From patchwork Fri Aug 26 22:03:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zach O'Keefe X-Patchwork-Id: 12956665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24E76C0502E for ; Fri, 26 Aug 2022 22:03:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 995B26B0074; Fri, 26 Aug 2022 18:03:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 96D7A6B0075; Fri, 26 Aug 2022 18:03:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 80B09940007; Fri, 26 Aug 2022 18:03:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7040B6B0074 for ; Fri, 26 Aug 2022 18:03:37 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 493C7120FE2 for ; Fri, 26 Aug 2022 22:03:37 +0000 (UTC) X-FDA: 79843121274.20.42A80D0 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf14.hostedemail.com (Postfix) with ESMTP id D23DC10001C for ; Fri, 26 Aug 2022 22:03:36 +0000 (UTC) Received: by mail-pj1-f73.google.com with SMTP id lx1-20020a17090b4b0100b001fd720458c3so737828pjb.1 for ; Fri, 26 Aug 2022 15:03:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc; bh=ZfMjBnU9jBash3JYHUYPEIOkw8Dh5mtl2yjBQXbzd8w=; b=lRCYS8VfUdwm2C4R1gez0VWa0kSo8O/qaTBR2zSmsfSTaXvVHg4DnAyzgeb2xZvEv3 xtWNe8PvzvdfQhF4mGp4bS2PzSXXw0ym1JNep/Xmoz2IV74exGjjNlE2YoLQNf85P5ZG ueqLdHFEpaN7QOjnctgb3/iF2Yemk5blhIU30bbA2S3hjYdslgKKo0PHEWp7GjB9qRlq PjZ0pd9xdfAC/TfooBjTcJYH5wfe6TTDgGR1bJz5zdvq/fMcmQDlV4pzBMXxAh3bON4t vXCa06t5QWRkVgMiWBSedGfN4tlY/kjX2qEVvvZoNjO56bi+mIGU6Xe5w8D2UxFs8OnG Jdkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc; bh=ZfMjBnU9jBash3JYHUYPEIOkw8Dh5mtl2yjBQXbzd8w=; b=gnw5cl5R6YlRdd2TSQ5Dsv9x/+Fo1NAy0GjS970sZpBgxFfMhKOPLl/Nj8GK9b/R4+ O4ojzZhs1S9A1IDvVle25YNgcmCFAjqHEvGtiNF2LdtMfAoqDE3tc3WdbGsdc/aV+hdW zM7TUCs553OYTHgaSrMbttVam3MQE+h3NHvTJPaDT9fVg9qVmfY6xlopAPVgewuYEEl9 yHw8M77TbVRuhJzsA19DXZTwddEpxdmCAh7JqlBSozq6cj3nY+7MDVi60AQ7nOx8vKWt dId4uSONfSxqJy47rj1LSXOcM59mNiOqrkWuGjoHjf+BeXrEbRmcaczICcFDG5r0Ne/L /MjA== X-Gm-Message-State: ACgBeo1uqtXm3KUBEuTnerEcfcucqsGPpaLxeEbbBfPzG5hxTSqk0Prx bG1SgrwnA/SRoohRLTWi5b00bJ9eR9NQg8Z7vRwP/wJFziffegBQgC4Pi4bVj2jnr/NYNhjKYGt GF+SSi10ULCRDs8ek1n6byPaGsZLSxUB5R39shb0N0tugfOOCFtY4qwdNXPs= X-Google-Smtp-Source: AA6agR4kvG2FLnY/ao//0NEq9QlaJoZJpTEl6AOSVFU/LTgmCWARNCcwmtvWZCAGEYYadyxUyGcBnV7sXIE3 X-Received: from zokeefe3.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:1b6]) (user=zokeefe job=sendgmr) by 2002:a05:6a00:2a04:b0:536:9d21:c177 with SMTP id ce4-20020a056a002a0400b005369d21c177mr5918144pfb.4.1661551415711; Fri, 26 Aug 2022 15:03:35 -0700 (PDT) Date: Fri, 26 Aug 2022 15:03:20 -0700 In-Reply-To: <20220826220329.1495407-1-zokeefe@google.com> Mime-Version: 1.0 References: <20220826220329.1495407-1-zokeefe@google.com> X-Mailer: git-send-email 2.37.2.672.g94769d06f0-goog Message-ID: <20220826220329.1495407-2-zokeefe@google.com> Subject: [PATCH mm-unstable v2 1/9] mm/shmem: add flag to enforce shmem THP in hugepage_vma_check() From: "Zach O'Keefe" To: linux-mm@kvack.org Cc: Andrew Morton , linux-api@vger.kernel.org, Axel Rasmussen , James Houghton , Hugh Dickins , Yang Shi , Miaohe Lin , David Hildenbrand , David Rientjes , Matthew Wilcox , Pasha Tatashin , Peter Xu , Rongwei Wang , SeongJae Park , Song Liu , Vlastimil Babka , Chris Kennelly , "Kirill A. Shutemov" , Minchan Kim , Patrick Xia , "Zach O'Keefe" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661551416; a=rsa-sha256; cv=none; b=T1Z31T/mHnLgAvrLAshyeY/tm4Ry+f+Ma9/jP6c92tehmvDjrIgsPEIBtzWfgpE8Twek7/ V6ZnWK9ukN9BztpoZ4PvAOdRVO1jm2suhx4oYo6wk+qtoGA03heVc21RJwECs0K/EorVM1 UKSq9HjQh4Heeb9O043DSAySyQG7a3s= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=lRCYS8Vf; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of 3N0MJYwcKCAI1qmgghgiqqing.eqonkpwz-oomxcem.qti@flex--zokeefe.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3N0MJYwcKCAI1qmgghgiqqing.eqonkpwz-oomxcem.qti@flex--zokeefe.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661551416; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZfMjBnU9jBash3JYHUYPEIOkw8Dh5mtl2yjBQXbzd8w=; b=FYI04ceyQINc6VWBxRYYuFnfiJDSFHcwu1tOyMCaQS6XpVsurfKofphgeFHZ0PP1TdytNh QY/65c6Hqw1Fmau0nOytNFXCSwDrWxTsdpJgAcCbuDMOzF/byh4UCbj6ist4sHrubn1T3D 9iY95LA4Z5o3BVSWZtel2/16L8931WU= Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=lRCYS8Vf; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of 3N0MJYwcKCAI1qmgghgiqqing.eqonkpwz-oomxcem.qti@flex--zokeefe.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3N0MJYwcKCAI1qmgghgiqqing.eqonkpwz-oomxcem.qti@flex--zokeefe.bounces.google.com X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: r6hkuxmf83gsn7iss3azsraf5jqe6sx6 X-Rspamd-Queue-Id: D23DC10001C X-HE-Tag: 1661551416-410886 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Extend 'mm/thp: add flag to enforce sysfs THP in hugepage_vma_check()' to shmem, allowing callers to ignore /sys/kernel/transparent_hugepage/shmem_enabled and tmpfs huge= mount. This is intended to be used by MADV_COLLAPSE, and the rationale is analogous to the anon/file case: MADV_COLLAPSE is not coupled to directives that advise the kernel's decisions on when THPs should be considered eligible. shmem/tmpfs always claims large folio support, regardless of sysfs or mount options. Signed-off-by: Zach O'Keefe --- include/linux/shmem_fs.h | 10 ++++++---- mm/huge_memory.c | 2 +- mm/shmem.c | 18 +++++++++--------- 3 files changed, 16 insertions(+), 14 deletions(-) diff --git a/include/linux/shmem_fs.h b/include/linux/shmem_fs.h index ff0b990de83d..f5e9b01dbf4c 100644 --- a/include/linux/shmem_fs.h +++ b/include/linux/shmem_fs.h @@ -92,11 +92,13 @@ extern struct page *shmem_read_mapping_page_gfp(struct address_space *mapping, extern void shmem_truncate_range(struct inode *inode, loff_t start, loff_t end); int shmem_unuse(unsigned int type); -extern bool shmem_is_huge(struct vm_area_struct *vma, - struct inode *inode, pgoff_t index); -static inline bool shmem_huge_enabled(struct vm_area_struct *vma) +extern bool shmem_is_huge(struct vm_area_struct *vma, struct inode *inode, + pgoff_t index, bool shmem_huge_force); +static inline bool shmem_huge_enabled(struct vm_area_struct *vma, + bool shmem_huge_force) { - return shmem_is_huge(vma, file_inode(vma->vm_file), vma->vm_pgoff); + return shmem_is_huge(vma, file_inode(vma->vm_file), vma->vm_pgoff, + shmem_huge_force); } extern unsigned long shmem_swap_usage(struct vm_area_struct *vma); extern unsigned long shmem_partial_swap_usage(struct address_space *mapping, diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 88d98241a635..b3acc8e3046d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -119,7 +119,7 @@ bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, * own flags. */ if (!in_pf && shmem_file(vma->vm_file)) - return shmem_huge_enabled(vma); + return shmem_huge_enabled(vma, !enforce_sysfs); /* Enforce sysfs THP requirements as necessary */ if (enforce_sysfs && diff --git a/mm/shmem.c b/mm/shmem.c index 42e5888bf84d..b9bab1abf142 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -472,20 +472,20 @@ static bool shmem_confirm_swap(struct address_space *mapping, static int shmem_huge __read_mostly = SHMEM_HUGE_NEVER; -bool shmem_is_huge(struct vm_area_struct *vma, - struct inode *inode, pgoff_t index) +bool shmem_is_huge(struct vm_area_struct *vma, struct inode *inode, + pgoff_t index, bool shmem_huge_force) { loff_t i_size; if (!S_ISREG(inode->i_mode)) return false; - if (shmem_huge == SHMEM_HUGE_DENY) - return false; if (vma && ((vma->vm_flags & VM_NOHUGEPAGE) || test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags))) return false; - if (shmem_huge == SHMEM_HUGE_FORCE) + if (shmem_huge == SHMEM_HUGE_FORCE || shmem_huge_force) return true; + if (shmem_huge == SHMEM_HUGE_DENY) + return false; switch (SHMEM_SB(inode->i_sb)->huge) { case SHMEM_HUGE_ALWAYS: @@ -680,8 +680,8 @@ static long shmem_unused_huge_count(struct super_block *sb, #define shmem_huge SHMEM_HUGE_DENY -bool shmem_is_huge(struct vm_area_struct *vma, - struct inode *inode, pgoff_t index) +bool shmem_is_huge(struct vm_area_struct *vma, struct inode *inode, + pgoff_t index, bool shmem_huge_force) { return false; } @@ -1069,7 +1069,7 @@ static int shmem_getattr(struct user_namespace *mnt_userns, STATX_ATTR_NODUMP); generic_fillattr(&init_user_ns, inode, stat); - if (shmem_is_huge(NULL, inode, 0)) + if (shmem_is_huge(NULL, inode, 0, false)) stat->blksize = HPAGE_PMD_SIZE; if (request_mask & STATX_BTIME) { @@ -1910,7 +1910,7 @@ static int shmem_getpage_gfp(struct inode *inode, pgoff_t index, return 0; } - if (!shmem_is_huge(vma, inode, index)) + if (!shmem_is_huge(vma, inode, index, false)) goto alloc_nohuge; huge_gfp = vma_thp_gfp_mask(vma);