From patchwork Thu Jun 16 17:48:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12884641 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A15FCCA47A for ; Thu, 16 Jun 2022 17:48:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 04E1E6B0073; Thu, 16 Jun 2022 13:48:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F13B86B0074; Thu, 16 Jun 2022 13:48:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D16B86B0075; Thu, 16 Jun 2022 13:48:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C0C206B0073 for ; Thu, 16 Jun 2022 13:48:50 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 8C14B216CF for ; Thu, 16 Jun 2022 17:48:50 +0000 (UTC) X-FDA: 79584834420.26.9D2E72F Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf08.hostedemail.com (Postfix) with ESMTP id 3CE9C16008C for ; Thu, 16 Jun 2022 17:48:50 +0000 (UTC) Received: by mail-pf1-f169.google.com with SMTP id bo5so2116246pfb.4 for ; Thu, 16 Jun 2022 10:48:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gQUColGb/PRKYuEV4lE8XSFjbzf2io5XZTvMnHxGXf0=; b=H8066BRclkgv160HBGUSksGljw5jEx74P81Wyl7DeklPUcfVRAzzn4ytpohhWZvS6Z bpso2AnaKouGt/5mw84y3KUdYhbE9jmtHQatT9ICKrULazn80V2zMIJSCZPItFdRljXh H6aFJpbnfQughZDnUFhPMMhlzqd0YguP5GLK2+gCQuBe0X6KLNggSWTXUJR9bdZxcrR7 WTTG+kcWkBdkQyWYbhl2priPPsi5ijjv+dcXEljTRUC2eP4QUXR4ZAyjeeVsvkurn61y TJjtSPV52f5tCl1bh01NNGhVozBWuNUlJ2X4U+VASX8ogw4/5DUTI2jK+RriHKc3J8IY DBug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gQUColGb/PRKYuEV4lE8XSFjbzf2io5XZTvMnHxGXf0=; b=zSsjxLYFH6NAokFAl2nZUfHwRR2aW2nuBjnC5ccRPlBheeCIKZl2j9MwQB4zv1p8J1 nICIam2nNDzmnT0XZXx/ABAHEI2yM+qNwIsRKikkgtCnDlOSLtXyJMgn6PpIGJQSQVQh D5+IrXhZZcK5UJgeVfeGrPlAnza/iVSTzzprzqHvlMCLYhrgwi5rThN+kxfyyDnlgrqn 64EPhyZHEEL6S43ejUeBiGHm7N9U5GZdDHokplhlL7ztPmayPOGDsDHsmE+qVv5XVr0E Vnh/U/oWQT4Hdc4ocOW8G46JFSTxf2DJzTlmGM24FLf/XQsRh9rLBaQ6GRUf4W0uVFsy 9l/Q== X-Gm-Message-State: AJIora/JKUm9y0bMrKbc3pG7z8rkfui8ntqukw09+SmmYXQV5ezp1NDB KNCzpj2f/d+3/D3vPn/XuYs= X-Google-Smtp-Source: AGRyM1vVU3rKwXy9n+PUA6CJmwhrnakM7UqbwEUBt0tkabuU8YP6EHHL4ouJ2CBwwzJvSF9Zcj+yCQ== X-Received: by 2002:a05:6a00:2392:b0:51b:fe0e:2b8 with SMTP id f18-20020a056a00239200b0051bfe0e02b8mr5799350pfc.84.1655401729182; Thu, 16 Jun 2022 10:48:49 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id z21-20020a17090a8b9500b001e8520b211bsm1818289pjn.53.2022.06.16.10.48.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Jun 2022 10:48:48 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, zokeefe@google.com, linmiaohe@huawei.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v5 PATCH 1/7] mm: khugepaged: check THP flag in hugepage_vma_check() Date: Thu, 16 Jun 2022 10:48:34 -0700 Message-Id: <20220616174840.1202070-2-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220616174840.1202070-1-shy828301@gmail.com> References: <20220616174840.1202070-1-shy828301@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=H8066BRc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655401730; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gQUColGb/PRKYuEV4lE8XSFjbzf2io5XZTvMnHxGXf0=; b=6PDfp2kAcwN8/8DYFq9A14X5cyotDM864AgPT5rqAKPt7Go2PMYti/D2mFpmLa27t5Roqu G+WhUM5oJrgeX9ffFS8JAqvGlt205InbCRhLufm4C2i23ku/PudnALZ7GifGoCkFdVnYNf fxmScKNiWYNYiEpRMHQWPQhWayjHMjY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655401730; a=rsa-sha256; cv=none; b=WumWuBPzkut5hry3UQWmlcfsml+TiApid67z17FaJHzrR9PPedxr8pZuPU11XmDjjR04yj IDPr8HNRVS7LjLhhN8JRGcklovlpgAqN/hv6ArnuaQ5ZWSwyIk6wNLofPXoILkrWUhem0V YoTfKMyQDiP+ybshihRgI/M0tyX4sFo= X-Rspam-User: X-Stat-Signature: mcmrzmefedax15beibxt56boggicggph X-Rspamd-Queue-Id: 3CE9C16008C Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=H8066BRc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf08.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam05 X-HE-Tag: 1655401730-812350 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently the THP flag check in hugepage_vma_check() will fallthrough if the flag is NEVER and VM_HUGEPAGE is set. This is not a problem for now since all the callers have the flag checked before or can't be invoked if the flag is NEVER. However, the following patch will call hugepage_vma_check() in more places, for example, page fault, so this flag must be checked in hugepge_vma_check(). Reviewed-by: Zach O'Keefe Reviewed-by: Miaohe Lin Signed-off-by: Yang Shi --- mm/khugepaged.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 476d79360101..b1dab94c0f1e 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -458,6 +458,9 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); + if (!khugepaged_enabled()) + return false; + /* THP settings require madvise. */ if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) return false; From patchwork Thu Jun 16 17:48:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12884642 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4139FC43334 for ; Thu, 16 Jun 2022 17:48:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C08C26B0074; Thu, 16 Jun 2022 13:48:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B917F6B0075; Thu, 16 Jun 2022 13:48:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 947636B0078; Thu, 16 Jun 2022 13:48:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 849D66B0074 for ; Thu, 16 Jun 2022 13:48:52 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 5E856162E for ; Thu, 16 Jun 2022 17:48:52 +0000 (UTC) X-FDA: 79584834504.24.93BD23B Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by imf26.hostedemail.com (Postfix) with ESMTP id A37BA1400A2 for ; Thu, 16 Jun 2022 17:48:51 +0000 (UTC) Received: by mail-pf1-f170.google.com with SMTP id w21so2147488pfc.0 for ; Thu, 16 Jun 2022 10:48:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Lf1Kp53gV5xVxcPrzuVVyrzHKBnqiH74ZtDbvhyF8zw=; b=qHbvu3U4WI6XTL+Bb7EcWbfSgzpXVwmBwdYi5b4fTDSMY/tMFb3YRbJy105vXefOI1 v+XLHWPz/vB0UgVN0u2mh1l+NLdiJ88GcEjUAoGZvoskG4KtWLQvWM29G0qBeynD9M9l ejsjQKiDU16xatYVZWK11/4EP942U/xh+kghEmAO2eS/VqNHbCd9twMGhSrNDNbfL3Fx 8AuHMCmNpCKAfw1BDm1meeWm/ujGolhpZ8paAlpESIC987HvF+vMDiRVmzs8ryP6UOof 4f4TUCiMMsSW1Qf6i2NxxcKj798ou3uvQlxSTGuXQ1ILAc8RKJd5hZjzKuHbjwjJ1504 8VjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Lf1Kp53gV5xVxcPrzuVVyrzHKBnqiH74ZtDbvhyF8zw=; b=HAJmIU6twwYO3cOp84oJCX3lVtFFfRoUKMb5ZuicIAnaVD2qpMlLsvB34o8rEYto5z wBaaSQpOP6y+gTNInkKqjbbdJEo8F8gBPDqQOHDfrtWtsmELcM5+hzRCzHY5wWdVD1gn byJbuaeb634OhaDZWIqj66M3qG9NatnXHOsdiRaNIDp6BL1MKev3yNssijyYwFYkCoZJ D696FgjzSWa0WSxZ+JdjVoPUPXgX8WifxBM/aXipdHlPkcX5knZyMcei5aJmmu0MAXZI O7qEs2zlfnCdKs+xQbLXhUIDMFkqQSwJtwrJPxc+yqsFGIVN6+4IUkjVuTzdCjoKWYQJ rulg== X-Gm-Message-State: AJIora81+6XjJxKMtC1/9IY6/gHT7c6flL5fu6VULXL95psD+zDWg5T4 Kx/bQV7ueF5Ij0XdGeDqobI= X-Google-Smtp-Source: AGRyM1sILDxv0QZoQXrOLOSuJVQZEjJnBYzToTfeZrKnwvvpIW3ALa81CvsweBmzReuWF3hmAbqADA== X-Received: by 2002:a63:894a:0:b0:3fc:a724:578c with SMTP id v71-20020a63894a000000b003fca724578cmr5543251pgd.499.1655401730679; Thu, 16 Jun 2022 10:48:50 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id z21-20020a17090a8b9500b001e8520b211bsm1818289pjn.53.2022.06.16.10.48.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Jun 2022 10:48:50 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, zokeefe@google.com, linmiaohe@huawei.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v5 PATCH 2/7] mm: thp: consolidate vma size check to transhuge_vma_suitable Date: Thu, 16 Jun 2022 10:48:35 -0700 Message-Id: <20220616174840.1202070-3-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220616174840.1202070-1-shy828301@gmail.com> References: <20220616174840.1202070-1-shy828301@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=qHbvu3U4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655401731; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Lf1Kp53gV5xVxcPrzuVVyrzHKBnqiH74ZtDbvhyF8zw=; b=tub632N2fPZAZTrUsqtXnLk5xx5oiGHwzBz6716wGw9Fpc3m1JbCelYt2oL9ioz6QWVeA3 D8nhYwx21IHsW/pEmIoFzql43kDlJMnXSNnmPwWHkAT5b9GKPthYArPcUyP1MG+5l5+TqR pfLiNX2qXxqdctYt7fCsb8Sq06T4Pjk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655401731; a=rsa-sha256; cv=none; b=7UuPyRl4qdwpPBXmcXwJj6ZPvvxCuvAosYK1fTWuM2XDSOLzWM9IOvf2WGNAoYrFpkBANK XowZjg5rr8a9n9dZH62Z1+ELy6W4Zi2+24T32GiSUu+nbQ7iMDZtDBWO5qoPOpYwTYEFh4 tQaLVAgTa92Skv62PHqXbid54cASWyc= X-Rspam-User: X-Stat-Signature: hxr3qfq9z5hdwcc3ota1p8s4f5xgty5q X-Rspamd-Queue-Id: A37BA1400A2 Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=qHbvu3U4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam05 X-HE-Tag: 1655401731-927129 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are couple of places that check whether the vma size is ok for THP or whether address fits, they are open coded and duplicate, use transhuge_vma_suitable() to do the job by passing in (vma->end - HPAGE_PMD_SIZE). Move vma size check into hugepage_vma_check(). This will make khugepaged_enter() is as same as khugepaged_enter_vma(). There is just one caller for khugepaged_enter(), replace it to khugepaged_enter_vma() and remove khugepaged_enter(). Reviewed-by: Zach O'Keefe Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 11 +++++++++++ include/linux/khugepaged.h | 14 -------------- mm/huge_memory.c | 2 +- mm/khugepaged.c | 19 ++++++------------- 4 files changed, 18 insertions(+), 28 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 648cb3ce7099..8a5a8bfce0f5 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -116,6 +116,17 @@ extern struct kobj_attribute shmem_enabled_attr; extern unsigned long transparent_hugepage_flags; +/* + * Do the below checks: + * - For file vma, check if the linear page offset of vma is + * HPAGE_PMD_NR aligned within the file. The hugepage is + * guaranteed to be hugepage-aligned within the file, but we must + * check that the PMD-aligned addresses in the VMA map to + * PMD-aligned offsets within the file, else the hugepage will + * not be PMD-mappable. + * - For all vmas, check if the haddr is in an aligned HPAGE_PMD_SIZE + * area. + */ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 392d34c3c59a..31ca8a7f78f4 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -51,16 +51,6 @@ static inline void khugepaged_exit(struct mm_struct *mm) if (test_bit(MMF_VM_HUGEPAGE, &mm->flags)) __khugepaged_exit(mm); } - -static inline void khugepaged_enter(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags)) - __khugepaged_enter(vma->vm_mm); - } -} #else /* CONFIG_TRANSPARENT_HUGEPAGE */ static inline void khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm) { @@ -68,10 +58,6 @@ static inline void khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm static inline void khugepaged_exit(struct mm_struct *mm) { } -static inline void khugepaged_enter(struct vm_area_struct *vma, - unsigned long vm_flags) -{ -} static inline void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 4f9bbb4eab23..b530462c4493 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -726,7 +726,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) return VM_FAULT_FALLBACK; if (unlikely(anon_vma_prepare(vma))) return VM_FAULT_OOM; - khugepaged_enter(vma, vma->vm_flags); + khugepaged_enter_vma(vma, vma->vm_flags); if (!(vmf->flags & FAULT_FLAG_WRITE) && !mm_forbids_zeropage(vma->vm_mm) && diff --git a/mm/khugepaged.c b/mm/khugepaged.c index b1dab94c0f1e..db0b334a7d1f 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -450,8 +450,8 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (vma_is_dax(vma)) return false; - if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - - vma->vm_pgoff, HPAGE_PMD_NR)) + /* Check alignment for file vma and size for both file and anon vma */ + if (!transhuge_vma_suitable(vma, (vma->vm_end - HPAGE_PMD_SIZE))) return false; /* Enabled via shmem mount options or sysfs settings. */ @@ -512,9 +512,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled() && - (((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) < - (vma->vm_end & HPAGE_PMD_MASK))) { + khugepaged_enabled()) { if (hugepage_vma_check(vma, vm_flags)) __khugepaged_enter(vma->vm_mm); } @@ -950,7 +948,6 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, struct vm_area_struct **vmap) { struct vm_area_struct *vma; - unsigned long hstart, hend; if (unlikely(khugepaged_test_exit(mm))) return SCAN_ANY_PROCESS; @@ -959,9 +956,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, if (!vma) return SCAN_VMA_NULL; - hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; - hend = vma->vm_end & HPAGE_PMD_MASK; - if (address < hstart || address + HPAGE_PMD_SIZE > hend) + if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; @@ -2147,10 +2142,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; continue; } - hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; - hend = vma->vm_end & HPAGE_PMD_MASK; - if (hstart >= hend) - goto skip; + hstart = round_up(vma->vm_start, HPAGE_PMD_SIZE); + hend = round_down(vma->vm_end, HPAGE_PMD_SIZE); if (khugepaged_scan.address > hend) goto skip; if (khugepaged_scan.address < hstart) From patchwork Thu Jun 16 17:48:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12884643 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBD17CCA47A for ; Thu, 16 Jun 2022 17:48:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5C9216B0075; Thu, 16 Jun 2022 13:48:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 54FFD6B0078; Thu, 16 Jun 2022 13:48:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C9266B007B; Thu, 16 Jun 2022 13:48:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2E3576B0075 for ; Thu, 16 Jun 2022 13:48:54 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 0671D80F97 for ; Thu, 16 Jun 2022 17:48:54 +0000 (UTC) X-FDA: 79584834588.24.D720E5D Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) by imf13.hostedemail.com (Postfix) with ESMTP id 9693320088 for ; Thu, 16 Jun 2022 17:48:53 +0000 (UTC) Received: by mail-pj1-f49.google.com with SMTP id k12-20020a17090a404c00b001eaabc1fe5dso2478470pjg.1 for ; Thu, 16 Jun 2022 10:48:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IVDYrUdqiRm9UcRbpXweUBj4/hDRVoO4CK1KpBPozys=; b=YoZ1bssMZl0ds5wLwIPKg0IItqCPaKU17iyetjJF+hT+7KvhMVimEkgRt95TZkLAxO FnqS18PmPzsWKkIght4RbuqwygilGASy2GLKqKg6k/vzT6t+DtM4UxAQl5b7P9A3TNNf eNO3UdSwBlFGrUig73CtXZrfbbh+sfQ3yu/6CT/tziWSALK5brivuz9r9Oul1PmCzrfZ vwzv81Kzs6hCvJuSzDap1wEz3A/BmNXbZucyPTub2avd33WiHFdjm29e8KsZtbG6Ob56 R8nESRo3DArWhf+2Xo28r7HWLmRlqKlf75OJ1nRX8ig9JYoj6D/6DI+3HTWP9r+gSf7n AW3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IVDYrUdqiRm9UcRbpXweUBj4/hDRVoO4CK1KpBPozys=; b=1NrcI++N6uamcbjkd0LEKkmUj1BUppkrYwOwI/kOQ0Yw7FyLTeT3LJN5fd25TFnLIT RJVUMFq6ojdenEAFfhvCpomChOWhP886yIb+s4cSVY+L3OOThFg9a6xiD6imXGHJkcjf J4l8vj0BNbviWnkY5lHT0tPJbsCQjl3NvCIxp9+oLehAEKybSEP1Ny2/x7fD8FAviyHK J861KCvDTDKg8t6wDuqkMLHQsf2D2/OygYlGRoqbWX0Ls9shPf2RaY4PRce92c8ycFEc DnfG9BRHdjK3aCsflJ18dTpBdip1urKjj6mlllfGs2JdAnJy3zJszmwq8LNnOR9yPm76 +FuA== X-Gm-Message-State: AJIora/TjAwrqXJHiUD3u5ZrsEQrDH0YUNxvco0p/uX7uzOhhYK6cVsL 0RG3JWZ3QdO6ZGWNHxmvJhU= X-Google-Smtp-Source: AGRyM1vxGK17GWzftPYeur/8Me6GEPOXRVJXvmUbi9GWcVUiu6WhWb2FJDsYzyvI0N7smHaYh/uyEg== X-Received: by 2002:a17:902:c403:b0:167:4a9f:2799 with SMTP id k3-20020a170902c40300b001674a9f2799mr5716030plk.56.1655401732435; Thu, 16 Jun 2022 10:48:52 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id z21-20020a17090a8b9500b001e8520b211bsm1818289pjn.53.2022.06.16.10.48.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Jun 2022 10:48:51 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, zokeefe@google.com, linmiaohe@huawei.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v5 PATCH 3/7] mm: khugepaged: better comments for anon vma check in hugepage_vma_revalidate Date: Thu, 16 Jun 2022 10:48:36 -0700 Message-Id: <20220616174840.1202070-4-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220616174840.1202070-1-shy828301@gmail.com> References: <20220616174840.1202070-1-shy828301@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=YoZ1bssM; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf13.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.49 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655401733; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IVDYrUdqiRm9UcRbpXweUBj4/hDRVoO4CK1KpBPozys=; b=k6XTIrYfvTrPri++y5uLawXR2ArEHl+753JU6TXU/5j2wqD7GazW4E0pZNm8Xd4qEi9Xdt B7xC0L2+wPXMGARWj5XHR62Wi8m0HHWqF74mM2IsJtfGpChecDlWStPoPA1smEXRg8l+k6 TvL9mYD3bV7Xr+Qk1SxNue7tOtcnWUM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655401733; a=rsa-sha256; cv=none; b=wa5n9zoaHbJZj4KkBlouFVjpDTCcKnXzUHMOIngKWHi/Ok3oosYSw3iC3UXBXwoxFdLBw8 EDwmMSu5iBsH66AGKH2N3WKmOJZ4EO0ivwqyFrRJDuFPXmv2wcA3VSQy4T0B/q/Nd8O99F HdZKmNEHVbcOFackOO5huL74AM612tk= X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 9693320088 X-Rspam-User: Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=YoZ1bssM; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf13.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.49 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Stat-Signature: b5hbuzm9ctkr91grd5xkzwhufwr9edf7 X-HE-Tag: 1655401733-606132 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The hugepage_vma_revalidate() needs to check if the vma is still anonymous vma or not since the address may be unmapped then remapped to file before khugepaged reaquired the mmap_lock. The old comment is not quite helpful, elaborate this with better comment. Reviewed-by: Zach O'Keefe Signed-off-by: Yang Shi --- mm/khugepaged.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index db0b334a7d1f..5baa394e34c8 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -960,7 +960,13 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; - /* Anon VMA expected */ + /* + * Anon VMA expected, the address may be unmapped then + * remapped to file after khugepaged reaquired the mmap_lock. + * + * hugepage_vma_check may return true for qualified file + * vmas. + */ if (!vma->anon_vma || !vma_is_anonymous(vma)) return SCAN_VMA_CHECK; return 0; From patchwork Thu Jun 16 17:48:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12884644 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 652BDC43334 for ; Thu, 16 Jun 2022 17:48:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A5D556B007D; Thu, 16 Jun 2022 13:48:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A0A576B007B; Thu, 16 Jun 2022 13:48:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85D5F6B007D; Thu, 16 Jun 2022 13:48:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 7664B6B0078 for ; Thu, 16 Jun 2022 13:48:55 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 4DC9D20703 for ; Thu, 16 Jun 2022 17:48:55 +0000 (UTC) X-FDA: 79584834630.25.45F3013 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf27.hostedemail.com (Postfix) with ESMTP id E1BDB4009C for ; Thu, 16 Jun 2022 17:48:54 +0000 (UTC) Received: by mail-pj1-f47.google.com with SMTP id 73-20020a17090a0fcf00b001eaee69f600so2138202pjz.1 for ; Thu, 16 Jun 2022 10:48:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SKzP8GvCz2PJf6EFnSMPoShRtGtH8yTLiOh2Lf/1mNw=; b=o6VtXrucgvo9V9vrl6em7ARq7LIYKBa6YZlKMEdy4FgPRadKHeJHGuino/92SIAf47 W5YJH0o3FkV2OlxUTftRIXQ1z6ePSIK0b0pozHwxajn/2+eAWl7VpUe1oEoxCWUQGpfh PGWBSbZBfsBXCM+Z2YGdzyXzGlIGxUxiENsmowMvYOBPiex+eS6XOOZAm1gjvkx47fAU p6memXJ0Vuq1uyU51/mX+zdh503EFoSnWzQYT2CrhScToA6fjxvN2ssJNUaR4hpqAzDp Ht/rlKGsHZYDj3WzSMKmKyXpAK8eEafaKStomvzNxTOr42LCJjlReO1s0H9E/t8Kf9kb Rfjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SKzP8GvCz2PJf6EFnSMPoShRtGtH8yTLiOh2Lf/1mNw=; b=DcZapu0wCQfaqjVHOOlXufssjL4CvvC+YfohMK17/k6jv3QkH0aAX95Pis803MzKh0 PCNQiS1E7W2x5TYDlQ8V7R8sXkaQdkGh3itwThVYzaKlWgM9M3teCtS3apQu2NAL86Lv 6Has55vkTKXkikpKzlQZ7rmN6Jk3h8HguwNkjcrv+4f+wIWyf8OmxMLlMGlevCiAGer1 FlnxTURZy3NpGuYeJ2aM7rTyPI+YZtSbMvdachHEFKFylPQ7qUp7+Q9eE8p7WqKIUDQK LzDNBkAH9N5dkRDIdckcbReOYYEjQhZWfPPoEkpa/bKt9coBhOKFDBnQ5urpPszuPctr WSLQ== X-Gm-Message-State: AJIora+b5xlbmecK0I2YrWieeeNVGgtAk8/Y4i2dgL2j3HmIaQjaGBMp 0CvmVGIC5s4oe2YJr60zXbM= X-Google-Smtp-Source: AGRyM1tD+obgwCVCrFhpKnWZxWNyZcVnZs4BoSmGflr9GWj6ak0KmkrqfKKE387/dApUt9IsDJQPzg== X-Received: by 2002:a17:902:76c2:b0:16a:205:f0cc with SMTP id j2-20020a17090276c200b0016a0205f0ccmr500472plt.171.1655401733973; Thu, 16 Jun 2022 10:48:53 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id z21-20020a17090a8b9500b001e8520b211bsm1818289pjn.53.2022.06.16.10.48.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Jun 2022 10:48:53 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, zokeefe@google.com, linmiaohe@huawei.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v5 PATCH 4/7] mm: thp: kill transparent_hugepage_active() Date: Thu, 16 Jun 2022 10:48:37 -0700 Message-Id: <20220616174840.1202070-5-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220616174840.1202070-1-shy828301@gmail.com> References: <20220616174840.1202070-1-shy828301@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=o6VtXruc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655401735; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SKzP8GvCz2PJf6EFnSMPoShRtGtH8yTLiOh2Lf/1mNw=; b=6bAfRw+O+dSljL6SoW4UQCi3vv0OTvTlbUPGkdmfFo8dQxbapx61CDivnZSDeYDBjWoetq p9ZPmMMG7AfL3cvcQj/mloFL2f2x/cE7DB16QeOzctUqqm0ewDkn03/oHEKMRT2ik67xih PO73Zu0EmX+SedgSWYaYGy6XTpfQpZc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655401735; a=rsa-sha256; cv=none; b=nLxQc6sp6SveqWh/kBJ3BOWyDRYavl+YFYBaVuKQK+bvqcF8ULeA/7gmOxuwIdi4b/irfP G1u4vG5VSc9C/MCKgqAgdQjI4iSliXWzhZirsuqMBI/EBdbQM3ZijCfIhlEh1jycU6ggbw 5WYPbFxxmvt8mzLb9YKGuDpp1Gzfv0M= X-Stat-Signature: z484ttxi1n9xxrhhn5i7tu68qm813j85 X-Rspamd-Queue-Id: E1BDB4009C X-Rspam-User: Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=o6VtXruc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf27.hostedemail.com: domain of shy828301@gmail.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam10 X-HE-Tag: 1655401734-152292 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The transparent_hugepage_active() was introduced to show THP eligibility bit in smaps in proc, smaps is the only user. But it actually does the similar check as hugepage_vma_check() which is used by khugepaged. We definitely don't have to maintain two similar checks, so kill transparent_hugepage_active(). This patch also fixed the wrong behavior for VM_NO_KHUGEPAGED vmas. Also move hugepage_vma_check() to huge_memory.c and huge_mm.h since it is not only for khugepaged anymore. Reviewed-by: Zach O'Keefe Signed-off-by: Yang Shi --- fs/proc/task_mmu.c | 2 +- include/linux/huge_mm.h | 16 +++++++----- include/linux/khugepaged.h | 2 -- mm/huge_memory.c | 50 +++++++++++++++++++++++++++++++------- mm/khugepaged.c | 48 +++--------------------------------- 5 files changed, 56 insertions(+), 62 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 37ccb5c9f4f8..39a40ec181e7 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -863,7 +863,7 @@ static int show_smap(struct seq_file *m, void *v) __show_smap(m, &mss, false); seq_printf(m, "THPeligible: %d\n", - transparent_hugepage_active(vma)); + hugepage_vma_check(vma, vma->vm_flags, true)); if (arch_pkeys_enabled()) seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma)); diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 8a5a8bfce0f5..64487bcd0c7b 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -202,7 +202,9 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma) !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode); } -bool transparent_hugepage_active(struct vm_area_struct *vma); +bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags, + bool smaps); #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ @@ -351,11 +353,6 @@ static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) return false; } -static inline bool transparent_hugepage_active(struct vm_area_struct *vma) -{ - return false; -} - static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { @@ -368,6 +365,13 @@ static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, return false; } +static inline bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags, + bool smaps) +{ + return false; +} + static inline void prep_transhuge_page(struct page *page) {} #define transparent_hugepage_flags 0UL diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 31ca8a7f78f4..ea5fd4c398f7 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -10,8 +10,6 @@ extern struct attribute_group khugepaged_attr_group; extern int khugepaged_init(void); extern void khugepaged_destroy(void); extern int start_stop_khugepaged(void); -extern bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags); extern void __khugepaged_enter(struct mm_struct *mm); extern void __khugepaged_exit(struct mm_struct *mm); extern void khugepaged_enter_vma(struct vm_area_struct *vma, diff --git a/mm/huge_memory.c b/mm/huge_memory.c index b530462c4493..a28c6100b491 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -69,21 +69,53 @@ static atomic_t huge_zero_refcount; struct page *huge_zero_page __read_mostly; unsigned long huge_zero_pfn __read_mostly = ~0UL; -bool transparent_hugepage_active(struct vm_area_struct *vma) +bool hugepage_vma_check(struct vm_area_struct *vma, + unsigned long vm_flags, + bool smaps) { - /* The addr is used to check if the vma size fits */ - unsigned long addr = (vma->vm_end & HPAGE_PMD_MASK) - HPAGE_PMD_SIZE; + if (!transhuge_vma_enabled(vma, vm_flags)) + return false; + + if (vm_flags & VM_NO_KHUGEPAGED) + return false; + + /* Don't run khugepaged against DAX vma */ + if (vma_is_dax(vma)) + return false; - if (!transhuge_vma_suitable(vma, addr)) + /* Check alignment for file vma and size for both file and anon vma */ + if (!transhuge_vma_suitable(vma, (vma->vm_end - HPAGE_PMD_SIZE))) return false; - if (vma_is_anonymous(vma)) - return __transparent_hugepage_enabled(vma); - if (vma_is_shmem(vma)) + + /* Enabled via shmem mount options or sysfs settings. */ + if (shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); - if (transhuge_vma_enabled(vma, vma->vm_flags) && file_thp_enabled(vma)) + + if (!khugepaged_enabled()) + return false; + + /* THP settings require madvise. */ + if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) + return false; + + /* Only regular file is valid */ + if (file_thp_enabled(vma)) return true; - return false; + if (!vma_is_anonymous(vma)) + return false; + + if (vma_is_temporary_stack(vma)) + return false; + + /* + * THPeligible bit of smaps should show 1 for proper VMAs even + * though anon_vma is not initialized yet. + */ + if (!vma->anon_vma) + return smaps; + + return true; } static bool get_huge_zero_page(void) diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 5baa394e34c8..3afd87f8c0b1 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -437,46 +437,6 @@ static inline int khugepaged_test_exit(struct mm_struct *mm) return atomic_read(&mm->mm_users) == 0; } -bool hugepage_vma_check(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - if (!transhuge_vma_enabled(vma, vm_flags)) - return false; - - if (vm_flags & VM_NO_KHUGEPAGED) - return false; - - /* Don't run khugepaged against DAX vma */ - if (vma_is_dax(vma)) - return false; - - /* Check alignment for file vma and size for both file and anon vma */ - if (!transhuge_vma_suitable(vma, (vma->vm_end - HPAGE_PMD_SIZE))) - return false; - - /* Enabled via shmem mount options or sysfs settings. */ - if (shmem_file(vma->vm_file)) - return shmem_huge_enabled(vma); - - if (!khugepaged_enabled()) - return false; - - /* THP settings require madvise. */ - if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) - return false; - - /* Only regular file is valid */ - if (file_thp_enabled(vma)) - return true; - - if (!vma->anon_vma || !vma_is_anonymous(vma)) - return false; - if (vma_is_temporary_stack(vma)) - return false; - - return true; -} - void __khugepaged_enter(struct mm_struct *mm) { struct mm_slot *mm_slot; @@ -513,7 +473,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags)) + if (hugepage_vma_check(vma, vm_flags, false)) __khugepaged_enter(vma->vm_mm); } } @@ -958,7 +918,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; - if (!hugepage_vma_check(vma, vma->vm_flags)) + if (!hugepage_vma_check(vma, vma->vm_flags, false)) return SCAN_VMA_CHECK; /* * Anon VMA expected, the address may be unmapped then @@ -1448,7 +1408,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr) * the valid THP. Add extra VM_HUGEPAGE so hugepage_vma_check() * will not fail the vma for missing VM_HUGEPAGE */ - if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE)) + if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE, false)) return; /* Keep pmd pgtable for uffd-wp; see comment in retract_page_tables() */ @@ -2143,7 +2103,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; break; } - if (!hugepage_vma_check(vma, vma->vm_flags)) { + if (!hugepage_vma_check(vma, vma->vm_flags, false)) { skip: progress++; continue; From patchwork Thu Jun 16 17:48:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12884645 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12D62CCA47A for ; Thu, 16 Jun 2022 17:48:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 94C296B0078; Thu, 16 Jun 2022 13:48:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8FA5A6B007B; Thu, 16 Jun 2022 13:48:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 74C2B6B007E; Thu, 16 Jun 2022 13:48:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 6563E6B0078 for ; Thu, 16 Jun 2022 13:48:57 -0400 (EDT) Received: from smtpin31.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 3376F163D for ; Thu, 16 Jun 2022 17:48:57 +0000 (UTC) X-FDA: 79584834714.31.5A44937 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by imf28.hostedemail.com (Postfix) with ESMTP id A4263C009E for ; Thu, 16 Jun 2022 17:48:56 +0000 (UTC) Received: by mail-pf1-f174.google.com with SMTP id z17so2113504pff.7 for ; Thu, 16 Jun 2022 10:48:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9rMZNhDmVhFfYI3dZDinmpFWJ+654cXj29ZWajToGJQ=; b=k54UC37gpxXu1HC6M7ieaY9VAO7o/YPUOu3hXBYkVkv9C9ybJ7kPZYEOP8IFLnqa/K b7SIBpXhgLFueHs3dPgXuUiOOqsWOJml11FXjJAmxamtSAesoQTB/a6eYS0IOrmmlW3e baHSJK3GJFK4JfnGIsv3RSKvHe4rsDTJkVQQNJb2ag7cv7IfjK4EtZiTm8uCh9tK/t1R hPTvYNDOs8R4yWRNcO9HttLDz1hx57EdefHssGDrstlaCNpwL44FwCRn5in6w/PrA+3V 6T/kd19emhBUlP+XHDsb2pnQCRpRfCDXfhv9VXTz+/fHjJJsirE5df6xjGI9jd5nA5UQ nIgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9rMZNhDmVhFfYI3dZDinmpFWJ+654cXj29ZWajToGJQ=; b=mZPrBUMKKB0tpOtZdIfBZ2YAD7ekr80vhipX883aWX7bKA2MCXEJkOoYW4xkMsmACD G8C+Yim6T3Tx5kiBiTPEyTIsVzazo8c74Jfitnmwneh8NhMPyrd7QSsF1MS17hF4gW30 YEVSKVrlQ9hFyqxE4Hz3uNLsMnyoxn8eVthyYC+umRZdaOC+upqZMB7nVG7JMJgtuvuI aHebBGOvBU/lRmSInfqavCiHj/aqma4rwEDeKpcbkxgh+OfSrj2dbhQDqJYpoyJdh3BY hXM431J9GpyB/7jES6z6M5Gf1NVWWDSZEKQlnlU9cCGC5OPXEuterJJLScNwTAmScdZt xleA== X-Gm-Message-State: AJIora9Y3uqjfEJKXjaIQbdtNNg3uvSlpa3ub9AEK+ggqXkHt7U3WE8C JuCjPgrGQiuIxQRuBvvLDIQ= X-Google-Smtp-Source: AGRyM1vl4tOydvP7hK0KqRHMn/OYMwWHFtGqeiwYKuHJr2LotWCPXbJStaJQm1Mior+02JTeRp6U3w== X-Received: by 2002:a05:6a00:98b:b0:51b:d730:c58 with SMTP id u11-20020a056a00098b00b0051bd7300c58mr6008432pfg.23.1655401735644; Thu, 16 Jun 2022 10:48:55 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id z21-20020a17090a8b9500b001e8520b211bsm1818289pjn.53.2022.06.16.10.48.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Jun 2022 10:48:54 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, zokeefe@google.com, linmiaohe@huawei.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v5 PATCH 5/7] mm: thp: kill __transhuge_page_enabled() Date: Thu, 16 Jun 2022 10:48:38 -0700 Message-Id: <20220616174840.1202070-6-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220616174840.1202070-1-shy828301@gmail.com> References: <20220616174840.1202070-1-shy828301@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655401736; a=rsa-sha256; cv=none; b=Mtwk4O54w6xm/AnRBF3Dge85tTpv+QFL+eUNsDeSm2fbwi3vLCv6i/ovwKO4AWmnMHOtxC /QrnPgRFV4xXjY8d1PYyfPQfAKpZRSc1zwjC+0wi+O5s8QBK7rQeh3Q8eLU0RvRExdvUaL Dj1aQ7/cc73I2nKOteSawJM8XIOkgYM= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=k54UC37g; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf28.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655401736; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9rMZNhDmVhFfYI3dZDinmpFWJ+654cXj29ZWajToGJQ=; b=NWbiWZ1YQOKDpdg6suBeXk2GYrCkjy/hPNrWlvsrg984ApgX0XZYX3pVNc5C9F+52YpwuH UW5jSaBVj5OOAJUNIH4447pTG9JPCQ6xUQ6XcfZqZtDfCB0n1Dsy9plfR15jDOv/IjIzRc /s7StG/vsSISkrzlFCiTOD/ImoxI+ZY= X-Rspamd-Queue-Id: A4263C009E X-Rspam-User: Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=k54UC37g; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf28.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam06 X-Stat-Signature: esf9x15imghg6g1fzjt6oq8dywgyyp4p X-HE-Tag: 1655401736-721032 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The page fault path checks THP eligibility with __transhuge_page_enabled() which does the similar thing as hugepage_vma_check(), so use hugepage_vma_check() instead. However page fault allows DAX and !anon_vma cases, so added a new flag, in_pf, to hugepage_vma_check() to make page fault work correctly. The in_pf flag is also used to skip shmem and file THP for page fault since shmem handles THP in its own shmem_fault() and file THP allocation on fault is not supported yet. Also remove hugepage_vma_enabled() since hugepage_vma_check() is the only caller now, it is not necessary to have a helper function. Reviewed-by: Zach O'Keefe Signed-off-by: Yang Shi --- fs/proc/task_mmu.c | 2 +- include/linux/huge_mm.h | 57 ++--------------------------------------- mm/huge_memory.c | 51 ++++++++++++++++++++++++++++-------- mm/khugepaged.c | 8 +++--- mm/memory.c | 7 +++-- 5 files changed, 52 insertions(+), 73 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 39a40ec181e7..cef72e49acc5 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -863,7 +863,7 @@ static int show_smap(struct seq_file *m, void *v) __show_smap(m, &mss, false); seq_printf(m, "THPeligible: %d\n", - hugepage_vma_check(vma, vma->vm_flags, true)); + hugepage_vma_check(vma, vma->vm_flags, true, false)); if (arch_pkeys_enabled()) seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma)); diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 64487bcd0c7b..cd8a6c5d9fe5 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -146,48 +146,6 @@ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, return true; } -static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - /* Explicitly disabled through madvise. */ - if ((vm_flags & VM_NOHUGEPAGE) || - test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) - return false; - return true; -} - -/* - * to be used on vmas which are known to support THP. - * Use transparent_hugepage_active otherwise - */ -static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) -{ - - /* - * If the hardware/firmware marked hugepage support disabled. - */ - if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_NEVER_DAX)) - return false; - - if (!transhuge_vma_enabled(vma, vma->vm_flags)) - return false; - - if (vma_is_temporary_stack(vma)) - return false; - - if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_FLAG)) - return true; - - if (vma_is_dax(vma)) - return true; - - if (transparent_hugepage_flags & - (1 << TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG)) - return !!(vma->vm_flags & VM_HUGEPAGE); - - return false; -} - static inline bool file_thp_enabled(struct vm_area_struct *vma) { struct inode *inode; @@ -204,7 +162,7 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma) bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, - bool smaps); + bool smaps, bool in_pf); #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ @@ -348,26 +306,15 @@ static inline bool folio_test_pmd_mappable(struct folio *folio) return false; } -static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) -{ - return false; -} - static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { return false; } -static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - return false; -} - static inline bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, - bool smaps) + bool smaps, bool in_pf) { return false; } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index a28c6100b491..d0c37d99917b 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -71,24 +71,50 @@ unsigned long huge_zero_pfn __read_mostly = ~0UL; bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags, - bool smaps) + bool smaps, bool in_pf) { - if (!transhuge_vma_enabled(vma, vm_flags)) + /* + * Explicitly disabled through madvise or prctl, or some + * architectures may disable THP for some mappings, for + * example, s390 kvm. + * */ + if ((vm_flags & VM_NOHUGEPAGE) || + test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) return false; - - if (vm_flags & VM_NO_KHUGEPAGED) + /* + * If the hardware/firmware marked hugepage support disabled. + */ + if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_NEVER_DAX)) return false; - /* Don't run khugepaged against DAX vma */ + /* khugepaged doesn't collapse DAX vma, but page fault is fine. */ if (vma_is_dax(vma)) + return in_pf; + + /* + * Special VMA and hugetlb VMA. + * Must be checked after dax since some dax mappings may have + * VM_MIXEDMAP set. + */ + if (vm_flags & VM_NO_KHUGEPAGED) return false; - /* Check alignment for file vma and size for both file and anon vma */ - if (!transhuge_vma_suitable(vma, (vma->vm_end - HPAGE_PMD_SIZE))) + /* + * Check alignment for file vma and size for both file and anon vma. + * + * Skip the check for page fault. Huge fault does the check in fault + * handlers. And this check is not suitable for huge PUD fault. + */ + if (!in_pf && + !transhuge_vma_suitable(vma, (vma->vm_end - HPAGE_PMD_SIZE))) return false; - /* Enabled via shmem mount options or sysfs settings. */ - if (shmem_file(vma->vm_file)) + /* + * Enabled via shmem mount options or sysfs settings. + * Must be done before hugepage flags check since shmem has its + * own flags. + */ + if (!in_pf && shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); if (!khugepaged_enabled()) @@ -99,7 +125,7 @@ bool hugepage_vma_check(struct vm_area_struct *vma, return false; /* Only regular file is valid */ - if (file_thp_enabled(vma)) + if (!in_pf && file_thp_enabled(vma)) return true; if (!vma_is_anonymous(vma)) @@ -111,9 +137,12 @@ bool hugepage_vma_check(struct vm_area_struct *vma, /* * THPeligible bit of smaps should show 1 for proper VMAs even * though anon_vma is not initialized yet. + * + * Allow page fault since anon_vma may be not initialized until + * the first page fault. */ if (!vma->anon_vma) - return smaps; + return (smaps || in_pf); return true; } diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 3afd87f8c0b1..2a676f37c921 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -473,7 +473,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags, false)) + if (hugepage_vma_check(vma, vm_flags, false, false)) __khugepaged_enter(vma->vm_mm); } } @@ -918,7 +918,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; - if (!hugepage_vma_check(vma, vma->vm_flags, false)) + if (!hugepage_vma_check(vma, vma->vm_flags, false, false)) return SCAN_VMA_CHECK; /* * Anon VMA expected, the address may be unmapped then @@ -1408,7 +1408,7 @@ void collapse_pte_mapped_thp(struct mm_struct *mm, unsigned long addr) * the valid THP. Add extra VM_HUGEPAGE so hugepage_vma_check() * will not fail the vma for missing VM_HUGEPAGE */ - if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE, false)) + if (!hugepage_vma_check(vma, vma->vm_flags | VM_HUGEPAGE, false, false)) return; /* Keep pmd pgtable for uffd-wp; see comment in retract_page_tables() */ @@ -2103,7 +2103,7 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; break; } - if (!hugepage_vma_check(vma, vma->vm_flags, false)) { + if (!hugepage_vma_check(vma, vma->vm_flags, false, false)) { skip: progress++; continue; diff --git a/mm/memory.c b/mm/memory.c index be724238a9d3..fee2884481f2 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4985,6 +4985,7 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, .gfp_mask = __get_fault_gfp_mask(vma), }; struct mm_struct *mm = vma->vm_mm; + unsigned long vm_flags = vma->vm_flags; pgd_t *pgd; p4d_t *p4d; vm_fault_t ret; @@ -4998,7 +4999,8 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, if (!vmf.pud) return VM_FAULT_OOM; retry_pud: - if (pud_none(*vmf.pud) && __transparent_hugepage_enabled(vma)) { + if (pud_none(*vmf.pud) && + hugepage_vma_check(vma, vm_flags, false, true)) { ret = create_huge_pud(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; @@ -5031,7 +5033,8 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, if (pud_trans_unstable(vmf.pud)) goto retry_pud; - if (pmd_none(*vmf.pmd) && __transparent_hugepage_enabled(vma)) { + if (pmd_none(*vmf.pmd) && + hugepage_vma_check(vma, vm_flags, false, true)) { ret = create_huge_pmd(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; From patchwork Thu Jun 16 17:48:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12884646 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0517C433EF for ; Thu, 16 Jun 2022 17:48:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CF5B36B007B; Thu, 16 Jun 2022 13:48:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CA3BF6B007E; Thu, 16 Jun 2022 13:48:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF59D6B0080; Thu, 16 Jun 2022 13:48:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A08246B007B for ; Thu, 16 Jun 2022 13:48:58 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 78FCC215BC for ; Thu, 16 Jun 2022 17:48:58 +0000 (UTC) X-FDA: 79584834756.22.1896764 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by imf18.hostedemail.com (Postfix) with ESMTP id 1E29B1C007C for ; Thu, 16 Jun 2022 17:48:57 +0000 (UTC) Received: by mail-pf1-f170.google.com with SMTP id y6so2093121pfr.13 for ; Thu, 16 Jun 2022 10:48:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QRTCWUz/v80cwRev6O447pCBMSdMFGV94XhB5oNBreQ=; b=Fm5abIw414WczmkiARiuOIB9/0ASIunFH2zFKcRTz+KBNKvcBYcmyc+rwz37WtNCMN Q91YdwCifI/utdZj36nMuMzQ4+xEtVBzz4U+2d8nLuOBijN4D6KkGIrGsfWfJqgzx88i N4serWC67hGg9tGb1K1E+W3IM/ktOQLXj8AjMMv2xSPVvxkLY8yTbIBB23hujhfKIjuF qR8R2Fl5Vg/GOWk7BTeueqJwqM3OPvf3ZTwxKrBpWmtBKIO+6AXuItMuxGrDC5dm3FVJ yOYhpdT39E1dzHRcviaiO05a4p0BqyMWfcINITHFuS6WnG9CxsdiuWgtrO7U4uWx+vZy 185Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QRTCWUz/v80cwRev6O447pCBMSdMFGV94XhB5oNBreQ=; b=YdDgYsOsLp+VrbTR1Xlj6sr6tnK30l/C3fOrEvl2nQOb2Ech156JgJgRWMLAFdq8un m0Tt/BVvol8HqA37AlBV0ZFHskunwa+Mu6tonBwVrF3bT0bfRbpzaZ37dqh+C0slFP8V qbb5abUROLlO3JBoeV11haKRh7doH5Wib5dEyvM2bQnvaL7grU4Vn9K/Ubz8lbJSlvFU AVPY2HQwW7NFq1rH3UiW3Ob6O0PkBkWQNuLGNkHHKi64mUxDd+XG5mboEFNIiZgCiFEi SyY6PBjJ3UZ04U32MbAiUCCKiBjpbDDMxoIg6ts2kKqXJwSFoaeglth8k1fGEPtZXcQN QXIg== X-Gm-Message-State: AJIora+S1UcmmhFE2W+pmUgfVVQs7oXtHFAfISXcZKX0deAt7uWwj2qp 0lHpDyyiv+rbs3gczj0OQMM= X-Google-Smtp-Source: AGRyM1vnH5OfaqcNXG9WeXIL0jSqmq4SmAlVZPgdAgD+zxW+JwxF7guIJVE0p/eybVg5QN/5LR3qRg== X-Received: by 2002:a05:6a00:1513:b0:51c:3ca8:47a4 with SMTP id q19-20020a056a00151300b0051c3ca847a4mr5967135pfu.48.1655401737201; Thu, 16 Jun 2022 10:48:57 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id z21-20020a17090a8b9500b001e8520b211bsm1818289pjn.53.2022.06.16.10.48.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Jun 2022 10:48:56 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, zokeefe@google.com, linmiaohe@huawei.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v5 PATCH 6/7] mm: khugepaged: reorg some khugepaged helpers Date: Thu, 16 Jun 2022 10:48:39 -0700 Message-Id: <20220616174840.1202070-7-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220616174840.1202070-1-shy828301@gmail.com> References: <20220616174840.1202070-1-shy828301@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Fm5abIw4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf18.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655401738; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QRTCWUz/v80cwRev6O447pCBMSdMFGV94XhB5oNBreQ=; b=5smDDNXWchLWj+6HihyU6uUOnEod/hrOxogDBk09N6HOhHFYXc5lOT8DOfjhuLUdVTN3uX I614mvZral8lK5gv7aolqhP5twGV/chxdvESj8M4yp5FqsVN/xWvBOcaU+EYPw6ezcXZxX WzYy71Kctj74VNVdw5jflv+oaNrMSGw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655401738; a=rsa-sha256; cv=none; b=iChhZrh4884W7rrSJEGnDwrLQ584HaPAbXFOwCVN0qjk1M3Y2kTV054jZJVx34hfDOosdN qT9huREmYjwESECXHbEvbxyKrYsJJNEJCivdpkovRjWournpYc0lZn13EL0QB2+Fjb5jD7 rueoL3F4MAlaas1603p4COcC/uR1aFM= X-Stat-Signature: 64dr6kdii846uyfwp54kyi9xfe4knucz X-Rspamd-Queue-Id: 1E29B1C007C X-Rspam-User: Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Fm5abIw4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf18.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam10 X-HE-Tag: 1655401737-85421 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The khugepaged_{enabled|always|req_madv} are not khugepaged only anymore, move them to huge_mm.h and rename to hugepage_flags_xxx, and remove khugepaged_req_madv due to no users. Also move khugepaged_defrag to khugepaged.c since its only caller is in that file, it doesn't have to be in a header file. Reviewed-by: Zach O'Keefe Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 8 ++++++++ include/linux/khugepaged.h | 14 -------------- mm/huge_memory.c | 4 ++-- mm/khugepaged.c | 18 +++++++++++------- 4 files changed, 21 insertions(+), 23 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index cd8a6c5d9fe5..ae3d8e2fd9e2 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -116,6 +116,14 @@ extern struct kobj_attribute shmem_enabled_attr; extern unsigned long transparent_hugepage_flags; +#define hugepage_flags_enabled() \ + (transparent_hugepage_flags & \ + ((1<flags)) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d0c37d99917b..0f2cce2d7041 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -117,11 +117,11 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (!in_pf && shmem_file(vma->vm_file)) return shmem_huge_enabled(vma); - if (!khugepaged_enabled()) + if (!hugepage_flags_enabled()) return false; /* THP settings require madvise. */ - if (!(vm_flags & VM_HUGEPAGE) && !khugepaged_always()) + if (!(vm_flags & VM_HUGEPAGE) && !hugepage_flags_always()) return false; /* Only regular file is valid */ diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 2a676f37c921..d8ebb60aae36 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -472,7 +472,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled()) { + hugepage_flags_enabled()) { if (hugepage_vma_check(vma, vm_flags, false, false)) __khugepaged_enter(vma->vm_mm); } @@ -763,6 +763,10 @@ static bool khugepaged_scan_abort(int nid) return false; } +#define khugepaged_defrag() \ + (transparent_hugepage_flags & \ + (1< X-Patchwork-Id: 12884647 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6604AC43334 for ; Thu, 16 Jun 2022 17:49:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DDDDE6B007E; Thu, 16 Jun 2022 13:49:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D8CFD6B0080; Thu, 16 Jun 2022 13:49:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C06286B0081; Thu, 16 Jun 2022 13:49:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A3BB46B007E for ; Thu, 16 Jun 2022 13:49:00 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 6A5386117B for ; Thu, 16 Jun 2022 17:49:00 +0000 (UTC) X-FDA: 79584834840.14.1C26F03 Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) by imf24.hostedemail.com (Postfix) with ESMTP id ADE4C18007D for ; Thu, 16 Jun 2022 17:48:59 +0000 (UTC) Received: by mail-pg1-f176.google.com with SMTP id e63so397264pgc.5 for ; Thu, 16 Jun 2022 10:48:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cbrfBk2owseXTl2TqGluY3ivP+501euUdLqfR5FVpCc=; b=F7+G3ESb9pcGIuGXoF2O/H03YvYsGpLhfNOWSQqYVP5yFLNGleH39xqFQ8cWIZuFho xg3heTbq6ePKtRc8GFH0juCyjezdh2hqyFjPBLzT9941kxHwTK7yLUeKVkiQKo6mDCxu se3LMMGqxW6HiTvu93X33WpwUBs73ecdVSU001ye3aHuc0r0loPqN+9Uj09NMTU9BWna ENe8DNgwYHvtpA6ITqyCZUYZN8mAOAXoWKj0lirO78K389Cubg2uLLRLQv4c9eITuXpW hzCgRhi/7HJwNubd85XqcCfY+TlKwnYPFZ+SI68tWv28Ig/OEhTEMpCtXj3Cf1NQPEVQ Vnzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cbrfBk2owseXTl2TqGluY3ivP+501euUdLqfR5FVpCc=; b=SgpKK4aIvyBh6aLVH7Vt79Sys7d2Yh6UMN9nGHfbi8/cWw0i2VM9CRV6VidlEeGwTD o98VQjTNwicrVRL2JOfGI/6FawiC8eoABLnKSmz/PhBlwzDFmh57Th5VaieJ6v1zsuuF XO0KFq8EytVutzJR1e+H5vn4YZCXKTM8uAAxWDl1S77MXdnuYw+DkxoJocH3Z/bzUDa7 zhepqd+u84mdzveK8yqRaYNADu0qsGnjfD+GvCb89zCFGHeP3btBkSEGTNSOWsOErCnM tvykdyY+FWckNNo8Lu8rF/b1Mxur70SiHkWvp6zTo2c0KBz4phAEvVZO1M1T1qEhJjAl OHVw== X-Gm-Message-State: AJIora9Bm8bVfQdBE2QnjiqYX1JP/gVZVA4Z+6zYntC3ZRiRoVqpWKK3 pjLFKoqUYCSaPTdrpVJXEcM= X-Google-Smtp-Source: AGRyM1vtFhWJ5FLzx5f3p3o/qmSs8NJRmjg3vyIZZSoaHxETIbvnT3f9vP9XDu44HCWoj/kKASXWTg== X-Received: by 2002:aa7:88cd:0:b0:51c:188e:fbeb with SMTP id k13-20020aa788cd000000b0051c188efbebmr5782611pff.81.1655401738804; Thu, 16 Jun 2022 10:48:58 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id z21-20020a17090a8b9500b001e8520b211bsm1818289pjn.53.2022.06.16.10.48.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Jun 2022 10:48:58 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, zokeefe@google.com, linmiaohe@huawei.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v5 PATCH 7/7] doc: proc: fix the description to THPeligible Date: Thu, 16 Jun 2022 10:48:40 -0700 Message-Id: <20220616174840.1202070-8-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220616174840.1202070-1-shy828301@gmail.com> References: <20220616174840.1202070-1-shy828301@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655401739; a=rsa-sha256; cv=none; b=ao8IR/UbB0N4INoZJtlT71ugLGlwziH/gJ3pi8uX0Mosolbali1+8EIwF+C28/0hid0zL9 Y/t4Nron7RFAHOVjRevHIgYy6OQTSoGdivRaUYhFEasnEJ/7RNcwci/qgphBMzms9D+Ulh S+VIARjfrlkkcihPf9wpnJhgpDiC9rI= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=F7+G3ESb; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of shy828301@gmail.com designates 209.85.215.176 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655401739; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cbrfBk2owseXTl2TqGluY3ivP+501euUdLqfR5FVpCc=; b=siGIiV1+aqEwGOiYtnQJYNCjXaPmD9Oao7PWV6bne4cOiqh1XFQ5lFia/X7X6M47Qppp7W LK/2iBCAu2frpDHowbuGJgMGHxxIxfdWOH6lR+tWAKEOtiNb/gr3hGhV+sw4LgEBAcA8EA zLkgSRKaL4j6UY4O9JNLebIEL1SeiIo= Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=F7+G3ESb; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of shy828301@gmail.com designates 209.85.215.176 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspam-User: X-Stat-Signature: sd7ryn1k17fe85kdccx68nxf18ip4wdi X-Rspamd-Queue-Id: ADE4C18007D X-Rspamd-Server: rspam08 X-HE-Tag: 1655401739-837713 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The THPeligible bit shows 1 if and only if the VMA is eligible for allocating THP and the THP is also PMD mappable. Some misaligned file VMAs may be eligible for allocating THP but the THP can't be mapped by PMD. Make this more explictly to avoid ambiguity. Reviewed-by: Zach O'Keefe Signed-off-by: Yang Shi --- Documentation/filesystems/proc.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/Documentation/filesystems/proc.rst b/Documentation/filesystems/proc.rst index 1bc91fb8c321..a5e41e636a1a 100644 --- a/Documentation/filesystems/proc.rst +++ b/Documentation/filesystems/proc.rst @@ -514,8 +514,10 @@ replaced by copy-on-write) part of the underlying shmem object out on swap. "SwapPss" shows proportional swap share of this mapping. Unlike "Swap", this does not take into account swapped out page of underlying shmem objects. "Locked" indicates whether the mapping is locked in memory or not. + "THPeligible" indicates whether the mapping is eligible for allocating THP -pages - 1 if true, 0 otherwise. It just shows the current status. +pages as well as the THP is PMD mappable or not - 1 if true, 0 otherwise. +It just shows the current status. "VmFlags" field deserves a separate description. This member represents the kernel flags associated with the particular virtual memory area in two letter