From patchwork Thu Jun 16 17:48:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12884642 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4139FC43334 for ; Thu, 16 Jun 2022 17:48:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C08C26B0074; Thu, 16 Jun 2022 13:48:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B917F6B0075; Thu, 16 Jun 2022 13:48:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 947636B0078; Thu, 16 Jun 2022 13:48:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 849D66B0074 for ; Thu, 16 Jun 2022 13:48:52 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 5E856162E for ; Thu, 16 Jun 2022 17:48:52 +0000 (UTC) X-FDA: 79584834504.24.93BD23B Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by imf26.hostedemail.com (Postfix) with ESMTP id A37BA1400A2 for ; Thu, 16 Jun 2022 17:48:51 +0000 (UTC) Received: by mail-pf1-f170.google.com with SMTP id w21so2147488pfc.0 for ; Thu, 16 Jun 2022 10:48:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Lf1Kp53gV5xVxcPrzuVVyrzHKBnqiH74ZtDbvhyF8zw=; b=qHbvu3U4WI6XTL+Bb7EcWbfSgzpXVwmBwdYi5b4fTDSMY/tMFb3YRbJy105vXefOI1 v+XLHWPz/vB0UgVN0u2mh1l+NLdiJ88GcEjUAoGZvoskG4KtWLQvWM29G0qBeynD9M9l ejsjQKiDU16xatYVZWK11/4EP942U/xh+kghEmAO2eS/VqNHbCd9twMGhSrNDNbfL3Fx 8AuHMCmNpCKAfw1BDm1meeWm/ujGolhpZ8paAlpESIC987HvF+vMDiRVmzs8ryP6UOof 4f4TUCiMMsSW1Qf6i2NxxcKj798ou3uvQlxSTGuXQ1ILAc8RKJd5hZjzKuHbjwjJ1504 8VjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Lf1Kp53gV5xVxcPrzuVVyrzHKBnqiH74ZtDbvhyF8zw=; b=HAJmIU6twwYO3cOp84oJCX3lVtFFfRoUKMb5ZuicIAnaVD2qpMlLsvB34o8rEYto5z wBaaSQpOP6y+gTNInkKqjbbdJEo8F8gBPDqQOHDfrtWtsmELcM5+hzRCzHY5wWdVD1gn byJbuaeb634OhaDZWIqj66M3qG9NatnXHOsdiRaNIDp6BL1MKev3yNssijyYwFYkCoZJ D696FgjzSWa0WSxZ+JdjVoPUPXgX8WifxBM/aXipdHlPkcX5knZyMcei5aJmmu0MAXZI O7qEs2zlfnCdKs+xQbLXhUIDMFkqQSwJtwrJPxc+yqsFGIVN6+4IUkjVuTzdCjoKWYQJ rulg== X-Gm-Message-State: AJIora81+6XjJxKMtC1/9IY6/gHT7c6flL5fu6VULXL95psD+zDWg5T4 Kx/bQV7ueF5Ij0XdGeDqobI= X-Google-Smtp-Source: AGRyM1sILDxv0QZoQXrOLOSuJVQZEjJnBYzToTfeZrKnwvvpIW3ALa81CvsweBmzReuWF3hmAbqADA== X-Received: by 2002:a63:894a:0:b0:3fc:a724:578c with SMTP id v71-20020a63894a000000b003fca724578cmr5543251pgd.499.1655401730679; Thu, 16 Jun 2022 10:48:50 -0700 (PDT) Received: from localhost.localdomain (c-67-174-241-145.hsd1.ca.comcast.net. [67.174.241.145]) by smtp.gmail.com with ESMTPSA id z21-20020a17090a8b9500b001e8520b211bsm1818289pjn.53.2022.06.16.10.48.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Jun 2022 10:48:50 -0700 (PDT) From: Yang Shi To: vbabka@suse.cz, kirill.shutemov@linux.intel.com, willy@infradead.org, zokeefe@google.com, linmiaohe@huawei.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [v5 PATCH 2/7] mm: thp: consolidate vma size check to transhuge_vma_suitable Date: Thu, 16 Jun 2022 10:48:35 -0700 Message-Id: <20220616174840.1202070-3-shy828301@gmail.com> X-Mailer: git-send-email 2.26.3 In-Reply-To: <20220616174840.1202070-1-shy828301@gmail.com> References: <20220616174840.1202070-1-shy828301@gmail.com> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=qHbvu3U4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=shy828301@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655401731; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Lf1Kp53gV5xVxcPrzuVVyrzHKBnqiH74ZtDbvhyF8zw=; b=tub632N2fPZAZTrUsqtXnLk5xx5oiGHwzBz6716wGw9Fpc3m1JbCelYt2oL9ioz6QWVeA3 D8nhYwx21IHsW/pEmIoFzql43kDlJMnXSNnmPwWHkAT5b9GKPthYArPcUyP1MG+5l5+TqR pfLiNX2qXxqdctYt7fCsb8Sq06T4Pjk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655401731; a=rsa-sha256; cv=none; b=7UuPyRl4qdwpPBXmcXwJj6ZPvvxCuvAosYK1fTWuM2XDSOLzWM9IOvf2WGNAoYrFpkBANK XowZjg5rr8a9n9dZH62Z1+ELy6W4Zi2+24T32GiSUu+nbQ7iMDZtDBWO5qoPOpYwTYEFh4 tQaLVAgTa92Skv62PHqXbid54cASWyc= X-Rspam-User: X-Stat-Signature: hxr3qfq9z5hdwcc3ota1p8s4f5xgty5q X-Rspamd-Queue-Id: A37BA1400A2 Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=qHbvu3U4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=shy828301@gmail.com X-Rspamd-Server: rspam05 X-HE-Tag: 1655401731-927129 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are couple of places that check whether the vma size is ok for THP or whether address fits, they are open coded and duplicate, use transhuge_vma_suitable() to do the job by passing in (vma->end - HPAGE_PMD_SIZE). Move vma size check into hugepage_vma_check(). This will make khugepaged_enter() is as same as khugepaged_enter_vma(). There is just one caller for khugepaged_enter(), replace it to khugepaged_enter_vma() and remove khugepaged_enter(). Reviewed-by: Zach O'Keefe Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 11 +++++++++++ include/linux/khugepaged.h | 14 -------------- mm/huge_memory.c | 2 +- mm/khugepaged.c | 19 ++++++------------- 4 files changed, 18 insertions(+), 28 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 648cb3ce7099..8a5a8bfce0f5 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -116,6 +116,17 @@ extern struct kobj_attribute shmem_enabled_attr; extern unsigned long transparent_hugepage_flags; +/* + * Do the below checks: + * - For file vma, check if the linear page offset of vma is + * HPAGE_PMD_NR aligned within the file. The hugepage is + * guaranteed to be hugepage-aligned within the file, but we must + * check that the PMD-aligned addresses in the VMA map to + * PMD-aligned offsets within the file, else the hugepage will + * not be PMD-mappable. + * - For all vmas, check if the haddr is in an aligned HPAGE_PMD_SIZE + * area. + */ static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long addr) { diff --git a/include/linux/khugepaged.h b/include/linux/khugepaged.h index 392d34c3c59a..31ca8a7f78f4 100644 --- a/include/linux/khugepaged.h +++ b/include/linux/khugepaged.h @@ -51,16 +51,6 @@ static inline void khugepaged_exit(struct mm_struct *mm) if (test_bit(MMF_VM_HUGEPAGE, &mm->flags)) __khugepaged_exit(mm); } - -static inline void khugepaged_enter(struct vm_area_struct *vma, - unsigned long vm_flags) -{ - if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled()) { - if (hugepage_vma_check(vma, vm_flags)) - __khugepaged_enter(vma->vm_mm); - } -} #else /* CONFIG_TRANSPARENT_HUGEPAGE */ static inline void khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm) { @@ -68,10 +58,6 @@ static inline void khugepaged_fork(struct mm_struct *mm, struct mm_struct *oldmm static inline void khugepaged_exit(struct mm_struct *mm) { } -static inline void khugepaged_enter(struct vm_area_struct *vma, - unsigned long vm_flags) -{ -} static inline void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 4f9bbb4eab23..b530462c4493 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -726,7 +726,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) return VM_FAULT_FALLBACK; if (unlikely(anon_vma_prepare(vma))) return VM_FAULT_OOM; - khugepaged_enter(vma, vma->vm_flags); + khugepaged_enter_vma(vma, vma->vm_flags); if (!(vmf->flags & FAULT_FLAG_WRITE) && !mm_forbids_zeropage(vma->vm_mm) && diff --git a/mm/khugepaged.c b/mm/khugepaged.c index b1dab94c0f1e..db0b334a7d1f 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -450,8 +450,8 @@ bool hugepage_vma_check(struct vm_area_struct *vma, if (vma_is_dax(vma)) return false; - if (vma->vm_file && !IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - - vma->vm_pgoff, HPAGE_PMD_NR)) + /* Check alignment for file vma and size for both file and anon vma */ + if (!transhuge_vma_suitable(vma, (vma->vm_end - HPAGE_PMD_SIZE))) return false; /* Enabled via shmem mount options or sysfs settings. */ @@ -512,9 +512,7 @@ void khugepaged_enter_vma(struct vm_area_struct *vma, unsigned long vm_flags) { if (!test_bit(MMF_VM_HUGEPAGE, &vma->vm_mm->flags) && - khugepaged_enabled() && - (((vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK) < - (vma->vm_end & HPAGE_PMD_MASK))) { + khugepaged_enabled()) { if (hugepage_vma_check(vma, vm_flags)) __khugepaged_enter(vma->vm_mm); } @@ -950,7 +948,6 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, struct vm_area_struct **vmap) { struct vm_area_struct *vma; - unsigned long hstart, hend; if (unlikely(khugepaged_test_exit(mm))) return SCAN_ANY_PROCESS; @@ -959,9 +956,7 @@ static int hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address, if (!vma) return SCAN_VMA_NULL; - hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; - hend = vma->vm_end & HPAGE_PMD_MASK; - if (address < hstart || address + HPAGE_PMD_SIZE > hend) + if (!transhuge_vma_suitable(vma, address)) return SCAN_ADDRESS_RANGE; if (!hugepage_vma_check(vma, vma->vm_flags)) return SCAN_VMA_CHECK; @@ -2147,10 +2142,8 @@ static unsigned int khugepaged_scan_mm_slot(unsigned int pages, progress++; continue; } - hstart = (vma->vm_start + ~HPAGE_PMD_MASK) & HPAGE_PMD_MASK; - hend = vma->vm_end & HPAGE_PMD_MASK; - if (hstart >= hend) - goto skip; + hstart = round_up(vma->vm_start, HPAGE_PMD_SIZE); + hend = round_down(vma->vm_end, HPAGE_PMD_SIZE); if (khugepaged_scan.address > hend) goto skip; if (khugepaged_scan.address < hstart)