From patchwork Tue Jul 11 20:20:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13309337 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFF6AEB64DD for ; Tue, 11 Jul 2023 20:21:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 14B466B0072; Tue, 11 Jul 2023 16:21:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0AD926B0075; Tue, 11 Jul 2023 16:21:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF3796B0078; Tue, 11 Jul 2023 16:20:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id CFE846B0072 for ; Tue, 11 Jul 2023 16:20:59 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id AD0A1B020B for ; Tue, 11 Jul 2023 20:20:59 +0000 (UTC) X-FDA: 81000449838.06.05AE24F Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf19.hostedemail.com (Postfix) with ESMTP id 071E11A0003 for ; Tue, 11 Jul 2023 20:20:57 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=wQ2iTu3C; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689106858; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GLnKPguWFW7/+2NrpNM1tuYQ/qesy2XQRrzmH9bU/NA=; b=DpWnO/s+gEKXi27wf90/5VQGveJlhYevCydzMbKLXEoyAEUyyDl5zxcOmM3Ze8afrKt32P vJ+/GodffFAFIxyWlyIU8DihnMwcwok7mrBbDW+5Z4AcxKMJE4MHg9YKknmevaydvBBP3U PwSyVPeD6/XtWDYiJhHzc1PmXysMG/8= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=wQ2iTu3C; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689106858; a=rsa-sha256; cv=none; b=zk0y2Ip0sJkaG8m54A8vDzPFPSQFOmhqSrTEKRFFyItosYRPLuRKkgflZvFZU/mvHaqOww bM5aTvpuHacEc7b1Fnf8FBWgw7jRUeb6hGjAS2n2PVaWd+B2Ipinu7Yc53H1qeBUa+XgJf 5Ejbq2egTEy5/XYCFmP3aAGMPvzwGtQ= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=GLnKPguWFW7/+2NrpNM1tuYQ/qesy2XQRrzmH9bU/NA=; b=wQ2iTu3CFpFz2xaUFdin9qQcLg Zo6SBBpIiOLLIMV6swSA+k3EKgZiUTrGU0JtQ0GqAtC1zBFq/djG4qxyKgxXL2rELIMTjX7PST0+t wEF09/oTC9SpZ1eYMICRhU549nBmS1JJGjx1yl6+G1tVkWh32Nnm4jfwsuZ1yc1aLIxLimEAr0Trj agjHHSe1c13m7UzUObkj9h9Ps3CS52CsrM3yseZX/Eq7zh8qaB/t5l99NlgY7UH+vEG1mGH8Nt+LF Xh2jqTBfKjMtc1j99Is0Levj2AGXa208pRFmDKx3Lc0dmOmQC3OHcoSsJ7qGFeTWMSqRSo0MyyldI H8WIznMA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qJJqr-00G1QB-Ju; Tue, 11 Jul 2023 20:20:49 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , Arjun Roy , Eric Dumazet , Suren Baghdasaryan , linux-fsdevel@vger.kernel.org, Punit Agrawal Subject: [PATCH v2 3/9] mm: Move FAULT_FLAG_VMA_LOCK check from handle_mm_fault() Date: Tue, 11 Jul 2023 21:20:41 +0100 Message-Id: <20230711202047.3818697-4-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230711202047.3818697-1-willy@infradead.org> References: <20230711202047.3818697-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 071E11A0003 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: gg6g7tyziuf8ibtksy3ycftdyah5oqen X-HE-Tag: 1689106857-283340 X-HE-Meta: U2FsdGVkX1+hxvZagdc935ai0QbuhjhHPz4s12DrKX0vch8GTzmC33Rd4NmAeJdMi0KJl4Bo/PIyQqOkr4gAMgvE2MBaM3q4kxvClZgJg4uyne9X/APsCwohiHeP7X6uiaEUNSqkyKd7iYqsqR+jNExI09kqcOmjXJdt6HvNLJilZuP/qr5l8Bk1ft1LaaNuJYairqWgwrFXSq6VzGNVdNh1vy78FgPjWwOuwe/g+fdzGI8ABvCp5BbmOXOD1JNLi7RFXSKrLebYYc22RO4czV4RvnD3SF1RFmT61/DPY+HliM8gCZxsNPxOkVRaCv8vlXvFbezL3I8DI52eHDxd8bB7I6BhrJNra02j2xHTwCuZwkbn4TUGXPY7N36PvyuhqP0S1hoSSnyLxSDPzXLxfH+gEBPI6SmGySZhdCEFnBILGHahi0gygVbTKPBAUVrTkwJMFijGPlPIHcFPQMcH2xwe8QBqYepR1fgrY/iJGvuJDkJwBJUhRg3Yo+eZW+WM+GIT3BNRKBelmj+ERocr1s35fKQHdqx8CR7gq5lZIwisRWFolvHK5Yb+T/PkbZfz11hdxCe0w0F8Nhj9i2CU35dZrkQl1z0sLEaMmxHZvDFo8dcfsWuFd/69ac2mRjqLgWWkZfZ0re11mqFMcZ7M3RR9+N5VjcD1OcLYcoX2NZX9dBsc+GprTTFWv+VYgevwv+5w+AR59lBc+jDrrjesGYUeakU5zm6lNVzbvmYDscgpvvl2DCLlkF42LbIfBEhk6/0sW65MiLAzdwdviN+tPRMhhopmT73rLkB1R21hTnGrSxEEwB8lJAY0KViHtiSQoC9U0rjSd2HGyZO9o55DYUASkxrhCgWSu2/1XFOdxg02DgtzaNhUhLDkf+TF/Al6tpaEdXo1gPlmGsOO0sHw/ObeEImcVR5oPdyeWwztvrcV0gZ8Mts/yFYDMOdeASxPrFmWWCxnZgnFoloT7Kz pvcz+myj T+bLUIF+tEeuUocJNgcesvvXdBM0FUa/LjyhfzSyOKKf3jbtK5B8uu0W0TC9ockFr8aLZc4g3lxwoMHC2nseSu4u4wkE7u1sn9x0tEAyx9emjn511PlprYLCWPNwZp/tBnWz/OiE/p3EM7LBzT/Ix3aj5b4bLdbfZjhbUSZ92Qfu7wTjrzVfTwMOOkz0K9cZJD/aEw7J2/fTxhB5Oy1dRSZNz86cx02J/I0Bg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Handle a little more of the page fault path outside the mmap sem. The hugetlb path doesn't need to check whether the VMA is anonymous; the VM_HUGETLB flag is only set on hugetlbfs VMAs. There should be no performance change from the previous commit; this is simply a step to ease bisection of any problems. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Suren Baghdasaryan --- mm/hugetlb.c | 6 ++++++ mm/memory.c | 18 +++++++++--------- 2 files changed, 15 insertions(+), 9 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index e4a28ce0667f..109e1ff92bc8 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6063,6 +6063,12 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, int need_wait_lock = 0; unsigned long haddr = address & huge_page_mask(h); + /* TODO: Handle faults under the VMA lock */ + if (flags & FAULT_FLAG_VMA_LOCK) { + vma_end_read(vma); + return VM_FAULT_RETRY; + } + /* * Serialize hugepage allocation and instantiation, so that we don't * get spurious allocation failures if two CPUs race to instantiate diff --git a/mm/memory.c b/mm/memory.c index f2dcc695f54e..6eda5c5f2069 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4998,10 +4998,10 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) } /* - * By the time we get here, we already hold the mm semaphore - * - * The mmap_lock may have been released depending on flags and our - * return value. See filemap_fault() and __folio_lock_or_retry(). + * On entry, we hold either the VMA lock or the mmap_lock + * (FAULT_FLAG_VMA_LOCK tells you which). If VM_FAULT_RETRY is set in + * the result, the mmap_lock is not held on exit. See filemap_fault() + * and __folio_lock_or_retry(). */ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, unsigned long address, unsigned int flags) @@ -5020,6 +5020,11 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, p4d_t *p4d; vm_fault_t ret; + if ((flags & FAULT_FLAG_VMA_LOCK) && !vma_is_anonymous(vma)) { + vma_end_read(vma); + return VM_FAULT_RETRY; + } + pgd = pgd_offset(mm, address); p4d = p4d_alloc(mm, pgd, address); if (!p4d) @@ -5247,11 +5252,6 @@ vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, goto out; } - if ((flags & FAULT_FLAG_VMA_LOCK) && !vma_is_anonymous(vma)) { - vma_end_read(vma); - return VM_FAULT_RETRY; - } - /* * Enable the memcg OOM handling for faults triggered in user * space. Kernel faults are handled more gracefully.