From patchwork Tue Jul 11 20:20:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13309338 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A966EB64DC for ; Tue, 11 Jul 2023 20:21:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5DEDD6B0075; Tue, 11 Jul 2023 16:21:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 591966B0078; Tue, 11 Jul 2023 16:21:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4301E6B007B; Tue, 11 Jul 2023 16:21:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 344BC6B0075 for ; Tue, 11 Jul 2023 16:21:01 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D7DD140397 for ; Tue, 11 Jul 2023 20:21:00 +0000 (UTC) X-FDA: 81000449880.29.C72F838 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf21.hostedemail.com (Postfix) with ESMTP id 1DC521C0022 for ; Tue, 11 Jul 2023 20:20:58 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="W/ZMMD3u"; dmarc=none; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689106859; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SWi2hnJa6UVy5RDmiU5nvVsgG2SVJpIZTkG5op8he4U=; b=p1obY8DoZN23BRX+pD0AYbRRgEjKgD9zltsNhPtq1kYNVLKjZrJP48BBSxV62FTYKhWmoR q4GeRQq2u1SUdcrsUgYRYl/2Wuyq1aUCBr7vwzSixYA951mzLM6wCzPuc44g4qjzW+AfNB wSjhYc3IQpgQdoqnVcezwbExx6s/SS0= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="W/ZMMD3u"; dmarc=none; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689106859; a=rsa-sha256; cv=none; b=NnptW8rIg+kS+xgmEHspQfP1jLfeUMtCkf3ivWGM7bEXEnJLeel7EW2gM9CYO8jWqCKL4x pCKo9C4gEK9JfKglnMF+yIkoy0gB0wPZBf+lZV5Xs7WSIiZQ5ax25NUJTpkKU6umvlYzzE q8mU3WhuubEXn9UuEXswEDX8+9GyujE= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=SWi2hnJa6UVy5RDmiU5nvVsgG2SVJpIZTkG5op8he4U=; b=W/ZMMD3uCnkA6CMhMyDdz/rsx8 js1sAmaboeOuGlO/QfP/tkIY64sHyudD8yzOcDG8w95UyCCrvW3+lRjx/ndAFDyBdq1yILWDVxX8s +xR3hsJqKfIvR/awLGd+z8FNyTkV10J7mb6UVwioKberqclMIvX8E8is2PY1eBNf85tLDI9xFcP6J ojVNyWZKBjenQ1R0trgEulrgdvA3/JGfcRNl/lFbRuHNBJHBPsQg4e2GGLqbbjN8nrFGqOYg/Pafl pfiG+sUyiO7cHtXp32JUp31f8x4BgNQtw5g4lLaF2ybHg3RjpLPYkq3hVEu9oje9EIX+X7z7w25Lp j1KRSO/g==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qJJqr-00G1QD-Un; Tue, 11 Jul 2023 20:20:50 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , Arjun Roy , Eric Dumazet , Suren Baghdasaryan , linux-fsdevel@vger.kernel.org, Punit Agrawal Subject: [PATCH v2 4/9] mm: Move FAULT_FLAG_VMA_LOCK check into handle_pte_fault() Date: Tue, 11 Jul 2023 21:20:42 +0100 Message-Id: <20230711202047.3818697-5-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230711202047.3818697-1-willy@infradead.org> References: <20230711202047.3818697-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: th8ehwzomugdt7816x7q5gr4zx687n5o X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1DC521C0022 X-HE-Tag: 1689106858-585703 X-HE-Meta: U2FsdGVkX1+UzyxcysV/cyzOKRR3/KcqBZtR2mP+yLuq2RMwAxoSZGw9jG26obKcWZ5wwnRp+5SCrK7c3FKmBwsiYwOd3NY5nc6fh7ulDuEOzV163XlG4D75GaokqDTe6KoDTYIfPuAhVrf599OVE0L9Pp9DJ/SpIcUXDhZZNyfanGdIiTPPxX0H/7tHu4LZJniWbSj7MtovvvrVASHD3ElKdAR72rwxAyeFTFoEZ73DthY3Ri5iZZM2s89fHSRx/0jkUtl1BIdVeq0CDsrR3SmiYclZ8vcaQEsTBIrv+WSm+ltvK+mPzyuvGlBpx+OZBn5yY9ePbR9YIXGNwsCdL+NcbKQMBKFQgEPJqa6iLn76DERST65qgt5FwulMDuAULTgTEkX/CFOMYXEoLN9AaxStft2rQyNU0x95JE8hk1oVaVr51/j3vwrudWsL/1EJloxOOV+Q1l74vwBdraGof35UzUGUjujOrP0Ht+0a4YaAKqwls0sIgtUJVXVVzVUZp+dZlK4nV0QJqTUVmyD8ORZLs9LplhtSMV6pATzo82vLdBmzYzLaKAppboTDqL+AXZ99OPTDVBpoA4GqkkIgQTuJ6ACzHoO3C+jHFc+U0i0bS4sxwyqCKMMzqfkK7voWd2oT1L51P90r8NfbHH+Pw0dPJebVF2AUekzG2sPTbBJpFVf/rdZAKcxrmHZg0ZOb/mYaketaq/bxGVCuYAVhrB2Kkv5ibO4fbbl+LH1O2zOMD1qYyXs5KMljkF3KgkncuHmet0rABmW771tp7oBOjyRUkvFBBvv3dhLUG+L/wcLbRcspfOjb97jSj42oXbnvAGwy2MRhFCXkoGIK9lp2yfgKeV70mQjPTv85+4sOoa+8DxMvPQpwbJnv6i6e88A2x/ISL2JKMsamj5Hr7411r5ETSbwrfqwceMok3QwA/xopqJTXd5fZinYnUXDnPqtL0l23LCKxZMce0+fVEeL 7wfvMpAK SR3ZmoH81mbo3ajS9XWxAibbRrTJYaVVEeyfTZGZRT3M2PUtL/31Ws2VlKcxZkfA/ce7T12a2kFCX7IJKwApdQgKMDcoNsUkxyEyTKonwUV10CGNPLNBX3gzSutyFtjBb6LHysIRUiudD/RGVf64B+aFEpzOCZ5PbSOz7UeX1g8YBe9zKSecexoZcCD6AK1Xmp6son9B85hu+ohub8MkdBaAsY88UD7sYQCdU X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Push the check down from __handle_mm_fault(). There's a mild upside to this patch in that we'll allocate the page tables while under the VMA lock rather than the mmap lock, reducing the hold time on the mmap lock, since the retry will find the page tables already populated. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Suren Baghdasaryan --- mm/memory.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 6eda5c5f2069..52f7fdd78380 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4924,6 +4924,11 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) { pte_t entry; + if ((vmf->flags & FAULT_FLAG_VMA_LOCK) && !vma_is_anonymous(vmf->vma)) { + vma_end_read(vmf->vma); + return VM_FAULT_RETRY; + } + if (unlikely(pmd_none(*vmf->pmd))) { /* * Leave __pte_alloc() until later: because vm_ops->fault may @@ -5020,11 +5025,6 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, p4d_t *p4d; vm_fault_t ret; - if ((flags & FAULT_FLAG_VMA_LOCK) && !vma_is_anonymous(vma)) { - vma_end_read(vma); - return VM_FAULT_RETRY; - } - pgd = pgd_offset(mm, address); p4d = p4d_alloc(mm, pgd, address); if (!p4d)