From patchwork Mon Jul 24 18:54:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 13325260 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A928DC04A94 for ; Mon, 24 Jul 2023 18:54:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE7C26B0078; Mon, 24 Jul 2023 14:54:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D216B6B007B; Mon, 24 Jul 2023 14:54:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C0FC96B007D; Mon, 24 Jul 2023 14:54:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B2EC36B0078 for ; Mon, 24 Jul 2023 14:54:23 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 7B649C0B70 for ; Mon, 24 Jul 2023 18:54:23 +0000 (UTC) X-FDA: 81047406006.10.F9AE02D Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf15.hostedemail.com (Postfix) with ESMTP id B5AF0A0003 for ; Mon, 24 Jul 2023 18:54:21 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=LpXGzVO1; dmarc=none; spf=none (imf15.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690224861; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=S+dCr3wFmOnYx33aA8EkljEdRO/JPPjH0b8bNo8hRK8=; b=LzMTMFNTCXkUs6so8E+V8HAkYhzXbHkj/Krss02YhfOQQLxj1ZnEX/EYaII4pn5FTspJTJ 2HHGfsq2xaCkMXdK7+enMNDSyahk6uPrfxF3xeHrDylLxnqabyU8YVLMQ1sJPVEewhmX6s 2mE4MURyp8vS+fHqgn52eriA04NpU2E= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=LpXGzVO1; dmarc=none; spf=none (imf15.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690224861; a=rsa-sha256; cv=none; b=o8onD3GmUzPFc/zbUoZbShauQWW7Phg+OpQxNfzVBpIEmR//gLR38oSfMqT5ghniSsHAc9 dW1DnjSG47k4cLEsGfjDYA627S+jjfkNV7TNnnPOiL8P+BuPcyb3BMhU/fZ04qzQO/uSRf PndlmnRimaE0pH/2nDYuEU9ixIE9RVs= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=S+dCr3wFmOnYx33aA8EkljEdRO/JPPjH0b8bNo8hRK8=; b=LpXGzVO1V3NFVNKSce0PI1K3mI 00HJIlamGKCVSJTU/TaXpSYtxAGCJ00H8nAvYbCONsEJF+t6kq7PAN1mvhkItteW71rqdc1PhVEbq 2gY6Z8zsNCr9W04TlSMo9Hy2HGC1WDAHIYECDH4Sbo1jdx+UmSmFhImPpi4G+6hpt3omrFGnPCioY r5OetU3asujp/vCVz0A1TrJNtGePESTSBDleEdYtuuJx/GTh05EMuZxNBZKUDjxrff0WVp9hInezm QXszgBXraH7AudLUVjQqC8IyJoZKy1QmoyTQLjTacx21fFkIGpppl2591qthTSu8Cfpv95BJwk1oO JSJPo0dQ==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qO0hA-004iRD-Jf; Mon, 24 Jul 2023 18:54:12 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Punit Agrawal , Suren Baghdasaryan Subject: [PATCH v3 07/10] mm: Move FAULT_FLAG_VMA_LOCK check down from do_fault() Date: Mon, 24 Jul 2023 19:54:07 +0100 Message-Id: <20230724185410.1124082-8-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230724185410.1124082-1-willy@infradead.org> References: <20230724185410.1124082-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: B5AF0A0003 X-Stat-Signature: faife5x4kokc81qf7fa6e6pmq67bmnau X-Rspam-User: X-HE-Tag: 1690224861-113040 X-HE-Meta: U2FsdGVkX1+Y+Slnvhri/b5U/mHO2m0e6D1OmX2LrU4/19em3iskeW7EDrGMluIbP5IgsQ9usmilBrzG+ngZf33ZMjfcwIUR1r0E1xIo0ZcEMuURdsiPR4m1qXnRlTQytuPzLu4JES+E6+2QuGndjyTSvzbJKXw0LKIp79NNvv5MSFH7TkNL2M7YmcnRH/S4caZTA7wzBGRh9V8Vwi9tr+a4Q8Rsbvj/aXea82WMbMwGYU4yjO2Gk1GJDTffFap0KZ0njEiT5n9uEl10i7dwyKvZwuDQQHKQT4w40Wk0uruNo9DhbbjQdSfgoZFSZx2/B23mmOsym2XfVITvfnGwVtTa3bfotxc2cXWMOKbtfY3+vW88Yll5DYcM81qsirFxbufCW7mJ5sHCVPx9f5txPKZIMafpKMppiP6oIM0zTNoxwJrBF1c5SMFyn86u5GZk1EExl509Kcj3HSOIqMR96MYjsEtbNLgyc3LhllkhD4PUdw4aO36CpZnNj8LUILub9HTlmQ3wo756i0lecES3KwyosrNBCX2jPlj5XgsBJoDsw+bOJ0pXF3H+bxc8tsaSVKP8sri1Mz3YUDr9HeWgCkutB3hZsuqMshpVvg+L4b86VglJVxvrsKR3uM6yOXqcDvNoNukU7SwkCWxuUGJrs1pc1EoW3iuNyzGcm0C/T1mQIr1llNxi3Km4SgsNfT3lcsUb1OBcqHVeUupzOO7fynGpB8i7oQy9ivaREU0+mmYmdc4Y42nqKQKCrlyZIuSUe1A+7s6xXswBvlnswR4ISIAOQUIsPFvunYdIyro87Bg0pXobH2VQJ4qMC2huY1jBWj/HXWSIWF8Bl2lkKacmUUwEbuG4qjMNfj5okZYDNP8Ee2rKVp2N8FlLH8HjQwQbqtGeVEyiBXgzRZ2svm250XSpmCOuPd6F6aW7baFKakcar+LeS/HNyE+UXEs3Gas56M/sqBjEMHypE71wRYC KLBxsSmq 9pB2JXeaKUXfahJDKoPRD3H8qeTasMGO9cUnqtUaVnvOt29w4k2abxFSDyhZ/zAjCLFSu21i+HGbIj1CY1W8KuXnqteMBbftctlvgBFlsb8cASpwbM+j/VWUbsJHMdduxLte/UmCq2tYB/nGZHMJRUhMCLd9+nIj3Ap3MYZGGTCIqMLmVnJK2vn+knJy/nI3OzdKC+12iDtPMhPN3rW6PuCRH6NyTOectiaPioBwKB62vEI4lBVnhqfFMag== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Perform the check at the start of do_read_fault(), do_cow_fault() and do_shared_fault() instead. Should be no performance change from the last commit. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Suren Baghdasaryan --- mm/memory.c | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 11b337876477..627a2abb969b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4661,6 +4661,11 @@ static vm_fault_t do_read_fault(struct vm_fault *vmf) vm_fault_t ret = 0; struct folio *folio; + if (vmf->flags & FAULT_FLAG_VMA_LOCK) { + vma_end_read(vmf->vma); + return VM_FAULT_RETRY; + } + /* * Let's call ->map_pages() first and use ->fault() as fallback * if page by the offset is not ready to be mapped (cold cache or @@ -4689,6 +4694,11 @@ static vm_fault_t do_cow_fault(struct vm_fault *vmf) struct vm_area_struct *vma = vmf->vma; vm_fault_t ret; + if (vmf->flags & FAULT_FLAG_VMA_LOCK) { + vma_end_read(vma); + return VM_FAULT_RETRY; + } + if (unlikely(anon_vma_prepare(vma))) return VM_FAULT_OOM; @@ -4729,6 +4739,11 @@ static vm_fault_t do_shared_fault(struct vm_fault *vmf) vm_fault_t ret, tmp; struct folio *folio; + if (vmf->flags & FAULT_FLAG_VMA_LOCK) { + vma_end_read(vma); + return VM_FAULT_RETRY; + } + ret = __do_fault(vmf); if (unlikely(ret & (VM_FAULT_ERROR | VM_FAULT_NOPAGE | VM_FAULT_RETRY))) return ret; @@ -4775,11 +4790,6 @@ static vm_fault_t do_fault(struct vm_fault *vmf) struct mm_struct *vm_mm = vma->vm_mm; vm_fault_t ret; - if (vmf->flags & FAULT_FLAG_VMA_LOCK){ - vma_end_read(vma); - return VM_FAULT_RETRY; - } - /* * The VMA was not fully populated on mmap() or missing VM_DONTEXPAND */