From patchwork Fri Jun 30 02:04:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13297513 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A5D1EB64D9 for ; Fri, 30 Jun 2023 02:04:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CADFD8D0006; Thu, 29 Jun 2023 22:04:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C10298D0001; Thu, 29 Jun 2023 22:04:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AAFBC8D0006; Thu, 29 Jun 2023 22:04:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 980A08D0001 for ; Thu, 29 Jun 2023 22:04:49 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 6340EA0A17 for ; Fri, 30 Jun 2023 02:04:49 +0000 (UTC) X-FDA: 80957770698.07.E978952 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf24.hostedemail.com (Postfix) with ESMTP id 9F676180007 for ; Fri, 30 Jun 2023 02:04:47 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=DUV7BVM9; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf24.hostedemail.com: domain of 3PjieZAYKCFMDFCz8w19916z.x97638FI-775Gvx5.9C1@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3PjieZAYKCFMDFCz8w19916z.x97638FI-775Gvx5.9C1@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688090687; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=V7p+xkeFpHpY+8imX6dpZwoTYVp943eibQ5qK+rLf3c=; b=jnhvjWUxG9fZiiar4pqup/+3EKvQQ4NUNLu+QBotxByWLJP5372qaBHQpxxKgSQ4iXop0f 9UcNx13RrHUFux6lbKAVvkhauiw91doI2hBiaJwoDci82egtTFAfJaJX56+rxJRG8c65w3 pSiueWKl5ozNiji+5+bSrPCYbFXbt6A= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=DUV7BVM9; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf24.hostedemail.com: domain of 3PjieZAYKCFMDFCz8w19916z.x97638FI-775Gvx5.9C1@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3PjieZAYKCFMDFCz8w19916z.x97638FI-775Gvx5.9C1@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688090687; a=rsa-sha256; cv=none; b=6P8/hD/H0zrUKIZPGJVNowtVUQ+MxqxhzdHrpXGqNgvxzsQpSs8CBrpzMndgVVb2PeQq6h YBk3uXm22TwEMu2DEIpirCmhDsii1YdSOiM9Bn+7FNieARPOlERRmFJW5lsjs1UcwKUfP7 T1gCs7XwJHT4uxesjg+RJPh7to0uTmc= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-56fffdea2d0so11520547b3.1 for ; Thu, 29 Jun 2023 19:04:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688090687; x=1690682687; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=V7p+xkeFpHpY+8imX6dpZwoTYVp943eibQ5qK+rLf3c=; b=DUV7BVM9nFkRrG4OmL2z1rbzNQPYA08eD/W6CM9ZTP8ycZScg2wT7L7+zDm47i3EEp YYXhyTKoKtmqRcvzOyTyV44bllZXd+7i1AMkEKCnp9VhDpEzAvKcS+2YEZK4NAZsYcoe y/bdw210HSJrZUvj1zbrUaU+YWomeS88ogbqihRlgl+T+b4TNg18Gr+BlED4NnaQkkkw f1IhR/XhDFUZ7ovgt4RYv6+3pQDpmsTR7as7++jPBwf7pFCGJ+iXgGbx70KDjUWJPZkG Jf9I49Iy4KlltuDt6TFCOy2p6d0dzZ9NTdp0L/acX6A4axhyfobo5rKVJIXrfSI6c/Al EGHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688090687; x=1690682687; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=V7p+xkeFpHpY+8imX6dpZwoTYVp943eibQ5qK+rLf3c=; b=ex6mAW8mkIk2L0eNUEtaJTEIVR9B/9ZJQpLJP8OJdqAZE32XpXkuPN1BOHWzJd165D Q5PMZUPQLNkZnvxUss0pmOAH9O4EfJgTKPWLGSGBjv/ZwzuFzUrwRWmumCW0Squ5AsQq aHFmYUz6iR+12gsdM5PEyeYOyKTUa+x79MDnDuus8yREOP2wMwFPZtUtjPvP/l5d7LGp Bskt6jyPiiYAar9evmqHWaKJnCxyckVvFe/0vF/kWMEJb8fhjKiqcKqeWMoE69oZnQcb a+C2A+2ZzZOQqltM+Q1wHKNxelAaTdP533y621T3Q+nMkHLPLwdJ/xXUdEkPqKzliBlm M7BA== X-Gm-Message-State: ABy/qLaWBBXjcRBCeCTe1QMq3BRe63IDgmVwN7STS+YtIZi/RPg0DJrn cEQSYgzqEw61Helg2Hn+7ljMX8YLOPU= X-Google-Smtp-Source: APBJJlEey6/2V3MIX/R8fPAMY5b+xzPIZFVw7itp9GblxpDJfWRGEmOCyuEH2lOm02vp1cSgPAo6FSaIDcg= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:1f11:a3d0:19a9:16e5]) (user=surenb job=sendgmr) by 2002:a5b:a01:0:b0:c1d:d3e6:d52e with SMTP id k1-20020a5b0a01000000b00c1dd3e6d52emr10821ybq.0.1688090686807; Thu, 29 Jun 2023 19:04:46 -0700 (PDT) Date: Thu, 29 Jun 2023 19:04:32 -0700 In-Reply-To: <20230630020436.1066016-1-surenb@google.com> Mime-Version: 1.0 References: <20230630020436.1066016-1-surenb@google.com> X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230630020436.1066016-4-surenb@google.com> Subject: [PATCH v6 3/6] mm: drop per-VMA lock when returning VM_FAULT_RETRY or VM_FAULT_COMPLETED From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: willy@infradead.org, hannes@cmpxchg.org, mhocko@suse.com, josef@toxicpanda.com, jack@suse.cz, ldufour@linux.ibm.com, laurent.dufour@fr.ibm.com, michel@lespinasse.org, liam.howlett@oracle.com, jglisse@google.com, vbabka@suse.cz, minchan@google.com, dave@stgolabs.net, punit.agrawal@bytedance.com, lstoakes@gmail.com, hdanton@sina.com, apopple@nvidia.com, peterx@redhat.com, ying.huang@intel.com, david@redhat.com, yuzhao@google.com, dhowells@redhat.com, hughd@google.com, viro@zeniv.linux.org.uk, brauner@kernel.org, pasha.tatashin@soleen.com, surenb@google.com, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, kernel-team@android.com X-Rspamd-Queue-Id: 9F676180007 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: e9qxkdf5o83mxhqx8f7ny5gigtiatmxa X-HE-Tag: 1688090687-156184 X-HE-Meta: U2FsdGVkX185GTlWoiYBWspX4Iad3QnGZQ7JPn+Txaeri0uNQF3jfnZilthqyEFW5dHwkp3pQsbQAFGzSelc4iNtz6YAdDp5+J6nCcI2zdF8gz/qFV95RhPni0+h9Q2PpSRK+GgeF104r42rXMg34uJazyw1H6J5Yy+aIWYYdx8WUSML3BPuT1OdpeKTCzadjX0cVl7GH9H1ImDmrfTiLe5nWLeAHYSpaglDfZ9Asy2MNgEoBOXbHhVsGuFpCptBH6Xdq9OIJxLZlZ95rbbx8ds9gw5f/WqEgACwhCmkxK448e3abCXFGENe2QTyRLIJr49UaGHq/ey3uzR4ZZB/43o1bZcrP/AnmbjCmU8mHbsgGw3jA8JEMcdaBHGxRKvKNpg3Yz7nMlWKgUIqqbYrpSV/OZopaN76rin8MuBxIbzkju++ID/+mCzcvqgmmPQ04YcIjcDMJfWPuO//9p6ePBzht7v7d9wYnJdjawIZHyxj5WIjxYU+zY+6eYNIWbuNFrskw0Sf7T/goSeTeobK4L9a4CcJ2PJufErIxAjj2NoixfY+ehNlYDyfh1EZfR47O3ov/lpfr7ZTN4K0i+s3eJv1lkQIOiQCmbwOnnZ7DBT9OSEzGLPDWFELT7S5HxZboEbySmwT+oRHEkMOpjoJ1hVDr/RPkbWAPvDY5437Au6Xz3ze3MXisIZsRSmQDj5wYy4E9zIi3G9Lf9qi9NEGgv7xJ/o/pKFSVPs0m2WxR5/ultGPJa8pGKEHSIDPRiMKJxS1U7430CZ0jyZ4xo58VKTw0MPs6ZX0A9VoC1NkZM3g4j9MXChhoOl1ZZ+l9Z/cqmBmCi3DAUfY75qH8wVcwarq0/M8wix+WNh7OO5XrAbVp/UxiD64zmv/7BJ0nPxU02VhTgWAyPtHJ+kF+RlC3HouCnMFSNh2duLzpn55yAPVQ07fymSIsa39KaKrJou0dzNYDlI8r/zhdt07nOW U5qquRhg wZxnvdHQN87HHV24DOgm7B4Z16taoIl8J7PQT3X0lhkkTxQMJd2BJsUDNKaVlIn9JTD0UO0DgNxJ91b8TloOZZCXJUghaFsH6zcABW6MDmjBgBvKIhRjO9+5q12zGTbYcXFnVr9vuXZgU5dfOCf/g/K4u4aGJT/KMFH4snXJjDOdaMnRiDst84+GFkaGhSucARReimg7fB4cJrRO92flVnT+GT+ST8KSidnzsrj+5yuDrzWY6RifPnuMCOwffHJPCL8B/2iqzqMYnqpqo0jkYEXa/F8r0+Xvq8h6UJeIRcDUve96nwEp8hPtDAbhd8OZrz0GjB8wNG3m/goiHoaTqmmt52+NRU3qdtZ/U/LNllmdfu88jyq07hKl6QHp+0rUzP2iHlqR1uoDRNs9QA+T0Uxy6TQ7QBTvxvfwud9bI1ZvEg/y/FZRval/aU+rQSy2U3Q8oQl65qrVF1S7r5RA9Gr3CySC7EBxlifEtvqbX83iRQQvQFGWlfXBsecwuMChioEUvTMKtGyIt4GNkWnLP5VJmfy4AELmABLE66sK2aGe1qK+AWhC8axXUDXV+NkPbtIohgT9kdVDesAs+2H5kjO9YP4+kCIJjvzFE2lFriHexujjuvUigOPZIuAHTi37GoBX6sdHXXpGnPw9acnOS4EXuSkeHLX6+CtIf+TG+PKlvHEypf6c8IDnpyw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: handle_mm_fault returning VM_FAULT_RETRY or VM_FAULT_COMPLETED means mmap_lock has been released. However with per-VMA locks behavior is different and the caller should still release it. To make the rules consistent for the caller, drop the per-VMA lock when returning VM_FAULT_RETRY or VM_FAULT_COMPLETED. Currently the only path returning VM_FAULT_RETRY under per-VMA locks is do_swap_page and no path returns VM_FAULT_COMPLETED for now. Signed-off-by: Suren Baghdasaryan Acked-by: Peter Xu --- arch/arm64/mm/fault.c | 3 ++- arch/powerpc/mm/fault.c | 3 ++- arch/s390/mm/fault.c | 3 ++- arch/x86/mm/fault.c | 3 ++- mm/memory.c | 12 ++++++++++++ 5 files changed, 20 insertions(+), 4 deletions(-) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 935f0a8911f9..9d78ff78b0e3 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -602,7 +602,8 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, goto lock_mmap; } fault = handle_mm_fault(vma, addr, mm_flags | FAULT_FLAG_VMA_LOCK, regs); - vma_end_read(vma); + if (!(fault & (VM_FAULT_RETRY | VM_FAULT_COMPLETED))) + vma_end_read(vma); if (!(fault & VM_FAULT_RETRY)) { count_vm_vma_lock_event(VMA_LOCK_SUCCESS); diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index 5bfdf6ecfa96..82954d0e6906 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -489,7 +489,8 @@ static int ___do_page_fault(struct pt_regs *regs, unsigned long address, } fault = handle_mm_fault(vma, address, flags | FAULT_FLAG_VMA_LOCK, regs); - vma_end_read(vma); + if (!(fault & (VM_FAULT_RETRY | VM_FAULT_COMPLETED))) + vma_end_read(vma); if (!(fault & VM_FAULT_RETRY)) { count_vm_vma_lock_event(VMA_LOCK_SUCCESS); diff --git a/arch/s390/mm/fault.c b/arch/s390/mm/fault.c index dbe8394234e2..40a71063949b 100644 --- a/arch/s390/mm/fault.c +++ b/arch/s390/mm/fault.c @@ -418,7 +418,8 @@ static inline vm_fault_t do_exception(struct pt_regs *regs, int access) goto lock_mmap; } fault = handle_mm_fault(vma, address, flags | FAULT_FLAG_VMA_LOCK, regs); - vma_end_read(vma); + if (!(fault & (VM_FAULT_RETRY | VM_FAULT_COMPLETED))) + vma_end_read(vma); if (!(fault & VM_FAULT_RETRY)) { count_vm_vma_lock_event(VMA_LOCK_SUCCESS); goto out; diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index e8711b2cafaf..56b4f9faf8c4 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -1341,7 +1341,8 @@ void do_user_addr_fault(struct pt_regs *regs, goto lock_mmap; } fault = handle_mm_fault(vma, address, flags | FAULT_FLAG_VMA_LOCK, regs); - vma_end_read(vma); + if (!(fault & (VM_FAULT_RETRY | VM_FAULT_COMPLETED))) + vma_end_read(vma); if (!(fault & VM_FAULT_RETRY)) { count_vm_vma_lock_event(VMA_LOCK_SUCCESS); diff --git a/mm/memory.c b/mm/memory.c index 0ae594703021..5f26c56ce979 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3730,6 +3730,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) if (vmf->flags & FAULT_FLAG_VMA_LOCK) { ret = VM_FAULT_RETRY; + vma_end_read(vma); goto out; } @@ -5182,6 +5183,17 @@ static vm_fault_t sanitize_fault_flags(struct vm_area_struct *vma, !is_cow_mapping(vma->vm_flags))) return VM_FAULT_SIGSEGV; } +#ifdef CONFIG_PER_VMA_LOCK + /* + * Per-VMA locks can't be used with FAULT_FLAG_RETRY_NOWAIT because of + * the assumption that lock is dropped on VM_FAULT_RETRY. + */ + if (WARN_ON_ONCE((*flags & + (FAULT_FLAG_VMA_LOCK | FAULT_FLAG_RETRY_NOWAIT)) == + (FAULT_FLAG_VMA_LOCK | FAULT_FLAG_RETRY_NOWAIT))) + return VM_FAULT_SIGSEGV; +#endif + return 0; }