From patchwork Mon Dec 16 19:24:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13910235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8154E77183 for ; Mon, 16 Dec 2024 19:24:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3AC946B00A9; Mon, 16 Dec 2024 14:24:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 335316B00AA; Mon, 16 Dec 2024 14:24:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1D50F6B00AB; Mon, 16 Dec 2024 14:24:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 01E506B00A9 for ; Mon, 16 Dec 2024 14:24:28 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id A1D56160914 for ; Mon, 16 Dec 2024 19:24:28 +0000 (UTC) X-FDA: 82901797230.03.3FAD3B3 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) by imf30.hostedemail.com (Postfix) with ESMTP id 2C90580015 for ; Mon, 16 Dec 2024 19:23:29 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=zPjeKIge; spf=pass (imf30.hostedemail.com: domain of 3aX5gZwYKCCoYaXKTHMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--surenb.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=3aX5gZwYKCCoYaXKTHMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734377044; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NdPtaL44NpTjP9eS1K5KVvptksJqAFYIrPxgR/8blQY=; b=h+gNg4bMWQibSnjpkuc87PgwkNuvnaj+M1QqD3/DFMYwol+baRjkEQW8moObWF50BNCxag 6qRXda93fxwsIIjSqhfUEJ3gMoAzhhcTWcZDfpBq4T9AKNTChc0TYD1lCBVt+z/Ltqa0uf bIKPHOjYb9sCfjg4su4tv5wcXhATEZk= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=zPjeKIge; spf=pass (imf30.hostedemail.com: domain of 3aX5gZwYKCCoYaXKTHMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--surenb.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=3aX5gZwYKCCoYaXKTHMUUMRK.IUSROTad-SSQbGIQ.UXM@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734377044; a=rsa-sha256; cv=none; b=uyFwPUuzwJmpRdGK9WPb48cKTe9z4EXOZwW5MrU51OAp+Uh56b+y0bDhbdADZh5WT3FLYU Qy0gtAve4LP5k/Ec1GM68AalSFFxRmP3f1XdblNmm8XO3mQtVlQXo1EZXqyqZfjg8xOdvk JnjywHZcMZiqo6eOofYQ7C7FLODfJRI= Received: by mail-pf1-f201.google.com with SMTP id d2e1a72fcca58-725e6fe074eso6613831b3a.0 for ; Mon, 16 Dec 2024 11:24:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1734377065; x=1734981865; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=NdPtaL44NpTjP9eS1K5KVvptksJqAFYIrPxgR/8blQY=; b=zPjeKIge8nqGATBWYUP7XOiwmCpAp0aEnBzXQqzPvkizOFxcVVY0/AuNt02YfQy628 qDyJy+mnGaAi4zB4QyTIdq9bXzpL85buJfNwYr8RVw6BaJRC4iZyeBF3W+0Bf4yBaxvC a4XB8aMURO5XUMu997AzmCyYc7X9xUicPqUTe+RsT5vBcdOr9O8Nb+o55jsettKF+FoU 5eZxUS64OSrbPtL3+fLoE7AkhX/o1hCJE//hoG6tdUdo6vW9FIRmqlPmYATFU6qSqCDN CLwhclP5viZRDoXZWwkCBowmVNbIj5eTxPLN14f4D+Jg9ry+VQ8E4T8HYVpu6e3YUzNU eSvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734377065; x=1734981865; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NdPtaL44NpTjP9eS1K5KVvptksJqAFYIrPxgR/8blQY=; b=AOZ1eFTnDILYx/5ZT3Zd0h4YwAJvBU1b+aSgEkEKpu1Sgdgl7J0pKPgAwiY9kB2fJW P8lRx2tHETAztf+9wNev+/CiFeOcrLEIX+5OAKyAl7SqO5gQBFZm/4FtrLOeYKSWpEEi MquuRZpEtc5AfJ53r94bltLi4HDmwKDCu/yqebXpVmTPpUDH2qqxm2VjlQFxIs3Ynbq+ qFUER2tjYS7pKOH1cd0kBpBrZKsQwYbdWlOYhBfpSU4H4XvA02wPNdyckQ4sHPjFPMX0 FYn/6b05I/8LAy4Zp3VfLLvmNw7M0qz8ViJKcoY5zn61vchP3lHhWZ8TxFGJb9CV4Zds xiMw== X-Forwarded-Encrypted: i=1; AJvYcCU/P1knJ7Q7ejFY0a9+DX3am0QKVOKOnIC6z1iAwSp0AZTdQ4rYMEE2Jny75oGPLILzABp1JRayyQ==@kvack.org X-Gm-Message-State: AOJu0YzY0h9fkYUuNL4gTwve3xxgGHSbOqn6XK1rIaoJlKBP2/ftwAm7 ILT0+i1EALqztxvOjKYLmSMpY3vBlHn4Eebrjsjvu3vePSLnqskmtWHCFcO4I/tObHPIBW+XMEs 5aA== X-Google-Smtp-Source: AGHT+IGebkhkuPAK7qZirmEWQkyXcqUHZOWu2H7W9h1+nHtKbr+4oTQtJb1gp01e4aHBI2bniuiuItfkaYg= X-Received: from pfbbs2.prod.google.com ([2002:a05:6a00:4442:b0:725:e2fd:dcf9]) (user=surenb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a00:4305:b0:725:973f:9d53 with SMTP id d2e1a72fcca58-7290c19ec16mr16656092b3a.15.1734377065528; Mon, 16 Dec 2024 11:24:25 -0800 (PST) Date: Mon, 16 Dec 2024 11:24:04 -0800 In-Reply-To: <20241216192419.2970941-1-surenb@google.com> Mime-Version: 1.0 References: <20241216192419.2970941-1-surenb@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20241216192419.2970941-2-surenb@google.com> Subject: [PATCH v6 01/16] mm: introduce vma_start_read_locked{_nested} helpers From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: peterz@infradead.org, willy@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mjguzik@gmail.com, oliver.sang@intel.com, mgorman@techsingularity.net, david@redhat.com, peterx@redhat.com, oleg@redhat.com, dave@stgolabs.net, paulmck@kernel.org, brauner@kernel.org, dhowells@redhat.com, hdanton@sina.com, hughd@google.com, lokeshgidra@google.com, minchan@google.com, jannh@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, klarasmodin@gmail.com, corbet@lwn.net, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com, surenb@google.com X-Rspamd-Queue-Id: 2C90580015 X-Rspamd-Server: rspam12 X-Stat-Signature: r8mxii5ogg9m6k5s64dn7obrjw17hqf8 X-Rspam-User: X-HE-Tag: 1734377009-921286 X-HE-Meta: U2FsdGVkX1/KZqYPxhZkQRN8A5rtjWc6vcVFrI/mgXJi8krYd7nZY9BCa24KS61ncW+o2za3IlHJCpYig/38A9K6921nNqCY2wjr+drlUY5kkzF7sufzxmqBqo4Fg69O2lK3baVBG9dZOsi23/hmn7k2V6YuBizFTbkc7sqd15Pndo8Ts9wQCNMTGD3uGCffcij5Wp+sW6Auh1oKxJXAQQ9jGSb++AuPTlANlkCmyG3iQCBwZT6VzkdOIMaAWdY/kqDFHWHf22r2nNvtx4YLOvuKrzVWQI+mpxFeO5fxEi07j/IGtUSmrrTwLxgQbI2QgmbtTDbAhfIaSFNp9dsGJ72xHz55Gi79FJVx5g6KHJuSsaUFC8DB8PKwQy4vqLA0ufes/GjOksLSgSNALx1beLtSfcYC8c5AnU3p7GjqGsXszEOJpe83zWF67fwlTcrr2kozGiDCnzK7D4R4wcHl7OUjoxA1+EiQ7mH62x8v6WxyT+mr8IBfN6ocG9yA3MZ3ZuK7a4SkYl+SeIyLHZuH9BZat6wLpoujRByiQ0m5uwaMPI695H4acBRjYA/3SteIqy1ixGDr/R1Ea1qmyO/Sd7p3SMs5gMGszGvjo3tQSaraVKJrdoCNcqPxCJahcI1d8Qtf9JvZ1FynLx+Qb4ERJfZGWBV6/NtY0EKJQrYmTSw29hhjEROVhMO/WlVfJpG5MNeZ0Hlg0kdjTOo7UfLiwyS7tRp90KCACJar/gToRsmBltgyL4LQICOaNM2nT8SFqKSojbvQfD1nHTOynt0+2YZUyN18iQ1iNClsJbJ2ExrgMHSYV3akbI738pulhoNlQd4vKl98t9xHn0JejeAO6fWlu+q1PdEz1UnumsQohw7LorHn3N1ifCkLJBdQSoEo4RJS6BCjXh1A3j0RmZLNLtbDPR8lpx6nlZ7tlmKwoCl8TupilFclOPOMLlMEUwlHClngSg1q/+rmI7Krqng lsY1v6WY pVgBl2MKgR3SLorfmsY01owgMDn0co8xc89RHbNoQtP9HYSbPLQGjptZTdhigaauk+onNsDSvoX5PYmEA6riu2TfoQowcnc+P+fBCG/mn087BuMwwRMki6dQUwy3jML2oGPZAHdspezYpGudIkbNEpSGkknX6MhgGa1qee7vcm+xyBByrlGAZLIW8X9RragNWJOqw3noIQ8J19t2Z58Urc0gKJ9hUyFiP5pPo8OsMYhcS3Bw3T0MPfvUq4eRMVDiuiUc9pgBOar6AScXX2dnHynG/MgrjK2IXNCMQtRO1sYICprS4VIyaotlAx0pNGscNdptDqpqPHtmnG7ED3eLsUBHOuBzJVdGY4QnqR2XQMOrT/TzGiWovBZSHvTVqDVSgmABSv7Ap1cuPyz1ylRplhJqpJb6ZzI1EyoeAys2+wC1ynHL9YfYvXhMarSocOEgAc1UR3t50dwqbZ80fm+MZcEHJJaezX8R1TN0dewJ6PFB0o16+4l47CJu8LITjW30fpQbPBquboN3ztzMq+yP9lC9VKnFq6PbYIhTIKW9htRYksUD3vTxUVtL9OV0plhNQOkrhZAiCSaIY4gBQ0RiCHy1F/fS9mMLhR07ybzWJBzXjOGM7ZYWpT6r/StPYT0cyLotLzLTS6+vG+cu7wHD19izCjucZ2yxwTDFzRseDAeUIvpg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.001380, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduce helper functions which can be used to read-lock a VMA when holding mmap_lock for read. Replace direct accesses to vma->vm_lock with these new helpers. Signed-off-by: Suren Baghdasaryan Reviewed-by: Lorenzo Stoakes Reviewed-by: Davidlohr Bueso Reviewed-by: Shakeel Butt Reviewed-by: Vlastimil Babka --- include/linux/mm.h | 24 ++++++++++++++++++++++++ mm/userfaultfd.c | 22 +++++----------------- 2 files changed, 29 insertions(+), 17 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 1352147a2648..3815a43ba504 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -735,6 +735,30 @@ static inline bool vma_start_read(struct vm_area_struct *vma) return true; } +/* + * Use only while holding mmap read lock which guarantees that locking will not + * fail (nobody can concurrently write-lock the vma). vma_start_read() should + * not be used in such cases because it might fail due to mm_lock_seq overflow. + * This functionality is used to obtain vma read lock and drop the mmap read lock. + */ +static inline void vma_start_read_locked_nested(struct vm_area_struct *vma, int subclass) +{ + mmap_assert_locked(vma->vm_mm); + down_read_nested(&vma->vm_lock->lock, subclass); +} + +/* + * Use only while holding mmap read lock which guarantees that locking will not + * fail (nobody can concurrently write-lock the vma). vma_start_read() should + * not be used in such cases because it might fail due to mm_lock_seq overflow. + * This functionality is used to obtain vma read lock and drop the mmap read lock. + */ +static inline void vma_start_read_locked(struct vm_area_struct *vma) +{ + mmap_assert_locked(vma->vm_mm); + down_read(&vma->vm_lock->lock); +} + static inline void vma_end_read(struct vm_area_struct *vma) { rcu_read_lock(); /* keeps vma alive till the end of up_read */ diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 8e16dc290ddf..bc9a66ec6a6e 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -84,16 +84,8 @@ static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm, mmap_read_lock(mm); vma = find_vma_and_prepare_anon(mm, address); - if (!IS_ERR(vma)) { - /* - * We cannot use vma_start_read() as it may fail due to - * false locked (see comment in vma_start_read()). We - * can avoid that by directly locking vm_lock under - * mmap_lock, which guarantees that nobody can lock the - * vma for write (vma_start_write()) under us. - */ - down_read(&vma->vm_lock->lock); - } + if (!IS_ERR(vma)) + vma_start_read_locked(vma); mmap_read_unlock(mm); return vma; @@ -1491,14 +1483,10 @@ static int uffd_move_lock(struct mm_struct *mm, mmap_read_lock(mm); err = find_vmas_mm_locked(mm, dst_start, src_start, dst_vmap, src_vmap); if (!err) { - /* - * See comment in uffd_lock_vma() as to why not using - * vma_start_read() here. - */ - down_read(&(*dst_vmap)->vm_lock->lock); + vma_start_read_locked(*dst_vmap); if (*dst_vmap != *src_vmap) - down_read_nested(&(*src_vmap)->vm_lock->lock, - SINGLE_DEPTH_NESTING); + vma_start_read_locked_nested(*src_vmap, + SINGLE_DEPTH_NESTING); } mmap_read_unlock(mm); return err;