From patchwork Tue Jan 23 23:10:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13528266 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3B57C47258 for ; Tue, 23 Jan 2024 23:10:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 678C66B0082; Tue, 23 Jan 2024 18:10:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 601486B0083; Tue, 23 Jan 2024 18:10:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4557A6B0085; Tue, 23 Jan 2024 18:10:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 311256B0082 for ; Tue, 23 Jan 2024 18:10:22 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id C901F1C14BA for ; Tue, 23 Jan 2024 23:10:21 +0000 (UTC) X-FDA: 81712121442.05.EE767E5 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf17.hostedemail.com (Postfix) with ESMTP id 4018A40020 for ; Tue, 23 Jan 2024 23:10:20 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=CisczW0i; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of 3W0ewZQYKCN4SURENBGOOGLE.COMLINUX-MMKVACK.ORG@flex--surenb.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3W0ewZQYKCN4SURENBGOOGLE.COMLINUX-MMKVACK.ORG@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706051420; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OLp6U6EsMSw583KQsA7U2jc3UsvPAv4xgtxefIxudO0=; b=h2ZXeH/q0v/ShjN+m/aCkV1AkHA7pUnaXY25HPiCfirOwlnDfC+Lh8Uz0bRsA1PVRCgP3Z DXCs/rCKVAgFSuk7q9A7su5I+5xhT/XcvO9Q0f7BHfWGvjWFGAEPTQFjMt7FeCKKFPBMTR pusRDOS8dX+cijqon/B3qJuhVYpVHFI= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=CisczW0i; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of 3W0ewZQYKCN4SURENBGOOGLE.COMLINUX-MMKVACK.ORG@flex--surenb.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3W0ewZQYKCN4SURENBGOOGLE.COMLINUX-MMKVACK.ORG@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706051420; a=rsa-sha256; cv=none; b=ReXfxV1Jtn/dvFmU+OrufG331VBX8jHS/aYMyD4DQJMnJ/cUJVUZwCD7aZJFExhupTC75q fQq6H4o/ATLjSzxdmNJo7stdOnDXhSGITDPxv7IXYHrsx21KYrnNGkNewdZ2h/bfy/8lJb 9wTelZArGwCh6nlermy1BK5rmuzpAi8= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-5f874219ff9so66801007b3.0 for ; Tue, 23 Jan 2024 15:10:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706051419; x=1706656219; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=OLp6U6EsMSw583KQsA7U2jc3UsvPAv4xgtxefIxudO0=; b=CisczW0ieVEh6tFi6AO+XedbqtsE3bHg+0jmGTz3lshqBKlnIZwkNcw5qUqxbabpr4 lNDM31Ya0d41U3uEXlTLJBqkF2fLbHA3Nfhr7ApbhuKj2/WcVb5pIBIGmxqs+yrGT5vP wqq07ZHd3KPBliguxl8L27JxkJUilrnjqCI0OpSeJSAUU6lvRZIs2Ff4zZ4wUVTrHiBe v5EOZvydgJRJXcnb314e7ssiGJEpCgiKCsSCUJES2XqrH9FvhydRvVab0Accp30XuD7l BfGPcVnc+KQ60PuA5zLQ1Bvo6xEI9Z0iFZ6Cv5T8430ShhOCihFKj+ZzeJd2Rnkwv1Ll he5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706051419; x=1706656219; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=OLp6U6EsMSw583KQsA7U2jc3UsvPAv4xgtxefIxudO0=; b=l8MaQ7VPCcpq1Wd1IBrjg365N4fLINBg8MRv5a5VI/jZSfFqlHDfog3NlG1QBeyNuh 1mgXA4p6986JFH+lsRQLnzO4r23uRwB7G8ZQEShyI2cWCUrVyFc0oOqbDaq1EjX3KOW5 fJlOsyLLxhqn30XnSt5j/JTQb2ss2bJ2JzHhosOHxemCFh2yr6VhD4Ep2kyyNLio+zIy u9Vf+aMubiBL36Pp+R4WD8mNBoAsJQzZznGembBC4vVBvP/cl7e2mB0r5YnuEfBWlrri keKGs88UdS4ZxxelVN4+Vst+EBoXQO1FTTWld0w17HVGydMQCaR9QZ8nLpBnJkN9wic5 eNbQ== X-Gm-Message-State: AOJu0YwZrrAichh80WYFmwklXwr+R/P0Oftwu6COk8i7pC1sRU5ded6s 6EyUAwQy6/gKiPS4IMxo11YzJDYMyd8J7lk99S1HLnrLbSP6w13ZFT7A5fqlAHl0S649IJKa01k qlg== X-Google-Smtp-Source: AGHT+IFX5qIebM/ZQiKUEXGTW3oxR6pBTWd2HR+W3VQ1MxVAngHzessR/YRxQzpsjGe8I1sOL4WmImG7jew= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:8fc3:c34f:d407:388]) (user=surenb job=sendgmr) by 2002:a81:9896:0:b0:5fc:7f94:da64 with SMTP id p144-20020a819896000000b005fc7f94da64mr134477ywg.5.1706051419411; Tue, 23 Jan 2024 15:10:19 -0800 (PST) Date: Tue, 23 Jan 2024 15:10:13 -0800 In-Reply-To: <20240123231014.3801041-1-surenb@google.com> Mime-Version: 1.0 References: <20240123231014.3801041-1-surenb@google.com> X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240123231014.3801041-2-surenb@google.com> Subject: [PATCH v2 2/3] mm: add mm_struct sequence number to detect write locks From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, dchinner@redhat.com, casey@schaufler-ca.com, ben.wolsieffer@hefring.com, paulmck@kernel.org, david@redhat.com, avagin@google.com, usama.anjum@collabora.com, peterx@redhat.com, hughd@google.com, ryan.roberts@arm.com, wangkefeng.wang@huawei.com, Liam.Howlett@Oracle.com, yuzhao@google.com, axelrasmussen@google.com, lstoakes@gmail.com, talumbau@google.com, willy@infradead.org, vbabka@suse.cz, mgorman@techsingularity.net, jhubbard@nvidia.com, vishal.moola@gmail.com, mathieu.desnoyers@efficios.com, dhowells@redhat.com, jgg@ziepe.ca, sidhartha.kumar@oracle.com, andriy.shevchenko@linux.intel.com, yangxingui@huawei.com, keescook@chromium.org, sj@kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, kernel-team@android.com, surenb@google.com X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 4018A40020 X-Stat-Signature: k6nip5srpnjg9h9crax6tnfkm5stepp9 X-Rspam-User: X-HE-Tag: 1706051420-269716 X-HE-Meta: U2FsdGVkX1/UIGVhUZimS0FI9PimWe1SG2KSAnLLF0Jq0HKVAxvyY/DXdD2mO37Y1F7Y1z04DIwk9NgqHnExf1ObDjBGLcvoKssfWsk6OMl2C0OHYAbNFbbByAur87DNAuGG9noVAaju2RS5ONQFqECU4OA3LUxOdzpWea0NyclNAtyloYOLJaHrL47gDPXXLXXGlSkaPldYowD7nlYzveM3UkBf9A3XYhc3H79kKbDmx/h2o3ryqBe3cVSMF6zpOQQYPjBqVB2+WToq2b4elCm3oLKv04p1e3QjsLLrX0YxAaoFMocSih2lxgng2W2tFu96fK0UBg8v6EDKU05YXu49N5jbqLj30GLG1yq37NRA2Yk5eSgk0FPP/BxbXrhynGYnqVTS6SBnD9vP41ngYCYoRRjoB2OuY6M07PNZvo4miIldYAbrd8pfR551wzzjszxqUE0CHxijoeTg9RbO79Csa+7QAsay9mvuh1JGvsRlGA2q2w3dZwroUyHIK44aA+X+zR5PHu2yrFgvAParboZdSmCAzhWb0/NtCKDAGQwVn91fuSDNEnFv97PQaQrJ5HBT7ejNnsNa9FngRcMXCpdAZL0/0gl3r+ryBw8IZnQ0NT66wRXfYf9z6KM60kNfPAuh1eEDEiQfVXyQSL8I0yn0QLVzKoZY7Q+oZyR1kdIlVpayRG8YN7zYMhtM5Jonrgik5G7T+UUomxY8zh6EQPa7/GYwMmf8JFTsqQ67xHCmkTOzSUa4jocn+P3MA4ZCWIifKCm1XUfs5BnlydTnDjGEAI2gKqwIuYH2+Hem0x8XS7NZqsCF44jfVc/HMg3fiX8GQlBBqQVKOMCYjcoMjHIo9Xn2fHFkpWB69UYoVzK2GtBc4D/xqmv9PkVDC81H8UYDVjzJJH6Rbp054DA6aepD7ZBj3MOSiTA8pV/XrErjbQlgvhhGpEKCZ2bKf86MsDwdNSxESVXUtbfjpSM FcsUNses efKGjrhAMIcd3BdmA11jJ5AlzLvrt8SgmRjAFlW9wVhr/tCx3zlm5SSZMalasHDpDLWSdB4WvOhFreqXMutvHXBqkLjuS2rIruTb1apLgTCMGkk08httYzCgdgcmWZ3geYDSEp37PNwxp6GM8XlgEGZDaLdnUyxxOw04AT9/j7yPS+cCzdiDIRMizHNlyB5NGcFihk1deOEjJsPCzrvqIF57jzUJfF3yNH82+mCxiVguqlp4pmDNvlMekm5pX8hPMMSUZYmZcShjqA/EiKtlGZcISPorMX+bBfzaHTvnbvIBLEbT61oFLrIBYqye66vd0Zpml6ROuxCVe8XLw0e20SSIEnE9DC0vaT5eJRgWQcESbKfkFsNtC5jyQPUGsMHeVzz7wRHg4qZnkrVkcW5OJhxMv9VmsYK/7kQYjXwZA1YC7DgmEOSW0rch+KRsNPrPolA9BPwnQAt7g1Yn1iymJUPDV0g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Provide a way for lockless mm_struct users to detect whether mm might have been changed since some specific point in time. The API provided allows the user to record a counter when it starts using the mm and later use that counter to check if anyone write-locked mmap_lock since the counter was recorded. Recording the counter value should be done while holding mmap_lock at least for reading to prevent the counter from concurrent changes. Every time mmap_lock is write-locked mm_struct updates its mm_wr_seq counter so that checks against counters recorded before that would fail, indicating a possibility of mm being modified. Signed-off-by: Suren Baghdasaryan --- include/linux/mm_types.h | 2 ++ include/linux/mmap_lock.h | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index bbe1223cd992..e749f7f09314 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -846,6 +846,8 @@ struct mm_struct { */ int mm_lock_seq; #endif + /* Counter incremented each time mm gets write-locked */ + unsigned long mm_wr_seq; unsigned long hiwater_rss; /* High-watermark of RSS usage */ diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h index 8d38dcb6d044..0197079cb6fe 100644 --- a/include/linux/mmap_lock.h +++ b/include/linux/mmap_lock.h @@ -106,6 +106,8 @@ static inline void mmap_write_lock(struct mm_struct *mm) { __mmap_lock_trace_start_locking(mm, true); down_write(&mm->mmap_lock); + /* Pairs with ACQUIRE semantics in mmap_write_seq_read */ + smp_store_release(&mm->mm_wr_seq, mm->mm_wr_seq + 1); __mmap_lock_trace_acquire_returned(mm, true, true); } @@ -113,6 +115,8 @@ static inline void mmap_write_lock_nested(struct mm_struct *mm, int subclass) { __mmap_lock_trace_start_locking(mm, true); down_write_nested(&mm->mmap_lock, subclass); + /* Pairs with ACQUIRE semantics in mmap_write_seq_read */ + smp_store_release(&mm->mm_wr_seq, mm->mm_wr_seq + 1); __mmap_lock_trace_acquire_returned(mm, true, true); } @@ -122,6 +126,10 @@ static inline int mmap_write_lock_killable(struct mm_struct *mm) __mmap_lock_trace_start_locking(mm, true); ret = down_write_killable(&mm->mmap_lock); + if (!ret) { + /* Pairs with ACQUIRE semantics in mmap_write_seq_read */ + smp_store_release(&mm->mm_wr_seq, mm->mm_wr_seq + 1); + } __mmap_lock_trace_acquire_returned(mm, true, ret == 0); return ret; } @@ -140,6 +148,20 @@ static inline void mmap_write_downgrade(struct mm_struct *mm) downgrade_write(&mm->mmap_lock); } +static inline unsigned long mmap_write_seq_read(struct mm_struct *mm) +{ + /* Pairs with RELEASE semantics in mmap_write_lock */ + return smp_load_acquire(&mm->mm_wr_seq); +} + +static inline void mmap_write_seq_record(struct mm_struct *mm, + unsigned long *mm_wr_seq) +{ + mmap_assert_locked(mm); + /* Nobody can concurrently modify since we hold the mmap_lock */ + *mm_wr_seq = mm->mm_wr_seq; +} + static inline void mmap_read_lock(struct mm_struct *mm) { __mmap_lock_trace_start_locking(mm, false);