From patchwork Tue Jan 23 23:10:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13528265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34B78C47DDB for ; Tue, 23 Jan 2024 23:10:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 874216B007B; Tue, 23 Jan 2024 18:10:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7FD816B0082; Tue, 23 Jan 2024 18:10:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 677126B0083; Tue, 23 Jan 2024 18:10:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 510336B007B for ; Tue, 23 Jan 2024 18:10:20 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2741E40712 for ; Tue, 23 Jan 2024 23:10:20 +0000 (UTC) X-FDA: 81712121400.20.F1F1258 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf10.hostedemail.com (Postfix) with ESMTP id 4A8E9C0016 for ; Tue, 23 Jan 2024 23:10:18 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="gx6r9V/y"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf10.hostedemail.com: domain of 3WUewZQYKCNwQSPCL9EMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3WUewZQYKCNwQSPCL9EMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706051418; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=oXyU4pkvCRYAyA2TUvsUC8dh5sAPzBBH4qagn6tOpF0=; b=SSyOG72KhppDXzO86sF8ULsoXxBlRDvFomz9v3HU/HOatDYp5fzPCZx2a+QmcGiLxLq9Dc b1W0sdKmGHKQDOdAFlC6iEdCrDZj+51heLr2YkERUwZSahFBT/7KjCz8W/rLRj308zOpvc rM6LHxaOEhTiVZeQ1Wbv65ewErR5wXE= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="gx6r9V/y"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf10.hostedemail.com: domain of 3WUewZQYKCNwQSPCL9EMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3WUewZQYKCNwQSPCL9EMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706051418; a=rsa-sha256; cv=none; b=S9d0OtC7s+iVBulz4NV0KUetvy3eOhfAmobGRDsNgfOGrFIut8UN7jyp4QCEp37/bSGA7d vhx9CmzgZFdg+OxjPKtqCI4B3F4uByR5881bCM8hg+BN/59N9bfvkF7AraszHbN/z6XRfr 9/SrH5c/ATheIgvJ29zjQM7jIVEePA0= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dbf1c3816a3so5766715276.1 for ; Tue, 23 Jan 2024 15:10:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706051417; x=1706656217; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=oXyU4pkvCRYAyA2TUvsUC8dh5sAPzBBH4qagn6tOpF0=; b=gx6r9V/y2qHq4kUdVnxHvZS/Xl+8rba0Eb5MHbQ4TFXhg3QxUHsAhSxVf9K7uOnAsu LGzfEJ/G3B0whQZJFQFhCqEm/ocuAnqtMKfUYkfQYwITrZWPVlEV+dWJ5to+5MlS09XR rM2fD/D6fL/5VSMIlMokr5Br5/ODf8Z4vV5Zq8ftCl+gS1iZ5DE03wRSAVE6DtMRBQtY ldQUwxi1X9T8DomYsGwWuRG6e+AxYLxAK7xeY45dop5prxsDZQuwuADBqvKHK+NtrTdc ZBq+BKOfhFI4tDkA8Gk0MFtRUyGhGrpa9a1UlSGIqV1C0SOdAuMVPKoYeag1k5KGVwY4 m+dA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706051417; x=1706656217; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=oXyU4pkvCRYAyA2TUvsUC8dh5sAPzBBH4qagn6tOpF0=; b=giZPb2My0CwsDaflxdRJ9yOadvUOL3LX3gR65JFGCtwAs3j9dldHw+dtCz1qvelesb VUA1Ra7QHrm/3Gaugy5pwGMA5YE28bNu0lH/o/FJ1KHNnKW2k22zD8RyeZN1yPXIWVGc SpjK+C87g3AezNpzBW0h/pd4zgQc4v3X15+fiCBCysB/Oq4vxRaasL925m57qIH08Lbh ppn165KyYaU5PwfR8blgn4hHwkMwg7sODHzSJoKJ2AbRStHl+e3Hwi1+z1F3TgIT7wrJ GpbYgNGosd2sY0JqmV2Ik+l8AX06+d8XMJQSeODr8363FCKe8rYKDUbj6LvYJ/yMcBUG IkXQ== X-Gm-Message-State: AOJu0Yzl4dsCg8Om6inlLHv4OvGIjcI/n8gy5a0JT18TgPd3XnAguhck Hyh6M3I2ITFIbGrDHApRniSS92JkM40zfEWlq5mLZ096Ew3/bVLm4pzpdbKmqSlEft2HcspsSQM Ong== X-Google-Smtp-Source: AGHT+IEgxI304pcV7zFy8N/84TxapaHhJhR9pXZLWhwJ6Iatmm8yKLxxB6dJA4KRTnvXodtXbWXNbbtsUkg= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:8fc3:c34f:d407:388]) (user=surenb job=sendgmr) by 2002:a05:6902:218b:b0:dc2:5456:d9ac with SMTP id dl11-20020a056902218b00b00dc25456d9acmr377765ybb.5.1706051417317; Tue, 23 Jan 2024 15:10:17 -0800 (PST) Date: Tue, 23 Jan 2024 15:10:12 -0800 Mime-Version: 1.0 X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240123231014.3801041-1-surenb@google.com> Subject: [PATCH v2 1/3] mm: make vm_area_struct anon_name field RCU-safe From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, dchinner@redhat.com, casey@schaufler-ca.com, ben.wolsieffer@hefring.com, paulmck@kernel.org, david@redhat.com, avagin@google.com, usama.anjum@collabora.com, peterx@redhat.com, hughd@google.com, ryan.roberts@arm.com, wangkefeng.wang@huawei.com, Liam.Howlett@Oracle.com, yuzhao@google.com, axelrasmussen@google.com, lstoakes@gmail.com, talumbau@google.com, willy@infradead.org, vbabka@suse.cz, mgorman@techsingularity.net, jhubbard@nvidia.com, vishal.moola@gmail.com, mathieu.desnoyers@efficios.com, dhowells@redhat.com, jgg@ziepe.ca, sidhartha.kumar@oracle.com, andriy.shevchenko@linux.intel.com, yangxingui@huawei.com, keescook@chromium.org, sj@kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, kernel-team@android.com, surenb@google.com X-Rspamd-Queue-Id: 4A8E9C0016 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: smz6zsmm8n8ojhiew6trtiu4y5ryz88d X-HE-Tag: 1706051418-747907 X-HE-Meta: U2FsdGVkX1/WI7UP9Lgk2yDJTyzt88/v1qb7LcKIMa6E8b5Fsn68/UYMjLjwaOgUsg8pXG63X4Js98ax0I04B1+uy3cYBiCN9b77NktUlwhIP61eFrLw5c+xUASncm9cmQuFolbyeGq5qnh20P+aYtKvv52PAw5cSgpJSTVmtGgQhmBcIQ+hEsCCnejDAfnpT8wGhWPTfXJTEICSoBcpl9MX1iL41QjZTcDB1zVUK9PFXPzd0R/7aeNjhY1scDJHiJYkMvZaxuNwOjtY5Q+bpzQ4vVJN1InQ3c4rnbbghYvPPDGJX4gRee5rTgfuJUe5Ya984pMxuxG46JYLzwJRPHHT7GADJW3eSOaKYBAMebsaOOOy8rTnpVg2hNhs/5BMrfxCCCNI3ETnCHdqNNmrqjraOfAbZHPn1ewMy/EHwvWkSJIGGJ0VpO9pgY24mLILH9Vpie8xGl4fCbofNoKSatNh952w0t2KcWVQWI02RY7r1sc1IUZTC7cuTYRm1tNZb/UUjk21q0AY3bElAnI5ZnTwqiIfSmiE8JalTE+iN0x8lSkd1TwGuGEpbJ9qJz2L5+EaNcgfyS9ZUwzF2g5O6lsfPxNs2fGzzP41LPpeZZ9iXWRF/zLPIGu8fotzrYXU2ZOYnrIcV+9XLnhUZZSbbpVVmhIzt4uiPm3cVGN8zxTkQHqpAH8Ll+h9qwRxwj6mC40BAISk+eaKuHqT9JH3fl6Ktm7HL4CDIUveiSQToxxZV47JzvADvcSQfIa6AePgwNg63exuoSdY8d/v0PONSLfVNn/QgL6jzkNyd8xjqUhane0E0086ignYEa/AjSFCNizY27ozxWrj+tq/oj8aqshBoG8oDo4CW+pHJb29+0qRbCcU+y2axvhsxpRkzWg2a9vAnbT9JXBv3lt1salzrIX6Ml3jOpiUPVr2bkVmop2pEya3bTzyK7BaA1YljTDRb7/xUtJYDk9opEvR9e4 0DkGrU72 AIubq+cBAd9lvt5ZZfxdCV6iizuDREaopSKENxmXu8a45yTD4HUC1RSsojVDyW+oivupfPQfYwsJr/cg8j/sTgy8GIggMrl1/7y1go9dLFy4JC9J1M/WdW5MgBTLgIbFWQKKu+VQ5xxsA10X1koP3Nzk6s1Xw9iMkUPXCTkGarsTtqteDAwkLjvZGtXytersqHQ5fi9NLD+4V8d4xnZT5wkrpFkKmCVGNDfuVc9OLZLwbZqi47QAs3OGmPzHvO3kmu/iAKsdvMrSsEWJdiyw0+3dn6ZtfF/1LtVA+ushgvm2oa0HT8v4cC8Gjpx+1ucyMkyis/Q7C0wVQ3iP6M6deBogLiP90psOFScMQ5n90Mq2tQKFe+uvUGVQexBQFlXdDW0IGLdrw8MOykOlxwEHxTYF54YQdK14YM9kBzX5y2lMwG9JJS/AqVMo3mT4ZNo274GuwZkhzbXXH7I1ZkxPNCZcYgunG5FSWLh5876yu3XQcUL7ebgeia7ShyT+bzMGfl5xCSErfDIVaQ57oeWk7lGP5QTvuLhXP+LDFts42MG0cYLvkXBrrM571Mkph6usziv9ztu0xsGq9N3LZ2543mobhY/MoFLwDnkP5HMra5YHeP1emEnR17wKEU10Ak1DgAPMypid4x9K/A3sDJFUdCQuIhwYJLNEpjcuH4+KHsekSIIYdO3CfbzhtnoSKXEhQdOxhv3/qhCDUfc2XuH6zpqUxLg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For lockless /proc/pid/maps reading we have to ensure all the fields used when generating the output are RCU-safe. The only pointer fields in vm_area_struct which are used to generate that file's output are vm_file and anon_name. vm_file is RCU-safe but anon_name is not. Make anon_name RCU-safe as well. Signed-off-by: Suren Baghdasaryan --- include/linux/mm_inline.h | 10 +++++++++- include/linux/mm_types.h | 3 ++- mm/madvise.c | 30 ++++++++++++++++++++++++++---- 3 files changed, 37 insertions(+), 6 deletions(-) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index f4fe593c1400..bbdb0ca857f1 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -389,7 +389,7 @@ static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma, struct anon_vma_name *anon_name = anon_vma_name(orig_vma); if (anon_name) - new_vma->anon_name = anon_vma_name_reuse(anon_name); + rcu_assign_pointer(new_vma->anon_name, anon_vma_name_reuse(anon_name)); } static inline void free_anon_vma_name(struct vm_area_struct *vma) @@ -411,6 +411,8 @@ static inline bool anon_vma_name_eq(struct anon_vma_name *anon_name1, !strcmp(anon_name1->name, anon_name2->name); } +struct anon_vma_name *anon_vma_name_get_rcu(struct vm_area_struct *vma); + #else /* CONFIG_ANON_VMA_NAME */ static inline void anon_vma_name_get(struct anon_vma_name *anon_name) {} static inline void anon_vma_name_put(struct anon_vma_name *anon_name) {} @@ -424,6 +426,12 @@ static inline bool anon_vma_name_eq(struct anon_vma_name *anon_name1, return true; } +static inline +struct anon_vma_name *anon_vma_name_get_rcu(struct vm_area_struct *vma) +{ + return NULL; +} + #endif /* CONFIG_ANON_VMA_NAME */ static inline void init_tlb_flush_pending(struct mm_struct *mm) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 8b611e13153e..bbe1223cd992 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -545,6 +545,7 @@ struct vm_userfaultfd_ctx {}; struct anon_vma_name { struct kref kref; + struct rcu_head rcu; /* The name needs to be at the end because it is dynamically sized. */ char name[]; }; @@ -699,7 +700,7 @@ struct vm_area_struct { * terminated string containing the name given to the vma, or NULL if * unnamed. Serialized by mmap_lock. Use anon_vma_name to access. */ - struct anon_vma_name *anon_name; + struct anon_vma_name __rcu *anon_name; #endif #ifdef CONFIG_SWAP atomic_long_t swap_readahead_info; diff --git a/mm/madvise.c b/mm/madvise.c index 912155a94ed5..0f222d464254 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -88,14 +88,15 @@ void anon_vma_name_free(struct kref *kref) { struct anon_vma_name *anon_name = container_of(kref, struct anon_vma_name, kref); - kfree(anon_name); + kfree_rcu(anon_name, rcu); } struct anon_vma_name *anon_vma_name(struct vm_area_struct *vma) { mmap_assert_locked(vma->vm_mm); - return vma->anon_name; + return rcu_dereference_protected(vma->anon_name, + rwsem_is_locked(&vma->vm_mm->mmap_lock)); } /* mmap_lock should be write-locked */ @@ -105,7 +106,7 @@ static int replace_anon_vma_name(struct vm_area_struct *vma, struct anon_vma_name *orig_name = anon_vma_name(vma); if (!anon_name) { - vma->anon_name = NULL; + rcu_assign_pointer(vma->anon_name, NULL); anon_vma_name_put(orig_name); return 0; } @@ -113,11 +114,32 @@ static int replace_anon_vma_name(struct vm_area_struct *vma, if (anon_vma_name_eq(orig_name, anon_name)) return 0; - vma->anon_name = anon_vma_name_reuse(anon_name); + rcu_assign_pointer(vma->anon_name, anon_vma_name_reuse(anon_name)); anon_vma_name_put(orig_name); return 0; } + +/* + * Returned anon_vma_name is stable due to elevated refcount but not guaranteed + * to be assigned to the original VMA after the call. + */ +struct anon_vma_name *anon_vma_name_get_rcu(struct vm_area_struct *vma) +{ + struct anon_vma_name __rcu *anon_name; + + WARN_ON_ONCE(!rcu_read_lock_held()); + + anon_name = rcu_dereference(vma->anon_name); + if (!anon_name) + return NULL; + + if (unlikely(!kref_get_unless_zero(&anon_name->kref))) + return NULL; + + return anon_name; +} + #else /* CONFIG_ANON_VMA_NAME */ static int replace_anon_vma_name(struct vm_area_struct *vma, struct anon_vma_name *anon_name)