From patchwork Mon Jan 15 18:38:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13520080 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E305C47258 for ; Mon, 15 Jan 2024 18:38:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C62DB6B0085; Mon, 15 Jan 2024 13:38:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BED0D6B0088; Mon, 15 Jan 2024 13:38:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A17246B0087; Mon, 15 Jan 2024 13:38:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 93AA36B0083 for ; Mon, 15 Jan 2024 13:38:45 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 73CDE120576 for ; Mon, 15 Jan 2024 18:38:45 +0000 (UTC) X-FDA: 81682406610.13.6726C4F Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf22.hostedemail.com (Postfix) with ESMTP id BC2F6C0019 for ; Mon, 15 Jan 2024 18:38:43 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=mtNryADL; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 3snulZQYKCHMjliVeSXffXcV.TfdcZelo-ddbmRTb.fiX@flex--surenb.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3snulZQYKCHMjliVeSXffXcV.TfdcZelo-ddbmRTb.fiX@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705343923; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oLlMsVPwk9YFRfJ1xkZa3gURgHKOqqqaOLfJjGTa1Bw=; b=xlfkALtW3StL2oXVA6kKU3zPlDZAQggetQyntSk9MWjlNdSJGI/KHgB0PUYGk7cxRf5mJE uzQt8SKGLZ0KmPCop/uiAjFPbLBUah7ZLowOMM5I/FC3dRTnbKxmHPAklQPqW8/qzuolIF kf4bhmh3dRJBqXzOd790AOk0kS1r7c8= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=mtNryADL; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 3snulZQYKCHMjliVeSXffXcV.TfdcZelo-ddbmRTb.fiX@flex--surenb.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3snulZQYKCHMjliVeSXffXcV.TfdcZelo-ddbmRTb.fiX@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705343923; a=rsa-sha256; cv=none; b=bu7t/PREGKGEIlaQaD/tF97wlyA5bphaiPl+Htnl4a/LLxZGmSn5NdVP6omIhqjxkhZbJL IQmkfl33MBhbXcDx1uk2wefBl0FfyKlVMQJgodQ/2/jZOq+v1OB6s1rWNvkorcVGWC2RAF hMtQsSmxaPPvyNiHeF/GeKNxd1y/ix4= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dbf3d102221so7034309276.1 for ; Mon, 15 Jan 2024 10:38:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1705343923; x=1705948723; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=oLlMsVPwk9YFRfJ1xkZa3gURgHKOqqqaOLfJjGTa1Bw=; b=mtNryADLi00Qn+nXLEZoWF/bag2mhQ4IgV61iC6qGTAW+0CQlF00GYxIMO9Bj9gBsS EYdmTYsDfD3wT6+M/uuKI+bvPUUL8OoE47WEcFvcLJNaW128uUSHOcv7WHyAOWlp2i6l DsdHEjeCdmQJhak3GTInMhhB6V5rHBh4ZnxJvgfNOckgPzdMLdqijjDE1mCAhbLTsb9w xFgIm2H/Rcn+eQDAoGhSas8g/B5XpAhnDH1NWesqfI9MGbAS9T6rEy/tJ+j4TXeBaykc 1ZkUOw2nD+VV43lV3IBUi5ChazqC3bqhFDZJYhozQzZjdUzweG/jlZJPj0CURv77qWbs f2zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705343923; x=1705948723; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oLlMsVPwk9YFRfJ1xkZa3gURgHKOqqqaOLfJjGTa1Bw=; b=phJ0GYhTZKPaaN7JW4BQFPtWyk+ycPdQE7CaB9A4yKQ4SXETFzQyiDOspWndBKL06q iQibdQ8gDkbe1nRg4Kj6fk8i1McAthqDEj3759qrP3A5ZvtBDofboLqPhH5B9vDK+TAQ ZV9xLNXWGKaZYCkMMF9wEEW6rlbQxKsvvBlnAZ4U7SA2r3RF2gPCjgk4/2dEjtgK8Vej XWT6IiaBAA3SHVAS/OLcGhV8D4Vc1t9NzFeuegt9OE8Jm8itZiKDRkf8ZGKfkPaPMKA2 Eau1HMqnNYWY2jjtR3hsy83ylLMEdQXDbiExuIgASewA5XbNp/fcKWncZImIw0fH9cs0 j52Q== X-Gm-Message-State: AOJu0YxUH6ddQQGIKb1CJmmafceyAvJ6kTnQzR59uWoNhgO7nmexBeqG VGiRpH3AxbV9MdgkzkHoIBm/2n40Zk0ZTN+YiQ== X-Google-Smtp-Source: AGHT+IETYM+vP+GcA1bz5ffGAn7xpFLG8rQcpZyAKLVas4y2HqU4DVH4yHlOoP/EqERN0tEXwr857q4NqX0= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:3af2:e48e:2785:270]) (user=surenb job=sendgmr) by 2002:a25:a292:0:b0:dc1:f71f:a0ad with SMTP id c18-20020a25a292000000b00dc1f71fa0admr1244231ybi.13.1705343922947; Mon, 15 Jan 2024 10:38:42 -0800 (PST) Date: Mon, 15 Jan 2024 10:38:34 -0800 In-Reply-To: <20240115183837.205694-1-surenb@google.com> Mime-Version: 1.0 References: <20240115183837.205694-1-surenb@google.com> X-Mailer: git-send-email 2.43.0.381.gb435a96ce8-goog Message-ID: <20240115183837.205694-2-surenb@google.com> Subject: [RFC 1/3] mm: make vm_area_struct anon_name field RCU-safe From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, dchinner@redhat.com, casey@schaufler-ca.com, ben.wolsieffer@hefring.com, paulmck@kernel.org, david@redhat.com, avagin@google.com, usama.anjum@collabora.com, peterx@redhat.com, hughd@google.com, ryan.roberts@arm.com, wangkefeng.wang@huawei.com, Liam.Howlett@Oracle.com, yuzhao@google.com, axelrasmussen@google.com, lstoakes@gmail.com, talumbau@google.com, willy@infradead.org, vbabka@suse.cz, mgorman@techsingularity.net, jhubbard@nvidia.com, vishal.moola@gmail.com, mathieu.desnoyers@efficios.com, dhowells@redhat.com, jgg@ziepe.ca, sidhartha.kumar@oracle.com, andriy.shevchenko@linux.intel.com, yangxingui@huawei.com, keescook@chromium.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, kernel-team@android.com, surenb@google.com X-Rspamd-Queue-Id: BC2F6C0019 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 15qjytr3qak3izpp7r59yo6zjb5fqk53 X-HE-Tag: 1705343923-9121 X-HE-Meta: U2FsdGVkX1+F501zwzszTya3j3XseglO1h35mbKbqjW7jOV2rp/vcLmR8NX+c3zSvStzUeDqELpbZgnxMlnZ2CZryk9TZb6w9Pe1he2/kAeg1l8bZ1yb4j7z4D8sKRqpUv3JX+RefxIJQL4sM4Y1/G86WTxlslkUbZNjEER3VMj3b92T0iPfAi3fswgT9xrT/T8jAOpjdEj1SNzYRFPdixpCvwyRfwVWVXg3OkEitk9XDCGlfQjrXD1L05bk7S12appG8I9rbI8Sof/pph8o0bJoQr9efR4QXAcs+1LKpCuP+F/+wDNLB+AyOR+Y3ki3bAlFMx8OOqDZBrP0R7NvikXnLpMm0rcvCrVIK1ks4iAKZlTKG5nIiucswSHqOCK06/oAPH0Ex+gCS93apmO5FnHGFF2vbPWNy/CP5kaeRVgDXWlkbo6vmrRCcgN36Qj46FmZWkRthV0Zjlf2Tj4ttIcbk2p3J30qCtOnz3gFwCDk8X7dLl98RXOT/zLW62JygLCGy/KNHKJiUYjLRncgUxc4Rx6VYLprT2DNJt3YzLyBrEYDu0sRDX0WMWH76K/1Std2At0gzhlatgiN+yxdpWSf/a7xdADEvlE/Cu/FmsgxtN7HcUyegW003W/xDsKhOlKrrVeOc5YmBjH3GAC15hsle/JrPo3RgCAuaYoECl12OZapcXfkvn61MYpdJvK5aEBd+Nbs5PKd477mGsWNDVNKPsRJr+MPfWiYu6cr/81Z4j2zv4hzq1B4fhuMNoFn9MZ8Ot1VR8wwCPqwX3RBIMWC4bT4JFtKx6jnjMUGLxSuF8prr39TB8Z7J/NYt7lq5op1L17h9b18F9QYw014QvXr1CJTtvlj5ylGfZ9eYYsnKAfBQ/NRIPbkaFI20VBPUc5gDfsvvQfCaqykReudUqTX/rMe/TgZ/A3GJTzIpkh5aecuzfs13o7TXjojkRj2U7cY6cZ7hMOOztJfn6P MIWOh9vP HQzr3tcTVnVyv1wLgySgc7vOF3CxRTVVFI4kKKXEvAff5lPM5k6NLP50MUGNHUqMSII+eTv1IRuzNY37PtWSWeecpNqm+c9j+jIToe+5WeiB8PBOVOk3q99nc6LMeIt5PCYOPfDxahxrWnJ+H2Tz4wiyJ53MsDQ62/tzevT9hGQCe4YAvelQO3S6j5yBtjza/u7C8+1itYLAc/e0GODlyQ7jEBLh6i/AuAO9OgB01KLpcSOIgWk2YDZNotFEqB64gMlIHDdleDdwDx2thz0ZUia9JkTl6boHty3F5VxUeY8iIdQuwduwlNpm0cNuFLEo9nNSskS/sBbrbwurTCMtTfNMrVVNHrHHKkMT2jnQCCG3UHRpwmzJYXD0wvjaTUULQvl3z2/lMbKGLtopPZfz5izYJ00QlLGsGrB0AlF4ZwYM2FY5nJklJR+q5fYfbaaF0ygQlfHqV1kUbFAhU6M0URP7OMgdYbNLvtkUy1gg+8w7e1SScrMDsnprNqrEp65bCdiH3Y70/99BQ1ISEn8cnMT9vRnce4nrdkWfYIanpARntemeoGoYzuixvA8Z3sJXjJ8+NjqDUF383ZM9+y5V/E1pfHVA99GF2tCDZSfI+LvDuGnUweVq6+efwNQzY4UgwIceEuRnwqh/nm8ikqj0gz7ytx+GLNAtP6r63TNre5geFfqtLRUJn9DjYXgfIb7C3dfU1eSXebvprQv/Fze18kz0T0A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For lockless /proc/pid/maps reading we have to ensure all the fields used when generating the output are RCU-safe. The only pointer fields in vm_area_struct which are used to generate that file's output are vm_file and anon_name. vm_file is RCU-safe but anon_name is not. Make anon_name RCU-safe as well. Signed-off-by: Suren Baghdasaryan --- include/linux/mm_inline.h | 10 +++++++++- include/linux/mm_types.h | 3 ++- mm/madvise.c | 30 ++++++++++++++++++++++++++---- 3 files changed, 37 insertions(+), 6 deletions(-) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index f4fe593c1400..bbdb0ca857f1 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -389,7 +389,7 @@ static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma, struct anon_vma_name *anon_name = anon_vma_name(orig_vma); if (anon_name) - new_vma->anon_name = anon_vma_name_reuse(anon_name); + rcu_assign_pointer(new_vma->anon_name, anon_vma_name_reuse(anon_name)); } static inline void free_anon_vma_name(struct vm_area_struct *vma) @@ -411,6 +411,8 @@ static inline bool anon_vma_name_eq(struct anon_vma_name *anon_name1, !strcmp(anon_name1->name, anon_name2->name); } +struct anon_vma_name *anon_vma_name_get_rcu(struct vm_area_struct *vma); + #else /* CONFIG_ANON_VMA_NAME */ static inline void anon_vma_name_get(struct anon_vma_name *anon_name) {} static inline void anon_vma_name_put(struct anon_vma_name *anon_name) {} @@ -424,6 +426,12 @@ static inline bool anon_vma_name_eq(struct anon_vma_name *anon_name1, return true; } +static inline +struct anon_vma_name *anon_vma_name_get_rcu(struct vm_area_struct *vma) +{ + return NULL; +} + #endif /* CONFIG_ANON_VMA_NAME */ static inline void init_tlb_flush_pending(struct mm_struct *mm) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index b2d3a88a34d1..1f0a30c00795 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -545,6 +545,7 @@ struct vm_userfaultfd_ctx {}; struct anon_vma_name { struct kref kref; + struct rcu_head rcu; /* The name needs to be at the end because it is dynamically sized. */ char name[]; }; @@ -699,7 +700,7 @@ struct vm_area_struct { * terminated string containing the name given to the vma, or NULL if * unnamed. Serialized by mmap_lock. Use anon_vma_name to access. */ - struct anon_vma_name *anon_name; + struct anon_vma_name __rcu *anon_name; #endif #ifdef CONFIG_SWAP atomic_long_t swap_readahead_info; diff --git a/mm/madvise.c b/mm/madvise.c index 912155a94ed5..0f222d464254 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -88,14 +88,15 @@ void anon_vma_name_free(struct kref *kref) { struct anon_vma_name *anon_name = container_of(kref, struct anon_vma_name, kref); - kfree(anon_name); + kfree_rcu(anon_name, rcu); } struct anon_vma_name *anon_vma_name(struct vm_area_struct *vma) { mmap_assert_locked(vma->vm_mm); - return vma->anon_name; + return rcu_dereference_protected(vma->anon_name, + rwsem_is_locked(&vma->vm_mm->mmap_lock)); } /* mmap_lock should be write-locked */ @@ -105,7 +106,7 @@ static int replace_anon_vma_name(struct vm_area_struct *vma, struct anon_vma_name *orig_name = anon_vma_name(vma); if (!anon_name) { - vma->anon_name = NULL; + rcu_assign_pointer(vma->anon_name, NULL); anon_vma_name_put(orig_name); return 0; } @@ -113,11 +114,32 @@ static int replace_anon_vma_name(struct vm_area_struct *vma, if (anon_vma_name_eq(orig_name, anon_name)) return 0; - vma->anon_name = anon_vma_name_reuse(anon_name); + rcu_assign_pointer(vma->anon_name, anon_vma_name_reuse(anon_name)); anon_vma_name_put(orig_name); return 0; } + +/* + * Returned anon_vma_name is stable due to elevated refcount but not guaranteed + * to be assigned to the original VMA after the call. + */ +struct anon_vma_name *anon_vma_name_get_rcu(struct vm_area_struct *vma) +{ + struct anon_vma_name __rcu *anon_name; + + WARN_ON_ONCE(!rcu_read_lock_held()); + + anon_name = rcu_dereference(vma->anon_name); + if (!anon_name) + return NULL; + + if (unlikely(!kref_get_unless_zero(&anon_name->kref))) + return NULL; + + return anon_name; +} + #else /* CONFIG_ANON_VMA_NAME */ static int replace_anon_vma_name(struct vm_area_struct *vma, struct anon_vma_name *anon_name)