From patchwork Mon Jan 15 18:38:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13520080 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E305C47258 for ; Mon, 15 Jan 2024 18:38:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C62DB6B0085; Mon, 15 Jan 2024 13:38:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BED0D6B0088; Mon, 15 Jan 2024 13:38:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A17246B0087; Mon, 15 Jan 2024 13:38:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 93AA36B0083 for ; Mon, 15 Jan 2024 13:38:45 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 73CDE120576 for ; Mon, 15 Jan 2024 18:38:45 +0000 (UTC) X-FDA: 81682406610.13.6726C4F Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf22.hostedemail.com (Postfix) with ESMTP id BC2F6C0019 for ; Mon, 15 Jan 2024 18:38:43 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=mtNryADL; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 3snulZQYKCHMjliVeSXffXcV.TfdcZelo-ddbmRTb.fiX@flex--surenb.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3snulZQYKCHMjliVeSXffXcV.TfdcZelo-ddbmRTb.fiX@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705343923; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oLlMsVPwk9YFRfJ1xkZa3gURgHKOqqqaOLfJjGTa1Bw=; b=xlfkALtW3StL2oXVA6kKU3zPlDZAQggetQyntSk9MWjlNdSJGI/KHgB0PUYGk7cxRf5mJE uzQt8SKGLZ0KmPCop/uiAjFPbLBUah7ZLowOMM5I/FC3dRTnbKxmHPAklQPqW8/qzuolIF kf4bhmh3dRJBqXzOd790AOk0kS1r7c8= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=mtNryADL; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 3snulZQYKCHMjliVeSXffXcV.TfdcZelo-ddbmRTb.fiX@flex--surenb.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3snulZQYKCHMjliVeSXffXcV.TfdcZelo-ddbmRTb.fiX@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705343923; a=rsa-sha256; cv=none; b=bu7t/PREGKGEIlaQaD/tF97wlyA5bphaiPl+Htnl4a/LLxZGmSn5NdVP6omIhqjxkhZbJL IQmkfl33MBhbXcDx1uk2wefBl0FfyKlVMQJgodQ/2/jZOq+v1OB6s1rWNvkorcVGWC2RAF hMtQsSmxaPPvyNiHeF/GeKNxd1y/ix4= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dbf3d102221so7034309276.1 for ; Mon, 15 Jan 2024 10:38:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1705343923; x=1705948723; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=oLlMsVPwk9YFRfJ1xkZa3gURgHKOqqqaOLfJjGTa1Bw=; b=mtNryADLi00Qn+nXLEZoWF/bag2mhQ4IgV61iC6qGTAW+0CQlF00GYxIMO9Bj9gBsS EYdmTYsDfD3wT6+M/uuKI+bvPUUL8OoE47WEcFvcLJNaW128uUSHOcv7WHyAOWlp2i6l DsdHEjeCdmQJhak3GTInMhhB6V5rHBh4ZnxJvgfNOckgPzdMLdqijjDE1mCAhbLTsb9w xFgIm2H/Rcn+eQDAoGhSas8g/B5XpAhnDH1NWesqfI9MGbAS9T6rEy/tJ+j4TXeBaykc 1ZkUOw2nD+VV43lV3IBUi5ChazqC3bqhFDZJYhozQzZjdUzweG/jlZJPj0CURv77qWbs f2zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705343923; x=1705948723; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=oLlMsVPwk9YFRfJ1xkZa3gURgHKOqqqaOLfJjGTa1Bw=; b=phJ0GYhTZKPaaN7JW4BQFPtWyk+ycPdQE7CaB9A4yKQ4SXETFzQyiDOspWndBKL06q iQibdQ8gDkbe1nRg4Kj6fk8i1McAthqDEj3759qrP3A5ZvtBDofboLqPhH5B9vDK+TAQ ZV9xLNXWGKaZYCkMMF9wEEW6rlbQxKsvvBlnAZ4U7SA2r3RF2gPCjgk4/2dEjtgK8Vej XWT6IiaBAA3SHVAS/OLcGhV8D4Vc1t9NzFeuegt9OE8Jm8itZiKDRkf8ZGKfkPaPMKA2 Eau1HMqnNYWY2jjtR3hsy83ylLMEdQXDbiExuIgASewA5XbNp/fcKWncZImIw0fH9cs0 j52Q== X-Gm-Message-State: AOJu0YxUH6ddQQGIKb1CJmmafceyAvJ6kTnQzR59uWoNhgO7nmexBeqG VGiRpH3AxbV9MdgkzkHoIBm/2n40Zk0ZTN+YiQ== X-Google-Smtp-Source: AGHT+IETYM+vP+GcA1bz5ffGAn7xpFLG8rQcpZyAKLVas4y2HqU4DVH4yHlOoP/EqERN0tEXwr857q4NqX0= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:3af2:e48e:2785:270]) (user=surenb job=sendgmr) by 2002:a25:a292:0:b0:dc1:f71f:a0ad with SMTP id c18-20020a25a292000000b00dc1f71fa0admr1244231ybi.13.1705343922947; Mon, 15 Jan 2024 10:38:42 -0800 (PST) Date: Mon, 15 Jan 2024 10:38:34 -0800 In-Reply-To: <20240115183837.205694-1-surenb@google.com> Mime-Version: 1.0 References: <20240115183837.205694-1-surenb@google.com> X-Mailer: git-send-email 2.43.0.381.gb435a96ce8-goog Message-ID: <20240115183837.205694-2-surenb@google.com> Subject: [RFC 1/3] mm: make vm_area_struct anon_name field RCU-safe From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, dchinner@redhat.com, casey@schaufler-ca.com, ben.wolsieffer@hefring.com, paulmck@kernel.org, david@redhat.com, avagin@google.com, usama.anjum@collabora.com, peterx@redhat.com, hughd@google.com, ryan.roberts@arm.com, wangkefeng.wang@huawei.com, Liam.Howlett@Oracle.com, yuzhao@google.com, axelrasmussen@google.com, lstoakes@gmail.com, talumbau@google.com, willy@infradead.org, vbabka@suse.cz, mgorman@techsingularity.net, jhubbard@nvidia.com, vishal.moola@gmail.com, mathieu.desnoyers@efficios.com, dhowells@redhat.com, jgg@ziepe.ca, sidhartha.kumar@oracle.com, andriy.shevchenko@linux.intel.com, yangxingui@huawei.com, keescook@chromium.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, kernel-team@android.com, surenb@google.com X-Rspamd-Queue-Id: BC2F6C0019 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 15qjytr3qak3izpp7r59yo6zjb5fqk53 X-HE-Tag: 1705343923-9121 X-HE-Meta: U2FsdGVkX1+F501zwzszTya3j3XseglO1h35mbKbqjW7jOV2rp/vcLmR8NX+c3zSvStzUeDqELpbZgnxMlnZ2CZryk9TZb6w9Pe1he2/kAeg1l8bZ1yb4j7z4D8sKRqpUv3JX+RefxIJQL4sM4Y1/G86WTxlslkUbZNjEER3VMj3b92T0iPfAi3fswgT9xrT/T8jAOpjdEj1SNzYRFPdixpCvwyRfwVWVXg3OkEitk9XDCGlfQjrXD1L05bk7S12appG8I9rbI8Sof/pph8o0bJoQr9efR4QXAcs+1LKpCuP+F/+wDNLB+AyOR+Y3ki3bAlFMx8OOqDZBrP0R7NvikXnLpMm0rcvCrVIK1ks4iAKZlTKG5nIiucswSHqOCK06/oAPH0Ex+gCS93apmO5FnHGFF2vbPWNy/CP5kaeRVgDXWlkbo6vmrRCcgN36Qj46FmZWkRthV0Zjlf2Tj4ttIcbk2p3J30qCtOnz3gFwCDk8X7dLl98RXOT/zLW62JygLCGy/KNHKJiUYjLRncgUxc4Rx6VYLprT2DNJt3YzLyBrEYDu0sRDX0WMWH76K/1Std2At0gzhlatgiN+yxdpWSf/a7xdADEvlE/Cu/FmsgxtN7HcUyegW003W/xDsKhOlKrrVeOc5YmBjH3GAC15hsle/JrPo3RgCAuaYoECl12OZapcXfkvn61MYpdJvK5aEBd+Nbs5PKd477mGsWNDVNKPsRJr+MPfWiYu6cr/81Z4j2zv4hzq1B4fhuMNoFn9MZ8Ot1VR8wwCPqwX3RBIMWC4bT4JFtKx6jnjMUGLxSuF8prr39TB8Z7J/NYt7lq5op1L17h9b18F9QYw014QvXr1CJTtvlj5ylGfZ9eYYsnKAfBQ/NRIPbkaFI20VBPUc5gDfsvvQfCaqykReudUqTX/rMe/TgZ/A3GJTzIpkh5aecuzfs13o7TXjojkRj2U7cY6cZ7hMOOztJfn6P MIWOh9vP HQzr3tcTVnVyv1wLgySgc7vOF3CxRTVVFI4kKKXEvAff5lPM5k6NLP50MUGNHUqMSII+eTv1IRuzNY37PtWSWeecpNqm+c9j+jIToe+5WeiB8PBOVOk3q99nc6LMeIt5PCYOPfDxahxrWnJ+H2Tz4wiyJ53MsDQ62/tzevT9hGQCe4YAvelQO3S6j5yBtjza/u7C8+1itYLAc/e0GODlyQ7jEBLh6i/AuAO9OgB01KLpcSOIgWk2YDZNotFEqB64gMlIHDdleDdwDx2thz0ZUia9JkTl6boHty3F5VxUeY8iIdQuwduwlNpm0cNuFLEo9nNSskS/sBbrbwurTCMtTfNMrVVNHrHHKkMT2jnQCCG3UHRpwmzJYXD0wvjaTUULQvl3z2/lMbKGLtopPZfz5izYJ00QlLGsGrB0AlF4ZwYM2FY5nJklJR+q5fYfbaaF0ygQlfHqV1kUbFAhU6M0URP7OMgdYbNLvtkUy1gg+8w7e1SScrMDsnprNqrEp65bCdiH3Y70/99BQ1ISEn8cnMT9vRnce4nrdkWfYIanpARntemeoGoYzuixvA8Z3sJXjJ8+NjqDUF383ZM9+y5V/E1pfHVA99GF2tCDZSfI+LvDuGnUweVq6+efwNQzY4UgwIceEuRnwqh/nm8ikqj0gz7ytx+GLNAtP6r63TNre5geFfqtLRUJn9DjYXgfIb7C3dfU1eSXebvprQv/Fze18kz0T0A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For lockless /proc/pid/maps reading we have to ensure all the fields used when generating the output are RCU-safe. The only pointer fields in vm_area_struct which are used to generate that file's output are vm_file and anon_name. vm_file is RCU-safe but anon_name is not. Make anon_name RCU-safe as well. Signed-off-by: Suren Baghdasaryan --- include/linux/mm_inline.h | 10 +++++++++- include/linux/mm_types.h | 3 ++- mm/madvise.c | 30 ++++++++++++++++++++++++++---- 3 files changed, 37 insertions(+), 6 deletions(-) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index f4fe593c1400..bbdb0ca857f1 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -389,7 +389,7 @@ static inline void dup_anon_vma_name(struct vm_area_struct *orig_vma, struct anon_vma_name *anon_name = anon_vma_name(orig_vma); if (anon_name) - new_vma->anon_name = anon_vma_name_reuse(anon_name); + rcu_assign_pointer(new_vma->anon_name, anon_vma_name_reuse(anon_name)); } static inline void free_anon_vma_name(struct vm_area_struct *vma) @@ -411,6 +411,8 @@ static inline bool anon_vma_name_eq(struct anon_vma_name *anon_name1, !strcmp(anon_name1->name, anon_name2->name); } +struct anon_vma_name *anon_vma_name_get_rcu(struct vm_area_struct *vma); + #else /* CONFIG_ANON_VMA_NAME */ static inline void anon_vma_name_get(struct anon_vma_name *anon_name) {} static inline void anon_vma_name_put(struct anon_vma_name *anon_name) {} @@ -424,6 +426,12 @@ static inline bool anon_vma_name_eq(struct anon_vma_name *anon_name1, return true; } +static inline +struct anon_vma_name *anon_vma_name_get_rcu(struct vm_area_struct *vma) +{ + return NULL; +} + #endif /* CONFIG_ANON_VMA_NAME */ static inline void init_tlb_flush_pending(struct mm_struct *mm) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index b2d3a88a34d1..1f0a30c00795 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -545,6 +545,7 @@ struct vm_userfaultfd_ctx {}; struct anon_vma_name { struct kref kref; + struct rcu_head rcu; /* The name needs to be at the end because it is dynamically sized. */ char name[]; }; @@ -699,7 +700,7 @@ struct vm_area_struct { * terminated string containing the name given to the vma, or NULL if * unnamed. Serialized by mmap_lock. Use anon_vma_name to access. */ - struct anon_vma_name *anon_name; + struct anon_vma_name __rcu *anon_name; #endif #ifdef CONFIG_SWAP atomic_long_t swap_readahead_info; diff --git a/mm/madvise.c b/mm/madvise.c index 912155a94ed5..0f222d464254 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -88,14 +88,15 @@ void anon_vma_name_free(struct kref *kref) { struct anon_vma_name *anon_name = container_of(kref, struct anon_vma_name, kref); - kfree(anon_name); + kfree_rcu(anon_name, rcu); } struct anon_vma_name *anon_vma_name(struct vm_area_struct *vma) { mmap_assert_locked(vma->vm_mm); - return vma->anon_name; + return rcu_dereference_protected(vma->anon_name, + rwsem_is_locked(&vma->vm_mm->mmap_lock)); } /* mmap_lock should be write-locked */ @@ -105,7 +106,7 @@ static int replace_anon_vma_name(struct vm_area_struct *vma, struct anon_vma_name *orig_name = anon_vma_name(vma); if (!anon_name) { - vma->anon_name = NULL; + rcu_assign_pointer(vma->anon_name, NULL); anon_vma_name_put(orig_name); return 0; } @@ -113,11 +114,32 @@ static int replace_anon_vma_name(struct vm_area_struct *vma, if (anon_vma_name_eq(orig_name, anon_name)) return 0; - vma->anon_name = anon_vma_name_reuse(anon_name); + rcu_assign_pointer(vma->anon_name, anon_vma_name_reuse(anon_name)); anon_vma_name_put(orig_name); return 0; } + +/* + * Returned anon_vma_name is stable due to elevated refcount but not guaranteed + * to be assigned to the original VMA after the call. + */ +struct anon_vma_name *anon_vma_name_get_rcu(struct vm_area_struct *vma) +{ + struct anon_vma_name __rcu *anon_name; + + WARN_ON_ONCE(!rcu_read_lock_held()); + + anon_name = rcu_dereference(vma->anon_name); + if (!anon_name) + return NULL; + + if (unlikely(!kref_get_unless_zero(&anon_name->kref))) + return NULL; + + return anon_name; +} + #else /* CONFIG_ANON_VMA_NAME */ static int replace_anon_vma_name(struct vm_area_struct *vma, struct anon_vma_name *anon_name) From patchwork Mon Jan 15 18:38:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13520081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A08E3C4707C for ; Mon, 15 Jan 2024 18:38:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 251686B0087; Mon, 15 Jan 2024 13:38:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1DA416B0088; Mon, 15 Jan 2024 13:38:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 02CD36B0089; Mon, 15 Jan 2024 13:38:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E34536B0087 for ; Mon, 15 Jan 2024 13:38:47 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BCAE41C11AA for ; Mon, 15 Jan 2024 18:38:47 +0000 (UTC) X-FDA: 81682406694.23.AF499AE Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf27.hostedemail.com (Postfix) with ESMTP id F125C4001D for ; Mon, 15 Jan 2024 18:38:45 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Kzn107eS; spf=pass (imf27.hostedemail.com: domain of 3tXulZQYKCHYmolYhVaiiafY.Wigfchor-ggepUWe.ila@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3tXulZQYKCHYmolYhVaiiafY.Wigfchor-ggepUWe.ila@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705343926; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ch6T6Lsnf4fqmqV9A6xFsxD4eG+jCeNRaJgu+LlgQig=; b=hn/w+AmxGUwBkuBx+0pKnJ4bvWK4W1ndMFUe5OxyQ4eaV2Om3S2R3qek5qVzrTHgLbYWir U9F3CRQ4YDMgdmqu8ioXp+a856J//44tlQ32FnHWQn40D8f/ERyL8k43Qcen1HFSVDM9mq vkTSs9Sches3oXqQ4bxL3DwcIRc5ktc= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Kzn107eS; spf=pass (imf27.hostedemail.com: domain of 3tXulZQYKCHYmolYhVaiiafY.Wigfchor-ggepUWe.ila@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3tXulZQYKCHYmolYhVaiiafY.Wigfchor-ggepUWe.ila@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705343926; a=rsa-sha256; cv=none; b=ZQlmzJ9mVWZVB+5Sk9DmwUEF8j/oXpUUAkbmdwLWwyY/+Kd79uXvlrZUwLup7C18/D2Oz9 YdDw1qAeTq+nYeiRKI8YIK5W0gssN2xfhmc9BF2QBMMOTNGcPRJuUnjurYbf4E8n6M2GEx cPrXnK6TpinBv4icKhlkd9LxZZyBwkU= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dbe9ef2422cso10926485276.0 for ; Mon, 15 Jan 2024 10:38:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1705343925; x=1705948725; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ch6T6Lsnf4fqmqV9A6xFsxD4eG+jCeNRaJgu+LlgQig=; b=Kzn107eSkT/Mtb4VptUlPYU/0Y184JHhufSAwykSoF5FWoX5kay9vbLMRWSrM28PMb weCQ/V3vzM2xAdNFvMpNYcOnKdX4kPI5P1uVyEqD7I0cavu/k+mEOfq//qQJVYrvezhY 2+b4FupuHyYtCzGJvMjL6f+D5Jza/ikyEBVYxFFVAU9RRsu2F81lxgxwcLrkfBFUtDS6 0aDBm4+m5W8UfrTyd+NBWNS2NAtPIupPBMO5rCfGByItyD6JSLRGe7s8VKiLUkd72NMP /9r3/0fm1BhznID/JuK+El+fbg3hPGcNuLgiSahx6qNg9iX6dZPtYNSmXbbt08Z9TiKX hTiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705343925; x=1705948725; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ch6T6Lsnf4fqmqV9A6xFsxD4eG+jCeNRaJgu+LlgQig=; b=qmnZPPl7+V6j/IAqySCGBw1tVao3Jhc6qz8yPuYnBC0/LwGphLe5DYds7gZWl3Jdy1 Ms+W7lASk/k8a2+5QmZuIeUXn7FyILp04HJF8Lrvgb7u6bP+A+JfaJAlh49DeqWxUZDS /Gx6KmfKMUbT+iLE/p4/aTDK1DhOE04XO0H0X/aFewMVX//3vdkRWP4KG0+HmXc1Ds2G SQF/AjgSXUOQ37snjqzHN2mnQsZhhg6yKMlaB+pLIz9StNuueG4DnxCAetf5jqyj86kD kJHB5KUl+KUtuqlTQEJWRYrNNY+YIBigmuIVOkBxxuKuiH0ll8ayRqkQPccQR056vvmr 5v7g== X-Gm-Message-State: AOJu0Yx2uFIouB7AbUanwlYD4ReACEJQ8+1efYRgAkm82LOdHdD6jD9T fxAZb0mzFBR1kv7jbBH+lXpe4PTNCVvShOWYUg== X-Google-Smtp-Source: AGHT+IGN2VvUUxNx3HDpn+LUP1maSJ793VbScOs1XRHuqJsGhFfM1RNtInfsiW1wKVri/L8s95oaaZe3g/g= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:3af2:e48e:2785:270]) (user=surenb job=sendgmr) by 2002:a05:6902:1364:b0:dbd:7149:a389 with SMTP id bt4-20020a056902136400b00dbd7149a389mr269675ybb.11.1705343925137; Mon, 15 Jan 2024 10:38:45 -0800 (PST) Date: Mon, 15 Jan 2024 10:38:35 -0800 In-Reply-To: <20240115183837.205694-1-surenb@google.com> Mime-Version: 1.0 References: <20240115183837.205694-1-surenb@google.com> X-Mailer: git-send-email 2.43.0.381.gb435a96ce8-goog Message-ID: <20240115183837.205694-3-surenb@google.com> Subject: [RFC 2/3] seq_file: add validate() operation to seq_operations From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, dchinner@redhat.com, casey@schaufler-ca.com, ben.wolsieffer@hefring.com, paulmck@kernel.org, david@redhat.com, avagin@google.com, usama.anjum@collabora.com, peterx@redhat.com, hughd@google.com, ryan.roberts@arm.com, wangkefeng.wang@huawei.com, Liam.Howlett@Oracle.com, yuzhao@google.com, axelrasmussen@google.com, lstoakes@gmail.com, talumbau@google.com, willy@infradead.org, vbabka@suse.cz, mgorman@techsingularity.net, jhubbard@nvidia.com, vishal.moola@gmail.com, mathieu.desnoyers@efficios.com, dhowells@redhat.com, jgg@ziepe.ca, sidhartha.kumar@oracle.com, andriy.shevchenko@linux.intel.com, yangxingui@huawei.com, keescook@chromium.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, kernel-team@android.com, surenb@google.com X-Rspamd-Queue-Id: F125C4001D X-Rspam-User: X-Stat-Signature: ntfpbpqkjiigii7z9prgi7xkrkeetxry X-Rspamd-Server: rspam01 X-HE-Tag: 1705343925-843216 X-HE-Meta: U2FsdGVkX19KhnAl0c+aaKMDdPLD+sjCAZnlcUrC13nW6613/XyUUvBdHywMFzr22BIHbx42XhBIj5kO62eO4TWOhdePxTVxzVHKzQ5bzK2pk1248zB0bAzUyZ6PcKnZwDuZnkruICPk1iefxwprvswo4xICBXPpHevRcsEptcjB4oeLQLDK5DX7J8CmkBcA0ApIQGTFunuDecJL5R45bAEerWW2/GJPLJJ+R8FbW0WvqJEC7N5/wVQxpF5tSLucJvFrpLNK5IdLmuoph9j5+3WQGemApNDJraatJwe8eygEPunOL7QAFVT2V/V3nyglY2MgusRx01QR4WtjKyujsRFe+O+UPRxchLKAu6NqhWdCJtBgyigCZnnFzJxYOYMnHfMM4u79eKOZ7KXGEwgl5/6hwlSr9CxIZXnWgeGyRNUdQ5O2k0IdhG71w5cgBWy81XuiC6FvVDV6HAqX/N9z44j0/2mEUeCHXy9v24TRwRECYm/Cfh2q+VQbVjHg8jrJ7uxEjU2nA1E3vqd8oq9UJEpvedwEkuwM9PdHnZe5FFMWfsCvNbLWsPLn/XaTptpGM2s7aXv+mqEmIy+AaEnVk+areMhDo+Elat5//f+50/sv9lNpFUH7edQE6+p56PoVblREw1Qw/dlSQWQAc0QSmi8e3qoxNX0VDAa1jPs1qrnu6t6aN2ud0OE3OmUlbUwvn8O5SnXNG6rmh9OjIDV/tyohTupAqzMk731w2VUM5NVjdrw9SjFrHomXhA9HVGJMZdpGrNLV4tNnologA2Qt6F+1k6Z8IwdegCQJK2I7J5dA39xks9OVj8FL2HMqD6p0kdQBtbdS+SJNMJKvrmywtbrE8Pod3Ql1DNZkKtFU8fFmIW2/2wMRMRnJ3MlLDZksLcGqWH8eaf2ZV2F0cAVxfi+k6fPz0nQ7EToBKNUqvevSDNO0NtY0N0Hb51xDBqXjoHGCO6ctG0s6WiXjD1Z FzFGUK50 lBHy+JhVJG4VTKje79eJGalp4dU87JU8ZHSrzRC0g5u7pbf7x1JPgAx+Siw2cMRyyofFjrkyhSfBvJkUmlW8I5OqXeTSma3Bp3oNYH4nhF6KCRdiJZKyue7xvEMkipfarTSxx/QUsp3Hm/t19Cv72OmQa3MQ1yWIyvO3OYzHmjlPtsRj+ZgaW53cJsaDlVgPQRH4NWb0jvMr2YizHvkbBdi5Fg+7pDvEkYHtXrLXJkW9ekKOT1WI3D3luy6LGvYW7EbihwE8tl/J7mjv+Xs2L+8b1whcv4vdfa4q2bJPGXWOek3egwQfhveWvuy/49ae6LUw2HHG86PFbsKUPltghGLgrgB+GV/y+fSQgYVwYo5pAlh51qFTbyjOshatLHVHauK2HRggUWRBuiNbRTLVbp2ndinkjsDjRPbBDEav9kJbdYbMP5+XnZ1EnG4YMeJZoxCHBfi/LjCMHNmGXAvK3NYPEdfN09pwc3QgT+6X4pwyUM/ivnoxw9WiLYJcvVlvl4UBV8kxO7GBzML+WYJn0ARtGkopn617hEIGgKTSq9OvA/MgEWICHsQxFSlYeup93dfNKbUoiaFdsg7KvZ5X1+DhLm+QXzXy9X9mLHEMFXwhf2WlVmy/WcCaIet9NA4pNNGcA1tjgFcjvctrdMdWG0lohskYRQdUgp0LbNOL//xK2YCJVkeEMSCa3SBYLpZRifWWhcUnemwOYaLh3yIAWQu1iA46yKOp0ljgG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: seq_file outputs data in chunks using seq_file.buf as the intermediate storage before outputting the generated data for the current chunk. It is possible for already buffered data to become stale before it gets reported. In certain situations it is desirable to regenerate that data instead of reporting the stale one. Provide a validate() operation called before outputting the buffered data to allow users to validate buffered data. To indicate valid data, user's validate callback should return 0, to request regeneration of the stale data it should return -EAGAIN, any other error will be considered fatal and read operation will be aborted. Signed-off-by: Suren Baghdasaryan --- fs/seq_file.c | 24 +++++++++++++++++++++++- include/linux/seq_file.h | 1 + 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/fs/seq_file.c b/fs/seq_file.c index f5fdaf3b1572..77833bbe5909 100644 --- a/fs/seq_file.c +++ b/fs/seq_file.c @@ -172,6 +172,8 @@ ssize_t seq_read_iter(struct kiocb *iocb, struct iov_iter *iter) { struct seq_file *m = iocb->ki_filp->private_data; size_t copied = 0; + loff_t orig_index; + size_t orig_count; size_t n; void *p; int err = 0; @@ -220,6 +222,10 @@ ssize_t seq_read_iter(struct kiocb *iocb, struct iov_iter *iter) if (m->count) // hadn't managed to copy everything goto Done; } + + orig_index = m->index; + orig_count = m->count; +Again: // get a non-empty record in the buffer m->from = 0; p = m->op->start(m, &m->index); @@ -278,6 +284,22 @@ ssize_t seq_read_iter(struct kiocb *iocb, struct iov_iter *iter) } } m->op->stop(m, p); + /* Note: we validate even if err<0 to prevent publishing copied data */ + if (m->op->validate) { + int val_err = m->op->validate(m, p); + + if (val_err) { + if (val_err == -EAGAIN) { + m->index = orig_index; + m->count = orig_count; + // data is stale, retry + goto Again; + } + // data is invalid, return the last error + err = val_err; + goto Done; + } + } n = copy_to_iter(m->buf, m->count, iter); copied += n; m->count -= n; @@ -572,7 +594,7 @@ static void single_stop(struct seq_file *p, void *v) int single_open(struct file *file, int (*show)(struct seq_file *, void *), void *data) { - struct seq_operations *op = kmalloc(sizeof(*op), GFP_KERNEL_ACCOUNT); + struct seq_operations *op = kzalloc(sizeof(*op), GFP_KERNEL_ACCOUNT); int res = -ENOMEM; if (op) { diff --git a/include/linux/seq_file.h b/include/linux/seq_file.h index 234bcdb1fba4..d0fefac2990f 100644 --- a/include/linux/seq_file.h +++ b/include/linux/seq_file.h @@ -34,6 +34,7 @@ struct seq_operations { void (*stop) (struct seq_file *m, void *v); void * (*next) (struct seq_file *m, void *v, loff_t *pos); int (*show) (struct seq_file *m, void *v); + int (*validate)(struct seq_file *m, void *v); }; #define SEQ_SKIP 1 From patchwork Mon Jan 15 18:38:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13520082 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0D2DC47258 for ; Mon, 15 Jan 2024 18:38:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3B17D6B0088; Mon, 15 Jan 2024 13:38:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2EBD56B0089; Mon, 15 Jan 2024 13:38:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 119CF6B008A; Mon, 15 Jan 2024 13:38:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id F0B9F6B0088 for ; Mon, 15 Jan 2024 13:38:49 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id B5B01A1CD2 for ; Mon, 15 Jan 2024 18:38:49 +0000 (UTC) X-FDA: 81682406778.13.4A6385D Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf01.hostedemail.com (Postfix) with ESMTP id 075B240012 for ; Mon, 15 Jan 2024 18:38:47 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="g/etgMAm"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of 3t3ulZQYKCHgoqnajXckkcha.Ykihejqt-iigrWYg.knc@flex--surenb.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3t3ulZQYKCHgoqnajXckkcha.Ykihejqt-iigrWYg.knc@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705343928; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=21zH497B8+dg0O/C7EwYJfD0SYQoqr7syeq2JJc4d60=; b=g5sSMK3yHwiemVExYN/8KMf3rNY/uDlamulPvKehVrmt7gOjrqUio2EfmDbOEJPs5Siswh UftfRQiRuvWEKOKqsnd8qSjtJCHZTb1VPGvtGFHiEFZegByHjCLCnlMrhCg5C4uRKyE6JO uhsYg2CsUKR6FMvJteXg/Fn9wxP9o6A= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="g/etgMAm"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf01.hostedemail.com: domain of 3t3ulZQYKCHgoqnajXckkcha.Ykihejqt-iigrWYg.knc@flex--surenb.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3t3ulZQYKCHgoqnajXckkcha.Ykihejqt-iigrWYg.knc@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705343928; a=rsa-sha256; cv=none; b=6KZiVWFiCuTRc30Prp6ESfUu7bSP6rNh9yMJj1a102ddJ4i3GVoap3E2HWMycPhIybGzaj XmtJO5u5KAvCMuxQKR6SkWYuVsKv2FZfvAIXK0Xi0uBzAnbxgac8obuoh+6cHw7aijkEdF eAVRXwYKD5XYuYnUyTb06dquI0s6mdU= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-5fc6463b0edso37611547b3.0 for ; Mon, 15 Jan 2024 10:38:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1705343927; x=1705948727; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=21zH497B8+dg0O/C7EwYJfD0SYQoqr7syeq2JJc4d60=; b=g/etgMAmwJ1WJmm0l+ZbuHwkVQqX7qMRmAyyztyOZ5GQhIr1F9TiuhYa3RT/Vns05t tgSZnxvimEYylOa7wLH8jvg75ygJqz1vRrqKBetmRcTCUo3j+KyDnboxcuNDa6U/RJmf AkSBEKzWFM7UGR7FrKftS5ejj7wCEv3o7HSAEMFE8UTRPcgBFZTbuZ7TTHRu6TFFuURp u/1fUlk758W/IjwLSaJxKnYB8qkomrznk3dbR6TZj4a1MEvCCm3sAdPmxIa/XUPBiksj GHmrJ/exeD2E3sBLGnCaSncRaVLvXa2BapqRfF8H2NCNqRg0HvgTM2vz7a/h84PsIS/I bjdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705343927; x=1705948727; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=21zH497B8+dg0O/C7EwYJfD0SYQoqr7syeq2JJc4d60=; b=W3itlah2LbWd0/BjqqarKTsOS4MPNu9BH1ll7re/tU2siaTfPhx2jWzof9lKp30TH7 5FlArE1I05kahg08he7CGpDLQ5bGKXYzk4NZW3jCh7HOO1G8/tZ1BLCL05zQ8ckI5qn5 cKgQ79uIBgZSrUnbDkWSgVbiqhFx3euBN6gPEM0vuBV85eEeuFMT8QCDinf9wzcVX1PK xUHZcltwPj3gJE4OEqDN6QtkFPTHN/8ThSBQitD7QGky2nke96OFQBA/yVML7Ko3Vlvu xv8aRPLdeRQDSfKUQ5yl/73ONNDNvJy6YqaDjz8as/FT2TxW6bmFoiJ9byPRbq7Dtchg jXCQ== X-Gm-Message-State: AOJu0Yxky914qp0O+lkUuP0SibzPit/ctW2dx0AolqCl0ulFQfMQk4HC NK6ZNSxNFNDcj+y5C/9DdIIOHaurggymUkM59g== X-Google-Smtp-Source: AGHT+IFB8/hnPbdwsoiv7tQuwUtxLgPQDLNW02byS+mXpmjbpqM8r5SRgtD+OQwpIXqVBxjlqXtVMNNKQrk= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:3af2:e48e:2785:270]) (user=surenb job=sendgmr) by 2002:a05:690c:805:b0:5fc:4ef9:9d6b with SMTP id bx5-20020a05690c080500b005fc4ef99d6bmr2038449ywb.9.1705343927162; Mon, 15 Jan 2024 10:38:47 -0800 (PST) Date: Mon, 15 Jan 2024 10:38:36 -0800 In-Reply-To: <20240115183837.205694-1-surenb@google.com> Mime-Version: 1.0 References: <20240115183837.205694-1-surenb@google.com> X-Mailer: git-send-email 2.43.0.381.gb435a96ce8-goog Message-ID: <20240115183837.205694-4-surenb@google.com> Subject: [RFC 3/3] mm/maps: read proc/pid/maps under RCU From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, dchinner@redhat.com, casey@schaufler-ca.com, ben.wolsieffer@hefring.com, paulmck@kernel.org, david@redhat.com, avagin@google.com, usama.anjum@collabora.com, peterx@redhat.com, hughd@google.com, ryan.roberts@arm.com, wangkefeng.wang@huawei.com, Liam.Howlett@Oracle.com, yuzhao@google.com, axelrasmussen@google.com, lstoakes@gmail.com, talumbau@google.com, willy@infradead.org, vbabka@suse.cz, mgorman@techsingularity.net, jhubbard@nvidia.com, vishal.moola@gmail.com, mathieu.desnoyers@efficios.com, dhowells@redhat.com, jgg@ziepe.ca, sidhartha.kumar@oracle.com, andriy.shevchenko@linux.intel.com, yangxingui@huawei.com, keescook@chromium.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, kernel-team@android.com, surenb@google.com X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 075B240012 X-Stat-Signature: azhjrdkeb6w6tujuf3cw789mmmyq5k1i X-Rspam-User: X-HE-Tag: 1705343927-201734 X-HE-Meta: U2FsdGVkX19SE+K4mXvIXnWTlIH6l73q0Mn8KK7ngngv+RBN2anMAWrzScu2lg7UiyukFatZN4UiQRY7Pk+7k8Q0M2q1XlOMXmeN7eAOi8WySZ/SAZXb7xoq3PBD4Oihd9FkOjkCSXhP3AOGl0IvRupZCW1HjwY/yd9NQsMD/gbQsqri9GbA6+xsrYITCr8ixfksjEZHFBsBLv8b/sbAlgdtM9ZQc2DdBhaS80cTTHZKnbgmMx6Trrm8+Fl7gCvouUar/xp/HBfbfXo76WSKbkw3xhZVusATJo0UWQHKH3Yf3igMBOhujHDcFIx/LsGjtyUd+aJrifz26EOuW5kvaRU3dRhLfIgPmJhjkID2Trx3N/NaDJa4I5Y9rqIWscTP8f3cHHNWXGARbGyXpFabNgDPVuFfe5cnzN9TRV/CNVsrVJyZpBFNvcEtHccSIMLIiSDNv8uuz8UXvaJ/5DxthDKxM3fN7ValJm/1Q1Jlcgczn0rFZfugpKEeovawT5uzrEarIf2QHZVwlpMtP/Qoy/5mK/WzJ2QhG7Im3xZNXkP+9uxlNRzPl8lXL8UDbXQb4tWs1UqvW+v/2n3mNsAV7i9az3FYVhWr6oi8zOm6YZFlHQrx8uwp7eAEhLFReXmJbOA7KgmvClXaEU64bVhkvGK3wx79zpBQGA9+dAiy6u8n8+XydesTWiTbmtmvwT3XdfHRiTTun56xzXXTytqomZj7rUo22F7EIVGprXGqocPRfCXRgai2PIznetY/wnc8NFxKtPGMq4yoac3CWsoJxiqmp/DhrW76zqrEZhvah49kasYSv+KK51hE1KSmB593YkTDc40OPBOacgbCKx7kEEZxLBRcFbUeWbQ9HQY9iCJPcnZ+ZAUQJkXCbfzSYut7l1dwMgMNXKXg2+5b9ku+lXwy7VnQC8DSpvBf4ikl49Mn1ZWOYmjo3AZ/8ngfTHyViKWmAoFXV6nk1CvF8at ekmuN+mM KceKH/wctC2/LgZ2UehF/b6OlDOb4SB4vnG/R+JwebU9XqmLqCTeCaou3YI+T+S8iXQ7f6w86W8HEEGVofD+Ctp4dv+4k3UzIfQnNzj/OCUHPU9PAA00D3x5L/MMcldHAkGHbC4vyoTyls6l4cKhzp9F5wvzISYkYLlPcMWdC3TOddwyTaaiSvMA9si1fcAb8LX/hrsx9NRracopbn/HE8f7CwdpaMkbTox3foeqIDkTMbxNtu+w3N/r6ahPoXXt4INnuqhPnuoFg5uryLpC5eLZQn0asxNvZpbDeOI0PI5hu3MvE9/WHSNKQcC3n+SjVKlSWZfPLsWzTlAmWj0zxrocOaH+Rw/lBwBMVuBfV4vn0QBgraaM7agAWajoIdYE/biyK3I+NQ07oxV28SUeAEEsuEN9kS8Rbzvs3S/I8DxSAdAGrzUR3whPdEzGTpJeJ97IEOeCS5TFn+Lyc//y2MGZFxs+/KGJ10ip//PhG8i3W+XDUbTQhVVKtmtOh+tb4/R5ly1/TFvcsy8s0ueCx0umPYcDPCUApJPbCwLKWMGbaIFAwxd6iaYITvM+kqAV4j3oO5fMHD+9Ot1XQ6w3C0WnruWJQPL5LZusK3+lrScZDnwGzlI5YPU1ZLP19bhTvkkfQzr93YVqa5l/TUVT8tnCDhr1trz46dIeRvn4vfqtHAuY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: With maple_tree supporting vma tree traversal under RCU and per-vma locks making vma access RCU-safe, /proc/pid/maps can be read under RCU and without the need to read-lock mmap_lock. However vma content can change from under us, therefore we need to pin pointer fields used when generating the output (currently only vm_file and anon_name). In addition, we validate data before publishing it to the user using new seq_file validate interface. This way we keep this mechanism consistent with the previous behavior where data tearing is possible only at page boundaries. This change is designed to reduce mmap_lock contention and prevent a process reading /proc/pid/maps files (often a low priority task, such as monitoring/data collection services) from blocking address space updates. Signed-off-by: Suren Baghdasaryan --- fs/proc/internal.h | 3 ++ fs/proc/task_mmu.c | 130 ++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 120 insertions(+), 13 deletions(-) diff --git a/fs/proc/internal.h b/fs/proc/internal.h index a71ac5379584..47233408550b 100644 --- a/fs/proc/internal.h +++ b/fs/proc/internal.h @@ -290,6 +290,9 @@ struct proc_maps_private { struct task_struct *task; struct mm_struct *mm; struct vma_iterator iter; + int mm_lock_seq; + struct anon_vma_name *anon_name; + struct file *vm_file; #ifdef CONFIG_NUMA struct mempolicy *task_mempolicy; #endif diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 62b16f42d5d2..d4305cfdca58 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -141,6 +141,22 @@ static struct vm_area_struct *proc_get_vma(struct proc_maps_private *priv, return vma; } +static const struct seq_operations proc_pid_maps_op; + +static inline bool needs_mmap_lock(struct seq_file *m) +{ +#ifdef CONFIG_PER_VMA_LOCK + /* + * smaps and numa_maps perform page table walk, therefore require + * mmap_lock but maps can be read under RCU. + */ + return m->op != &proc_pid_maps_op; +#else + /* Without per-vma locks VMA access is not RCU-safe */ + return true; +#endif +} + static void *m_start(struct seq_file *m, loff_t *ppos) { struct proc_maps_private *priv = m->private; @@ -162,11 +178,17 @@ static void *m_start(struct seq_file *m, loff_t *ppos) return NULL; } - if (mmap_read_lock_killable(mm)) { - mmput(mm); - put_task_struct(priv->task); - priv->task = NULL; - return ERR_PTR(-EINTR); + if (needs_mmap_lock(m)) { + if (mmap_read_lock_killable(mm)) { + mmput(mm); + put_task_struct(priv->task); + priv->task = NULL; + return ERR_PTR(-EINTR); + } + } else { + /* For memory barrier see the comment for mm_lock_seq in mm_struct */ + priv->mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq); + rcu_read_lock(); } vma_iter_init(&priv->iter, mm, last_addr); @@ -195,7 +217,10 @@ static void m_stop(struct seq_file *m, void *v) return; release_task_mempolicy(priv); - mmap_read_unlock(mm); + if (needs_mmap_lock(m)) + mmap_read_unlock(mm); + else + rcu_read_unlock(); mmput(mm); put_task_struct(priv->task); priv->task = NULL; @@ -283,8 +308,10 @@ show_map_vma(struct seq_file *m, struct vm_area_struct *vma) start = vma->vm_start; end = vma->vm_end; show_vma_header_prefix(m, start, end, flags, pgoff, dev, ino); - if (mm) - anon_name = anon_vma_name(vma); + if (mm) { + anon_name = needs_mmap_lock(m) ? anon_vma_name(vma) : + anon_vma_name_get_rcu(vma); + } /* * Print the dentry name for named mappings, and a @@ -338,19 +365,96 @@ show_map_vma(struct seq_file *m, struct vm_area_struct *vma) seq_puts(m, name); } seq_putc(m, '\n'); + if (anon_name && !needs_mmap_lock(m)) + anon_vma_name_put(anon_name); +} + +/* + * Pin vm_area_struct fields used by show_map_vma. We also copy pinned fields + * into proc_maps_private because by the time put_vma_fields() is called, VMA + * might have changed and these fields might be pointing to different objects. + */ +static bool get_vma_fields(struct vm_area_struct *vma, struct proc_maps_private *priv) +{ + if (vma->vm_file) { + priv->vm_file = get_file_rcu(&vma->vm_file); + if (!priv->vm_file) + return false; + + } else + priv->vm_file = NULL; + + if (vma->anon_name) { + priv->anon_name = anon_vma_name_get_rcu(vma); + if (!priv->anon_name) { + if (priv->vm_file) { + fput(priv->vm_file); + return false; + } + } + } else + priv->anon_name = NULL; + + return true; +} + +static void put_vma_fields(struct proc_maps_private *priv) +{ + if (priv->anon_name) + anon_vma_name_put(priv->anon_name); + if (priv->vm_file) + fput(priv->vm_file); } static int show_map(struct seq_file *m, void *v) { - show_map_vma(m, v); + struct proc_maps_private *priv = m->private; + + if (needs_mmap_lock(m)) + show_map_vma(m, v); + else { + /* + * Stop immediately if the VMA changed from under us. + * Validation step will prevent publishing already cached data. + */ + if (!get_vma_fields(v, priv)) + return -EAGAIN; + + show_map_vma(m, v); + put_vma_fields(priv); + } + return 0; } +static int validate_map(struct seq_file *m, void *v) +{ + if (!needs_mmap_lock(m)) { + struct proc_maps_private *priv = m->private; + int mm_lock_seq; + + /* For memory barrier see the comment for mm_lock_seq in mm_struct */ + mm_lock_seq = smp_load_acquire(&priv->mm->mm_lock_seq); + if (mm_lock_seq != priv->mm_lock_seq) { + /* + * mmap_lock contention is detected. Wait for mmap_lock + * write to be released, discard stale data and retry. + */ + mmap_read_lock(priv->mm); + mmap_read_unlock(priv->mm); + return -EAGAIN; + } + } + return 0; + +} + static const struct seq_operations proc_pid_maps_op = { - .start = m_start, - .next = m_next, - .stop = m_stop, - .show = show_map + .start = m_start, + .next = m_next, + .stop = m_stop, + .show = show_map, + .validate = validate_map, }; static int pid_maps_open(struct inode *inode, struct file *file)