From patchwork Thu Oct 17 08:06:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13839589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39608D2F7FD for ; Thu, 17 Oct 2024 08:06:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BBCBB6B007B; Thu, 17 Oct 2024 04:06:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B6D146B0082; Thu, 17 Oct 2024 04:06:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A34EB6B0083; Thu, 17 Oct 2024 04:06:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 7F5A56B007B for ; Thu, 17 Oct 2024 04:06:08 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id DE731AC651 for ; Thu, 17 Oct 2024 08:05:46 +0000 (UTC) X-FDA: 82682360922.21.0E9936F Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf27.hostedemail.com (Postfix) with ESMTP id 10EBE40012 for ; Thu, 17 Oct 2024 08:05:56 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=K7efwpBA; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf27.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729152259; a=rsa-sha256; cv=none; b=xBbTDLw9p60gPwNF71rOA4biFH/K3a63LNP8adN7LGOqdQf1YVrJIRAzuYFzI849tZwaOA oYDkYQzoJYPtMq4QS7pQHRzJuTwkLNPMBumyBQ3Qq+eonjaBh0ETAMWAjqCXBDHqbxlBkH DPTcDMkjCWc/tCOl3RCLyqgRkAv1Vi0= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=K7efwpBA; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf27.hostedemail.com: domain of namhyung@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=namhyung@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729152259; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=GyxAp9lZDxJT+HAhhcZ8C6BAVsJFktd28VYuQBxlOOY=; b=m16XZGU91KJ/6/XNsw227KU6g4AoO1nIAyoB37Y/96TDhtxxDw/qPkRXD46r9xfIc5quzt iK/Vm2t/LcgnsMpZu9HUiyaXU9OVnNjwVik4763wP1iGxXwVi7Q34on+s2KiAUX1o2q4ox 9JhaBPRVgNQbN6ORuZmLVR+T9mYGmno= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 68BBA5C54CC; Thu, 17 Oct 2024 08:06:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AA9B8C4CEC3; Thu, 17 Oct 2024 08:06:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729152365; bh=xhFeg8d3ZWTLiCtqZ3vBuXhl5x01QSY54MvjjIElYqk=; h=From:To:Cc:Subject:Date:From; b=K7efwpBAPFGbHuhzunUc2QaAoYXQGvgJ0Z4GkctsmDARFen9LFPnov4tOVlsL6ajn MtuBCbLOr705HX4kPXojoTFNYCmqyUJT3cw4vFplqwBkkjUfQosAWmp/xO3VD0Ayby Nys6vuOCh5t7WOYofWupYigpR6FTnYZILFI7c0ndK+NtB+b+k7NBvhLniC/rdQ/2dh P9jlEzaK3mO2wPV6ioY6EO4S9W/Vyz7tJ+/CZ2uvTcmStDfck+WCcO9DA60ZgpMeHL c8OVaEOWifVvOh8F00/dqILMx00M3QektgYxY5yGXqFZK9R82GHrh9cG5n3qluMTaQ miQ6c5PJmvWug== From: Namhyung Kim To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , LKML , bpf@vger.kernel.org, Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Arnaldo Carvalho de Melo , Kees Cook Subject: [PATCH bpf-next 1/2] bpf: Add open coded version of kmem_cache iterator Date: Thu, 17 Oct 2024 01:06:03 -0700 Message-ID: <20241017080604.541872-1-namhyung@kernel.org> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog MIME-Version: 1.0 X-Rspamd-Queue-Id: 10EBE40012 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 9qnpqe61g45angfcpnqgxtucj9m81xmh X-HE-Tag: 1729152356-318167 X-HE-Meta: U2FsdGVkX1+5yRYt/YjMLiwmEx8TY4ApdGYgpBNUWMOp4o2Flv8wFrPGUxi2dzehmvX5vMRLxXjfEYB/RmQd1+5bSBJyIhiZVrPpue8C/eDAKEi4CXrMjl9/sBxEJt9gAHBKabK7fGLsrfQ6hazsgMoS5nz09IaEHNFH7Sp0U9l99nHy2Whi983kZ5VvAar5MFlIOyrSs54j/aQeRrYBe2PBUSNgx7EojCNx7f4lx+efs1MImRnMjrmyJDZvFUQDUyk1NvLwMuvc8KNxNbsYi2HLRoYMGji/MBoyi1KVXOdHuQy7R9MZZWVSduHwRh3u5v/pKoQa2H/HqYGiEw3LZk2UkkVElO1Vf+RumNxulAWgt7uJbC3iD1OkB2/nv33UzQLSrHviV3hMoDdBkKKRPqyFVYwADvuAiH3o6oKx5pgIFwzBsapNQkDjHR81gMqzOVVg+rvqebp5m2815HNivVlGxtrtVBz1glLFMJ+jgb54qdqBr5q2hb7NDkxzzSPmjEWfnTslWV57sdVtNDYw9YWGVdRdoq/1UMKoXUBdBVWDXnuwvFuZd6zQM5PRX9j8XtIVIgSO0JrPT5arokLAUsEwK9DpHYbgMcAygooyMQ5U9h8tIR9zIH7Vtg0ise7kLr+tzGqUcCEHYkjmwmq+HbXP74JIyk37lfd/bAWlUesXoq1alguPIb0ju3ePwY8JWc8N9KipG9Ct5auVZESRVTEhVzLbnjwqeI7ue/cpgfKQ6TiCiPOBZ9WENJ4K3OM9suexS04S7UgxkjjigFcqaA/YRWjNC7I/HJGfDPftGreMrZqd+TSxkPk5RYc04gn+Wca5R568tVb6CrCoNpXC28yW65LeCRJjHSfpRO+S3aQdhSkIpLFg2ZWpw5yYS4GtyspH3+Gv0SqQYGsVtS/SPYsnYH96Aw1ZASE0kWIuV/0pWBvJbvmjRM7xVKAl52/NSMv3vGCEmJ8rOubAcsw EQXc1rsu k4jvw4vioTuZeUoaxBGz6sN7OT2Hv4oc1R6Y7GIuZz0j/yU6LnHWh068NVvV755MWunyaekD3Rb3VH/eThO5R5CcpK4t81AnGNG59GXglh/qtvl4Q5zwi5SQXJoMt6FhqQUQNgxyar6n2O7PZ0oXzzbeJTY58bN9pCVfRWdKSaeTS9WXm/lUg8DlGrc1ip5GLVwc0lEMFyjSvq0i5/6AN55MEsne49DF5w7jbnvUrrQSNt+HOScsbM/4e68RLwpnot+o+Pi76W2Vyk8XAz14CTUmkUKUWntVyBv5jwgO12VVlEgOXSGESd77dTI90decMaNN/P7OgyoVhTJ6AyaaTATYa61N2qACewS5e X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a new open coded iterator for kmem_cache which can be called from a BPF program like below. It doesn't take any argument and traverses all kmem_cache entries. struct kmem_cache *pos; bpf_for_each(kmem_cache, pos) { ... } As it needs to grab slab_mutex, it should be called from sleepable BPF programs only. Signed-off-by: Namhyung Kim --- kernel/bpf/helpers.c | 3 ++ kernel/bpf/kmem_cache_iter.c | 87 ++++++++++++++++++++++++++++++++++++ 2 files changed, 90 insertions(+) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 073e6f04f4d765ff..d1dfa4f335577914 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -3111,6 +3111,9 @@ BTF_ID_FLAGS(func, bpf_iter_bits_next, KF_ITER_NEXT | KF_RET_NULL) BTF_ID_FLAGS(func, bpf_iter_bits_destroy, KF_ITER_DESTROY) BTF_ID_FLAGS(func, bpf_copy_from_user_str, KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_get_kmem_cache) +BTF_ID_FLAGS(func, bpf_iter_kmem_cache_new, KF_ITER_NEW | KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_iter_kmem_cache_next, KF_ITER_NEXT | KF_RET_NULL | KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_iter_kmem_cache_destroy, KF_ITER_DESTROY | KF_SLEEPABLE) BTF_KFUNCS_END(common_btf_ids) static const struct btf_kfunc_id_set common_kfunc_set = { diff --git a/kernel/bpf/kmem_cache_iter.c b/kernel/bpf/kmem_cache_iter.c index ebc101d7da51b57c..31ddaf452b20a458 100644 --- a/kernel/bpf/kmem_cache_iter.c +++ b/kernel/bpf/kmem_cache_iter.c @@ -145,6 +145,93 @@ static const struct bpf_iter_seq_info kmem_cache_iter_seq_info = { .seq_ops = &kmem_cache_iter_seq_ops, }; +/* open-coded version */ +struct bpf_iter_kmem_cache { + __u64 __opaque[1]; +} __attribute__((aligned(8))); + +struct bpf_iter_kmem_cache_kern { + struct kmem_cache *pos; +} __attribute__((aligned(8))); + +__bpf_kfunc_start_defs(); + +__bpf_kfunc int bpf_iter_kmem_cache_new(struct bpf_iter_kmem_cache *it) +{ + struct bpf_iter_kmem_cache_kern *kit = (void *)it; + + BUILD_BUG_ON(sizeof(*kit) > sizeof(*it)); + BUILD_BUG_ON(__alignof__(*kit) != __alignof__(*it)); + + kit->pos = NULL; + return 0; +} + +__bpf_kfunc struct kmem_cache *bpf_iter_kmem_cache_next(struct bpf_iter_kmem_cache *it) +{ + struct bpf_iter_kmem_cache_kern *kit = (void *)it; + struct kmem_cache *prev = kit->pos; + struct kmem_cache *next; + bool destroy = false; + + mutex_lock(&slab_mutex); + + if (list_empty(&slab_caches)) { + mutex_unlock(&slab_mutex); + return NULL; + } + + if (prev == NULL) + next = list_first_entry(&slab_caches, struct kmem_cache, list); + else if (list_last_entry(&slab_caches, struct kmem_cache, list) == prev) + next = NULL; + else + next = list_next_entry(prev, list); + + /* boot_caches have negative refcount, don't touch them */ + if (next && next->refcount > 0) + next->refcount++; + + /* Skip kmem_cache_destroy() for active entries */ + if (prev && prev->refcount > 1) + prev->refcount--; + else if (prev && prev->refcount == 1) + destroy = true; + + mutex_unlock(&slab_mutex); + + if (destroy) + kmem_cache_destroy(prev); + + kit->pos = next; + return next; +} + +__bpf_kfunc void bpf_iter_kmem_cache_destroy(struct bpf_iter_kmem_cache *it) +{ + struct bpf_iter_kmem_cache_kern *kit = (void *)it; + struct kmem_cache *s = kit->pos; + bool destroy = false; + + if (s == NULL) + return; + + mutex_lock(&slab_mutex); + + /* Skip kmem_cache_destroy() for active entries */ + if (s->refcount > 1) + s->refcount--; + else if (s->refcount == 1) + destroy = true; + + mutex_unlock(&slab_mutex); + + if (destroy) + kmem_cache_destroy(s); +} + +__bpf_kfunc_end_defs(); + static void bpf_iter_kmem_cache_show_fdinfo(const struct bpf_iter_aux_info *aux, struct seq_file *seq) { From patchwork Thu Oct 17 08:06:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Namhyung Kim X-Patchwork-Id: 13839590 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90D5BD2F7FC for ; Thu, 17 Oct 2024 08:06:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0E8766B0082; Thu, 17 Oct 2024 04:06:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 099DF6B0083; Thu, 17 Oct 2024 04:06:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E7B4B6B0088; Thu, 17 Oct 2024 04:06:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C2C976B0082 for ; Thu, 17 Oct 2024 04:06:09 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id B05ED12113E for ; Thu, 17 Oct 2024 08:05:59 +0000 (UTC) X-FDA: 82682361174.29.FC02EAA Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf11.hostedemail.com (Postfix) with ESMTP id 8CF1040011 for ; Thu, 17 Oct 2024 08:05:56 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=W7b8fumq; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of namhyung@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=namhyung@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729152320; a=rsa-sha256; cv=none; b=RvQ2YXVyxgWDN5RCFAKOTDReKmky9A78umnADMZRHXlfCyh5u5bjJZsAe0WWFUJ9mxOsbe xChZ+NWCYyAWRw/0cz8QDRTTR/supP303Jc3jiYLSYMomS9endFXW2wxRSKzYXmvybrp51 I733mG40JR9iUAZvUk9PDgSMgTnqrcw= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=W7b8fumq; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of namhyung@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=namhyung@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729152320; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Pe2f9OYh4G5DNUIrEyBBFtuXb8kicNEHApn0/T33CYs=; b=hCWyeYjc3MkVXzqPWMUsPqN2AAV253gLxnqJt7bI/xew+DHO/+fEsmRM7SUjaieHy1wPfx zfBUsmum8/ZKgOZ7ixaEnQBXGtCpNUV5VyS6R4iwE1asp33XdZKiw1tt92nV4XcvZNzZQg UhtVaf2xqO0WFn+UOIqbycQfRGe0vLg= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id DF901A4384F; Thu, 17 Oct 2024 08:05:57 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B419DC4CECD; Thu, 17 Oct 2024 08:06:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729152366; bh=yT8+3HibLytUwdjsKhrLp1hPXDVTvisDL7LIxziLei0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=W7b8fumqVoqSkV7fsBqyP+/nbc0CsubJ4ryJu0S1h6SxTMOTt6thxyqZJnMt1bPCl +BGpQsGcCHm5sajQSMrVzZ/UFM/K+IDRS6QMBh4mlB9uIhVSkEFgDxdUtDwRrejHgB ZOpwBdP8TaR7z3tJ9fLZtn+rXMVolW/aSmyQmf2k7O36peSRsB7zBQdXYHx84g5ENh EOWx+hmxShGFDwm6Ougmx8zwVGqXca1yjW4feQC1i+jLmgALSmeA92gXEUKSor8rjH bS+JeVGYvPFMWXWSqPEUaOOqvzo5/SveIa9K8QCEEwS3iPYl1ImIHKlvNIDcR4zdaG hqxIczeyKBw0Q== From: Namhyung Kim To: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko Cc: Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , LKML , bpf@vger.kernel.org, Andrew Morton , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Arnaldo Carvalho de Melo , Kees Cook Subject: [PATCH bpf-next 2/2] selftests/bpf: Add a test for open coded kmem_cache iter Date: Thu, 17 Oct 2024 01:06:04 -0700 Message-ID: <20241017080604.541872-2-namhyung@kernel.org> X-Mailer: git-send-email 2.47.0.rc1.288.g06298d1525-goog In-Reply-To: <20241017080604.541872-1-namhyung@kernel.org> References: <20241017080604.541872-1-namhyung@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 8CF1040011 X-Rspamd-Server: rspam01 X-Stat-Signature: 5icsarzp79jgeasp3ygsw7ra171xwzqh X-HE-Tag: 1729152356-810970 X-HE-Meta: U2FsdGVkX1+gAqlJ55eT934XaEyU+oQl/k+0sa9zEK8WTZ25v/MWQH9IpFU2KzA/GHrKjNRuWoSFwrY1XvEUGPh2ws20bRD8QWKwjj1/hnd9V5L36IulYCj0xDBb0e/O0ulkP7Q6s7akz/0b459hLFeKCQW27fZPz/jJN8Dwcg8ZsvIHzA7O62k96aubEbboB9fqXWETLWWwPGfiQhSxxcX8mcLgez+aiKTNczhTmQK0G340LoG5mCzb+c7RJNYMG+ziy+jUdYSqXc/ZU/iTdLFyDylq1V2K92lAa6dbnTo/YWJeUwaIsjpUSVt0QBlFXXqY/GtDyFayzrzCLiATneTEJl5mDTDKtMlG/t+pMjbIIRiVsSBi6M4lQahnRSAe7M0G1dCg4TKI+AWWv4lIPEsMUhB6he1n6yBCzI0p/xPI0+MrQpCf1VG9lRTE0tT9iDrG8C2EF9sL4iAyucnxvfqYcO/Is8QEQa2+J7EVrvA/qQxY3tvV+oW3hSNgXjrEW8lhmMmlYJx8aDsSmkOOHaXBGsOVmfG28fU+MTRiSyM/5erXNE6DMWMtBDnf0c/eCaSMys7Czz1fEIRpqtwHS6UvtVBXgqabFMqqL98tMVfx39X9a7D0lZ7DC6gCLLrHFiQ87yDmgZYlEFvDbVur7klmEVJz/vHJKSTk6bZQRByLmHPV50flk23qNqQ5siR0SNYaopVFc2O92ulES8Bl8XkJfe7P10DBLieyJJSA3WMBL5G+ywPwzQ88B6Z0YX453T4m6B2rebTmbQWWD/iCh8+rSOI8FPBm3pXW9qp/Hdzod0KEwrWWZ3GhtCTwPkVuuTN4CDx0735qWSMDBEHLJh9v2kVaNQOPeG48JpGFC1ooSot9QWQUrxDbA7zFmr0rsUx2B0tSBXZhWrUip1J0EKpsWd7EWZYXmEUoAgA/jZ5E0I5R9kFbNghNrUbzyhc+B3cDqLHg/3a4CYOMrkS LItYkbAW gYi3eE/Hdx7p7AMXBYDU3K6NDWHiKGwhldn71wJtcO+yShdY+XSXVKZsrkEQhurkMAbjnXRM1cYpOynyE4XC71VD03c6navat6g5dsE8Y3nR5rFe64ZOjkVbPwSKWdp5EGtQpFOjcgKy920r62fOpiEaLux9JvFRFv0UENM1+/8cLo5I+IQvJ/rpPKC0LD5FR7VjZ4fSryV3Il9Lxz1WawmaxN2PN+K7EAVStW9B3l0nBNWMHAi+vMPjfjv0cxHJ8vcUR9rsA7gNavqA7XWGb5XmQERVWbuROMUfDUxkdKi0KWRFpbTm2kvBOBe3b+Khf0MHjfkICVCT3K3v8dodIWDEsohz50tpQhhpQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The new subtest is attached to sleepable fentry of syncfs() syscall. It iterates the kmem_cache using bpf_for_each loop and count the number of entries. Finally it checks it with the number of entries from the regular iterator. $ ./vmtest.sh -- ./test_progs -t kmem_cache_iter ... #130/1 kmem_cache_iter/check_task_struct:OK #130/2 kmem_cache_iter/check_slabinfo:OK #130/3 kmem_cache_iter/open_coded_iter:OK #130 kmem_cache_iter:OK Summary: 1/3 PASSED, 0 SKIPPED, 0 FAILED Also simplify the code by using attach routine of the skeleton. Signed-off-by: Namhyung Kim --- .../testing/selftests/bpf/bpf_experimental.h | 6 ++++ .../bpf/prog_tests/kmem_cache_iter.c | 28 +++++++++++-------- .../selftests/bpf/progs/kmem_cache_iter.c | 24 ++++++++++++++++ 3 files changed, 46 insertions(+), 12 deletions(-) diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h index b0668f29f7b394eb..cd8ecd39c3f3c68d 100644 --- a/tools/testing/selftests/bpf/bpf_experimental.h +++ b/tools/testing/selftests/bpf/bpf_experimental.h @@ -582,4 +582,10 @@ extern int bpf_wq_set_callback_impl(struct bpf_wq *wq, unsigned int flags__k, void *aux__ign) __ksym; #define bpf_wq_set_callback(timer, cb, flags) \ bpf_wq_set_callback_impl(timer, cb, flags, NULL) + +struct bpf_iter_kmem_cache; +extern int bpf_iter_kmem_cache_new(struct bpf_iter_kmem_cache *it) __weak __ksym; +extern struct kmem_cache *bpf_iter_kmem_cache_next(struct bpf_iter_kmem_cache *it) __weak __ksym; +extern void bpf_iter_kmem_cache_destroy(struct bpf_iter_kmem_cache *it) __weak __ksym; + #endif diff --git a/tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c b/tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c index 848d8fc9171fae45..a1fd3bc57c0b21bb 100644 --- a/tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c +++ b/tools/testing/selftests/bpf/prog_tests/kmem_cache_iter.c @@ -68,12 +68,18 @@ static void subtest_kmem_cache_iter_check_slabinfo(struct kmem_cache_iter *skel) fclose(fp); } +static void subtest_kmem_cache_iter_open_coded(struct kmem_cache_iter *skel) +{ + /* To trigger the open coded iterator attached to the syscall */ + syncfs(0); + + /* It should be same as we've seen from the explicit iterator */ + ASSERT_EQ(skel->bss->open_coded_seen, skel->bss->kmem_cache_seen, "open_code_seen_eq"); +} + void test_kmem_cache_iter(void) { - DECLARE_LIBBPF_OPTS(bpf_iter_attach_opts, opts); struct kmem_cache_iter *skel = NULL; - union bpf_iter_link_info linfo = {}; - struct bpf_link *link; char buf[256]; int iter_fd; @@ -81,16 +87,12 @@ void test_kmem_cache_iter(void) if (!ASSERT_OK_PTR(skel, "kmem_cache_iter__open_and_load")) return; - opts.link_info = &linfo; - opts.link_info_len = sizeof(linfo); - - link = bpf_program__attach_iter(skel->progs.slab_info_collector, &opts); - if (!ASSERT_OK_PTR(link, "attach_iter")) + if (!ASSERT_OK(kmem_cache_iter__attach(skel), "skel_attach")) goto destroy; - iter_fd = bpf_iter_create(bpf_link__fd(link)); + iter_fd = bpf_iter_create(bpf_link__fd(skel->links.slab_info_collector)); if (!ASSERT_GE(iter_fd, 0, "iter_create")) - goto free_link; + goto detach; memset(buf, 0, sizeof(buf)); while (read(iter_fd, buf, sizeof(buf) > 0)) { @@ -105,11 +107,13 @@ void test_kmem_cache_iter(void) subtest_kmem_cache_iter_check_task_struct(skel); if (test__start_subtest("check_slabinfo")) subtest_kmem_cache_iter_check_slabinfo(skel); + if (test__start_subtest("open_coded_iter")) + subtest_kmem_cache_iter_open_coded(skel); close(iter_fd); -free_link: - bpf_link__destroy(link); +detach: + kmem_cache_iter__detach(skel); destroy: kmem_cache_iter__destroy(skel); } diff --git a/tools/testing/selftests/bpf/progs/kmem_cache_iter.c b/tools/testing/selftests/bpf/progs/kmem_cache_iter.c index 72c9dafecd98406b..4c44aa279a5328fe 100644 --- a/tools/testing/selftests/bpf/progs/kmem_cache_iter.c +++ b/tools/testing/selftests/bpf/progs/kmem_cache_iter.c @@ -2,6 +2,8 @@ /* Copyright (c) 2024 Google */ #include "bpf_iter.h" +#include "bpf_experimental.h" +#include "bpf_misc.h" #include #include @@ -33,6 +35,7 @@ extern struct kmem_cache *bpf_get_kmem_cache(u64 addr) __ksym; /* Result, will be checked by userspace */ int task_struct_found; int kmem_cache_seen; +int open_coded_seen; SEC("iter/kmem_cache") int slab_info_collector(struct bpf_iter__kmem_cache *ctx) @@ -85,3 +88,24 @@ int BPF_PROG(check_task_struct) task_struct_found = -2; return 0; } + +SEC("fentry.s/" SYS_PREFIX "sys_syncfs") +int open_coded_iter(const void *ctx) +{ + struct kmem_cache *s; + + bpf_for_each(kmem_cache, s) { + struct kmem_cache_result *r; + int idx = open_coded_seen; + + r = bpf_map_lookup_elem(&slab_result, &idx); + if (r == NULL) + break; + + open_coded_seen++; + + if (r->obj_size != s->size) + break; + } + return 0; +}