From patchwork Tue Nov 12 08:25:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 13871852 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1617D79C4; Tue, 12 Nov 2024 08:26:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731399987; cv=none; b=ZJ7Ess4sfNZO+XEcYYRNhmqxZD4Cq3eiP8woKaEnPgFaoUUPeG6wsyXVjSG65UxrrGrOpNrC8WZoU/dw3zsMz/6yJinYcXBp+Ooiw1h23HMF9F2EJucynQyDhKRD2IeOuaBe88UyBmjqwsBGM0VuuJhvzCx8s3bFWodJJELxZeg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731399987; c=relaxed/simple; bh=6wcgg1hEfQa6Nf8Cxbudvhg88dmxNzVd4PTSML8QbyY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WEEat5nPFn7GDdd7ltxPR1R5ydJImJ0Tpk5oqocvBHiE8YkVV9eD4Nqm0GhfYG61J0jW+VTTplfj3LaqbyWSYLd3zT8iXhdrlQGDMpwViMZ2hYWx5lsWi1oPCX2mArK4dFlVdBs6hl6pxdqE3LBZt95uR0qs7bcxeCMZdHfjbTc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=NxxdiwZ8; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="NxxdiwZ8" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8EEFEC4CECD; Tue, 12 Nov 2024 08:26:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1731399986; bh=6wcgg1hEfQa6Nf8Cxbudvhg88dmxNzVd4PTSML8QbyY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NxxdiwZ84rBFnZJhsmTsjlyzGMqioZUjKB9lcCR6fUMtLlEhwcvhg9dzmGf+5/6q3 SOWR2qY++JLe3q274y1KIaUZevOfCGuOWjj5olTs3Z3Z5/TdZJk6Z4Y8ntfaWqxDVI ZU6M/4Wk2omO2zKSIehbGRunvsz08odris3gPzU09bzpaqeyFU+QNR9Xbg08U1/V/M p3UJMGNBg96ohHpjy7N1v1Bb3eMHvR03YMejsZNXjfNaEeQGkkekcP4iqVu0j1e5mT IobDkCXjgKYDJTv0cG6DXAlQ0Q3CUU9tWmGxvnzZuhEROIHgsMbcgFVcTk79l5OhBo lgG3BAXTk3cFw== From: Song Liu To: bpf@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-security-module@vger.kernel.org Cc: kernel-team@meta.com, andrii@kernel.org, eddyz87@gmail.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, kpsingh@kernel.org, mattbobrowski@google.com, amir73il@gmail.com, repnop@google.com, jlayton@kernel.org, josef@toxicpanda.com, mic@digikod.net, gnoack@google.com, Song Liu Subject: [PATCH bpf-next 1/4] bpf: lsm: Remove hook to bpf_task_storage_free Date: Tue, 12 Nov 2024 00:25:55 -0800 Message-ID: <20241112082600.298035-2-song@kernel.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241112082600.298035-1-song@kernel.org> References: <20241112082600.298035-1-song@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net free_task() already calls bpf_task_storage_free(). It is not necessary to call bpf_task_storage_free() again on security_task_free(). Remove the hook to bpf_task_storage_free. Signed-off-by: Song Liu --- security/bpf/hooks.c | 1 - 1 file changed, 1 deletion(-) diff --git a/security/bpf/hooks.c b/security/bpf/hooks.c index 3663aec7bcbd..db759025abe1 100644 --- a/security/bpf/hooks.c +++ b/security/bpf/hooks.c @@ -13,7 +13,6 @@ static struct security_hook_list bpf_lsm_hooks[] __ro_after_init = { #include #undef LSM_HOOK LSM_HOOK_INIT(inode_free_security, bpf_inode_storage_free), - LSM_HOOK_INIT(task_free, bpf_task_storage_free), }; static const struct lsm_id bpf_lsmid = { From patchwork Tue Nov 12 08:25:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 13871853 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6497079C4; Tue, 12 Nov 2024 08:26:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731399995; cv=none; b=SEcC0lAqSqITaxfD+k5HmbbBIkRxj9fqKvkuLkiralXf9VC0+jDiP3jCp2rYMHcs2cufJXXOTfcXOVcrqLWMZPUEjQRXU4u4vfJQJy4KdVEOI2oCQYHZHw869o5mb6iHpzKufECYHDrBJpvdYcXJkiWRwGWizBgIEEGJS49GFb4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731399995; c=relaxed/simple; bh=/PswumvMtl3pBMdLPHsAXoB0Om0Fp55HD6qa3VSzBxk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YXKnarv8NMwrHGvKApFkRRhIwWmXDDz3c61U+ooTAZjUf3Pw4iBMbUZyhZUk42EPfNdh016LYFKUWVLtsrzQjJUhyco0uNjJ8YcEhd1nMYyuHnmPmHB6lCbuX8016XIBjR7dLu7ngInbRnIQA4CM17CEMuw8jIrg4Dqt6yUKmBg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=LSplI+aH; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="LSplI+aH" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CE6A3C4CECD; Tue, 12 Nov 2024 08:26:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1731399995; bh=/PswumvMtl3pBMdLPHsAXoB0Om0Fp55HD6qa3VSzBxk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LSplI+aHKNUWF+G8VeBtGfvRr2lsCeZ6kPYn650W/GzaTRvGWPiry36KV4BKkLagS noMsOad98ClpGkhJT51RibGxrbIwF65I5DDtqx6W51noh+7fhq3kCVOaUdrJ6bYgSm BIH6EdXfsC/5vzn+Y0J+WghmFw5Nk/+5QUjx4r2/AHXFbjHCjoCqGKXbiOBU0yMh07 yCgAvUcykvb9JEhta0ygtSME2rEsL8pXX0JOxrD1mOTLiTF3q5Y3r77tCFd0pDT8pS 6WpYanQ8VG2TtLcS4Ugke0WaCvr6Kzg0xMu/dFlWFCQMA8k54euUw+er5CpcDey4zi a6CZivRtOfA/g== From: Song Liu To: bpf@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-security-module@vger.kernel.org Cc: kernel-team@meta.com, andrii@kernel.org, eddyz87@gmail.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, kpsingh@kernel.org, mattbobrowski@google.com, amir73il@gmail.com, repnop@google.com, jlayton@kernel.org, josef@toxicpanda.com, mic@digikod.net, gnoack@google.com, Song Liu Subject: [PATCH bpf-next 2/4] bpf: Make bpf inode storage available to tracing program Date: Tue, 12 Nov 2024 00:25:56 -0800 Message-ID: <20241112082600.298035-3-song@kernel.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241112082600.298035-1-song@kernel.org> References: <20241112082600.298035-1-song@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net inode storage can be useful for non-LSM program. For example, file* tools from bcc/libbpf-tools can use inode storage instead of hash map; fanotify fastpath [1] can also use inode storage to store useful data. Make inode storage available for tracing program. Move bpf inode storage from a security blob to inode->i_bpf_storage, and adjust related code accordingly. [1] https://lore.kernel.org/linux-fsdevel/20241029231244.2834368-1-song@kernel.org/ Signed-off-by: Song Liu --- fs/inode.c | 1 + include/linux/bpf.h | 9 +++++++++ include/linux/bpf_lsm.h | 29 ----------------------------- include/linux/fs.h | 4 ++++ kernel/bpf/Makefile | 3 +-- kernel/bpf/bpf_inode_storage.c | 32 +++++--------------------------- kernel/bpf/bpf_lsm.c | 4 ---- kernel/trace/bpf_trace.c | 4 ++++ security/bpf/hooks.c | 6 ------ 9 files changed, 24 insertions(+), 68 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index 8dabb224f941..3c679578169f 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -286,6 +286,7 @@ static struct inode *alloc_inode(struct super_block *sb) void __destroy_inode(struct inode *inode) { BUG_ON(inode_has_buffers(inode)); + bpf_inode_storage_free(inode); inode_detach_wb(inode); security_inode_free(inode); fsnotify_inode_delete(inode); diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 1b84613b10ac..0b31d2e74df6 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -2672,6 +2672,7 @@ struct bpf_link *bpf_link_by_id(u32 id); const struct bpf_func_proto *bpf_base_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog); void bpf_task_storage_free(struct task_struct *task); +void bpf_inode_storage_free(struct inode *inode); void bpf_cgrp_storage_free(struct cgroup *cgroup); bool bpf_prog_has_kfunc_call(const struct bpf_prog *prog); const struct btf_func_model * @@ -2942,6 +2943,10 @@ static inline void bpf_task_storage_free(struct task_struct *task) { } +static inline void bpf_inode_storage_free(struct inode *inode) +{ +} + static inline bool bpf_prog_has_kfunc_call(const struct bpf_prog *prog) { return false; @@ -3305,6 +3310,10 @@ extern const struct bpf_func_proto bpf_task_storage_get_recur_proto; extern const struct bpf_func_proto bpf_task_storage_get_proto; extern const struct bpf_func_proto bpf_task_storage_delete_recur_proto; extern const struct bpf_func_proto bpf_task_storage_delete_proto; +extern const struct bpf_func_proto bpf_inode_storage_get_proto; +extern const struct bpf_func_proto bpf_inode_storage_get_recur_proto; +extern const struct bpf_func_proto bpf_inode_storage_delete_proto; +extern const struct bpf_func_proto bpf_inode_storage_delete_recur_proto; extern const struct bpf_func_proto bpf_for_each_map_elem_proto; extern const struct bpf_func_proto bpf_btf_find_by_name_kind_proto; extern const struct bpf_func_proto bpf_sk_setsockopt_proto; diff --git a/include/linux/bpf_lsm.h b/include/linux/bpf_lsm.h index aefcd6564251..a819c2f0a062 100644 --- a/include/linux/bpf_lsm.h +++ b/include/linux/bpf_lsm.h @@ -19,31 +19,12 @@ #include #undef LSM_HOOK -struct bpf_storage_blob { - struct bpf_local_storage __rcu *storage; -}; - -extern struct lsm_blob_sizes bpf_lsm_blob_sizes; - int bpf_lsm_verify_prog(struct bpf_verifier_log *vlog, const struct bpf_prog *prog); bool bpf_lsm_is_sleepable_hook(u32 btf_id); bool bpf_lsm_is_trusted(const struct bpf_prog *prog); -static inline struct bpf_storage_blob *bpf_inode( - const struct inode *inode) -{ - if (unlikely(!inode->i_security)) - return NULL; - - return inode->i_security + bpf_lsm_blob_sizes.lbs_inode; -} - -extern const struct bpf_func_proto bpf_inode_storage_get_proto; -extern const struct bpf_func_proto bpf_inode_storage_delete_proto; -void bpf_inode_storage_free(struct inode *inode); - void bpf_lsm_find_cgroup_shim(const struct bpf_prog *prog, bpf_func_t *bpf_func); int bpf_lsm_get_retval_range(const struct bpf_prog *prog, @@ -66,16 +47,6 @@ static inline int bpf_lsm_verify_prog(struct bpf_verifier_log *vlog, return -EOPNOTSUPP; } -static inline struct bpf_storage_blob *bpf_inode( - const struct inode *inode) -{ - return NULL; -} - -static inline void bpf_inode_storage_free(struct inode *inode) -{ -} - static inline void bpf_lsm_find_cgroup_shim(const struct bpf_prog *prog, bpf_func_t *bpf_func) { diff --git a/include/linux/fs.h b/include/linux/fs.h index 3559446279c1..479097e4dd5b 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -79,6 +79,7 @@ struct fs_context; struct fs_parameter_spec; struct fileattr; struct iomap_ops; +struct bpf_local_storage; extern void __init inode_init(void); extern void __init inode_init_early(void); @@ -648,6 +649,9 @@ struct inode { #ifdef CONFIG_SECURITY void *i_security; #endif +#ifdef CONFIG_BPF_SYSCALL + struct bpf_local_storage __rcu *i_bpf_storage; +#endif /* Stat data, not accessed from path walking */ unsigned long i_ino; diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile index 105328f0b9c0..73604c7130f1 100644 --- a/kernel/bpf/Makefile +++ b/kernel/bpf/Makefile @@ -10,8 +10,7 @@ obj-$(CONFIG_BPF_SYSCALL) += syscall.o verifier.o inode.o helpers.o tnum.o log.o obj-$(CONFIG_BPF_SYSCALL) += bpf_iter.o map_iter.o task_iter.o prog_iter.o link_iter.o obj-$(CONFIG_BPF_SYSCALL) += hashtab.o arraymap.o percpu_freelist.o bpf_lru_list.o lpm_trie.o map_in_map.o bloom_filter.o obj-$(CONFIG_BPF_SYSCALL) += local_storage.o queue_stack_maps.o ringbuf.o -obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o -obj-${CONFIG_BPF_LSM} += bpf_inode_storage.o +obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o bpf_inode_storage.o obj-$(CONFIG_BPF_SYSCALL) += disasm.o mprog.o obj-$(CONFIG_BPF_JIT) += trampoline.o obj-$(CONFIG_BPF_SYSCALL) += btf.o memalloc.o diff --git a/kernel/bpf/bpf_inode_storage.c b/kernel/bpf/bpf_inode_storage.c index 44ccebc745e5..8d5a9bfe6643 100644 --- a/kernel/bpf/bpf_inode_storage.c +++ b/kernel/bpf/bpf_inode_storage.c @@ -21,16 +21,11 @@ DEFINE_BPF_STORAGE_CACHE(inode_cache); -static struct bpf_local_storage __rcu ** -inode_storage_ptr(void *owner) +static struct bpf_local_storage __rcu **inode_storage_ptr(void *owner) { struct inode *inode = owner; - struct bpf_storage_blob *bsb; - bsb = bpf_inode(inode); - if (!bsb) - return NULL; - return &bsb->storage; + return &inode->i_bpf_storage; } static struct bpf_local_storage_data *inode_storage_lookup(struct inode *inode, @@ -39,14 +34,9 @@ static struct bpf_local_storage_data *inode_storage_lookup(struct inode *inode, { struct bpf_local_storage *inode_storage; struct bpf_local_storage_map *smap; - struct bpf_storage_blob *bsb; - - bsb = bpf_inode(inode); - if (!bsb) - return NULL; inode_storage = - rcu_dereference_check(bsb->storage, bpf_rcu_lock_held()); + rcu_dereference_check(inode->i_bpf_storage, bpf_rcu_lock_held()); if (!inode_storage) return NULL; @@ -57,15 +47,10 @@ static struct bpf_local_storage_data *inode_storage_lookup(struct inode *inode, void bpf_inode_storage_free(struct inode *inode) { struct bpf_local_storage *local_storage; - struct bpf_storage_blob *bsb; - - bsb = bpf_inode(inode); - if (!bsb) - return; rcu_read_lock(); - local_storage = rcu_dereference(bsb->storage); + local_storage = rcu_dereference(inode->i_bpf_storage); if (!local_storage) { rcu_read_unlock(); return; @@ -95,8 +80,6 @@ static long bpf_fd_inode_storage_update_elem(struct bpf_map *map, void *key, if (fd_empty(f)) return -EBADF; - if (!inode_storage_ptr(file_inode(fd_file(f)))) - return -EBADF; sdata = bpf_local_storage_update(file_inode(fd_file(f)), (struct bpf_local_storage_map *)map, @@ -136,12 +119,7 @@ BPF_CALL_5(bpf_inode_storage_get, struct bpf_map *, map, struct inode *, inode, if (flags & ~(BPF_LOCAL_STORAGE_GET_F_CREATE)) return (unsigned long)NULL; - /* explicitly check that the inode_storage_ptr is not - * NULL as inode_storage_lookup returns NULL in this case and - * bpf_local_storage_update expects the owner to have a - * valid storage pointer. - */ - if (!inode || !inode_storage_ptr(inode)) + if (!inode) return (unsigned long)NULL; sdata = inode_storage_lookup(inode, map, true); diff --git a/kernel/bpf/bpf_lsm.c b/kernel/bpf/bpf_lsm.c index 3bc61628ab25..6b6e0730e515 100644 --- a/kernel/bpf/bpf_lsm.c +++ b/kernel/bpf/bpf_lsm.c @@ -231,10 +231,6 @@ bpf_lsm_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) } switch (func_id) { - case BPF_FUNC_inode_storage_get: - return &bpf_inode_storage_get_proto; - case BPF_FUNC_inode_storage_delete: - return &bpf_inode_storage_delete_proto; #ifdef CONFIG_NET case BPF_FUNC_sk_storage_get: return &bpf_sk_storage_get_proto; diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 949a3870946c..262bd101ea0b 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1553,6 +1553,10 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) if (bpf_prog_check_recur(prog)) return &bpf_task_storage_delete_recur_proto; return &bpf_task_storage_delete_proto; + case BPF_FUNC_inode_storage_get: + return &bpf_inode_storage_get_proto; + case BPF_FUNC_inode_storage_delete: + return &bpf_inode_storage_delete_proto; case BPF_FUNC_for_each_map_elem: return &bpf_for_each_map_elem_proto; case BPF_FUNC_snprintf: diff --git a/security/bpf/hooks.c b/security/bpf/hooks.c index db759025abe1..67719a04bb0b 100644 --- a/security/bpf/hooks.c +++ b/security/bpf/hooks.c @@ -12,7 +12,6 @@ static struct security_hook_list bpf_lsm_hooks[] __ro_after_init = { LSM_HOOK_INIT(NAME, bpf_lsm_##NAME), #include #undef LSM_HOOK - LSM_HOOK_INIT(inode_free_security, bpf_inode_storage_free), }; static const struct lsm_id bpf_lsmid = { @@ -28,12 +27,7 @@ static int __init bpf_lsm_init(void) return 0; } -struct lsm_blob_sizes bpf_lsm_blob_sizes __ro_after_init = { - .lbs_inode = sizeof(struct bpf_storage_blob), -}; - DEFINE_LSM(bpf) = { .name = "bpf", .init = bpf_lsm_init, - .blobs = &bpf_lsm_blob_sizes }; From patchwork Tue Nov 12 08:25:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 13871854 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1FD579C4; Tue, 12 Nov 2024 08:26:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400003; cv=none; b=lHP+LQC34YQVfeY4PPE1EXyCqfe0e79sYruFSftWiu3tHwoC+Pz1cOOlDGgK6YIsoOADg2J0+Y1joWorP0HgN0GTadD6pnH/wzRdSdCuSZ7Ypasujy8OQem6kRN1nc++n8zXdicD4cCJCK+i+U8Hejm/ef0Z8ayQCWeTJUDuNII= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400003; c=relaxed/simple; bh=b9ty9QVi0FdLwT9dydl2t4X5i3nc9IOhUfyVHSCcObk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=K+bWR7ZXO45Gt7i2Xw4CR9btkcZ9YjtnjpaslFKwgxMOQCXnUE7F7bJjbVk+tomM5xbhZqwiL0lv2cxA8JrDvJxSz3/6k3woM2rLoaoyzAwhQpcOYhWDbgdKzOC904fxeWKmWLwZPNwZz7q2ig0EH8+Wil8JBV5IhXjv3T38Whw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TZmXNgNa; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TZmXNgNa" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4D9D8C4CECD; Tue, 12 Nov 2024 08:26:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1731400002; bh=b9ty9QVi0FdLwT9dydl2t4X5i3nc9IOhUfyVHSCcObk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TZmXNgNaHPoS7Cbk270RzGLoiaJMDcx4ZbYyapTKpf0p7xF+iEpmRZlbbSYcR3HzI kw4JmA+XG+1m9DyYRozu+yXy5Zed37w36TTmgNexZF2DqvA/5eMELtQAQbcZlGF+Dr QnctU2kDZmH9sHzpHm19jN1LKcy5fXusXHDTGewJEleIFr5LzY7oepHeloBoioSUE3 6121EAI8V25WlFuqfm/Esgn1agzNCivaVHeulZ2XJ49eKX2ZYFmYxNCuqQ2zkkcOZW kbEt0VEEbQtuEClrfVRUJ8Ihn+BEOUTAis0Mikg484o1Fo7QeX6lh1+rvXFy/8rsRu Y9/hbPcxcMhfw== From: Song Liu To: bpf@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-security-module@vger.kernel.org Cc: kernel-team@meta.com, andrii@kernel.org, eddyz87@gmail.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, kpsingh@kernel.org, mattbobrowski@google.com, amir73il@gmail.com, repnop@google.com, jlayton@kernel.org, josef@toxicpanda.com, mic@digikod.net, gnoack@google.com, Song Liu Subject: [PATCH bpf-next 3/4] bpf: Add recursion avoid logic for inode storage Date: Tue, 12 Nov 2024 00:25:57 -0800 Message-ID: <20241112082600.298035-4-song@kernel.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241112082600.298035-1-song@kernel.org> References: <20241112082600.298035-1-song@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net The logic is same as task local storage. Signed-off-by: Song Liu --- kernel/bpf/bpf_inode_storage.c | 156 +++++++++++++++++++++++++++------ kernel/trace/bpf_trace.c | 4 + 2 files changed, 135 insertions(+), 25 deletions(-) diff --git a/kernel/bpf/bpf_inode_storage.c b/kernel/bpf/bpf_inode_storage.c index cd4dc266ebff..ef539f4fe583 100644 --- a/kernel/bpf/bpf_inode_storage.c +++ b/kernel/bpf/bpf_inode_storage.c @@ -21,6 +21,31 @@ DEFINE_BPF_STORAGE_CACHE(inode_cache); +static DEFINE_PER_CPU(int, bpf_inode_storage_busy); + +static void bpf_inode_storage_lock(void) +{ + migrate_disable(); + this_cpu_inc(bpf_inode_storage_busy); +} + +static void bpf_inode_storage_unlock(void) +{ + this_cpu_dec(bpf_inode_storage_busy); + migrate_enable(); +} + +static bool bpf_inode_storage_trylock(void) +{ + migrate_disable(); + if (unlikely(this_cpu_inc_return(bpf_inode_storage_busy) != 1)) { + this_cpu_dec(bpf_inode_storage_busy); + migrate_enable(); + return false; + } + return true; +} + static struct bpf_local_storage __rcu **inode_storage_ptr(void *owner) { struct inode *inode = owner; @@ -56,7 +81,9 @@ void bpf_inode_storage_free(struct inode *inode) return; } + bpf_inode_storage_lock(); bpf_local_storage_destroy(local_storage); + bpf_inode_storage_unlock(); rcu_read_unlock(); } @@ -68,7 +95,9 @@ static void *bpf_fd_inode_storage_lookup_elem(struct bpf_map *map, void *key) if (fd_empty(f)) return ERR_PTR(-EBADF); + bpf_inode_storage_lock(); sdata = inode_storage_lookup(file_inode(fd_file(f)), map, true); + bpf_inode_storage_unlock(); return sdata ? sdata->data : NULL; } @@ -81,13 +110,16 @@ static long bpf_fd_inode_storage_update_elem(struct bpf_map *map, void *key, if (fd_empty(f)) return -EBADF; + bpf_inode_storage_lock(); sdata = bpf_local_storage_update(file_inode(fd_file(f)), (struct bpf_local_storage_map *)map, value, map_flags, false, GFP_ATOMIC); + bpf_inode_storage_unlock(); return PTR_ERR_OR_ZERO(sdata); } -static int inode_storage_delete(struct inode *inode, struct bpf_map *map) +static int inode_storage_delete(struct inode *inode, struct bpf_map *map, + bool nobusy) { struct bpf_local_storage_data *sdata; @@ -95,6 +127,9 @@ static int inode_storage_delete(struct inode *inode, struct bpf_map *map) if (!sdata) return -ENOENT; + if (!nobusy) + return -EBUSY; + bpf_selem_unlink(SELEM(sdata), false); return 0; @@ -102,60 +137,111 @@ static int inode_storage_delete(struct inode *inode, struct bpf_map *map) static long bpf_fd_inode_storage_delete_elem(struct bpf_map *map, void *key) { + int err; + CLASS(fd_raw, f)(*(int *)key); if (fd_empty(f)) return -EBADF; - return inode_storage_delete(file_inode(fd_file(f)), map); + bpf_inode_storage_lock(); + err = inode_storage_delete(file_inode(fd_file(f)), map, true); + bpf_inode_storage_unlock(); + return err; +} + +static void *__bpf_inode_storage_get(struct bpf_map *map, struct inode *inode, + void *value, u64 flags, gfp_t gfp_flags, bool nobusy) +{ + struct bpf_local_storage_data *sdata; + + /* explicitly check that the inode not NULL */ + if (!inode) + return NULL; + + sdata = inode_storage_lookup(inode, map, true); + if (sdata) + return sdata->data; + + /* only allocate new storage, when the inode is refcounted */ + if (atomic_read(&inode->i_count) && + flags & BPF_LOCAL_STORAGE_GET_F_CREATE) { + sdata = bpf_local_storage_update( + inode, (struct bpf_local_storage_map *)map, value, + BPF_NOEXIST, false, gfp_flags); + return IS_ERR(sdata) ? NULL : sdata->data; + } + + return NULL; } /* *gfp_flags* is a hidden argument provided by the verifier */ -BPF_CALL_5(bpf_inode_storage_get, struct bpf_map *, map, struct inode *, inode, +BPF_CALL_5(bpf_inode_storage_get_recur, struct bpf_map *, map, struct inode *, inode, void *, value, u64, flags, gfp_t, gfp_flags) { - struct bpf_local_storage_data *sdata; + bool nobusy; + void *data; WARN_ON_ONCE(!bpf_rcu_lock_held()); if (flags & ~(BPF_LOCAL_STORAGE_GET_F_CREATE)) return (unsigned long)NULL; - /* explicitly check that the inode_storage_ptr is not - * NULL as inode_storage_lookup returns NULL in this case and - * bpf_local_storage_update expects the owner to have a - * valid storage pointer. - */ - if (!inode || !inode_storage_ptr(inode)) + nobusy = bpf_inode_storage_trylock(); + data = __bpf_inode_storage_get(map, inode, value, flags, gfp_flags, nobusy); + if (nobusy) + bpf_inode_storage_unlock(); + return (unsigned long)data; +} + +/* *gfp_flags* is a hidden argument provided by the verifier */ +BPF_CALL_5(bpf_inode_storage_get, struct bpf_map *, map, struct inode *, inode, + void *, value, u64, flags, gfp_t, gfp_flags) +{ + void *data; + + WARN_ON_ONCE(!bpf_rcu_lock_held()); + if (flags & ~(BPF_LOCAL_STORAGE_GET_F_CREATE)) return (unsigned long)NULL; - sdata = inode_storage_lookup(inode, map, true); - if (sdata) - return (unsigned long)sdata->data; + bpf_inode_storage_lock(); + data = __bpf_inode_storage_get(map, inode, value, flags, gfp_flags, true); + bpf_inode_storage_unlock(); + return (unsigned long)data; +} + +BPF_CALL_2(bpf_inode_storage_delete_recur, struct bpf_map *, map, struct inode *, inode) +{ + bool nobusy; + int ret; + + WARN_ON_ONCE(!bpf_rcu_lock_held()); + if (!inode) + return -EINVAL; + nobusy = bpf_inode_storage_trylock(); /* This helper must only called from where the inode is guaranteed * to have a refcount and cannot be freed. */ - if (flags & BPF_LOCAL_STORAGE_GET_F_CREATE) { - sdata = bpf_local_storage_update( - inode, (struct bpf_local_storage_map *)map, value, - BPF_NOEXIST, false, gfp_flags); - return IS_ERR(sdata) ? (unsigned long)NULL : - (unsigned long)sdata->data; - } - - return (unsigned long)NULL; + ret = inode_storage_delete(inode, map, nobusy); + if (nobusy) + bpf_inode_storage_unlock(); + return ret; } -BPF_CALL_2(bpf_inode_storage_delete, - struct bpf_map *, map, struct inode *, inode) +BPF_CALL_2(bpf_inode_storage_delete, struct bpf_map *, map, struct inode *, inode) { + int ret; + WARN_ON_ONCE(!bpf_rcu_lock_held()); if (!inode) return -EINVAL; + bpf_inode_storage_lock(); /* This helper must only called from where the inode is guaranteed * to have a refcount and cannot be freed. */ - return inode_storage_delete(inode, map); + ret = inode_storage_delete(inode, map, true); + bpf_inode_storage_unlock(); + return ret; } static int notsupp_get_next_key(struct bpf_map *map, void *key, @@ -191,6 +277,17 @@ const struct bpf_map_ops inode_storage_map_ops = { BTF_ID_LIST_SINGLE(bpf_inode_storage_btf_ids, struct, inode) +const struct bpf_func_proto bpf_inode_storage_get_recur_proto = { + .func = bpf_inode_storage_get_recur, + .gpl_only = false, + .ret_type = RET_PTR_TO_MAP_VALUE_OR_NULL, + .arg1_type = ARG_CONST_MAP_PTR, + .arg2_type = ARG_PTR_TO_BTF_ID_OR_NULL, + .arg2_btf_id = &bpf_inode_storage_btf_ids[0], + .arg3_type = ARG_PTR_TO_MAP_VALUE_OR_NULL, + .arg4_type = ARG_ANYTHING, +}; + const struct bpf_func_proto bpf_inode_storage_get_proto = { .func = bpf_inode_storage_get, .gpl_only = false, @@ -202,6 +299,15 @@ const struct bpf_func_proto bpf_inode_storage_get_proto = { .arg4_type = ARG_ANYTHING, }; +const struct bpf_func_proto bpf_inode_storage_delete_recur_proto = { + .func = bpf_inode_storage_delete_recur, + .gpl_only = false, + .ret_type = RET_INTEGER, + .arg1_type = ARG_CONST_MAP_PTR, + .arg2_type = ARG_PTR_TO_BTF_ID_OR_NULL, + .arg2_btf_id = &bpf_inode_storage_btf_ids[0], +}; + const struct bpf_func_proto bpf_inode_storage_delete_proto = { .func = bpf_inode_storage_delete, .gpl_only = false, diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 262bd101ea0b..4616f5430a5e 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -1554,8 +1554,12 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) return &bpf_task_storage_delete_recur_proto; return &bpf_task_storage_delete_proto; case BPF_FUNC_inode_storage_get: + if (bpf_prog_check_recur(prog)) + return &bpf_inode_storage_get_recur_proto; return &bpf_inode_storage_get_proto; case BPF_FUNC_inode_storage_delete: + if (bpf_prog_check_recur(prog)) + return &bpf_inode_storage_delete_recur_proto; return &bpf_inode_storage_delete_proto; case BPF_FUNC_for_each_map_elem: return &bpf_for_each_map_elem_proto; From patchwork Tue Nov 12 08:25:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Song Liu X-Patchwork-Id: 13871856 X-Patchwork-Delegate: bpf@iogearbox.net Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DEAEE20C01C; Tue, 12 Nov 2024 08:26:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400018; cv=none; b=ltcnzLwYhJPhGMxGkLsaHLAhoD1NzcZYrCvx9DMgn5m8nl0jk60mf3J4xxsw+00K6dDD8we6aNe7Gfq7H1AtwqCL5RCuYQUdB0Sg1qjI5zG+BUCccLSFXXDn4ejPySGp1Xi60PZLRWwnLv2UA3aAzCsCVxGj7QenEdmG/TV5avc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731400018; c=relaxed/simple; bh=h4/pxwZqbYL0YDJ6s/BsT2Km5NWKmXNu7R1WBTg/gjw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=s+UDYlTjjIjtarM8EfRRbNuK8/AXnZB8oHcfJVzIi2H9wBt8UZfVURHaQaT62Rp0p4C/L4m922TTUqonVxhqP459OQtqaKSHw0WnVKEQyT35eI0QwrOhkxmhv/kwarmEYHVA9x8O9VuPQWFF0jCcXWbrHCGpM2lR2399+v7t5gQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=IOfXLL+d; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="IOfXLL+d" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4D2F1C4CED7; Tue, 12 Nov 2024 08:26:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1731400017; bh=h4/pxwZqbYL0YDJ6s/BsT2Km5NWKmXNu7R1WBTg/gjw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IOfXLL+dHM1DjsnCIe8PuLDQi5NBdJeMbiMKUjI9udDVHGnieO6R2MKYlRJIGJiP1 6Nto+hblkhXXqe6ecJpGkJPiOTmNpKOWyhUyQGy4al9Jzj0+3NEeM4L3f+eYwznymd aXea7LH9T1t1Z2kjjM4Ljt/KLkcIXJ2N9mLSRWulud39sHn6mgHd/AUgMtS9ifvs/c HmhYXq4pK6b0rCTL3uPsKRnleYwRQK1TTcdGUQ0Qv8iIR90CXb3Qv482vmb45WWRbI 6upjVu1xiIqk9CfVFfsN0HDyqTMVHSR1VpnCgUcxzqMLSSfUpcnDIqKpMGBxzG+oAW QdGc/n9p9KFCQ== From: Song Liu To: bpf@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-security-module@vger.kernel.org Cc: kernel-team@meta.com, andrii@kernel.org, eddyz87@gmail.com, ast@kernel.org, daniel@iogearbox.net, martin.lau@linux.dev, viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, kpsingh@kernel.org, mattbobrowski@google.com, amir73il@gmail.com, repnop@google.com, jlayton@kernel.org, josef@toxicpanda.com, mic@digikod.net, gnoack@google.com, Song Liu Subject: [PATCH bpf-next 4/4] selftest/bpf: Add test for inode local storage recursion Date: Tue, 12 Nov 2024 00:25:59 -0800 Message-ID: <20241112082600.298035-6-song@kernel.org> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241112082600.298035-1-song@kernel.org> References: <20241112082600.298035-1-song@kernel.org> Precedence: bulk X-Mailing-List: bpf@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: bpf@iogearbox.net Add selftest to cover recursion of bpf local storage functions. When inode local storage function is traced, helpers that access inode local storage should return -EBUSY. The recurring program is attached to inode_storage_lookup(). This is not an ideal target for recurrsion tests. However, given that the target function have to take "struct inode *" argument, there isn't a better target function for the tests. Test results showed that inode_storage_lookup() is inlined in s390x. Work around this by adding this test to DENYLIST.s390x. Signed-off-by: Song Liu --- tools/testing/selftests/bpf/DENYLIST.s390x | 1 + .../bpf/prog_tests/inode_local_storage.c | 70 +++++++++++++++ .../bpf/progs/inode_storage_recursion.c | 90 +++++++++++++++++++ 3 files changed, 161 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/inode_local_storage.c create mode 100644 tools/testing/selftests/bpf/progs/inode_storage_recursion.c diff --git a/tools/testing/selftests/bpf/DENYLIST.s390x b/tools/testing/selftests/bpf/DENYLIST.s390x index 3ebd77206f98..6b8c9c9ec754 100644 --- a/tools/testing/selftests/bpf/DENYLIST.s390x +++ b/tools/testing/selftests/bpf/DENYLIST.s390x @@ -1,5 +1,6 @@ # TEMPORARY # Alphabetical order get_stack_raw_tp # user_stack corrupted user stack (no backchain userspace) +inode_localstorage/recursion # target function (inode_storage_lookup) is inlined on s390) stacktrace_build_id # compare_map_keys stackid_hmap vs. stackmap err -2 errno 2 (?) verifier_iterating_callbacks diff --git a/tools/testing/selftests/bpf/prog_tests/inode_local_storage.c b/tools/testing/selftests/bpf/prog_tests/inode_local_storage.c new file mode 100644 index 000000000000..8dc44ebb8431 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/inode_local_storage.c @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ + +#include +#include +#include +#include "inode_storage_recursion.skel.h" + +#define TDIR "/tmp/inode_local_storage" +#define TDIR_PARENT "/tmp" + +static void test_recursion(void) +{ + struct inode_storage_recursion *skel; + struct bpf_prog_info info; + __u32 info_len = sizeof(info); + int err, prog_fd, map_fd, inode_fd = -1; + long value; + + skel = inode_storage_recursion__open_and_load(); + if (!ASSERT_OK_PTR(skel, "skel_open_and_load")) + return; + + skel->bss->test_pid = getpid(); + + err = inode_storage_recursion__attach(skel); + if (!ASSERT_OK(err, "skel_attach")) + goto out; + + err = mkdir(TDIR, 0755); + if (!ASSERT_OK(err, "mkdir " TDIR)) + goto out; + + inode_fd = open(TDIR_PARENT, O_RDONLY | O_CLOEXEC); + if (!ASSERT_OK_FD(inode_fd, "open inode_fd")) + goto out; + + /* Detach so that the following lookup won't trigger + * trace_inode_storage_lookup and further change the values. + */ + inode_storage_recursion__detach(skel); + map_fd = bpf_map__fd(skel->maps.inode_map); + err = bpf_map_lookup_elem(map_fd, &inode_fd, &value); + ASSERT_OK(err, "lookup inode_map"); + ASSERT_EQ(value, 201, "inode_map value"); + ASSERT_EQ(skel->bss->nr_del_errs, 1, "bpf_task_storage_delete busy"); + + prog_fd = bpf_program__fd(skel->progs.trace_inode_mkdir); + memset(&info, 0, sizeof(info)); + err = bpf_prog_get_info_by_fd(prog_fd, &info, &info_len); + ASSERT_OK(err, "get prog info"); + ASSERT_EQ(info.recursion_misses, 0, "trace_inode_mkdir prog recursion"); + + prog_fd = bpf_program__fd(skel->progs.trace_inode_storage_lookup); + memset(&info, 0, sizeof(info)); + err = bpf_prog_get_info_by_fd(prog_fd, &info, &info_len); + ASSERT_OK(err, "get prog info"); + ASSERT_EQ(info.recursion_misses, 3, "trace_inode_storage_lookup prog recursion"); + +out: + rmdir(TDIR); + close(inode_fd); + inode_storage_recursion__destroy(skel); +} + +void test_inode_localstorage(void) +{ + if (test__start_subtest("recursion")) + test_recursion(); +} diff --git a/tools/testing/selftests/bpf/progs/inode_storage_recursion.c b/tools/testing/selftests/bpf/progs/inode_storage_recursion.c new file mode 100644 index 000000000000..18db141f8235 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/inode_storage_recursion.c @@ -0,0 +1,90 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2024 Meta Platforms, Inc. and affiliates. */ + +#include "vmlinux.h" +#include +#include + +#ifndef EBUSY +#define EBUSY 16 +#endif + +char _license[] SEC("license") = "GPL"; +int nr_del_errs; +int test_pid; + +struct { + __uint(type, BPF_MAP_TYPE_INODE_STORAGE); + __uint(map_flags, BPF_F_NO_PREALLOC); + __type(key, int); + __type(value, long); +} inode_map SEC(".maps"); + +/* inode_storage_lookup is not an ideal hook for recursion tests, as it + * is static and more likely to get inlined. However, there isn't a + * better function for the test. This is because we need to call + * bpf_inode_storage_* helpers with an inode intput. Unlike task local + * storage, for which we can use bpf_get_current_task_btf() to get task + * pointer with BTF, for inode local storage, we need the get the inode + * pointer from function arguments. Other functions, such as, + * bpf_local_storage_get() does not take inode as input. + * + * As a compromise, we may need to skip this test for some architectures. + */ +SEC("fentry/inode_storage_lookup") +int BPF_PROG(trace_inode_storage_lookup, struct inode *inode) +{ + struct task_struct *task = bpf_get_current_task_btf(); + long *ptr; + int err; + + if (!test_pid || task->pid != test_pid) + return 0; + + /* This doesn't have BPF_LOCAL_STORAGE_GET_F_CREATE, so it will + * not trigger on the first call of bpf_inode_storage_get() below. + * + * This is called twice, recursion_misses += 2. + */ + ptr = bpf_inode_storage_get(&inode_map, inode, 0, 0); + if (ptr) { + *ptr += 1; + + /* This is called once, recursion_misses += 1. */ + err = bpf_inode_storage_delete(&inode_map, inode); + if (err == -EBUSY) + nr_del_errs++; + } + + return 0; +} + +SEC("fentry/security_inode_mkdir") +int BPF_PROG(trace_inode_mkdir, struct inode *dir, + struct dentry *dentry, + int mode) +{ + struct task_struct *task = bpf_get_current_task_btf(); + long *ptr; + + if (!test_pid || task->pid != test_pid) + return 0; + + /* Trigger trace_inode_storage_lookup, the first time */ + ptr = bpf_inode_storage_get(&inode_map, dir, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + + /* trace_inode_storage_lookup cannot get ptr, so *ptr is 0 */ + if (ptr && !*ptr) + *ptr = 200; + + /* Trigger trace_inode_storage_lookup, the first time. + * trace_inode_storage_lookup can now get ptr and increase the + * value. + */ + bpf_inode_storage_get(&inode_map, dir, 0, + BPF_LOCAL_STORAGE_GET_F_CREATE); + + return 0; + +}