From patchwork Mon Sep 27 14:59:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12520087 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0B1FC433FE for ; Mon, 27 Sep 2021 14:59:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B3A3960F4A for ; Mon, 27 Sep 2021 14:59:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234960AbhI0PBc (ORCPT ); Mon, 27 Sep 2021 11:01:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36244 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234963AbhI0PB3 (ORCPT ); Mon, 27 Sep 2021 11:01:29 -0400 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5DF7C061575; Mon, 27 Sep 2021 07:59:49 -0700 (PDT) Received: by mail-pg1-x544.google.com with SMTP id g184so18058098pgc.6; Mon, 27 Sep 2021 07:59:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UNZa4pX6CrOjmwpsRtuxbm3kwFwAQQHvu29MrD4kvsI=; b=q6Rvtkfv1DsxI1EjbszB3fUgGbGKpiX1PdtQFndX6FmG5BVVSVsf2vwWTNiQDYx1Sg u6ywc/sfS74CEx3+3JGogaEjPfkC59dzXJk2q2skE5MMsMT94C8E1e5BaEvPrcdYce1F QYYzxByumv2FTnbVLavSJ3lLgRLT4W1puNFLjXX9fWa8OoeFKfiCAokD2/FLQxsxKCuL WEcYRYqQJshL80h1ROPfM9OdpNqnP3m3IkNvUp1e+wpQlRS4arK6W3sTe9So4gqsPgYt 7aVV4SOZy8X9c+qF4LBfTAHwUOtskd4RJgOTwipeUaOcC8z9KCgNHCCFcanFVswosQsN ahWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UNZa4pX6CrOjmwpsRtuxbm3kwFwAQQHvu29MrD4kvsI=; b=R+xU4bDpxq9VpynfBbWxoAGuMoPJ3TXwtbOudtJbRhPcPJRCegEOl5N1Jw9/J72esA E6rbGS0qL6RRfXjZ5M6yrs6+rfGe5Wjq/J/dVuw0wxcBG1dN7IfdG83nPFAEnuHCzwO8 796hbYvSYs9WarB15NkxMN34rOb1yXL8KqEFtdT0DHSS0ghVNvL9yFYPDsVdQXFIGihE nX00JmaMotmWUQhcBFMtROiHPI9hbqLEfB5xOy4joQS3C31XN+Vn+vMzBm7Or2ovIx7R xqTmFBr4oEILvSq5xkR2yK973AD17YnZ00zzBBI5leL7IFcUqbLpIqUHXHt6HaXVU3WL iFtw== X-Gm-Message-State: AOAM530jGdC7WzJM1MdjiDOlp7tWYalPYC04enZ1y8QUfg4XGOgd69rM QwQJh6h1Q69xy+kjbCKyUH5dP/YKLJI= X-Google-Smtp-Source: ABdhPJw8mGuY0fE3v5lTm0LXUCbH5OoEQwWFCCYz546ILKXeMu+8P1wVcrqj7/rbyGqVpGHKwBbqqQ== X-Received: by 2002:a63:b552:: with SMTP id u18mr97230pgo.223.1632754789148; Mon, 27 Sep 2021 07:59:49 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id v14sm17597522pfi.111.2021.09.27.07.59.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 07:59:48 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v5 01/12] bpf: Introduce BPF support for kernel module function calls Date: Mon, 27 Sep 2021 20:29:30 +0530 Message-Id: <20210927145941.1383001-2-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210927145941.1383001-1-memxor@gmail.com> References: <20210927145941.1383001-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=17853; h=from:subject; bh=PIEKGBvUcY+R+dN7C9VMbMtrGuMeXglfuoJ5Uhv1+wU=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhUdxOHr4oo+cc/yw1wDLqLxxlh1amT6SDbc7SjzZx JkJpxNaJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYVHcTgAKCRBM4MiGSL8RykRrEA CWdj7Pc+3ZT5QlmqehsifNX2uVznkP3XzaqhfOJxLHAYSUg5318nF6/VCGHPXOdCWpeshyb5xIS1no 95/CKhGtoProW159ZXqSuLzZbZuR8dLpW86NbEOORW8Q3ti0Kp5uD3NadwC44tJi4EzPQJ1d1xkNfE GgY1+T+gyttf+guR9bGkTDLjp2vTfL4tWv3l/k+e9PsPV+rVObucixjsVA3Fia2xWIy/C2frK+AO8Q HM5cUP+pQcPiMRoc0py83IlLW61q3ojQNrP/aM+VPSBkRHJ7TKlsnEvseNQVMwy9eYVgl1OS2pIEtX KmBq5smuU01BytnWalhBkorbk2ae+Wk+NulBpcEdMHoufVT0ZICT3LsmO0kLloTFn9KijEpf+RxbQO cyg/mIJfW6uljJOektsNH2GAZ0ZtjislMQ3lWNxPXPQKLnwqmGxqFYpzW91/xK/nkHkqTztgiULxiQ pot7GAFuEnZhAZAlDKraSbrDOXTFB53REWX2RRE+/7jVFbVoT4L9teQVi3nRBfM3ukF02VbPEImdhB utEhukoT6SC6J3aX0N7ChM5gTBfOjaOeIWJKslLqvN58pTN5WJfzTZlEqv9dEZ4loJCmy3n70HK2kp OUl8XRaA2WfxaIRtQthKSArgcau/GQHcb1XzvWgkJuEg/AnmwNKmn+AphOVA== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This change adds support on the kernel side to allow for BPF programs to call kernel module functions. Userspace will prepare an array of module BTF fds that is passed in during BPF_PROG_LOAD using fd_array parameter. In the kernel, the module BTFs are placed in the auxilliary struct for bpf_prog, and loaded as needed. The verifier then uses insn->off to index into the fd_array. insn->off 0 is reserved for vmlinux BTF (for backwards compat), so userspace must use an fd_array index > 0 for module kfunc support. kfunc_btf_tab is sorted based on offset in an array, and each offset corresponds to one descriptor, with a max limit up to 256 such module BTFs. We also change existing kfunc_tab to distinguish each element based on imm, off pair as each such call will now be distinct. Another change is to check_kfunc_call callback, which now include a struct module * pointer, this is to be used in later patch such that the kfunc_id and module pointer are matched for dynamically registered BTF sets from loadable modules, so that same kfunc_id in two modules doesn't lead to check_kfunc_call succeeding. For the duration of the check_kfunc_call, the reference to struct module exists, as it returns the pointer stored in kfunc_btf_tab. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 8 +- include/linux/bpf_verifier.h | 2 + kernel/bpf/core.c | 4 + kernel/bpf/verifier.c | 202 ++++++++++++++++++++++++++++++----- net/bpf/test_run.c | 2 +- net/ipv4/bpf_tcp_ca.c | 2 +- 6 files changed, 188 insertions(+), 32 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index b6c45a6cbbba..81b1b3d0d546 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -511,7 +511,7 @@ struct bpf_verifier_ops { const struct btf_type *t, int off, int size, enum bpf_access_type atype, u32 *next_btf_id); - bool (*check_kfunc_call)(u32 kfunc_btf_id); + bool (*check_kfunc_call)(u32 kfunc_btf_id, struct module *owner); }; struct bpf_prog_offload_ops { @@ -874,6 +874,7 @@ struct bpf_prog_aux { void *jit_data; /* JIT specific data. arch dependent */ struct bpf_jit_poke_descriptor *poke_tab; struct bpf_kfunc_desc_tab *kfunc_tab; + struct bpf_kfunc_btf_tab *kfunc_btf_tab; u32 size_poke_tab; struct bpf_ksym ksym; const struct bpf_prog_ops *ops; @@ -1636,7 +1637,7 @@ int bpf_prog_test_run_raw_tp(struct bpf_prog *prog, int bpf_prog_test_run_sk_lookup(struct bpf_prog *prog, const union bpf_attr *kattr, union bpf_attr __user *uattr); -bool bpf_prog_test_check_kfunc_call(u32 kfunc_id); +bool bpf_prog_test_check_kfunc_call(u32 kfunc_id, struct module *owner); bool btf_ctx_access(int off, int size, enum bpf_access_type type, const struct bpf_prog *prog, struct bpf_insn_access_aux *info); @@ -1857,7 +1858,8 @@ static inline int bpf_prog_test_run_sk_lookup(struct bpf_prog *prog, return -ENOTSUPP; } -static inline bool bpf_prog_test_check_kfunc_call(u32 kfunc_id) +static inline bool bpf_prog_test_check_kfunc_call(u32 kfunc_id, + struct module *owner) { return false; } diff --git a/include/linux/bpf_verifier.h b/include/linux/bpf_verifier.h index 5424124dbe36..c8a78e830fca 100644 --- a/include/linux/bpf_verifier.h +++ b/include/linux/bpf_verifier.h @@ -527,5 +527,7 @@ int bpf_check_attach_target(struct bpf_verifier_log *log, const struct bpf_prog *tgt_prog, u32 btf_id, struct bpf_attach_target_info *tgt_info); +void bpf_free_kfunc_btf_tab(struct bpf_kfunc_btf_tab *tab); + #endif /* _LINUX_BPF_VERIFIER_H */ diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 6fddc13fe67f..084df01625cd 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -32,6 +32,7 @@ #include #include #include +#include #include #include @@ -2255,6 +2256,9 @@ static void bpf_prog_free_deferred(struct work_struct *work) int i; aux = container_of(work, struct bpf_prog_aux, work); +#ifdef CONFIG_BPF_SYSCALL + bpf_free_kfunc_btf_tab(aux->kfunc_btf_tab); +#endif bpf_free_used_maps(aux); bpf_free_used_btfs(aux); if (bpf_prog_is_dev_bound(aux)) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index e76b55917905..5b465d67cf65 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1626,52 +1626,173 @@ static int add_subprog(struct bpf_verifier_env *env, int off) return env->subprog_cnt - 1; } +#define MAX_KFUNC_DESCS 256 +#define MAX_KFUNC_BTFS 256 + struct bpf_kfunc_desc { struct btf_func_model func_model; u32 func_id; s32 imm; + u16 offset; +}; + +struct bpf_kfunc_btf { + struct btf *btf; + struct module *module; + u16 offset; }; -#define MAX_KFUNC_DESCS 256 struct bpf_kfunc_desc_tab { struct bpf_kfunc_desc descs[MAX_KFUNC_DESCS]; u32 nr_descs; }; -static int kfunc_desc_cmp_by_id(const void *a, const void *b) +struct bpf_kfunc_btf_tab { + struct bpf_kfunc_btf descs[MAX_KFUNC_BTFS]; + u32 nr_descs; +}; + +static int kfunc_desc_cmp_by_id_off(const void *a, const void *b) { const struct bpf_kfunc_desc *d0 = a; const struct bpf_kfunc_desc *d1 = b; /* func_id is not greater than BTF_MAX_TYPE */ - return d0->func_id - d1->func_id; + return d0->func_id - d1->func_id ?: d0->offset - d1->offset; +} + +static int kfunc_btf_cmp_by_off(const void *a, const void *b) +{ + const struct bpf_kfunc_btf *d0 = a; + const struct bpf_kfunc_btf *d1 = b; + + return d0->offset - d1->offset; } static const struct bpf_kfunc_desc * -find_kfunc_desc(const struct bpf_prog *prog, u32 func_id) +find_kfunc_desc(const struct bpf_prog *prog, u32 func_id, u16 offset) { struct bpf_kfunc_desc desc = { .func_id = func_id, + .offset = offset, }; struct bpf_kfunc_desc_tab *tab; tab = prog->aux->kfunc_tab; return bsearch(&desc, tab->descs, tab->nr_descs, - sizeof(tab->descs[0]), kfunc_desc_cmp_by_id); + sizeof(tab->descs[0]), kfunc_desc_cmp_by_id_off); +} + +static struct btf *__find_kfunc_desc_btf(struct bpf_verifier_env *env, + s16 offset, struct module **btf_modp) +{ + struct bpf_kfunc_btf kf_btf = { .offset = offset }; + struct bpf_kfunc_btf_tab *tab; + struct bpf_kfunc_btf *b; + struct module *mod; + struct btf *btf; + int btf_fd; + + tab = env->prog->aux->kfunc_btf_tab; + b = bsearch(&kf_btf, tab->descs, tab->nr_descs, + sizeof(tab->descs[0]), kfunc_btf_cmp_by_off); + if (!b) { + if (tab->nr_descs == MAX_KFUNC_BTFS) { + verbose(env, "too many different module BTFs\n"); + return ERR_PTR(-E2BIG); + } + + if (bpfptr_is_null(env->fd_array)) { + verbose(env, "kfunc offset > 0 without fd_array is invalid\n"); + return ERR_PTR(-EPROTO); + } + + if (copy_from_bpfptr_offset(&btf_fd, env->fd_array, + offset * sizeof(btf_fd), + sizeof(btf_fd))) + return ERR_PTR(-EFAULT); + + btf = btf_get_by_fd(btf_fd); + if (IS_ERR(btf)) + return btf; + + if (!btf_is_module(btf)) { + verbose(env, "BTF fd for kfunc is not a module BTF\n"); + btf_put(btf); + return ERR_PTR(-EINVAL); + } + + mod = btf_try_get_module(btf); + if (!mod) { + btf_put(btf); + return ERR_PTR(-ENXIO); + } + + b = &tab->descs[tab->nr_descs++]; + b->btf = btf; + b->module = mod; + b->offset = offset; + + sort(tab->descs, tab->nr_descs, sizeof(tab->descs[0]), + kfunc_btf_cmp_by_off, NULL); + } + if (btf_modp) + *btf_modp = b->module; + return b->btf; } -static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) +void bpf_free_kfunc_btf_tab(struct bpf_kfunc_btf_tab *tab) +{ + if (!tab) + return; + + while (tab->nr_descs--) { + module_put(tab->descs[tab->nr_descs].module); + btf_put(tab->descs[tab->nr_descs].btf); + } + kfree(tab); +} + +static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, + u32 func_id, s16 offset, + struct module **btf_modp) +{ + struct btf *kfunc_btf; + + if (offset) { + if (offset < 0) { + /* In the future, this can be allowed to increase limit + * of fd index into fd_array, interpreted as u16. + */ + verbose(env, "negative offset disallowed for kernel module function call\n"); + return ERR_PTR(-EINVAL); + } + + kfunc_btf = __find_kfunc_desc_btf(env, offset, btf_modp); + if (IS_ERR_OR_NULL(kfunc_btf)) { + verbose(env, "cannot find module BTF for func_id %u\n", func_id); + return kfunc_btf ?: ERR_PTR(-ENOENT); + } + return kfunc_btf; + } + return btf_vmlinux ?: ERR_PTR(-ENOENT); +} + +static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) { const struct btf_type *func, *func_proto; + struct bpf_kfunc_btf_tab *btf_tab; struct bpf_kfunc_desc_tab *tab; struct bpf_prog_aux *prog_aux; struct bpf_kfunc_desc *desc; const char *func_name; + struct btf *desc_btf; unsigned long addr; int err; prog_aux = env->prog->aux; tab = prog_aux->kfunc_tab; + btf_tab = prog_aux->kfunc_btf_tab; if (!tab) { if (!btf_vmlinux) { verbose(env, "calling kernel function is not supported without CONFIG_DEBUG_INFO_BTF\n"); @@ -1699,7 +1820,20 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) prog_aux->kfunc_tab = tab; } - if (find_kfunc_desc(env->prog, func_id)) + if (!btf_tab && offset) { + btf_tab = kzalloc(sizeof(*btf_tab), GFP_KERNEL); + if (!btf_tab) + return -ENOMEM; + prog_aux->kfunc_btf_tab = btf_tab; + } + + desc_btf = find_kfunc_desc_btf(env, func_id, offset, NULL); + if (IS_ERR(desc_btf)) { + verbose(env, "failed to find BTF for kernel function\n"); + return PTR_ERR(desc_btf); + } + + if (find_kfunc_desc(env->prog, func_id, offset)) return 0; if (tab->nr_descs == MAX_KFUNC_DESCS) { @@ -1707,20 +1841,20 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) return -E2BIG; } - func = btf_type_by_id(btf_vmlinux, func_id); + func = btf_type_by_id(desc_btf, func_id); if (!func || !btf_type_is_func(func)) { verbose(env, "kernel btf_id %u is not a function\n", func_id); return -EINVAL; } - func_proto = btf_type_by_id(btf_vmlinux, func->type); + func_proto = btf_type_by_id(desc_btf, func->type); if (!func_proto || !btf_type_is_func_proto(func_proto)) { verbose(env, "kernel function btf_id %u does not have a valid func_proto\n", func_id); return -EINVAL; } - func_name = btf_name_by_offset(btf_vmlinux, func->name_off); + func_name = btf_name_by_offset(desc_btf, func->name_off); addr = kallsyms_lookup_name(func_name); if (!addr) { verbose(env, "cannot find address for kernel function %s\n", @@ -1731,12 +1865,13 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) desc = &tab->descs[tab->nr_descs++]; desc->func_id = func_id; desc->imm = BPF_CAST_CALL(addr) - __bpf_call_base; - err = btf_distill_func_proto(&env->log, btf_vmlinux, + desc->offset = offset; + err = btf_distill_func_proto(&env->log, desc_btf, func_proto, func_name, &desc->func_model); if (!err) sort(tab->descs, tab->nr_descs, sizeof(tab->descs[0]), - kfunc_desc_cmp_by_id, NULL); + kfunc_desc_cmp_by_id_off, NULL); return err; } @@ -1815,7 +1950,7 @@ static int add_subprog_and_kfunc(struct bpf_verifier_env *env) } else if (bpf_pseudo_call(insn)) { ret = add_subprog(env, i + insn->imm + 1); } else { - ret = add_kfunc_call(env, insn->imm); + ret = add_kfunc_call(env, insn->imm, insn->off); } if (ret < 0) @@ -2152,12 +2287,17 @@ static int get_prev_insn_idx(struct bpf_verifier_state *st, int i, static const char *disasm_kfunc_name(void *data, const struct bpf_insn *insn) { const struct btf_type *func; + struct btf *desc_btf; if (insn->src_reg != BPF_PSEUDO_KFUNC_CALL) return NULL; - func = btf_type_by_id(btf_vmlinux, insn->imm); - return btf_name_by_offset(btf_vmlinux, func->name_off); + desc_btf = find_kfunc_desc_btf(data, insn->imm, insn->off, NULL); + if (IS_ERR(desc_btf)) + return ""; + + func = btf_type_by_id(desc_btf, insn->imm); + return btf_name_by_offset(desc_btf, func->name_off); } /* For given verifier state backtrack_insn() is called from the last insn to @@ -6485,23 +6625,29 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) struct bpf_reg_state *regs = cur_regs(env); const char *func_name, *ptr_type_name; u32 i, nargs, func_id, ptr_type_id; + struct module *btf_mod = NULL; const struct btf_param *args; + struct btf *desc_btf; int err; + desc_btf = find_kfunc_desc_btf(env, insn->imm, insn->off, &btf_mod); + if (IS_ERR(desc_btf)) + return PTR_ERR(desc_btf); + func_id = insn->imm; - func = btf_type_by_id(btf_vmlinux, func_id); - func_name = btf_name_by_offset(btf_vmlinux, func->name_off); - func_proto = btf_type_by_id(btf_vmlinux, func->type); + func = btf_type_by_id(desc_btf, func_id); + func_name = btf_name_by_offset(desc_btf, func->name_off); + func_proto = btf_type_by_id(desc_btf, func->type); if (!env->ops->check_kfunc_call || - !env->ops->check_kfunc_call(func_id)) { + !env->ops->check_kfunc_call(func_id, btf_mod)) { verbose(env, "calling kernel function %s is not allowed\n", func_name); return -EACCES; } /* Check the arguments */ - err = btf_check_kfunc_arg_match(env, btf_vmlinux, func_id, regs); + err = btf_check_kfunc_arg_match(env, desc_btf, func_id, regs); if (err) return err; @@ -6509,15 +6655,15 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) mark_reg_not_init(env, regs, caller_saved[i]); /* Check return type */ - t = btf_type_skip_modifiers(btf_vmlinux, func_proto->type, NULL); + t = btf_type_skip_modifiers(desc_btf, func_proto->type, NULL); if (btf_type_is_scalar(t)) { mark_reg_unknown(env, regs, BPF_REG_0); mark_btf_func_reg_size(env, BPF_REG_0, t->size); } else if (btf_type_is_ptr(t)) { - ptr_type = btf_type_skip_modifiers(btf_vmlinux, t->type, + ptr_type = btf_type_skip_modifiers(desc_btf, t->type, &ptr_type_id); if (!btf_type_is_struct(ptr_type)) { - ptr_type_name = btf_name_by_offset(btf_vmlinux, + ptr_type_name = btf_name_by_offset(desc_btf, ptr_type->name_off); verbose(env, "kernel function %s returns pointer type %s %s is not supported\n", func_name, btf_type_str(ptr_type), @@ -6525,7 +6671,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) return -EINVAL; } mark_reg_known_zero(env, regs, BPF_REG_0); - regs[BPF_REG_0].btf = btf_vmlinux; + regs[BPF_REG_0].btf = desc_btf; regs[BPF_REG_0].type = PTR_TO_BTF_ID; regs[BPF_REG_0].btf_id = ptr_type_id; mark_btf_func_reg_size(env, BPF_REG_0, sizeof(void *)); @@ -6536,7 +6682,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) for (i = 0; i < nargs; i++) { u32 regno = i + 1; - t = btf_type_skip_modifiers(btf_vmlinux, args[i].type, NULL); + t = btf_type_skip_modifiers(desc_btf, args[i].type, NULL); if (btf_type_is_ptr(t)) mark_btf_func_reg_size(env, regno, sizeof(void *)); else @@ -11076,7 +11222,8 @@ static int do_check(struct bpf_verifier_env *env) env->jmps_processed++; if (opcode == BPF_CALL) { if (BPF_SRC(insn->code) != BPF_K || - insn->off != 0 || + (insn->src_reg != BPF_PSEUDO_KFUNC_CALL + && insn->off != 0) || (insn->src_reg != BPF_REG_0 && insn->src_reg != BPF_PSEUDO_CALL && insn->src_reg != BPF_PSEUDO_KFUNC_CALL) || @@ -12432,6 +12579,7 @@ static int jit_subprogs(struct bpf_verifier_env *env) func[i]->aux->stack_depth = env->subprog_info[i].stack_depth; func[i]->jit_requested = 1; func[i]->aux->kfunc_tab = prog->aux->kfunc_tab; + func[i]->aux->kfunc_btf_tab = prog->aux->kfunc_btf_tab; func[i]->aux->linfo = prog->aux->linfo; func[i]->aux->nr_linfo = prog->aux->nr_linfo; func[i]->aux->jited_linfo = prog->aux->jited_linfo; @@ -12621,7 +12769,7 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, /* insn->imm has the btf func_id. Replace it with * an address (relative to __bpf_base_call). */ - desc = find_kfunc_desc(env->prog, insn->imm); + desc = find_kfunc_desc(env->prog, insn->imm, insn->off); if (!desc) { verbose(env, "verifier internal error: kernel function descriptor not found for func_id %u\n", insn->imm); diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c index fcb2f493f710..fe5c34f414a2 100644 --- a/net/bpf/test_run.c +++ b/net/bpf/test_run.c @@ -241,7 +241,7 @@ BTF_ID(func, bpf_kfunc_call_test2) BTF_ID(func, bpf_kfunc_call_test3) BTF_SET_END(test_sk_kfunc_ids) -bool bpf_prog_test_check_kfunc_call(u32 kfunc_id) +bool bpf_prog_test_check_kfunc_call(u32 kfunc_id, struct module *owner) { return btf_id_set_contains(&test_sk_kfunc_ids, kfunc_id); } diff --git a/net/ipv4/bpf_tcp_ca.c b/net/ipv4/bpf_tcp_ca.c index 0dcee9df1326..b3afd3361f34 100644 --- a/net/ipv4/bpf_tcp_ca.c +++ b/net/ipv4/bpf_tcp_ca.c @@ -255,7 +255,7 @@ BTF_ID(func, bbr_set_state) #endif /* CONFIG_X86 */ BTF_SET_END(bpf_tcp_ca_kfunc_ids) -static bool bpf_tcp_ca_check_kfunc_call(u32 kfunc_btf_id) +static bool bpf_tcp_ca_check_kfunc_call(u32 kfunc_btf_id, struct module *owner) { return btf_id_set_contains(&bpf_tcp_ca_kfunc_ids, kfunc_btf_id); } From patchwork Mon Sep 27 14:59:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12520089 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24202C433EF for ; Mon, 27 Sep 2021 14:59:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0FDA160EE2 for ; Mon, 27 Sep 2021 14:59:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234968AbhI0PBd (ORCPT ); Mon, 27 Sep 2021 11:01:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36264 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234833AbhI0PBb (ORCPT ); Mon, 27 Sep 2021 11:01:31 -0400 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA36DC061769; Mon, 27 Sep 2021 07:59:52 -0700 (PDT) Received: by mail-pg1-x544.google.com with SMTP id 75so1675269pga.3; Mon, 27 Sep 2021 07:59:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YC/5g23SPpiZ7A5VfOJx+Ff2nYKoj2UfUfHHpa2IJDY=; b=YlW+cCsID+sm/E9I595RTis9Dyr9U/qkj7DwYq4bUEj0tJAz8cY4SHwJlbjm3VDda4 NTHbAcbER6Qtba6xM3kzPg79PYtVePhEn7zkheSPYJ+o7DSMw/EKyJONn/vi7RS0gOsU rWnWH89blEKxHIYvm9NonJCvUahF9mNJ9tatGIsMoJ6pkkXQeylVVpOojoGL3fwaKWkD e/00oNqlNqcaT3ycueM35ZxDTgxzAyidl00kDKaO4k3tsSyeqwq1lvm/Qvdo5Yl7s74b YuNGFni9nxpsDPcMySibOdSiH/PKcBOd2Xwh4DerCLiDoFn7QFWOhqHMMYyI5lsnJqsp LBPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YC/5g23SPpiZ7A5VfOJx+Ff2nYKoj2UfUfHHpa2IJDY=; b=Z0p/2JxD83nIHgeKqcBtxXrvfNMWm+eOOKTvzy+4EZwrgH9ythZfCqjLtY+HTG+/T/ y0Cp5Mmqm/Xou/9vBfFWtqq7Owo2ltfqZwnYIxCNNHFoGYC+aeQFzOmYbFVbQB8r0Xl4 RH9KNdbtOuNZ8fBrsToXBwr376SxbBCyO3U+BsqGK2O8Hl25gvcNdbLVoY55jgi3i9tG j691dSgANnIDTl3q5dMVRaNSn5nFCUdyHr+pYgn2Sgj+m1S5HeALpQ98x1q6oE0wzAL5 OeSElxUotbLGG41si/a5kHiB7YNjtEa5iTYAIjbv6/tibCs+AA3W0MDf4bPJdoWS4Tq4 Tgiw== X-Gm-Message-State: AOAM531yR4kcOgE2OXUPMAxvtWtLWZ0ZNT0xk+Ne9fFCw10rRKhiN04T DVQczzeFrGKhs28xaGSDX4xYmSlciqg= X-Google-Smtp-Source: ABdhPJxDVlyE5OpEzOeh0phQc3G4ubss858xwIQYonUw079dBQLBRg9mmzVOSkEoGyI8bqVIJlcbqg== X-Received: by 2002:a65:6658:: with SMTP id z24mr134892pgv.266.1632754792171; Mon, 27 Sep 2021 07:59:52 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id mt17sm16244745pjb.38.2021.09.27.07.59.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 07:59:51 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v5 02/12] bpf: Be conservative while processing invalid kfunc calls Date: Mon, 27 Sep 2021 20:29:31 +0530 Message-Id: <20210927145941.1383001-3-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210927145941.1383001-1-memxor@gmail.com> References: <20210927145941.1383001-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2214; h=from:subject; bh=FKRyNZlxBXZubpWKuL4SYjb1veTjvmV2mok2OrudHXU=; b=owEBbAKT/ZANAwAIAUzgyIZIvxHKAcsmYgBhUdxOvRD+YlTis4ZpxqZMjImP98Z2FoCqLKhYqPfs a52iZuSJAjIEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYVHcTgAKCRBM4MiGSL8Ryn3fD/ jUltPQyOQmSBE2DmxdqSSaXYt41MY+KG12tZNCADyi7SNqzKvpmarEUVp+SX1ddUAfjSDyPKRe8j2U GMhjw2a2npffhD+fFf5wxZSpA3s53J76Egc61rHTY48YCb4hT/qLc3nnE+5cZFGk4WrJgwOAM01eFJ aNQic5d20p0Kni1RKk/krxsRKxdS9IhTz9IQ76Ufm+/y3nh2k/gWPicyOyjGFAv5mF07ag8cvOO6Fu oL8Iawvb6zgvuwtE553e8r9N/Cn1a+4p1j6yWd8bSJlD5GZH5i2YoqJo84jBXkneA5TAOw0VVm6X7Z fGb1OkuXphAM8HMSdDntQfCi50JWzxXy22cu020otE7DUwa8Wb6Pg8JOWB/iW1LkfQUpzt2ycmiktv qAnlAk2JXW8fxXSNhrfzhp/YKFiV5suwEJZMl7oHUugAIF0TWjkPRlsNdC5NEoNBGlT8BtksZK/rAb f2OaTHw8BXJ1DzlWu+pqkn3rTHlOiUJk0TRjX7dTUeIRA20sNCvAABVH6Pnl7f89F2aS3iciuUlaL7 REInb7Qnw+b1TBwibrOL69Dx8GkqQzh2/XGeGTBdGIhQa6wK1noHXOQxGqY+7jFw63u1r6p+DU4zQD GxfxsMuIyRwhInr4eHzCVQ0MqKOO59ShhFVxslknIC03kic9xxcUy/2c3b X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This patch also modifies the BPF verifier to only return error for invalid kfunc calls specially marked by userspace (with insn->imm == 0, insn->off == 0) after the verifier has eliminated dead instructions. This can be handled in the fixup stage, and skip processing during add and check stages. If such an invalid call is dropped, the fixup stage will not encounter insn->imm as 0, otherwise it bails out and returns an error. This will be exposed as weak ksym support in libbpf in later patches. Signed-off-by: Kumar Kartikeya Dwivedi --- kernel/bpf/verifier.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 5b465d67cf65..58c86670dc55 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1820,6 +1820,15 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) prog_aux->kfunc_tab = tab; } + /* func_id == 0 is always invalid, but instead of returning an error, be + * conservative and wait until the code elimination pass before returning + * error, so that invalid calls that get pruned out can be in BPF programs + * loaded from userspace. It is also required that offset be untouched + * for such calls. + */ + if (!func_id && !offset) + return 0; + if (!btf_tab && offset) { btf_tab = kzalloc(sizeof(*btf_tab), GFP_KERNEL); if (!btf_tab) @@ -6630,6 +6639,10 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) struct btf *desc_btf; int err; + /* skip for now, but return error when we find this in fixup_kfunc_call */ + if (!insn->imm) + return 0; + desc_btf = find_kfunc_desc_btf(env, insn->imm, insn->off, &btf_mod); if (IS_ERR(desc_btf)) return PTR_ERR(desc_btf); @@ -12766,6 +12779,11 @@ static int fixup_kfunc_call(struct bpf_verifier_env *env, { const struct bpf_kfunc_desc *desc; + if (!insn->imm) { + verbose(env, "invalid kernel function call not eliminated in verifier pass\n"); + return -EINVAL; + } + /* insn->imm has the btf func_id. Replace it with * an address (relative to __bpf_base_call). */ From patchwork Mon Sep 27 14:59:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12520091 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C540BC4332F for ; Mon, 27 Sep 2021 15:00:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B188160EE2 for ; Mon, 27 Sep 2021 15:00:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234833AbhI0PBf (ORCPT ); Mon, 27 Sep 2021 11:01:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36278 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234946AbhI0PBd (ORCPT ); Mon, 27 Sep 2021 11:01:33 -0400 Received: from mail-pl1-x641.google.com (mail-pl1-x641.google.com [IPv6:2607:f8b0:4864:20::641]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C2660C061575; Mon, 27 Sep 2021 07:59:55 -0700 (PDT) Received: by mail-pl1-x641.google.com with SMTP id n2so11938776plk.12; Mon, 27 Sep 2021 07:59:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SmxgKFkRGgoEqMaY9cRCsw185bTTD612fO7JwVu7FH4=; b=VA9pGzY0DZzQWDCEaO9RV3nzCNCQmqT4UeUMb5vwXtUYFzwe/K9EoWaV7NaUkJBcR1 qcS3Ql1Xhk7+wxc2BixmM2THku/82RwqJpNO9Df99/YsNy3BRrLufO113RssGXnccd8l s6rBYGiRIoilQyMT2/LIrCXfdsq2CXkX+8rvJPVR0IABaatpayEB9E3TRF4XoFMs9UaB UlfkLqVnmZ+4sTa/5/vNmlkMNeZIUU+cyCxTTMTJGC8BOblWoqDOZFf4F7s6NNb0cn4j gbKnhuoEujkfdJX+CMCJPXU+QwHxrKEWETvGdP4BpoENZ6cC0KQB3MKX0LTNkqXMs5lF hmDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SmxgKFkRGgoEqMaY9cRCsw185bTTD612fO7JwVu7FH4=; b=DSCPmNHUwaL1MGAHn+Na+QDA43ng2t6qk9TG1rZ0PCaaZpmYT5RqnMcxUQ9Hu161Gy qnLbnzo7Hw4lkXLn3Mu9LJVzd6sN32VjWbEclpbe91/LHdPUu70ec3Ubvi204KOu2rhz sh3C3wfLW2oDoJ4sx96WaIHQspPJDR7nZNMH9FH1vSwo1SFwgfTcvPecWtELoRuP6pTS HPZEJb8OZPCpAdxQSG53UkpFTquyLtwltjdWNZBPrTLdK2U7YQwhrszcqRJtDfvayrnP /UlZktWhJOGbrqNFbq6wejR+4YvFiHAtTnZcTSBz6U8eZ1Wfb7trfAjvITQ1rRl/NEhF 1wZw== X-Gm-Message-State: AOAM5316dGuZCAO56geHgu8ArOPSMdhTqoDZqw87aR6GdGumARSLPZsI TklQhcDVbCQAC77/unsOCWwDA1LwZGE= X-Google-Smtp-Source: ABdhPJzDVdiEcWtO5KkQjafkHz6RzL2hqGNX61ORC3ZPJZ0/uRuZN0cAh2BY+/yKrfbsNu0KS9OqRQ== X-Received: by 2002:a17:90a:6708:: with SMTP id n8mr20185101pjj.219.1632754795121; Mon, 27 Sep 2021 07:59:55 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id 77sm17224443pfu.219.2021.09.27.07.59.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 07:59:54 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v5 03/12] bpf: btf: Introduce helpers for dynamic BTF set registration Date: Mon, 27 Sep 2021 20:29:32 +0530 Message-Id: <20210927145941.1383001-4-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210927145941.1383001-1-memxor@gmail.com> References: <20210927145941.1383001-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4928; h=from:subject; bh=Z49+ZfOJfNIcBx7xGCBn4+PTlYIzy90X8Bk3C1i1vSo=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhUdxOfrZyWFi8bHU8zOBS4ia5K3qEpJqBiICjwsXY NqOXh4SJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYVHcTgAKCRBM4MiGSL8RygSYD/ oCcHLqCOnw9Jn0rR+yajvlJiOXz09bELbVSG6cPjUykyJCh+ats9huCNKk2tqr6GB4BTBuz6tskaj0 I7233A+iWV1aTwYt8xCA1Ynpj25Ywf8yo+PCWp2XlaZevGWScX3XoUj7BFvRBkH6TbDnAoOOoYTUdt otHuJhplVbmy/37YirzCo3rZyVa2YokxehDvWzgTrN6O5TexJkAxRGBza9yDfIqGUJIx0oOTMAMTj+ Zmx9LeU6UNgcg4fjV18nRf36Bu/xdppJwrsuGVEpLH8bVmQM06KqGm2AIKDLh0jH7eb2Uhb4KQ4h0J J5e+Ep2ZmedAfA13QN/jkNVh/TNZb94sVNahRfYkOUiFVY6doKKHACpH4Ntfy1eKe2pwxgXgjqgpVy KJWdN3B9h6RtvknScOFgI1gHoO/vjTJviv/H6ZrdPZsq/anqf+6WQ9Y2eN6YI1+/d46L0iHUp5G/WK qavbkdGqIZGoz0vvohNevNnjvsM11GjUHS4ZmPKyGQ8qc29J9muJZt6sl6A2a6q05ephHtjvrKoVQk BIjV32qpMa9hNzujgakAC1y0VWUZIzKn1q7EGTIWtz8aNCsO5VUXha0q5VpgnAGmxRLG+hqTpLzK3q pf+oDRWZO6JOA5WLGBe67iTMDBSnZX+iqqEB9h+iOGINuEGku4e0VrBUsmCw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This adds helpers for registering btf_id_set from modules and the check_kfunc_call callback that can be used to look them up. With in kernel sets, the way this is supposed to work is, in kernel callback looks up within the in-kernel kfunc whitelist, and then defers to the dynamic BTF set lookup if it doesn't find the BTF id. If there is no in-kernel BTF id set, this callback can be used directly. Also fix includes for btf.h and bpfptr.h so that they can included in isolation. This is in preparation for their usage in tcp_bbr, tcp_cubic and tcp_dctcp modules in the next patch. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpfptr.h | 1 + include/linux/btf.h | 31 +++++++++++++++++++++++++ kernel/bpf/btf.c | 51 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 83 insertions(+) diff --git a/include/linux/bpfptr.h b/include/linux/bpfptr.h index 546e27fc6d46..46e1757d06a3 100644 --- a/include/linux/bpfptr.h +++ b/include/linux/bpfptr.h @@ -3,6 +3,7 @@ #ifndef _LINUX_BPFPTR_H #define _LINUX_BPFPTR_H +#include #include typedef sockptr_t bpfptr_t; diff --git a/include/linux/btf.h b/include/linux/btf.h index 214fde93214b..382c00d5cede 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -5,6 +5,7 @@ #define _LINUX_BTF_H 1 #include +#include #include #include @@ -238,4 +239,34 @@ static inline const char *btf_name_by_offset(const struct btf *btf, } #endif +struct kfunc_btf_id_set { + struct list_head list; + struct btf_id_set *set; + struct module *owner; +}; + +struct kfunc_btf_id_list; + +#ifdef CONFIG_DEBUG_INFO_BTF_MODULES +void register_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s); +void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s); +#else +static inline void register_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s) +{ +} +static inline void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s) +{ +} +#endif + +#define DECLARE_CHECK_KFUNC_CALLBACK(type) \ + bool __bpf_##type##_check_kfunc_call(u32 kfunc_id, struct module *owner) +#define DEFINE_KFUNC_BTF_ID_SET(set, name) \ + struct kfunc_btf_id_set name = { LIST_HEAD_INIT(name.list), (set), \ + THIS_MODULE } + #endif diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index c3d605b22473..5a8806cfecd0 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6343,3 +6343,54 @@ const struct bpf_func_proto bpf_btf_find_by_name_kind_proto = { }; BTF_ID_LIST_GLOBAL_SINGLE(btf_task_struct_ids, struct, task_struct) + +struct kfunc_btf_id_list { + struct list_head list; + struct mutex mutex; +}; + +#ifdef CONFIG_DEBUG_INFO_BTF_MODULES + +void register_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s) +{ + mutex_lock(&l->mutex); + list_add(&s->list, &l->list); + mutex_unlock(&l->mutex); +} +EXPORT_SYMBOL_GPL(register_kfunc_btf_id_set); + +void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, + struct kfunc_btf_id_set *s) +{ + mutex_lock(&l->mutex); + list_del_init(&s->list); + mutex_unlock(&l->mutex); +} +EXPORT_SYMBOL_GPL(unregister_kfunc_btf_id_set); + +#endif + +#define DEFINE_KFUNC_BTF_ID_LIST(name) \ + struct kfunc_btf_id_list name = { LIST_HEAD_INIT(name.list), \ + __MUTEX_INITIALIZER(name.mutex) }; \ + EXPORT_SYMBOL_GPL(name) + +#define DEFINE_CHECK_KFUNC_CALLBACK(type, list_name) \ + bool __bpf_##type##_check_kfunc_call(u32 kfunc_id, \ + struct module *owner) \ + { \ + struct kfunc_btf_id_set *s; \ + if (!owner || !IS_ENABLED(CONFIG_DEBUG_INFO_BTF_MODULES)) \ + return false; \ + mutex_lock(&list_name.mutex); \ + list_for_each_entry(s, &list_name.list, list) { \ + if (s->owner == owner && \ + btf_id_set_contains(s->set, kfunc_id)) { \ + mutex_unlock(&list_name.mutex); \ + return true; \ + } \ + } \ + mutex_unlock(&list_name.mutex); \ + return false; \ + } From patchwork Mon Sep 27 14:59:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12520093 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8A2BC433EF for ; Mon, 27 Sep 2021 15:00:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A905160EE2 for ; Mon, 27 Sep 2021 15:00:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234946AbhI0PBk (ORCPT ); Mon, 27 Sep 2021 11:01:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36310 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234995AbhI0PBi (ORCPT ); Mon, 27 Sep 2021 11:01:38 -0400 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6209C061769; Mon, 27 Sep 2021 07:59:58 -0700 (PDT) Received: by mail-pj1-x1043.google.com with SMTP id k23so12668708pji.0; Mon, 27 Sep 2021 07:59:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=J0qwPwLHdv9Ry0hnhF4U+31mLrql64Mi3Dl7eMKHw3Q=; b=B6rVpfVLWdS7tGXJ1b2FjbSR4R1W7k8i6hfNfELR93xXd73/8liTFQSas09bmG+A1e yJVW9qss01ulEb9XzprpHbdEOr56H6klQhw2qOvTTX9mrp/B/R2+oAJ4EcjgRG9NmACf al6A1VM6fDPybJI4TZttZHkI9UpipGc/awzCjSAFMwrThaGV9zikHokk8R0H7RAubLui JqVVCyf9+ppx+40cQ1yRs94FQvzaN2tWtnl4+/9ifamdOVFYogZG11EmZj9nu82eucun q/9AuhXNJIlodbqiOA2QqtrGaWXN3ed+CGsgB6Mdhfq2LoWuiJ0J4x0Z+qhqQlEF1b9Y fAGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=J0qwPwLHdv9Ry0hnhF4U+31mLrql64Mi3Dl7eMKHw3Q=; b=rTakyvj1uL7efzI8pEtiFxneUm8/03OYsviIoRb5+OXQNm/fo8owWGpwzs1IOme9r7 iw8TZqrs+9Xvj1Fcd0MuwBvdmhNyPzTQoKrqWkDJ8yWUhOg7LMR+pVWeGPsjFy4v17a3 +Nu942JcEIIBR7VYIBPS7I2PLU5y/XKqkRWksWgMeMI1rj6wqaAYc2y3uxjteQNbQJBz DoIEHldixnJNrqmBC+GfN7n2dFIniqzMS5UpHXxO3kAQucacND8cDBGouOpUGssWQuNF PYhjaA4ysfZ4IAwgdVuvJSFjcs1821l2S2P3QiNlhBoEPL/4RYDqJeQi9oaUxV1Jgugf qv0A== X-Gm-Message-State: AOAM532SxUWkznJyJAS6hy60aFktb/i8lL43GZXUfdCI6Q7//v2daYt4 IKvZTUfVWm46jyaPodVQpDaKAfjGL5w= X-Google-Smtp-Source: ABdhPJxPeAxuJg+A8pMGqTHPjEakf+JbP0wqbz7NWgXV+VM3n0PDOLQdIAjUMyD/8WF96+S/XvjgtA== X-Received: by 2002:a17:90a:19d2:: with SMTP id 18mr378179pjj.217.1632754798072; Mon, 27 Sep 2021 07:59:58 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id j24sm17450611pfh.65.2021.09.27.07.59.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 07:59:57 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v5 04/12] tools: Allow specifying base BTF file in resolve_btfids Date: Mon, 27 Sep 2021 20:29:33 +0530 Message-Id: <20210927145941.1383001-5-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210927145941.1383001-1-memxor@gmail.com> References: <20210927145941.1383001-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3590; h=from:subject; bh=JQF5FTCXoiG0ws4sHCqjBu7XKk8EtlYkG/YdFjxrcpU=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhUdxOWhWKmXQXqjEOOQQo0Qvqm2W8oSyAf2VVF5+s gNtOhXOJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYVHcTgAKCRBM4MiGSL8RyiB/D/ 4mCpUZFjf0BR5A161S4uBUOybLaEmmULoQ+Q0X0HYU5DVGZS1tWfoCsyiCRL1zwfPMtkPY0ix+yHy2 QHJyrWhJ+0xjWQrQ2pXvfAslvl4SET2xJ+jGWti/qv2Xb7XdkFac+JqhS9kBqfv2vO2RHZpvKbINd0 s2Slr2RrJ46ZgYPhfw7xI3oLfoaJX42bNL9Vji5FGmLbb2F09DhVrlDAQlNcMjFwLhRcC1jLKBHLQY siHu0HQDsCBiYhBOakOYX4ZhGvB+mAOZQCoU4p50ggwGPN+E0g02Fei0/3x4tlp8Z3Jt+C6mSsqwyl KPpCLWAkN411FormJnWApC+ukcAsg0eLC1u38wwXzQ3oNFPe9qooAJsCAzKCuEnSLvva9MvOkNqGM8 oD44HY2Q32l+XE1wN+4mbH/9ptXFHvHxejGYAOFgJ0SIpGOnoKBXIcNzB5Q3MtmcDl0bDTMAPo4zx4 osiY0qNvY9a63mm0cfar8ccrqwglbUP9bN4jOItpEzxYx56Hn9JWn3UMQLDAMvPoAlxma8eVSLIlrG kvd1bEbjp7gy2KN7aL024lAJ0UGYV325SdyeLS56aVxf+RnkPqDmMO5M5JCwlM5YaDIUScaXhK4BIG Q5tgaLGKKb3EaxtoHV8iEODeTfE+GIrkYTYQYG7Tv2liksm4E+2BIDAx6Y0A== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This commits allows specifying the base BTF for resolving btf id lists/sets during link time in the resolve_btfids tool. The base BTF is set to NULL if no path is passed. This allows resolving BTF ids for module kernel objects. Also, drop the --no-fail option, as it is only used in case .BTF_ids section is not present, instead make no-fail the default mode. The long option name is same as that of pahole. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/bpf/resolve_btfids/main.c | 28 +++++++++++++++++++--------- tools/testing/selftests/bpf/Makefile | 2 +- 2 files changed, 20 insertions(+), 10 deletions(-) diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c index de6365b53c9c..c6c3e613858a 100644 --- a/tools/bpf/resolve_btfids/main.c +++ b/tools/bpf/resolve_btfids/main.c @@ -89,6 +89,7 @@ struct btf_id { struct object { const char *path; const char *btf; + const char *base_btf_path; struct { int fd; @@ -477,16 +478,27 @@ static int symbols_resolve(struct object *obj) int nr_structs = obj->nr_structs; int nr_unions = obj->nr_unions; int nr_funcs = obj->nr_funcs; + struct btf *base_btf = NULL; int err, type_id; struct btf *btf; __u32 nr_types; - btf = btf__parse(obj->btf ?: obj->path, NULL); + if (obj->base_btf_path) { + base_btf = btf__parse(obj->base_btf_path, NULL); + err = libbpf_get_error(base_btf); + if (err) { + pr_err("FAILED: load base BTF from %s: %s\n", + obj->base_btf_path, strerror(-err)); + return -1; + } + } + + btf = btf__parse_split(obj->btf ?: obj->path, base_btf); err = libbpf_get_error(btf); if (err) { pr_err("FAILED: load BTF from %s: %s\n", obj->btf ?: obj->path, strerror(-err)); - return -1; + goto out; } err = -1; @@ -545,6 +557,7 @@ static int symbols_resolve(struct object *obj) err = 0; out: + btf__free(base_btf); btf__free(btf); return err; } @@ -678,7 +691,6 @@ static const char * const resolve_btfids_usage[] = { int main(int argc, const char **argv) { - bool no_fail = false; struct object obj = { .efile = { .idlist_shndx = -1, @@ -695,8 +707,8 @@ int main(int argc, const char **argv) "be more verbose (show errors, etc)"), OPT_STRING(0, "btf", &obj.btf, "BTF data", "BTF data"), - OPT_BOOLEAN(0, "no-fail", &no_fail, - "do not fail if " BTF_IDS_SECTION " section is not found"), + OPT_STRING('b', "btf_base", &obj.base_btf_path, "file", + "path of file providing base BTF"), OPT_END() }; int err = -1; @@ -717,10 +729,8 @@ int main(int argc, const char **argv) */ if (obj.efile.idlist_shndx == -1 || obj.efile.symbols_shndx == -1) { - if (no_fail) - return 0; - pr_err("FAILED to find needed sections\n"); - return -1; + pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n"); + return 0; } if (symbols_collect(&obj)) diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 326ea75ce99e..e1ce73be7a5b 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -453,7 +453,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ | $(TRUNNER_BINARY)-extras $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) -o $$@ - $(Q)$(RESOLVE_BTFIDS) --no-fail --btf $(TRUNNER_OUTPUT)/btf_data.o $$@ + $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.o $$@ endef From patchwork Mon Sep 27 14:59:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12520095 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2948C433F5 for ; Mon, 27 Sep 2021 15:00:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DC8D060FED for ; Mon, 27 Sep 2021 15:00:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235005AbhI0PBw (ORCPT ); Mon, 27 Sep 2021 11:01:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36312 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234951AbhI0PBk (ORCPT ); Mon, 27 Sep 2021 11:01:40 -0400 Received: from mail-pl1-x641.google.com (mail-pl1-x641.google.com [IPv6:2607:f8b0:4864:20::641]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CAA11C061714; Mon, 27 Sep 2021 08:00:01 -0700 (PDT) Received: by mail-pl1-x641.google.com with SMTP id y5so9525842pll.3; Mon, 27 Sep 2021 08:00:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yKGmIS7C5DRrVfELf8OojS9SVi5z+gHo+jOm3l5DvEc=; b=ACFgI6sllMP/LwNdaCCb8EiqyYij3/A82Je9jYYLG9Cdu1fgaffpsMQyH0JsWJheCs H+1tbcIepiPhM7xtB9mmDFH1WtE4nJ/kuuQpjfqv0I4iwZFutiS97xqGHtO7DWsVw5So JDMNOyUknSvdkld0RhA16k0DlCRPLnTI5RPqB1sPvekixKcl4WmY2Os0hb8XUfMeNo/m OB1Ujai1ZfQNlwKctdnAa9Gy4udabbrRcz0o1DooRnvYli8Z4a+ufMxi0gwAmPfRM8o+ 1WhErtmrh5r+7wMi2PVR2hDJ9vYnHusww59l3acyyXo3GD4taA54zETDld+ycXA8e6i1 tO9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yKGmIS7C5DRrVfELf8OojS9SVi5z+gHo+jOm3l5DvEc=; b=1WHoI5fzI3sOir1oVgQFl2bfnN/ijqzmmymLgarIkrIlwbdL6zGxGA1DDHCE7UkBBh fgILH45fGX/4ON7cIbw2dnEdyvQEObaxCe6wbeYSAkO6bmxP+TdhtiFkneglpogoJFiw uyLg4UKf/hFeXN7WrwDnnbesAJBhe+2ieJBHUGuzOU5G4MpOlsvyMDsO6J/C+Jjbe8yQ h+s3HhyKFABRuNDsKytQ1aFUURBOjE8JVVir7D7aDDFSWUC1ZbiXgxAJ6cmfdmyphv2g iIsuRjCKjcRKOXrtqZQ7IG6OlhPy8kuseit4E6MUntfxRqtmY3pwtkY9TVJ8f+IoIwnZ DnCg== X-Gm-Message-State: AOAM530E6oaS8IyYWyPq5xoa4ZrG0xQycxGKtbB3GzV1gmMFUnXYduqe XObRofg/E4gtk/fgTFGwUVZcXBPUnZk= X-Google-Smtp-Source: ABdhPJyr43nWj2VLU5e2ZNy8ABDxLdour5C8cIWIhcRsTO1CAbZ874QwHvXFQCnseA2Usw672QZflg== X-Received: by 2002:a17:903:1103:b0:13a:1dd7:485f with SMTP id n3-20020a170903110300b0013a1dd7485fmr35154plh.6.1632754801081; Mon, 27 Sep 2021 08:00:01 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id j3sm18103385pgn.12.2021.09.27.08.00.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 08:00:00 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v5 05/12] bpf: Enable TCP congestion control kfunc from modules Date: Mon, 27 Sep 2021 20:29:34 +0530 Message-Id: <20210927145941.1383001-6-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210927145941.1383001-1-memxor@gmail.com> References: <20210927145941.1383001-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=9091; h=from:subject; bh=rxa0xEHmglr7YFOfcijdyTW/nCW7eae0WTlG7/lN0qc=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhUdxO10m1Pya+mmEs93dy9pGb1k5n2oTmvWarBWWf aUdSFlWJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYVHcTgAKCRBM4MiGSL8RyotfD/ 9SlQLapkXC+3/bqYEg6IW8q+F2qN9UNA4COctoVI+q9JvpNpsO5M8DKUHXKjbFWDWEkMJrqmcNN04p fKrWQn7UQghfCbz633S1hoQbNrHcvUp0r9AeohJ6yAfapF/3pYY2ApRKD7KxRgHwQVHJYu8VIv6NgI Vy7eRR/WJAgR+syBu3fY6Z/35GxdL7m7G0n/v5ieuRyEBVQTzAx2ZSBKXXXHrlBBK/EYE1iXZBjS5Z SNzEiJr5N6h6ODYl3HZ9Ubj0FnWW+QWACiu5fu/EeCrb77MFTR+YGmo+6D9oWRbYptHOyLXOv/rWo/ W+cu7cc2hDa3t6JEZPCR8zGY799Siu0NPOFP3mlF7UYcJQqzY8sihwNXUqJKX7B/iBl3jByO2zDO2W Zw1JZq9sCQdZlH8pXW+0o7Vi1m1O4YXGMOCcMOoPWvRkugZCtmlrexU8cQnOzxpVHgMyxzLMqJb9cV WCFo0MBeJ9c7/n03pr0TrBe9ktRy8TLnYnqgU8k2aQdboE5u0IWl3ngsmmLqp4D95VNGoaiyq7aP3b haZdObDIACWL+8OXrToI+SDMuB8KdqJ8mvq6O/eZ6gn5shmB/5R6p7J5dD9Em6ZTgIxK5rD1kIKHsS 9Oxw8v9GbTZdbSGxBc/TcK5uwHSWMUkM+kbP3AY5Lme46XH/7ghXHFjivmtQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This commit moves BTF ID lookup into the newly added registration helper, in a way that the bbr, cubic, and dctcp implementation set up their sets in the bpf_tcp_ca kfunc_btf_set list, while the ones not dependent on modules are looked up from the wrapper function. This lifts the restriction for them to be compiled as built in objects, and can be loaded as modules if required. Also modify Makefile.modfinal to call resolve_btfids for each module. Note that since kernel kfunc_ids never overlap with module kfunc_ids, we only match the owner for module btf id sets. See following commits for background on use of: CONFIG_X86 ifdef: 569c484f9995 (bpf: Limit static tcp-cc functions in the .BTF_ids list to x86) CONFIG_DYNAMIC_FTRACE ifdef: 7aae231ac93b (bpf: tcp: Limit calling some tcp cc functions to CONFIG_DYNAMIC_FTRACE) Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/btf.h | 4 ++++ kernel/bpf/btf.c | 3 +++ net/ipv4/bpf_tcp_ca.c | 34 +++------------------------------- net/ipv4/tcp_bbr.c | 28 +++++++++++++++++++++++++++- net/ipv4/tcp_cubic.c | 26 +++++++++++++++++++++++++- net/ipv4/tcp_dctcp.c | 26 +++++++++++++++++++++++++- scripts/Makefile.modfinal | 1 + 7 files changed, 88 insertions(+), 34 deletions(-) diff --git a/include/linux/btf.h b/include/linux/btf.h index 382c00d5cede..8dc65ff02bd6 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -269,4 +269,8 @@ static inline void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, struct kfunc_btf_id_set name = { LIST_HEAD_INIT(name.list), (set), \ THIS_MODULE } +extern struct kfunc_btf_id_list bpf_tcp_ca_kfunc_list; + +DECLARE_CHECK_KFUNC_CALLBACK(tcp_ca); + #endif diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 5a8806cfecd0..423744378fc8 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6394,3 +6394,6 @@ EXPORT_SYMBOL_GPL(unregister_kfunc_btf_id_set); mutex_unlock(&list_name.mutex); \ return false; \ } + +DEFINE_KFUNC_BTF_ID_LIST(bpf_tcp_ca_kfunc_list); +DEFINE_CHECK_KFUNC_CALLBACK(tcp_ca, bpf_tcp_ca_kfunc_list); diff --git a/net/ipv4/bpf_tcp_ca.c b/net/ipv4/bpf_tcp_ca.c index b3afd3361f34..dae1515d81dd 100644 --- a/net/ipv4/bpf_tcp_ca.c +++ b/net/ipv4/bpf_tcp_ca.c @@ -223,41 +223,13 @@ BTF_ID(func, tcp_reno_cong_avoid) BTF_ID(func, tcp_reno_undo_cwnd) BTF_ID(func, tcp_slow_start) BTF_ID(func, tcp_cong_avoid_ai) -#ifdef CONFIG_X86 -#ifdef CONFIG_DYNAMIC_FTRACE -#if IS_BUILTIN(CONFIG_TCP_CONG_CUBIC) -BTF_ID(func, cubictcp_init) -BTF_ID(func, cubictcp_recalc_ssthresh) -BTF_ID(func, cubictcp_cong_avoid) -BTF_ID(func, cubictcp_state) -BTF_ID(func, cubictcp_cwnd_event) -BTF_ID(func, cubictcp_acked) -#endif -#if IS_BUILTIN(CONFIG_TCP_CONG_DCTCP) -BTF_ID(func, dctcp_init) -BTF_ID(func, dctcp_update_alpha) -BTF_ID(func, dctcp_cwnd_event) -BTF_ID(func, dctcp_ssthresh) -BTF_ID(func, dctcp_cwnd_undo) -BTF_ID(func, dctcp_state) -#endif -#if IS_BUILTIN(CONFIG_TCP_CONG_BBR) -BTF_ID(func, bbr_init) -BTF_ID(func, bbr_main) -BTF_ID(func, bbr_sndbuf_expand) -BTF_ID(func, bbr_undo_cwnd) -BTF_ID(func, bbr_cwnd_event) -BTF_ID(func, bbr_ssthresh) -BTF_ID(func, bbr_min_tso_segs) -BTF_ID(func, bbr_set_state) -#endif -#endif /* CONFIG_DYNAMIC_FTRACE */ -#endif /* CONFIG_X86 */ BTF_SET_END(bpf_tcp_ca_kfunc_ids) static bool bpf_tcp_ca_check_kfunc_call(u32 kfunc_btf_id, struct module *owner) { - return btf_id_set_contains(&bpf_tcp_ca_kfunc_ids, kfunc_btf_id); + if (btf_id_set_contains(&bpf_tcp_ca_kfunc_ids, kfunc_btf_id)) + return true; + return __bpf_tcp_ca_check_kfunc_call(kfunc_btf_id, owner); } static const struct bpf_verifier_ops bpf_tcp_ca_verifier_ops = { diff --git a/net/ipv4/tcp_bbr.c b/net/ipv4/tcp_bbr.c index 6274462b86b4..ec5550089b4d 100644 --- a/net/ipv4/tcp_bbr.c +++ b/net/ipv4/tcp_bbr.c @@ -56,6 +56,8 @@ * otherwise TCP stack falls back to an internal pacing using one high * resolution timer per TCP socket and may use more resources. */ +#include +#include #include #include #include @@ -1152,14 +1154,38 @@ static struct tcp_congestion_ops tcp_bbr_cong_ops __read_mostly = { .set_state = bbr_set_state, }; +BTF_SET_START(tcp_bbr_kfunc_ids) +#ifdef CONFIG_X86 +#ifdef CONFIG_DYNAMIC_FTRACE +BTF_ID(func, bbr_init) +BTF_ID(func, bbr_main) +BTF_ID(func, bbr_sndbuf_expand) +BTF_ID(func, bbr_undo_cwnd) +BTF_ID(func, bbr_cwnd_event) +BTF_ID(func, bbr_ssthresh) +BTF_ID(func, bbr_min_tso_segs) +BTF_ID(func, bbr_set_state) +#endif +#endif +BTF_SET_END(tcp_bbr_kfunc_ids) + +static DEFINE_KFUNC_BTF_ID_SET(&tcp_bbr_kfunc_ids, tcp_bbr_kfunc_btf_set); + static int __init bbr_register(void) { + int ret; + BUILD_BUG_ON(sizeof(struct bbr) > ICSK_CA_PRIV_SIZE); - return tcp_register_congestion_control(&tcp_bbr_cong_ops); + ret = tcp_register_congestion_control(&tcp_bbr_cong_ops); + if (ret) + return ret; + register_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_bbr_kfunc_btf_set); + return 0; } static void __exit bbr_unregister(void) { + unregister_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_bbr_kfunc_btf_set); tcp_unregister_congestion_control(&tcp_bbr_cong_ops); } diff --git a/net/ipv4/tcp_cubic.c b/net/ipv4/tcp_cubic.c index 4a30deaa9a37..5e9d9c51164c 100644 --- a/net/ipv4/tcp_cubic.c +++ b/net/ipv4/tcp_cubic.c @@ -25,6 +25,8 @@ */ #include +#include +#include #include #include #include @@ -482,8 +484,25 @@ static struct tcp_congestion_ops cubictcp __read_mostly = { .name = "cubic", }; +BTF_SET_START(tcp_cubic_kfunc_ids) +#ifdef CONFIG_X86 +#ifdef CONFIG_DYNAMIC_FTRACE +BTF_ID(func, cubictcp_init) +BTF_ID(func, cubictcp_recalc_ssthresh) +BTF_ID(func, cubictcp_cong_avoid) +BTF_ID(func, cubictcp_state) +BTF_ID(func, cubictcp_cwnd_event) +BTF_ID(func, cubictcp_acked) +#endif +#endif +BTF_SET_END(tcp_cubic_kfunc_ids) + +static DEFINE_KFUNC_BTF_ID_SET(&tcp_cubic_kfunc_ids, tcp_cubic_kfunc_btf_set); + static int __init cubictcp_register(void) { + int ret; + BUILD_BUG_ON(sizeof(struct bictcp) > ICSK_CA_PRIV_SIZE); /* Precompute a bunch of the scaling factors that are used per-packet @@ -514,11 +533,16 @@ static int __init cubictcp_register(void) /* divide by bic_scale and by constant Srtt (100ms) */ do_div(cube_factor, bic_scale * 10); - return tcp_register_congestion_control(&cubictcp); + ret = tcp_register_congestion_control(&cubictcp); + if (ret) + return ret; + register_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_cubic_kfunc_btf_set); + return 0; } static void __exit cubictcp_unregister(void) { + unregister_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_cubic_kfunc_btf_set); tcp_unregister_congestion_control(&cubictcp); } diff --git a/net/ipv4/tcp_dctcp.c b/net/ipv4/tcp_dctcp.c index 79f705450c16..0d7ab3cc7b61 100644 --- a/net/ipv4/tcp_dctcp.c +++ b/net/ipv4/tcp_dctcp.c @@ -36,6 +36,8 @@ * Glenn Judd */ +#include +#include #include #include #include @@ -236,14 +238,36 @@ static struct tcp_congestion_ops dctcp_reno __read_mostly = { .name = "dctcp-reno", }; +BTF_SET_START(tcp_dctcp_kfunc_ids) +#ifdef CONFIG_X86 +#ifdef CONFIG_DYNAMIC_FTRACE +BTF_ID(func, dctcp_init) +BTF_ID(func, dctcp_update_alpha) +BTF_ID(func, dctcp_cwnd_event) +BTF_ID(func, dctcp_ssthresh) +BTF_ID(func, dctcp_cwnd_undo) +BTF_ID(func, dctcp_state) +#endif +#endif +BTF_SET_END(tcp_dctcp_kfunc_ids) + +static DEFINE_KFUNC_BTF_ID_SET(&tcp_dctcp_kfunc_ids, tcp_dctcp_kfunc_btf_set); + static int __init dctcp_register(void) { + int ret; + BUILD_BUG_ON(sizeof(struct dctcp) > ICSK_CA_PRIV_SIZE); - return tcp_register_congestion_control(&dctcp); + ret = tcp_register_congestion_control(&dctcp); + if (ret) + return ret; + register_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_dctcp_kfunc_btf_set); + return 0; } static void __exit dctcp_unregister(void) { + unregister_kfunc_btf_id_set(&bpf_tcp_ca_kfunc_list, &tcp_dctcp_kfunc_btf_set); tcp_unregister_congestion_control(&dctcp); } diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index ff805777431c..1fb45b011e4b 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -41,6 +41,7 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ -f vmlinux ]; then \ LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J --btf_base vmlinux $@; \ + $(RESOLVE_BTFIDS) -b vmlinux $@; \ else \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ fi; From patchwork Mon Sep 27 14:59:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12520097 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0E99C4332F for ; Mon, 27 Sep 2021 15:00:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8AB9460EE2 for ; Mon, 27 Sep 2021 15:00:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235032AbhI0PCD (ORCPT ); Mon, 27 Sep 2021 11:02:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36374 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235051AbhI0PBu (ORCPT ); Mon, 27 Sep 2021 11:01:50 -0400 Received: from mail-pl1-x641.google.com (mail-pl1-x641.google.com [IPv6:2607:f8b0:4864:20::641]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DF174C06176F; Mon, 27 Sep 2021 08:00:04 -0700 (PDT) Received: by mail-pl1-x641.google.com with SMTP id y5so9525938pll.3; Mon, 27 Sep 2021 08:00:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LULoxn4nL4Q36b5bUqKCWXkoihb4lldZkQC1RkIgXd8=; b=KDq7YwKTDV6ZZfzRm9VkrYmrP8I4virQ7pppMG1A1b9sDFYJIfUarG7sd1tQEnHH6A qqmEi1q4qIjPRToyA8XhcDXDldDJctJF7SuWcjtjFdOOQ2/QQHzYaxTfyU1V0SUU7BEQ EkWY3Xy1IByFJTDWJFErOnegYsFoLZSmjidY+5YozMWTI3O1exirSCmfqIl+s+nqWeTj uCkpMvcU4PLmrkhsNn1USBLQze8+ZM74WZUc/Psl18OT20rPc9wJfXny/r3ake5qnaGD Sj4PdogkCufzhXcVVdhF7eS0pAfLBPZHGebk86+0XAYUzQBIhPOV4Odtg4R09YgpaaKt 0luA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LULoxn4nL4Q36b5bUqKCWXkoihb4lldZkQC1RkIgXd8=; b=AVcWVfOfvcz3WIP0p2fO9OqfPsT1rkuNF5fqZa817WS0rq8kScBwr30LSOrbcOjz5y c42CT7kwZnCZMhXyt25WCn+XM1I0Dl89qxAoe4AENbmqJfXq6QxpXd+hZHs4uzY7eJWs qGXMWpd0wqVO0sD5lPcQZP92BqBMQ8hf/Chclh0otwN0NXazEp8iWPYrHzC6oeJD8YY/ I24GiART2pQvVEXnpaB/N4Kkv2OUKdRy9GVPet6TJ8VbV3g/eY+nTPvbqGIBFSnpeDtk NXv53WI+ISuC8ltNh/iRB7voul1581JJUw/6V9RuqMzMYLEc6XqJTOOmQnPUOFog4eDh sQcQ== X-Gm-Message-State: AOAM530K4S8kRVSQTtZLBH+bx8UrGYOLr/t3y3IcWUUCr9W0NQ9tcfFA OHyjcakYPhUkkLu+qbputg3/XtvS3X4= X-Google-Smtp-Source: ABdhPJyzbmTrOEtLHO7kpsWWnkXfma0xhw60TT1WYFi6OBf0IUJxxacsFeFUWqwp5jh0YSjg1Ug1OA== X-Received: by 2002:a17:90a:9404:: with SMTP id r4mr425485pjo.240.1632754804155; Mon, 27 Sep 2021 08:00:04 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id v7sm16028016pjk.37.2021.09.27.08.00.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 08:00:03 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v5 06/12] libbpf: Support kernel module function calls Date: Mon, 27 Sep 2021 20:29:35 +0530 Message-Id: <20210927145941.1383001-7-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210927145941.1383001-1-memxor@gmail.com> References: <20210927145941.1383001-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=10698; h=from:subject; bh=LkCWHWBbKelQhH0cNbXGEJiHjG3OhDjbAgl0mOtIX4w=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhUdxOWxo47GVMSoMjTwONEK9cWhmLs+/1REW2RdAQ dOF1ASOJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYVHcTgAKCRBM4MiGSL8RyhCXD/ 9emdkktcc6AWXzv/oOwOeUwdOwuuqGLqcDPYn3jcpUrXFWABrBy9LDuNjaFFbNRwS3b34pWggeCi1L V6/5m6IPhJ5+boFyhgP6icO7joVGvI+3WaJsCjY9K4z2tCJYdLJpsgfwTe0X2FGGLcYj9W8BxDC18E YTas0wjp0pgVmOqrbfYmwRH1ffSFUpMX3puJm2jdntv/SztWSi2wZqbtH2ELnNBZflCYYD57d7qS5T dL76AOC4Yq4SzGrjOMz8TPcVObN867qL9EtiruzLz956AHcBGDIq0CcBWiEYgQjQ/3MRwEs7RU5bgD ST12bFeRBwMPUWTluvZvhavvmgVQojXGOmSsvQ3McNyQ5IOR+WYVzLMamXughmHUDByLWpmaMznk9B Qx/A97KcUeg6dgcp7IEdybJNsGrpDJLkUYThJD/7jbnhJjEjr7Vddf1EldfjOsMpZ13aTPccNgKo9v XmW0BqnTuaOqI4ks+yapzxxnoz1Tt1ZssctmLti2IvF+OHy5C7VZk51w97wb1164wP0ybXZOwbt+ya Dfl4zoter3DqcP3vf369FRvOyZ9byyrLUq2QXI19ulpr++cdEgP51CFNxln12XptYK7OUwpG/+4o2M lany1v+Nbl25UK2CoATdrlV5qMk6F7besvM+CD9SRXx6jIetOTrx01WmwHoQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This patch adds libbpf support for kernel module function call support. The fd_array parameter is used during BPF program load is used to pass module BTFs referenced by the program. insn->off is set to index into this array, but starts from 1, because insn->off as 0 is reserved for btf_vmlinux. We try to use existing insn->off for a module, since the kernel limits the maximum distinct module BTFs for kfuncs to 256, and also because index must never exceed the maximum allowed value that can fit in insn->off (INT16_MAX). In the future, if kernel interprets signed offset as unsigned for kfunc calls, this limit can be increased to UINT16_MAX. Also introduce a btf__find_by_name_kind_own helper to start searching from module BTF's start id when we know that the BTF ID is not present in vmlinux BTF (in find_ksym_btf_id). Signed-off-by: Kumar Kartikeya Dwivedi --- tools/lib/bpf/bpf.c | 1 + tools/lib/bpf/btf.c | 19 ++++++-- tools/lib/bpf/libbpf.c | 80 +++++++++++++++++++++++---------- tools/lib/bpf/libbpf_internal.h | 3 ++ 4 files changed, 76 insertions(+), 27 deletions(-) diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 2401fad090c5..7d1741ceaa32 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -264,6 +264,7 @@ int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr) attr.line_info_rec_size = load_attr->line_info_rec_size; attr.line_info_cnt = load_attr->line_info_cnt; attr.line_info = ptr_to_u64(load_attr->line_info); + attr.fd_array = ptr_to_u64(load_attr->fd_array); if (load_attr->name) memcpy(attr.prog_name, load_attr->name, diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c index 6ad63e4d418a..f1d872b3fbf4 100644 --- a/tools/lib/bpf/btf.c +++ b/tools/lib/bpf/btf.c @@ -695,15 +695,16 @@ __s32 btf__find_by_name(const struct btf *btf, const char *type_name) return libbpf_err(-ENOENT); } -__s32 btf__find_by_name_kind(const struct btf *btf, const char *type_name, - __u32 kind) +static __s32 __btf__find_by_name_kind(const struct btf *btf, + const char *type_name, __u32 kind, + bool own) { __u32 i, nr_types = btf__get_nr_types(btf); if (kind == BTF_KIND_UNKN || !strcmp(type_name, "void")) return 0; - for (i = 1; i <= nr_types; i++) { + for (i = own ? btf->start_id : 1; i <= nr_types; i++) { const struct btf_type *t = btf__type_by_id(btf, i); const char *name; @@ -717,6 +718,18 @@ __s32 btf__find_by_name_kind(const struct btf *btf, const char *type_name, return libbpf_err(-ENOENT); } +__s32 btf__find_by_name_kind_own(const struct btf *btf, const char *type_name, + __u32 kind) +{ + return __btf__find_by_name_kind(btf, type_name, kind, true); +} + +__s32 btf__find_by_name_kind(const struct btf *btf, const char *type_name, + __u32 kind) +{ + return __btf__find_by_name_kind(btf, type_name, kind, false); +} + static bool btf_is_modifiable(const struct btf *btf) { return (void *)btf->hdr != btf->raw_data; diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index ef5db34bf913..0f4d203f926e 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -420,6 +420,11 @@ struct extern_desc { /* local btf_id of the ksym extern's type. */ __u32 type_id; + /* offset to be patched in for insn->off, this is 0 for + * vmlinux BTF, and BTF fd index in obj->fd_array for + * module BTF + */ + __s16 offset; } ksym; }; }; @@ -431,6 +436,7 @@ struct module_btf { char *name; __u32 id; int fd; + int fd_array_idx; }; struct bpf_object { @@ -516,6 +522,10 @@ struct bpf_object { void *priv; bpf_object_clear_priv_t clear_priv; + int *fd_array; + size_t fd_array_cap; + size_t fd_array_cnt; + char path[]; }; #define obj_elf_valid(o) ((o)->efile.elf) @@ -1145,6 +1155,9 @@ static struct bpf_object *bpf_object__new(const char *path, obj->kern_version = get_kernel_version(); obj->loaded = false; + /* We cannot use index 0 for module BTF fds */ + obj->fd_array_cnt = 1; + INIT_LIST_HEAD(&obj->list); list_add(&obj->list, &bpf_objects_list); return obj; @@ -5357,6 +5370,7 @@ bpf_object__relocate_data(struct bpf_object *obj, struct bpf_program *prog) ext = &obj->externs[relo->sym_off]; insn[0].src_reg = BPF_PSEUDO_KFUNC_CALL; insn[0].imm = ext->ksym.kernel_btf_id; + insn[0].off = ext->ksym.offset; break; case RELO_SUBPROG_ADDR: if (insn[0].src_reg != BPF_PSEUDO_FUNC) { @@ -6151,6 +6165,7 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt, } load_attr.log_level = prog->log_level; load_attr.prog_flags = prog->prog_flags; + load_attr.fd_array = prog->obj->fd_array; if (prog->obj->gen_loader) { bpf_gen__prog_load(prog->obj->gen_loader, &load_attr, @@ -6661,13 +6676,14 @@ static int bpf_object__read_kallsyms_file(struct bpf_object *obj) static int find_ksym_btf_id(struct bpf_object *obj, const char *ksym_name, __u16 kind, struct btf **res_btf, - int *res_btf_fd) + struct module_btf **res_mod_btf) { - int i, id, btf_fd, err; + struct module_btf *mod_btf; struct btf *btf; + int i, id, err; btf = obj->btf_vmlinux; - btf_fd = 0; + mod_btf = NULL; id = btf__find_by_name_kind(btf, ksym_name, kind); if (id == -ENOENT) { @@ -6676,10 +6692,10 @@ static int find_ksym_btf_id(struct bpf_object *obj, const char *ksym_name, return err; for (i = 0; i < obj->btf_module_cnt; i++) { - btf = obj->btf_modules[i].btf; - /* we assume module BTF FD is always >0 */ - btf_fd = obj->btf_modules[i].fd; - id = btf__find_by_name_kind(btf, ksym_name, kind); + /* we assume module_btf's BTF FD is always >0 */ + mod_btf = &obj->btf_modules[i]; + btf = mod_btf->btf; + id = btf__find_by_name_kind_own(btf, ksym_name, kind); if (id != -ENOENT) break; } @@ -6688,7 +6704,7 @@ static int find_ksym_btf_id(struct bpf_object *obj, const char *ksym_name, return -ESRCH; *res_btf = btf; - *res_btf_fd = btf_fd; + *res_mod_btf = mod_btf; return id; } @@ -6697,11 +6713,12 @@ static int bpf_object__resolve_ksym_var_btf_id(struct bpf_object *obj, { const struct btf_type *targ_var, *targ_type; __u32 targ_type_id, local_type_id; + struct module_btf *mod_btf = NULL; const char *targ_var_name; - int id, btf_fd = 0, err; struct btf *btf = NULL; + int id, err; - id = find_ksym_btf_id(obj, ext->name, BTF_KIND_VAR, &btf, &btf_fd); + id = find_ksym_btf_id(obj, ext->name, BTF_KIND_VAR, &btf, &mod_btf); if (id == -ESRCH && ext->is_weak) { return 0; } else if (id < 0) { @@ -6736,7 +6753,7 @@ static int bpf_object__resolve_ksym_var_btf_id(struct bpf_object *obj, } ext->is_set = true; - ext->ksym.kernel_btf_obj_fd = btf_fd; + ext->ksym.kernel_btf_obj_fd = mod_btf ? mod_btf->fd : 0; ext->ksym.kernel_btf_id = id; pr_debug("extern (var ksym) '%s': resolved to [%d] %s %s\n", ext->name, id, btf_kind_str(targ_var), targ_var_name); @@ -6748,40 +6765,52 @@ static int bpf_object__resolve_ksym_func_btf_id(struct bpf_object *obj, struct extern_desc *ext) { int local_func_proto_id, kfunc_proto_id, kfunc_id; + struct module_btf *mod_btf = NULL; const struct btf_type *kern_func; - struct btf *kern_btf = NULL; - int ret, kern_btf_fd = 0; + struct btf *btf = NULL; + int ret; local_func_proto_id = ext->ksym.type_id; - kfunc_id = find_ksym_btf_id(obj, ext->name, BTF_KIND_FUNC, - &kern_btf, &kern_btf_fd); + kfunc_id = find_ksym_btf_id(obj, ext->name, BTF_KIND_FUNC, &btf, &mod_btf); if (kfunc_id < 0) { pr_warn("extern (func ksym) '%s': not found in kernel BTF\n", ext->name); return kfunc_id; } - if (kern_btf != obj->btf_vmlinux) { - pr_warn("extern (func ksym) '%s': function in kernel module is not supported\n", - ext->name); - return -ENOTSUP; - } - - kern_func = btf__type_by_id(kern_btf, kfunc_id); + kern_func = btf__type_by_id(btf, kfunc_id); kfunc_proto_id = kern_func->type; ret = bpf_core_types_are_compat(obj->btf, local_func_proto_id, - kern_btf, kfunc_proto_id); + btf, kfunc_proto_id); if (ret <= 0) { pr_warn("extern (func ksym) '%s': func_proto [%d] incompatible with kernel [%d]\n", ext->name, local_func_proto_id, kfunc_proto_id); return -EINVAL; } + /* set index for module BTF fd in fd_array, if unset */ + if (mod_btf && !mod_btf->fd_array_idx) { + /* insn->off is s16 */ + if (obj->fd_array_cnt == INT16_MAX) { + pr_warn("extern (func ksym) '%s': module BTF fd index %d too big to fit in bpf_insn offset\n", + ext->name, mod_btf->fd_array_idx); + return -E2BIG; + } + + ret = libbpf_ensure_mem((void **)&obj->fd_array, &obj->fd_array_cap, sizeof(int), + obj->fd_array_cnt + 1); + if (ret) + return ret; + mod_btf->fd_array_idx = obj->fd_array_cnt; + /* we assume module BTF FD is always >0 */ + obj->fd_array[obj->fd_array_cnt++] = mod_btf->fd; + } + ext->is_set = true; - ext->ksym.kernel_btf_obj_fd = kern_btf_fd; ext->ksym.kernel_btf_id = kfunc_id; + ext->ksym.offset = mod_btf ? mod_btf->fd_array_idx : 0; pr_debug("extern (func ksym) '%s': resolved to kernel [%d]\n", ext->name, kfunc_id); @@ -6941,6 +6970,9 @@ int bpf_object__load_xattr(struct bpf_object_load_attr *attr) err = bpf_gen__finish(obj->gen_loader); } + /* clean up fd_array */ + zfree(&obj->fd_array); + /* clean up module BTFs */ for (i = 0; i < obj->btf_module_cnt; i++) { close(obj->btf_modules[i].fd); diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h index ceb0c98979bc..1969d0ce35cf 100644 --- a/tools/lib/bpf/libbpf_internal.h +++ b/tools/lib/bpf/libbpf_internal.h @@ -291,6 +291,7 @@ struct bpf_prog_load_params { __u32 log_level; char *log_buf; size_t log_buf_sz; + int *fd_array; }; int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr); @@ -401,6 +402,8 @@ int btf_type_visit_type_ids(struct btf_type *t, type_id_visit_fn visit, void *ct int btf_type_visit_str_offs(struct btf_type *t, str_off_visit_fn visit, void *ctx); int btf_ext_visit_type_ids(struct btf_ext *btf_ext, type_id_visit_fn visit, void *ctx); int btf_ext_visit_str_offs(struct btf_ext *btf_ext, str_off_visit_fn visit, void *ctx); +__s32 btf__find_by_name_kind_own(const struct btf *btf, const char *type_name, + __u32 kind); extern enum libbpf_strict_mode libbpf_mode; From patchwork Mon Sep 27 14:59:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12520099 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2F3EC433EF for ; Mon, 27 Sep 2021 15:00:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C5FDA60EE2 for ; Mon, 27 Sep 2021 15:00:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235040AbhI0PCL (ORCPT ); Mon, 27 Sep 2021 11:02:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36324 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235004AbhI0PBw (ORCPT ); Mon, 27 Sep 2021 11:01:52 -0400 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9FE45C061776; Mon, 27 Sep 2021 08:00:08 -0700 (PDT) Received: by mail-pj1-x1042.google.com with SMTP id k23-20020a17090a591700b001976d2db364so233098pji.2; Mon, 27 Sep 2021 08:00:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=75YvYvEsvh397a9t/GUbdRglLQeQDw2VgrXGcjLzh6g=; b=cQi+YaMiAiHQahXir/RNLlE1zThQCWwHoWHtM2/KwmHQ/yXwKZC6A0iRcOikLz4kgh 7rB7kBAucuBz5WKAPVGFEv9ZkzdHKkOb4rj05JedtPO+zThVePWT5fE04BxLgsqJkgWm CiMfi7UTdzxHlXd1i6ZC47QkD2ogHBrKT+PgH6TfVQEv6M4wKxGdyWJD60mTbS+i5J86 Ov3MxaImoVUWio9Y6CdBvxssvbGMdOZv8qpen5xDUkx1b745PcO9eZ8L62n7XqMIOCa7 0qq0q9DvIzCxoqrudOODXKc30aWf0em3/7CVy5qoB8OIrmRjTBJl6U41Dwy/n16VV9BB Se4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=75YvYvEsvh397a9t/GUbdRglLQeQDw2VgrXGcjLzh6g=; b=OxskayLL3iuhk/d6pJHOMlOaafx5cgSvdkMTMcscwXgIVm0BtluEdM2Hnlgv250xMf MYyEJVwDOoTt5pFSQNolsweMSwqQO5ITfARkyrjEJ/aYAmkoO5EC0EaA7d9dTIkYb+U1 nrPv6DGpOIIHcz7zrwLtmtGEO4eYuQR6tvQXXSFsd5jnD06aCac4ENAPs5wfxSgg2fks NmRXYpxaaiw2mn0qKvP1Hvl5LEPmmNeKDfTAN3PtRwt6d1E9oStNbc8q8afvz13PTAxN HHcA2Z/hA1wgvTNHoaphx9OQ5v9j+gOPto4Se08JndEQpdZDEaGMoUp6P5XHvX5dE9qy XkRw== X-Gm-Message-State: AOAM5325LJJu4DC1/VHEqHscYErx+te9fSuEiT6Jfq/lqThxV+B/6frR keVJVzl/rRGTB45xxiFO8z15zExNPKY= X-Google-Smtp-Source: ABdhPJz3rN3SBIkvGvp7418FwRbIR4sELkcB8T99Mpwm13ScVZlATAx5zCIAUpNA64xRMK5WJPoMbg== X-Received: by 2002:a17:90a:7d05:: with SMTP id g5mr20544865pjl.14.1632754807474; Mon, 27 Sep 2021 08:00:07 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id z19sm17732012pfk.109.2021.09.27.08.00.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 08:00:07 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Andrii Nakryiko , Alexei Starovoitov , Daniel Borkmann , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v5 07/12] libbpf: Resolve invalid weak kfunc calls with imm = 0, off = 0 Date: Mon, 27 Sep 2021 20:29:36 +0530 Message-Id: <20210927145941.1383001-8-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210927145941.1383001-1-memxor@gmail.com> References: <20210927145941.1383001-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2562; h=from:subject; bh=wtMr11ltrcGBXIzDIKBO9mdVxLsLz4SfHdSlEf2aLB0=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhUdxOQjYnkUVAWCr8yujN3XJ9YSo64qV7Mo30r5tv chPJrlOJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYVHcTgAKCRBM4MiGSL8RykQ1D/ 9dOO5YiM6ggRBpU+pApOFFtKbnOXSEzeAxJzstHQ8UIDnQ8OsjCV7JNvjJrguBXQXIEM5kQ8xBNlxK zjef5Je/p01u8XMBFfxjj2UmETvsk+fVLc5IFJquFs7iQpXvHJCnHPhEexYA4BHZ/PRSN0WbwtcFyz 0ieByNFs62zDcJnLVFzxjSrmF+F9XR0Kgpk6ClsrP0ZUQTKKnzfjb1B3/lqJFq/tA7cJiLQMpzYCvx vCfhAWC92ObwnwFbxP8K9o/bnz8rfS+Ou0YdnHcjrkp+KTpd0yOjrfGHmeGRO4PX54zEwGkC2QhSWD 3NS8SjzMA0dgtpy4xNnlXWFFV6Yl8AR4sHIi+GvBWwRE6298cw+pPxRnaE6rm+We2He47hdu2oRP3X GlrUPXMzeqbRvHEZ9un1/iQKrvvzuv5z1rgCjmB9cGJzQYoKGApumWcpKON/ESJ8pmMkOY85qnKY3S cIrx3U2mau4zJzA1Xgtsxaz2scTSlhJNrw1/aEyT8mfNYN5EcddG8+OQey0EeCswldeGRRUnqs37tq fKSMYIbKFlr3GKk9JwwflbEoo0pppW09engOeiwCnNcSuGUHynFmzJO9f3vQqNAVBMGbeSH9Y4R5Vy 0hEl6mGhuygKduzUUqCEDlANTMvgzrp92RQF4lmMO7O/RUyKmv1gwyx8udPQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Preserve these calls as it allows verifier to succeed in loading the program if they are determined to be unreachable after dead code elimination during program load. If not, the verifier will fail at runtime. This is done for ext->is_weak symbols similar to the case for variable ksyms. Acked-by: Andrii Nakryiko Signed-off-by: Kumar Kartikeya Dwivedi --- tools/lib/bpf/libbpf.c | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 0f4d203f926e..4b2d0511c1e7 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -3416,11 +3416,6 @@ static int bpf_object__collect_externs(struct bpf_object *obj) return -ENOTSUP; } } else if (strcmp(sec_name, KSYMS_SEC) == 0) { - if (btf_is_func(t) && ext->is_weak) { - pr_warn("extern weak function %s is unsupported\n", - ext->name); - return -ENOTSUP; - } ksym_sec = sec; ext->type = EXT_KSYM; skip_mods_and_typedefs(obj->btf, t->type, @@ -5369,8 +5364,13 @@ bpf_object__relocate_data(struct bpf_object *obj, struct bpf_program *prog) case RELO_EXTERN_FUNC: ext = &obj->externs[relo->sym_off]; insn[0].src_reg = BPF_PSEUDO_KFUNC_CALL; - insn[0].imm = ext->ksym.kernel_btf_id; - insn[0].off = ext->ksym.offset; + if (ext->is_set) { + insn[0].imm = ext->ksym.kernel_btf_id; + insn[0].off = ext->ksym.offset; + } else { /* unresolved weak kfunc */ + insn[0].imm = 0; + insn[0].off = 0; + } break; case RELO_SUBPROG_ADDR: if (insn[0].src_reg != BPF_PSEUDO_FUNC) { @@ -6719,9 +6719,9 @@ static int bpf_object__resolve_ksym_var_btf_id(struct bpf_object *obj, int id, err; id = find_ksym_btf_id(obj, ext->name, BTF_KIND_VAR, &btf, &mod_btf); - if (id == -ESRCH && ext->is_weak) { - return 0; - } else if (id < 0) { + if (id < 0) { + if (id == -ESRCH && ext->is_weak) + return 0; pr_warn("extern (var ksym) '%s': not found in kernel BTF\n", ext->name); return id; @@ -6774,7 +6774,9 @@ static int bpf_object__resolve_ksym_func_btf_id(struct bpf_object *obj, kfunc_id = find_ksym_btf_id(obj, ext->name, BTF_KIND_FUNC, &btf, &mod_btf); if (kfunc_id < 0) { - pr_warn("extern (func ksym) '%s': not found in kernel BTF\n", + if (kfunc_id == -ESRCH && ext->is_weak) + return 0; + pr_warn("extern (func ksym) '%s': not found in kernel or module BTFs\n", ext->name); return kfunc_id; } From patchwork Mon Sep 27 14:59:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12520101 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95020C433EF for ; Mon, 27 Sep 2021 15:00:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7C79960F4A for ; Mon, 27 Sep 2021 15:00:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235059AbhI0PCQ (ORCPT ); Mon, 27 Sep 2021 11:02:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36394 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235060AbhI0PBx (ORCPT ); Mon, 27 Sep 2021 11:01:53 -0400 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35952C06177A; Mon, 27 Sep 2021 08:00:12 -0700 (PDT) Received: by mail-pl1-x643.google.com with SMTP id n2so11939367plk.12; Mon, 27 Sep 2021 08:00:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JgNyFr51slssP3bhc3yykZKy36rLj3siDcsI+KkBHs4=; b=qPbXuG4bTnnni4lEo7J3b4TZubygDchx6c/mHMVPjjwqqevV40RG7FjN+8BrXAgiWO 8KKSj6DjCD8iim722xL1YK2kMHY7imWzIhwV/130B7rmR+JnfHBYGnq9m+ljEfJd2mTo xcCsipTe0DaKtvEF6Bw/rtC7ZOCvVmH8sZvG3GC3Nm6zvCVMNKTQS7WMsoxOLpSKve5d V+OtWZT4YIcm/2trSDd2tblUs6boD8+x50p1elFWgD4SP6ALjSsFZWK/0143kj9UtPKr 0zREsOn6krzn7vDInW3IeaSCIpHeiCJbTNsPL+F/FMhh1hycL2hLyrtCBUvJ/uTzYzrN TuVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JgNyFr51slssP3bhc3yykZKy36rLj3siDcsI+KkBHs4=; b=icZBh23Q6zW/mR0er/TLlzCogUXbqaknba15vLSU0eDQch7zpWdLw0b0t15NH2JR9p Zb6oSVJ/DHLjRJhvaNgyx9rbbRsqBiap03VWLbUOremI2BRudTK+anIikxvPbtCJiKU6 BpjIphrZv3jTSyVp1AzLy4xKIZX2KCnE0pebDzLP4cXJoe8FkhFdY9mWlfOPhoFllD87 6dJAzqPDB8/whxQ8F4V9t5isYVzQ3BVlyNN9z6OvILxea4Yy1oerHa6Gnp2AyRe0MDOR cm5EZQuaU5Tp7V/cbIjxFBDQj6X6Qty5gBPpLMAkEdFAc8zbPxt1MEK7itbvq55XZzZZ 7Ubw== X-Gm-Message-State: AOAM532sDglCV9ts5iHNXZaMv81LWoNdQMvGUwN/vyiijenaNgj/iMNW T2lw7Ih5Z9SRz+851ywfT2HaFli3TcI= X-Google-Smtp-Source: ABdhPJxp1HIVz3BceARgXwzQZSfy3TNdv4hzf+EbV/5qck3DLWTozynqWe2eFB42jwByOvBRZRzmew== X-Received: by 2002:a17:902:c184:b0:13e:2e49:8a92 with SMTP id d4-20020a170902c18400b0013e2e498a92mr284757pld.2.1632754810714; Mon, 27 Sep 2021 08:00:10 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id g27sm18214183pfk.173.2021.09.27.08.00.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 08:00:10 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v5 08/12] libbpf: Make gen_loader data aligned. Date: Mon, 27 Sep 2021 20:29:37 +0530 Message-Id: <20210927145941.1383001-9-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210927145941.1383001-1-memxor@gmail.com> References: <20210927145941.1383001-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1157; i=memxor@gmail.com; h=from:subject; bh=BpTAouLmASMkUTTW2rl1vdawRxwdtufN6uz0MPaNZ7o=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhUdxPnHrX6N9UhfkxogwpVC7f85lVGqZtHZurK6Xl H0Khi6iJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYVHcTwAKCRBM4MiGSL8Ryn/JEA CK113XGjkFqRU4jQYtao3vB8BLO1tw7GBpl1fxYOB9GGUIYL1Zo9YhY93dH1Ok4K2Ff3ix9HqB3/Y3 TJd38C5uprNpTQnmi7TokRgQHOb+jkSpckLrqPQAiUq0UlgV4wmOQj1+kxkNK6LU+u1hkvIxUV0bFP d4RB1FGpnK4dIPtP63v5W32g7h2vLKSniu/GkJonV85AdgOzcwMIcZeSG+4Ly9sPjpVWh3rKoOzmXc A1LlcdbjbCswocu8v7BKrCojZ04FLop8PBY1GiKb62H6hyBNAaWN39M4ZGlUwOaePOgsVkXWqfWp2S zqFzIpqyKWtcpJMW90HW+CgzfGrwAO37pqjw1V/ijm73stRAZqowCRCOmlaOEQSKo3EgObutwxBsP3 TOqE81an4Y6jGuU+QCYceG19u97IaQARTMQi2ChFpOQUno0UXc7FPFYIjKfI38DIBBQs9sDI4R/vyJ xbmgOq2PmaiD35fkpJSM8913wQl542qmY+UVzoDef9AsRJzg3FwPaKg8lJJVU04QZA54+aIe2iCRMz NaOk1dDu8YuL+16hjTk2AREjIDICI5UZIZo6mGDsAqLgciahIY/Ne2GyLLrJeMGrXH4xhWlQoVQb+V 8ZNdBwUgpfnPFrnHMEe27Up9gJDIJoEfytqfmUfA2Lpl3c56CsmRZmHUn5pw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net From: Alexei Starovoitov Align gen_loader data to 8 byte boundary to make sure union bpf_attr, bpf_insns and other structs are aligned. Signed-off-by: Alexei Starovoitov --- tools/lib/bpf/gen_loader.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c index 8df718a6b142..80087b13877f 100644 --- a/tools/lib/bpf/gen_loader.c +++ b/tools/lib/bpf/gen_loader.c @@ -5,6 +5,7 @@ #include #include #include +#include #include "btf.h" #include "bpf.h" #include "libbpf.h" @@ -135,13 +136,17 @@ void bpf_gen__init(struct bpf_gen *gen, int log_level) static int add_data(struct bpf_gen *gen, const void *data, __u32 size) { + __u32 size8 = roundup(size, 8); + __u64 zero = 0; void *prev; - if (realloc_data_buf(gen, size)) + if (realloc_data_buf(gen, size8)) return 0; prev = gen->data_cur; memcpy(gen->data_cur, data, size); gen->data_cur += size; + memcpy(gen->data_cur, &zero, size8 - size); + gen->data_cur += size8 - size; return prev - gen->data_start; } From patchwork Mon Sep 27 14:59:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12520105 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F627C433F5 for ; Mon, 27 Sep 2021 15:00:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3A1CD60EE2 for ; Mon, 27 Sep 2021 15:00:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234995AbhI0PCR (ORCPT ); Mon, 27 Sep 2021 11:02:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235018AbhI0PBy (ORCPT ); Mon, 27 Sep 2021 11:01:54 -0400 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C3C78C061575; Mon, 27 Sep 2021 08:00:14 -0700 (PDT) Received: by mail-pg1-x544.google.com with SMTP id s11so18013075pgr.11; Mon, 27 Sep 2021 08:00:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sLvD35TQAg5fScK2rABT6UWa0KNMaDbRArip1oGfUE0=; b=NneEMaYU6ACjv6QVAKlN0aACQJu5I5U0fxv+hUO8+MmINXgS9RdZA447pv4gn4E0tr h3A0KufHQ8zS18eLLjdzfZCAvv1rq/+ZaAczyzRD7FquyJ2HNF3WgJ87XjbtY2vdUU3I zyXFWgr9tkPB3WWq+huiDRH5ODW2/dcvV+ohVgthD/e3PWXCz5lHJjqVO4xIzhAmU2CC 7CPK2OBlrIInL8NFOyLG8swss5O5wGUAm/aKZzyZIBsDpClKF5d4Nl+GnuuYRl/DkhSF vHVHp5NcIVzGJ2B18keBuaFghMa9/zy2Oy8Z7vjpTT1GJWH9c+YZOl05Iy09nP679X43 IZnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sLvD35TQAg5fScK2rABT6UWa0KNMaDbRArip1oGfUE0=; b=uzgShVC14owOI0M/Wkx0sV7r3jKyPub0nAU2GTQLoWDCj+0NaR4Q3icm4SIGLyEJ0X J/U4sJT/yjN1Iqg4/JVlbFN6JUjvyaon1AgaDsfkqdZcOMwUNKjDRfOczqEx5Yj/04Ld u57WV00QB7cH2oIgIB9cGvBb4Mg6zMxF7c28sSYZqeQmghm7CPpEK8Toq8GCIZjRdOJu 3P2KOky0MtuBA1TSCtaMcZfgh563QHwf8H8U3c8NolovGdyw2cdYFPPCbtX7exSd74ld Nc2iYip4yBClabhdEd8YhAVcrNk6fjy5ZfrZQcSWeS8Ty/uOgpRPqvkZikd1sYIxgCG4 1nbw== X-Gm-Message-State: AOAM532oxnjm5B7h7cTvgEcMfkdMdLWpbYUqM49eZJYUax7uA3/hs8Un /ffGTstlFC2gfJGlUQqArSbXuQXFwg0= X-Google-Smtp-Source: ABdhPJyt7NTHRrsm1kPBDhKnZYluStdhnGC+pdq2Kw8u1rcJsDAQIk/VB2NIEBkG7doexPLE0i6PFQ== X-Received: by 2002:a62:5f03:0:b0:445:38d5:98bd with SMTP id t3-20020a625f03000000b0044538d598bdmr320324pfb.81.1632754813861; Mon, 27 Sep 2021 08:00:13 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id p17sm12381833pfo.9.2021.09.27.08.00.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 08:00:13 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v5 09/12] libbpf: Update gen_loader to emit BTF_KIND_FUNC relocations Date: Mon, 27 Sep 2021 20:29:38 +0530 Message-Id: <20210927145941.1383001-10-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210927145941.1383001-1-memxor@gmail.com> References: <20210927145941.1383001-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=16905; h=from:subject; bh=fnnjeIgaSooClYn9mVCKrfuzISQ0gfK/+IjxZ4XVMlg=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhUdxP0/nVWmPtVIlQPIL+j5jLvImsYH76E9JzPLCE WbfQ1iOJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYVHcTwAKCRBM4MiGSL8Ryv6vEA CLF5do8D7ojJPGa1SbRAPhWYW+iyzRsWfc1tUHbba9w90Yzt8jFRN+L53sE85AHxPrFC5rlRet647T tZEIS39W2F6N0H60FLIJRiyijVZF/7kx6lABjEglEBeieGS46W5DIz9wSWVBZX9gxNy5jAKrglJE6w E1AQOg9yeuZ0vPhIdHuE6PTvwpFJpQ0g+2gvmJGYRCvFqJK3FhGPsmb+61VT8Yr6TCoeIfrKrdxRCz VialSsnZn4GrZFsn//v/WiLti7ymd2u44ZCN8OfFRzBYAlU17I9/RRodgYI7bqbzytzG0o/Kndhs26 R7gOk0Y094tCR68K0YEcdIGMpT7oBwQOztZmaN37eBq2AIfT9DEV9GvbP+T7wK6wdwN+rDzTopOjIy +db3u3YpaYiVwxaa/dMraKTpzig1qDIaNqRyDTISxd0tfgHHpOua6cE2LmqVzLLWFoldO2c38/FvYA 8JJKA/xOhLrApZVBl0kx9kcWyjAcnBXEC/9IQOcDeDTIha1Pv2DkrwPhEctdU2O8KcEna/JyI9qVk8 nQDGPOCvyAaXBt/jPXXOT6uLIYJysBRnV1916mqJMi9ZRLbYri/XLR+7l9/5fxkJGB0SUrVllew/0C +d3RIshOhUBllCXgd4673wbJQqSreGpL/vkuvUIYujnGVZrb2Uvrg4tRpZ3w== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This change updates the BPF syscall loader to relocate BTF_KIND_FUNC relocations, with support for weak kfunc relocations. The general idea is to move map_fds to loader map, and also use the data for storing kfunc BTF fds. Since both reuse the fd_array parameter, they need to be kept together. For map_fds, we reserve MAX_USED_MAPS slots in a region, and for kfunc, we reserve MAX_KFUNC_DESCS. This is done so that insn->off has more chances of being <= INT16_MAX than treating data map as a sparse array and adding fd as needed. When the MAX_KFUNC_DESCS limit is reached, we fall back to the sparse array model, so that as long as it does remain <= INT16_MAX, we pass an index relative to the start of fd_array. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/lib/bpf/bpf_gen_internal.h | 14 +- tools/lib/bpf/gen_loader.c | 243 +++++++++++++++++++++++++++---- tools/lib/bpf/libbpf.c | 8 +- 3 files changed, 228 insertions(+), 37 deletions(-) diff --git a/tools/lib/bpf/bpf_gen_internal.h b/tools/lib/bpf/bpf_gen_internal.h index 615400391e57..b278293599f6 100644 --- a/tools/lib/bpf/bpf_gen_internal.h +++ b/tools/lib/bpf/bpf_gen_internal.h @@ -7,6 +7,13 @@ struct ksym_relo_desc { const char *name; int kind; int insn_idx; + bool is_weak; +}; + +struct kfunc_desc { + const char *name; + int ref; + int off; }; struct bpf_gen { @@ -24,6 +31,10 @@ struct bpf_gen { int relo_cnt; char attach_target[128]; int attach_kind; + struct kfunc_desc *kfuncs; + __u32 nr_kfuncs; + int fd_array; + int nr_fd_array; }; void bpf_gen__init(struct bpf_gen *gen, int log_level); @@ -36,6 +47,7 @@ void bpf_gen__prog_load(struct bpf_gen *gen, struct bpf_prog_load_params *load_a void bpf_gen__map_update_elem(struct bpf_gen *gen, int map_idx, void *value, __u32 value_size); void bpf_gen__map_freeze(struct bpf_gen *gen, int map_idx); void bpf_gen__record_attach_target(struct bpf_gen *gen, const char *name, enum bpf_attach_type type); -void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, int kind, int insn_idx); +void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, bool is_weak, int kind, + int insn_idx); #endif diff --git a/tools/lib/bpf/gen_loader.c b/tools/lib/bpf/gen_loader.c index 80087b13877f..4656c6412601 100644 --- a/tools/lib/bpf/gen_loader.c +++ b/tools/lib/bpf/gen_loader.c @@ -16,6 +16,8 @@ #define MAX_USED_MAPS 64 #define MAX_USED_PROGS 32 +#define MAX_KFUNC_DESCS 256 +#define MAX_FD_ARRAY_SZ (MAX_USED_PROGS + MAX_KFUNC_DESCS) /* The following structure describes the stack layout of the loader program. * In addition R6 contains the pointer to context. @@ -30,7 +32,6 @@ */ struct loader_stack { __u32 btf_fd; - __u32 map_fd[MAX_USED_MAPS]; __u32 prog_fd[MAX_USED_PROGS]; __u32 inner_map_fd; }; @@ -143,13 +144,58 @@ static int add_data(struct bpf_gen *gen, const void *data, __u32 size) if (realloc_data_buf(gen, size8)) return 0; prev = gen->data_cur; - memcpy(gen->data_cur, data, size); + if (data) + memcpy(gen->data_cur, data, size); gen->data_cur += size; memcpy(gen->data_cur, &zero, size8 - size); gen->data_cur += size8 - size; return prev - gen->data_start; } +/* Get index for map_fd/btf_fd slot in reserved fd_array, or in data relative + * to start of fd_array. Caller can decide if it is usable or not. + */ +static int fd_array_init(struct bpf_gen *gen) +{ + if (!gen->fd_array) + gen->fd_array = add_data(gen, NULL, MAX_FD_ARRAY_SZ * sizeof(int)); + return gen->fd_array; +} + +static int add_map_fd(struct bpf_gen *gen) +{ + if (!fd_array_init(gen)) + return 0; + if (gen->nr_maps == MAX_USED_MAPS) { + pr_warn("Total maps exceeds %d\n", MAX_USED_MAPS); + gen->error = -E2BIG; + return 0; + } + return gen->nr_maps++; +} + +static int add_kfunc_btf_fd(struct bpf_gen *gen) +{ + int cur; + + if (!fd_array_init(gen)) + return 0; + if (gen->nr_fd_array == MAX_KFUNC_DESCS) { + cur = add_data(gen, NULL, sizeof(int)); + if (!cur) + return 0; + return (cur - gen->fd_array) / sizeof(int); + } + return MAX_USED_MAPS + gen->nr_fd_array++; +} + +static int blob_fd_array_off(struct bpf_gen *gen, int index) +{ + if (!gen->fd_array) + return 0; + return gen->fd_array + (index * sizeof(int)); +} + static int insn_bytes_to_bpf_size(__u32 sz) { switch (sz) { @@ -171,14 +217,22 @@ static void emit_rel_store(struct bpf_gen *gen, int off, int data) emit(gen, BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 0)); } -/* *(u64 *)(blob + off) = (u64)(void *)(%sp + stack_off) */ -static void emit_rel_store_sp(struct bpf_gen *gen, int off, int stack_off) +static void move_blob2blob(struct bpf_gen *gen, int off, int size, int blob_off) { - emit(gen, BPF_MOV64_REG(BPF_REG_0, BPF_REG_10)); - emit(gen, BPF_ALU64_IMM(BPF_ADD, BPF_REG_0, stack_off)); + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_7, BPF_PSEUDO_MAP_IDX_VALUE, + 0, 0, 0, blob_off)); + emit(gen, BPF_LDX_MEM(insn_bytes_to_bpf_size(size), BPF_REG_0, BPF_REG_7, 0)); emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, 0, 0, 0, off)); - emit(gen, BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, 0)); + emit(gen, BPF_STX_MEM(insn_bytes_to_bpf_size(size), BPF_REG_1, BPF_REG_0, 0)); +} + +static void move_blob2ctx(struct bpf_gen *gen, int ctx_off, int size, int blob_off) +{ + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, + 0, 0, 0, blob_off)); + emit(gen, BPF_LDX_MEM(insn_bytes_to_bpf_size(size), BPF_REG_0, BPF_REG_1, 0)); + emit(gen, BPF_STX_MEM(insn_bytes_to_bpf_size(size), BPF_REG_6, BPF_REG_0, ctx_off)); } static void move_ctx2blob(struct bpf_gen *gen, int off, int size, int ctx_off, @@ -326,11 +380,11 @@ int bpf_gen__finish(struct bpf_gen *gen) offsetof(struct bpf_prog_desc, prog_fd), 4, stack_off(prog_fd[i])); for (i = 0; i < gen->nr_maps; i++) - move_stack2ctx(gen, - sizeof(struct bpf_loader_ctx) + - sizeof(struct bpf_map_desc) * i + - offsetof(struct bpf_map_desc, map_fd), 4, - stack_off(map_fd[i])); + move_blob2ctx(gen, + sizeof(struct bpf_loader_ctx) + + sizeof(struct bpf_map_desc) * i + + offsetof(struct bpf_map_desc, map_fd), 4, + blob_fd_array_off(gen, i)); emit(gen, BPF_MOV64_IMM(BPF_REG_0, 0)); emit(gen, BPF_EXIT_INSN()); pr_debug("gen: finish %d\n", gen->error); @@ -390,7 +444,7 @@ void bpf_gen__map_create(struct bpf_gen *gen, { int attr_size = offsetofend(union bpf_attr, btf_vmlinux_value_type_id); bool close_inner_map_fd = false; - int map_create_attr; + int map_create_attr, idx; union bpf_attr attr; memset(&attr, 0, attr_size); @@ -467,9 +521,11 @@ void bpf_gen__map_create(struct bpf_gen *gen, gen->error = -EDOM; /* internal bug */ return; } else { - emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_10, BPF_REG_7, - stack_off(map_fd[map_idx]))); - gen->nr_maps++; + /* add_map_fd does gen->nr_maps++ */ + idx = add_map_fd(gen); + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_1, BPF_PSEUDO_MAP_IDX_VALUE, + 0, 0, 0, blob_fd_array_off(gen, idx))); + emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_7, 0)); } if (close_inner_map_fd) emit_sys_close_stack(gen, stack_off(inner_map_fd)); @@ -511,8 +567,8 @@ static void emit_find_attach_target(struct bpf_gen *gen) */ } -void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, int kind, - int insn_idx) +void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, bool is_weak, + int kind, int insn_idx) { struct ksym_relo_desc *relo; @@ -524,11 +580,119 @@ void bpf_gen__record_extern(struct bpf_gen *gen, const char *name, int kind, gen->relos = relo; relo += gen->relo_cnt; relo->name = name; + relo->is_weak = is_weak; relo->kind = kind; relo->insn_idx = insn_idx; gen->relo_cnt++; } +/* returns existing kfunc_desc with ref incremented, or inserts a new one */ +static struct kfunc_desc *get_kfunc_desc(struct bpf_gen *gen, const char *name) +{ + struct kfunc_desc *kdesc; + + for (int i = 0; i < gen->nr_kfuncs; i++) { + if (!strcmp(gen->kfuncs[i].name, name)) { + gen->kfuncs[i].ref++; + return &gen->kfuncs[i]; + } + } + kdesc = libbpf_reallocarray(gen->kfuncs, gen->nr_kfuncs + 1, sizeof(*kdesc)); + if (!kdesc) { + gen->error = -ENOMEM; + return NULL; + } + gen->kfuncs = kdesc; + kdesc = &gen->kfuncs[gen->nr_kfuncs++]; + kdesc->name = name; + kdesc->ref = 1; + kdesc->off = 0; + return kdesc; +} + +/* Expects: + * BPF_REG_0 - pointer to instruction + * BPF_REG_7 - return value of bpf_btf_find_by_name_kind + * + * We need to reuse BTF fd for same symbol otherwise each relocation takes a new + * index, while kernel limits total kfunc BTFs to 256. For duplicate symbols, + * this would mean a new BTF fd index for each entry. By pairing symbol name + * with index, we get the insn->imm, insn->off pairing that kernel uses for + * kfunc_tab, which becomes the effective limit even though all of them may + * share same index in fd_array (such that kfunc_btf_tab has 1 element). + */ +static void emit_relo_kfunc_btf(struct bpf_gen *gen, struct ksym_relo_desc *relo, int insn) +{ + struct kfunc_desc *kdesc; + int btf_fd_idx; + + kdesc = get_kfunc_desc(gen, relo->name); + if (!kdesc) + return; + + btf_fd_idx = kdesc->ref > 1 ? kdesc->off : add_kfunc_btf_fd(gen); + if (btf_fd_idx > INT16_MAX) { + pr_warn("BTF fd off %d for kfunc %s exceeds INT16_MAX, cannot process relocation\n", + btf_fd_idx, relo->name); + gen->error = -E2BIG; + return; + } + /* load slot pointer */ + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_8, BPF_PSEUDO_MAP_IDX_VALUE, + 0, 0, 0, blob_fd_array_off(gen, btf_fd_idx))); + /* Try to map one insn->off to one insn->imm */ + if (kdesc->ref > 1) { + emit(gen, BPF_MOV64_REG(BPF_REG_9, BPF_REG_7)); + goto skip_btf_fd; + } else { + /* cannot use index == 0 */ + if (!btf_fd_idx) { + btf_fd_idx = add_kfunc_btf_fd(gen); + /* shift to next slot */ + emit(gen, BPF_ALU64_IMM(BPF_ADD, BPF_REG_8, sizeof(int))); + } + kdesc->off = btf_fd_idx; + } + + /* set a default value */ + emit(gen, BPF_ST_MEM(BPF_W, BPF_REG_8, 0, 0)); + emit(gen, BPF_MOV64_REG(BPF_REG_9, BPF_REG_7)); + /* store BTF fd if ret < 0 */ + emit(gen, BPF_JMP_IMM(BPF_JSLT, BPF_REG_7, 0, 3)); + emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_7, 32)); + /* store BTF fd in slot */ + emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_8, BPF_REG_7, 0)); + emit(gen, BPF_MOV64_REG(BPF_REG_7, BPF_REG_9)); +skip_btf_fd: + /* remember BTF fd to skip insn->off store for vmlinux case */ + emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_9, 32)); + /* set a default value */ + emit(gen, BPF_ST_MEM(BPF_H, BPF_REG_0, offsetof(struct bpf_insn, off), 0)); + /* skip insn->off store if ret < 0 */ + emit(gen, BPF_JMP_IMM(BPF_JSLT, BPF_REG_7, 0, 2)); + /* skip if vmlinux BTF */ + emit(gen, BPF_JMP_IMM(BPF_JEQ, BPF_REG_9, 0, 1)); + /* store index into insn[insn_idx].off */ + emit(gen, BPF_ST_MEM(BPF_H, BPF_REG_0, offsetof(struct bpf_insn, off), btf_fd_idx)); + /* close fd that we skipped storing in fd_array */ + if (kdesc->ref > 1) { + emit(gen, BPF_MOV64_REG(BPF_REG_1, BPF_REG_9)); + __emit_sys_close(gen); + } + if (!gen->log_level) + return; + /* reload register 0, overwritten by sys_close */ + if (kdesc->ref > 1) + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_0, BPF_PSEUDO_MAP_IDX_VALUE, + 0, 0, 0, insn)); + emit(gen, BPF_LDX_MEM(BPF_W, BPF_REG_7, BPF_REG_0, + offsetof(struct bpf_insn, imm))); + emit(gen, BPF_LDX_MEM(BPF_H, BPF_REG_8, BPF_REG_0, + offsetof(struct bpf_insn, off))); + debug_regs(gen, BPF_REG_7, BPF_REG_8, "sym (%s:count=%d): imm: %%d, off: %%d", + relo->name, kdesc->ref); +} + static void emit_relo(struct bpf_gen *gen, struct ksym_relo_desc *relo, int insns) { int name, insn, len = strlen(relo->name) + 1; @@ -544,18 +708,24 @@ static void emit_relo(struct bpf_gen *gen, struct ksym_relo_desc *relo, int insn emit(gen, BPF_EMIT_CALL(BPF_FUNC_btf_find_by_name_kind)); emit(gen, BPF_MOV64_REG(BPF_REG_7, BPF_REG_0)); debug_ret(gen, "find_by_name_kind(%s,%d)", relo->name, relo->kind); - emit_check_err(gen); + /* if not weak kfunc, emit err check */ + if (relo->kind != BTF_KIND_FUNC || !relo->is_weak) + emit_check_err(gen); + insn = insns + sizeof(struct bpf_insn) * relo->insn_idx; + emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_0, BPF_PSEUDO_MAP_IDX_VALUE, 0, 0, 0, insn)); + /* set a default value */ + emit(gen, BPF_ST_MEM(BPF_W, BPF_REG_0, offsetof(struct bpf_insn, imm), 0)); + /* skip success case store if ret < 0 */ + emit(gen, BPF_JMP_IMM(BPF_JSLT, BPF_REG_7, 0, 1)); /* store btf_id into insn[insn_idx].imm */ - insn = insns + sizeof(struct bpf_insn) * relo->insn_idx + - offsetof(struct bpf_insn, imm); - emit2(gen, BPF_LD_IMM64_RAW_FULL(BPF_REG_0, BPF_PSEUDO_MAP_IDX_VALUE, - 0, 0, 0, insn)); - emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7, 0)); + emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7, offsetof(struct bpf_insn, imm))); if (relo->kind == BTF_KIND_VAR) { /* store btf_obj_fd into insn[insn_idx + 1].imm */ emit(gen, BPF_ALU64_IMM(BPF_RSH, BPF_REG_7, 32)); emit(gen, BPF_STX_MEM(BPF_W, BPF_REG_0, BPF_REG_7, - sizeof(struct bpf_insn))); + sizeof(struct bpf_insn) + offsetof(struct bpf_insn, imm))); + } else if (relo->kind == BTF_KIND_FUNC) { + emit_relo_kfunc_btf(gen, relo, insn); } } @@ -571,6 +741,16 @@ static void cleanup_relos(struct bpf_gen *gen, int insns) { int i, insn; + for (i = 0; i < gen->nr_kfuncs; i++) { + emit_sys_close_blob(gen, blob_fd_array_off(gen, gen->kfuncs[i].off)); + if (gen->kfuncs[i].off < MAX_FD_ARRAY_SZ) + gen->nr_fd_array--; + } + if (gen->nr_kfuncs) { + free(gen->kfuncs); + gen->nr_kfuncs = 0; + gen->kfuncs = NULL; + } for (i = 0; i < gen->relo_cnt; i++) { if (gen->relos[i].kind != BTF_KIND_VAR) continue; @@ -637,9 +817,8 @@ void bpf_gen__prog_load(struct bpf_gen *gen, /* populate union bpf_attr with a pointer to line_info */ emit_rel_store(gen, attr_field(prog_load_attr, line_info), line_info); - /* populate union bpf_attr fd_array with a pointer to stack where map_fds are saved */ - emit_rel_store_sp(gen, attr_field(prog_load_attr, fd_array), - stack_off(map_fd[0])); + /* populate union bpf_attr fd_array with a pointer to data where map_fds are saved */ + emit_rel_store(gen, attr_field(prog_load_attr, fd_array), gen->fd_array); /* populate union bpf_attr with user provided log details */ move_ctx2blob(gen, attr_field(prog_load_attr, log_level), 4, @@ -706,8 +885,8 @@ void bpf_gen__map_update_elem(struct bpf_gen *gen, int map_idx, void *pvalue, emit(gen, BPF_EMIT_CALL(BPF_FUNC_copy_from_user)); map_update_attr = add_data(gen, &attr, attr_size); - move_stack2blob(gen, attr_field(map_update_attr, map_fd), 4, - stack_off(map_fd[map_idx])); + move_blob2blob(gen, attr_field(map_update_attr, map_fd), 4, + blob_fd_array_off(gen, map_idx)); emit_rel_store(gen, attr_field(map_update_attr, key), key); emit_rel_store(gen, attr_field(map_update_attr, value), value); /* emit MAP_UPDATE_ELEM command */ @@ -725,8 +904,8 @@ void bpf_gen__map_freeze(struct bpf_gen *gen, int map_idx) memset(&attr, 0, attr_size); pr_debug("gen: map_freeze: idx %d\n", map_idx); map_freeze_attr = add_data(gen, &attr, attr_size); - move_stack2blob(gen, attr_field(map_freeze_attr, map_fd), 4, - stack_off(map_fd[map_idx])); + move_blob2blob(gen, attr_field(map_freeze_attr, map_fd), 4, + blob_fd_array_off(gen, map_idx)); /* emit MAP_FREEZE command */ emit_sys_bpf(gen, BPF_MAP_FREEZE, map_freeze_attr, attr_size); debug_ret(gen, "map_freeze"); diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 4b2d0511c1e7..b7c11eb40766 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -6268,12 +6268,12 @@ static int bpf_program__record_externs(struct bpf_program *prog) ext->name); return -ENOTSUP; } - bpf_gen__record_extern(obj->gen_loader, ext->name, BTF_KIND_VAR, - relo->insn_idx); + bpf_gen__record_extern(obj->gen_loader, ext->name, ext->is_weak, + BTF_KIND_VAR, relo->insn_idx); break; case RELO_EXTERN_FUNC: - bpf_gen__record_extern(obj->gen_loader, ext->name, BTF_KIND_FUNC, - relo->insn_idx); + bpf_gen__record_extern(obj->gen_loader, ext->name, ext->is_weak, + BTF_KIND_FUNC, relo->insn_idx); break; default: continue; From patchwork Mon Sep 27 14:59:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12520103 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C940BC43217 for ; Mon, 27 Sep 2021 15:00:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B1CD160FF2 for ; Mon, 27 Sep 2021 15:00:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234945AbhI0PCT (ORCPT ); Mon, 27 Sep 2021 11:02:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36318 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234996AbhI0PCC (ORCPT ); Mon, 27 Sep 2021 11:02:02 -0400 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 03DBEC061740; Mon, 27 Sep 2021 08:00:18 -0700 (PDT) Received: by mail-pg1-x541.google.com with SMTP id s75so1540117pgs.5; Mon, 27 Sep 2021 08:00:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HAo200QE9/huK+nWNKe5juvI7U/KIFLH0A3znfnQYDM=; b=G4uzPoyz5U7he4rqhKyzV9PdfXoNM1wZUs9RQGDfCHDLwAzq/vYuMWIBC39lPiDEzY GfA2kFxAnm2afuHzfq19koDb0fCItoQ9fdYdixMtSBnkW1d0UW0xB+NzUpolJbMQx2DI oYi03lP7Trn0sMQBjao+lbE6Ro9rG+5HnsAyU6UUqoaF96FNddLEWS85DGzPrxsA0Hzz GiQa7ZXaOBDd0mSibtEvYgQok9OKVX4ZFjxRa1uSYvaqb8wY/pUrR3Eus3XzvN0cgY85 dJAEYlba6JhZ/TywAcziGWSZT7lA3BoVgMUqbt0xLJHQDBv4WO3xWYhFppNhv7lFtDky xLFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HAo200QE9/huK+nWNKe5juvI7U/KIFLH0A3znfnQYDM=; b=olBX06XC+eZxmTc3Ty6UqAwX2gYZFMpWDlKYcbgHi4hZQ3lAfGvJI6WCvwvkSycpGF AsP6At5t2oxzcapyM+pJ7wAOvosRUYrkYhDJRkARwnLqIkQWJlZI6PkdrPKjJzihLSkO LdyUxbdmnPb5DCLVpMxu/ET3pxk2Ds1YBFn4fknwdZQd+5wvwQ5eobnUUkksHu2rh9IJ PdfK1z3mpI4OwpTkcZnImpt1ZLo+CGaW4s2YDxkcSjkKHvY0N69iAMneOkteYVmf/9pY RYbs1cPmdz4N9TA3ki9MV90khLdpwbKQfY42mIwEUPAgr81FUOGuOr0MEe28WRlke1ms H89Q== X-Gm-Message-State: AOAM533zna6GetSJh8gRNvkXO5aCDyEZmUrYwOuUBfOl7kNguuRlsRO6 hlrppR7p8KYvhYSWlGu9nzOpc45GYRo= X-Google-Smtp-Source: ABdhPJyqk7wr4DiStlyvCNzbEhOxLauKxA0qxSBVRSKDs33S0cxej4mdoOkpEvKpy6Ril6TGG/cksA== X-Received: by 2002:a62:2587:0:b0:44b:2d81:8520 with SMTP id l129-20020a622587000000b0044b2d818520mr100916pfl.43.1632754817176; Mon, 27 Sep 2021 08:00:17 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id x13sm17450357pfp.133.2021.09.27.08.00.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 08:00:16 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v5 10/12] libbpf: Fix skel_internal.h to set errno on loader retval < 0 Date: Mon, 27 Sep 2021 20:29:39 +0530 Message-Id: <20210927145941.1383001-11-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210927145941.1383001-1-memxor@gmail.com> References: <20210927145941.1383001-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1647; h=from:subject; bh=PVTKS+rfPpxdSU9d74iSymeJVwfhEWVxpXAvuJFJ9qs=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhUdxPrACVEi7CiY1lhZ+7KHtZuFVP74ebqAo6CHjW 2E6erlWJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYVHcTwAKCRBM4MiGSL8RyvLkD/ 44fEfgf6NG+0oe9IJLBj4cCxNZ5Zsfu2h0U4i2h24cw6PYDoSzrGjwvnMRsEDfUQliloPhwU5zT/zI J+U6jFNkxQ1WfCKksOHKbDaiGdWHxFAr9OzqQuIs6JD5Dvla9FntOa/aCPP2eAl54gd2E0NOt9li+J g2phHQhrofK690XQJRmb5302zDf/fHa6LX+nefl8bxVRWQuYx4zSOsEy7lHYqtXyHwTEPYrEBf43Wt +IoRpkWeEM88z2ciLkeQhMhghXtI4kvCX4+uMwgpNDQ5mthpV1Ch+PaPZAmYcMqlD5x2c3938ZIRdA YC5Sf2iTpN++d9CC239Io112EAW0cahlQw9GlI9uFJR9bITdb5BrYeisJveOOphy8JAbEd7zCHDkDa iiu8Ue3uKDvJry6VY2baeOzYYOnNfH3PtjIwqgywDoPfVATRyp3FskJFLgYvNcx9LW5b+yGHyqOf9P Dy/te3xFSEeMEb28PcM7DJBuJ7TuUz+V8c9qDSziXB2WbwO3cbpiXmiOjPxZpLp/vmGAgsw5ebUzAP /7sMP49qhEmUwI175tqWGG86CrmA7Lg15Yl3pf9FUQYNf63AtaR/JdoX74UOpQOeqmejyKL6s547+o yTNpLlfMic+rl43Fuuvp0Drk0rPhKm6xi6oRUs31gLlfX4VlEc+IpazALcqg== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net When the loader indicates an internal error (result of a checked bpf system call), it returns the result in attr.test.retval. However, tests that rely on ASSERT_OK_PTR on NULL (returned from light skeleton) may miss that NULL denotes an error if errno is set to 0. This would result in skel pointer being NULL, while ASSERT_OK_PTR returning 1, leading to a SEGV on dereference of skel, because libbpf_get_error relies on the assumption that errno is always set in case of error for ptr == NULL. In particular, this was observed for the ksyms_module test. When executed using `./test_progs -t ksyms`, prior tests manipulated errno and the test didn't crash when it failed at ksyms_module load, while using `./test_progs -t ksyms_module` crashed due to errno being untouched. Fixes: 67234743736a (libbpf: Generate loader program out of BPF ELF file.) Signed-off-by: Kumar Kartikeya Dwivedi --- tools/lib/bpf/skel_internal.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tools/lib/bpf/skel_internal.h b/tools/lib/bpf/skel_internal.h index b22b50c1b173..9cf66702fa8d 100644 --- a/tools/lib/bpf/skel_internal.h +++ b/tools/lib/bpf/skel_internal.h @@ -105,10 +105,12 @@ static inline int bpf_load_and_run(struct bpf_load_and_run_opts *opts) err = skel_sys_bpf(BPF_PROG_RUN, &attr, sizeof(attr)); if (err < 0 || (int)attr.test.retval < 0) { opts->errstr = "failed to execute loader prog"; - if (err < 0) + if (err < 0) { err = -errno; - else + } else { err = (int)attr.test.retval; + errno = -err; + } goto out; } err = 0; From patchwork Mon Sep 27 14:59:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12520107 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2370C433EF for ; Mon, 27 Sep 2021 15:00:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DE61160F4A for ; Mon, 27 Sep 2021 15:00:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235031AbhI0PC0 (ORCPT ); Mon, 27 Sep 2021 11:02:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36378 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235043AbhI0PCK (ORCPT ); Mon, 27 Sep 2021 11:02:10 -0400 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 86BD9C06176D; Mon, 27 Sep 2021 08:00:21 -0700 (PDT) Received: by mail-pf1-x444.google.com with SMTP id g2so12014859pfc.6; Mon, 27 Sep 2021 08:00:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jJYGi1zkIeuw27rkatjXNG5PEkuC/aMccRb7Npsrk9s=; b=q5dk9iQ7iCsM/K/R7AOh3Xh+VKGOsRc+XWSHpmZv7Caq0zi96dmGo/zw3DT8i4i+BM K16jgFMsa4y+jTHux/zPYXYb8oLU/7IgmOIjUjg80pMgI8I3WdqMWda0BPAjATYUHq3P hsf8xryrvEdk81fK8Ym6mkMVxkFbt2TrPSnKtYFk507KFw9fZMQy/Ydgws0jKkEUxGgv /UwTrPmdfojkawu/sVmNl39t1PpoAFXvcozhUCCA8i+gOgSbrQYNON+Ohw3C54CEPw44 eKfIxdaItxKqHHjkp1uIBGm+TCMzL/Pc5ggVS2J8RQVVpZk92BcIWwHYvZ8I9DdPj7gF k7Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jJYGi1zkIeuw27rkatjXNG5PEkuC/aMccRb7Npsrk9s=; b=Jav1dzuYRHnl4wkL770sOthFFbxROCyt3q3hwScslCiTJ12GdlCg24aZRroSXwPZQZ Vs107OfFVyT3c+JPA5cDtnoxT6kEi3bgL547wUFTT3VnsvneNDMFp+1fWtQbE0flN9Cu mDVNIfY0sqwZjHr4NV0Jkf8OJsP7dtCY7wFDZNZsV5XtfJyj6zewxxl23AvTLW5MREbt rrx9pTzQK9TlHmT0Z/oGZit/mzXdfS/2Ssb4ny1b1lDEOW/vqU/v25QDdt6diHZe1DUo JWAsqPeWCM4r9ro4eMSM7yHRIjFnMYXhAOBABZgntEeIGYtl6yvwCeABWMu+mKZUISFa JjmQ== X-Gm-Message-State: AOAM530okQfscMyX+B1ncclcxf8YecMwB+pS0wKSawgDHL/UEaoPjbU6 mKovVL3LAE5WEp6Epsokw04ClyEfrjI= X-Google-Smtp-Source: ABdhPJwoN4tQsmBitCfE1dnKZP4S+jfZABWlTDVcnRaCZ6nkQY/wbZatxrhC44gSAHJliIUkWbCfwQ== X-Received: by 2002:a62:804b:0:b0:44b:9369:5def with SMTP id j72-20020a62804b000000b0044b93695defmr348896pfd.29.1632754820812; Mon, 27 Sep 2021 08:00:20 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id x9sm20216953pjp.50.2021.09.27.08.00.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 08:00:20 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Song Liu , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v5 11/12] bpf: selftests: Fix fd cleanup in get_branch_snapshot Date: Mon, 27 Sep 2021 20:29:40 +0530 Message-Id: <20210927145941.1383001-12-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210927145941.1383001-1-memxor@gmail.com> References: <20210927145941.1383001-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1605; h=from:subject; bh=c77Pl0CO4LVQOtoLfE/1wWJcU2DcI56k8l974/sm848=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhUdxPnociSklHbvXNLA4XcWc4STUr38dyotdAygqz WTAXQ/GJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYVHcTwAKCRBM4MiGSL8Ryu0FEA Cq2BmaI/LXHaK+pfYuffSPliajGf3d1uIk/d6k/Sa+nmvMUDQl/uj1Kb77dVLSg5Cx/gvuwDJxwG9W 3puModv2mQtTQqlZNOszo2FgMc3YN872a6bk/GeAamBUVU+xweoUHTx9mzQToC936eBYjXiV/Tfpt5 Nmwyw7t1n/sBcldGT0tbgFFYHwTOya9MJocZz/2IjSYZ57GiEL8kHpqhvX250685FjlthhZngnJ7nT fcmlM91ZFK1v+htQEPELu9CW89pfmqX5paXIhbB6AKFIFWkVMZgmnxgCjYYksh/ncWbKr9JIMdfy5d y8iN2xkfB8HSENCYyW6HaZ6PRqFJdbwfY/+8m81Iizi2SMDAbBxwgQBMVp5h2adMr0uC37uYdWKumA h80YDFPGqsloGRRyD5QTJwuLXnHwr6zs+AopruaSqN+RD9ygeIIqvZm0Ny9ZDsiZCrExibhSRiqCtE zVroiX6hpfCM3cK0ODe5h/vaDU+/tAqSSjnNyCld4lNQgPwUdBX6o2QR1O6++AMLdDD8P1NePjr6rx 12ecGYMI11ZqKptCCXbVaYTPSTlBxLoltFoWPKMF6pNn6bt7EzumEPuDbH1wi/sP/CeaYBSpXHp0Y7 L6Q7SgehZ2BTTm3PAI4FWbaDquq60kKyM7oNYeGxeEuYczTOUlW0lI1+SUSQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net Cleanup code uses while (cpu++ < cpu_cnt) for closing fds, which means it starts iterating from 1 for closing fds. If the first fd is -1, it skips over it and closes garbage fds (typically zero) in the remaining array. This leads to test failures for future tests when they end up storing fd 0 (as the slot becomes free due to close(0)) in ldimm64's BTF fd, ending up trying to match module BTF id with vmlinux. This was observed as spurious CI failure for the ksym_module_libbpf and module_attach tests. The test ends up closing fd 0 and breaking libbpf's assumption that module BTF fd will always be > 0, which leads to the kernel thinking that we are pointing to a BTF ID in vmlinux BTF. Cc: Song Liu Fixes: 025bd7c753aa (selftests/bpf: Add test for bpf_get_branch_snapshot) Signed-off-by: Kumar Kartikeya Dwivedi Acked-by: Song Liu --- tools/testing/selftests/bpf/prog_tests/get_branch_snapshot.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/get_branch_snapshot.c b/tools/testing/selftests/bpf/prog_tests/get_branch_snapshot.c index f81db9135ae4..67e86f8d8677 100644 --- a/tools/testing/selftests/bpf/prog_tests/get_branch_snapshot.c +++ b/tools/testing/selftests/bpf/prog_tests/get_branch_snapshot.c @@ -38,10 +38,9 @@ static int create_perf_events(void) static void close_perf_events(void) { - int cpu = 0; - int fd; + int cpu, fd; - while (cpu++ < cpu_cnt) { + for (cpu = 0; cpu < cpu_cnt; cpu++) { fd = pfd_array[cpu]; if (fd < 0) break; From patchwork Mon Sep 27 14:59:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 12520109 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE4B6C4332F for ; Mon, 27 Sep 2021 15:00:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CDC4660EE2 for ; Mon, 27 Sep 2021 15:00:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235093AbhI0PC1 (ORCPT ); Mon, 27 Sep 2021 11:02:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235045AbhI0PCL (ORCPT ); Mon, 27 Sep 2021 11:02:11 -0400 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9DAEDC06177E; Mon, 27 Sep 2021 08:00:24 -0700 (PDT) Received: by mail-pl1-x643.google.com with SMTP id t11so11957422plq.11; Mon, 27 Sep 2021 08:00:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6sE66zJDQq6Lg7OPKq9GTbHJbssrncUby2t9sXV4jV0=; b=j1MCkFk8k9bEQGFlS9+bxiPDO3hJoj1e1W8CNhMrOu8PQzl0WpSHF4tB7GqK8+8wiA hsi+0knUDFzEC35D0w8SR2MNXykthUaqnTBxy/ser1SXzPZfjnud5zi5tDz4QDanN5PF o9bK6zPQLJwDDQC0MUvJf3C/911uLNjSEXFUxy5zwAxrKlYLnMO5exlyzeMl30WEETeX bJ9h9cXeQRMiuoc1UoUNW4U16XkPbeg+ZpckPhI9O97RSbzeeAYXiggVXUaAoymY4qIS 3ztST6NvN3qVP89/UZZ8IB6p7cBjYpMJQMXKMKXUKUKnGHDWMCymviOrScyueRRlpHXg iQHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6sE66zJDQq6Lg7OPKq9GTbHJbssrncUby2t9sXV4jV0=; b=4I3bYkF7syFNab0ODKCLEWHNajEalRAQcuOUjzwacddmP8VcZFA4zTLMJNB2J5wQSB Qs4qN8YLjw4KV2NL8fOCoN2gqMJl84n6bhqqQFHzg519WPnbKvl5IdmKKPi1AUHQufck U4M2YMCePq1+taGBxz1aAzMrmRA/giszXyjayJ+cSEezfgyDx2ZTXcyqNnSEYnYYQBwS pb8Z1FYWrC8pEZRGC8PinTZt5IwtPkrN+F8lMP4O1M1xSz3aiHtg5y2wjnvo6NTrgO5Q y3cwxB6RbFG0kqoCy6Z8ADlUsM/z6Dq616yG7d5rKXVHkqKiZ9VzhyWyWpx/tIPJvcRp 5Hew== X-Gm-Message-State: AOAM5323suzoEgujo551nxMvr5dVjKChjkOKbXuDAz5c7HVwc+a7J9L7 APX1s5Tcreg0U86g8AO7ZnjHmEAtSUk= X-Google-Smtp-Source: ABdhPJz0+IynyrqPKFQURhpnDbt1eqb8eO7ClZwaHvv6bDK5yJZDkwKEd2FJf0U/ebyAnNzLnSwhcA== X-Received: by 2002:a17:902:e88a:b0:13c:9ca3:f724 with SMTP id w10-20020a170902e88a00b0013c9ca3f724mr471451plg.10.1632754823929; Mon, 27 Sep 2021 08:00:23 -0700 (PDT) Received: from localhost ([2405:201:6014:d058:a28d:3909:6ed5:29e7]) by smtp.gmail.com with ESMTPSA id 2sm19738272pjt.23.2021.09.27.08.00.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Sep 2021 08:00:23 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?q?=C3=B8rgensen?= , netdev@vger.kernel.org Subject: [PATCH bpf-next v5 12/12] bpf: selftests: Add selftests for module kfunc support Date: Mon, 27 Sep 2021 20:29:41 +0530 Message-Id: <20210927145941.1383001-13-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210927145941.1383001-1-memxor@gmail.com> References: <20210927145941.1383001-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=11846; h=from:subject; bh=CtSiGElH/UrODUZz0IPrVPwlgUYMWrvlaeteZpqX6go=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhUdxPC+iDwZJfOAzoI3NkCTxafmbnxTReN+vX9+W5 XEwpXnuJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYVHcTwAKCRBM4MiGSL8Ryvg/D/ 9X2R3KXoAS4e9BQ3w1PDkgYDBZuAJUxk+jEJZsal5WLFvnneImm2ywSGcKofqfjPugMh8ozw+k1EDt RGe6/4b5HG2VlLV/fVJnM1FT9UMUeofhjaKV5sP+8u8LXEXB8uAzR1INp0WYvwnUqEWLjH9nVv3dDa lZsIEVtTAzz/svR8wTAlGB+M7hfVPJZaZyQJn4YHq+N6+x7TlPro6F/F449xtiYwzdaoib5zbfyq17 jnDSplpds3MK9Eu0UXnVPFpoDGxAg0FETfGnj2EzWx4pvDP8MQv/XIpbR41UrDjlrbjuJY9Z1uGKzp x6CADcBbINz0fSGXkv4gwwU2tZnpf1CviTzZqIW8YVa93KRR3BN0LwyAKecd5OMT/7GmF82dmQ5uqT SKwyOOuMl6xr9WQe7FIC3KwSWuPjqxdTzJs2OzmxYt6ZVjJOR/r96OSHv45WXtSwbGSFEBP4uiFrmV 0VXuWwTAubL9i0Qzsy5saMnksvCKeo2cEGx7BniOD4jIVAH3Dx4P3Cd5s/PCz7rg+122cT4EP34sDg 9sTv9DikF4eQmA+Ba1P1KrR1Gb7RAN+7ArrQHH2Czux9F/9maHJ93wWCYBacf9fgOk6DYHllRTaxvK C1KoyRz8CCJMyoI/DH0MGMYz05UfilHM3hsG8DdWpGvg7NGeZzmT4qAQuHpQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net This adds selftests that tests the success and failure path for modules kfuncs (in presence of invalid kfunc calls) for both libbpf and gen_loader. It also adds a prog_test kfunc_btf_id_list so that we can add module BTF ID set from bpf_testmod. This also introduces a couple of test cases to verifier selftests for validating whether we get an error or not depending on if invalid kfunc call remains after elimination of unreachable instructions. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/btf.h | 2 + kernel/bpf/btf.c | 2 + net/bpf/test_run.c | 5 +- tools/testing/selftests/bpf/Makefile | 8 ++-- .../selftests/bpf/bpf_testmod/bpf_testmod.c | 23 +++++++++- .../selftests/bpf/prog_tests/ksyms_module.c | 29 ++++++------ .../bpf/prog_tests/ksyms_module_libbpf.c | 28 +++++++++++ .../selftests/bpf/progs/test_ksyms_module.c | 46 ++++++++++++++----- tools/testing/selftests/bpf/verifier/calls.c | 23 ++++++++++ 9 files changed, 135 insertions(+), 31 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/ksyms_module_libbpf.c diff --git a/include/linux/btf.h b/include/linux/btf.h index 8dc65ff02bd6..4075cc24b88a 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -270,7 +270,9 @@ static inline void unregister_kfunc_btf_id_set(struct kfunc_btf_id_list *l, THIS_MODULE } extern struct kfunc_btf_id_list bpf_tcp_ca_kfunc_list; +extern struct kfunc_btf_id_list prog_test_kfunc_list; DECLARE_CHECK_KFUNC_CALLBACK(tcp_ca); +DECLARE_CHECK_KFUNC_CALLBACK(prog_test); #endif diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 423744378fc8..2d84b739ec22 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6397,3 +6397,5 @@ EXPORT_SYMBOL_GPL(unregister_kfunc_btf_id_set); DEFINE_KFUNC_BTF_ID_LIST(bpf_tcp_ca_kfunc_list); DEFINE_CHECK_KFUNC_CALLBACK(tcp_ca, bpf_tcp_ca_kfunc_list); +DEFINE_KFUNC_BTF_ID_LIST(prog_test_kfunc_list); +DEFINE_CHECK_KFUNC_CALLBACK(prog_test, prog_test_kfunc_list); diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c index fe5c34f414a2..e2b5f71af37e 100644 --- a/net/bpf/test_run.c +++ b/net/bpf/test_run.c @@ -2,6 +2,7 @@ /* Copyright (c) 2017 Facebook */ #include +#include #include #include #include @@ -243,7 +244,9 @@ BTF_SET_END(test_sk_kfunc_ids) bool bpf_prog_test_check_kfunc_call(u32 kfunc_id, struct module *owner) { - return btf_id_set_contains(&test_sk_kfunc_ids, kfunc_id); + if (btf_id_set_contains(&test_sk_kfunc_ids, kfunc_id)) + return true; + return __bpf_prog_test_check_kfunc_call(kfunc_id, owner); } static void *bpf_test_init(const union bpf_attr *kattr, u32 size, diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index e1ce73be7a5b..df461699932d 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -174,6 +174,7 @@ $(OUTPUT)/bpf_testmod.ko: $(VMLINUX_BTF) $(wildcard bpf_testmod/Makefile bpf_tes $(Q)$(RM) bpf_testmod/bpf_testmod.ko # force re-compilation $(Q)$(MAKE) $(submake_extras) -C bpf_testmod $(Q)cp bpf_testmod/bpf_testmod.ko $@ + $(Q)$(RESOLVE_BTFIDS) -b $(VMLINUX_BTF) bpf_testmod.ko $(OUTPUT)/test_stub.o: test_stub.c $(BPFOBJ) $(call msg,CC,,$@) @@ -315,8 +316,9 @@ LINKED_SKELS := test_static_linked.skel.h linked_funcs.skel.h \ linked_vars.skel.h linked_maps.skel.h LSKELS := kfunc_call_test.c fentry_test.c fexit_test.c fexit_sleep.c \ - test_ksyms_module.c test_ringbuf.c atomics.c trace_printk.c \ - trace_vprintk.c + test_ringbuf.c atomics.c trace_printk.c trace_vprintk.c +# Generate both light skeleton and libbpf skeleton for these +LSKELS_EXTRA := test_ksyms_module.c SKEL_BLACKLIST += $$(LSKELS) test_static_linked.skel.h-deps := test_static_linked1.o test_static_linked2.o @@ -346,7 +348,7 @@ TRUNNER_BPF_OBJS := $$(patsubst %.c,$$(TRUNNER_OUTPUT)/%.o, $$(TRUNNER_BPF_SRCS) TRUNNER_BPF_SKELS := $$(patsubst %.c,$$(TRUNNER_OUTPUT)/%.skel.h, \ $$(filter-out $(SKEL_BLACKLIST) $(LINKED_BPF_SRCS),\ $$(TRUNNER_BPF_SRCS))) -TRUNNER_BPF_LSKELS := $$(patsubst %.c,$$(TRUNNER_OUTPUT)/%.lskel.h, $$(LSKELS)) +TRUNNER_BPF_LSKELS := $$(patsubst %.c,$$(TRUNNER_OUTPUT)/%.lskel.h, $$(LSKELS) $$(LSKELS_EXTRA)) TRUNNER_BPF_SKELS_LINKED := $$(addprefix $$(TRUNNER_OUTPUT)/,$(LINKED_SKELS)) TEST_GEN_FILES += $$(TRUNNER_BPF_OBJS) diff --git a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c index 50fc5561110a..b892948dc134 100644 --- a/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c +++ b/tools/testing/selftests/bpf/bpf_testmod/bpf_testmod.c @@ -1,5 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright (c) 2020 Facebook */ +#include +#include #include #include #include @@ -13,6 +15,12 @@ DEFINE_PER_CPU(int, bpf_testmod_ksym_percpu) = 123; +noinline void +bpf_testmod_test_mod_kfunc(int i) +{ + *(int *)this_cpu_ptr(&bpf_testmod_ksym_percpu) = i; +} + noinline int bpf_testmod_loop_test(int n) { int i, sum = 0; @@ -71,13 +79,26 @@ static struct bin_attribute bin_attr_bpf_testmod_file __ro_after_init = { .write = bpf_testmod_test_write, }; +BTF_SET_START(bpf_testmod_kfunc_ids) +BTF_ID(func, bpf_testmod_test_mod_kfunc) +BTF_SET_END(bpf_testmod_kfunc_ids) + +static DEFINE_KFUNC_BTF_ID_SET(&bpf_testmod_kfunc_ids, bpf_testmod_kfunc_btf_set); + static int bpf_testmod_init(void) { - return sysfs_create_bin_file(kernel_kobj, &bin_attr_bpf_testmod_file); + int ret; + + ret = sysfs_create_bin_file(kernel_kobj, &bin_attr_bpf_testmod_file); + if (ret) + return ret; + register_kfunc_btf_id_set(&prog_test_kfunc_list, &bpf_testmod_kfunc_btf_set); + return 0; } static void bpf_testmod_exit(void) { + unregister_kfunc_btf_id_set(&prog_test_kfunc_list, &bpf_testmod_kfunc_btf_set); return sysfs_remove_bin_file(kernel_kobj, &bin_attr_bpf_testmod_file); } diff --git a/tools/testing/selftests/bpf/prog_tests/ksyms_module.c b/tools/testing/selftests/bpf/prog_tests/ksyms_module.c index 2cd5cded543f..831447878d7b 100644 --- a/tools/testing/selftests/bpf/prog_tests/ksyms_module.c +++ b/tools/testing/selftests/bpf/prog_tests/ksyms_module.c @@ -2,30 +2,29 @@ /* Copyright (c) 2021 Facebook */ #include -#include -#include +#include #include "test_ksyms_module.lskel.h" -static int duration; - void test_ksyms_module(void) { - struct test_ksyms_module* skel; + struct test_ksyms_module *skel; + int retval; int err; - skel = test_ksyms_module__open_and_load(); - if (CHECK(!skel, "skel_open", "failed to open skeleton\n")) + if (!env.has_testmod) { + test__skip(); return; + } - err = test_ksyms_module__attach(skel); - if (CHECK(err, "skel_attach", "skeleton attach failed: %d\n", err)) + skel = test_ksyms_module__open_and_load(); + if (!ASSERT_OK_PTR(skel, "test_ksyms_module__open_and_load")) + return; + err = bpf_prog_test_run(skel->progs.load.prog_fd, 1, &pkt_v4, sizeof(pkt_v4), + NULL, NULL, (__u32 *)&retval, NULL); + if (!ASSERT_OK(err, "bpf_prog_test_run")) goto cleanup; - - usleep(1); - - ASSERT_EQ(skel->bss->triggered, true, "triggered"); - ASSERT_EQ(skel->bss->out_mod_ksym_global, 123, "global_ksym_val"); - + ASSERT_EQ(retval, 0, "retval"); + ASSERT_EQ(skel->bss->out_bpf_testmod_ksym, 42, "bpf_testmod_ksym"); cleanup: test_ksyms_module__destroy(skel); } diff --git a/tools/testing/selftests/bpf/prog_tests/ksyms_module_libbpf.c b/tools/testing/selftests/bpf/prog_tests/ksyms_module_libbpf.c new file mode 100644 index 000000000000..e6343ef63af9 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/ksyms_module_libbpf.c @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include "test_ksyms_module.skel.h" + +void test_ksyms_module_libbpf(void) +{ + struct test_ksyms_module *skel; + int retval, err; + + if (!env.has_testmod) { + test__skip(); + return; + } + + skel = test_ksyms_module__open_and_load(); + if (!ASSERT_OK_PTR(skel, "test_ksyms_module__open")) + return; + err = bpf_prog_test_run(bpf_program__fd(skel->progs.load), 1, &pkt_v4, + sizeof(pkt_v4), NULL, NULL, (__u32 *)&retval, NULL); + if (!ASSERT_OK(err, "bpf_prog_test_run")) + goto cleanup; + ASSERT_EQ(retval, 0, "retval"); + ASSERT_EQ(skel->bss->out_bpf_testmod_ksym, 42, "bpf_testmod_ksym"); +cleanup: + test_ksyms_module__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/test_ksyms_module.c b/tools/testing/selftests/bpf/progs/test_ksyms_module.c index d6a0b3086b90..5fc50deef3b4 100644 --- a/tools/testing/selftests/bpf/progs/test_ksyms_module.c +++ b/tools/testing/selftests/bpf/progs/test_ksyms_module.c @@ -2,24 +2,48 @@ /* Copyright (c) 2021 Facebook */ #include "vmlinux.h" - #include +#define X_0(x) +#define X_1(x) x X_0(x) +#define X_2(x) x X_1(x) +#define X_3(x) x X_2(x) +#define X_4(x) x X_3(x) +#define X_5(x) x X_4(x) +#define X_6(x) x X_5(x) +#define X_7(x) x X_6(x) +#define X_8(x) x X_7(x) +#define X_9(x) x X_8(x) +#define X_10(x) x X_9(x) +#define REPEAT_256(Y) X_2(X_10(X_10(Y))) X_5(X_10(Y)) X_6(Y) + extern const int bpf_testmod_ksym_percpu __ksym; +extern void bpf_testmod_test_mod_kfunc(int i) __ksym; +extern void bpf_testmod_invalid_mod_kfunc(void) __ksym __weak; -int out_mod_ksym_global = 0; -bool triggered = false; +int out_bpf_testmod_ksym = 0; +const volatile int x = 0; -SEC("raw_tp/sys_enter") -int handler(const void *ctx) +SEC("classifier") +int load(struct __sk_buff *skb) { - int *val; - __u32 cpu; - - val = (int *)bpf_this_cpu_ptr(&bpf_testmod_ksym_percpu); - out_mod_ksym_global = *val; - triggered = true; + /* This will be kept by clang, but removed by verifier. Since it is + * marked as __weak, libbpf and gen_loader don't error out if BTF ID + * is not found for it, instead imm and off is set to 0 for it. + */ + if (x) + bpf_testmod_invalid_mod_kfunc(); + bpf_testmod_test_mod_kfunc(42); + out_bpf_testmod_ksym = *(int *)bpf_this_cpu_ptr(&bpf_testmod_ksym_percpu); + return 0; +} +SEC("classifier") +int load_256(struct __sk_buff *skb) +{ + /* this will fail if kfunc doesn't reuse its own btf fd index */ + REPEAT_256(bpf_testmod_test_mod_kfunc(42);); + bpf_testmod_test_mod_kfunc(42); return 0; } diff --git a/tools/testing/selftests/bpf/verifier/calls.c b/tools/testing/selftests/bpf/verifier/calls.c index 336a749673d1..d7b74eb28333 100644 --- a/tools/testing/selftests/bpf/verifier/calls.c +++ b/tools/testing/selftests/bpf/verifier/calls.c @@ -1,3 +1,26 @@ +{ + "calls: invalid kfunc call not eliminated", + .insns = { + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_KFUNC_CALL, 0, 0), + BPF_MOV64_IMM(BPF_REG_0, 1), + BPF_EXIT_INSN(), + }, + .prog_type = BPF_PROG_TYPE_TRACEPOINT, + .result = REJECT, + .errstr = "invalid kernel function call not eliminated in verifier pass", +}, +{ + "calls: invalid kfunc call unreachable", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1), + BPF_JMP_IMM(BPF_JGT, BPF_REG_0, 0, 2), + BPF_RAW_INSN(BPF_JMP | BPF_CALL, 0, BPF_PSEUDO_KFUNC_CALL, 0, 0), + BPF_MOV64_IMM(BPF_REG_0, 1), + BPF_EXIT_INSN(), + }, + .prog_type = BPF_PROG_TYPE_TRACEPOINT, + .result = ACCEPT, +}, { "calls: basic sanity", .insns = {