| Message ID | 20211118112455.475349-10-jolsa@kernel.org (mailing list archive) |
|---|---|
| State | Changes Requested |
| Delegated to: | BPF |
| Headers | show |
| Series | bpf: Add batch support for attaching trampolines | expand |
On Thu, Nov 18, 2021 at 12:24:35PM +0100, Jiri Olsa wrote: > + > +DEFINE_BPF_MULTI_FUNC(unsigned long a1, unsigned long a2, > + unsigned long a3, unsigned long a4, > + unsigned long a5, unsigned long a6) This is probably a bit too x86 specific. May be make add all 12 args? Or other places would need to be tweaked? > +BTF_ID_LIST_SINGLE(bpf_multi_func_btf_id, func, bpf_multi_func) ... > - prog->aux->attach_btf_id = attr->attach_btf_id; > + prog->aux->attach_btf_id = multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id; Just ignoring that was passed in uattr? Maybe instead of ignoring dopr BPF_F_MULTI_FUNC and make libbpf point to that btf_id instead? Then multi or not can be checked with if (attr->attach_btf_id == bpf_multi_func_btf_id[0]).
On Thu, Nov 18, 2021 at 08:11:59PM -0800, Alexei Starovoitov wrote: > On Thu, Nov 18, 2021 at 12:24:35PM +0100, Jiri Olsa wrote: > > + > > +DEFINE_BPF_MULTI_FUNC(unsigned long a1, unsigned long a2, > > + unsigned long a3, unsigned long a4, > > + unsigned long a5, unsigned long a6) > > This is probably a bit too x86 specific. May be make add all 12 args? > Or other places would need to be tweaked? I think si, I'll check > > > +BTF_ID_LIST_SINGLE(bpf_multi_func_btf_id, func, bpf_multi_func) > ... > > - prog->aux->attach_btf_id = attr->attach_btf_id; > > + prog->aux->attach_btf_id = multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id; > > Just ignoring that was passed in uattr? > Maybe instead of ignoring dopr BPF_F_MULTI_FUNC and make libbpf > point to that btf_id instead? > Then multi or not can be checked with if (attr->attach_btf_id == bpf_multi_func_btf_id[0]). > nice idea, it might fit better than the flag thanks, jirka
On Mon, Nov 22, 2021 at 12:15 PM Jiri Olsa <jolsa@redhat.com> wrote: > > On Thu, Nov 18, 2021 at 08:11:59PM -0800, Alexei Starovoitov wrote: > > On Thu, Nov 18, 2021 at 12:24:35PM +0100, Jiri Olsa wrote: > > > + > > > +DEFINE_BPF_MULTI_FUNC(unsigned long a1, unsigned long a2, > > > + unsigned long a3, unsigned long a4, > > > + unsigned long a5, unsigned long a6) > > > > This is probably a bit too x86 specific. May be make add all 12 args? > > Or other places would need to be tweaked? > > I think si, I'll check > > > > > > +BTF_ID_LIST_SINGLE(bpf_multi_func_btf_id, func, bpf_multi_func) > > ... > > > - prog->aux->attach_btf_id = attr->attach_btf_id; > > > + prog->aux->attach_btf_id = multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id; > > > > Just ignoring that was passed in uattr? > > Maybe instead of ignoring dopr BPF_F_MULTI_FUNC and make libbpf > > point to that btf_id instead? > > Then multi or not can be checked with if (attr->attach_btf_id == bpf_multi_func_btf_id[0]). > > > > nice idea, it might fit better than the flag Instead of a flag we can also use a different expected_attach_type (FENTRY vs FENTRY_MULTI, etc). As for attach_btf_id, why can't we just enforce it as 0? > > thanks, > jirka >
On Wed, Nov 24, 2021 at 01:51:36PM -0800, Andrii Nakryiko wrote: > On Mon, Nov 22, 2021 at 12:15 PM Jiri Olsa <jolsa@redhat.com> wrote: > > > > On Thu, Nov 18, 2021 at 08:11:59PM -0800, Alexei Starovoitov wrote: > > > On Thu, Nov 18, 2021 at 12:24:35PM +0100, Jiri Olsa wrote: > > > > + > > > > +DEFINE_BPF_MULTI_FUNC(unsigned long a1, unsigned long a2, > > > > + unsigned long a3, unsigned long a4, > > > > + unsigned long a5, unsigned long a6) > > > > > > This is probably a bit too x86 specific. May be make add all 12 args? > > > Or other places would need to be tweaked? > > > > I think si, I'll check > > > > > > > > > +BTF_ID_LIST_SINGLE(bpf_multi_func_btf_id, func, bpf_multi_func) > > > ... > > > > - prog->aux->attach_btf_id = attr->attach_btf_id; > > > > + prog->aux->attach_btf_id = multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id; > > > > > > Just ignoring that was passed in uattr? > > > Maybe instead of ignoring dopr BPF_F_MULTI_FUNC and make libbpf > > > point to that btf_id instead? > > > Then multi or not can be checked with if (attr->attach_btf_id == bpf_multi_func_btf_id[0]). > > > > > > > nice idea, it might fit better than the flag > > Instead of a flag we can also use a different expected_attach_type > (FENTRY vs FENTRY_MULTI, etc). right, you already asked for that - https://lore.kernel.org/bpf/YS9k26rRcUJVS%2Fvx@krava/ I still think it'd mean more code while this way we just use current fentry/fexit code paths with few special handling for multi programs > As for attach_btf_id, why can't we just > enforce it as 0? there's prog->aux->attach_func_proto that needs to be set based on attach_btf_id, and is checked later in btf_ctx_access jirka > > > > > thanks, > > jirka > > >
On Sun, Nov 28, 2021 at 9:41 AM Jiri Olsa <jolsa@redhat.com> wrote: > > On Wed, Nov 24, 2021 at 01:51:36PM -0800, Andrii Nakryiko wrote: > > On Mon, Nov 22, 2021 at 12:15 PM Jiri Olsa <jolsa@redhat.com> wrote: > > > > > > On Thu, Nov 18, 2021 at 08:11:59PM -0800, Alexei Starovoitov wrote: > > > > On Thu, Nov 18, 2021 at 12:24:35PM +0100, Jiri Olsa wrote: > > > > > + > > > > > +DEFINE_BPF_MULTI_FUNC(unsigned long a1, unsigned long a2, > > > > > + unsigned long a3, unsigned long a4, > > > > > + unsigned long a5, unsigned long a6) > > > > > > > > This is probably a bit too x86 specific. May be make add all 12 args? > > > > Or other places would need to be tweaked? > > > > > > I think si, I'll check > > > > > > > > > > > > +BTF_ID_LIST_SINGLE(bpf_multi_func_btf_id, func, bpf_multi_func) > > > > ... > > > > > - prog->aux->attach_btf_id = attr->attach_btf_id; > > > > > + prog->aux->attach_btf_id = multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id; > > > > > > > > Just ignoring that was passed in uattr? > > > > Maybe instead of ignoring dopr BPF_F_MULTI_FUNC and make libbpf > > > > point to that btf_id instead? > > > > Then multi or not can be checked with if (attr->attach_btf_id == bpf_multi_func_btf_id[0]). > > > > > > > > > > nice idea, it might fit better than the flag > > > > Instead of a flag we can also use a different expected_attach_type > > (FENTRY vs FENTRY_MULTI, etc). > > right, you already asked for that - https://lore.kernel.org/bpf/YS9k26rRcUJVS%2Fvx@krava/ > > I still think it'd mean more code while this way we just use > current fentry/fexit code paths with few special handling > for multi programs > I don't see how it makes much difference for kernel implementation. Checking expected_attach_type vs checking prog_flags is about the same amount of code. The big advantage of new expected_attach_type (or prog_type) is that it will be very obvious in all sorts of diagnostics tooling (think bpftool prog show output, etc). prog_flags are almost invisible and it will be the last thing that users will think about. I'd try to minimize the usage of prog_flags overall. > > As for attach_btf_id, why can't we just > > enforce it as 0? > > there's prog->aux->attach_func_proto that needs to be set based > on attach_btf_id, and is checked later in btf_ctx_access right: if (attach_btf_id == 0) prog->aux->attach_func_proto = &special_func_model_or_proto_or_whatever_that_does_not_have_to_be_known_to_libbpf_and_outside_world_ever; ;) let's keep implementation details as internal implementation details, instead of dumping all that to UAPI > > jirka > > > > > > > > > thanks, > > > jirka > > > > > >
On Tue, Nov 30, 2021 at 11:17:44PM -0800, Andrii Nakryiko wrote: > On Sun, Nov 28, 2021 at 9:41 AM Jiri Olsa <jolsa@redhat.com> wrote: > > > > On Wed, Nov 24, 2021 at 01:51:36PM -0800, Andrii Nakryiko wrote: > > > On Mon, Nov 22, 2021 at 12:15 PM Jiri Olsa <jolsa@redhat.com> wrote: > > > > > > > > On Thu, Nov 18, 2021 at 08:11:59PM -0800, Alexei Starovoitov wrote: > > > > > On Thu, Nov 18, 2021 at 12:24:35PM +0100, Jiri Olsa wrote: > > > > > > + > > > > > > +DEFINE_BPF_MULTI_FUNC(unsigned long a1, unsigned long a2, > > > > > > + unsigned long a3, unsigned long a4, > > > > > > + unsigned long a5, unsigned long a6) > > > > > > > > > > This is probably a bit too x86 specific. May be make add all 12 args? > > > > > Or other places would need to be tweaked? > > > > > > > > I think si, I'll check > > > > > > > > > > > > > > > +BTF_ID_LIST_SINGLE(bpf_multi_func_btf_id, func, bpf_multi_func) > > > > > ... > > > > > > - prog->aux->attach_btf_id = attr->attach_btf_id; > > > > > > + prog->aux->attach_btf_id = multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id; > > > > > > > > > > Just ignoring that was passed in uattr? > > > > > Maybe instead of ignoring dopr BPF_F_MULTI_FUNC and make libbpf > > > > > point to that btf_id instead? > > > > > Then multi or not can be checked with if (attr->attach_btf_id == bpf_multi_func_btf_id[0]). > > > > > > > > > > > > > nice idea, it might fit better than the flag > > > > > > Instead of a flag we can also use a different expected_attach_type > > > (FENTRY vs FENTRY_MULTI, etc). > > > > right, you already asked for that - https://lore.kernel.org/bpf/YS9k26rRcUJVS%2Fvx@krava/ > > > > I still think it'd mean more code while this way we just use > > current fentry/fexit code paths with few special handling > > for multi programs > > > > I don't see how it makes much difference for kernel implementation. > Checking expected_attach_type vs checking prog_flags is about the same > amount of code. The big advantage of new expected_attach_type (or > prog_type) is that it will be very obvious in all sorts of diagnostics > tooling (think bpftool prog show output, etc). prog_flags are almost > invisible and it will be the last thing that users will think about. > I'd try to minimize the usage of prog_flags overall. ok, I'll check on that.. I recall adding this new type in many expected_attach_type switches, which made me think the new flag will be easier > > > > As for attach_btf_id, why can't we just > > > enforce it as 0? > > > > there's prog->aux->attach_func_proto that needs to be set based > > on attach_btf_id, and is checked later in btf_ctx_access > > right: > > if (attach_btf_id == 0) > prog->aux->attach_func_proto = > &special_func_model_or_proto_or_whatever_that_does_not_have_to_be_known_to_libbpf_and_outside_world_ever; > > ;) let's keep implementation details as internal implementation > details, instead of dumping all that to UAPI ok, we can do that ;-) thanks, jirka
diff --git a/include/linux/bpf.h b/include/linux/bpf.h index c93c629b5725..35f484f323f3 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -881,6 +881,7 @@ struct bpf_prog_aux { bool func_proto_unreliable; bool sleepable; bool tail_call_reachable; + bool multi_func; struct hlist_node tramp_hlist; /* BTF_KIND_FUNC_PROTO for valid attach_btf_id */ const struct btf_type *attach_func_proto; diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index fc8b344eecba..ca05e35e0478 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1111,6 +1111,13 @@ enum bpf_link_type { */ #define BPF_F_SLEEPABLE (1U << 4) +/* If BPF_F_MULTI_FUNC is used in BPF_PROG_LOAD command, the verifier does + * not expect BTF ID for the program, instead it assumes it's function + * with 6 u64 arguments. No trampoline is created for the program. Such + * program can be attached to multiple functions. + */ +#define BPF_F_MULTI_FUNC (1U << 5) + /* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: * diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 0df8b2f3d982..648155f0a4b1 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -31,6 +31,7 @@ #include <linux/bpf-netns.h> #include <linux/rcupdate_trace.h> #include <linux/memcontrol.h> +#include <linux/btf_ids.h> #define IS_FD_ARRAY(map) ((map)->map_type == BPF_MAP_TYPE_PERF_EVENT_ARRAY || \ (map)->map_type == BPF_MAP_TYPE_CGROUP_ARRAY || \ @@ -2040,7 +2041,8 @@ static int bpf_prog_load_check_attach(enum bpf_prog_type prog_type, enum bpf_attach_type expected_attach_type, struct btf *attach_btf, u32 btf_id, - struct bpf_prog *dst_prog) + struct bpf_prog *dst_prog, + bool multi_func) { if (btf_id) { if (btf_id > BTF_MAX_TYPE) @@ -2060,6 +2062,14 @@ bpf_prog_load_check_attach(enum bpf_prog_type prog_type, } } + if (multi_func) { + if (prog_type != BPF_PROG_TYPE_TRACING) + return -EINVAL; + if (!attach_btf || btf_id) + return -EINVAL; + return 0; + } + if (attach_btf && (!btf_id || dst_prog)) return -EINVAL; @@ -2183,6 +2193,16 @@ static bool is_perfmon_prog_type(enum bpf_prog_type prog_type) } } +#define DEFINE_BPF_MULTI_FUNC(args...) \ + extern int bpf_multi_func(args); \ + int __init bpf_multi_func(args) { return 0; } + +DEFINE_BPF_MULTI_FUNC(unsigned long a1, unsigned long a2, + unsigned long a3, unsigned long a4, + unsigned long a5, unsigned long a6) + +BTF_ID_LIST_SINGLE(bpf_multi_func_btf_id, func, bpf_multi_func) + /* last field in 'union bpf_attr' used by this command */ #define BPF_PROG_LOAD_LAST_FIELD fd_array @@ -2193,6 +2213,7 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) struct btf *attach_btf = NULL; int err; char license[128]; + bool multi_func; bool is_gpl; if (CHECK_ATTR(BPF_PROG_LOAD)) @@ -2202,7 +2223,8 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) BPF_F_ANY_ALIGNMENT | BPF_F_TEST_STATE_FREQ | BPF_F_SLEEPABLE | - BPF_F_TEST_RND_HI32)) + BPF_F_TEST_RND_HI32 | + BPF_F_MULTI_FUNC)) return -EINVAL; if (!IS_ENABLED(CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS) && @@ -2233,6 +2255,8 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) if (is_perfmon_prog_type(type) && !perfmon_capable()) return -EPERM; + multi_func = attr->prog_flags & BPF_F_MULTI_FUNC; + /* attach_prog_fd/attach_btf_obj_fd can specify fd of either bpf_prog * or btf, we need to check which one it is */ @@ -2251,7 +2275,7 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) return -ENOTSUPP; } } - } else if (attr->attach_btf_id) { + } else if (attr->attach_btf_id || multi_func) { /* fall back to vmlinux BTF, if BTF type ID is specified */ attach_btf = bpf_get_btf_vmlinux(); if (IS_ERR(attach_btf)) @@ -2264,7 +2288,7 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) bpf_prog_load_fixup_attach_type(attr); if (bpf_prog_load_check_attach(type, attr->expected_attach_type, attach_btf, attr->attach_btf_id, - dst_prog)) { + dst_prog, multi_func)) { if (dst_prog) bpf_prog_put(dst_prog); if (attach_btf) @@ -2284,10 +2308,11 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) prog->expected_attach_type = attr->expected_attach_type; prog->aux->attach_btf = attach_btf; - prog->aux->attach_btf_id = attr->attach_btf_id; + prog->aux->attach_btf_id = multi_func ? bpf_multi_func_btf_id[0] : attr->attach_btf_id; prog->aux->dst_prog = dst_prog; prog->aux->offload_requested = !!attr->prog_ifindex; prog->aux->sleepable = attr->prog_flags & BPF_F_SLEEPABLE; + prog->aux->multi_func = multi_func; err = security_bpf_prog_alloc(prog->aux); if (err) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index d4249ef6ca7e..af6a39bbb0dc 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -13983,7 +13983,8 @@ static int check_attach_btf_id(struct bpf_verifier_env *env) if (!bpf_iter_prog_supported(prog)) return -EINVAL; return 0; - } + } else if (prog->aux->multi_func) + return prog->type == BPF_PROG_TYPE_TRACING ? 0 : -EINVAL; if (prog->type == BPF_PROG_TYPE_LSM) { ret = bpf_lsm_verify_prog(&env->log, prog); diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index fc8b344eecba..ca05e35e0478 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1111,6 +1111,13 @@ enum bpf_link_type { */ #define BPF_F_SLEEPABLE (1U << 4) +/* If BPF_F_MULTI_FUNC is used in BPF_PROG_LOAD command, the verifier does + * not expect BTF ID for the program, instead it assumes it's function + * with 6 u64 arguments. No trampoline is created for the program. Such + * program can be attached to multiple functions. + */ +#define BPF_F_MULTI_FUNC (1U << 5) + /* When BPF ldimm64's insn[0].src_reg != 0 then this can have * the following extensions: *
Adding support to load tracing program with new BPF_F_MULTI_FUNC flag, that allows the program to be loaded without specific function to be attached to. Such program will be allowed to be attached to multiple functions in following patches. Signed-off-by: Jiri Olsa <jolsa@kernel.org> --- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 7 +++++++ kernel/bpf/syscall.c | 35 +++++++++++++++++++++++++++++----- kernel/bpf/verifier.c | 3 ++- tools/include/uapi/linux/bpf.h | 7 +++++++ 5 files changed, 47 insertions(+), 6 deletions(-)