| Message ID | 20210905100914.33007-1-hengqi.chen@gmail.com (mailing list archive) |
|---|---|
| State | Changes Requested |
| Delegated to: | BPF |
| Headers | show |
| Series | [bpf-next,1/2] libbpf: Support uniform BTF-defined key/value specification across all BPF maps | expand |
| Context | Check | Description |
|---|---|---|
| netdev/cover_letter | success | Link |
| netdev/fixes_present | success | Link |
| netdev/patch_count | success | Link |
| netdev/tree_selection | success | Clearly marked for bpf-next |
| netdev/subject_prefix | success | Link |
| netdev/cc_maintainers | warning | 4 maintainers not CCed: songliubraving@fb.com kpsingh@kernel.org kafai@fb.com netdev@vger.kernel.org |
| netdev/source_inline | success | Was 0 now: 0 |
| netdev/verify_signedoff | success | Link |
| netdev/module_param | success | Was 0 now: 0 |
| netdev/build_32bit | success | Errors and warnings before: 0 this patch: 0 |
| netdev/kdoc | success | Errors and warnings before: 0 this patch: 0 |
| netdev/verify_fixes | success | Link |
| netdev/checkpatch | success | total: 0 errors, 0 warnings, 0 checks, 47 lines checked |
| netdev/build_allmodconfig_warn | success | Errors and warnings before: 0 this patch: 0 |
| netdev/header_inline | success | Link |
| bpf/vmtest-bpf-next-PR | success | PR summary |
| bpf/vmtest-bpf-next | success | VM_Test |
On Sun, Sep 5, 2021 at 3:09 AM Hengqi Chen <hengqi.chen@gmail.com> wrote: > > A bunch of BPF maps do not support specifying types for key and value. s/types/BTF types/, it's a bit confusing otherwise > This is non-uniform and inconvenient[0]. Currently, libbpf uses a retry > logic which removes BTF type IDs when BPF map creation failed. Instead > of retrying, this commit recognizes those specialized map and removes s/map/maps/ > BTF type IDs when creating BPF map. > > [0] Closes: https://github.com/libbpf/libbpf/issues/355 > > Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com> > --- For patch sets consisting of two or more patches, we ask for a cover letter, so for the next revision please provide a cover letter with an overall description of what the series is about. > tools/lib/bpf/libbpf.c | 35 ++++++++++++++++++++--------------- > 1 file changed, 20 insertions(+), 15 deletions(-) > > diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c > index 88d8825fc6f6..7068c4d07337 100644 > --- a/tools/lib/bpf/libbpf.c > +++ b/tools/lib/bpf/libbpf.c > @@ -4613,6 +4613,26 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b > create_attr.inner_map_fd = map->inner_map_fd; > } > > + if (def->type == BPF_MAP_TYPE_PERF_EVENT_ARRAY || > + def->type == BPF_MAP_TYPE_STACK_TRACE || > + def->type == BPF_MAP_TYPE_CGROUP_ARRAY || > + def->type == BPF_MAP_TYPE_ARRAY_OF_MAPS || > + def->type == BPF_MAP_TYPE_HASH_OF_MAPS || > + def->type == BPF_MAP_TYPE_DEVMAP || > + def->type == BPF_MAP_TYPE_SOCKMAP || > + def->type == BPF_MAP_TYPE_CPUMAP || > + def->type == BPF_MAP_TYPE_XSKMAP || > + def->type == BPF_MAP_TYPE_SOCKHASH || > + def->type == BPF_MAP_TYPE_QUEUE || > + def->type == BPF_MAP_TYPE_STACK || > + def->type == BPF_MAP_TYPE_DEVMAP_HASH) { > + create_attr.btf_fd = 0; > + create_attr.btf_key_type_id = 0; > + create_attr.btf_value_type_id = 0; > + map->btf_key_type_id = 0; > + map->btf_value_type_id = 0; > + } Let's do this as a more succinct switch statement. Consider also slightly rearranging entries to keep "related" map types together: - SOCKMAP + SOCKHASH - DEVMAP + DEVMAP_HASH + CPUMAP + XSKMAP Thanks! > + > if (obj->gen_loader) { > bpf_gen__map_create(obj->gen_loader, &create_attr, is_inner ? -1 : map - obj->maps); > /* Pretend to have valid FD to pass various fd >= 0 checks. > @@ -4622,21 +4642,6 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b > } else { > map->fd = bpf_create_map_xattr(&create_attr); > } > - if (map->fd < 0 && (create_attr.btf_key_type_id || > - create_attr.btf_value_type_id)) { > - char *cp, errmsg[STRERR_BUFSIZE]; > - > - err = -errno; > - cp = libbpf_strerror_r(err, errmsg, sizeof(errmsg)); > - pr_warn("Error in bpf_create_map_xattr(%s):%s(%d). Retrying without BTF.\n", > - map->name, cp, err); > - create_attr.btf_fd = 0; > - create_attr.btf_key_type_id = 0; > - create_attr.btf_value_type_id = 0; > - map->btf_key_type_id = 0; > - map->btf_value_type_id = 0; > - map->fd = bpf_create_map_xattr(&create_attr); > - } > Please don't remove this fallback logic. There are multiple situations where libbpf might need to retry map creation without BTF. > err = map->fd < 0 ? -errno : 0; > > -- > 2.25.1 >
On 9/9/21 12:26 PM, Andrii Nakryiko wrote: > On Sun, Sep 5, 2021 at 3:09 AM Hengqi Chen <hengqi.chen@gmail.com> wrote: >> >> A bunch of BPF maps do not support specifying types for key and value. > > s/types/BTF types/, it's a bit confusing otherwise > >> This is non-uniform and inconvenient[0]. Currently, libbpf uses a retry >> logic which removes BTF type IDs when BPF map creation failed. Instead >> of retrying, this commit recognizes those specialized map and removes > > s/map/maps/ > >> BTF type IDs when creating BPF map. >> >> [0] Closes: https://github.com/libbpf/libbpf/issues/355 >> >> Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com> >> --- > > For patch sets consisting of two or more patches, we ask for a cover > letter, so for the next revision please provide a cover letter with an > overall description of what the series is about. > Hello, Andrii Sorry for the long delay. Will send a v2 for review. >> tools/lib/bpf/libbpf.c | 35 ++++++++++++++++++++--------------- >> 1 file changed, 20 insertions(+), 15 deletions(-) >> >> diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c >> index 88d8825fc6f6..7068c4d07337 100644 >> --- a/tools/lib/bpf/libbpf.c >> +++ b/tools/lib/bpf/libbpf.c >> @@ -4613,6 +4613,26 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b >> create_attr.inner_map_fd = map->inner_map_fd; >> } >> >> + if (def->type == BPF_MAP_TYPE_PERF_EVENT_ARRAY || >> + def->type == BPF_MAP_TYPE_STACK_TRACE || >> + def->type == BPF_MAP_TYPE_CGROUP_ARRAY || >> + def->type == BPF_MAP_TYPE_ARRAY_OF_MAPS || >> + def->type == BPF_MAP_TYPE_HASH_OF_MAPS || >> + def->type == BPF_MAP_TYPE_DEVMAP || >> + def->type == BPF_MAP_TYPE_SOCKMAP || >> + def->type == BPF_MAP_TYPE_CPUMAP || >> + def->type == BPF_MAP_TYPE_XSKMAP || >> + def->type == BPF_MAP_TYPE_SOCKHASH || >> + def->type == BPF_MAP_TYPE_QUEUE || >> + def->type == BPF_MAP_TYPE_STACK || >> + def->type == BPF_MAP_TYPE_DEVMAP_HASH) { >> + create_attr.btf_fd = 0; >> + create_attr.btf_key_type_id = 0; >> + create_attr.btf_value_type_id = 0; >> + map->btf_key_type_id = 0; >> + map->btf_value_type_id = 0; >> + } > > Let's do this as a more succinct switch statement. Consider also > slightly rearranging entries to keep "related" map types together: > - SOCKMAP + SOCKHASH > - DEVMAP + DEVMAP_HASH + CPUMAP + XSKMAP > > Thanks! >> >> + >> if (obj->gen_loader) { >> bpf_gen__map_create(obj->gen_loader, &create_attr, is_inner ? -1 : map - obj->maps); >> /* Pretend to have valid FD to pass various fd >= 0 checks. >> @@ -4622,21 +4642,6 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b >> } else { >> map->fd = bpf_create_map_xattr(&create_attr); >> } >> - if (map->fd < 0 && (create_attr.btf_key_type_id || >> - create_attr.btf_value_type_id)) { >> - char *cp, errmsg[STRERR_BUFSIZE]; >> - >> - err = -errno; >> - cp = libbpf_strerror_r(err, errmsg, sizeof(errmsg)); >> - pr_warn("Error in bpf_create_map_xattr(%s):%s(%d). Retrying without BTF.\n", >> - map->name, cp, err); >> - create_attr.btf_fd = 0; >> - create_attr.btf_key_type_id = 0; >> - create_attr.btf_value_type_id = 0; >> - map->btf_key_type_id = 0; >> - map->btf_value_type_id = 0; >> - map->fd = bpf_create_map_xattr(&create_attr); >> - } >> > > Please don't remove this fallback logic. There are multiple situations > where libbpf might need to retry map creation without BTF. > >> err = map->fd < 0 ? -errno : 0; >> >> -- >> 2.25.1 >>
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 88d8825fc6f6..7068c4d07337 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -4613,6 +4613,26 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b create_attr.inner_map_fd = map->inner_map_fd; } + if (def->type == BPF_MAP_TYPE_PERF_EVENT_ARRAY || + def->type == BPF_MAP_TYPE_STACK_TRACE || + def->type == BPF_MAP_TYPE_CGROUP_ARRAY || + def->type == BPF_MAP_TYPE_ARRAY_OF_MAPS || + def->type == BPF_MAP_TYPE_HASH_OF_MAPS || + def->type == BPF_MAP_TYPE_DEVMAP || + def->type == BPF_MAP_TYPE_SOCKMAP || + def->type == BPF_MAP_TYPE_CPUMAP || + def->type == BPF_MAP_TYPE_XSKMAP || + def->type == BPF_MAP_TYPE_SOCKHASH || + def->type == BPF_MAP_TYPE_QUEUE || + def->type == BPF_MAP_TYPE_STACK || + def->type == BPF_MAP_TYPE_DEVMAP_HASH) { + create_attr.btf_fd = 0; + create_attr.btf_key_type_id = 0; + create_attr.btf_value_type_id = 0; + map->btf_key_type_id = 0; + map->btf_value_type_id = 0; + } + if (obj->gen_loader) { bpf_gen__map_create(obj->gen_loader, &create_attr, is_inner ? -1 : map - obj->maps); /* Pretend to have valid FD to pass various fd >= 0 checks. @@ -4622,21 +4642,6 @@ static int bpf_object__create_map(struct bpf_object *obj, struct bpf_map *map, b } else { map->fd = bpf_create_map_xattr(&create_attr); } - if (map->fd < 0 && (create_attr.btf_key_type_id || - create_attr.btf_value_type_id)) { - char *cp, errmsg[STRERR_BUFSIZE]; - - err = -errno; - cp = libbpf_strerror_r(err, errmsg, sizeof(errmsg)); - pr_warn("Error in bpf_create_map_xattr(%s):%s(%d). Retrying without BTF.\n", - map->name, cp, err); - create_attr.btf_fd = 0; - create_attr.btf_key_type_id = 0; - create_attr.btf_value_type_id = 0; - map->btf_key_type_id = 0; - map->btf_value_type_id = 0; - map->fd = bpf_create_map_xattr(&create_attr); - } err = map->fd < 0 ? -errno : 0;
A bunch of BPF maps do not support specifying types for key and value. This is non-uniform and inconvenient[0]. Currently, libbpf uses a retry logic which removes BTF type IDs when BPF map creation failed. Instead of retrying, this commit recognizes those specialized map and removes BTF type IDs when creating BPF map. [0] Closes: https://github.com/libbpf/libbpf/issues/355 Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com> --- tools/lib/bpf/libbpf.c | 35 ++++++++++++++++++++--------------- 1 file changed, 20 insertions(+), 15 deletions(-)