diff mbox series

[bpf] libbpf: skip base btf sanity checks

Message ID 20240624090908.171231-1-atenart@kernel.org (mailing list archive)
State Accepted
Commit c73a9683cb21012b6c0f14217974837151c527a8
Delegated to: BPF
Headers show
Series [bpf] libbpf: skip base btf sanity checks | expand

Checks

Context Check Description
netdev/series_format success Single patches do not need cover letters
netdev/tree_selection success Clearly marked for bpf
netdev/ynl success Generated files up to date; no warnings/errors; no diff in generated;
netdev/fixes_present success Fixes tag present in non-next series
netdev/header_inline success No static functions without inline keyword in header files
netdev/build_32bit fail Errors and warnings before: 45 this patch: 45
netdev/build_tools success Errors and warnings before: 1 this patch: 1
netdev/cc_maintainers fail 3 blamed authors not CCed: daniel@iogearbox.net song@kernel.org alan.maguire@oracle.com; 11 maintainers not CCed: jolsa@kernel.org john.fastabend@gmail.com ast@kernel.org haoluo@google.com song@kernel.org martin.lau@linux.dev daniel@iogearbox.net yonghong.song@linux.dev alan.maguire@oracle.com kpsingh@kernel.org sdf@fomichev.me
netdev/build_clang fail Errors and warnings before: 36 this patch: 36
netdev/verify_signedoff success Signed-off-by tag matches author and committer
netdev/deprecated_api success None detected
netdev/check_selftest success No net selftest shell script
netdev/verify_fixes success Fixes tag looks correct
netdev/build_allmodconfig_warn fail Errors and warnings before: 45 this patch: 45
netdev/checkpatch success total: 0 errors, 0 warnings, 0 checks, 8 lines checked
netdev/build_clang_rust success No Rust files in patch. Skipping build
netdev/kdoc success Errors and warnings before: 0 this patch: 0
netdev/source_inline success Was 0 now: 0
bpf/vmtest-bpf-PR success PR summary
bpf/vmtest-bpf-VM_Test-0 success Logs for Lint
bpf/vmtest-bpf-VM_Test-1 success Logs for ShellCheck
bpf/vmtest-bpf-VM_Test-3 success Logs for Validate matrix.py
bpf/vmtest-bpf-VM_Test-5 success Logs for aarch64-gcc / build-release
bpf/vmtest-bpf-VM_Test-2 success Logs for Unittests
bpf/vmtest-bpf-VM_Test-4 success Logs for aarch64-gcc / build / build for aarch64 with gcc
bpf/vmtest-bpf-VM_Test-9 success Logs for aarch64-gcc / test (test_verifier, false, 360) / test_verifier on aarch64 with gcc
bpf/vmtest-bpf-VM_Test-12 success Logs for s390x-gcc / build-release
bpf/vmtest-bpf-VM_Test-10 success Logs for aarch64-gcc / veristat
bpf/vmtest-bpf-VM_Test-11 success Logs for s390x-gcc / build / build for s390x with gcc
bpf/vmtest-bpf-VM_Test-17 success Logs for s390x-gcc / veristat
bpf/vmtest-bpf-VM_Test-20 success Logs for x86_64-gcc / build-release
bpf/vmtest-bpf-VM_Test-18 success Logs for set-matrix
bpf/vmtest-bpf-VM_Test-21 success Logs for x86_64-gcc / test (test_maps, false, 360) / test_maps on x86_64 with gcc
bpf/vmtest-bpf-VM_Test-27 success Logs for x86_64-gcc / veristat / veristat on x86_64 with gcc
bpf/vmtest-bpf-VM_Test-24 success Logs for x86_64-gcc / test (test_progs_no_alu32_parallel, true, 30) / test_progs_no_alu32_parallel on x86_64 with gcc
bpf/vmtest-bpf-VM_Test-19 success Logs for x86_64-gcc / build / build for x86_64 with gcc
bpf/vmtest-bpf-VM_Test-29 success Logs for x86_64-llvm-17 / build-release / build for x86_64 with llvm-17-O2
bpf/vmtest-bpf-VM_Test-26 success Logs for x86_64-gcc / test (test_verifier, false, 360) / test_verifier on x86_64 with gcc
bpf/vmtest-bpf-VM_Test-25 success Logs for x86_64-gcc / test (test_progs_parallel, true, 30) / test_progs_parallel on x86_64 with gcc
bpf/vmtest-bpf-VM_Test-28 success Logs for x86_64-llvm-17 / build / build for x86_64 with llvm-17
bpf/vmtest-bpf-VM_Test-30 success Logs for x86_64-llvm-17 / test (test_maps, false, 360) / test_maps on x86_64 with llvm-17
bpf/vmtest-bpf-VM_Test-33 success Logs for x86_64-llvm-17 / test (test_verifier, false, 360) / test_verifier on x86_64 with llvm-17
bpf/vmtest-bpf-VM_Test-41 success Logs for x86_64-llvm-18 / test (test_verifier, false, 360) / test_verifier on x86_64 with llvm-18
bpf/vmtest-bpf-VM_Test-35 success Logs for x86_64-llvm-18 / build / build for x86_64 with llvm-18
bpf/vmtest-bpf-VM_Test-42 success Logs for x86_64-llvm-18 / veristat
bpf/vmtest-bpf-VM_Test-34 success Logs for x86_64-llvm-17 / veristat
bpf/vmtest-bpf-VM_Test-37 success Logs for x86_64-llvm-18 / test (test_maps, false, 360) / test_maps on x86_64 with llvm-18
bpf/vmtest-bpf-VM_Test-36 success Logs for x86_64-llvm-18 / build-release / build for x86_64 with llvm-18-O2
bpf/vmtest-bpf-VM_Test-6 success Logs for aarch64-gcc / test (test_maps, false, 360) / test_maps on aarch64 with gcc
bpf/vmtest-bpf-VM_Test-7 success Logs for aarch64-gcc / test (test_progs, false, 360) / test_progs on aarch64 with gcc
bpf/vmtest-bpf-VM_Test-22 success Logs for x86_64-gcc / test (test_progs, false, 360) / test_progs on x86_64 with gcc
bpf/vmtest-bpf-VM_Test-8 success Logs for aarch64-gcc / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on aarch64 with gcc
bpf/vmtest-bpf-VM_Test-23 success Logs for x86_64-gcc / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on x86_64 with gcc
bpf/vmtest-bpf-VM_Test-32 success Logs for x86_64-llvm-17 / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on x86_64 with llvm-17
bpf/vmtest-bpf-VM_Test-39 success Logs for x86_64-llvm-18 / test (test_progs_cpuv4, false, 360) / test_progs_cpuv4 on x86_64 with llvm-18
bpf/vmtest-bpf-VM_Test-31 success Logs for x86_64-llvm-17 / test (test_progs, false, 360) / test_progs on x86_64 with llvm-17
bpf/vmtest-bpf-VM_Test-40 success Logs for x86_64-llvm-18 / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on x86_64 with llvm-18
bpf/vmtest-bpf-VM_Test-38 success Logs for x86_64-llvm-18 / test (test_progs, false, 360) / test_progs on x86_64 with llvm-18
bpf/vmtest-bpf-VM_Test-13 success Logs for s390x-gcc / test (test_maps, false, 360) / test_maps on s390x with gcc
bpf/vmtest-bpf-VM_Test-16 success Logs for s390x-gcc / test (test_verifier, false, 360) / test_verifier on s390x with gcc
bpf/vmtest-bpf-VM_Test-14 success Logs for s390x-gcc / test (test_progs, false, 360) / test_progs on s390x with gcc
bpf/vmtest-bpf-VM_Test-15 success Logs for s390x-gcc / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on s390x with gcc

Commit Message

Antoine Tenart June 24, 2024, 9:09 a.m. UTC 
When upgrading to libbpf 1.3 we noticed a big performance hit while
loading programs using CORE on non base-BTF symbols. This was tracked
down to the new BTF sanity check logic. The issue is the base BTF
definitions are checked first for the base BTF and then again for every
module BTF.

Loading 5 dummy programs (using libbpf-rs) that are using CORE on a
non-base BTF symbol on my system:
- Before this fix: 3s.
- With this fix: 0.1s.

Fix this by only checking the types starting at the BTF start id. This
should ensure the base BTF is still checked as expected but only once
(btf->start_id == 1 when creating the base BTF), and then only
additional types are checked for each module BTF.

Fixes: 3903802bb99a ("libbpf: Add basic BTF sanity validation")
Signed-off-by: Antoine Tenart <atenart@kernel.org>
---
 tools/lib/bpf/btf.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Alan Maguire June 24, 2024, 1:30 p.m. UTC | #1 
On 24/06/2024 10:09, Antoine Tenart wrote:
> When upgrading to libbpf 1.3 we noticed a big performance hit while
> loading programs using CORE on non base-BTF symbols. This was tracked
> down to the new BTF sanity check logic. The issue is the base BTF
> definitions are checked first for the base BTF and then again for every
> module BTF.
> 
> Loading 5 dummy programs (using libbpf-rs) that are using CORE on a
> non-base BTF symbol on my system:
> - Before this fix: 3s.
> - With this fix: 0.1s.
> 
> Fix this by only checking the types starting at the BTF start id. This
> should ensure the base BTF is still checked as expected but only once
> (btf->start_id == 1 when creating the base BTF), and then only
> additional types are checked for each module BTF.
> 
> Fixes: 3903802bb99a ("libbpf: Add basic BTF sanity validation")
> Signed-off-by: Antoine Tenart <atenart@kernel.org>

This looks good to me.

Reviewed-by: Alan Maguire <alan.maguire@oracle.com>

> ---
>  tools/lib/bpf/btf.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index 2d0840ef599a..142060bbce0a 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -598,7 +598,7 @@ static int btf_sanity_check(const struct btf *btf)
>  	__u32 i, n = btf__type_cnt(btf);
>  	int err;
>  
> -	for (i = 1; i < n; i++) {
> +	for (i = btf->start_id; i < n; i++) {
>  		t = btf_type_by_id(btf, i);
>  		err = btf_validate_type(btf, t, i);
>  		if (err)
Andrii Nakryiko June 24, 2024, 4:18 p.m. UTC | #2 
On Mon, Jun 24, 2024 at 2:09 AM Antoine Tenart <atenart@kernel.org> wrote:
>
> When upgrading to libbpf 1.3 we noticed a big performance hit while
> loading programs using CORE on non base-BTF symbols. This was tracked
> down to the new BTF sanity check logic. The issue is the base BTF
> definitions are checked first for the base BTF and then again for every
> module BTF.
>
> Loading 5 dummy programs (using libbpf-rs) that are using CORE on a
> non-base BTF symbol on my system:
> - Before this fix: 3s.
> - With this fix: 0.1s.
>
> Fix this by only checking the types starting at the BTF start id. This
> should ensure the base BTF is still checked as expected but only once
> (btf->start_id == 1 when creating the base BTF), and then only
> additional types are checked for each module BTF.
>
> Fixes: 3903802bb99a ("libbpf: Add basic BTF sanity validation")
> Signed-off-by: Antoine Tenart <atenart@kernel.org>
> ---
>  tools/lib/bpf/btf.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>

Makes total sense, thanks, applied!

> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index 2d0840ef599a..142060bbce0a 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -598,7 +598,7 @@ static int btf_sanity_check(const struct btf *btf)
>         __u32 i, n = btf__type_cnt(btf);
>         int err;
>
> -       for (i = 1; i < n; i++) {
> +       for (i = btf->start_id; i < n; i++) {
>                 t = btf_type_by_id(btf, i);
>                 err = btf_validate_type(btf, t, i);
>                 if (err)
> --
> 2.45.2
>
patchwork-bot+netdevbpf@kernel.org June 24, 2024, 8:10 p.m. UTC | #3 
Hello:

This patch was applied to bpf/bpf-next.git (master)
by Andrii Nakryiko <andrii@kernel.org>:

On Mon, 24 Jun 2024 11:09:07 +0200 you wrote:
> When upgrading to libbpf 1.3 we noticed a big performance hit while
> loading programs using CORE on non base-BTF symbols. This was tracked
> down to the new BTF sanity check logic. The issue is the base BTF
> definitions are checked first for the base BTF and then again for every
> module BTF.
> 
> Loading 5 dummy programs (using libbpf-rs) that are using CORE on a
> non-base BTF symbol on my system:
> - Before this fix: 3s.
> - With this fix: 0.1s.
> 
> [...]

Here is the summary with links:
  - [bpf] libbpf: skip base btf sanity checks
    https://git.kernel.org/bpf/bpf-next/c/c73a9683cb21

You are awesome, thank you!
diff mbox series

Patch

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 2d0840ef599a..142060bbce0a 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -598,7 +598,7 @@  static int btf_sanity_check(const struct btf *btf)
 	__u32 i, n = btf__type_cnt(btf);
 	int err;
 
-	for (i = 1; i < n; i++) {
+	for (i = btf->start_id; i < n; i++) {
 		t = btf_type_by_id(btf, i);
 		err = btf_validate_type(btf, t, i);
 		if (err)