diff mbox series

[6/6] selftests/bpf: Skip ENOTSUPP in ASSERT_GE

Message ID a4e9f830d9e10de3ce5c56d0e15fcc0f209aa089.1720075006.git.tanggeliang@kylinos.cn (mailing list archive)
State Superseded
Delegated to: BPF
Headers show
Series skip ENOTSUPP for BPF selftests | expand

Checks

Context Check Description
netdev/tree_selection success Not a local patch
bpf/vmtest-bpf-next-PR success PR summary
bpf/vmtest-bpf-next-VM_Test-0 success Logs for Lint
bpf/vmtest-bpf-next-VM_Test-1 success Logs for ShellCheck
bpf/vmtest-bpf-next-VM_Test-2 success Logs for Unittests
bpf/vmtest-bpf-next-VM_Test-3 success Logs for Validate matrix.py
bpf/vmtest-bpf-next-VM_Test-5 success Logs for aarch64-gcc / build-release
bpf/vmtest-bpf-next-VM_Test-4 success Logs for aarch64-gcc / build / build for aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-16 success Logs for s390x-gcc / test (test_verifier, false, 360) / test_verifier on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-17 success Logs for s390x-gcc / veristat
bpf/vmtest-bpf-next-VM_Test-18 success Logs for set-matrix
bpf/vmtest-bpf-next-VM_Test-19 success Logs for x86_64-gcc / build / build for x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-20 success Logs for x86_64-gcc / build-release
bpf/vmtest-bpf-next-VM_Test-21 success Logs for x86_64-gcc / test (test_maps, false, 360) / test_maps on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-24 success Logs for x86_64-gcc / test (test_progs_no_alu32_parallel, true, 30) / test_progs_no_alu32_parallel on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-25 success Logs for x86_64-gcc / test (test_progs_parallel, true, 30) / test_progs_parallel on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-26 success Logs for x86_64-gcc / test (test_verifier, false, 360) / test_verifier on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-28 success Logs for x86_64-llvm-17 / build / build for x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-12 success Logs for s390x-gcc / build-release
bpf/vmtest-bpf-next-VM_Test-9 success Logs for aarch64-gcc / test (test_verifier, false, 360) / test_verifier on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-10 success Logs for aarch64-gcc / veristat
bpf/vmtest-bpf-next-VM_Test-29 success Logs for x86_64-llvm-17 / build-release / build for x86_64 with llvm-17-O2
bpf/vmtest-bpf-next-VM_Test-30 success Logs for x86_64-llvm-17 / test (test_maps, false, 360) / test_maps on x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-11 success Logs for s390x-gcc / build / build for s390x with gcc
bpf/vmtest-bpf-next-VM_Test-13 success Logs for s390x-gcc / test (test_maps, false, 360) / test_maps on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-33 success Logs for x86_64-llvm-17 / test (test_verifier, false, 360) / test_verifier on x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-34 success Logs for x86_64-llvm-17 / veristat
bpf/vmtest-bpf-next-VM_Test-35 success Logs for x86_64-llvm-18 / build / build for x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-36 success Logs for x86_64-llvm-18 / build-release / build for x86_64 with llvm-18-O2
bpf/vmtest-bpf-next-VM_Test-37 success Logs for x86_64-llvm-18 / test (test_maps, false, 360) / test_maps on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-41 success Logs for x86_64-llvm-18 / test (test_verifier, false, 360) / test_verifier on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-42 success Logs for x86_64-llvm-18 / veristat
bpf/vmtest-bpf-next-VM_Test-15 success Logs for s390x-gcc / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on s390x with gcc
bpf/vmtest-bpf-next-VM_Test-22 success Logs for x86_64-gcc / test (test_progs, false, 360) / test_progs on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-23 success Logs for x86_64-gcc / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-27 success Logs for x86_64-gcc / veristat / veristat on x86_64 with gcc
bpf/vmtest-bpf-next-VM_Test-6 success Logs for aarch64-gcc / test (test_maps, false, 360) / test_maps on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-31 success Logs for x86_64-llvm-17 / test (test_progs, false, 360) / test_progs on x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-32 success Logs for x86_64-llvm-17 / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on x86_64 with llvm-17
bpf/vmtest-bpf-next-VM_Test-8 success Logs for aarch64-gcc / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-7 success Logs for aarch64-gcc / test (test_progs, false, 360) / test_progs on aarch64 with gcc
bpf/vmtest-bpf-next-VM_Test-38 success Logs for x86_64-llvm-18 / test (test_progs, false, 360) / test_progs on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-39 success Logs for x86_64-llvm-18 / test (test_progs_cpuv4, false, 360) / test_progs_cpuv4 on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-40 success Logs for x86_64-llvm-18 / test (test_progs_no_alu32, false, 360) / test_progs_no_alu32 on x86_64 with llvm-18
bpf/vmtest-bpf-next-VM_Test-14 success Logs for s390x-gcc / test (test_progs, false, 360) / test_progs on s390x with gcc

Commit Message

Geliang Tang July 4, 2024, 6:48 a.m. UTC 
From: Geliang Tang <tanggeliang@kylinos.cn>

There are still some "ENOTSUPP" (-524) errors left when running BPF
selftests on a Loongarch platform since ASSERT_GE() are used there to
check the return values, not ASSERT_OK():

'''
 test_bpf_cookie:PASS:skel_open 0 nsec
 #17/1    bpf_cookie/kprobe:OK
 #17/2    bpf_cookie/multi_kprobe_link_api:OK
 #17/3    bpf_cookie/multi_kprobe_attach_api:OK
 #17/4    bpf_cookie/uprobe:OK
 #17/5    bpf_cookie/multi_uprobe_attach_api:OK
 #17/6    bpf_cookie/tracepoint:OK
 #17/7    bpf_cookie/perf_event:OK
 tracing_subtest:FAIL:fentry.link_create unexpected fentry.link_create: \
						actual -524 < expected 0
 #17/8    bpf_cookie/trampoline:FAIL
 lsm_subtest:FAIL:lsm.link_create unexpected lsm.link_create: \
						actual -524 < expected 0
 #17/9    bpf_cookie/lsm:FAIL
 #17/10   bpf_cookie/tp_btf:OK
 #17/11   bpf_cookie/raw_tp:OK
 #17      bpf_cookie:FAIL
 ... ...
 test_module_fentry_shadow:PASS:load_vmlinux_btf 0 nsec
 test_module_fentry_shadow:PASS:get_bpf_testmod_btf_fd 0 nsec
 test_module_fentry_shadow:PASS:btf_get_from_fd 0 nsec
 test_module_fentry_shadow:PASS:btf_find_by_name 0 nsec
 test_module_fentry_shadow:PASS:btf_find_by_name 0 nsec
 test_module_fentry_shadow:PASS:bpf_prog_load 0 nsec
 test_module_fentry_shadow:FAIL:bpf_link_create unexpected \
				bpf_link_create: actual -524 < expected 0
 #168     module_fentry_shadow:FAIL
'''

Just like in ASSERT_OK(), this patch skips ENOTSUPP (524) and ENOTSUP (95)
in ASSERT_GT() too. With this change, the new output of these selftests
look like:

'''
 #17/1    bpf_cookie/kprobe:OK
 #17/2    bpf_cookie/multi_kprobe_link_api:OK
 #17/3    bpf_cookie/multi_kprobe_attach_api:OK
 #17/4    bpf_cookie/uprobe:OK
 #17/5    bpf_cookie/multi_uprobe_attach_api:OK
 #17/6    bpf_cookie/tracepoint:OK
 #17/7    bpf_cookie/perf_event:OK
 #17/8    bpf_cookie/trampoline:SKIP
 #17/9    bpf_cookie/lsm:SKIP
 #17/10   bpf_cookie/tp_btf:SKIP
 #17/11   bpf_cookie/raw_tp:SKIP
 #17      bpf_cookie:OK (SKIP: 4/11)
 ... ...
 #168     module_fentry_shadow:SKIP
'''

Signed-off-by: Geliang Tang <tanggeliang@kylinos.cn>
---
 tools/testing/selftests/bpf/test_progs.h | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

Comments

Andrii Nakryiko July 8, 2024, 6:56 p.m. UTC | #1 
On Wed, Jul 3, 2024 at 11:48 PM Geliang Tang <geliang@kernel.org> wrote:
>
> From: Geliang Tang <tanggeliang@kylinos.cn>
>
> There are still some "ENOTSUPP" (-524) errors left when running BPF
> selftests on a Loongarch platform since ASSERT_GE() are used there to
> check the return values, not ASSERT_OK():
>
> '''
>  test_bpf_cookie:PASS:skel_open 0 nsec
>  #17/1    bpf_cookie/kprobe:OK
>  #17/2    bpf_cookie/multi_kprobe_link_api:OK
>  #17/3    bpf_cookie/multi_kprobe_attach_api:OK
>  #17/4    bpf_cookie/uprobe:OK
>  #17/5    bpf_cookie/multi_uprobe_attach_api:OK
>  #17/6    bpf_cookie/tracepoint:OK
>  #17/7    bpf_cookie/perf_event:OK
>  tracing_subtest:FAIL:fentry.link_create unexpected fentry.link_create: \
>                                                 actual -524 < expected 0
>  #17/8    bpf_cookie/trampoline:FAIL
>  lsm_subtest:FAIL:lsm.link_create unexpected lsm.link_create: \
>                                                 actual -524 < expected 0
>  #17/9    bpf_cookie/lsm:FAIL
>  #17/10   bpf_cookie/tp_btf:OK
>  #17/11   bpf_cookie/raw_tp:OK
>  #17      bpf_cookie:FAIL
>  ... ...
>  test_module_fentry_shadow:PASS:load_vmlinux_btf 0 nsec
>  test_module_fentry_shadow:PASS:get_bpf_testmod_btf_fd 0 nsec
>  test_module_fentry_shadow:PASS:btf_get_from_fd 0 nsec
>  test_module_fentry_shadow:PASS:btf_find_by_name 0 nsec
>  test_module_fentry_shadow:PASS:btf_find_by_name 0 nsec
>  test_module_fentry_shadow:PASS:bpf_prog_load 0 nsec
>  test_module_fentry_shadow:FAIL:bpf_link_create unexpected \
>                                 bpf_link_create: actual -524 < expected 0
>  #168     module_fentry_shadow:FAIL
> '''
>
> Just like in ASSERT_OK(), this patch skips ENOTSUPP (524) and ENOTSUP (95)
> in ASSERT_GT() too. With this change, the new output of these selftests
> look like:
>
> '''
>  #17/1    bpf_cookie/kprobe:OK
>  #17/2    bpf_cookie/multi_kprobe_link_api:OK
>  #17/3    bpf_cookie/multi_kprobe_attach_api:OK
>  #17/4    bpf_cookie/uprobe:OK
>  #17/5    bpf_cookie/multi_uprobe_attach_api:OK
>  #17/6    bpf_cookie/tracepoint:OK
>  #17/7    bpf_cookie/perf_event:OK
>  #17/8    bpf_cookie/trampoline:SKIP
>  #17/9    bpf_cookie/lsm:SKIP
>  #17/10   bpf_cookie/tp_btf:SKIP
>  #17/11   bpf_cookie/raw_tp:SKIP
>  #17      bpf_cookie:OK (SKIP: 4/11)
>  ... ...
>  #168     module_fentry_shadow:SKIP
> '''
>
> Signed-off-by: Geliang Tang <tanggeliang@kylinos.cn>
> ---
>  tools/testing/selftests/bpf/test_progs.h | 10 +++++++---
>  1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/test_progs.h b/tools/testing/selftests/bpf/test_progs.h
> index 464aa12feada..8a5ce360aa1c 100644
> --- a/tools/testing/selftests/bpf/test_progs.h
> +++ b/tools/testing/selftests/bpf/test_progs.h
> @@ -294,9 +294,13 @@ int test__join_cgroup(const char *path);
>         typeof(actual) ___act = (actual);                               \
>         typeof(expected) ___exp = (expected);                           \
>         bool ___ok = ___act >= ___exp;                                  \
> -       CHECK(!___ok, (name),                                           \
> -             "unexpected %s: actual %lld < expected %lld\n",           \
> -             (name), (long long)(___act), (long long)(___exp));        \
> +       if (___act == -ENOTSUPP || ___act == -ENOTSUP ||                \
> +           errno == ENOTSUPP || errno == ENOTSUP)                      \
> +               test__skip();                                           \

absolutely not, if ENOTSUP or ENOTSUPP is special for some tests, that
should be handled explicitly. This is not correct in general and we
should not just silently skip some test on such errors.

> +       else                                                            \
> +               CHECK(!___ok, (name),                                   \
> +                     "unexpected %s: actual %lld < expected %lld\n",   \
> +                     (name), (long long)(___act), (long long)(___exp));\
>         ___ok;                                                          \
>  })
>
> --
> 2.43.0
>
diff mbox series

Patch

diff --git a/tools/testing/selftests/bpf/test_progs.h b/tools/testing/selftests/bpf/test_progs.h
index 464aa12feada..8a5ce360aa1c 100644
--- a/tools/testing/selftests/bpf/test_progs.h
+++ b/tools/testing/selftests/bpf/test_progs.h
@@ -294,9 +294,13 @@  int test__join_cgroup(const char *path);
 	typeof(actual) ___act = (actual);				\
 	typeof(expected) ___exp = (expected);				\
 	bool ___ok = ___act >= ___exp;					\
-	CHECK(!___ok, (name),						\
-	      "unexpected %s: actual %lld < expected %lld\n",		\
-	      (name), (long long)(___act), (long long)(___exp));	\
+	if (___act == -ENOTSUPP || ___act == -ENOTSUP ||		\
+	    errno == ENOTSUPP || errno == ENOTSUP)			\
+		test__skip();						\
+	else								\
+		CHECK(!___ok, (name),					\
+		      "unexpected %s: actual %lld < expected %lld\n",	\
+		      (name), (long long)(___act), (long long)(___exp));\
 	___ok;								\
 })