| Message ID | 20230123145148.2791939-4-eddyz87@gmail.com (mailing list archive) |
|---|---|
| State | RFC |
| Delegated to: | BPF |
| Headers | show |
| Series | test_verifier tests migration to inline assembly | expand |
On Mon, Jan 23, 2023 at 6:52 AM Eduard Zingerman <eddyz87@gmail.com> wrote: > > Automatically generate boilerplate code necessary to run tests that > use test_loader.c. > > Adds a target 'prog_tests/test_loader_auto_wrappers.c' as part of > rulesets for 'test_progs' and 'test_progs-no_alu32'. The content of > this C file is generated by make and has the following structure: > > #include <test_progs.h> > > #include "some_test_1.skel.h" > #include "some_test_2.skel.h" > ... > > void test_some_test_1(void) { RUN_TESTS(some_test_1); } > void test_some_test_2(void) { RUN_TESTS(some_test_2); } > ... > > Here RUN_TESTS is a macro defined in test_progs.h, it expands to a > code that uses test_loader.c:test_loader__run_subtests() function to > load tests specified by appropriate skel.h. > > In order to get the list of tests included in > 'test_loader_auto_wrappers.c' the generation script looks for > 'progs/*.c' files that contain a special comment: > > /* Use test_loader marker */ > > Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> > --- It feels like this is a bit of an overkill, tbh. There are $ ls verifier/*.c | wc -l 94 files. We can move each migrated set of tests from verifier/xxx.c to progs/verifier_xxx.c. And then just have just manually maintained prog_tests/verifier.c file where for each converted test we have one #include and one void test_some_test_1(void) { RUN_TESTS(some_test_1); }. It sometimes would useful to add some extra debugging printfs in such a file, so having it auto generated would be actually an inconvenience. And that on top of further Makefile complication. For initial conversion we can auto-generate this file, of course. And then for each migrated file adding 2 lines manually doesn't seem like a big deal? > tools/testing/selftests/bpf/Makefile | 34 +++++++++++++++++++ > .../selftests/bpf/prog_tests/.gitignore | 1 + > 2 files changed, 35 insertions(+) > [...]
On Wed, 2023-01-25 at 17:43 -0800, Andrii Nakryiko wrote: > On Mon, Jan 23, 2023 at 6:52 AM Eduard Zingerman <eddyz87@gmail.com> wrote: > > > > Automatically generate boilerplate code necessary to run tests that > > use test_loader.c. > > > > Adds a target 'prog_tests/test_loader_auto_wrappers.c' as part of > > rulesets for 'test_progs' and 'test_progs-no_alu32'. The content of > > this C file is generated by make and has the following structure: > > > > #include <test_progs.h> > > > > #include "some_test_1.skel.h" > > #include "some_test_2.skel.h" > > ... > > > > void test_some_test_1(void) { RUN_TESTS(some_test_1); } > > void test_some_test_2(void) { RUN_TESTS(some_test_2); } > > ... > > > > Here RUN_TESTS is a macro defined in test_progs.h, it expands to a > > code that uses test_loader.c:test_loader__run_subtests() function to > > load tests specified by appropriate skel.h. > > > > In order to get the list of tests included in > > 'test_loader_auto_wrappers.c' the generation script looks for > > 'progs/*.c' files that contain a special comment: > > > > /* Use test_loader marker */ > > > > Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> > > --- > > It feels like this is a bit of an overkill, tbh. There are > > $ ls verifier/*.c | wc -l > 94 > > files. We can move each migrated set of tests from verifier/xxx.c to > progs/verifier_xxx.c. And then just have just manually maintained > prog_tests/verifier.c file where for each converted test we have one > #include and one void test_some_test_1(void) { RUN_TESTS(some_test_1); > }. > > It sometimes would useful to add some extra debugging printfs in such > a file, so having it auto generated would be actually an > inconvenience. And that on top of further Makefile complication. > > For initial conversion we can auto-generate this file, of course. And > then for each migrated file adding 2 lines manually doesn't seem like > a big deal? Ok, I'll remove the makefile changes. > > > > > tools/testing/selftests/bpf/Makefile | 34 +++++++++++++++++++ > > .../selftests/bpf/prog_tests/.gitignore | 1 + > > 2 files changed, 35 insertions(+) > > > > [...]
diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 26e66f9a0977..66ba2941677c 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -398,6 +398,11 @@ TRUNNER_OUTPUT := $(OUTPUT)$(if $2,/)$2 TRUNNER_BINARY := $1$(if $2,-)$2 TRUNNER_TEST_OBJS := $$(patsubst %.c,$$(TRUNNER_OUTPUT)/%.test.o, \ $$(notdir $$(wildcard $(TRUNNER_TESTS_DIR)/*.c))) +ifneq ($(TRUNNER_TEST_LOADER_AUTO_WRAPPERS),) +TRUNNER_TEST_OBJS := $$(filter-out $$(TRUNNER_OUTPUT)/test_loader_auto_wrappers.test.o, \ + $$(TRUNNER_TEST_OBJS)) +TRUNNER_TEST_OBJS += $$(TRUNNER_OUTPUT)/test_loader_auto_wrappers.test.o +endif TRUNNER_EXTRA_OBJS := $$(patsubst %.c,$$(TRUNNER_OUTPUT)/%.o, \ $$(filter %.c,$(TRUNNER_EXTRA_SOURCES))) TRUNNER_EXTRA_HDRS := $$(filter %.h,$(TRUNNER_EXTRA_SOURCES)) @@ -482,6 +487,32 @@ $(TRUNNER_TESTS_HDR): $(TRUNNER_TESTS_DIR)/*.c ) > $$@) endif +ifneq ($(TRUNNER_TEST_LOADER_AUTO_WRAPPERS),) +ifeq ($($(TRUNNER_TESTS_DIR)-test-loader-auto-wrappers-c),) +$(TRUNNER_TESTS_DIR)-test-loader-auto-wrappers-c := y +$(TRUNNER_TESTS_DIR)/test_loader_auto_wrappers.c: $(TRUNNER_BPF_PROGS_DIR)/*.c + $$(call msg,GEN-TEST,$(TRUNNER_BINARY),$$@) + $$(shell (echo '/* Generated source, do not edit */'; \ + tests=$$$$(grep --null -lF '/* Use test_loader marker */' \ + $$(TRUNNER_BPF_PROGS_DIR)/*.c \ + | xargs -0 -I {} basename {} .c); \ + echo "#include <test_progs.h>"; \ + echo ""; \ + for case in $$$$tests; \ + do \ + echo "#include \"$$$$case.skel.h\""; \ + done; \ + echo ""; \ + for case in $$$$tests; \ + do \ + printf "void %-50s { %-50s }\n" \ + "test_$$$$case(void)" \ + "RUN_TESTS($$$$case);"; \ + done) > $$@) +$(TRUNNER_TESTS_HDR): $(TRUNNER_TESTS_DIR)/test_loader_auto_wrappers.c +endif +endif # TRUNNER_TEST_LOADER_AUTO_WRAPPERS + # compile individual test files # Note: we cd into output directory to ensure embedded BPF object is found $(TRUNNER_TEST_OBJS): $(TRUNNER_OUTPUT)/%.test.o: \ @@ -537,6 +568,7 @@ TRUNNER_EXTRA_FILES := $(OUTPUT)/urandom_read $(OUTPUT)/bpf_testmod.ko \ verify_sig_setup.sh \ $(wildcard progs/btf_dump_test_case_*.c) \ $(wildcard progs/*.bpf.o) +TRUNNER_TEST_LOADER_AUTO_WRAPPERS := t TRUNNER_BPF_BUILD_RULE := CLANG_BPF_BUILD_RULE TRUNNER_BPF_CFLAGS := $(BPF_CFLAGS) $(CLANG_CFLAGS) -DENABLE_ATOMICS_TESTS $(eval $(call DEFINE_TEST_RUNNER,test_progs)) @@ -560,6 +592,7 @@ TRUNNER_EXTRA_SOURCES := test_maps.c TRUNNER_EXTRA_FILES := TRUNNER_BPF_BUILD_RULE := $$(error no BPF objects should be built) TRUNNER_BPF_CFLAGS := +TRUNNER_TEST_LOADER_AUTO_WRAPPERS := $(eval $(call DEFINE_TEST_RUNNER,test_maps)) # Define test_verifier test runner. @@ -625,6 +658,7 @@ $(OUTPUT)/veristat: $(OUTPUT)/veristat.o EXTRA_CLEAN := $(TEST_CUSTOM_PROGS) $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ + prog_tests/test_loader_auto_wrappers.c \ feature bpftool \ $(addprefix $(OUTPUT)/,*.o *.skel.h *.lskel.h *.subskel.h \ no_alu32 bpf_gcc bpf_testmod.ko \ diff --git a/tools/testing/selftests/bpf/prog_tests/.gitignore b/tools/testing/selftests/bpf/prog_tests/.gitignore index 89c4a3d37544..6a937c1cd78d 100644 --- a/tools/testing/selftests/bpf/prog_tests/.gitignore +++ b/tools/testing/selftests/bpf/prog_tests/.gitignore @@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only tests.h +test_loader_auto_wrappers.c
Automatically generate boilerplate code necessary to run tests that use test_loader.c. Adds a target 'prog_tests/test_loader_auto_wrappers.c' as part of rulesets for 'test_progs' and 'test_progs-no_alu32'. The content of this C file is generated by make and has the following structure: #include <test_progs.h> #include "some_test_1.skel.h" #include "some_test_2.skel.h" ... void test_some_test_1(void) { RUN_TESTS(some_test_1); } void test_some_test_2(void) { RUN_TESTS(some_test_2); } ... Here RUN_TESTS is a macro defined in test_progs.h, it expands to a code that uses test_loader.c:test_loader__run_subtests() function to load tests specified by appropriate skel.h. In order to get the list of tests included in 'test_loader_auto_wrappers.c' the generation script looks for 'progs/*.c' files that contain a special comment: /* Use test_loader marker */ Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> --- tools/testing/selftests/bpf/Makefile | 34 +++++++++++++++++++ .../selftests/bpf/prog_tests/.gitignore | 1 + 2 files changed, 35 insertions(+)