| Message ID | 20241021193310.2014131-1-mcgrof@kernel.org (mailing list archive) |
|---|---|
| State | Handled Elsewhere |
| Headers | show |
| Series | [v3] selftests: add new kallsyms selftests | expand |
| Context | Check | Description |
|---|---|---|
| mcgrof/vmtest-modules-next-PR | success | PR summary |
| mcgrof/vmtest-modules-next-VM_Test-1 | success | Logs for Run kdevops CI on Self-hosted Runner |
| mcgrof/vmtest-modules-next-VM_Test-0 | success | Logs for Run kdevops CI on Self-hosted Runner |
| mcgrof/vmtest-main-PR | fail | merge-conflict |
On 10/21/24 12:33, Luis Chamberlain wrote: ... > +gen_template_module_exit() > +{ > + cat <<____END_MODULE > +static int __init auto_test_module_init(void) > +{ > + return auto_runtime_test(); > +} > +module_init(auto_test_module_init); > + > +static void __exit auto_test_module_exit(void) > +{ > +} > +module_exit(auto_test_module_exit); > + > +MODULE_AUTHOR("Luis Chamberlain <mcgrof@kernel.org>"); > +MODULE_LICENSE("GPL"); > +____END_MODULE > +} Since commit 1fffe7a34c89 ("script: modpost: emit a warning when the description is missing"), a module without a MODULE_DESCRIPTION() will result in a warning when built with make W=1. Is that a concern here? Should we add a MODULE_DESCRIPTION()? /jeff
On Wed, Oct 23, 2024 at 10:28:49AM -0700, Jeff Johnson wrote: > On 10/21/24 12:33, Luis Chamberlain wrote: > ... > > +gen_template_module_exit() > > +{ > > + cat <<____END_MODULE > > +static int __init auto_test_module_init(void) > > +{ > > + return auto_runtime_test(); > > +} > > +module_init(auto_test_module_init); > > + > > +static void __exit auto_test_module_exit(void) > > +{ > > +} > > +module_exit(auto_test_module_exit); > > + > > +MODULE_AUTHOR("Luis Chamberlain <mcgrof@kernel.org>"); > > +MODULE_LICENSE("GPL"); > > +____END_MODULE > > +} > > Since commit 1fffe7a34c89 ("script: modpost: emit a warning when the > description is missing"), a module without a MODULE_DESCRIPTION() will > result in a warning when built with make W=1. Is that a concern here? > Should we add a MODULE_DESCRIPTION()? News to me, I'll send a follup patch with just that alone as I already merged this onto modules-next. Luis
Hi Luis, On Mon, Oct 21, 2024 at 12:33 PM Luis Chamberlain <mcgrof@kernel.org> wrote: > > diff --git a/lib/tests/module/gen_test_kallsyms.sh b/lib/tests/module/gen_test_kallsyms.sh > new file mode 100755 > index 000000000000..e85f10dc11bd > --- /dev/null > +++ b/lib/tests/module/gen_test_kallsyms.sh > @@ -0,0 +1,128 @@ [..] > +gen_template_module_data_b() > +{ > + printf "\nextern int auto_test_a_%010d;\n\n" 28 > + echo "static int auto_runtime_test(void)" > + echo "{" > + printf "\nreturn auto_test_a_%010d;\n" 28 > + echo "}" > +} FYI, I get a warning when loading test_kallsyms_b because the init function return value is >0: # modprobe test_kallsyms_b [ 11.154496] do_init_module: 'test_kallsyms_b'->init suspiciously returned 255, it should follow 0/-E convention [ 11.154496] do_init_module: loading module anyway... [ 11.156156] CPU: 3 UID: 0 PID: 116 Comm: modprobe Not tainted 6.12.0-rc5-00020-g897cb2ff413d #1 [ 11.156832] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.2-0-gea1b7a073390-prebuilt.qemu.org 04/01/2014 [ 11.157762] Call Trace: [ 11.158914] <TASK> [ 11.159253] dump_stack_lvl+0x3f/0xb0 [ 11.160279] do_init_module+0x1f4/0x200 [ 11.160586] __se_sys_finit_module+0x30c/0x400 [ 11.160948] do_syscall_64+0xd0/0x1a0 [ 11.161255] ? arch_exit_to_user_mode_prepare+0x11/0x60 [ 11.161659] ? irqentry_exit_to_user_mode+0x8e/0xb0 [ 11.162052] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 11.162598] RIP: 0033:0x7f5843968cf6 [ 11.163076] Code: 48 89 57 30 48 8b 04 24 48 89 47 38 e9 1e 9a 02 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 3a fd ff ff c3 48 c7 c6 01 00 00 00 e9 a1 [ 11.164465] RSP: 002b:00007ffefcc92d68 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [ 11.165046] RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00007f5843968cf6 [ 11.165656] RDX: 0000000000000000 RSI: 00000000320429e0 RDI: 0000000000000003 [ 11.166220] RBP: 00000000320429e0 R08: 0000000000000074 R09: 0000000000000000 [ 11.166804] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000032042b50 [ 11.167378] R13: 0000000000000001 R14: 0000000032042c10 R15: 0000000000000000 [ 11.168007] </TASK> > diff --git a/tools/testing/selftests/module/find_symbol.sh b/tools/testing/selftests/module/find_symbol.sh > new file mode 100755 > index 000000000000..140364d3c49f > --- /dev/null > +++ b/tools/testing/selftests/module/find_symbol.sh > @@ -0,0 +1,81 @@ [..] > +test_reqs() > +{ > + if ! which modprobe 2> /dev/null > /dev/null; then > + echo "$0: You need modprobe installed" >&2 > + exit $ksft_skip > + fi > + > + if ! which kmod 2> /dev/null > /dev/null; then > + echo "$0: You need kmod installed" >&2 > + exit $ksft_skip > + fi Is there a reason to test for kmod? I don't see it called directly in this script. Also, shouldn't you add the module directory to TARGETS in tools/testing/selftests/Makefile? Otherwise the script won't be installed with the rest of kselftests. Sami
On Tue, Oct 29, 2024 at 09:24:30AM -0700, Sami Tolvanen wrote: > Hi Luis, > > On Mon, Oct 21, 2024 at 12:33 PM Luis Chamberlain <mcgrof@kernel.org> wrote: > > > > diff --git a/lib/tests/module/gen_test_kallsyms.sh b/lib/tests/module/gen_test_kallsyms.sh > > new file mode 100755 > > index 000000000000..e85f10dc11bd > > --- /dev/null > > +++ b/lib/tests/module/gen_test_kallsyms.sh > > @@ -0,0 +1,128 @@ > [..] > > +gen_template_module_data_b() > > +{ > > + printf "\nextern int auto_test_a_%010d;\n\n" 28 > > + echo "static int auto_runtime_test(void)" > > + echo "{" > > + printf "\nreturn auto_test_a_%010d;\n" 28 > > + echo "}" > > +} > > FYI, I get a warning when loading test_kallsyms_b because the init > function return value is >0: This fixes it. From b776d662c8e05d67c7879d0f6f5208dd431d900a Mon Sep 17 00:00:00 2001 From: Luis Chamberlain <mcgrof@kernel.org> Date: Wed, 6 Nov 2024 00:17:21 +0000 Subject: [PATCH] tests/module/gen_test_kallsyms.sh: use 0 value for variables Use 0 for the values as we use them for the return value on init to keep the test modules simple. This fixes a splat reported do_init_module: 'test_kallsyms_b'->init suspiciously returned 255, it should follow 0/-E convention do_init_module: loading module anyway... CPU: 5 UID: 0 PID: 1873 Comm: modprobe Not tainted 6.12.0-rc3+ #4 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 2024.08-1 09/18/2024 Call Trace: <TASK> dump_stack_lvl+0x56/0x80 do_init_module.cold+0x21/0x26 init_module_from_file+0x88/0xf0 idempotent_init_module+0x108/0x300 __x64_sys_finit_module+0x5a/0xb0 do_syscall_64+0x4b/0x110 entry_SYSCALL_64_after_hwframe+0x76/0x7e RIP: 0033:0x7f4f3a718839 Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff> RSP: 002b:00007fff97d1a9e8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 RAX: ffffffffffffffda RBX: 000055b94001ab90 RCX: 00007f4f3a718839 RDX: 0000000000000000 RSI: 000055b910e68a10 RDI: 0000000000000004 RBP: 0000000000000000 R08: 00007f4f3a7f1b20 R09: 000055b94001c5b0 R10: 0000000000000040 R11: 0000000000000246 R12: 000055b910e68a10 R13: 0000000000040000 R14: 000055b94001ad60 R15: 0000000000000000 </TASK> do_init_module: 'test_kallsyms_b'->init suspiciously returned 255, it should follow 0/-E convention do_init_module: loading module anyway... CPU: 1 UID: 0 PID: 1884 Comm: modprobe Not tainted 6.12.0-rc3+ #4 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 2024.08-1 09/18/2024 Call Trace: <TASK> dump_stack_lvl+0x56/0x80 do_init_module.cold+0x21/0x26 init_module_from_file+0x88/0xf0 idempotent_init_module+0x108/0x300 __x64_sys_finit_module+0x5a/0xb0 do_syscall_64+0x4b/0x110 entry_SYSCALL_64_after_hwframe+0x76/0x7e RIP: 0033:0x7ffaa5d18839 Reported-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> --- lib/tests/module/gen_test_kallsyms.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/tests/module/gen_test_kallsyms.sh b/lib/tests/module/gen_test_kallsyms.sh index ae5966f1f904..3f2c626350ad 100755 --- a/lib/tests/module/gen_test_kallsyms.sh +++ b/lib/tests/module/gen_test_kallsyms.sh @@ -32,7 +32,7 @@ gen_num_syms() PREFIX=$1 NUM=$2 for i in $(seq 1 $NUM); do - printf "int auto_test_%s_%010d = 0xff;\n" $PREFIX $i + printf "int auto_test_%s_%010d = 0;\n" $PREFIX $i printf "EXPORT_SYMBOL_GPL(auto_test_%s_%010d);\n" $PREFIX $i done echo
Hi Luis, On Mon, Oct 21, 2024 at 9:33 PM Luis Chamberlain <mcgrof@kernel.org> wrote: > We lack find_symbol() selftests, so add one. This let's us stress test > improvements easily on find_symbol() or optimizations. It also inherently > allows us to test the limits of kallsyms on Linux today. > > We test a pathalogical use case for kallsyms by introducing modules > which are automatically written for us with a larger number of symbols. > We have 4 kallsyms test modules: > > A: has KALLSYSMS_NUMSYMS exported symbols > B: uses one of A's symbols > C: adds KALLSYMS_SCALE_FACTOR * KALLSYSMS_NUMSYMS exported > D: adds 2 * the symbols than C > > By using anything much larger than KALLSYSMS_NUMSYMS as 10,000 and > KALLSYMS_SCALE_FACTOR of 8 we segfault today. So we're capped at > around 160000 symbols somehow today. We can inpsect that issue at > our leasure later, but for now the real value to this test is that > this will easily allow us to test improvements on find_symbol(). > > We want to enable this test on allyesmodconfig builds so we can't > use this combination, so instead just use a safe value for now and > be informative on the Kconfig symbol documentation about where our > thresholds are for testers. We default then to KALLSYSMS_NUMSYMS of > just 100 and KALLSYMS_SCALE_FACTOR of 8. > > On x86_64 we can use perf, for other architectures we just use 'time' > and allow for customizations. For example a future enhancements could > be done for parisc to check for unaligned accesses which triggers a > special special exception handler assembler code inside the kernel. > The negative impact on performance is so large on parisc that it > keeps track of its accesses on /proc/cpuinfo as UAH: > > IRQ: CPU0 CPU1 > 3: 1332 0 SuperIO ttyS0 > 7: 1270013 0 SuperIO pata_ns87415 > 64: 320023012 320021431 CPU timer > 65: 17080507 20624423 CPU IPI > UAH: 10948640 58104 Unaligned access handler traps > > While at it, this tidies up lib/ test modules to allow us to have > a new directory for them. The amount of test modules under lib/ > is insane. > > This should also hopefully showcase how to start doing basic > self module writing code, which may be more useful for more complex > cases later in the future. > > Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> Thanks for your patch, which is now commit 84b4a51fce4ccc66 ("selftests: add new kallsyms selftests") upstream. > @@ -2903,6 +2903,111 @@ config TEST_KMOD > > If unsure, say N. > > +config TEST_RUNTIME > + bool > + > +config TEST_RUNTIME_MODULE > + bool > + > +config TEST_KALLSYMS > + tristate "module kallsyms find_symbol() test" > + depends on m > + select TEST_RUNTIME > + select TEST_RUNTIME_MODULE > + select TEST_KALLSYMS_A > + select TEST_KALLSYMS_B > + select TEST_KALLSYMS_C > + select TEST_KALLSYMS_D > + help > + This allows us to stress test find_symbol() through the kallsyms > + used to place symbols on the kernel ELF kallsyms and modules kallsyms > + where we place kernel symbols such as exported symbols. > + > + We have four test modules: > + > + A: has KALLSYSMS_NUMSYMS exported symbols > + B: uses one of A's symbols > + C: adds KALLSYMS_SCALE_FACTOR * KALLSYSMS_NUMSYMS exported > + D: adds 2 * the symbols than C > + > + We stress test find_symbol() through two means: > + > + 1) Upon load of B it will trigger simplify_symbols() to look for the > + one symbol it uses from the module A with tons of symbols. This is an > + indirect way for us to have B call resolve_symbol_wait() upon module > + load. This will eventually call find_symbol() which will eventually > + try to find the symbols used with find_exported_symbol_in_section(). > + find_exported_symbol_in_section() uses bsearch() so a binary search > + for each symbol. Binary search will at worst be O(log(n)) so the > + larger TEST_MODULE_KALLSYSMS the worse the search. > + > + 2) The selftests should load C first, before B. Upon B's load towards > + the end right before we call module B's init routine we get > + complete_formation() called on the module. That will first check > + for duplicate symbols with the call to verify_exported_symbols(). > + That is when we'll force iteration on module C's insane symbol list. > + Since it has 10 * KALLSYMS_NUMSYMS it means we can first test > + just loading B without C. The amount of time it takes to load C Vs > + B can give us an idea of the impact growth of the symbol space and > + give us projection. Module A only uses one symbol from B so to allow > + this scaling in module C to be proportional, if it used more symbols > + then the first test would be doing more and increasing just the > + search space would be slightly different. The last module, module D > + will just increase the search space by twice the number of symbols in > + C so to allow for full projects. > + > + tools/testing/selftests/module/find_symbol.sh > + > + The current defaults will incur a build delay of about 7 minutes > + on an x86_64 with only 8 cores. Enable this only if you want to > + stress test find_symbol() with thousands of symbols. At the same > + time this is also useful to test building modules with thousands of > + symbols, and if BTF is enabled this also stress tests adding BTF > + information for each module. Currently enabling many more symbols > + will segfault the build system. Despite the warning, I gave this a try on m68k (cross-compiled on i7 ;-). However, I didn't notice any extra-ordinary build times. Also, when running the test manually on ARAnyM, everything runs in the blink of an eye. I didn't use the script, but ran all commands manually. I tried insmodding a/b/c/d, c/a/b, a/c/d/b. Is this expected? Thanks! $ wc -l lib/tests/module/test_kallsyms_*.c 233 lib/tests/module/test_kallsyms_a.c 22 lib/tests/module/test_kallsyms_a.mod.c 35 lib/tests/module/test_kallsyms_b.c 21 lib/tests/module/test_kallsyms_b.mod.c 1633 lib/tests/module/test_kallsyms_c.c 21 lib/tests/module/test_kallsyms_c.mod.c 3233 lib/tests/module/test_kallsyms_d.c 21 lib/tests/module/test_kallsyms_d.mod.c 5219 total Gr{oetje,eeting}s, Geert
On Thu, Nov 28, 2024 at 03:10:34PM +0100, Geert Uytterhoeven wrote: > Despite the warning, I gave this a try on m68k (cross-compiled on i7 ;-). > However, I didn't notice any extra-ordinary build times. I had some large values before this got merged and due to this slowing down builds 2x I reduced the defaults. The current defaults are no known as TEST_KALLSYMS_LARGE. > Also, when running the test manually on ARAnyM, everything runs > in the blink of an eye. I didn't use the script, but ran all commands > manually. I tried insmodding a/b/c/d, c/a/b, a/c/d/b. > > Is this expected? I tried to clarify the ranges known we can use to mess with this test a bit more in the recent fixes on modules-next [0] so since you want to see some things blow up try TEST_KALLSYMS_MAX, on some systems this is reported to crash the build. It is why I had reduced the original values a bit... [0] git://git.kernel.org/pub/scm/linux/kernel/git/modules/linux.git modules-next Luis
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 7315f643817a..b5929721fc63 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -2903,6 +2903,111 @@ config TEST_KMOD If unsure, say N. +config TEST_RUNTIME + bool + +config TEST_RUNTIME_MODULE + bool + +config TEST_KALLSYMS + tristate "module kallsyms find_symbol() test" + depends on m + select TEST_RUNTIME + select TEST_RUNTIME_MODULE + select TEST_KALLSYMS_A + select TEST_KALLSYMS_B + select TEST_KALLSYMS_C + select TEST_KALLSYMS_D + help + This allows us to stress test find_symbol() through the kallsyms + used to place symbols on the kernel ELF kallsyms and modules kallsyms + where we place kernel symbols such as exported symbols. + + We have four test modules: + + A: has KALLSYSMS_NUMSYMS exported symbols + B: uses one of A's symbols + C: adds KALLSYMS_SCALE_FACTOR * KALLSYSMS_NUMSYMS exported + D: adds 2 * the symbols than C + + We stress test find_symbol() through two means: + + 1) Upon load of B it will trigger simplify_symbols() to look for the + one symbol it uses from the module A with tons of symbols. This is an + indirect way for us to have B call resolve_symbol_wait() upon module + load. This will eventually call find_symbol() which will eventually + try to find the symbols used with find_exported_symbol_in_section(). + find_exported_symbol_in_section() uses bsearch() so a binary search + for each symbol. Binary search will at worst be O(log(n)) so the + larger TEST_MODULE_KALLSYSMS the worse the search. + + 2) The selftests should load C first, before B. Upon B's load towards + the end right before we call module B's init routine we get + complete_formation() called on the module. That will first check + for duplicate symbols with the call to verify_exported_symbols(). + That is when we'll force iteration on module C's insane symbol list. + Since it has 10 * KALLSYMS_NUMSYMS it means we can first test + just loading B without C. The amount of time it takes to load C Vs + B can give us an idea of the impact growth of the symbol space and + give us projection. Module A only uses one symbol from B so to allow + this scaling in module C to be proportional, if it used more symbols + then the first test would be doing more and increasing just the + search space would be slightly different. The last module, module D + will just increase the search space by twice the number of symbols in + C so to allow for full projects. + + tools/testing/selftests/module/find_symbol.sh + + The current defaults will incur a build delay of about 7 minutes + on an x86_64 with only 8 cores. Enable this only if you want to + stress test find_symbol() with thousands of symbols. At the same + time this is also useful to test building modules with thousands of + symbols, and if BTF is enabled this also stress tests adding BTF + information for each module. Currently enabling many more symbols + will segfault the build system. + + If unsure, say N. + +if TEST_KALLSYMS + +config TEST_KALLSYMS_A + tristate + depends on m + +config TEST_KALLSYMS_B + tristate + depends on m + +config TEST_KALLSYMS_C + tristate + depends on m + +config TEST_KALLSYMS_D + tristate + depends on m + +config TEST_KALLSYMS_NUMSYMS + int "test kallsyms number of symbols" + default 100 + help + The number of symbols to create on TEST_KALLSYMS_A, only one of which + module TEST_KALLSYMS_B will use. This also will be used + for how many symbols TEST_KALLSYMS_C will have, scaled up by + TEST_KALLSYMS_SCALE_FACTOR. Note that setting this to 10,000 will + trigger a segfault today, don't use anything close to it unless + you are aware that this should not be used for automated build tests. + +config TEST_KALLSYMS_SCALE_FACTOR + int "test kallsyms scale factor" + default 8 + help + How many more unusued symbols will TEST_KALLSYSMS_C have than + TEST_KALLSYMS_A. If 8, then module C will have 8 * syms + than module A. Then TEST_KALLSYMS_D will have double the amount + of symbols than C so to allow projections. + +endif # TEST_KALLSYMS + config TEST_DEBUG_VIRTUAL tristate "Test CONFIG_DEBUG_VIRTUAL feature" depends on DEBUG_VIRTUAL diff --git a/lib/Makefile b/lib/Makefile index 773adf88af41..ae720c7eb996 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -96,6 +96,7 @@ obj-$(CONFIG_TEST_XARRAY) += test_xarray.o obj-$(CONFIG_TEST_MAPLE_TREE) += test_maple_tree.o obj-$(CONFIG_TEST_PARMAN) += test_parman.o obj-$(CONFIG_TEST_KMOD) += test_kmod.o +obj-$(CONFIG_TEST_RUNTIME) += tests/ obj-$(CONFIG_TEST_DEBUG_VIRTUAL) += test_debug_virtual.o obj-$(CONFIG_TEST_MEMCAT_P) += test_memcat_p.o obj-$(CONFIG_TEST_OBJAGG) += test_objagg.o diff --git a/lib/tests/Makefile b/lib/tests/Makefile new file mode 100644 index 000000000000..8e4f42cb9c54 --- /dev/null +++ b/lib/tests/Makefile @@ -0,0 +1 @@ +obj-$(CONFIG_TEST_RUNTIME_MODULE) += module/ diff --git a/lib/tests/module/.gitignore b/lib/tests/module/.gitignore new file mode 100644 index 000000000000..8be7891b250f --- /dev/null +++ b/lib/tests/module/.gitignore @@ -0,0 +1,4 @@ +test_kallsyms_a.c +test_kallsyms_b.c +test_kallsyms_c.c +test_kallsyms_d.c diff --git a/lib/tests/module/Makefile b/lib/tests/module/Makefile new file mode 100644 index 000000000000..af5c27b996cb --- /dev/null +++ b/lib/tests/module/Makefile @@ -0,0 +1,15 @@ +obj-$(CONFIG_TEST_KALLSYMS_A) += test_kallsyms_a.o +obj-$(CONFIG_TEST_KALLSYMS_B) += test_kallsyms_b.o +obj-$(CONFIG_TEST_KALLSYMS_C) += test_kallsyms_c.o +obj-$(CONFIG_TEST_KALLSYMS_D) += test_kallsyms_d.o + +$(obj)/%.c: FORCE + @$(kecho) " GEN $@" + $(Q)$(srctree)/lib/tests/module/gen_test_kallsyms.sh $@\ + $(CONFIG_TEST_KALLSYMS_NUMSYMS) \ + $(CONFIG_TEST_KALLSYMS_SCALE_FACTOR) + +clean-files += test_kallsyms_a.c +clean-files += test_kallsyms_b.c +clean-files += test_kallsyms_c.c +clean-files += test_kallsyms_d.c diff --git a/lib/tests/module/gen_test_kallsyms.sh b/lib/tests/module/gen_test_kallsyms.sh new file mode 100755 index 000000000000..e85f10dc11bd --- /dev/null +++ b/lib/tests/module/gen_test_kallsyms.sh @@ -0,0 +1,128 @@ +#!/bin/bash + +TARGET=$(basename $1) +DIR=lib/tests/module +TARGET="$DIR/$TARGET" +NUM_SYMS=$2 +SCALE_FACTOR=$3 +TEST_TYPE=$(echo $TARGET | sed -e 's|lib/tests/module/test_kallsyms_||g') +TEST_TYPE=$(echo $TEST_TYPE | sed -e 's|.c||g') + +gen_template_module_header() +{ + cat <<____END_MODULE +// SPDX-License-Identifier: GPL-2.0-or-later OR copyleft-next-0.3.1 +/* + * Copyright (C) 2023 Luis Chamberlain <mcgrof@kernel.org> + * + * Automatically generated code for testing, do not edit manually. + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include <linux/init.h> +#include <linux/module.h> +#include <linux/printk.h> + +____END_MODULE +} + +gen_num_syms() +{ + PREFIX=$1 + NUM=$2 + for i in $(seq 1 $NUM); do + printf "int auto_test_%s_%010d = 0xff;\n" $PREFIX $i + printf "EXPORT_SYMBOL_GPL(auto_test_%s_%010d);\n" $PREFIX $i + done + echo +} + +gen_template_module_data_a() +{ + gen_num_syms a $1 + cat <<____END_MODULE +static int auto_runtime_test(void) +{ + return 0; +} + +____END_MODULE +} + +gen_template_module_data_b() +{ + printf "\nextern int auto_test_a_%010d;\n\n" 28 + echo "static int auto_runtime_test(void)" + echo "{" + printf "\nreturn auto_test_a_%010d;\n" 28 + echo "}" +} + +gen_template_module_data_c() +{ + gen_num_syms c $1 + cat <<____END_MODULE +static int auto_runtime_test(void) +{ + return 0; +} + +____END_MODULE +} + +gen_template_module_data_d() +{ + gen_num_syms d $1 + cat <<____END_MODULE +static int auto_runtime_test(void) +{ + return 0; +} + +____END_MODULE +} + +gen_template_module_exit() +{ + cat <<____END_MODULE +static int __init auto_test_module_init(void) +{ + return auto_runtime_test(); +} +module_init(auto_test_module_init); + +static void __exit auto_test_module_exit(void) +{ +} +module_exit(auto_test_module_exit); + +MODULE_AUTHOR("Luis Chamberlain <mcgrof@kernel.org>"); +MODULE_LICENSE("GPL"); +____END_MODULE +} + +case $TEST_TYPE in + a) + gen_template_module_header > $TARGET + gen_template_module_data_a $NUM_SYMS >> $TARGET + gen_template_module_exit >> $TARGET + ;; + b) + gen_template_module_header > $TARGET + gen_template_module_data_b >> $TARGET + gen_template_module_exit >> $TARGET + ;; + c) + gen_template_module_header > $TARGET + gen_template_module_data_c $((NUM_SYMS * SCALE_FACTOR)) >> $TARGET + gen_template_module_exit >> $TARGET + ;; + d) + gen_template_module_header > $TARGET + gen_template_module_data_d $((NUM_SYMS * SCALE_FACTOR * 2)) >> $TARGET + gen_template_module_exit >> $TARGET + ;; + *) + ;; +esac diff --git a/tools/testing/selftests/module/Makefile b/tools/testing/selftests/module/Makefile new file mode 100644 index 000000000000..6132d7ddb08b --- /dev/null +++ b/tools/testing/selftests/module/Makefile @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Makefile for module loading selftests + +# No binaries, but make sure arg-less "make" doesn't trigger "run_tests" +all: + +TEST_PROGS := find_symbol.sh + +include ../lib.mk + +# Nothing to clean up. +clean: diff --git a/tools/testing/selftests/module/config b/tools/testing/selftests/module/config new file mode 100644 index 000000000000..b0c206b1ad47 --- /dev/null +++ b/tools/testing/selftests/module/config @@ -0,0 +1,3 @@ +CONFIG_TEST_RUNTIME=y +CONFIG_TEST_RUNTIME_MODULE=y +CONFIG_TEST_KALLSYMS=m diff --git a/tools/testing/selftests/module/find_symbol.sh b/tools/testing/selftests/module/find_symbol.sh new file mode 100755 index 000000000000..140364d3c49f --- /dev/null +++ b/tools/testing/selftests/module/find_symbol.sh @@ -0,0 +1,81 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later OR copyleft-next-0.3.1 +# Copyright (C) 2023 Luis Chamberlain <mcgrof@kernel.org> +# +# This is a stress test script for kallsyms through find_symbol() + +set -e + +# Kselftest framework requirement - SKIP code is 4. +ksft_skip=4 + +test_reqs() +{ + if ! which modprobe 2> /dev/null > /dev/null; then + echo "$0: You need modprobe installed" >&2 + exit $ksft_skip + fi + + if ! which kmod 2> /dev/null > /dev/null; then + echo "$0: You need kmod installed" >&2 + exit $ksft_skip + fi + + if ! which perf 2> /dev/null > /dev/null; then + echo "$0: You need perf installed" >&2 + exit $ksft_skip + fi + + uid=$(id -u) + if [ $uid -ne 0 ]; then + echo $msg must be run as root >&2 + exit $ksft_skip + fi +} + +load_mod() +{ + local STATS="-e duration_time" + STATS="$STATS -e user_time" + STATS="$STATS -e system_time" + STATS="$STATS -e page-faults" + local MOD=$1 + + local ARCH="$(uname -m)" + case "${ARCH}" in + x86_64) + perf stat $STATS $MODPROBE test_kallsyms_b + ;; + *) + time $MODPROBE test_kallsyms_b + exit 1 + ;; + esac +} + +remove_all() +{ + $MODPROBE -r test_kallsyms_b + for i in a b c d; do + $MODPROBE -r test_kallsyms_$i + done +} +test_reqs + +MODPROBE=$(</proc/sys/kernel/modprobe) + +remove_all +load_mod test_kallsyms_b +remove_all + +# Now pollute the namespace +$MODPROBE test_kallsyms_c +load_mod test_kallsyms_b + +# Now pollute the namespace with twice the number of symbols than the last time +remove_all +$MODPROBE test_kallsyms_c +$MODPROBE test_kallsyms_d +load_mod test_kallsyms_b + +exit 0
We lack find_symbol() selftests, so add one. This let's us stress test improvements easily on find_symbol() or optimizations. It also inherently allows us to test the limits of kallsyms on Linux today. We test a pathalogical use case for kallsyms by introducing modules which are automatically written for us with a larger number of symbols. We have 4 kallsyms test modules: A: has KALLSYSMS_NUMSYMS exported symbols B: uses one of A's symbols C: adds KALLSYMS_SCALE_FACTOR * KALLSYSMS_NUMSYMS exported D: adds 2 * the symbols than C By using anything much larger than KALLSYSMS_NUMSYMS as 10,000 and KALLSYMS_SCALE_FACTOR of 8 we segfault today. So we're capped at around 160000 symbols somehow today. We can inpsect that issue at our leasure later, but for now the real value to this test is that this will easily allow us to test improvements on find_symbol(). We want to enable this test on allyesmodconfig builds so we can't use this combination, so instead just use a safe value for now and be informative on the Kconfig symbol documentation about where our thresholds are for testers. We default then to KALLSYSMS_NUMSYMS of just 100 and KALLSYMS_SCALE_FACTOR of 8. On x86_64 we can use perf, for other architectures we just use 'time' and allow for customizations. For example a future enhancements could be done for parisc to check for unaligned accesses which triggers a special special exception handler assembler code inside the kernel. The negative impact on performance is so large on parisc that it keeps track of its accesses on /proc/cpuinfo as UAH: IRQ: CPU0 CPU1 3: 1332 0 SuperIO ttyS0 7: 1270013 0 SuperIO pata_ns87415 64: 320023012 320021431 CPU timer 65: 17080507 20624423 CPU IPI UAH: 10948640 58104 Unaligned access handler traps While at it, this tidies up lib/ test modules to allow us to have a new directory for them. The amount of test modules under lib/ is insane. This should also hopefully showcase how to start doing basic self module writing code, which may be more useful for more complex cases later in the future. Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> --- I posted the v2 [0] a long time ago to account for some Helge's patches and provide a vehicle for testing but Masahiro noted this should be addressed on the asm code, not the linker script. This test still is useful so I am posting it separately. [0] https://lkml.kernel.org/r/20240129192644.3359978-1-mcgrof@kernel.org Changes on v3: - Drop Helge's patches - Adjust to allow safe build tests for now with sane settings, otherwise we'd fail allmodconfig build tests. lib/Kconfig.debug | 105 ++++++++++++++ lib/Makefile | 1 + lib/tests/Makefile | 1 + lib/tests/module/.gitignore | 4 + lib/tests/module/Makefile | 15 ++ lib/tests/module/gen_test_kallsyms.sh | 128 ++++++++++++++++++ tools/testing/selftests/module/Makefile | 12 ++ tools/testing/selftests/module/config | 3 + tools/testing/selftests/module/find_symbol.sh | 81 +++++++++++ 9 files changed, 350 insertions(+) create mode 100644 lib/tests/Makefile create mode 100644 lib/tests/module/.gitignore create mode 100644 lib/tests/module/Makefile create mode 100755 lib/tests/module/gen_test_kallsyms.sh create mode 100644 tools/testing/selftests/module/Makefile create mode 100644 tools/testing/selftests/module/config create mode 100755 tools/testing/selftests/module/find_symbol.sh