| Message ID | 20201025214842.5924-5-linux@rasmusvillemoes.dk (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | deterministic random testing | expand |
On Sun 2020-10-25 22:48:42, Rasmus Villemoes wrote: > The printf test suite does each test with a few different buffer sizes > to ensure vsnprintf() behaves correctly with respect to truncation and > size reporting. It calls vsnprintf() with a buffer size that is > guaranteed to be big enough, a buffer size of 0 to ensure that nothing > gets written to the buffer, but it also calls vsnprintf() with a > buffer size chosen to guarantee the output gets truncated somewhere in > the middle. > > That buffer size is chosen randomly to increase the chance of finding > some corner case bug (for example, there used to be some %p<foo> > extension that would fail to produce any output if there wasn't room > enough for it all, despite the requirement of producing as much as > there's room for). I'm not aware of that having found anything yet, > but should it happen, it's annoying not to be able to repeat the > test with the same sequence of truncated lengths. > > For demonstration purposes, if we break one of the test cases > deliberately, we still get different buffer sizes if we don't pass the > seed parameter: > > root@(none):/# modprobe test_printf > [ 15.317783] test_printf: vsnprintf(buf, 18, "%piS|%pIS", ...) wrote '127.000.000.001|1', expected '127-000.000.001|1' > [ 15.323182] test_printf: failed 3 out of 388 tests > [ 15.324034] test_printf: random seed used was 0x278bb9311979cc91 > modprobe: ERROR: could not insert 'test_printf': Invalid argument > > root@(none):/# modprobe test_printf > [ 13.940909] test_printf: vsnprintf(buf, 22, "%piS|%pIS", ...) wrote '127.000.000.001|127.0', expected '127-000.000.001|127.0' > [ 13.944744] test_printf: failed 3 out of 388 tests > [ 13.945607] test_printf: random seed used was 0x9f72eee1c9dc02e5 > modprobe: ERROR: could not insert 'test_printf': Invalid argument > > but to repeat a specific sequence of tests, we can do > > root@(none):/# modprobe test_printf seed=0x9f72eee1c9dc02e5 > [ 448.328685] test_printf: vsnprintf(buf, 22, "%piS|%pIS", ...) wrote '127.000.000.001|127.0', expected '127-000.000.001|127.0' > [ 448.331650] test_printf: failed 3 out of 388 tests > [ 448.332295] test_printf: random seed used was 0x9f72eee1c9dc02e5 > modprobe: ERROR: could not insert 'test_printf': Invalid argument > > Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk> Great feature! Reviewed-by: Petr Mladek <pmladek@suse.com> Best Regards, Petr
diff --git a/lib/test_printf.c b/lib/test_printf.c index 1ed4a27390cb621715ab..bbea8b807d1eafe67e01 100644 --- a/lib/test_printf.c +++ b/lib/test_printf.c @@ -24,6 +24,7 @@ #include <linux/property.h> +#define KSTM_RANDOM 1 #include "../tools/testing/selftests/kselftest_module.h" #define BUF_SIZE 256 @@ -111,8 +112,14 @@ __test(const char *expect, int elen, const char *fmt, ...) * be able to print it as expected. */ failed_tests += do_test(BUF_SIZE, expect, elen, fmt, ap); - rand = 1 + prandom_u32_max(elen+1); - /* Since elen < BUF_SIZE, we have 1 <= rand <= BUF_SIZE. */ + rand = prandom_u32_range_state(&rnd_state, 1, elen + 1); + /* + * Except for elen == 0, we have 1 <= rand <= elen < BUF_SIZE, + * i.e., the output is guaranteed to be truncated somewhere in + * the middle, and we're not pretending the buffer to be + * larger than it really is. For the boring case of elen == 0, + * rand is 1, which is of course also <= BUF_SIZE. + */ failed_tests += do_test(rand, expect, elen, fmt, ap); failed_tests += do_test(0, expect, elen, fmt, ap);
The printf test suite does each test with a few different buffer sizes to ensure vsnprintf() behaves correctly with respect to truncation and size reporting. It calls vsnprintf() with a buffer size that is guaranteed to be big enough, a buffer size of 0 to ensure that nothing gets written to the buffer, but it also calls vsnprintf() with a buffer size chosen to guarantee the output gets truncated somewhere in the middle. That buffer size is chosen randomly to increase the chance of finding some corner case bug (for example, there used to be some %p<foo> extension that would fail to produce any output if there wasn't room enough for it all, despite the requirement of producing as much as there's room for). I'm not aware of that having found anything yet, but should it happen, it's annoying not to be able to repeat the test with the same sequence of truncated lengths. For demonstration purposes, if we break one of the test cases deliberately, we still get different buffer sizes if we don't pass the seed parameter: root@(none):/# modprobe test_printf [ 15.317783] test_printf: vsnprintf(buf, 18, "%piS|%pIS", ...) wrote '127.000.000.001|1', expected '127-000.000.001|1' [ 15.323182] test_printf: failed 3 out of 388 tests [ 15.324034] test_printf: random seed used was 0x278bb9311979cc91 modprobe: ERROR: could not insert 'test_printf': Invalid argument root@(none):/# modprobe test_printf [ 13.940909] test_printf: vsnprintf(buf, 22, "%piS|%pIS", ...) wrote '127.000.000.001|127.0', expected '127-000.000.001|127.0' [ 13.944744] test_printf: failed 3 out of 388 tests [ 13.945607] test_printf: random seed used was 0x9f72eee1c9dc02e5 modprobe: ERROR: could not insert 'test_printf': Invalid argument but to repeat a specific sequence of tests, we can do root@(none):/# modprobe test_printf seed=0x9f72eee1c9dc02e5 [ 448.328685] test_printf: vsnprintf(buf, 22, "%piS|%pIS", ...) wrote '127.000.000.001|127.0', expected '127-000.000.001|127.0' [ 448.331650] test_printf: failed 3 out of 388 tests [ 448.332295] test_printf: random seed used was 0x9f72eee1c9dc02e5 modprobe: ERROR: could not insert 'test_printf': Invalid argument Signed-off-by: Rasmus Villemoes <linux@rasmusvillemoes.dk> --- lib/test_printf.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)