| Message ID | 20230911-kms-slow-tests-v1-2-d3800a69a1a1@kernel.org (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | drm/tests: Flag slow kunit tests as such | expand |
On Mon, Sep 11, 2023 at 11:51:06AM +0200, Maxime Ripard wrote: > Kunit recently gained a speed attribute that allows to filter out slow > tests. A slow test is defined in the documentation as a test taking more > than a second to execute. > > Let's flag the few tests that are doing so on my machine when running: > > ./tools/testing/kunit/kunit.py run --kunitconfig=drivers/gpu/drm/tests \ > --cross_compile aarch64-linux-gnu- --arch arm64 > > Suggested-by: David Gow <davidgow@google.com> > Signed-off-by: Maxime Ripard <mripard@kernel.org> Ugh ... not a fan. igt has a really bad habit of making disastrously combinatorial tests with impossible runtimes, and then just filtering these out so it's still fast. Maybe some stress tests for overall system make sense like this, but absolutely not for unit tests. And I did spot check some of these, they're just combinatorial explosions with large repetition counts and some fun stuff like going through prime numbers because surely that's a good idea. Imo delete them all, and if that causes a real gap in coverage, ask the authors to write some actual good unit tests for these corner cases. Cheers, Sima > --- > drivers/gpu/drm/tests/drm_buddy_test.c | 2 +- > drivers/gpu/drm/tests/drm_mm_test.c | 14 +++++++------- > 2 files changed, 8 insertions(+), 8 deletions(-) > > diff --git a/drivers/gpu/drm/tests/drm_buddy_test.c b/drivers/gpu/drm/tests/drm_buddy_test.c > index 09ee6f6af896..6f79cde2df55 100644 > --- a/drivers/gpu/drm/tests/drm_buddy_test.c > +++ b/drivers/gpu/drm/tests/drm_buddy_test.c > @@ -742,7 +742,7 @@ static struct kunit_case drm_buddy_tests[] = { > KUNIT_CASE(drm_test_buddy_alloc_range), > KUNIT_CASE(drm_test_buddy_alloc_optimistic), > KUNIT_CASE(drm_test_buddy_alloc_pessimistic), > - KUNIT_CASE(drm_test_buddy_alloc_smoke), > + KUNIT_CASE_SLOW(drm_test_buddy_alloc_smoke), > KUNIT_CASE(drm_test_buddy_alloc_pathological), > {} > }; > diff --git a/drivers/gpu/drm/tests/drm_mm_test.c b/drivers/gpu/drm/tests/drm_mm_test.c > index 186b28dc7038..c1e662c2a76c 100644 > --- a/drivers/gpu/drm/tests/drm_mm_test.c > +++ b/drivers/gpu/drm/tests/drm_mm_test.c > @@ -2228,23 +2228,23 @@ module_param(max_prime, uint, 0400); > static struct kunit_case drm_mm_tests[] = { > KUNIT_CASE(drm_test_mm_init), > KUNIT_CASE(drm_test_mm_debug), > - KUNIT_CASE(drm_test_mm_reserve), > - KUNIT_CASE(drm_test_mm_insert), > - KUNIT_CASE(drm_test_mm_replace), > - KUNIT_CASE(drm_test_mm_insert_range), > + KUNIT_CASE_SLOW(drm_test_mm_reserve), > + KUNIT_CASE_SLOW(drm_test_mm_insert), > + KUNIT_CASE_SLOW(drm_test_mm_replace), > + KUNIT_CASE_SLOW(drm_test_mm_insert_range), > KUNIT_CASE(drm_test_mm_frag), > KUNIT_CASE(drm_test_mm_align), > KUNIT_CASE(drm_test_mm_align32), > KUNIT_CASE(drm_test_mm_align64), > - KUNIT_CASE(drm_test_mm_evict), > + KUNIT_CASE_SLOW(drm_test_mm_evict), > KUNIT_CASE(drm_test_mm_evict_range), > KUNIT_CASE(drm_test_mm_topdown), > KUNIT_CASE(drm_test_mm_bottomup), > KUNIT_CASE(drm_test_mm_lowest), > KUNIT_CASE(drm_test_mm_highest), > KUNIT_CASE(drm_test_mm_color), > - KUNIT_CASE(drm_test_mm_color_evict), > - KUNIT_CASE(drm_test_mm_color_evict_range), > + KUNIT_CASE_SLOW(drm_test_mm_color_evict), > + KUNIT_CASE_SLOW(drm_test_mm_color_evict_range), > {} > }; > > > -- > 2.41.0 >
Hi Sima, (For some reason, it looks like your mailer sets up the headers to reply to every recipient but you) On Tue, Sep 12, 2023 at 09:36:12AM +0200, Daniel Vetter wrote: > On Mon, Sep 11, 2023 at 11:51:06AM +0200, Maxime Ripard wrote: > > Kunit recently gained a speed attribute that allows to filter out slow > > tests. A slow test is defined in the documentation as a test taking more > > than a second to execute. > > > > Let's flag the few tests that are doing so on my machine when running: > > > > ./tools/testing/kunit/kunit.py run --kunitconfig=drivers/gpu/drm/tests \ > > --cross_compile aarch64-linux-gnu- --arch arm64 > > > > Suggested-by: David Gow <davidgow@google.com> > > Signed-off-by: Maxime Ripard <mripard@kernel.org> > > Ugh ... not a fan. > > igt has a really bad habit of making disastrously combinatorial tests with > impossible runtimes, and then just filtering these out so it's still fast. > > Maybe some stress tests for overall system make sense like this, but > absolutely not for unit tests. I agree, I didn't want to reduce testing though. > And I did spot check some of these, they're just combinatorial > explosions with large repetition counts and some fun stuff like going > through prime numbers because surely that's a good idea. > > Imo delete them all, and if that causes a real gap in coverage, ask > the authors to write some actual good unit tests for these corner > cases. Ack, I will send a patch doing so. Thanks! Maxime
diff --git a/drivers/gpu/drm/tests/drm_buddy_test.c b/drivers/gpu/drm/tests/drm_buddy_test.c index 09ee6f6af896..6f79cde2df55 100644 --- a/drivers/gpu/drm/tests/drm_buddy_test.c +++ b/drivers/gpu/drm/tests/drm_buddy_test.c @@ -742,7 +742,7 @@ static struct kunit_case drm_buddy_tests[] = { KUNIT_CASE(drm_test_buddy_alloc_range), KUNIT_CASE(drm_test_buddy_alloc_optimistic), KUNIT_CASE(drm_test_buddy_alloc_pessimistic), - KUNIT_CASE(drm_test_buddy_alloc_smoke), + KUNIT_CASE_SLOW(drm_test_buddy_alloc_smoke), KUNIT_CASE(drm_test_buddy_alloc_pathological), {} }; diff --git a/drivers/gpu/drm/tests/drm_mm_test.c b/drivers/gpu/drm/tests/drm_mm_test.c index 186b28dc7038..c1e662c2a76c 100644 --- a/drivers/gpu/drm/tests/drm_mm_test.c +++ b/drivers/gpu/drm/tests/drm_mm_test.c @@ -2228,23 +2228,23 @@ module_param(max_prime, uint, 0400); static struct kunit_case drm_mm_tests[] = { KUNIT_CASE(drm_test_mm_init), KUNIT_CASE(drm_test_mm_debug), - KUNIT_CASE(drm_test_mm_reserve), - KUNIT_CASE(drm_test_mm_insert), - KUNIT_CASE(drm_test_mm_replace), - KUNIT_CASE(drm_test_mm_insert_range), + KUNIT_CASE_SLOW(drm_test_mm_reserve), + KUNIT_CASE_SLOW(drm_test_mm_insert), + KUNIT_CASE_SLOW(drm_test_mm_replace), + KUNIT_CASE_SLOW(drm_test_mm_insert_range), KUNIT_CASE(drm_test_mm_frag), KUNIT_CASE(drm_test_mm_align), KUNIT_CASE(drm_test_mm_align32), KUNIT_CASE(drm_test_mm_align64), - KUNIT_CASE(drm_test_mm_evict), + KUNIT_CASE_SLOW(drm_test_mm_evict), KUNIT_CASE(drm_test_mm_evict_range), KUNIT_CASE(drm_test_mm_topdown), KUNIT_CASE(drm_test_mm_bottomup), KUNIT_CASE(drm_test_mm_lowest), KUNIT_CASE(drm_test_mm_highest), KUNIT_CASE(drm_test_mm_color), - KUNIT_CASE(drm_test_mm_color_evict), - KUNIT_CASE(drm_test_mm_color_evict_range), + KUNIT_CASE_SLOW(drm_test_mm_color_evict), + KUNIT_CASE_SLOW(drm_test_mm_color_evict_range), {} };
Kunit recently gained a speed attribute that allows to filter out slow tests. A slow test is defined in the documentation as a test taking more than a second to execute. Let's flag the few tests that are doing so on my machine when running: ./tools/testing/kunit/kunit.py run --kunitconfig=drivers/gpu/drm/tests \ --cross_compile aarch64-linux-gnu- --arch arm64 Suggested-by: David Gow <davidgow@google.com> Signed-off-by: Maxime Ripard <mripard@kernel.org> --- drivers/gpu/drm/tests/drm_buddy_test.c | 2 +- drivers/gpu/drm/tests/drm_mm_test.c | 14 +++++++------- 2 files changed, 8 insertions(+), 8 deletions(-)