diff mbox series

[v4,4/4] kunit: Report test parameter results as (K)TAP subtests

Message ID 20211102073014.2901870-4-davidgow@google.com (mailing list archive)
State Accepted
Commit 44b7da5fcd4c99de1ec5cc783cdd605398246280
Delegated to: Brendan Higgins
Headers show
Series [v4,1/4] kunit: tool: Do not error on tests without test plans | expand

Commit Message

David Gow Nov. 2, 2021, 7:30 a.m. UTC 
Currently, the results for individial parameters in a parameterised test
are simply output as (K)TAP diagnostic lines.

As kunit_tool now supports nested subtests, report each parameter as its
own subtest.

For example, here's what the output now looks like:
	# Subtest: inode_test_xtimestamp_decoding
	ok 1 - 1901-12-13 Lower bound of 32bit < 0 timestamp, no extra bits
	ok 2 - 1969-12-31 Upper bound of 32bit < 0 timestamp, no extra bits
	ok 3 - 1970-01-01 Lower bound of 32bit >=0 timestamp, no extra bits
	ok 4 - 2038-01-19 Upper bound of 32bit >=0 timestamp, no extra bits
	ok 5 - 2038-01-19 Lower bound of 32bit <0 timestamp, lo extra sec bit on
	ok 6 - 2106-02-07 Upper bound of 32bit <0 timestamp, lo extra sec bit on
	ok 7 - 2106-02-07 Lower bound of 32bit >=0 timestamp, lo extra sec bit on
	ok 8 - 2174-02-25 Upper bound of 32bit >=0 timestamp, lo extra sec bit on
	ok 9 - 2174-02-25 Lower bound of 32bit <0 timestamp, hi extra sec bit on
	ok 10 - 2242-03-16 Upper bound of 32bit <0 timestamp, hi extra sec bit on
	ok 11 - 2242-03-16 Lower bound of 32bit >=0 timestamp, hi extra sec bit on
	ok 12 - 2310-04-04 Upper bound of 32bit >=0 timestamp, hi extra sec bit on
	ok 13 - 2310-04-04 Upper bound of 32bit>=0 timestamp, hi extra sec bit 1. 1 ns
	ok 14 - 2378-04-22 Lower bound of 32bit>= timestamp. Extra sec bits 1. Max ns
	ok 15 - 2378-04-22 Lower bound of 32bit >=0 timestamp. All extra sec bits on
	ok 16 - 2446-05-10 Upper bound of 32bit >=0 timestamp. All extra sec bits on
	# inode_test_xtimestamp_decoding: pass:16 fail:0 skip:0 total:16
	ok 1 - inode_test_xtimestamp_decoding

Signed-off-by: David Gow <davidgow@google.com>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
---

Changes since v3:
https://lore.kernel.org/linux-kselftest/20211028064154.2301049-4-davidgow@google.com/
- Fix the missing log line which ended up in patch 3 by mistake.

Changes since v2:
https://lore.kernel.org/linux-kselftest/20211027013702.2039566-4-davidgow@google.com/
- No changes to this patch.


 lib/kunit/test.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

Comments

Brendan Higgins Dec. 7, 2021, 8:30 p.m. UTC | #1 
On Tue, Nov 2, 2021 at 3:30 AM David Gow <davidgow@google.com> wrote:
>
> Currently, the results for individial parameters in a parameterised test
> are simply output as (K)TAP diagnostic lines.
>
> As kunit_tool now supports nested subtests, report each parameter as its
> own subtest.
>
> For example, here's what the output now looks like:
>         # Subtest: inode_test_xtimestamp_decoding
>         ok 1 - 1901-12-13 Lower bound of 32bit < 0 timestamp, no extra bits
>         ok 2 - 1969-12-31 Upper bound of 32bit < 0 timestamp, no extra bits
>         ok 3 - 1970-01-01 Lower bound of 32bit >=0 timestamp, no extra bits
>         ok 4 - 2038-01-19 Upper bound of 32bit >=0 timestamp, no extra bits
>         ok 5 - 2038-01-19 Lower bound of 32bit <0 timestamp, lo extra sec bit on
>         ok 6 - 2106-02-07 Upper bound of 32bit <0 timestamp, lo extra sec bit on
>         ok 7 - 2106-02-07 Lower bound of 32bit >=0 timestamp, lo extra sec bit on
>         ok 8 - 2174-02-25 Upper bound of 32bit >=0 timestamp, lo extra sec bit on
>         ok 9 - 2174-02-25 Lower bound of 32bit <0 timestamp, hi extra sec bit on
>         ok 10 - 2242-03-16 Upper bound of 32bit <0 timestamp, hi extra sec bit on
>         ok 11 - 2242-03-16 Lower bound of 32bit >=0 timestamp, hi extra sec bit on
>         ok 12 - 2310-04-04 Upper bound of 32bit >=0 timestamp, hi extra sec bit on
>         ok 13 - 2310-04-04 Upper bound of 32bit>=0 timestamp, hi extra sec bit 1. 1 ns
>         ok 14 - 2378-04-22 Lower bound of 32bit>= timestamp. Extra sec bits 1. Max ns
>         ok 15 - 2378-04-22 Lower bound of 32bit >=0 timestamp. All extra sec bits on
>         ok 16 - 2446-05-10 Upper bound of 32bit >=0 timestamp. All extra sec bits on
>         # inode_test_xtimestamp_decoding: pass:16 fail:0 skip:0 total:16
>         ok 1 - inode_test_xtimestamp_decoding
>
> Signed-off-by: David Gow <davidgow@google.com>
> Reviewed-by: Daniel Latypov <dlatypov@google.com>

Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
diff mbox series

Patch

diff --git a/lib/kunit/test.c b/lib/kunit/test.c
index f96498ede2cc..c7ed4aabec04 100644
--- a/lib/kunit/test.c
+++ b/lib/kunit/test.c
@@ -512,6 +512,8 @@  int kunit_run_tests(struct kunit_suite *suite)
 			/* Get initial param. */
 			param_desc[0] = '\0';
 			test.param_value = test_case->generate_params(NULL, param_desc);
+			kunit_log(KERN_INFO, &test, KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
+				  "# Subtest: %s", test_case->name);
 
 			while (test.param_value) {
 				kunit_run_case_catch_errors(suite, test_case, &test);
@@ -522,9 +524,8 @@  int kunit_run_tests(struct kunit_suite *suite)
 				}
 
 				kunit_log(KERN_INFO, &test,
-					  KUNIT_SUBTEST_INDENT
-					  "# %s: %s %d - %s",
-					  test_case->name,
+					  KUNIT_SUBTEST_INDENT KUNIT_SUBTEST_INDENT
+					  "%s %d - %s",
 					  kunit_status_to_ok_not_ok(test.status),
 					  test.param_index + 1, param_desc);