| Message ID | 20220223103446.2681293-2-nrb@linux.ibm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Detecting crashes in TAP output | expand |
diff --git a/scripts/runtime.bash b/scripts/runtime.bash index 6d5fced94246..7bb70d50012a 100644 --- a/scripts/runtime.bash +++ b/scripts/runtime.bash @@ -60,8 +60,10 @@ function print_result() if [ -z "$reason" ]; then echo "`$status` $testname $summary" + RUNTIME_log_stderr "$testname" <<< "$status: $summary" else echo "`$status` $testname ($reason)" + RUNTIME_log_stderr "$testname" <<< "$status: $reason" fi }
Currently, when a test exits prematurely for whatever reason (crash, guest exception, ...) the test is correctly reported as FAIL by run_tests.sh - even when all report()s up to that point passed. The failure is reported by print_result which checks (among others) the exit code of the arch-specific run script. But, as soon as one enables the TAP output in run_tests.sh, there is no way to see that any more, as the print_result is discarded and only the run script's output is converted to TAP. External test runners relying on TAP output will thus believe everything is fine even though we got a crash. The same also applies to the logfiles. As a simple fix, have print_result also print to RUNTIME_log_stderr. For each test, we will then get an additional test line in the TAP which reports the test's result: not ok 36 - css: (35 tests, 2 unexpected failures) The log files will also contain the result: FAIL: (35 tests, 2 unexpected failures) This makes it easy to see whether we had a premature exit in the test. The disadvantage being the number of test lines in the TAP will no longer match the number of report()s in a test. Signed-off-by: Nico Boehr <nrb@linux.ibm.com> --- scripts/runtime.bash | 2 ++ 1 file changed, 2 insertions(+)