| Message ID | 20200821123744.33173-3-mhartmay@linux.ibm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Use same test names in the default and the TAP13 output format | expand |
On Fri, Aug 21, 2020 at 02:37:44PM +0200, Marc Hartmayer wrote: > Use the same test names in the TAP13 output as in the default output > format. This makes the output more consistent. To achieve this, we > need to pass the test name as an argument to the function > `process_test_output`. > > Before this change: > $ ./run_tests.sh > PASS selftest-setup (14 tests) > ... > > vs. > > $ ./run_tests.sh -t > TAP version 13 > ok 1 - selftest: true > ok 2 - selftest: argc == 3 > ... > > After this change: > $ ./run_tests.sh > PASS selftest-setup (14 tests) > ... > > vs. > > $ ./run_tests.sh -t > TAP version 13 > ok 1 - selftest-setup: true > ok 2 - selftest-setup: argc == 3 > ... > > While at it, introduce a local variable `kernel` in > `RUNTIME_log_stdout` since this makes the function easier to read. > > Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com> > --- > run_tests.sh | 15 +++++++++------ > scripts/runtime.bash | 6 +++--- > 2 files changed, 12 insertions(+), 9 deletions(-) > > diff --git a/run_tests.sh b/run_tests.sh > index 01e36dcfa06e..b5812336866f 100755 > --- a/run_tests.sh > +++ b/run_tests.sh > @@ -81,18 +81,19 @@ if [[ $tap_output == "no" ]]; then > postprocess_suite_output() { cat; } > else > process_test_output() { > + local testname="$1" > CR=$'\r' > while read -r line; do > line="${line%$CR}" > case "${line:0:4}" in > PASS) > - echo "ok TEST_NUMBER - ${line#??????}" >&3 > + echo "ok TEST_NUMBER - ${testname}:${line#*:*:}" >&3 > ;; > FAIL) > - echo "not ok TEST_NUMBER - ${line#??????}" >&3 > + echo "not ok TEST_NUMBER - ${testname}:${line#*:*:}" >&3 > ;; > SKIP) > - echo "ok TEST_NUMBER - ${line#??????} # skip" >&3 > + echo "ok TEST_NUMBER - ${testname}:${line#*:*:} # skip" >&3 > ;; > *) > ;; > @@ -114,12 +115,14 @@ else > } > fi > > -RUNTIME_log_stderr () { process_test_output; } > +RUNTIME_log_stderr () { process_test_output "$1"; } > RUNTIME_log_stdout () { > + local testname="$1" > if [ "$PRETTY_PRINT_STACKS" = "yes" ]; then > - ./scripts/pretty_print_stacks.py $1 | process_test_output > + local kernel="$2" > + ./scripts/pretty_print_stacks.py "$kernel" | process_test_output "$testname" > else > - process_test_output > + process_test_output "$testname" > fi > } > > diff --git a/scripts/runtime.bash b/scripts/runtime.bash > index caa4c5ba18cc..294e6b15a5e2 100644 > --- a/scripts/runtime.bash > +++ b/scripts/runtime.bash > @@ -140,10 +140,10 @@ function run() > # extra_params in the config file may contain backticks that need to be > # expanded, so use eval to start qemu. Use "> >(foo)" instead of a pipe to > # preserve the exit status. > - summary=$(eval $cmdline 2> >(RUNTIME_log_stderr) \ > - > >(tee >(RUNTIME_log_stdout $kernel) | extract_summary)) > + summary=$(eval $cmdline 2> >(RUNTIME_log_stderr $testname) \ > + > >(tee >(RUNTIME_log_stdout $testname $kernel) | extract_summary)) > ret=$? > - [ "$STANDALONE" != "yes" ] && echo > >(RUNTIME_log_stdout $kernel) > + [ "$STANDALONE" != "yes" ] && echo > >(RUNTIME_log_stdout $testname $kernel) > > if [ $ret -eq 0 ]; then > print_result "PASS" $testname "$summary" > -- > 2.25.4 > Reviewed-by: Andrew Jones <drjones@redhat.com>
On 8/21/20 2:37 PM, Marc Hartmayer wrote: > Use the same test names in the TAP13 output as in the default output > format. This makes the output more consistent. To achieve this, we > need to pass the test name as an argument to the function > `process_test_output`. > > Before this change: > $ ./run_tests.sh > PASS selftest-setup (14 tests) > ... > > vs. > > $ ./run_tests.sh -t > TAP version 13 > ok 1 - selftest: true > ok 2 - selftest: argc == 3 > ... > > After this change: > $ ./run_tests.sh > PASS selftest-setup (14 tests) > ... > > vs. > > $ ./run_tests.sh -t > TAP version 13 > ok 1 - selftest-setup: true > ok 2 - selftest-setup: argc == 3 This doesn't work for me, we can't just drop prefixes. I.e. it needs to be "testname: prefix1: prefix2: prefixn: Test text" selftest-setup: selftest: true > ... > > While at it, introduce a local variable `kernel` in > `RUNTIME_log_stdout` since this makes the function easier to read. > > Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com> > --- > run_tests.sh | 15 +++++++++------ > scripts/runtime.bash | 6 +++--- > 2 files changed, 12 insertions(+), 9 deletions(-) > > diff --git a/run_tests.sh b/run_tests.sh > index 01e36dcfa06e..b5812336866f 100755 > --- a/run_tests.sh > +++ b/run_tests.sh > @@ -81,18 +81,19 @@ if [[ $tap_output == "no" ]]; then > postprocess_suite_output() { cat; } > else > process_test_output() { > + local testname="$1" > CR=$'\r' > while read -r line; do > line="${line%$CR}" > case "${line:0:4}" in > PASS) > - echo "ok TEST_NUMBER - ${line#??????}" >&3 > + echo "ok TEST_NUMBER - ${testname}:${line#*:*:}" >&3 > ;; > FAIL) > - echo "not ok TEST_NUMBER - ${line#??????}" >&3 > + echo "not ok TEST_NUMBER - ${testname}:${line#*:*:}" >&3 > ;; > SKIP) > - echo "ok TEST_NUMBER - ${line#??????} # skip" >&3 > + echo "ok TEST_NUMBER - ${testname}:${line#*:*:} # skip" >&3 > ;; > *) > ;; > @@ -114,12 +115,14 @@ else > } > fi > > -RUNTIME_log_stderr () { process_test_output; } > +RUNTIME_log_stderr () { process_test_output "$1"; } > RUNTIME_log_stdout () { > + local testname="$1" > if [ "$PRETTY_PRINT_STACKS" = "yes" ]; then > - ./scripts/pretty_print_stacks.py $1 | process_test_output > + local kernel="$2" > + ./scripts/pretty_print_stacks.py "$kernel" | process_test_output "$testname" > else > - process_test_output > + process_test_output "$testname" > fi > } > > diff --git a/scripts/runtime.bash b/scripts/runtime.bash > index caa4c5ba18cc..294e6b15a5e2 100644 > --- a/scripts/runtime.bash > +++ b/scripts/runtime.bash > @@ -140,10 +140,10 @@ function run() > # extra_params in the config file may contain backticks that need to be > # expanded, so use eval to start qemu. Use "> >(foo)" instead of a pipe to > # preserve the exit status. > - summary=$(eval $cmdline 2> >(RUNTIME_log_stderr) \ > - > >(tee >(RUNTIME_log_stdout $kernel) | extract_summary)) > + summary=$(eval $cmdline 2> >(RUNTIME_log_stderr $testname) \ > + > >(tee >(RUNTIME_log_stdout $testname $kernel) | extract_summary)) > ret=$? > - [ "$STANDALONE" != "yes" ] && echo > >(RUNTIME_log_stdout $kernel) > + [ "$STANDALONE" != "yes" ] && echo > >(RUNTIME_log_stdout $testname $kernel) > > if [ $ret -eq 0 ]; then > print_result "PASS" $testname "$summary" >
On Fri, Aug 21, 2020 at 04:05 PM +0200, Janosch Frank <frankja@linux.ibm.com> wrote: > On 8/21/20 2:37 PM, Marc Hartmayer wrote: >> Use the same test names in the TAP13 output as in the default output >> format. This makes the output more consistent. To achieve this, we >> need to pass the test name as an argument to the function >> `process_test_output`. >> >> Before this change: >> $ ./run_tests.sh >> PASS selftest-setup (14 tests) >> ... >> >> vs. >> >> $ ./run_tests.sh -t >> TAP version 13 >> ok 1 - selftest: true >> ok 2 - selftest: argc == 3 >> ... >> >> After this change: >> $ ./run_tests.sh >> PASS selftest-setup (14 tests) >> ... >> >> vs. >> >> $ ./run_tests.sh -t >> TAP version 13 >> ok 1 - selftest-setup: true >> ok 2 - selftest-setup: argc == 3 > > This doesn't work for me, we can't just drop prefixes. > > I.e. it needs to be "testname: prefix1: prefix2: prefixn: Test text" > selftest-setup: selftest: true Okay, I’ll fix that. […snip]
diff --git a/run_tests.sh b/run_tests.sh index 01e36dcfa06e..b5812336866f 100755 --- a/run_tests.sh +++ b/run_tests.sh @@ -81,18 +81,19 @@ if [[ $tap_output == "no" ]]; then postprocess_suite_output() { cat; } else process_test_output() { + local testname="$1" CR=$'\r' while read -r line; do line="${line%$CR}" case "${line:0:4}" in PASS) - echo "ok TEST_NUMBER - ${line#??????}" >&3 + echo "ok TEST_NUMBER - ${testname}:${line#*:*:}" >&3 ;; FAIL) - echo "not ok TEST_NUMBER - ${line#??????}" >&3 + echo "not ok TEST_NUMBER - ${testname}:${line#*:*:}" >&3 ;; SKIP) - echo "ok TEST_NUMBER - ${line#??????} # skip" >&3 + echo "ok TEST_NUMBER - ${testname}:${line#*:*:} # skip" >&3 ;; *) ;; @@ -114,12 +115,14 @@ else } fi -RUNTIME_log_stderr () { process_test_output; } +RUNTIME_log_stderr () { process_test_output "$1"; } RUNTIME_log_stdout () { + local testname="$1" if [ "$PRETTY_PRINT_STACKS" = "yes" ]; then - ./scripts/pretty_print_stacks.py $1 | process_test_output + local kernel="$2" + ./scripts/pretty_print_stacks.py "$kernel" | process_test_output "$testname" else - process_test_output + process_test_output "$testname" fi } diff --git a/scripts/runtime.bash b/scripts/runtime.bash index caa4c5ba18cc..294e6b15a5e2 100644 --- a/scripts/runtime.bash +++ b/scripts/runtime.bash @@ -140,10 +140,10 @@ function run() # extra_params in the config file may contain backticks that need to be # expanded, so use eval to start qemu. Use "> >(foo)" instead of a pipe to # preserve the exit status. - summary=$(eval $cmdline 2> >(RUNTIME_log_stderr) \ - > >(tee >(RUNTIME_log_stdout $kernel) | extract_summary)) + summary=$(eval $cmdline 2> >(RUNTIME_log_stderr $testname) \ + > >(tee >(RUNTIME_log_stdout $testname $kernel) | extract_summary)) ret=$? - [ "$STANDALONE" != "yes" ] && echo > >(RUNTIME_log_stdout $kernel) + [ "$STANDALONE" != "yes" ] && echo > >(RUNTIME_log_stdout $testname $kernel) if [ $ret -eq 0 ]; then print_result "PASS" $testname "$summary"
Use the same test names in the TAP13 output as in the default output format. This makes the output more consistent. To achieve this, we need to pass the test name as an argument to the function `process_test_output`. Before this change: $ ./run_tests.sh PASS selftest-setup (14 tests) ... vs. $ ./run_tests.sh -t TAP version 13 ok 1 - selftest: true ok 2 - selftest: argc == 3 ... After this change: $ ./run_tests.sh PASS selftest-setup (14 tests) ... vs. $ ./run_tests.sh -t TAP version 13 ok 1 - selftest-setup: true ok 2 - selftest-setup: argc == 3 ... While at it, introduce a local variable `kernel` in `RUNTIME_log_stdout` since this makes the function easier to read. Signed-off-by: Marc Hartmayer <mhartmay@linux.ibm.com> --- run_tests.sh | 15 +++++++++------ scripts/runtime.bash | 6 +++--- 2 files changed, 12 insertions(+), 9 deletions(-)