| Message ID | 20201105105051.64258-3-po-hsu.lin@canonical.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | selftests: pmtu.sh: improve the test result processing | expand |
On Thu, 5 Nov 2020 18:50:51 +0800 Po-Hsu Lin wrote: > This test will treat all non-zero return codes as failures, it will > make the pmtu.sh test script being marked as FAILED when some > sub-test got skipped. > > Improve the result processing by > * Only mark the whole test script as SKIP when all of the > sub-tests were skipped > * If the sub-tests were either passed or skipped, the overall > result will be PASS > * If any of them has failed, the overall result will be FAIL > * Treat other return codes (e.g. 127 for command not found) as FAIL > > Signed-off-by: Po-Hsu Lin <po-hsu.lin@canonical.com> Patch 1 looks like a cleanup while patch 2 is more of a fix, can we separate the two and apply the former to -next and latter to 5.10? They shouldn't conflict, right? > diff --git a/tools/testing/selftests/net/pmtu.sh b/tools/testing/selftests/net/pmtu.sh > index fb53987..5c86fb1 100755 > --- a/tools/testing/selftests/net/pmtu.sh > +++ b/tools/testing/selftests/net/pmtu.sh > @@ -1652,7 +1652,23 @@ run_test() { > return $ret > ) > ret=$? > - [ $ret -ne 0 ] && exitcode=1 > + case $ret in > + 0) > + all_skipped=false > + [ $exitcode=$ksft_skip ] && exitcode=0 > + ;; > + 1) > + all_skipped=false > + exitcode=1 > + ;; Does it make sense to remove this case? The handling is identical to the default case *). > + $ksft_skip) > + [ $all_skipped = true ] && exitcode=$ksft_skip > + ;; > + *) > + all_skipped=false > + exitcode=1 > + ;; > + esac > > return $ret > } > @@ -1786,6 +1802,7 @@ usage() { > # > exitcode=0 > desc=0 > +all_skipped=true > > while getopts :ptv o > do
On Sun, Nov 8, 2020 at 7:02 AM Jakub Kicinski <kuba@kernel.org> wrote: > > On Thu, 5 Nov 2020 18:50:51 +0800 Po-Hsu Lin wrote: > > This test will treat all non-zero return codes as failures, it will > > make the pmtu.sh test script being marked as FAILED when some > > sub-test got skipped. > > > > Improve the result processing by > > * Only mark the whole test script as SKIP when all of the > > sub-tests were skipped > > * If the sub-tests were either passed or skipped, the overall > > result will be PASS > > * If any of them has failed, the overall result will be FAIL > > * Treat other return codes (e.g. 127 for command not found) as FAIL > > > > Signed-off-by: Po-Hsu Lin <po-hsu.lin@canonical.com> > > Patch 1 looks like a cleanup while patch 2 is more of a fix, can we > separate the two and apply the former to -next and latter to 5.10? > They shouldn't conflict, right? > Hello Jakub, Yes the first patch is just changing return code to $ksft_skip, the real fix is the second one. However the second patch was based on the first one, if we want to apply them separately we might need to change this $ksft_skip handling part in the second patch. What should I do to deal with this? Resend the former for -next and rebase + resend the latter (plus the fix to remove case 1) for 5.10 without the former patch? Thanks! > > diff --git a/tools/testing/selftests/net/pmtu.sh b/tools/testing/selftests/net/pmtu.sh > > index fb53987..5c86fb1 100755 > > --- a/tools/testing/selftests/net/pmtu.sh > > +++ b/tools/testing/selftests/net/pmtu.sh > > @@ -1652,7 +1652,23 @@ run_test() { > > return $ret > > ) > > ret=$? > > - [ $ret -ne 0 ] && exitcode=1 > > + case $ret in > > + 0) > > + all_skipped=false > > + [ $exitcode=$ksft_skip ] && exitcode=0 > > + ;; > > + 1) > > + all_skipped=false > > + exitcode=1 > > + ;; > > Does it make sense to remove this case? The handling is identical to > the default case *). > Yes you're right, we can remove this part. > > + $ksft_skip) > > + [ $all_skipped = true ] && exitcode=$ksft_skip > > + ;; > > + *) > > + all_skipped=false > > + exitcode=1 > > + ;; > > + esac > > > > return $ret > > } > > @@ -1786,6 +1802,7 @@ usage() { > > # > > exitcode=0 > > desc=0 > > +all_skipped=true > > > > while getopts :ptv o > > do >
On Mon, 9 Nov 2020 11:42:33 +0800 Po-Hsu Lin wrote: > On Sun, Nov 8, 2020 at 7:02 AM Jakub Kicinski <kuba@kernel.org> wrote: > > > > On Thu, 5 Nov 2020 18:50:51 +0800 Po-Hsu Lin wrote: > > > This test will treat all non-zero return codes as failures, it will > > > make the pmtu.sh test script being marked as FAILED when some > > > sub-test got skipped. > > > > > > Improve the result processing by > > > * Only mark the whole test script as SKIP when all of the > > > sub-tests were skipped > > > * If the sub-tests were either passed or skipped, the overall > > > result will be PASS > > > * If any of them has failed, the overall result will be FAIL > > > * Treat other return codes (e.g. 127 for command not found) as FAIL > > > > > > Signed-off-by: Po-Hsu Lin <po-hsu.lin@canonical.com> > > > > Patch 1 looks like a cleanup while patch 2 is more of a fix, can we > > separate the two and apply the former to -next and latter to 5.10? > > They shouldn't conflict, right? > > > > Hello Jakub, > > Yes the first patch is just changing return code to $ksft_skip, the > real fix is the second one. However the second patch was based on the > first one, if we want to apply them separately we might need to change > this $ksft_skip handling part in the second patch. Ah, I misread the situation, ksft_skip is 4, not 2, so the patch is more than just refactoring. > What should I do to deal with this? > Resend the former for -next and rebase + resend the latter (plus the > fix to remove case 1) for 5.10 without the former patch? Let's apply both of the patches to net-next if that's fine with you. Indeed detangling them is may be more effort that it's worth.
On Tue, Nov 10, 2020 at 2:09 AM Jakub Kicinski <kuba@kernel.org> wrote: > > On Mon, 9 Nov 2020 11:42:33 +0800 Po-Hsu Lin wrote: > > On Sun, Nov 8, 2020 at 7:02 AM Jakub Kicinski <kuba@kernel.org> wrote: > > > > > > On Thu, 5 Nov 2020 18:50:51 +0800 Po-Hsu Lin wrote: > > > > This test will treat all non-zero return codes as failures, it will > > > > make the pmtu.sh test script being marked as FAILED when some > > > > sub-test got skipped. > > > > > > > > Improve the result processing by > > > > * Only mark the whole test script as SKIP when all of the > > > > sub-tests were skipped > > > > * If the sub-tests were either passed or skipped, the overall > > > > result will be PASS > > > > * If any of them has failed, the overall result will be FAIL > > > > * Treat other return codes (e.g. 127 for command not found) as FAIL > > > > > > > > Signed-off-by: Po-Hsu Lin <po-hsu.lin@canonical.com> > > > > > > Patch 1 looks like a cleanup while patch 2 is more of a fix, can we > > > separate the two and apply the former to -next and latter to 5.10? > > > They shouldn't conflict, right? > > > > > > > Hello Jakub, > > > > Yes the first patch is just changing return code to $ksft_skip, the > > real fix is the second one. However the second patch was based on the > > first one, if we want to apply them separately we might need to change > > this $ksft_skip handling part in the second patch. > > Ah, I misread the situation, ksft_skip is 4, not 2, so the patch is > more than just refactoring. > > > What should I do to deal with this? > > Resend the former for -next and rebase + resend the latter (plus the > > fix to remove case 1) for 5.10 without the former patch? > > Let's apply both of the patches to net-next if that's fine with you. > Indeed detangling them is may be more effort that it's worth. That would be great, but allow me to resend V2 to get rid of case 1 first. Thanks!
diff --git a/tools/testing/selftests/net/pmtu.sh b/tools/testing/selftests/net/pmtu.sh index fb53987..5c86fb1 100755 --- a/tools/testing/selftests/net/pmtu.sh +++ b/tools/testing/selftests/net/pmtu.sh @@ -1652,7 +1652,23 @@ run_test() { return $ret ) ret=$? - [ $ret -ne 0 ] && exitcode=1 + case $ret in + 0) + all_skipped=false + [ $exitcode=$ksft_skip ] && exitcode=0 + ;; + 1) + all_skipped=false + exitcode=1 + ;; + $ksft_skip) + [ $all_skipped = true ] && exitcode=$ksft_skip + ;; + *) + all_skipped=false + exitcode=1 + ;; + esac return $ret } @@ -1786,6 +1802,7 @@ usage() { # exitcode=0 desc=0 +all_skipped=true while getopts :ptv o do
This test will treat all non-zero return codes as failures, it will make the pmtu.sh test script being marked as FAILED when some sub-test got skipped. Improve the result processing by * Only mark the whole test script as SKIP when all of the sub-tests were skipped * If the sub-tests were either passed or skipped, the overall result will be PASS * If any of them has failed, the overall result will be FAIL * Treat other return codes (e.g. 127 for command not found) as FAIL Signed-off-by: Po-Hsu Lin <po-hsu.lin@canonical.com> --- tools/testing/selftests/net/pmtu.sh | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-)