| Message ID | CANeU7QnHYuW7-h+wKFcQZe=6xNUZgNdSAtLnqdAm=OBNARvR-Q@mail.gmail.com (mailing list archive) |
|---|---|
| State | Superseded, archived |
| Headers | show |
On Fri, Nov 18, 2016 at 12:13:58AM +0800, Christopher Li wrote: > On Thu, Nov 3, 2016 at 5:45 AM, Luc Van Oostenryck > <luc.vanoostenryck@gmail.com> wrote: > > Such situation may simply show that what was tested is now fixed > > and that it's juste the test annotation which need to be adapted, > > but can be a sign that something else is broken. > > > > Reporting the exact result (failure/success, known-to-fail/expect-to-succeed) > > make the testsuite more useful and allow to use more efficiently > > git-bisect or other automated testing tools. > > I like what you are doing as a result. > > But I think the implementation has room to improve. > I don't like deep and messy "else" statement. > > I attach a patch which I modify base on yours. I hope it is > easier to read. Care to review it? Yes, I much prefer like this. Thanks. But there is something I would like to check but I can't do that now. I'll come back on it later. Luc -- To unsubscribe from this list: send the line "unsubscribe linux-sparse" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
--- .pc/9409919-3-5-testsuite-report-as-error-tests-known-to-fail-but-which-succeed.patch/validation/test-suite 2016-10-24 17:37:29.303846000 +0800 +++ validation/test-suite 2016-11-17 23:56:35.404392454 +0800 @@ -161,18 +161,25 @@ do_test() test_failed=1 fi - if [ "$test_failed" -eq "1" ]; then - ko_tests=`expr $ko_tests + 1` - get_tag "check-known-to-fail" $file - if [ "$?" -eq "0" ]; then + get_tag "check-known-to-fail" $file + must_fail=`expr "$?" = 0` + known_ko_tests=`expr $known_ko_tests + $must_fail` + + if [ "$must_fail" -eq "1" ]; then + if [ "$test_failed" -eq "1" ]; then echo "info: test '$file' is known to fail" - known_ko_tests=`expr $known_ko_tests + 1` + else + echo "info: test '$file' is known to fail but succeed!" + test_failed=1 fi - return 1 + fi + + if [ "$test_failed" -eq "1" ]; then + ko_tests=`expr $ko_tests + 1` else ok_tests=`expr $ok_tests + 1` - return 0 fi + return $test_failed } do_test_suite()