| Message ID | 1448628036-7410-1-git-send-email-gabriel.feceoru@intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 27 November 2015 at 12:40, Gabriel Feceoru <gabriel.feceoru@intel.com> wrote: > This script takes multiple test results and prints the diffs, useful > for detecting noise in subsequent executions. > It gives the option to re-run these tests. This feature might be useful in Piglit itself. It sounds similar to the ideas mentioned in this thread: http://lists.freedesktop.org/archives/piglit/2015-November/018197.html > > Signed-off-by: Gabriel Feceoru <gabriel.feceoru@intel.com> > --- > scripts/retest-diff.sh | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 72 insertions(+) > create mode 100755 scripts/retest-diff.sh > > diff --git a/scripts/retest-diff.sh b/scripts/retest-diff.sh > new file mode 100755 > index 0000000..aee1d05 > --- /dev/null > +++ b/scripts/retest-diff.sh > @@ -0,0 +1,72 @@ > +#!/bin/bash > +# > +# Copyright © 2015 Intel Corporation > +# > +# Permission is hereby granted, free of charge, to any person obtaining a > +# copy of this software and associated documentation files (the "Software"), > +# to deal in the Software without restriction, including without limitation > +# the rights to use, copy, modify, merge, publish, distribute, sublicense, > +# and/or sell copies of the Software, and to permit persons to whom the > +# Software is furnished to do so, subject to the following conditions: > +# > +# The above copyright notice and this permission notice (including the next > +# paragraph) shall be included in all copies or substantial portions of the > +# Software. > +# > +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL > +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING > +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS > +# IN THE SOFTWARE. > + > +ROOT="`dirname $0`" > +ROOT="`readlink -f $ROOT/..`" > +IGT_TEST_ROOT="$ROOT/tests" > +RESULTS="$ROOT/results" > +PIGLIT="$ROOT/piglit/piglit" > + > +if [ ! -d "$IGT_TEST_ROOT" ]; then > + echo "Error: could not find tests directory." > + exit 1 > +fi > + > +function print_help { > + echo "Usage: $0 [<Results Paths> ...]" > + echo "" > +} > + > + > +if [ $# -lt 2 ]; then > + print_help > + exit 1 > +fi > + > +RESULT_FILES=$@ > + > +for result in $RESULT_FILES; do > + if [ ! -e $result ]; then > + echo "Wrong result paths" > + exit 1 > + fi > +done > + > + > +TESTS=`$PIGLIT summary console -d $RESULT_FILES | grep igt | cut -d':' -f1` > +if [[ -z $TESTS ]]; then > + exit 1 > +else > + RED='\033[0;31m' > + NC='\033[0m' > + printf "$RED $TESTS $NC " | sed -e 's/ /\n/g' > +fi > + > + > +read -p "Dow you want to run these tests [y/N]? " -n 2 -r > +if [[ $REPLY =~ ^[Yy]$ ]]; then > + mkdir -p $RESULTS > + TEST_LIST=($TESTS) > + TEST_PARAMS=`printf "%s " "${TEST_LIST[@]/#/-t }"` > + sudo IGT_TEST_ROOT=$IGT_TEST_ROOT $PIGLIT run igt $RESULTS $TEST_PARAMS > +fi > -- > 1.9.1 > > _______________________________________________ > Intel-gfx mailing list > Intel-gfx@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/intel-gfx
On Fri, Nov 27, 2015 at 02:53:36PM +0000, Thomas Wood wrote: > On 27 November 2015 at 12:40, Gabriel Feceoru <gabriel.feceoru@intel.com> wrote: > > This script takes multiple test results and prints the diffs, useful > > for detecting noise in subsequent executions. > > It gives the option to re-run these tests. > > This feature might be useful in Piglit itself. It sounds similar to > the ideas mentioned in this thread: > > http://lists.freedesktop.org/archives/piglit/2015-November/018197.html Yeah agreed, imo this should be implemented in piglit proper. We could have different knobs to retest just regressions, fixes, or all changes even. -Daniel > > > > > > Signed-off-by: Gabriel Feceoru <gabriel.feceoru@intel.com> > > --- > > scripts/retest-diff.sh | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++ > > 1 file changed, 72 insertions(+) > > create mode 100755 scripts/retest-diff.sh > > > > diff --git a/scripts/retest-diff.sh b/scripts/retest-diff.sh > > new file mode 100755 > > index 0000000..aee1d05 > > --- /dev/null > > +++ b/scripts/retest-diff.sh > > @@ -0,0 +1,72 @@ > > +#!/bin/bash > > +# > > +# Copyright © 2015 Intel Corporation > > +# > > +# Permission is hereby granted, free of charge, to any person obtaining a > > +# copy of this software and associated documentation files (the "Software"), > > +# to deal in the Software without restriction, including without limitation > > +# the rights to use, copy, modify, merge, publish, distribute, sublicense, > > +# and/or sell copies of the Software, and to permit persons to whom the > > +# Software is furnished to do so, subject to the following conditions: > > +# > > +# The above copyright notice and this permission notice (including the next > > +# paragraph) shall be included in all copies or substantial portions of the > > +# Software. > > +# > > +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > > +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > > +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL > > +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > > +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING > > +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS > > +# IN THE SOFTWARE. > > + > > +ROOT="`dirname $0`" > > +ROOT="`readlink -f $ROOT/..`" > > +IGT_TEST_ROOT="$ROOT/tests" > > +RESULTS="$ROOT/results" > > +PIGLIT="$ROOT/piglit/piglit" > > + > > +if [ ! -d "$IGT_TEST_ROOT" ]; then > > + echo "Error: could not find tests directory." > > + exit 1 > > +fi > > + > > +function print_help { > > + echo "Usage: $0 [<Results Paths> ...]" > > + echo "" > > +} > > + > > + > > +if [ $# -lt 2 ]; then > > + print_help > > + exit 1 > > +fi > > + > > +RESULT_FILES=$@ > > + > > +for result in $RESULT_FILES; do > > + if [ ! -e $result ]; then > > + echo "Wrong result paths" > > + exit 1 > > + fi > > +done > > + > > + > > +TESTS=`$PIGLIT summary console -d $RESULT_FILES | grep igt | cut -d':' -f1` > > +if [[ -z $TESTS ]]; then > > + exit 1 > > +else > > + RED='\033[0;31m' > > + NC='\033[0m' > > + printf "$RED $TESTS $NC " | sed -e 's/ /\n/g' > > +fi > > + > > + > > +read -p "Dow you want to run these tests [y/N]? " -n 2 -r > > +if [[ $REPLY =~ ^[Yy]$ ]]; then > > + mkdir -p $RESULTS > > + TEST_LIST=($TESTS) > > + TEST_PARAMS=`printf "%s " "${TEST_LIST[@]/#/-t }"` > > + sudo IGT_TEST_ROOT=$IGT_TEST_ROOT $PIGLIT run igt $RESULTS $TEST_PARAMS > > +fi > > -- > > 1.9.1 > > > > _______________________________________________ > > Intel-gfx mailing list > > Intel-gfx@lists.freedesktop.org > > http://lists.freedesktop.org/mailman/listinfo/intel-gfx > _______________________________________________ > Intel-gfx mailing list > Intel-gfx@lists.freedesktop.org > http://lists.freedesktop.org/mailman/listinfo/intel-gfx
diff --git a/scripts/retest-diff.sh b/scripts/retest-diff.sh new file mode 100755 index 0000000..aee1d05 --- /dev/null +++ b/scripts/retest-diff.sh @@ -0,0 +1,72 @@ +#!/bin/bash +# +# Copyright © 2015 Intel Corporation +# +# Permission is hereby granted, free of charge, to any person obtaining a +# copy of this software and associated documentation files (the "Software"), +# to deal in the Software without restriction, including without limitation +# the rights to use, copy, modify, merge, publish, distribute, sublicense, +# and/or sell copies of the Software, and to permit persons to whom the +# Software is furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice (including the next +# paragraph) shall be included in all copies or substantial portions of the +# Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +# IN THE SOFTWARE. + +ROOT="`dirname $0`" +ROOT="`readlink -f $ROOT/..`" +IGT_TEST_ROOT="$ROOT/tests" +RESULTS="$ROOT/results" +PIGLIT="$ROOT/piglit/piglit" + +if [ ! -d "$IGT_TEST_ROOT" ]; then + echo "Error: could not find tests directory." + exit 1 +fi + +function print_help { + echo "Usage: $0 [<Results Paths> ...]" + echo "" +} + + +if [ $# -lt 2 ]; then + print_help + exit 1 +fi + +RESULT_FILES=$@ + +for result in $RESULT_FILES; do + if [ ! -e $result ]; then + echo "Wrong result paths" + exit 1 + fi +done + + +TESTS=`$PIGLIT summary console -d $RESULT_FILES | grep igt | cut -d':' -f1` +if [[ -z $TESTS ]]; then + exit 1 +else + RED='\033[0;31m' + NC='\033[0m' + printf "$RED $TESTS $NC " | sed -e 's/ /\n/g' +fi + + +read -p "Dow you want to run these tests [y/N]? " -n 2 -r +if [[ $REPLY =~ ^[Yy]$ ]]; then + mkdir -p $RESULTS + TEST_LIST=($TESTS) + TEST_PARAMS=`printf "%s " "${TEST_LIST[@]/#/-t }"` + sudo IGT_TEST_ROOT=$IGT_TEST_ROOT $PIGLIT run igt $RESULTS $TEST_PARAMS +fi
This script takes multiple test results and prints the diffs, useful for detecting noise in subsequent executions. It gives the option to re-run these tests. Signed-off-by: Gabriel Feceoru <gabriel.feceoru@intel.com> --- scripts/retest-diff.sh | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) create mode 100755 scripts/retest-diff.sh