From patchwork Mon Jul 7 10:49:05 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Thomas Wood X-Patchwork-Id: 4494421 Return-Path: X-Original-To: patchwork-intel-gfx@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 58B6BBEEAA for ; Mon, 7 Jul 2014 10:49:17 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 6156B2024C for ; Mon, 7 Jul 2014 10:49:16 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) by mail.kernel.org (Postfix) with ESMTP id 4E99E20225 for ; Mon, 7 Jul 2014 10:49:15 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C406B6E259; Mon, 7 Jul 2014 03:49:14 -0700 (PDT) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from mail-wi0-f179.google.com (mail-wi0-f179.google.com [209.85.212.179]) by gabe.freedesktop.org (Postfix) with ESMTP id A9B556E259 for ; Mon, 7 Jul 2014 03:49:13 -0700 (PDT) Received: by mail-wi0-f179.google.com with SMTP id cc10so6473651wib.6 for ; Mon, 07 Jul 2014 03:49:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-type:content-transfer-encoding; bh=na1X5NfqTkcFGtlRr0PZkOnYvWunu0MF227H50hIZzE=; b=E5lJDzQ4+tHWRBzqC5I7EGPILH5wxnGMKXlx5ENPCoMd6ZPftApUeE1AoAxtQ4wyXx 8JaCL2Ig+PDELbhLGl4Dg6VWc7gKzgXkB2iVyNvdLzTINvhtrWTNAI/OlfKA88sHA7Nf 6EY4pr4fm3+wXsqMH7y0NeARRS1xPvefENYnIvoHCjoUkD2cyxT4DooQ6YdZ++ss8PW6 etH5LVkdVw59j3MkoFuP+GL0FbHAyEyG4x+hZHzF2SdZzf3yAJamMLb/SZ0XkTPN6ulJ EpPJ/WQlFbKVYy2h7RyCvNQcZtnKc7Ve40WOQ/sjOcJ1dWGcGPaqQ7gwP4y4bhsHBjE1 6C5Q== X-Gm-Message-State: ALoCoQnp2OtRbiB5MetbBb3Bh1LqnxrF1we4rqYT3tKQF6akK3mu4xCGeLTE/nSy3n4RRZG/YgBt X-Received: by 10.194.221.6 with SMTP id qa6mr30894902wjc.39.1404730148113; Mon, 07 Jul 2014 03:49:08 -0700 (PDT) Received: from pistachio.icx.intel.com ([83.217.123.106]) by mx.google.com with ESMTPSA id 19sm88654924wjz.3.2014.07.07.03.49.06 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Jul 2014 03:49:07 -0700 (PDT) From: Thomas Wood To: intel-gfx@lists.freedesktop.org Date: Mon, 7 Jul 2014 11:49:05 +0100 Message-Id: <1404730145-15931-1-git-send-email-thomas.wood@intel.com> X-Mailer: git-send-email 1.9.3 MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH i-g-t] scripts: add a script to help run tests with Piglit X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add a script to facilitate running the tests with Piglit by providing simplified options for listing, filtering and creating summaries of test runs. Signed-off-by: Thomas Wood --- .gitignore | 3 ++ scripts/Makefile.am | 2 +- scripts/run-tests.sh | 120 +++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 124 insertions(+), 1 deletion(-) create mode 100755 scripts/run-tests.sh diff --git a/.gitignore b/.gitignore index 5b9ac8f..f40b4f6 100644 --- a/.gitignore +++ b/.gitignore @@ -88,3 +88,6 @@ version.h gtk-doc.make gtk-doc.m4 + +piglit +results diff --git a/scripts/Makefile.am b/scripts/Makefile.am index baf3612..b77f0eb 100644 --- a/scripts/Makefile.am +++ b/scripts/Makefile.am @@ -1,3 +1,3 @@ -dist_noinst_SCRIPTS = who.sh +dist_noinst_SCRIPTS = who.sh run-tests.sh noinst_PYTHON = throttle.py diff --git a/scripts/run-tests.sh b/scripts/run-tests.sh new file mode 100755 index 0000000..cdb6e08 --- /dev/null +++ b/scripts/run-tests.sh @@ -0,0 +1,120 @@ +#!/bin/bash +# +# Copyright © 2014 Intel Corporation +# +# Permission is hereby granted, free of charge, to any person obtaining a +# copy of this software and associated documentation files (the "Software"), +# to deal in the Software without restriction, including without limitation +# the rights to use, copy, modify, merge, publish, distribute, sublicense, +# and/or sell copies of the Software, and to permit persons to whom the +# Software is furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice (including the next +# paragraph) shall be included in all copies or substantial portions of the +# Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +# IN THE SOFTWARE. + + +ROOT=`dirname $0` +ROOT=`readlink -f $ROOT/..` +IGT_TEST_ROOT=$ROOT/tests +RESULTS=$ROOT/results +PIGLIT=`which piglit 2> /dev/null` + +if [ ! -d "$IGT_TEST_ROOT" ]; then + echo "Error: could not find tests directory." + exit 1 +fi + +if [ ! -f "$IGT_TEST_ROOT/single-tests.txt" ]; then + echo "Error: test list not found." + echo "Please run make in the tests directory to generate the test list." +fi + +SINGLE_TEST_LIST=`cat $IGT_TEST_ROOT/single-tests.txt | sed -e '/TESTLIST/d' -e 's/ /\n/g'` +MULTI_TEST_LIST=`cat $IGT_TEST_ROOT/multi-tests.txt | sed -e '/TESTLIST/d' -e 's/ /\n/g'` + +function download_piglit { + git clone git://anongit.freedesktop.org/piglit $ROOT/piglit +} + +function print_help { + echo "Usage: run-tests.sh [options]" + echo "Available options:" + echo " -d download Piglit to $ROOT/piglit" + echo " -h display this help message" + echo " -l list all available tests" + echo " -r store the results in directory" + echo " (default: $RESULTS)" + echo " -s create html summary" + echo " -t only include tests that match the regular expression" + echo " -v enable verbose mode" + echo " -x exclude tests that match the regular expression" + echo "" + echo "Useful patterns for test filtering are described in tests/NAMING-CONVENTION" +} + +function list_tests { + echo "$SINGLE_TEST_LIST" + for test in $MULTI_TEST_LIST; do + SUBTESTS=`$IGT_TEST_ROOT/$test --list-subtests` + for subtest in $SUBTESTS; do + echo "$test/$subtest" + done + done +} + +while getopts ":dhlr:st:vx:" opt; do + case $opt in + d) download_piglit; exit ;; + h) print_help; exit ;; + l) list_tests; exit ;; + r) RESULTS="$OPTARG" ;; + s) SUMMARY="html" ;; + t) FILTER="-t $OPTARG" ;; + v) VERBOSE="-v" ;; + x) EXCLUDE="-x $OPTARG" ;; + :) + echo "Option -$OPTARG requires an argument." + exit 1 + ;; + \?) + echo "Unknown option: -$OPTARG" + print_help + exit 1 + ;; + esac +done +shift $(($OPTIND-1)) + +if [ "x$1" != "x" ]; then + echo "Unknown option: $1" + print_help + exit 1 +fi + +if [ "x$PIGLIT" == "x" ]; then + PIGLIT="$ROOT/piglit/piglit" +fi + +if [ ! -x "$PIGLIT" ]; then + echo "Could not find Piglit." + echo "Please install Piglit or use -d to download Piglit locally." + exit 1 +fi + +mkdir -p $RESULTS + +sudo IGT_TEST_ROOT=$IGT_TEST_ROOT $PIGLIT run igt $RESULTS $VERBOSE $EXCLUDE $FILTER + +if [ "$SUMMARY" == "html" ]; then + $PIGLIT summary html --overwrite $RESULTS/html $RESULTS + echo "HTML summary has been written to $RESULTS/html/index.html" +fi