From patchwork Fri Mar 17 13:25:18 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Petri Latvala X-Patchwork-Id: 9630739 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 108D860245 for ; Fri, 17 Mar 2017 13:25:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0839F1FF1E for ; Fri, 17 Mar 2017 13:25:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F13E7284A5; Fri, 17 Mar 2017 13:25:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 89C2A1FF1E for ; Fri, 17 Mar 2017 13:25:58 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 205356ED3B; Fri, 17 Mar 2017 13:25:50 +0000 (UTC) X-Original-To: intel-gfx@lists.freedesktop.org Delivered-To: intel-gfx@lists.freedesktop.org Received: from johanna1.inet.fi (mta-out1.inet.fi [62.71.2.229]) by gabe.freedesktop.org (Postfix) with ESMTP id 6FB6B6ED3F for ; Fri, 17 Mar 2017 13:25:49 +0000 (UTC) RazorGate-KAS: Status: not_detected RazorGate-KAS: Rate: 0 RazorGate-KAS: Envelope from: RazorGate-KAS: Version: 5.5.3 RazorGate-KAS: LuaCore: 80 2014-11-10_18-01-23 260f8afb9361da3c7edfd3a8e3a4ca908191ad29 RazorGate-KAS: Lua profiles 69136 [Nov 12 2014] RazorGate-KAS: Method: none Received: from hufflepuff.adrinael.net (84.250.112.128) by johanna1.inet.fi (9.0.002.03-2-gbe5d057) id 58C29BB000F4E767; Fri, 17 Mar 2017 15:25:49 +0200 Received: from adrinael by hufflepuff.adrinael.net with local (Exim 4.84_2) (envelope-from ) id 1cort8-00019z-13; Fri, 17 Mar 2017 15:25:50 +0200 From: Petri Latvala To: intel-gfx@lists.freedesktop.org Date: Fri, 17 Mar 2017 15:25:18 +0200 Message-Id: <1489757119-4280-4-git-send-email-petri.latvala@intel.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1489757119-4280-1-git-send-email-petri.latvala@intel.com> References: <1489757119-4280-1-git-send-email-petri.latvala@intel.com> MIME-Version: 1.0 Subject: [Intel-gfx] [PATCH i-g-t 3/4] intel-ci: Build-time check for testlists X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" X-Virus-Scanned: ClamAV using ClamSMTP Test at check and distcheck that testlist files only contain tests that are still present. Signed-off-by: Petri Latvala --- configure.ac | 1 + tests/Makefile.am | 2 ++ tests/intel-ci/.gitignore | 1 + tests/intel-ci/Makefile.am | 42 +++++++++++++++++++++++++ tests/intel-ci/check_testlists.sh | 65 +++++++++++++++++++++++++++++++++++++++ tests/intel-ci/checked_lists.txt | 1 + 6 files changed, 112 insertions(+) create mode 100644 tests/intel-ci/.gitignore create mode 100644 tests/intel-ci/Makefile.am create mode 100755 tests/intel-ci/check_testlists.sh create mode 100644 tests/intel-ci/checked_lists.txt diff --git a/configure.ac b/configure.ac index 12b0ab0..9aa6068 100644 --- a/configure.ac +++ b/configure.ac @@ -341,6 +341,7 @@ AC_CONFIG_FILES([ man/Makefile scripts/Makefile tests/Makefile + tests/intel-ci/Makefile tools/Makefile tools/null_state_gen/Makefile tools/registers/Makefile diff --git a/tests/Makefile.am b/tests/Makefile.am index 2080b3f..001106f 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -1,5 +1,7 @@ include Makefile.sources +SUBDIRS = . intel-ci + if HAVE_LIBDRM_NOUVEAU TESTS_progs_M += $(NOUVEAU_TESTS_M) endif diff --git a/tests/intel-ci/.gitignore b/tests/intel-ci/.gitignore new file mode 100644 index 0000000..1a8f133 --- /dev/null +++ b/tests/intel-ci/.gitignore @@ -0,0 +1 @@ +all-tests.testlist diff --git a/tests/intel-ci/Makefile.am b/tests/intel-ci/Makefile.am new file mode 100644 index 0000000..de92b24 --- /dev/null +++ b/tests/intel-ci/Makefile.am @@ -0,0 +1,42 @@ +if BUILD_TESTS + +TESTLIST = $(top_builddir)/tests/test-list.txt + +ALL_TESTS = all-tests.testlist +CHECKED_TESTLISTS = checked_lists.txt + +check_DATA = $(ALL_TESTS) +$(ALL_TESTS): $(TESTLIST) + echo "# This file is automatically generated at build-time." > $@ + echo "# It contains a list of all available (sub)tests, " >> $@ + echo "# for checking other lists against it." >> $@ + echo >> $@ + for test in `cat $(TESTLIST)`; do \ + if [ "$$test" = "TESTLIST" -o "$$test" = "END" ]; then \ + continue; \ + fi; \ + if [ -x $(top_builddir)/tests/$$test ]; then \ + testprog=$(top_builddir)/tests/$$test; \ + else \ + testprog=$(top_srcdir)/tests/$$test; \ + fi; \ + progbase=`basename $$testprog | tr '[:upper:]' '[:lower:]'`; \ + if ./$$testprog --list-subtests > /dev/null ; then \ + for subtest in `./$$testprog --list-subtests`; do \ + subname=`echo $$subtest | tr '[:upper:]' '[:lower:]'`; \ + echo "igt@$$progbase@$$subname" >> $@; \ + done; \ + else \ + echo "igt@$$progbase" >> $@; \ + fi; \ + done + +CLEANFILES = $(ALL_TESTS) + +check_SCRIPTS = check_testlists.sh + +EXTRA_DIST = $(CHECKED_TESTLISTS) $(shell cat $(CHECKED_TESTLISTS)) $(check_SCRIPTS) + +TESTS = $(check_SCRIPTS) + +endif diff --git a/tests/intel-ci/check_testlists.sh b/tests/intel-ci/check_testlists.sh new file mode 100755 index 0000000..648364d --- /dev/null +++ b/tests/intel-ci/check_testlists.sh @@ -0,0 +1,65 @@ +#!/bin/sh +# +# Copyright © 2017 Intel Corporation +# +# Permission is hereby granted, free of charge, to any person obtaining a +# copy of this software and associated documentation files (the "Software"), +# to deal in the Software without restriction, including without limitation +# the rights to use, copy, modify, merge, publish, distribute, sublicense, +# and/or sell copies of the Software, and to permit persons to whom the +# Software is furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice (including the next +# paragraph) shall be included in all copies or substantial portions of the +# Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS +# IN THE SOFTWARE. + +# +# Check that testlist files don't have tests that are not available +# + +if [ -z "$top_builddir" ]; then + top_builddir="$(dirname $0)" +fi + +# allow to run this script from top directory +TESTLISTS=`cat $top_builddir/checked_lists.txt` +if [ $? -ne 0 ]; then + # distcheck requires this hack + TESTLISTS=$(cat checked_lists.txt) + if [ $? -ne 0 ]; then + echo "Error: Could not read list of testlists" + exit 99 + fi +fi + +ALLTESTS="$top_builddir/all-tests.testlist" +if [ ! -r "$ALLTESTS" ]; then + # distcheck requires this hack + ALLTESTS=all-tests.testlist + if [ ! -r "$ALLTESTS" ]; then + echo "Error: Could not read list of all tests" + exit 99 + fi +fi + +failure=0 +fail () { + echo "FAIL: Test $1 in test list $2 does not exist" + failure=1 +} + +for testlist in $TESTLISTS; do + for testname in `cat "$testlist" | sed 's/#.*$//g'`; do + grep -q "^$testname\$" "$ALLTESTS" || fail "$testname" "$testlist" + done +done + +exit $failure diff --git a/tests/intel-ci/checked_lists.txt b/tests/intel-ci/checked_lists.txt new file mode 100644 index 0000000..2f8a386 --- /dev/null +++ b/tests/intel-ci/checked_lists.txt @@ -0,0 +1 @@ +fast-feedback.testlist