From patchwork Tue Apr 9 23:55:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 10892879 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 18D801708 for ; Tue, 9 Apr 2019 23:56:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 003E828387 for ; Tue, 9 Apr 2019 23:56:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E8B9628727; Tue, 9 Apr 2019 23:56:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 63A8928387 for ; Tue, 9 Apr 2019 23:56:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726756AbfDIX4M (ORCPT ); Tue, 9 Apr 2019 19:56:12 -0400 Received: from mail-pl1-f194.google.com ([209.85.214.194]:37010 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726741AbfDIX4L (ORCPT ); Tue, 9 Apr 2019 19:56:11 -0400 Received: by mail-pl1-f194.google.com with SMTP id w23so200489ply.4 for ; Tue, 09 Apr 2019 16:56:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=L3VkJhqGguQYNQ544jAt4E93KwkMKUR3iV5epN09oo4=; b=FGS7DyCAibmO6BnoFRAXuW6EO5qHF33CfL9ulkM+rkuwlxH3lmfHIqw/NnNECTMi2b jzGNttVsuTEwo3TbfkIhPdN15iQ4XpI2ZlasfTR84WCrGy1bLNdsE/0Tw2r2bsq8tNZ+ XPog2lwyk5oGn4vqqjsvkynybRpilyl7zFRz8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=L3VkJhqGguQYNQ544jAt4E93KwkMKUR3iV5epN09oo4=; b=Tgm8eG3ybvQI/1x98LZ9oh73raSmGPMAZIfEbp2ZqswbxEgON8Lijw51ESZOU4MmHg CYfH06RfubAUFt/Z3DbmvXk8IprWhYFvHR2jqfwRdELpWV5w9K1Wnuxiw7EqC2QENn1a s6qFcQ2s12IU4iNjI4fX2BrJxC7iJC5q6188MIk982Q/7PezjcO615qN0eCXXP1sF7nz vTx/fgvtOmvZp94XdNSRM1xcjZxs7ZpTZgoZFrGxUXzVqh2xJ043VS3NftCm5HxXkOM3 U1n8CmLfLWh6K/MisS8gcfbApyla+5Zi+WU9ybhSHX11BtNHmYBHrKgPU+7KYKa49xA0 Wvvw== X-Gm-Message-State: APjAAAXwM/0s0ekGKAu6EzNds0K2xaZ/Y42jux0LSpRUtKNk0/eYkRsK lLcNbSP4DHPHDKTzFK0Gs3XSvQ== X-Google-Smtp-Source: APXvYqwlLmqoJZ5GRKTAZypnHlB2wXwRwYwg3faz2aZh/pEhyM6PY/ubL2HWq2DHyyeRPPs13bhd2g== X-Received: by 2002:a17:902:b94a:: with SMTP id h10mr40582160pls.82.1554854170438; Tue, 09 Apr 2019 16:56:10 -0700 (PDT) Received: from www.outflux.net (173-164-112-133-Oregon.hfc.comcastbusiness.net. [173.164.112.133]) by smtp.gmail.com with ESMTPSA id g67sm52591223pfg.94.2019.04.09.16.56.07 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 09 Apr 2019 16:56:09 -0700 (PDT) From: Kees Cook To: Shuah Khan Cc: Kees Cook , linux-kselftest@vger.kernel.org Subject: [PATCH 6/6] selftests: Move test output to diagnostic lines Date: Tue, 9 Apr 2019 16:55:56 -0700 Message-Id: <20190409235556.3967-7-keescook@chromium.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190409235556.3967-1-keescook@chromium.org> References: <20190409235556.3967-1-keescook@chromium.org> Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This changes the selftest output so that each test's output is prefixed with "# " as a TAP "diagnostic line". This creates a bit of a kernel-specific TAP dialect where the diagnostics precede the results. The TAP spec isn't entirely clear about this, though, so I think it's the correct solution so as to keep interactive runs making sense. If the output _followed_ the result line in the spec-suggested YAML form, each test would dump all of its output at once instead of as it went, making debugging harder. This does, however, solve the recursive TAP output problem, as sub-tests will simply be prefixed by "# ". Parsing sub-tests becomes a simple problem of just removing the first two characters of a given top-level test's diagnostic output, and parsing the results. Note that the shell construct needed to both get an exit code from the first command in a pipe and still filter the pipe (to add the "# " prefix) uses a POSIX solution rather than the bash "pipefail" option which is not supported by dash. Since some test environments may have a very minimal set of utilities available, the new prefixing code will fall back to doing line-at-a-time prefixing if perl and/or stdbuf are not available. Signed-off-by: Kees Cook --- tools/testing/selftests/Makefile | 1 + tools/testing/selftests/kselftest/prefix.pl | 23 +++++++++++++ tools/testing/selftests/kselftest/runner.sh | 37 ++++++++++++++++++--- tools/testing/selftests/lib.mk | 1 + 4 files changed, 58 insertions(+), 4 deletions(-) create mode 100755 tools/testing/selftests/kselftest/prefix.pl diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index 72178d28f9c6..2e92a9727a4d 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -133,6 +133,7 @@ ifdef INSTALL_PATH @# Ask all targets to install their files mkdir -p $(INSTALL_PATH)/kselftest install -m 744 kselftest/runner.sh $(INSTALL_PATH)/kselftest/ + install -m 744 kselftest/prefix.pl $(INSTALL_PATH)/kselftest/ @for TARGET in $(TARGETS); do \ BUILD_TARGET=$$BUILD/$$TARGET; \ make OUTPUT=$$BUILD_TARGET -C $$TARGET INSTALL_PATH=$(INSTALL_PATH)/$$TARGET install; \ diff --git a/tools/testing/selftests/kselftest/prefix.pl b/tools/testing/selftests/kselftest/prefix.pl new file mode 100755 index 000000000000..ec7e48118183 --- /dev/null +++ b/tools/testing/selftests/kselftest/prefix.pl @@ -0,0 +1,23 @@ +#!/usr/bin/perl +# SPDX-License-Identifier: GPL-2.0 +# Prefix all lines with "# ", unbuffered. Command being piped in may need +# to have unbuffering forced with "stdbuf -i0 -o0 -e0 $cmd". +use strict; + +binmode STDIN; +binmode STDOUT; + +STDOUT->autoflush(1); + +my $needed = 1; +while (1) { + my $char; + my $bytes = sysread(STDIN, $char, 1); + exit 0 if ($bytes == 0); + if ($needed) { + print "# "; + $needed = 0; + } + print $char; + $needed = 1 if ($char eq "\n"); +} diff --git a/tools/testing/selftests/kselftest/runner.sh b/tools/testing/selftests/kselftest/runner.sh index 5793660dbe3f..f8b545a6e0e9 100644 --- a/tools/testing/selftests/kselftest/runner.sh +++ b/tools/testing/selftests/kselftest/runner.sh @@ -5,6 +5,34 @@ export KSFT_TAP_LEVEL=1 export skip_rc=4 export logfile=/dev/stdout +# There isn't a shell-agnostic way to find the path of a sourced file, +# so we must rely on BASE_DIR being set to find other tools. +if [ -z "$BASE_DIR" ]; then + echo "Error: BASE_DIR must be set before sourcing." >&2 + exit 1 +fi + +# If Perl is unavailable, we must fall back to line-at-a-time prefixing +# with sed instead of unbuffered output. +tap_prefix() +{ + if [ ! -x /usr/bin/perl ]; then + sed -e 's/^/# /' + else + "$BASE_DIR"/kselftest/prefix.pl + fi +} + +# If stdbuf is unavailable, we must fall back to line-at-a-time piping. +tap_unbuffer() +{ + if ! which stdbuf >/dev/null ; then + "$@" + else + stdbuf -i0 -o0 -e0 "$@" + fi +} + run_one() { DIR="$1" @@ -14,10 +42,9 @@ run_one() BASENAME_TEST=$(basename $TEST) TEST_HDR_MSG="selftests: $DIR: $BASENAME_TEST" - echo "$TEST_HDR_MSG" - echo "========================================" + echo "# $TEST_HDR_MSG" if [ ! -x "$TEST" ]; then - echo -n "$TEST_HDR_MSG: Warning: file $TEST is " + echo -n "# Warning: file $TEST is " if [ ! -e "$TEST" ]; then echo "missing!" else @@ -26,7 +53,9 @@ run_one() echo "not ok $test_num $TEST_HDR_MSG" else cd `dirname $TEST` > /dev/null - (./$BASENAME_TEST >> "$logfile" 2>&1 && + (((((tap_unbuffer ./$BASENAME_TEST 2>&1; echo $? >&3) | + tap_prefix >&4) 3>&1) | + (read xs; exit $xs)) 4>>"$logfile" && echo "ok $test_num $TEST_HDR_MSG") || (if [ $? -eq $skip_rc ]; then \ echo "not ok $test_num $TEST_HDR_MSG # SKIP" diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk index 8be13d117101..5c6505bec8be 100644 --- a/tools/testing/selftests/lib.mk +++ b/tools/testing/selftests/lib.mk @@ -33,6 +33,7 @@ endif .ONESHELL: define RUN_TESTS + BASE_DIR="$(selfdir)"; \ . $(selfdir)/kselftest/runner.sh; \ run_many $(1) endef