From patchwork Mon Jan 24 18:56:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Schindelin X-Patchwork-Id: 12722815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75C90C433FE for ; Mon, 24 Jan 2022 19:01:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344441AbiAXTBj (ORCPT ); Mon, 24 Jan 2022 14:01:39 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44172 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S245569AbiAXS7B (ORCPT ); Mon, 24 Jan 2022 13:59:01 -0500 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6E519C061252 for ; Mon, 24 Jan 2022 10:56:19 -0800 (PST) Received: by mail-wm1-x32d.google.com with SMTP id l35-20020a05600c1d2300b0034d477271c1so34187wms.3 for ; Mon, 24 Jan 2022 10:56:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=0+6ZofhIChFrhi0R37fNMY11dRkTRc5ZEYlAU1BlzXA=; b=hAYWloshxoO9iMvkry+wMyXe87nQayYaoMr/lNU7l3xEvoVpYP0vQJTbgRG0EUZOuy rn79m8/FDG1yILxx+B+YkM6FD0AbpkEwItGJoNjGOenAxNcU7ONZYoWi3W2TZVDjhypi XSQW1NJLVKdbtVWFUQfRS1nZiziqn52oH7NLWmWU6tOhtB4mXC7jRbp0VTb+Lxzgqeav kkne3WBPuOgDGjR00GJ+N3bPq6KvMSwk2JrTbunHesI0ET2Du/yL5de95MRy1ZiU/I2V QqG8q3hoQTwP5sLPxj2r/cZCVP7WaPG/WhDVOPSB566qNEBmeSWG/ijZgfu9USlkGBvz QEbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=0+6ZofhIChFrhi0R37fNMY11dRkTRc5ZEYlAU1BlzXA=; b=pVDTbcjB4HZAA9VA9n48QTjDQPi4rRRQbURoPUpN+nzMyCbcoCcHFf1d2EaKm5BsTU USVj7Cz4wwLWOn7+8uUgnAkPfEd2hzlO0C/9JiXq6shWMweHBjGy8W23+GUJ1SOJb7Qh oDdaiOaQZS+Erd+1a44HMsT3/sZaOY/LT7uohzoVQYDlcTFBQT97/nWR74O5Z7wgcjhk UFx2Nap/mv9AS9Xkz2wSSRhQBDsqK/wvw/sL4Al/P+zKiTst3EAppt5TQ9RnPcpz6uKL 50XV+1kpqmoZE8Djchyxhc/E1Z4nSPC9WjZuBYLrEeImieHzJY7Et7zaGrrIVpbQlAsA crfw== X-Gm-Message-State: AOAM531bHANiGSlYA7IHXwgv+QCKviGO/M60LeYA4ICFkOx9WsYbF7Av 34gSyEbI3BNv5fUC130Q5hV7XXogZUQ= X-Google-Smtp-Source: ABdhPJwYPeXcO9uUDMHvhP8bcZz0tesdHN/EfOuFMfivlq1ywsH2aY4t78un+n4kLnwrbx7WLCLxpA== X-Received: by 2002:a7b:c219:: with SMTP id x25mr3030756wmi.41.1643050577943; Mon, 24 Jan 2022 10:56:17 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id h1sm14463268wrq.82.2022.01.24.10.56.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 10:56:17 -0800 (PST) Message-Id: In-Reply-To: References: Date: Mon, 24 Jan 2022 18:56:06 +0000 Subject: [PATCH 1/9] ci: fix code style Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin From: Johannes Schindelin In b92cb86ea14 (travis-ci: check that all build artifacts are .gitignore-d, 2017-12-31), a function was introduced with a code style that is different from the surrounding code: it added the opening curly brace on its own line, when all the existing functions in the same file cuddle that brace on the same line as the function name. Let's make the code style consistent again. Signed-off-by: Johannes Schindelin --- ci/lib.sh | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index 9d28ab50fb4..ebb502640fa 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -69,8 +69,7 @@ skip_good_tree () { exit 0 } -check_unignored_build_artifacts () -{ +check_unignored_build_artifacts () { ! git ls-files --other --exclude-standard --error-unmatch \ -- ':/*' 2>/dev/null || { From patchwork Mon Jan 24 18:56:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Schindelin X-Patchwork-Id: 12722816 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC794C4332F for ; Mon, 24 Jan 2022 19:01:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344518AbiAXTBl (ORCPT ); Mon, 24 Jan 2022 14:01:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44728 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344775AbiAXS7M (ORCPT ); Mon, 24 Jan 2022 13:59:12 -0500 Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 20CE8C06125D for ; Mon, 24 Jan 2022 10:56:21 -0800 (PST) Received: by mail-wr1-x42c.google.com with SMTP id h10so460554wrb.5 for ; Mon, 24 Jan 2022 10:56:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=l5LgNAQ7EM+LvbeG09TiAtpeOQEXEidrSkjZl/ZTkBk=; b=f3gQtyYYJMC8G6F0UL0dLE092aLv9Jh9+kKDaWd7BdL9MvzDl7qWHXizgqNlZZLjOT IZJi8EK9CxriLBKCX9XPhOwOviHKzSS3j2xVGPplT3M1JN5wII7G58RVxL1HVTJdq9S6 Z7+68lQDvuxl1AJPcMADbq2s6WFb4mpxUf37E4wieZE6joBSFByDeIZzej9cBMNoX8uL xSnQI3eqf5OWm1bskql5x4UwWiKyv+VtpIBtmzH9ybLEmogfSrLslSJOJbN+DQHLWa3u yf1XA7rjqE42rAEh1/Oou0Eq6ZbnLoePfrERDHrIZAOnt+R3FJUinKeJg9g9HMeFhbTf 0upA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=l5LgNAQ7EM+LvbeG09TiAtpeOQEXEidrSkjZl/ZTkBk=; b=jQaW1h1wjKTYxLOGj14WjHlXqIknefZpLxAeUGimTn0k+RQ4I1bJhYmopx5014Gkls d1ZlD0MSdazBknFAv9SaJW3ipQpe9ucESMumG/rqyUoS6Vaz+8IqdXmqFnCGD2lSR8Xz AFf3rAbOeEFvUr5VAg2lxe9sHLLk8LOIsiujqVzFi8HGXZVOEBn/UX2OJyZjZtQ9GVON DBGjaBNQk5WOE2OF10xbTeO5zZSuez0mPsQ0MxqNkr2O5IxhSYFkQARgMvKUQlO/ePve ffro4hY9jH6hjlD10kPqCgpGIAKGG2mf2yfxWsXcYySSNN8g+O2r50JHM1QWDUm2bKsf I5ew== X-Gm-Message-State: AOAM531lR/wjBoBKKvnAtD7NfNUHkAGTJdkCw7ohOcCJIZzX/2dAly5j jHa/bXG50b99UPSLcgYD/12hT547stI= X-Google-Smtp-Source: ABdhPJxx0WVaGalkEAC3/ybAFBF99Kpuyzdf8FnwTPxAQWwJ7AFQ8Kv5VFPGAvRTRGTPsu412N+VnA== X-Received: by 2002:adf:cd0d:: with SMTP id w13mr15521270wrm.522.1643050579173; Mon, 24 Jan 2022 10:56:19 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id l13sm18956829wry.87.2022.01.24.10.56.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 10:56:18 -0800 (PST) Message-Id: In-Reply-To: References: Date: Mon, 24 Jan 2022 18:56:07 +0000 Subject: [PATCH 2/9] ci/run-build-and-tests: take a more high-level view Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin From: Johannes Schindelin In the web UI of GitHub workflows, failed runs are presented with the job step that failed auto-expanded. In the current setup, this is not helpful at all because that shows only the output of `prove`, which says which test failed, but not in what way. What would help understand the reader what went wrong is the verbose test output of the failed test. The logs of the failed runs do contain that verbose test output, but it is shown in the _next_ step (which is marked as succeeding, and is therefore _not_ auto-expanded). Anyone not intimately familiar with this would completely miss the verbose test output, being left mostly puzzled with the test failures. We are about to show the failed test cases' output in the _same_ step, so that the user has a much easier time to figure out what was going wrong. But first, we must partially revert the change that tried to improve the CI runs by combining the `Makefile` targets to build into a single `make` invocation. That might have sounded like a good idea at the time, but it does make it rather impossible for the CI script to determine whether the _build_ failed, or the _tests_. If the tests were run at all, that is. So let's go back to calling `make` for the build, and call `make test` separately so that we can easily detect that _that_ invocation failed, and react appropriately. Signed-off-by: Johannes Schindelin --- ci/run-build-and-tests.sh | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index 280dda7d285..b70373c172f 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -10,7 +10,7 @@ windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";; *) ln -s "$cache_dir/.prove" t/.prove;; esac -export MAKE_TARGETS="all test" +run_tests=t case "$jobname" in linux-gcc) @@ -41,14 +41,18 @@ pedantic) # Don't run the tests; we only care about whether Git can be # built. export DEVOPTS=pedantic - export MAKE_TARGETS=all + run_tests= ;; esac # Any new "test" targets should not go after this "make", but should # adjust $MAKE_TARGETS. Otherwise compilation-only targets above will # start running tests. -make $MAKE_TARGETS +make +if test -n "$run_tests" +then + make test +fi check_unignored_build_artifacts save_good_tree From patchwork Mon Jan 24 18:56:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Schindelin X-Patchwork-Id: 12722817 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9C44C4332F for ; Mon, 24 Jan 2022 19:01:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344616AbiAXTBn (ORCPT ); Mon, 24 Jan 2022 14:01:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344824AbiAXS7R (ORCPT ); Mon, 24 Jan 2022 13:59:17 -0500 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A175CC06127D for ; Mon, 24 Jan 2022 10:56:22 -0800 (PST) Received: by mail-wr1-x433.google.com with SMTP id u15so15390816wrt.3 for ; Mon, 24 Jan 2022 10:56:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=+WgQC6mVmYEOoBPfo1TF1x/EPmyp1B7i6UgGdnGKrBY=; b=ZasrQ6nIlEW0Rxf2mjciwrKV9O96bEoI7d69r9czDwpV1wqnQcvCqJUyd/QGbpropq 6DKFMjEDuwlb20N+nERkemHQZCBYL9ftAghTpxkJ5n9uDGxdxH2BbOq4ybzy3WwIEXhQ ATjjEP4jD3ha7iHnIMHpoYpGPNOTmO75FCymsdYSq3wSkPFiZ0HFLsEupEM4Ax/tr5v3 VG6KAUtZHnLlCRUKnODd4+r73hW9aXx/2NdK0J171QECqpQKAYkgoDp9Mah3wScJz8BL NNYRRjt2LlVvUXxXzqLpDGBH18O+YPU06cZclqu4SzN8BM3JoQMH+mH8THW7sE19yoVr vOEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=+WgQC6mVmYEOoBPfo1TF1x/EPmyp1B7i6UgGdnGKrBY=; b=epjWHhtksY0lfmo6JkPFjR3sIXlsSeHcAMHrwNKbrXRLF+8Sjm/ZiQQJa1y9LBb62B 1o6hVvoyTh/ci0DEuBsyn3/N37/XAOEv+re9eLpjW/jAuO9A+n9LFvEJZl7O9tuivtr5 Az0zaLbJ7DulJu+mizpwkIKTip7xEAWjpjPUGiV/XEIJH/z/b/pD0c7dql6O19ENECGm JAmW1BClRN10i9+aX+k5hXCLylHcKlIwlcwUOanOXGOZmcqjjZBHz2ab8Qm9h7a+Plue 3jD2ugRqQ0y3fDVOcJWBB073Cr2fKe1+uxXw6KaooO6r32pdKktnKC87oyf7+Jbdf1iu v+pQ== X-Gm-Message-State: AOAM533jMEC6g1p2k69kvS4/YSSMau8bABrcraMocBIVIc5whnosQVuJ 0/WXG2oCU/rXryH4c0wAzHPb5z51j+g= X-Google-Smtp-Source: ABdhPJyDvK8MP8SU67/Hlb3G2uv1B0s4Eg/KZprznQmMxRkuMzmN0799gdl3aNU45RHNThdjSmpXOA== X-Received: by 2002:a05:6000:1634:: with SMTP id v20mr15301031wrb.376.1643050581019; Mon, 24 Jan 2022 10:56:21 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id u3sm4383451wrs.55.2022.01.24.10.56.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 10:56:20 -0800 (PST) Message-Id: <98891b0d3f7927086350cc9523736ae2eb7c1ee2.1643050574.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 24 Jan 2022 18:56:08 +0000 Subject: [PATCH 3/9] ci: make it easier to find failed tests' logs in the GitHub workflow Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin From: Johannes Schindelin You currently have to know a lot of implementation details when investigating test failures in the CI runs. The first step is easy: the failed job is marked quite clearly, but when opening it, the failed step is expanded, which in our case is the one running `ci/run-build-and-tests.sh`. This step, most notably, only offers a high-level view of what went wrong: it prints the output of `prove` which merely tells the reader which test script failed. The actually interesting part is in the detailed log of said failed test script. But that log is shown in the CI run's step that runs `ci/print-test-failures.sh`. And that step is _not_ expanded in the web UI by default. Let's help the reader by showing the failed tests' detailed logs in the step that is expanded automatically, i.e. directly after the test suite failed. This also helps the situation where the _build_ failed and the `print-test-failures` step was executed under the assumption that the _test suite_ failed, and consequently failed to find any failed tests. An alternative way to implement this patch would be to source `ci/print-test-failures.sh` in the `handle_test_failures` function to show these logs. However, over the course of the next few commits, we want to introduce some grouping which would be harder to achieve that way (for example, we do want a leaner, and colored, preamble for each failed test script, and it would be trickier to accommodate the lack of nested groupings in GitHub workflows' output). Signed-off-by: Johannes Schindelin --- .github/workflows/main.yml | 12 ------------ ci/lib.sh | 23 +++++++++++++++++++++++ ci/run-build-and-tests.sh | 3 ++- ci/run-test-slice.sh | 3 ++- 4 files changed, 27 insertions(+), 14 deletions(-) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index c35200defb9..3fa88b78b6d 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -119,10 +119,6 @@ jobs: - name: test shell: bash run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10 - - name: ci/print-test-failures.sh - if: failure() - shell: bash - run: ci/print-test-failures.sh - name: Upload failed tests' directories if: failure() && env.FAILED_TEST_ARTIFACTS != '' uses: actions/upload-artifact@v2 @@ -204,10 +200,6 @@ jobs: env: NO_SVN_TESTS: 1 run: . /etc/profile && ci/run-test-slice.sh ${{matrix.nr}} 10 - - name: ci/print-test-failures.sh - if: failure() - shell: bash - run: ci/print-test-failures.sh - name: Upload failed tests' directories if: failure() && env.FAILED_TEST_ARTIFACTS != '' uses: actions/upload-artifact@v2 @@ -261,8 +253,6 @@ jobs: - uses: actions/checkout@v2 - run: ci/install-dependencies.sh - run: ci/run-build-and-tests.sh - - run: ci/print-test-failures.sh - if: failure() - name: Upload failed tests' directories if: failure() && env.FAILED_TEST_ARTIFACTS != '' uses: actions/upload-artifact@v2 @@ -292,8 +282,6 @@ jobs: - uses: actions/checkout@v1 - run: ci/install-docker-dependencies.sh - run: ci/run-build-and-tests.sh - - run: ci/print-test-failures.sh - if: failure() - name: Upload failed tests' directories if: failure() && env.FAILED_TEST_ARTIFACTS != '' uses: actions/upload-artifact@v1 diff --git a/ci/lib.sh b/ci/lib.sh index ebb502640fa..2b2c0932320 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -78,6 +78,10 @@ check_unignored_build_artifacts () { } } +handle_failed_tests () { + return 1 +} + # GitHub Action doesn't set TERM, which is required by tput export TERM=${TERM:-dumb} @@ -123,6 +127,25 @@ then CI_JOB_ID="$GITHUB_RUN_ID" CC="${CC:-gcc}" DONT_SKIP_TAGS=t + handle_failed_tests () { + mkdir -p t/failed-test-artifacts + echo "FAILED_TEST_ARTIFACTS=t/failed-test-artifacts" >>$GITHUB_ENV + + for test_exit in t/test-results/*.exit + do + test 0 != "$(cat "$test_exit")" || continue + + test_name="${test_exit%.exit}" + test_name="${test_name##*/}" + printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n" + cat "t/test-results/$test_name.out" + + trash_dir="t/trash directory.$test_name" + cp "t/test-results/$test_name.out" t/failed-test-artifacts/ + tar czf t/failed-test-artifacts/"$test_name".trash.tar.gz "$trash_dir" + done + return 1 + } cache_dir="$HOME/none" diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index b70373c172f..e49f9eaa8c0 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -51,7 +51,8 @@ esac make if test -n "$run_tests" then - make test + make test || + handle_failed_tests fi check_unignored_build_artifacts diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index f8c2c3106a2..63358c23e11 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -12,6 +12,7 @@ esac make --quiet -C t T="$(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh | - tr '\n' ' ')" + tr '\n' ' ')" || +handle_failed_tests check_unignored_build_artifacts From patchwork Mon Jan 24 18:56:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Schindelin X-Patchwork-Id: 12722818 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B546C433F5 for ; Mon, 24 Jan 2022 19:01:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237045AbiAXTBq (ORCPT ); Mon, 24 Jan 2022 14:01:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44396 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345043AbiAXS7p (ORCPT ); Mon, 24 Jan 2022 13:59:45 -0500 Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 62C95C06118C for ; Mon, 24 Jan 2022 10:56:25 -0800 (PST) Received: by mail-wr1-x429.google.com with SMTP id r14so15388901wrp.2 for ; Mon, 24 Jan 2022 10:56:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=Mg3KhFwO9GAljunFpr+hdC3tt+UoQvn0pI9Ay70kHO8=; b=OFtyOUs8WSeIGsDXQqmzJtVmlPiEAqGAJcxcihJeSzP4gg/emKSZu+AGs3DCY6eja6 5BJSj7bVWau9REdyLxIR4ZPJfGeG3p7IbdQ1demJEzK/kc0hz2suy4YViT2IRfUZ1eGd TvgdzKBRVEpROShcJY89n+O7mX6vjjBKDDKZnK3aiFoz5MzGpvv16Xx4RAewegVgvbD+ Knx6hc7L72bcJBWaf2braXFccjMlQKxBSt/mDzdmG+zO4CXy6ISWuzLV5UP3tBB6ciAE /x8g0Mi++AY3DsjFt8paxVA7A9rMOLdbgFh3GzWZ9LGFLQlA1Fn2FJ2tjrQsBRLGmPPw 0TSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=Mg3KhFwO9GAljunFpr+hdC3tt+UoQvn0pI9Ay70kHO8=; b=2TepEaWeRYpg5MmYexMxsws7yjyVuAdMFRpDl0TGa6VsBsdAPUCTpskXEHvEUFTldg QrM+X3rOEkbJC8g1mbIgwoPp8wvG5T6dlvQEp3iqt8qo0sFA98LdNS4I+R2tIvk33d39 j1avVT/NAJFqBwYKuLEo7T9TUYUp+AyGHPKjMQimKFUjNYCEfeyB25W6p7GUqfoccUwU FcSKq4Y2ZWD2Xn1O5kzjkrpMbUvrvviViUWLuv6MmCKZeNsnPtT1itDOuuPxG9nljGjv OZhdA+qa22aCOCdPt++sFfy+wqb3NyUYovgqSwatlyBRdbUHrrQppJGW0rRgS4yyQgL8 O0xw== X-Gm-Message-State: AOAM530JsLlK50nn7+6DpuWx9YI2vAHMnpod4EfJOhicPobYzpWG0KCw LHTh9K8n2EoCfOULlHtIaHL/f5Wbw84= X-Google-Smtp-Source: ABdhPJx5lMvKZIo4SE1awHVEFJWjKNlg+EZkPQTRQwDVKc48Y1HS1FxwSQ5HPVguQAa0CrAStwN5TQ== X-Received: by 2002:adf:9dc9:: with SMTP id q9mr7821727wre.203.1643050583784; Mon, 24 Jan 2022 10:56:23 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n10sm122342wmr.25.2022.01.24.10.56.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 10:56:21 -0800 (PST) Message-Id: <9333ba781b8240f704e739b00d274f8c3d887e39.1643050574.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 24 Jan 2022 18:56:09 +0000 Subject: [PATCH 4/9] ci/run-build-and-tests: add some structure to the GitHub workflow output Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin From: Johannes Schindelin The current output of Git's GitHub workflow can be quite confusing, especially for contributors new to the project. To make it more helpful, let's introduce some collapsible grouping. Initially, readers will see the high-level view of what actually happened (did the build fail, or the test suite?). To drill down, the respective group can be expanded. Note: sadly, workflow output currently cannot contain any nested groups (see https://github.com/actions/runner/issues/802 for details), therefore we take pains to ensure to end any previous group before starting a new one. Signed-off-by: Johannes Schindelin --- ci/lib.sh | 55 ++++++++++++++++++++++++++++++++++----- ci/run-build-and-tests.sh | 4 +-- ci/run-test-slice.sh | 2 +- 3 files changed, 51 insertions(+), 10 deletions(-) diff --git a/ci/lib.sh b/ci/lib.sh index 2b2c0932320..4ed8f40ab02 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -1,5 +1,49 @@ # Library of functions shared by all CI scripts +if test true != "$GITHUB_ACTIONS" +then + begin_group () { :; } + end_group () { :; } + + group () { + shift + "$@" + } + set -x +else + begin_group () { + need_to_end_group=t + echo "::group::$1" >&2 + set -x + } + + end_group () { + test -n "$need_to_end_group" || return 0 + set +x + need_to_end_group= + echo '::endgroup::' >&2 + } + trap end_group EXIT + + group () { + set +x + begin_group "$1" + shift + "$@" + res=$? + end_group + return $res + } + + begin_group "CI setup" +fi + +# Set 'exit on error' for all CI scripts to let the caller know that +# something went wrong. +# Set tracing executed commands, primarily setting environment variables +# and installing dependencies. +set -e + skip_branch_tip_with_tag () { # Sometimes, a branch is pushed at the same time the tag that points # at the same commit as the tip of the branch is pushed, and building @@ -88,12 +132,6 @@ export TERM=${TERM:-dumb} # Clear MAKEFLAGS that may come from the outside world. export MAKEFLAGS= -# Set 'exit on error' for all CI scripts to let the caller know that -# something went wrong. -# Set tracing executed commands, primarily setting environment variables -# and installing dependencies. -set -ex - if test -n "$SYSTEM_COLLECTIONURI" || test -n "$SYSTEM_TASKDEFINITIONSURI" then CI_TYPE=azure-pipelines @@ -138,7 +176,7 @@ then test_name="${test_exit%.exit}" test_name="${test_name##*/}" printf "\\e[33m\\e[1m=== Failed test: ${test_name} ===\\e[m\\n" - cat "t/test-results/$test_name.out" + group "Failed test: $test_name" cat "t/test-results/$test_name.out" trash_dir="t/trash directory.$test_name" cp "t/test-results/$test_name.out" t/failed-test-artifacts/ @@ -234,3 +272,6 @@ linux-leaks) esac MAKEFLAGS="$MAKEFLAGS CC=${CC:-cc}" + +end_group +set -x diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index e49f9eaa8c0..5516f45f7fe 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -48,10 +48,10 @@ esac # Any new "test" targets should not go after this "make", but should # adjust $MAKE_TARGETS. Otherwise compilation-only targets above will # start running tests. -make +group Build make if test -n "$run_tests" then - make test || + group "Run tests" make test || handle_failed_tests fi check_unignored_build_artifacts diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index 63358c23e11..a3c67956a8d 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -10,7 +10,7 @@ windows*) cmd //c mklink //j t\\.prove "$(cygpath -aw "$cache_dir/.prove")";; *) ln -s "$cache_dir/.prove" t/.prove;; esac -make --quiet -C t T="$(cd t && +group "Run tests" make --quiet -C t T="$(cd t && ./helper/test-tool path-utils slice-tests "$1" "$2" t[0-9]*.sh | tr '\n' ' ')" || handle_failed_tests From patchwork Mon Jan 24 18:56:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Schindelin X-Patchwork-Id: 12722819 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5304C433F5 for ; Mon, 24 Jan 2022 19:01:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344661AbiAXTBt (ORCPT ); Mon, 24 Jan 2022 14:01:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45266 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345068AbiAXS7q (ORCPT ); Mon, 24 Jan 2022 13:59:46 -0500 Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0988BC061190 for ; Mon, 24 Jan 2022 10:56:27 -0800 (PST) Received: by mail-wr1-x42f.google.com with SMTP id r14so15389045wrp.2 for ; Mon, 24 Jan 2022 10:56:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=x3+QTZjSRvoWaKsczsulWNBMqebhwXtC5ksMNH2RWTM=; b=X9iwkbvURgzMUObixEj/bC/vqpKBaz4bNBOH1gUGrIduLse3fA9sEmeR0opruxQNB8 8S5fRnVIfphk2NMdMB+BrubuZsCwSAKi87Shk7msQ3BqhU2KmGrwjKOE8LZqFp4bZzDX 8hyXRnXyke2fgcp3Mn/gekSK55Ajn6bhtAXEzeWMKDBzhCgGBfhY3hV6ZDgpuLeiEOy9 wOSZxjVs4SC6NF3zYIIfQp/poAOT55vb6HIRssapPwrc1qR9aBXxTwtb7XONSfjsMKiu GpWJX3yIliFM/i9GzbMV1+z3E4HKPETDX5/Nju1JdP8sem7RkXR0Yzgxcfms90slPjp5 zk8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=x3+QTZjSRvoWaKsczsulWNBMqebhwXtC5ksMNH2RWTM=; b=OvqqokmA7cnuvAYlc07ZwUbDjO2L65VoP3UbkS2TnudOWVGmpm1KZ1BKRU8xh14f7V WQS89skSpw4Jke71m9Xs0J0FsJ/tulgwIcJv8NqS9TY1oLP5gR9Pmm1YtDHEfVNKfpoU dHwDnEKmtWtcngpSU+Rwrk27kzXzeSuu/lAQYSwH7dVLt/3p7CdQgrhGMJdBSy49d5Fm GDXFUHYtNJ+03XGOmiBvrSc46/tGbBi+rW6zLLtYxa/x3AHmxdbYgoRPM8hi1/r04oC6 cteOHqKBinRZl4eIqVYCTuBhzCQGHDibw5POEu4wJXFKEocW845aChiWAakPs2TWj3eB Tyew== X-Gm-Message-State: AOAM533RKIYaIVyoMhe2OewnebVvgjSfBWIRDzCNXeHWQbAe89JT8ea5 F8IuBqq5eFuC6zTKnLbiNQ0uIpQ9N38= X-Google-Smtp-Source: ABdhPJwoijrb8VJ4ETWOZlNNcwSO5Xu/ExJCcS89i/El0cO+vj/0IqPrOedBj2vvrHP4bHjp+2TVKQ== X-Received: by 2002:a05:6000:11c3:: with SMTP id i3mr15153417wrx.662.1643050585359; Mon, 24 Jan 2022 10:56:25 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id n14sm96477wmq.42.2022.01.24.10.56.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 10:56:24 -0800 (PST) Message-Id: <94dcbe1bc437dc9738cfc635e9085d743eb4ec98.1643050574.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 24 Jan 2022 18:56:10 +0000 Subject: [PATCH 5/9] tests: refactor --write-junit-xml code Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin From: Johannes Schindelin The code writing JUnit XML is interspersed directly with all the code in `t/test-lib.sh`, and it is therefore not only ill-separated, but introducing yet another output format would make the situation even worse. Let's introduce an abstraction layer by hiding the JUnit XML code behind four new functions that are supposed to be called before and after each test and test case. This is not just an academic exercise, refactoring for refactoring's sake. We _actually_ want to introduce such a new output format, to make it substantially easier to diagnose test failures in our GitHub workflow, therefore we do need this refactoring. This commit is best viewed with `git show --color-moved --color-moved-ws=allow-indentation-change `. Signed-off-by: Johannes Schindelin --- t/test-lib-junit.sh | 126 ++++++++++++++++++++++++++++++++++++++++++++ t/test-lib.sh | 124 ++++++------------------------------------- 2 files changed, 142 insertions(+), 108 deletions(-) create mode 100644 t/test-lib-junit.sh diff --git a/t/test-lib-junit.sh b/t/test-lib-junit.sh new file mode 100644 index 00000000000..9d55d74d764 --- /dev/null +++ b/t/test-lib-junit.sh @@ -0,0 +1,126 @@ +# Library of functions to format test scripts' output in JUnit XML +# format, to support Git's test suite result to be presented in an +# easily digestible way on Azure Pipelines. +# +# Copyright (c) 2022 Johannes Schindelin +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see http://www.gnu.org/licenses/ . +# +# The idea is for `test-lib.sh` to source this file when the user asks +# for JUnit XML; these functions will then override (empty) functions +# that are are called at the appropriate times during the test runs. + +start_test_output () { + junit_xml_dir="$TEST_OUTPUT_DIRECTORY/out" + mkdir -p "$junit_xml_dir" + junit_xml_base=${1##*/} + junit_xml_path="$junit_xml_dir/TEST-${junit_xml_base%.sh}.xml" + junit_attrs="name=\"${junit_xml_base%.sh}\"" + junit_attrs="$junit_attrs timestamp=\"$(TZ=UTC \ + date +%Y-%m-%dT%H:%M:%S)\"" + write_junit_xml --truncate "" " " + junit_suite_start=$(test-tool date getnanos) + if test -n "$GIT_TEST_TEE_OUTPUT_FILE" + then + GIT_TEST_TEE_OFFSET=0 + fi +} + +start_test_case_output () { + junit_start=$(test-tool date getnanos) +} + +finalize_test_case_output () { + test_case_result=$1 + shift + case "$test_case_result" in + ok) + set "$*" + ;; + failure) + junit_insert="" + junit_insert="$junit_insert $(xml_attr_encode \ + "$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE" + then + test-tool path-utils skip-n-bytes \ + "$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET + else + printf '%s\n' "$@" | sed 1d + fi)")" + junit_insert="$junit_insert" + if test -n "$GIT_TEST_TEE_OUTPUT_FILE" + then + junit_insert="$junit_insert$(xml_attr_encode \ + "$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")" + fi + set "$1" " $junit_insert" + ;; + fixed) + set "$* (breakage fixed)" + ;; + broken) + set "$* (known breakage)" + ;; + skip) + message="$(xml_attr_encode "$skipped_reason")" + set "$1" " " + ;; + esac + + junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\"" + shift + junit_attrs="$junit_attrs classname=\"$this_test\"" + junit_attrs="$junit_attrs time=\"$(test-tool \ + date getnanos $junit_start)\"" + write_junit_xml "$(printf '%s\n' \ + " " "$@" " ")" + junit_have_testcase=t +} + +finalize_test_output () { + if test -n "$junit_xml_path" + then + test -n "$junit_have_testcase" || { + junit_start=$(test-tool date getnanos) + write_junit_xml_testcase "all tests skipped" + } + + # adjust the overall time + junit_time=$(test-tool date getnanos $junit_suite_start) + sed -e "s/\(]*/& time=\"$junit_time\"/" \ + -e '/^ *<\/testsuite/d' \ + <"$junit_xml_path" >"$junit_xml_path.new" + mv "$junit_xml_path.new" "$junit_xml_path" + + write_junit_xml " " "" + write_junit_xml= + fi +} + +write_junit_xml () { + case "$1" in + --truncate) + >"$junit_xml_path" + junit_have_testcase= + shift + ;; + esac + printf '%s\n' "$@" >>"$junit_xml_path" +} + +xml_attr_encode () { + printf '%s\n' "$@" | test-tool xml-encode +} diff --git a/t/test-lib.sh b/t/test-lib.sh index 0f7a137c7d8..e13e1cb9124 100644 --- a/t/test-lib.sh +++ b/t/test-lib.sh @@ -107,6 +107,12 @@ mark_option_requires_arg () { store_arg_to=$2 } +# These functions can be overridden e.g. to output JUnit XML +start_test_output () { :; } +start_test_case_output () { :; } +finalize_test_case_output () { :; } +finalize_test_output () { :; } + parse_option () { local opt="$1" @@ -166,7 +172,7 @@ parse_option () { tee=t ;; --write-junit-xml) - write_junit_xml=t + . "$TEST_DIRECTORY/test-lib-junit.sh" ;; --stress) stress=t ;; @@ -613,7 +619,7 @@ exec 6<&0 exec 7>&2 _error_exit () { - finalize_junit_xml + finalize_test_output GIT_EXIT_OK=t exit 1 } @@ -723,35 +729,13 @@ trap '{ code=$?; set +x; } 2>/dev/null; exit $code' INT TERM HUP # the test_expect_* functions instead. test_ok_ () { - if test -n "$write_junit_xml" - then - write_junit_xml_testcase "$*" - fi + finalize_test_case_output ok "$@" test_success=$(($test_success + 1)) say_color "" "ok $test_count - $@" } test_failure_ () { - if test -n "$write_junit_xml" - then - junit_insert="" - junit_insert="$junit_insert $(xml_attr_encode \ - "$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE" - then - test-tool path-utils skip-n-bytes \ - "$GIT_TEST_TEE_OUTPUT_FILE" $GIT_TEST_TEE_OFFSET - else - printf '%s\n' "$@" | sed 1d - fi)")" - junit_insert="$junit_insert" - if test -n "$GIT_TEST_TEE_OUTPUT_FILE" - then - junit_insert="$junit_insert$(xml_attr_encode \ - "$(cat "$GIT_TEST_TEE_OUTPUT_FILE")")" - fi - write_junit_xml_testcase "$1" " $junit_insert" - fi + finalize_test_case_output failure "$@" test_failure=$(($test_failure + 1)) say_color error "not ok $test_count - $1" shift @@ -760,19 +744,13 @@ test_failure_ () { } test_known_broken_ok_ () { - if test -n "$write_junit_xml" - then - write_junit_xml_testcase "$* (breakage fixed)" - fi + finalize_test_case_output fixed "$@" test_fixed=$(($test_fixed+1)) say_color error "ok $test_count - $@ # TODO known breakage vanished" } test_known_broken_failure_ () { - if test -n "$write_junit_xml" - then - write_junit_xml_testcase "$* (known breakage)" - fi + finalize_test_case_output broken "$@" test_broken=$(($test_broken+1)) say_color warn "not ok $test_count - $@ # TODO known breakage" } @@ -1049,10 +1027,7 @@ test_start_ () { test_count=$(($test_count+1)) maybe_setup_verbose maybe_setup_valgrind - if test -n "$write_junit_xml" - then - junit_start=$(test-tool date getnanos) - fi + start_test_case_output } test_finish_ () { @@ -1103,12 +1078,7 @@ test_skip () { case "$to_skip" in t) - if test -n "$write_junit_xml" - then - message="$(xml_attr_encode "$skipped_reason")" - write_junit_xml_testcase "$1" \ - " " - fi + finalize_test_case_output skip "$@" say_color skip "ok $test_count # skip $1 ($skipped_reason)" : true @@ -1124,53 +1094,6 @@ test_at_end_hook_ () { : } -write_junit_xml () { - case "$1" in - --truncate) - >"$junit_xml_path" - junit_have_testcase= - shift - ;; - esac - printf '%s\n' "$@" >>"$junit_xml_path" -} - -xml_attr_encode () { - printf '%s\n' "$@" | test-tool xml-encode -} - -write_junit_xml_testcase () { - junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\"" - shift - junit_attrs="$junit_attrs classname=\"$this_test\"" - junit_attrs="$junit_attrs time=\"$(test-tool \ - date getnanos $junit_start)\"" - write_junit_xml "$(printf '%s\n' \ - " " "$@" " ")" - junit_have_testcase=t -} - -finalize_junit_xml () { - if test -n "$write_junit_xml" && test -n "$junit_xml_path" - then - test -n "$junit_have_testcase" || { - junit_start=$(test-tool date getnanos) - write_junit_xml_testcase "all tests skipped" - } - - # adjust the overall time - junit_time=$(test-tool date getnanos $junit_suite_start) - sed -e "s/\(]*/& time=\"$junit_time\"/" \ - -e '/^ *<\/testsuite/d' \ - <"$junit_xml_path" >"$junit_xml_path.new" - mv "$junit_xml_path.new" "$junit_xml_path" - - write_junit_xml " " "" - write_junit_xml= - fi -} - test_atexit_cleanup=: test_atexit_handler () { # In a succeeding test script 'test_atexit_handler' is invoked @@ -1193,7 +1116,7 @@ test_done () { # removed, so the commands can access pidfiles and socket files. test_atexit_handler - finalize_junit_xml + finalize_test_output if test -z "$HARNESS_ACTIVE" then @@ -1484,22 +1407,7 @@ fi # in subprocesses like git equals our $PWD (for pathname comparisons). cd -P "$TRASH_DIRECTORY" || exit 1 -if test -n "$write_junit_xml" -then - junit_xml_dir="$TEST_OUTPUT_DIRECTORY/out" - mkdir -p "$junit_xml_dir" - junit_xml_base=${0##*/} - junit_xml_path="$junit_xml_dir/TEST-${junit_xml_base%.sh}.xml" - junit_attrs="name=\"${junit_xml_base%.sh}\"" - junit_attrs="$junit_attrs timestamp=\"$(TZ=UTC \ - date +%Y-%m-%dT%H:%M:%S)\"" - write_junit_xml --truncate "" " " - junit_suite_start=$(test-tool date getnanos) - if test -n "$GIT_TEST_TEE_OUTPUT_FILE" - then - GIT_TEST_TEE_OFFSET=0 - fi -fi +start_test_output "$0" # Convenience # A regexp to match 5 and 35 hexdigits From patchwork Mon Jan 24 18:56:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Schindelin X-Patchwork-Id: 12722820 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3954FC433FE for ; Mon, 24 Jan 2022 19:01:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344109AbiAXTBy (ORCPT ); Mon, 24 Jan 2022 14:01:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45296 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345178AbiAXS7z (ORCPT ); Mon, 24 Jan 2022 13:59:55 -0500 Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30BDAC0619C0 for ; Mon, 24 Jan 2022 10:56:28 -0800 (PST) Received: by mail-wm1-x32c.google.com with SMTP id r9-20020a1c4409000000b0034e043aaac7so8640wma.5 for ; Mon, 24 Jan 2022 10:56:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=iGAE3SZGJU7eojvkhG0qydiDez1X+uPg8UYp38uGbF0=; b=eFy71cgRS3uc/7GsQl0qv9RgJ3d1MCMFd2ViMax0lwkRzikMIsMsiXpVDVPGyf00/z hYqgooPyWXjeECqf4e7qveBhCDBz75RSzEgDCcoWkgU5gAVAzqHDr0Hn59dzJqbAKxKp iM1axcOGkMM5uFydkmtDqYzdFqKiJFYkrfXnMEJ0PLJCk7sOZarl4N1lI3Pya1IFIRC1 zNbDDFZp7nWPAtpp9y1RndyzEFjdc2G0gN+O1G4nuGjsViYWXLoN3xRgfOxj0BnpUN3M f2gpjv6Yy0HpgGgQT+kO0CpZjrOfKqPU5tX8f3TgWl9sYbVKIZfUaycJYsi8p4Xvxgmc U6ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=iGAE3SZGJU7eojvkhG0qydiDez1X+uPg8UYp38uGbF0=; b=f5glT4QyX4FBpHZqXUTl9bEyceWNSbfDoFqp3iMFrxbsmWyhZ19uqOZh0f9dM39L66 cnMFrNUPPrSn+aQtUrPvLGJiGvYW8QfpbwlGE3e+v0O15iarSzoJQknUYRA/HS6a/Yfy 4ZnVAM/iFD+ZJxycVyf/0fdnYucR5idEnBT+ZrPPly0odtzoDuZeSsdQv843aDJNyK8I nuujaKzEalEe7SOqMgZE4y15hO9jq6KBJk3EK1yjCcmUXRBt/c2STi1Ganv2KpBEzL5p Z2vRBBRgdsmauFXyamhjgkJKsL9lcqzjGKMBDe5DN0qsquFOz37urnuvUgQhF+70+jeV KA+A== X-Gm-Message-State: AOAM531lsdQh7IV4r8PbrM0txYRZ9HIp5RaX0FezH4MHYWPmOYRoja8W aT8COGCuYPszFyrQGZ0j6jTh8vaJbwY= X-Google-Smtp-Source: ABdhPJzFYFVS54Le9iaDiyc/7zfZwVtlowmETklsQsHM1vePmdPqOaZmFhWH8PtNxzuE9jGmmH+MSQ== X-Received: by 2002:a1c:cc0f:: with SMTP id h15mr2969555wmb.38.1643050586534; Mon, 24 Jan 2022 10:56:26 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id j19sm175719wmq.17.2022.01.24.10.56.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 10:56:25 -0800 (PST) Message-Id: <41230100091e3c53fa87516c9db697ad385d5873.1643050574.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 24 Jan 2022 18:56:11 +0000 Subject: [PATCH 6/9] test(junit): avoid line feeds in XML attributes Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin From: Johannes Schindelin In the test case's output, we do want newline characters, but in the XML attributes we do not want them. However, the `xml_attr_encode` function always adds a Line Feed at the end (which are then encoded as ` `, even for XML attributes. This seems not to faze Azure Pipelines' XML parser, but it still is incorrect, so let's fix it. Signed-off-by: Johannes Schindelin --- t/test-lib-junit.sh | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/t/test-lib-junit.sh b/t/test-lib-junit.sh index 9d55d74d764..c959183c7e2 100644 --- a/t/test-lib-junit.sh +++ b/t/test-lib-junit.sh @@ -50,7 +50,7 @@ finalize_test_case_output () { ;; failure) junit_insert="" + junit_insert="$junit_insert $(xml_attr_encode --no-lf "$1")\">" junit_insert="$junit_insert $(xml_attr_encode \ "$(if test -n "$GIT_TEST_TEE_OUTPUT_FILE" then @@ -74,12 +74,12 @@ finalize_test_case_output () { set "$* (known breakage)" ;; skip) - message="$(xml_attr_encode "$skipped_reason")" + message="$(xml_attr_encode --no-lf "$skipped_reason")" set "$1" " " ;; esac - junit_attrs="name=\"$(xml_attr_encode "$this_test.$test_count $1")\"" + junit_attrs="name=\"$(xml_attr_encode --no-lf "$this_test.$test_count $1")\"" shift junit_attrs="$junit_attrs classname=\"$this_test\"" junit_attrs="$junit_attrs time=\"$(test-tool \ @@ -122,5 +122,11 @@ write_junit_xml () { } xml_attr_encode () { - printf '%s\n' "$@" | test-tool xml-encode + if test "x$1" = "x--no-lf" + then + shift + printf '%s' "$*" | test-tool xml-encode + else + printf '%s\n' "$@" | test-tool xml-encode + fi } From patchwork Mon Jan 24 18:56:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Schindelin X-Patchwork-Id: 12722823 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12808C433EF for ; Mon, 24 Jan 2022 19:02:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344415AbiAXTCN (ORCPT ); Mon, 24 Jan 2022 14:02:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1345173AbiAXS7z (ORCPT ); Mon, 24 Jan 2022 13:59:55 -0500 Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB5FFC0619C3 for ; Mon, 24 Jan 2022 10:56:28 -0800 (PST) Received: by mail-wm1-x331.google.com with SMTP id r2-20020a1c2b02000000b0034f7b261169so36646wmr.2 for ; Mon, 24 Jan 2022 10:56:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:in-reply-to:references:from:date:subject:fcc :content-transfer-encoding:mime-version:to:cc; bh=0cTVlvy3koO8kIFX46QMstD+ERk03HMw6aT8GNBxMTM=; b=JmgLikkhxHoG2J97PFaZvTa8lwGk+HZOHk3+eLiidHz5H3exheIWxEqaGEfxsFLyLw y/OXzwMakzZA2rQ25abMippB5QdvO/XXNyCz4OK1NvLuXuYO/X8ja3gtVxYN9zBZrJEZ fYEsEF9VOApXXR1L2br/DWdu7Z9+bKxrzCWPiABunuhJVI34pJJlSOqPZUYa7z9QFiRe LpUbPuFzsE1BghKbGpf4qAKFEtXpHSIA1YK3HBKo9OQNu64AU0N2Sc8XaVK6ETL4AlRq By9JkrhHiDTX4o6qabSZcpp7//i1r4Rl3T/p9kgswG38CYKhm9SeKBT5aO+QhMQ2TG8J L+Hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:in-reply-to:references:from:date :subject:fcc:content-transfer-encoding:mime-version:to:cc; bh=0cTVlvy3koO8kIFX46QMstD+ERk03HMw6aT8GNBxMTM=; b=AzYpexclwXEjoGzWT33Xim7hYSrkNPaQ3yPDnsNA9taHLtQ6Kx5iHqgCKqUO7W6Kqd Xnt/GDrPI2C5FSMorVg+kPx6lcdRaKIS0UqIQhwctb47j2lwM803uW48E08EfthhBMw/ kT8WAf723XKnSQITnxDOi/zrNaFXXGDCMOkR/+xswPV36b4wchjTSmkUL4EVDYuXfRXW 3+mUJEGgJ9peQQ4MZlU2gF3P+TbbI/gCHow/GqMPmLcn58iULcNvZaKMPnyxZ5dgEwey 70s8Y8f+t7Y5/Z3Pi62KbwclFQpzcQpAxEj+6frRAJkx0Rf++3JK/PEY2asiQWb2gWXl T1Ig== X-Gm-Message-State: AOAM532BTdObHwVhY4RcwMwpx63ChAVf6zOM+/GU4mtA8kHq3Wd43hsg Z5sCZObk9A74wYsf9WrhYGscPJe1qe0= X-Google-Smtp-Source: ABdhPJx/Aw8pJG5RXwEp5rg4uJ/2xZ8Um4Kz00QrjsCDM6AhkO8h0W/o/GqvwlEh6YNYWovjn3C2Rg== X-Received: by 2002:a1c:448a:: with SMTP id r132mr362978wma.11.1643050587358; Mon, 24 Jan 2022 10:56:27 -0800 (PST) Received: from [127.0.0.1] ([13.74.141.28]) by smtp.gmail.com with ESMTPSA id a6sm7044804wrx.101.2022.01.24.10.56.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Jan 2022 10:56:26 -0800 (PST) Message-Id: <98b32630fcd76d591ce64f19314f3081134f541e.1643050574.git.gitgitgadget@gmail.com> In-Reply-To: References: Date: Mon, 24 Jan 2022 18:56:12 +0000 Subject: [PATCH 7/9] ci: optionally mark up output in the GitHub workflow Fcc: Sent MIME-Version: 1.0 To: git@vger.kernel.org Cc: Johannes Schindelin , Johannes Schindelin Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org From: Johannes Schindelin From: Johannes Schindelin A couple of commands exist to spruce up the output in GitHub workflows: https://docs.github.com/en/actions/learn-github-actions/workflow-commands-for-github-actions In addition to the `::group::