From patchwork Mon May 6 19:57:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13655981 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E835915B0E2 for ; Mon, 6 May 2024 19:57:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715025463; cv=none; b=ED/yitWNyDGOUBN1oxOoDuameLlnUP86Trkx4IKmgNL3XEmN8nalVZHLPeMuGxjAyeI3YTA68hSNPKph8vzhB8KXXSiE3k3vIjy3mS3MnLaL/aqS6cktLWMxhL6bOLQw4cVhaEjQmdJubdeJVq+qO+mChms45q3Qtas482+zLPE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715025463; c=relaxed/simple; bh=o62CKTIvI1Bf3LeBdOeVxugbTn71QMnHEOdZQlI6UVk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=kMR5BiqmiXiibpaqZeDmh+CEmpkJsJfRidk3Pyfg1IsfnRpD3zq4unsOFaIcAhbQVd6jNZm0Dv9Az+GMVsnXwjbMzMbFwqfCrd5bm13nNBxrGeMfRCGy1Ug4n3/gwHwx7TORaeONwWhYhp6uUv9xIMNtIYWck7DLRTRKAyZ2kpk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=18fnC6JU; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="18fnC6JU" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-61e0c296333so58573597b3.1 for ; Mon, 06 May 2024 12:57:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715025461; x=1715630261; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=BDw6O8iYt3nuF7enVVC/N8T20/aisT3WXkf8IlWKiCE=; b=18fnC6JUm3dExRtj1GjJFzV8Lvm+mRx5uGGTD/tVhswunTwrRVSZ61HJSkXG4fHiwp lxVmL73fD8l10C4oFd3Y8BDE6MRoue1TODi1NU8lgvrd12cUDrk6I+itxkf9gjSGSVCg NHDyNJNiAHzCewJ0T/xxUSYDK7MIiJgpboyAKD91TzhEj1Fe9egSSV7fLdJdi81lVZck /YRSqAozCzd8InOFEMa/x30zB7laUrR05rwRG+y4FRTL+zt11HXaOYzmNkAkVszPpPAt b9AOMJojuE8bld061wecjkXn2PnjM8VapWDm3q6SIGwDZqOElFvIrG00DtUOGh93UyCJ CswQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715025461; x=1715630261; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=BDw6O8iYt3nuF7enVVC/N8T20/aisT3WXkf8IlWKiCE=; b=o1BBDicY0hjVXHWxCgyhn22EGCvGSOI7pfxIzFYAbkdyZax/Qae9UNFxuZ4i4NAsw2 xzo5qRha8MABTqCYR8/iCPBuJnuBg2b3Opo4I+pLHtamUHJYoVcVUw2Afm8t89SBx+I1 isFws0jbkriRkmOMGtzybpKLFJyY1+A6C6Dth58tBOK5AxAEF24rvzK2u3kZwscMwq9+ qmzbWZSCXmPL4qpkfU9kvH8FQvpUcYhfr0G2mO/glr+UXjYIit3HU7iT4pCQT839CBDE QgboFxMGm6wSktLztwJIKVzFABShJxzTQ8Jbwn71Ypl7HTdr0FRM1gAVwpn3IX7xEPvo gBxQ== X-Gm-Message-State: AOJu0YzQA+vfMkdxCdHmRdgTVq1nvgTLVTYL2SYFUWtmHGUw5madHs0Y gxujNzy5paOZJGrBcoTKI60gVH1Gb5x62bRSI+W5Mxjw7nGvJ+A1zNViHm+6bbuJb7HXztSAuBB PoPDCvG1xRpeWkDRCq98MDLy/6o1pM57n3I37kW4RIp7jxPGdA1rWeYPHx+v9yJEqs3OJ99/1X6 ISxltQk8ktfRZnGYShUsa9Um60Zg4KNS6KYdK/qJk= X-Google-Smtp-Source: AGHT+IGDqPLDv7yUeSxrpCQ3rz88TTRk5JXL4wwTERmwJEHBCeguZX8ExCtgrN/Eh09m2iju2v8q+bsxMDqDRA== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:cf3e:6b4e:994:dc81]) (user=steadmon job=sendgmr) by 2002:a0d:ea04:0:b0:618:5009:cb71 with SMTP id t4-20020a0dea04000000b006185009cb71mr156090ywe.5.1715025460840; Mon, 06 May 2024 12:57:40 -0700 (PDT) Date: Mon, 6 May 2024 12:57:31 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.45.0.rc1.225.g2a3ae87e7f-goog Message-ID: <677745110078501e611faf3d1facd1518e0453a2.1715024899.git.steadmon@google.com> Subject: [PATCH v6 1/7] t0080: turn t-basic unit test into a helper From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net While t/unit-tests/t-basic.c uses the unit-test framework added in e137fe3b29 (unit tests: add TAP unit test framework, 2023-11-09), it is not a true unit test in that it intentionally fails in order to exercise various codepaths in the unit-test framework. Thus, we intentionally exclude it when running unit tests through the various t/Makefile targets. Instead, it is executed by t0080-unit-test-output.sh, which verifies its output follows the TAP format expected for the various pass, skip, or fail cases. As such, it makes more sense for t-basic to be a helper item for t0080-unit-test-output.sh, so let's move it to t/helper/test-example-tap.c and adjust Makefiles as necessary. Signed-off-by: Josh Steadmon --- Makefile | 4 ++-- t/Makefile | 2 +- .../t-basic.c => helper/test-example-tap.c} | 5 ++-- t/helper/test-tool.c | 1 + t/helper/test-tool.h | 1 + t/t0080-unit-test-output.sh | 24 +++++++++---------- 6 files changed, 20 insertions(+), 17 deletions(-) rename t/{unit-tests/t-basic.c => helper/test-example-tap.c} (95%) diff --git a/Makefile b/Makefile index 23723367b8..ba55d817ee 100644 --- a/Makefile +++ b/Makefile @@ -802,6 +802,7 @@ TEST_BUILTINS_OBJS += test-dump-split-index.o TEST_BUILTINS_OBJS += test-dump-untracked-cache.o TEST_BUILTINS_OBJS += test-env-helper.o TEST_BUILTINS_OBJS += test-example-decorate.o +TEST_BUILTINS_OBJS += test-example-tap.o TEST_BUILTINS_OBJS += test-find-pack.o TEST_BUILTINS_OBJS += test-fsmonitor-client.o TEST_BUILTINS_OBJS += test-genrandom.o @@ -1338,7 +1339,6 @@ THIRD_PARTY_SOURCES += compat/regex/% THIRD_PARTY_SOURCES += sha1collisiondetection/% THIRD_PARTY_SOURCES += sha1dc/% -UNIT_TEST_PROGRAMS += t-basic UNIT_TEST_PROGRAMS += t-mem-pool UNIT_TEST_PROGRAMS += t-strbuf UNIT_TEST_PROGRAMS += t-ctype @@ -3218,7 +3218,7 @@ perf: all .PRECIOUS: $(TEST_OBJS) -t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) +t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) $(UNIT_TEST_DIR)/test-lib.o t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) $(REFTABLE_TEST_LIB) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(filter %.a,$^) $(LIBS) diff --git a/t/Makefile b/t/Makefile index 2d95046f26..4861edafe6 100644 --- a/t/Makefile +++ b/t/Makefile @@ -48,7 +48,7 @@ CHAINLINTTESTS = $(sort $(patsubst chainlint/%.test,%,$(wildcard chainlint/*.tes CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl UNIT_TEST_SOURCES = $(wildcard unit-tests/t-*.c) UNIT_TEST_PROGRAMS = $(patsubst unit-tests/%.c,unit-tests/bin/%$(X),$(UNIT_TEST_SOURCES)) -UNIT_TESTS = $(sort $(filter-out unit-tests/bin/t-basic%,$(UNIT_TEST_PROGRAMS))) +UNIT_TESTS = $(sort $(UNIT_TEST_PROGRAMS)) # `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`) # checks all tests in all scripts via a single invocation, so tell individual diff --git a/t/unit-tests/t-basic.c b/t/helper/test-example-tap.c similarity index 95% rename from t/unit-tests/t-basic.c rename to t/helper/test-example-tap.c index fda1ae59a6..d072ad559f 100644 --- a/t/unit-tests/t-basic.c +++ b/t/helper/test-example-tap.c @@ -1,4 +1,5 @@ -#include "test-lib.h" +#include "test-tool.h" +#include "t/unit-tests/test-lib.h" /* * The purpose of this "unit test" is to verify a few invariants of the unit @@ -69,7 +70,7 @@ static void t_empty(void) ; /* empty */ } -int cmd_main(int argc, const char **argv) +int cmd__example_tap(int argc, const char **argv) { test_res = TEST(check_res = check_int(1, ==, 1), "passing test"); TEST(t_res(1), "passing test and assertion return 1"); diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c index 33b9501c21..bb5c04c9c0 100644 --- a/t/helper/test-tool.c +++ b/t/helper/test-tool.c @@ -29,6 +29,7 @@ static struct test_cmd cmds[] = { { "dump-untracked-cache", cmd__dump_untracked_cache }, { "env-helper", cmd__env_helper }, { "example-decorate", cmd__example_decorate }, + { "example-tap", cmd__example_tap }, { "find-pack", cmd__find_pack }, { "fsmonitor-client", cmd__fsmonitor_client }, { "genrandom", cmd__genrandom }, diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h index b72f07ded9..38001bd1c6 100644 --- a/t/helper/test-tool.h +++ b/t/helper/test-tool.h @@ -23,6 +23,7 @@ int cmd__dump_untracked_cache(int argc, const char **argv); int cmd__dump_reftable(int argc, const char **argv); int cmd__env_helper(int argc, const char **argv); int cmd__example_decorate(int argc, const char **argv); +int cmd__example_tap(int argc, const char **argv); int cmd__find_pack(int argc, const char **argv); int cmd__fsmonitor_client(int argc, const char **argv); int cmd__genrandom(int argc, const char **argv); diff --git a/t/t0080-unit-test-output.sh b/t/t0080-unit-test-output.sh index 961b54b06c..83b1e3b7f5 100755 --- a/t/t0080-unit-test-output.sh +++ b/t/t0080-unit-test-output.sh @@ -8,50 +8,50 @@ test_expect_success 'TAP output from unit tests' ' cat >expect <<-EOF && ok 1 - passing test ok 2 - passing test and assertion return 1 - # check "1 == 2" failed at t/unit-tests/t-basic.c:76 + # check "1 == 2" failed at t/helper/test-example-tap.c:77 # left: 1 # right: 2 not ok 3 - failing test ok 4 - failing test and assertion return 0 not ok 5 - passing TEST_TODO() # TODO ok 6 - passing TEST_TODO() returns 1 - # todo check ${SQ}check(x)${SQ} succeeded at t/unit-tests/t-basic.c:25 + # todo check ${SQ}check(x)${SQ} succeeded at t/helper/test-example-tap.c:26 not ok 7 - failing TEST_TODO() ok 8 - failing TEST_TODO() returns 0 - # check "0" failed at t/unit-tests/t-basic.c:30 + # check "0" failed at t/helper/test-example-tap.c:31 # skipping test - missing prerequisite - # skipping check ${SQ}1${SQ} at t/unit-tests/t-basic.c:32 + # skipping check ${SQ}1${SQ} at t/helper/test-example-tap.c:33 ok 9 - test_skip() # SKIP ok 10 - skipped test returns 1 # skipping test - missing prerequisite ok 11 - test_skip() inside TEST_TODO() # SKIP ok 12 - test_skip() inside TEST_TODO() returns 1 - # check "0" failed at t/unit-tests/t-basic.c:48 + # check "0" failed at t/helper/test-example-tap.c:49 not ok 13 - TEST_TODO() after failing check ok 14 - TEST_TODO() after failing check returns 0 - # check "0" failed at t/unit-tests/t-basic.c:56 + # check "0" failed at t/helper/test-example-tap.c:57 not ok 15 - failing check after TEST_TODO() ok 16 - failing check after TEST_TODO() returns 0 - # check "!strcmp("\thello\\\\", "there\"\n")" failed at t/unit-tests/t-basic.c:61 + # check "!strcmp("\thello\\\\", "there\"\n")" failed at t/helper/test-example-tap.c:62 # left: "\011hello\\\\" # right: "there\"\012" - # check "!strcmp("NULL", NULL)" failed at t/unit-tests/t-basic.c:62 + # check "!strcmp("NULL", NULL)" failed at t/helper/test-example-tap.c:63 # left: "NULL" # right: NULL - # check "${SQ}a${SQ} == ${SQ}\n${SQ}" failed at t/unit-tests/t-basic.c:63 + # check "${SQ}a${SQ} == ${SQ}\n${SQ}" failed at t/helper/test-example-tap.c:64 # left: ${SQ}a${SQ} # right: ${SQ}\012${SQ} - # check "${SQ}\\\\${SQ} == ${SQ}\\${SQ}${SQ}" failed at t/unit-tests/t-basic.c:64 + # check "${SQ}\\\\${SQ} == ${SQ}\\${SQ}${SQ}" failed at t/helper/test-example-tap.c:65 # left: ${SQ}\\\\${SQ} # right: ${SQ}\\${SQ}${SQ} not ok 17 - messages from failing string and char comparison - # BUG: test has no checks at t/unit-tests/t-basic.c:91 + # BUG: test has no checks at t/helper/test-example-tap.c:92 not ok 18 - test with no checks ok 19 - test with no checks returns 0 1..19 EOF - ! "$GIT_BUILD_DIR"/t/unit-tests/bin/t-basic >actual && + ! test-tool example-tap >actual && test_cmp expect actual ' From patchwork Mon May 6 19:57:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13655982 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9248015ADB9 for ; Mon, 6 May 2024 19:57:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715025465; cv=none; b=IqsW3sE54mFFgYDEJ3dEq+dphbnlWee+adWr1/QHvfWZ5GkFbOb3elwVWx+1x1TeE0xfXIoP9NXcNYUcQKS8Lv30gMjR1BcvoZiMb5hhIE0aj2cSDOZPuAN+scp+gK7YwOIdHL7WOXenHSr3rNWiiCjGZHoIw8ysS30wiB1iIbE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715025465; c=relaxed/simple; bh=6hfv9dJ3mVPGavbF2drQpmNg/FsQobE4oNEmIyL6aEw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=uPaFRxJERMIuAufIvob3BCBFkf+VGsxNPAMV269DW/7ai3oeQ2S23Lo1SXYnKzp4nPyBmCiaOFwGg61INO5C6RO9eXfwNT11BMfNeMoZEcHuZQtoe/G84CQi45aYga4WJD3V2gN55LlTgeytbbcSoVuT+anoEHOtiutpPw3arnU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=a2BaarAu; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="a2BaarAu" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-de51daf900cso4309297276.1 for ; Mon, 06 May 2024 12:57:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715025462; x=1715630262; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=d3Lfr1Wdvxj4xlm3iGGkW/blqhNIFqjiFyFUhXWUFOY=; b=a2BaarAuYF5TEi3WR9a7EWg/S9y5m/83kqBvF5F4AndXMVSEqW3IsLJyrruWzE/whf kWhnYZJNO26xw3SWNQCgmliEmgX5HIxE1uFuCauzp6NB1ta6cMNykfcvnA1oDedi7/Jh 4MkAsflQrLOdxx27+TVQlxYMNTRQozDbHlM0PGsBVaCiX7suRqWID/IJtEM8C5T0bd/p 1rsh4ahjUJLXep7+nNeDZ/H4cBj6y0a04btqhCSGwodZ0Dh6qZihRpVk4FQCDc0v6xf7 xgDkaejqJ09nRq65o3QkhVw4YXNsNleyhAPddM6cX6PMss32fYCKujwcnTyY3FbO10vj pDBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715025462; x=1715630262; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=d3Lfr1Wdvxj4xlm3iGGkW/blqhNIFqjiFyFUhXWUFOY=; b=kzhV/L/cENN0EyOz4NnMG7oJW0yoIRs4+MJ1PI+wsR57AHEsjCKOBT7U8tscCXBjM7 QOA8sgYx2lUj7rJvY2oXzTt3/tCoA8k8kawRCn3epND2B9oGXp5K0dniWzcVvd9Z8TYY FxFcnyyNUXT0L/D6VX7NRPzd6jgYGKGY6xpVfdESDw5XPJgL+tDXUdel19CS9Rh9L/KG vWyPmWQewkAsll7UOkVDH78GFv0NkBiZ2KFDm5Z3f6eBxWRq4MatLvzbBOflhZammhxD 0okSJNEaKFIMsZJ5NdVXgpz44Mh6nZKJL4qUzu4A/lwPmwf5QRHR5eMkUYgZqMZQl73h R6FQ== X-Gm-Message-State: AOJu0YzztqPKaBPBv7w3so0JnmmU+u2F/G6NwzA5FHyh8vqx8z5cpyW5 tl349quDv8oh8rwT7doGuJM0zDAuXXl1uvEFqHnZNqTXHE/1dpGplAPpJ+mvK38gXMQYQL2ai61 /QiQNRNOUfKRo7LZdhXcsK+ehxQXQih25c/OomLBcxYu5HuubMSCXKQgEq2HA5CGX5g7WOpkJ2Z afrX0fiQ/hJf2eq66Aq8qD83GVoA8fP1GptoPzsmo= X-Google-Smtp-Source: AGHT+IEnD/6+3ZdwTmoDLBEX4eFdQabOsZyWxEmxB1vQ/RCTM/a1f1N6jAxBLg4wZA101gwa553hVimZ+mjNag== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:cf3e:6b4e:994:dc81]) (user=steadmon job=sendgmr) by 2002:a05:6902:1505:b0:dcc:6065:2b3d with SMTP id q5-20020a056902150500b00dcc60652b3dmr3602540ybu.8.1715025462598; Mon, 06 May 2024 12:57:42 -0700 (PDT) Date: Mon, 6 May 2024 12:57:32 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.45.0.rc1.225.g2a3ae87e7f-goog Message-ID: <24f47f8fc7e931c2e15fa985635e3036ec1123ba.1715024899.git.steadmon@google.com> Subject: [PATCH v6 2/7] test-tool run-command testsuite: get shell from env From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net When running tests through `test-tool run-command testsuite`, we currently hardcode `sh` as the command interpreter. As discussed in [1], this is incorrect, and we should be using the shell set in TEST_SHELL_PATH instead. Add a shell_path field in struct testsuite so that we can pass this to the task runner callback. If this is non-null, we'll use it as the argv[0] of the subprocess. Otherwise, we'll just execute the test program directly. We will use this feature in a later commit to enable running binary executable unit tests. However, for now when setting up the struct testsuite in testsuite(), use the value of TEST_SHELL_PATH if it's set, otherwise keep the original behavior by defaulting to `sh`. [1] https://lore.kernel.org/git/20240123005913.GB835964@coredump.intra.peff.net/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index c0ed8722c8..a41a54d9cb 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -65,6 +65,7 @@ struct testsuite { struct string_list tests, failed; int next; int quiet, immediate, verbose, verbose_log, trace, write_junit_xml; + const char *shell_path; }; #define TESTSUITE_INIT { \ .tests = STRING_LIST_INIT_DUP, \ @@ -80,7 +81,9 @@ static int next_test(struct child_process *cp, struct strbuf *err, void *cb, return 0; test = suite->tests.items[suite->next++].string; - strvec_pushl(&cp->args, "sh", test, NULL); + if (suite->shell_path) + strvec_push(&cp->args, suite->shell_path); + strvec_push(&cp->args, test); if (suite->quiet) strvec_push(&cp->args, "--quiet"); if (suite->immediate) @@ -162,6 +165,10 @@ static int testsuite(int argc, const char **argv) if (max_jobs <= 0) max_jobs = online_cpus(); + suite.shell_path = getenv("TEST_SHELL_PATH"); + if (!suite.shell_path) + suite.shell_path = "sh"; + dir = opendir("."); if (!dir) die("Could not open the current directory"); From patchwork Mon May 6 19:57:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13655983 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7357E15B10A for ; Mon, 6 May 2024 19:57:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715025466; cv=none; b=mcAjCLvOcJZoWpXm7/lGHVECsvbyXVZTkuPTGqrkh4IcfSWz6t5TJoXGMNk4+vBs3GRjtlaoWuqoTI7VmgnRXUJ3sjv9I1NJWGkZTwQ9pmJF/lQCRfiB+Rd1+7W7x/VBzrxI+tCFX/hd23G++jJaWT31BxRtdwgVWWiVNKAUgQ8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715025466; c=relaxed/simple; bh=X/iVPdS8BI9Q3kNKSfBuXD+4g6EyVJeXX91CTqEoRpc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=gXOyrZPaPhoWSsthgMJkyEW15lf/kQgotRKThVL5LKzIyCTlyGhMM6Vd9BMSZllsaOFM6c8GX2M1KDvunF7spdy4oWy5R8YY21CvpU+usQsOwdt9rFLVTo1OjoGL/os+yJanJtaTD5tdT2m7gS4INH55eRFmeAAX3FjZ5tSoV0M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=J6MZOa0Y; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="J6MZOa0Y" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-61be8f7ed6eso29679707b3.0 for ; Mon, 06 May 2024 12:57:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715025464; x=1715630264; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/0MsO1dJdkmzETo3Vb/aHe95tnWVsUlLJnBhvpvKhaE=; b=J6MZOa0YuaUz/wa860SviwDUatwcfGiqVCix/IUg8NZN6eWguwSN4S6NYifgioMbOb /g+ipbSzKInkJhh5bLFD9Agkw2d/ZcGuleFVtdPde6FLDJEuL1ndlMI6JoN1L3dbqKmx 1SvVIarKcvwfC9bySgdxV3ic/Ie1e47EN2xu3Xuafp7Hj451LWAREMMdUd09c+6ZUjyE 540gPZag/VRXcUG5lDCbY59sNvDLmXRwpISvqyMCNyt4YCJku4LOOu8Ax6eBLDleDyIi /fZN0W+AwFIJERgV3plEZ4/xTx+2Uqwcs4gs9Y99UrHQKk+aefb9HRkrBbmG9/QXM+CY kGzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715025464; x=1715630264; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/0MsO1dJdkmzETo3Vb/aHe95tnWVsUlLJnBhvpvKhaE=; b=dHH70mL6XEP8vIr1DIWgicQESNDTBcRIoxsu/z2AfE1y6A21m6UHWANpROeNDmu2V/ XQ6Lvnn2L6fRvshKrCk+vj2Qf/DCOMyjrIStkzfcuPMFeBI/pfHMiGq84jcal8441hRT ddSORA+1Jj4Z/rs4aNizJHwyx6bSvT3RDXfAGbj+5xaRYypCsyyDxh2m2jegBHDyjJfW oSW+8LXT27xMjUSPg9e6N0QSDok82nVWyX9lBWUbEY8OnG6biSuhRdDLal45RyjrD0Gk 6f+C6gzeC+qjAZLv30M6kQdSMsLW3aGlGhzv0ZdMsudScKTrbsCrc08XguP4K9UGO+8T X2Jw== X-Gm-Message-State: AOJu0YwAgeUmb6iNxbfFfAOjWM6/CvLzG58TWoMO2GcQJdsO91HpHp9v 92y9C0Awb1CvXa0ZYN+QzW0HaFnznRuLu9o5wz3MkmeJePZJ6GG3iNQCUW4KEnrl+y6F7svzq+p GWIKevWAawzOzaZeIyMduFQtwCkI/adWZ6ods1r3EWNbPp9aKItOr8m4VMUu13m04MHzCqUibWc uQ2cqHPjktYulj0YKheks7xpQDaPKR5s/VNl6a3vQ= X-Google-Smtp-Source: AGHT+IFwhA1dDNcXZ2+xdWRWIkM4DlElaeEg3ySaHagKtQrfBdcDvT1Tb7sDpoR2qP2RuF2dgJ1/whFWBJkXhg== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:cf3e:6b4e:994:dc81]) (user=steadmon job=sendgmr) by 2002:a0d:ca4a:0:b0:620:32ea:e1d4 with SMTP id m71-20020a0dca4a000000b0062032eae1d4mr165541ywd.0.1715025464393; Mon, 06 May 2024 12:57:44 -0700 (PDT) Date: Mon, 6 May 2024 12:57:33 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.45.0.rc1.225.g2a3ae87e7f-goog Message-ID: <4a16a3ec241d83b3de424ce835db24eee6e46f0f.1715024899.git.steadmon@google.com> Subject: [PATCH v6 3/7] test-tool run-command testsuite: remove hardcoded filter From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net `test-tool run-command testsuite` currently assumes that it will only be running the shell test suite, and therefore filters out anything that does not match a hardcoded pattern of "t[0-9][0-9][0-9][0-9]-*.sh". Later in this series, we'll adapt `test-tool run-command testsuite` to also support unit tests, which do not follow the same naming conventions as the shell tests, so this hardcoded pattern is inconvenient. Since `testsuite` also allows specifying patterns on the command-line, let's just remove this pattern. As noted in [1], there are no longer any uses of `testsuite` in our codebase, it should be OK to break backwards compatibility in this case. We also add a new filter to avoid trying to execute "." and "..", so that users who wish to execute every test in a directory can do so without specifying a pattern. [1] https://lore.kernel.org/git/850ea42c-f103-68d5-896b-9120e2628686@gmx.de/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index a41a54d9cb..e6bd792274 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -175,9 +175,7 @@ static int testsuite(int argc, const char **argv) while ((d = readdir(dir))) { const char *p = d->d_name; - if (*p != 't' || !isdigit(p[1]) || !isdigit(p[2]) || - !isdigit(p[3]) || !isdigit(p[4]) || p[5] != '-' || - !ends_with(p, ".sh")) + if (!strcmp(p, ".") || !strcmp(p, "..")) continue; /* No pattern: match all */ From patchwork Mon May 6 19:57:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13655984 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E2FF15B11C for ; Mon, 6 May 2024 19:57:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715025468; cv=none; b=IeMPs+6aFWZnCQSi9g2P20GwCcg88yYGYNnwN002zB0cLXVOCftHFI4Fbg1pg8XOXwEQsacR8sIerHjX6lTuXPNCkkJ2Y6X46OOiE7n9APL+4ZPVY1sWCgPgFYw14hHFasiBjeDnXKBKbwNs/0iimMZ2d2CoUYG32xyBf1VAJM8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715025468; c=relaxed/simple; bh=eQNBRzpwi9Q80F5M2qevhCm5xqKSdviDYAvMTxBeg9E=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=XJ1Fwr3Tpj9DH0aVwxiOWUmV2GCNbX34HA9SpGQ3t8dXdyFZKf0lZ7KCzUH69o3ns+uLNUoJiNPiYPiDgEE3zptd3h0YuD1uQQYLIdE/i20gDE0bN6ylOqQJAtCCuzL6LTNbH9RJYhy8AR9saIHXldjPQ25EemP5LNLkaoDegt4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lNLDa21M; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lNLDa21M" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-de603db5d6aso5294138276.2 for ; Mon, 06 May 2024 12:57:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715025466; x=1715630266; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=MbAr47xneUW6J1uzpOQp3hKuBIpPm/xQmnkQPf/MfVY=; b=lNLDa21MLNsZkiPpvTUc/zpDuKs1rXyqhYgQ/Ygkfai/01fJS6te7NrStCVZD1QtqA WEO3qgfhOZTNTb5GgI5lBeU2FKArcwWSfXseatdjZVfLnJDKFWIfas8C71zQWgPnqVhi zaBM9pQyKvqihuxMEWRHbA0D+G0sOttBwIrHs6uSgoyaM6tXZwSkm4j1NL672To28RpR WLqJvs9lSwjKFMY9dFEVdLT9AoZvsLJP7lYNHsNk42S4difksAQs+EWe7uEP9EwrNClF rjAEdWKHp2+6JJbMDY1uWh55CSnsLcb8Ja+AVC1FOO1H48QP3xkegQ3toWIB2ZrefyqV 9W3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715025466; x=1715630266; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=MbAr47xneUW6J1uzpOQp3hKuBIpPm/xQmnkQPf/MfVY=; b=vQC3GKqfxXJYoPuXjqRUQ3fpTu8Gm/01raqa5XUoXYZgFSjlUsenlZ0PT5KVNklbLu KIOhYZtb+11UnlPc+AnNIeD1hYn1T5vesYjO5JQZsTWhviFIoDPVj1UwUPB0QidLzYAt QOQlQSedJkbDHe4y4NvOBTgqlRZeXXzZ8esttrdvRlt9VwS/DV8D9kKpFPEXXbPmUP9Q O0QIaRB+F0seM5JG00kAjojbFBwrYKVIZhR7Ypimh6k+fGeZKdJuBeIXbsbxrnKvq2d4 3OssjR+jL6mq8MbcIYUtqpv0iJ1muuIE0bXpyzK0usgWpozxTqzut4osMKYhXfV9KWK1 6Osw== X-Gm-Message-State: AOJu0YwbNWhOxnoCnbwbvwbe/e2XkDQZA+zzLr6FHebBHDibc4fUDQrU VzhLPX+AM14JKXywZC9xM4lgQAsIA2Bphw69g1QMPD0LPdyMkZtUz1yeWosscoXHzE23SwNGi7a zUPAPS4ahE7v0HuQVmSVW163mC90LL0RwyOfoe++zRYYVAl8IPKkD1U2J+ZytTDOWCvMmafyMbd R94pQH96i2AeGN1BKCSSPi+eVXsuJsjjLl1adHmps= X-Google-Smtp-Source: AGHT+IGOMOWvkUac69NLnnkJj54d0L6f+bBC+OSyhxEpIWCiIp7Q1JKDqxmOMN1dtEZT+w+WkSic7lHGQsswYw== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:cf3e:6b4e:994:dc81]) (user=steadmon job=sendgmr) by 2002:a05:6902:c05:b0:de5:2694:45ba with SMTP id fs5-20020a0569020c0500b00de5269445bamr3791232ybb.0.1715025466162; Mon, 06 May 2024 12:57:46 -0700 (PDT) Date: Mon, 6 May 2024 12:57:34 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.45.0.rc1.225.g2a3ae87e7f-goog Message-ID: Subject: [PATCH v6 4/7] test-tool run-command testsuite: support unit tests From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net Teach the testsuite runner in `test-tool run-command testsuite` how to run unit tests: if TEST_SHELL_PATH is not set, run the programs directly from CWD, rather than defaulting to "sh" as an interpreter. With this change, you can now use test-tool to run the unit tests: $ make $ cd t/unit-tests/bin $ ../../helper/test-tool run-command testsuite This should be helpful on Windows to allow running tests without requiring Perl (for `prove`), as discussed in [1] and [2]. This again breaks backwards compatibility, as it is now required to set TEST_SHELL_PATH properly for executing shell scripts, but again, as noted in [2], there are no longer any such invocations in our codebase. [1] https://lore.kernel.org/git/nycvar.QRO.7.76.6.2109091323150.59@tvgsbejvaqbjf.bet/ [2] https://lore.kernel.org/git/850ea42c-f103-68d5-896b-9120e2628686@gmx.de/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index e6bd792274..61eb1175fe 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -158,6 +158,8 @@ static int testsuite(int argc, const char **argv) .task_finished = test_finished, .data = &suite, }; + struct strbuf progpath = STRBUF_INIT; + size_t path_prefix_len; argc = parse_options(argc, argv, NULL, options, testsuite_usage, PARSE_OPT_STOP_AT_NON_OPTION); @@ -165,9 +167,13 @@ static int testsuite(int argc, const char **argv) if (max_jobs <= 0) max_jobs = online_cpus(); + /* + * If we run without a shell, execute the programs directly from CWD. + */ suite.shell_path = getenv("TEST_SHELL_PATH"); if (!suite.shell_path) - suite.shell_path = "sh"; + strbuf_addstr(&progpath, "./"); + path_prefix_len = progpath.len; dir = opendir("."); if (!dir) @@ -180,13 +186,17 @@ static int testsuite(int argc, const char **argv) /* No pattern: match all */ if (!argc) { - string_list_append(&suite.tests, p); + strbuf_setlen(&progpath, path_prefix_len); + strbuf_addstr(&progpath, p); + string_list_append(&suite.tests, progpath.buf); continue; } for (i = 0; i < argc; i++) if (!wildmatch(argv[i], p, 0)) { - string_list_append(&suite.tests, p); + strbuf_setlen(&progpath, path_prefix_len); + strbuf_addstr(&progpath, p); + string_list_append(&suite.tests, progpath.buf); break; } } @@ -213,6 +223,7 @@ static int testsuite(int argc, const char **argv) string_list_clear(&suite.tests, 0); string_list_clear(&suite.failed, 0); + strbuf_release(&progpath); return ret; } From patchwork Mon May 6 19:57:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13655985 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0121F15B10A for ; Mon, 6 May 2024 19:57:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715025470; cv=none; b=P0nbngdaz9dLh7HCJkYYjIxgfQ0dCwMAAN8lFvVTfd6QGJO4w3ISDXPYEzQjZhGJFR0ISrRZAv/Ox8hSuxJPwKkzMzp4WJxGAb3JrieYI+ECbapLaJZL+ZQNUM/21UMbN4FyJrTQIOOaYDMHc1kC0DfWOwZQzCVoB/l114o7bOM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715025470; c=relaxed/simple; bh=vvkdPxgBTLNm9rYCJNcB2ylQeVVO/xBvfqaKfqvx1c0=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=DTc9OCUcj1YuTxlBcVZIItj9jYPGUaYTRpFrtq5OqOXCd8KxHPvVN6AVJlXiNzq73inyDMCcbkJKEXuLzG3zbTFxnahWQknvVnWYC0IHpVzZP0EXo+7WgSWiKNCO/Dm5BjobR/Hn1PXhwWxbu3PVgZLPK1tnfVZ332tpZasEB+4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=hDNQXBMm; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="hDNQXBMm" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-61e0c296333so58574497b3.1 for ; Mon, 06 May 2024 12:57:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715025468; x=1715630268; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Te4JAQhYWNwMg09FSYQHk0fhr/4qReTBh2xzBw5qAyU=; b=hDNQXBMmYGXjL6CVDiCh+HeJSCr7f8qM2+Eis6uhO1M816DYoG6XZEyi/CtL4x3ycP MO/51qvlbGxbR5gcAvcEJHmEcKjR98mCm5IzKzD9kPRSarImOBOnuCMd/GqMVUHhU4SU 9ajU9hx6UkJ0P+K3BlWhfk/7gK0BPUddVoN7vTDGVXUqoSK6tWExhKcpEvkaRAplQ8XE RPNFxwyvAHCWuyAdW3vdvimxzT4uJo3noizFpPzv7iBZL1pmIgKAdR5DUbtxiyXylb5t vs7WJrY3DEMyE4JyCRU22OOl067cSn6r2of0wJXvvl201YgakmM4KZP0DCKHWD3H1J1T tYZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715025468; x=1715630268; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Te4JAQhYWNwMg09FSYQHk0fhr/4qReTBh2xzBw5qAyU=; b=QGE3YVklWuJfWQGnE2bvjfZhMtg12i3K5/sazkUharOw435KFn0oaISZRu1GJpL5w4 OQFpxoRulWf8vlcGUzj3kloOVsQqaSXr/N0S3UbRtdZhcnkQJW/GaeNiMX+GD5wlvn/6 08B0yqIjasNskWluQFoc98y9bePQtG3ceE/6C8bHF3zTZkUUkeNxBFWjArSUuD2LuyS9 uV0cVvZ8iV7l1oTvNcBB2w5WF4Gzod/r/sEPbTeoyy7u7gLTH8Y7ySU45Bt6X1GwSKcm vLXFFQvRCIgPGZL9TrPKva4eoON+ROgvh0nuGGs+bM+Z9ouq6ROxN+6n9wx6V2s79nYE fflA== X-Gm-Message-State: AOJu0YydT6pwaQcst6xR+dPij/XkkJoJ9Ntp9POAVfJu/UHgeyNJTEsD LS+MCRhbRRp+Djji3hWY03Y1/BW/i/kIMDrvNlaMDFioTrLDJqABbUjx5Wh4WN8/4VJWY/28db2 fOWMJ8+tLNRj+59rMlsO+/9ECZGbh/swchSrRP0Y541qDYXOFPC/VzWT2V5rhwKYO4XYxLKVEG0 RI3k+wVtA/VdSR/qKaNKphi61YUtzYKMeJSC5ETX8= X-Google-Smtp-Source: AGHT+IGeq9GLwbZmTWtGoxVnz/gjplzMlg0dHuLiEalodAOwaQGXpwcJCaUHhRSirA6B3P7K917Gm3HCmrCgfA== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:cf3e:6b4e:994:dc81]) (user=steadmon job=sendgmr) by 2002:a0d:ea04:0:b0:618:5009:cb71 with SMTP id t4-20020a0dea04000000b006185009cb71mr156143ywe.5.1715025467997; Mon, 06 May 2024 12:57:47 -0700 (PDT) Date: Mon, 6 May 2024 12:57:35 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.45.0.rc1.225.g2a3ae87e7f-goog Message-ID: Subject: [PATCH v6 5/7] unit tests: add rule for running with test-tool From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net In the previous commit, we added support in test-tool for running collections of unit tests. Now, add rules in t/Makefile for running in this way. This new rule can be executed from the top-level Makefile via `make DEFAULT_UNIT_TEST_TARGET=unit-tests-test-tool unit-tests`, or by setting DEFAULT_UNIT_TEST_TARGET in config.mak. Signed-off-by: Josh Steadmon --- Makefile | 2 +- t/Makefile | 10 +++++++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index ba55d817ee..b0d1f04b4d 100644 --- a/Makefile +++ b/Makefile @@ -3870,5 +3870,5 @@ $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o $(UNIT_TEST_DIR)/ .PHONY: build-unit-tests unit-tests build-unit-tests: $(UNIT_TEST_PROGS) -unit-tests: $(UNIT_TEST_PROGS) +unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X $(MAKE) -C t/ unit-tests diff --git a/t/Makefile b/t/Makefile index 4861edafe6..0ae04f1e42 100644 --- a/t/Makefile +++ b/t/Makefile @@ -49,6 +49,7 @@ CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl UNIT_TEST_SOURCES = $(wildcard unit-tests/t-*.c) UNIT_TEST_PROGRAMS = $(patsubst unit-tests/%.c,unit-tests/bin/%$(X),$(UNIT_TEST_SOURCES)) UNIT_TESTS = $(sort $(UNIT_TEST_PROGRAMS)) +UNIT_TESTS_NO_DIR = $(notdir $(UNIT_TESTS)) # `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`) # checks all tests in all scripts via a single invocation, so tell individual @@ -76,7 +77,7 @@ $(T): $(UNIT_TESTS): @echo "*** $@ ***"; $@ -.PHONY: unit-tests unit-tests-raw unit-tests-prove +.PHONY: unit-tests unit-tests-raw unit-tests-prove unit-tests-test-tool unit-tests: $(DEFAULT_UNIT_TEST_TARGET) unit-tests-raw: $(UNIT_TESTS) @@ -84,6 +85,13 @@ unit-tests-raw: $(UNIT_TESTS) unit-tests-prove: @echo "*** prove - unit tests ***"; $(PROVE) $(GIT_PROVE_OPTS) $(UNIT_TESTS) +unit-tests-test-tool: + @echo "*** test-tool - unit tests **" + ( \ + cd unit-tests/bin && \ + ../../helper/test-tool$X run-command testsuite $(UNIT_TESTS_NO_DIR)\ + ) + pre-clean: $(RM) -r '$(TEST_RESULTS_DIRECTORY_SQ)' From patchwork Mon May 6 19:57:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13655986 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B69515B135 for ; Mon, 6 May 2024 19:57:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715025472; cv=none; b=U4rIlfWq+11JymAnWO19Re6Ge5mEX6MLXewF06q71fP8l8dPP2teFOkyKdBxe/e0U3+rM1VY8/wJFWl0B7VcXhIsL0p0fkt6BFYkLLGDr8DUO0U0v6P1ueWazFhmPCDAMIspOO1xQiHh2tGOpl39p3l4t/sx/omujdpJM1Ogd/0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715025472; c=relaxed/simple; bh=7Hc2WJPOTOOO+0w9uaXRa8/G/OpOTTxAnZbRwBxQmEA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=f/U5CFdLcGbK4W7Ro2kNTgnVM28NNfD/JLSZPmygTsvt/iJzRIZGliRtdPYsjzvR5jXcbsWVYxGpV2J9LnAxY4xXDpPtc/h14odrIvbk06u8rnE69OpauCpuxkbaSx0jVRaAuPFQshwB1Zt02y+dDQTih0b+tQ664yRChk1INtg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=xY+Pft/k; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="xY+Pft/k" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-61bef0accddso59658617b3.0 for ; Mon, 06 May 2024 12:57:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715025470; x=1715630270; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=RU+AFoAaCv+5oI4Ce2i9HE/Kl2wGSJ9eLdx5U2U+nR0=; b=xY+Pft/kkpNyIPUeRzYFb5siJnE31fStDCtCUbZTw17dd8lnyb3ghUHilb67wmi5+o NyFIZ+/6WdFCmKqOMErej0WjwHzK2hQ6HmA734RbNmFY5JoHBEAipaGelUNkphFdo7fH u3AzHPVSBu9oYyB2HdROizXVMxfaDiEHy7AduW2hG/hdJ3MHbnoVl6p1W7tjd0HEtuMD t54+WOQxZmbKrhHRbu6+wFG2VQ4TkYWTcbqKInLMHQ/apkmW1hXBC82huohb1Ri8oqKn dFmZ66U0RqnjZ5SGUWycq2PPjSHLddDTKnvfkCibEzoLyip1tG45MZLkTqhyMrIoAlq2 7eqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715025470; x=1715630270; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=RU+AFoAaCv+5oI4Ce2i9HE/Kl2wGSJ9eLdx5U2U+nR0=; b=MNnkM5j6Qey5gdJNdqMXLC241EiKQALjTnWF7RkTpLfdjqwajSPdG9KQAdZdmRWbx6 taB+eEzaeZ/OZOR4oUDtTVX5xnkaxl0D2lVxGDqMsEpt3ajunIXSeixfYz6v2PPCxMGy I3EEn3uLmQxWycm1y4fAP6o/12U8wgYXcDf82ZpMPNlY8KrVv04uEw/bjZXfiX7z1IjW Mm7Q9vn4BXoID5JloKgIGdCCu4XFq9x7+ou2v+AYERagG1oa6h0rgYJS9JDQTl0ooTPV pwMgFrJUtd6VGxrlX9GC2AKfDjGAIHuUiE0LfnjB2DbvXDlK9/cokwrFSqIGIG6YOyAj 1y6Q== X-Gm-Message-State: AOJu0YxmxWL/ZpJOtObd/oqLdLPEq5VKFfFFEUnda9noQxD/8VhNrzy/ Lhtfs/W/r2KSLIQeQBhBzlZwtrQAsP+Pa4P7b5zL/5Tz6mB36dWLoB/BylscoohLLybIS+2Udzh cfiTUlmUn0rqE1AoHWUbCvqgVSXIPYwzAk1THdV+nwppILL74zCDj8oZG2SgC6rTITaC66Y8qe2 flomA3Z0LLKlDW7zDdVxxlwZLDNSrd2l1/brbKrTQ= X-Google-Smtp-Source: AGHT+IFNgSO1qeSnG3C6OUc3N3rISh6jZ+CREBw58pQDShr4sPVP3DirLEJERGDyBBTgpzS1X6+EZM516y/QLQ== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:cf3e:6b4e:994:dc81]) (user=steadmon job=sendgmr) by 2002:a0d:eb04:0:b0:61b:ebab:ce9b with SMTP id u4-20020a0deb04000000b0061bebabce9bmr2969493ywe.3.1715025469663; Mon, 06 May 2024 12:57:49 -0700 (PDT) Date: Mon, 6 May 2024 12:57:36 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.45.0.rc1.225.g2a3ae87e7f-goog Message-ID: Subject: [PATCH v6 6/7] t/Makefile: run unit tests alongside shell tests From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net From: Jeff King Add a wrapper script to allow `prove` to run both shell tests and unit tests from a single invocation. This avoids issues around running prove twice in CI, as discussed in [1]. Additionally, this moves the unit tests into the main dev workflow, so that errors can be spotted more quickly. Accordingly, we remove the separate unit tests step for Linux CI. (We leave the Windows CI unit-test step as-is, because the sharding scheme there involves selecting specific test files rather than running `make test`.) [1] https://lore.kernel.org/git/pull.1613.git.1699894837844.gitgitgadget@gmail.com/ Signed-off-by: Jeff King Signed-off-by: Josh Steadmon --- ci/run-build-and-tests.sh | 2 -- t/Makefile | 2 +- t/run-test.sh | 18 ++++++++++++++++++ 3 files changed, 19 insertions(+), 3 deletions(-) create mode 100755 t/run-test.sh diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index 7a1466b868..2528f25e31 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -50,8 +50,6 @@ if test -n "$run_tests" then group "Run tests" make test || handle_failed_tests - group "Run unit tests" \ - make DEFAULT_UNIT_TEST_TARGET=unit-tests-prove unit-tests fi check_unignored_build_artifacts diff --git a/t/Makefile b/t/Makefile index 0ae04f1e42..b2eb9f770b 100644 --- a/t/Makefile +++ b/t/Makefile @@ -68,7 +68,7 @@ failed: test -z "$$failed" || $(MAKE) $$failed prove: pre-clean check-chainlint $(TEST_LINT) - @echo "*** prove ***"; $(CHAINLINTSUPPRESS) $(PROVE) --exec '$(TEST_SHELL_PATH_SQ)' $(GIT_PROVE_OPTS) $(T) :: $(GIT_TEST_OPTS) + @echo "*** prove (shell & unit tests) ***"; $(CHAINLINTSUPPRESS) TEST_SHELL_PATH='$(TEST_SHELL_PATH_SQ)' $(PROVE) --exec ./run-test.sh $(GIT_PROVE_OPTS) $(T) $(UNIT_TESTS) :: $(GIT_TEST_OPTS) $(MAKE) clean-except-prove-cache $(T): diff --git a/t/run-test.sh b/t/run-test.sh new file mode 100755 index 0000000000..13c353b91b --- /dev/null +++ b/t/run-test.sh @@ -0,0 +1,18 @@ +#!/bin/sh + +# A simple wrapper to run shell tests via TEST_SHELL_PATH, +# or exec unit tests directly. + +case "$1" in +*.sh) + if test -z "${TEST_SHELL_PATH}" + then + echo >&2 "ERROR: TEST_SHELL_PATH is empty or not set" + exit 1 + fi + exec "${TEST_SHELL_PATH}" "$@" + ;; +*) + exec "$@" + ;; +esac From patchwork Mon May 6 19:57:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13655987 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 76DE515B0F5 for ; Mon, 6 May 2024 19:57:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715025473; cv=none; b=fTgCtBW1rtnM9wn6PnAZUlE4ESSAnUXOh7XitAHS8EcORYW/jDL5fq5GFfWFFEC2Ma9ENYF6ZqdRISiX4+Ksu9+F+yIRwwbQ1bMjWDo0S5/bu9tSBZZFJ+xzTOAP95kyU+Mjs79ZLnYgiS1YTulB0/9x6AU+EVBFGgSrYgK0nz8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715025473; c=relaxed/simple; bh=zetJarc9VvVfP5g8Pf++IhqS7O1bUyM8FX+xZ57IOeg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Gd2OIwbrU+AJtGW23UeOA9hw9LLj5XOtG+IoKDWcF5G0MzczMRh1vVOUOWZKehUu7F0Ptvp85coZXTFVcCXlXjE0VasMHO8IS2yXX/q8fPUv6cVDD1WYy0kC6KxRucYEs9EOWvqAblnHumHqMnuXC/5dlmGAuzTA3Ay3Rxv9TTM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=3xFx64xa; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="3xFx64xa" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-61c9e36888bso43630397b3.2 for ; Mon, 06 May 2024 12:57:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715025471; x=1715630271; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=eR9zye7G9mIUUIaJbjHlBf7VEXXzfcREVNaFQjnXpLA=; b=3xFx64xaY5+UxUDvaEbIPZsCCjcSv4m3fsnjYYWSvdyIOeEW0h5OPMc3cLw8r1D+aE TJ2nP8N6oCKPqZWiFVZgMIxAeVlKLWekZ7JV3mqUJWNLARBiBB2Id6cV0SdUptZzCdB6 g3/LJ+CyW4u+akHzuwmt5lg2InSmOhYIIhCLOinnyaQE9ZJYzCE8ni0tbWSgF5gCMWnZ wN4O19gzkm6X+Runr3woyZz1KWHEpsL3iJfrVwkCOFPsojnc85Q5KRLZE4FO7ZBqhgAn 4Wfqf1oX6fh3mRKYaG5G0bKtaP66PSZLvrevGxjC4DuufabHnScaWuAZCa35O142JOLR DPqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715025471; x=1715630271; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=eR9zye7G9mIUUIaJbjHlBf7VEXXzfcREVNaFQjnXpLA=; b=W6euwOdo1jK7wvtU17jI2WIlh23epF1VDmqEeYLSyaWUe5hDBZFD1WY3LBDRIQVnM7 6EEYo/Xp0OnwGT8TmxVsQ54OR7n1+PQpZNwbrAZgjwIGLiWp5nAtMgdn3Q9Is7Qih3ps pqxQFoSb27lkVNJ8u5DW1+BsSk7RGrO0nEjKnyvzMTcky1MSA+0BambZK1G2g5pt3LWx YMWW8dri+GI6qYNOwrC/00FO6JyFjGim37orMQ5OoEDNaWvpdEwrBLchSVoYRCwQuK+2 B1vkx4syz16bjxKHFq6QIS2gGXbRh0CTCartWcXl2RZe9c/x/IYOfGqik3KViHaHVb8R F9Rg== X-Gm-Message-State: AOJu0YwmwaHLFZZvTd1q7GJB8hfgtkSwdGY9QHeylw/NSnv8ZcYMnw/q fBQkCRulhyYsvbPTO7qUdbcD6g/+zgEEqtHrLBA+Xrh3p1axtwcHuhx0ywYQfDEAdUwmKxf1rB1 G4uR26dK8Grobu+wAqf1Z2ob0Mvpzo+wTQ5xOW9/NpQMfoJoL8r/INk3z+cfwcDV/7qCDJWI39B Bpc+RPF0se4JetZpvcxa6XRsL8ky5IBsF0ol08a/o= X-Google-Smtp-Source: AGHT+IHfbee9Z+uE/bbh76DLvPKX6Q7YBNmqIJsLuZ80Ei8XDkhc/5M3JlEVVPoFmDq1KXpAncui6ZG1BwZapg== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:cf3e:6b4e:994:dc81]) (user=steadmon job=sendgmr) by 2002:a0d:d857:0:b0:61b:e524:f91a with SMTP id a84-20020a0dd857000000b0061be524f91amr2555432ywe.10.1715025471629; Mon, 06 May 2024 12:57:51 -0700 (PDT) Date: Mon, 6 May 2024 12:57:37 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.45.0.rc1.225.g2a3ae87e7f-goog Message-ID: <2f5853a7fae7967ba959ae16c81c90b6be52a536.1715024899.git.steadmon@google.com> Subject: [PATCH v6 7/7] ci: use test-tool as unit test runner on Windows From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net Although the previous commit changed t/Makefile to run unit tests alongside shell tests, the Windows CI still needs a separate unit-tests step due to how the test sharding works. We want to avoid using `prove` as a test running on Windows due to performance issues [1], so use the new test-tool runner instead. [1] https://lore.kernel.org/git/850ea42c-f103-68d5-896b-9120e2628686@gmx.de/ Signed-off-by: Josh Steadmon --- ci/run-test-slice.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index ae8094382f..e167e646f7 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -17,7 +17,7 @@ handle_failed_tests # We only have one unit test at the moment, so run it in the first slice if [ "$1" == "0" ] ; then - group "Run unit tests" make --quiet -C t unit-tests-prove + group "Run unit tests" make --quiet -C t unit-tests-test-tool fi check_unignored_build_artifacts