From patchwork Wed Apr 24 19:14:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13642419 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 748DE2C1A2 for ; Wed, 24 Apr 2024 19:14:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713986060; cv=none; b=F3j/W5Oa6eJ0dPLFQJWKiDRSOsipdseVEzUrTGfiZ+QDre6D+97NiSjeQZGzF0v817lGzeTPYNxqHUucpedcPmBy4dSjZjfl9X6GbB3TOTq3KDACMqya+mZB4BbLvyWPfv5Kc+t95omG+RRcYv555zIYwTAf6vMqtroAqNoljdY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713986060; c=relaxed/simple; bh=abkWUpMoC0T+ghrLh4FiMZLr2+gH+0Kxtin45Yj6fPc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hSYNa2KFdUNkSHWsn6Sq0N9aaVcq79SI2oGUVFDxR4crJxKspDTDLqR3ulfG0P8BN5Cia2PpbsIeUJ5Wp6+mbGNooz8W0W+dEBHQS3PltCCG7iRJQmVIc95LdqNu+JZvA6JM3xBbqAr5HSX8ljjCWv7ve2JA2BTy1lgQ4Rw8XNk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=bll0AkuD; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="bll0AkuD" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6150dcdf83fso3186867b3.2 for ; Wed, 24 Apr 2024 12:14:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713986057; x=1714590857; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=SfeotoyLSYB/cVnv0MDF6dF+MaCPywMQ3wYNJf/YyWk=; b=bll0AkuDDqxK0O8GeSfHmcDLuaNi1RLO6oHSVIYvSwkFTP4R1qDLyfhdvTR5oM45uP KPyQ1NHJpTkPg2e4nagInQGSWQNABPrzje0SfQixHYHH905080TWGHXxN8IARJ286Onu LjDgeH5BWj4rKTZ+oszSHQdw4nTcWOzyQ5u3/hPP8RkDLIDz11HZqJ/uQ/gtJXgaiqDC JmjbR4ot3tpXskBcN8qiw4NHnIcHfUi1pN/2/CBOofGExkNKSzRsscwPfOnO2BNBrtCt JfHxf1mNFnGfdxliCi0Z9jm3kv6+H/w4gXnWPkIkPyk7zOXpctIndSD/3PuVi2qLQeov NcRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713986057; x=1714590857; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=SfeotoyLSYB/cVnv0MDF6dF+MaCPywMQ3wYNJf/YyWk=; b=wi5yRITcLoEItI/whfgfD9vD5qGYYhqJ7v1xgWaeYf5Zbq3LcddTDqJz/lc8iYb6ys jvgxfVJmRoNaBVTT8i1bwQZuK4WfnaSC5OpMcrd85eAaq4wkLslamNKlJyatQ+ID58vs hq4Rr0HYqKUYcCR4aMocIOX5VHIFeI659Lg3K3IFk7tBP3WiUc92npIyDfEVinS7OhRV d39hGuko9Ooe6axUiBbSWHNwgq1YMGP3ikh3mC9kalnG6pi8MmDvOegpf5lNrxglNg7Q BLg7zVkL7dVzmkwv5AC2S7TquH8lv8ksL7+3vOtL1LhAG06vLTsvlij5sUAaPRX+JtpB wpBQ== X-Gm-Message-State: AOJu0YzAC7bvQ2gCMEXJm/dTx1YKfhcPtw8iw7KkXFVP2oCmyj1ElHZa i8y2XSTmNIUoYFyjWyZtKlBPBfR8HczsoBmx6WUTmzGYflaHhTHDM/G3kkFU+R5/PMn/hk7PUJg TMny0LGdhwSVJv0sMu/qhAZjqZOWP7BRJ/nAP1jdksXNodWilFgvxGryhlO4Z8UguI3IPoHGGHS SyC0ThH85q8D/8u1qUqKLIXW+M8Zxo9yl0h5WLWq8= X-Google-Smtp-Source: AGHT+IEOm61RnVaRdKY3fpnVywXadgD8TQq4THez310bDQM+j4a/wjjNpli7BI8V4Mu2a6042mShRWTtBrTnxw== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:5580:46bd:98b:ecbb]) (user=steadmon job=sendgmr) by 2002:a0d:db92:0:b0:618:80c0:1a2e with SMTP id d140-20020a0ddb92000000b0061880c01a2emr745986ywe.6.1713986057383; Wed, 24 Apr 2024 12:14:17 -0700 (PDT) Date: Wed, 24 Apr 2024 12:14:07 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.44.0.769.g3c40516874-goog Message-ID: <677745110078501e611faf3d1facd1518e0453a2.1713985716.git.steadmon@google.com> Subject: [PATCH v4 1/7] t0080: turn t-basic unit test into a helper From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net While t/unit-tests/t-basic.c uses the unit-test framework added in e137fe3b29 (unit tests: add TAP unit test framework, 2023-11-09), it is not a true unit test in that it intentionally fails in order to exercise various codepaths in the unit-test framework. Thus, we intentionally exclude it when running unit tests through the various t/Makefile targets. Instead, it is executed by t0080-unit-test-output.sh, which verifies its output follows the TAP format expected for the various pass, skip, or fail cases. As such, it makes more sense for t-basic to be a helper item for t0080-unit-test-output.sh, so let's move it to t/helper/test-example-tap.c and adjust Makefiles as necessary. Signed-off-by: Josh Steadmon --- Makefile | 4 ++-- t/Makefile | 2 +- .../t-basic.c => helper/test-example-tap.c} | 5 ++-- t/helper/test-tool.c | 1 + t/helper/test-tool.h | 1 + t/t0080-unit-test-output.sh | 24 +++++++++---------- 6 files changed, 20 insertions(+), 17 deletions(-) rename t/{unit-tests/t-basic.c => helper/test-example-tap.c} (95%) diff --git a/Makefile b/Makefile index 23723367b8..ba55d817ee 100644 --- a/Makefile +++ b/Makefile @@ -802,6 +802,7 @@ TEST_BUILTINS_OBJS += test-dump-split-index.o TEST_BUILTINS_OBJS += test-dump-untracked-cache.o TEST_BUILTINS_OBJS += test-env-helper.o TEST_BUILTINS_OBJS += test-example-decorate.o +TEST_BUILTINS_OBJS += test-example-tap.o TEST_BUILTINS_OBJS += test-find-pack.o TEST_BUILTINS_OBJS += test-fsmonitor-client.o TEST_BUILTINS_OBJS += test-genrandom.o @@ -1338,7 +1339,6 @@ THIRD_PARTY_SOURCES += compat/regex/% THIRD_PARTY_SOURCES += sha1collisiondetection/% THIRD_PARTY_SOURCES += sha1dc/% -UNIT_TEST_PROGRAMS += t-basic UNIT_TEST_PROGRAMS += t-mem-pool UNIT_TEST_PROGRAMS += t-strbuf UNIT_TEST_PROGRAMS += t-ctype @@ -3218,7 +3218,7 @@ perf: all .PRECIOUS: $(TEST_OBJS) -t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) +t/helper/test-tool$X: $(patsubst %,t/helper/%,$(TEST_BUILTINS_OBJS)) $(UNIT_TEST_DIR)/test-lib.o t/helper/test-%$X: t/helper/test-%.o GIT-LDFLAGS $(GITLIBS) $(REFTABLE_TEST_LIB) $(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $@ $(ALL_LDFLAGS) $(filter %.o,$^) $(filter %.a,$^) $(LIBS) diff --git a/t/Makefile b/t/Makefile index 2d95046f26..4861edafe6 100644 --- a/t/Makefile +++ b/t/Makefile @@ -48,7 +48,7 @@ CHAINLINTTESTS = $(sort $(patsubst chainlint/%.test,%,$(wildcard chainlint/*.tes CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl UNIT_TEST_SOURCES = $(wildcard unit-tests/t-*.c) UNIT_TEST_PROGRAMS = $(patsubst unit-tests/%.c,unit-tests/bin/%$(X),$(UNIT_TEST_SOURCES)) -UNIT_TESTS = $(sort $(filter-out unit-tests/bin/t-basic%,$(UNIT_TEST_PROGRAMS))) +UNIT_TESTS = $(sort $(UNIT_TEST_PROGRAMS)) # `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`) # checks all tests in all scripts via a single invocation, so tell individual diff --git a/t/unit-tests/t-basic.c b/t/helper/test-example-tap.c similarity index 95% rename from t/unit-tests/t-basic.c rename to t/helper/test-example-tap.c index fda1ae59a6..d072ad559f 100644 --- a/t/unit-tests/t-basic.c +++ b/t/helper/test-example-tap.c @@ -1,4 +1,5 @@ -#include "test-lib.h" +#include "test-tool.h" +#include "t/unit-tests/test-lib.h" /* * The purpose of this "unit test" is to verify a few invariants of the unit @@ -69,7 +70,7 @@ static void t_empty(void) ; /* empty */ } -int cmd_main(int argc, const char **argv) +int cmd__example_tap(int argc, const char **argv) { test_res = TEST(check_res = check_int(1, ==, 1), "passing test"); TEST(t_res(1), "passing test and assertion return 1"); diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c index 33b9501c21..bb5c04c9c0 100644 --- a/t/helper/test-tool.c +++ b/t/helper/test-tool.c @@ -29,6 +29,7 @@ static struct test_cmd cmds[] = { { "dump-untracked-cache", cmd__dump_untracked_cache }, { "env-helper", cmd__env_helper }, { "example-decorate", cmd__example_decorate }, + { "example-tap", cmd__example_tap }, { "find-pack", cmd__find_pack }, { "fsmonitor-client", cmd__fsmonitor_client }, { "genrandom", cmd__genrandom }, diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h index b72f07ded9..38001bd1c6 100644 --- a/t/helper/test-tool.h +++ b/t/helper/test-tool.h @@ -23,6 +23,7 @@ int cmd__dump_untracked_cache(int argc, const char **argv); int cmd__dump_reftable(int argc, const char **argv); int cmd__env_helper(int argc, const char **argv); int cmd__example_decorate(int argc, const char **argv); +int cmd__example_tap(int argc, const char **argv); int cmd__find_pack(int argc, const char **argv); int cmd__fsmonitor_client(int argc, const char **argv); int cmd__genrandom(int argc, const char **argv); diff --git a/t/t0080-unit-test-output.sh b/t/t0080-unit-test-output.sh index 961b54b06c..83b1e3b7f5 100755 --- a/t/t0080-unit-test-output.sh +++ b/t/t0080-unit-test-output.sh @@ -8,50 +8,50 @@ test_expect_success 'TAP output from unit tests' ' cat >expect <<-EOF && ok 1 - passing test ok 2 - passing test and assertion return 1 - # check "1 == 2" failed at t/unit-tests/t-basic.c:76 + # check "1 == 2" failed at t/helper/test-example-tap.c:77 # left: 1 # right: 2 not ok 3 - failing test ok 4 - failing test and assertion return 0 not ok 5 - passing TEST_TODO() # TODO ok 6 - passing TEST_TODO() returns 1 - # todo check ${SQ}check(x)${SQ} succeeded at t/unit-tests/t-basic.c:25 + # todo check ${SQ}check(x)${SQ} succeeded at t/helper/test-example-tap.c:26 not ok 7 - failing TEST_TODO() ok 8 - failing TEST_TODO() returns 0 - # check "0" failed at t/unit-tests/t-basic.c:30 + # check "0" failed at t/helper/test-example-tap.c:31 # skipping test - missing prerequisite - # skipping check ${SQ}1${SQ} at t/unit-tests/t-basic.c:32 + # skipping check ${SQ}1${SQ} at t/helper/test-example-tap.c:33 ok 9 - test_skip() # SKIP ok 10 - skipped test returns 1 # skipping test - missing prerequisite ok 11 - test_skip() inside TEST_TODO() # SKIP ok 12 - test_skip() inside TEST_TODO() returns 1 - # check "0" failed at t/unit-tests/t-basic.c:48 + # check "0" failed at t/helper/test-example-tap.c:49 not ok 13 - TEST_TODO() after failing check ok 14 - TEST_TODO() after failing check returns 0 - # check "0" failed at t/unit-tests/t-basic.c:56 + # check "0" failed at t/helper/test-example-tap.c:57 not ok 15 - failing check after TEST_TODO() ok 16 - failing check after TEST_TODO() returns 0 - # check "!strcmp("\thello\\\\", "there\"\n")" failed at t/unit-tests/t-basic.c:61 + # check "!strcmp("\thello\\\\", "there\"\n")" failed at t/helper/test-example-tap.c:62 # left: "\011hello\\\\" # right: "there\"\012" - # check "!strcmp("NULL", NULL)" failed at t/unit-tests/t-basic.c:62 + # check "!strcmp("NULL", NULL)" failed at t/helper/test-example-tap.c:63 # left: "NULL" # right: NULL - # check "${SQ}a${SQ} == ${SQ}\n${SQ}" failed at t/unit-tests/t-basic.c:63 + # check "${SQ}a${SQ} == ${SQ}\n${SQ}" failed at t/helper/test-example-tap.c:64 # left: ${SQ}a${SQ} # right: ${SQ}\012${SQ} - # check "${SQ}\\\\${SQ} == ${SQ}\\${SQ}${SQ}" failed at t/unit-tests/t-basic.c:64 + # check "${SQ}\\\\${SQ} == ${SQ}\\${SQ}${SQ}" failed at t/helper/test-example-tap.c:65 # left: ${SQ}\\\\${SQ} # right: ${SQ}\\${SQ}${SQ} not ok 17 - messages from failing string and char comparison - # BUG: test has no checks at t/unit-tests/t-basic.c:91 + # BUG: test has no checks at t/helper/test-example-tap.c:92 not ok 18 - test with no checks ok 19 - test with no checks returns 0 1..19 EOF - ! "$GIT_BUILD_DIR"/t/unit-tests/bin/t-basic >actual && + ! test-tool example-tap >actual && test_cmp expect actual ' From patchwork Wed Apr 24 19:14:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13642420 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 474B75E07E for ; Wed, 24 Apr 2024 19:14:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713986061; cv=none; b=EkbfHJvigkVloGkooFhZxqoIuXdzCDRSeVi5pkKVOVM3GVGp4k7QEnC3EbssYEdBHP3/tJdwYgDE9Hf9LkasHsQJGJmAqxqcccZcJr4BAA8K1aacMdK/jrCJt/2GVfJ3M75piZvbqbmPcKgZiZW3oEkI/MxYCF3JvWGPmQH1+oE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713986061; c=relaxed/simple; bh=lbt9TJAPElNie2SlD/5S4WTeE/79cDloHMdXhhDb7jk=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=Jem3EiFk0HTMyJmv2ajMTfjRHSrg6wmytKJASa8TqAoMOql2d8sprRe0p2WrIk3l8C57sL0JDqYMW6z9Zgw2FprkjEZ5ax/ZgkcbJNs/D/9iAEK6EcfxlS8XT6HGTdrwI0QhL41/kOb97qWcpwbxlfVb5YGb7d3AtzisAjukJck= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=X6XdiMMI; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="X6XdiMMI" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-61ac32822bcso3182537b3.1 for ; Wed, 24 Apr 2024 12:14:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713986059; x=1714590859; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=GhLNRLpJH1PUCRG33l0/jTFly3a2tm8jftvh3mdfOBk=; b=X6XdiMMIQ3HysEVDo/rE+9DOrEDC0Sh8nuXSWgItkzlCLnL0NOQwjJQfedQ4eNeLCq uc1Bk5cjAQOp1UnN3uiNHfUJrldxQBGaNLT/qClMM/DrxWLWrELxEPx/kazzmEafVWYp vgNQ1KbBACWL/ck7Qld9UnlOM6QRDNbDLBdRHDmkgoblE+5Z6pOxR+jZWD8a4wkyubdz 1HB3JrEBLuSv4avuluE/lkL2QzG8rfcMgPH7OAsy/Y54kJLcW3vIft61eSJwcXF775FP eQJyTJ9vMCuEpgZnxYmxyaEbrqzL9CQEGGvjYH9bY+avz8mscI5c82j0X7jlefvDl1KC v7Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713986059; x=1714590859; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GhLNRLpJH1PUCRG33l0/jTFly3a2tm8jftvh3mdfOBk=; b=TpQNRXto9E8dAEYIYn7B+CYd7h1QKGSiX/FCyxh1MiMHz3tu59CbvIp0FVl79/uBp3 z8fIUivIQebFH6hJTDHaVMB0KPSyxLLvhSf7U58lPLN0z3/QP81g+srdopWGT9FX/Tip PBrF9qv2U8Wdqjz00T24oPm47mMgZo6FZa+/tG1gCGTK0tNY18iGyIWWGCBsKrRq8hV8 sdHSVHj1DATCPHkAb2Mmz0IGGbTn/z5PQX/kLr/jWGx+FFy3Et7/xIBmY14zmIISJqU8 iBgeZ1j1/fj2p3+ym53gxazy6Fug1sRuQbU96WhqyHsyCKoceg4y4ipJhW0KhGlqbUk2 gidg== X-Gm-Message-State: AOJu0Yy7WkZqUnRu7unxywPEmSsmcKQPponi087nOT44C0nzqkA8VQUK 3fCSe0P/mBIozzIxN9CFcFi3qLlRNNZq+IW31ctvgJZGGxTKMIEsPBFqCI06GjKhsublKER2lYS rkGADVQP0mihRB289LVQ2n8u29HZI1rZJZfdVtw2FBvA2WmT6jbvZGKp/QZE6zMtuZ/muLgXoAf ViLxxw694HnPJTfWG/GTXDHOO0SH6PiLG9RxxDnc8= X-Google-Smtp-Source: AGHT+IEG5Sbp/f5oNu8yYbHBuPcxr4mXR8noXIQdDiKjOXHmRXDBX4TfVg1n9ASY8VvQq+xnf0Mn7d5DjPduZQ== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:5580:46bd:98b:ecbb]) (user=steadmon job=sendgmr) by 2002:a81:69c4:0:b0:618:9588:e9db with SMTP id e187-20020a8169c4000000b006189588e9dbmr846668ywc.2.1713986059348; Wed, 24 Apr 2024 12:14:19 -0700 (PDT) Date: Wed, 24 Apr 2024 12:14:08 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.44.0.769.g3c40516874-goog Message-ID: <24f47f8fc7e931c2e15fa985635e3036ec1123ba.1713985716.git.steadmon@google.com> Subject: [PATCH v4 2/7] test-tool run-command testsuite: get shell from env From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net When running tests through `test-tool run-command testsuite`, we currently hardcode `sh` as the command interpreter. As discussed in [1], this is incorrect, and we should be using the shell set in TEST_SHELL_PATH instead. Add a shell_path field in struct testsuite so that we can pass this to the task runner callback. If this is non-null, we'll use it as the argv[0] of the subprocess. Otherwise, we'll just execute the test program directly. We will use this feature in a later commit to enable running binary executable unit tests. However, for now when setting up the struct testsuite in testsuite(), use the value of TEST_SHELL_PATH if it's set, otherwise keep the original behavior by defaulting to `sh`. [1] https://lore.kernel.org/git/20240123005913.GB835964@coredump.intra.peff.net/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index c0ed8722c8..a41a54d9cb 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -65,6 +65,7 @@ struct testsuite { struct string_list tests, failed; int next; int quiet, immediate, verbose, verbose_log, trace, write_junit_xml; + const char *shell_path; }; #define TESTSUITE_INIT { \ .tests = STRING_LIST_INIT_DUP, \ @@ -80,7 +81,9 @@ static int next_test(struct child_process *cp, struct strbuf *err, void *cb, return 0; test = suite->tests.items[suite->next++].string; - strvec_pushl(&cp->args, "sh", test, NULL); + if (suite->shell_path) + strvec_push(&cp->args, suite->shell_path); + strvec_push(&cp->args, test); if (suite->quiet) strvec_push(&cp->args, "--quiet"); if (suite->immediate) @@ -162,6 +165,10 @@ static int testsuite(int argc, const char **argv) if (max_jobs <= 0) max_jobs = online_cpus(); + suite.shell_path = getenv("TEST_SHELL_PATH"); + if (!suite.shell_path) + suite.shell_path = "sh"; + dir = opendir("."); if (!dir) die("Could not open the current directory"); From patchwork Wed Apr 24 19:14:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13642421 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 876157442D for ; Wed, 24 Apr 2024 19:14:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713986063; cv=none; b=ZqzqUwntSOVRk0JR6n5Lnl2Z6pDYU7MduJBP1uSMXxMBCtZMp2dqDUjTIp5dHP8uxlCB1K1cK1HaWTQBopHGDUtYPTIkJ/66VlushUxM6tFZzEwt28tMJQWf3bz/f/lzun827KDjLqK6rOD6+NHRjJ/1/j6+KPwd+ptnBpmOJC0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713986063; c=relaxed/simple; bh=HwlXhseibk7XuU6OSoV6FJp1xhufZ3etSRxMUF+5XyQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=szekgYdv5cF/qeIUJibvwt7z3xyXrO+dQkNxMZreK/z2Os+dfLzAmgTE4hqrPVAPGk8oX36ZCUAxkCy61z4WjmClLWa4uIyHlwrQlJAVN3OKxdVoN1VcrntqADOo8uXi4pOObNtHYfMwkV3iLDKsWRFGaJjUBKko6uRp5YCO2EQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=dlUnY3fP; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="dlUnY3fP" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-de54be7066bso405818276.0 for ; Wed, 24 Apr 2024 12:14:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713986061; x=1714590861; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=WXus1Gkcc1750Jiv0uDUx9Fuhc2v2RRkLeZb18hkEbA=; b=dlUnY3fPeCN7FuY8stZs5lpu/zkfeoEzTQe5pCnYRU7Xe+uJMcQnyAATJgIuaM1UqU 8CHb8YKmj8mHZQ+KS8ZOqnb1D3VepaBz5rl0NNcjuKuHHV5lL9hv7FeufnYahjnhTBSI 0Bi7N2jPXcUjncXy5YbzuAYGHEbihiTdJCCPhS9jdUH9KoQ14k+yzWQONaUSJjmAPLm5 WlLmsv1wXwyFamZdTJPNb8dflphQOLbos5Es3MCSr5FhjT79J9DSQgIO0vuIJ8ZUeKgW m8rSHyBYgQj1YWVzjToZrG5C0Q+nZqp6F+CfpcKHVBdS5tem53zCo5MowCb0mWvn84DL 01ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713986061; x=1714590861; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=WXus1Gkcc1750Jiv0uDUx9Fuhc2v2RRkLeZb18hkEbA=; b=dXg+GKbYPm22M1wYEqQsW1r72GpUmfn/jBj2OC6ThTIwtUb1EH4JoIRVMOjOxpPFeu MqABz/oWjMK+y9fFnpoSCyLBB9OHynwGZPGDvMUVnswO/+Yhwa+J3YFGvYKFOm6ZxFBp 3TTRACVQleUSbXI3z7XcGh6zoatmj1XIxv/ax/TKwaibmkkax0K3iNKhkOmH7ulVSekt vaelOxukMN0TBvl/9ga1bdr+h2FF1UfUXAkYrz6yf1s5IFLYtger15e9QmQNywqh5KlI bJyAkm1TtWqFzECQPF8uTiLr/LUufJLSrKuejYYiJ4J9BMn5uWJVcZUx3VQ3v4jjXDvy h9xw== X-Gm-Message-State: AOJu0Yyb1tPm2oiLggpO9rgJ8GsUkHnN+lhhLweHWgPWO3gpThyQPf3j jRYGxwff27KkjWporISUgH1Xzkk3C20HH4P9o7oVepYTJTxWf1bMjqq3uX5R6/ZPy1ItmGB82oW kfCJ2MhR1rh11usZO52MLuMv8o9f0Ci5eJYlC0BS0uFvKT0NptymtDdGsldUWNNB2IMbSwNwfUr O6q3PjP7LLAhKb1VF8Qg1MemDdm1O06qEmfGozOi4= X-Google-Smtp-Source: AGHT+IFVhOYRJjyMKOaHdsTMNyhu1wjIkzksTNvRqJtO5zf2EPdrRgxeYLnp7haq6JwNS6egdtXHz+YgbjYd2w== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:5580:46bd:98b:ecbb]) (user=steadmon job=sendgmr) by 2002:a05:6902:1024:b0:dcc:e1a6:aca9 with SMTP id x4-20020a056902102400b00dcce1a6aca9mr1108889ybt.9.1713986061354; Wed, 24 Apr 2024 12:14:21 -0700 (PDT) Date: Wed, 24 Apr 2024 12:14:09 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.44.0.769.g3c40516874-goog Message-ID: <4a16a3ec241d83b3de424ce835db24eee6e46f0f.1713985716.git.steadmon@google.com> Subject: [PATCH v4 3/7] test-tool run-command testsuite: remove hardcoded filter From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net `test-tool run-command testsuite` currently assumes that it will only be running the shell test suite, and therefore filters out anything that does not match a hardcoded pattern of "t[0-9][0-9][0-9][0-9]-*.sh". Later in this series, we'll adapt `test-tool run-command testsuite` to also support unit tests, which do not follow the same naming conventions as the shell tests, so this hardcoded pattern is inconvenient. Since `testsuite` also allows specifying patterns on the command-line, let's just remove this pattern. As noted in [1], there are no longer any uses of `testsuite` in our codebase, it should be OK to break backwards compatibility in this case. We also add a new filter to avoid trying to execute "." and "..", so that users who wish to execute every test in a directory can do so without specifying a pattern. [1] https://lore.kernel.org/git/850ea42c-f103-68d5-896b-9120e2628686@gmx.de/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index a41a54d9cb..e6bd792274 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -175,9 +175,7 @@ static int testsuite(int argc, const char **argv) while ((d = readdir(dir))) { const char *p = d->d_name; - if (*p != 't' || !isdigit(p[1]) || !isdigit(p[2]) || - !isdigit(p[3]) || !isdigit(p[4]) || p[5] != '-' || - !ends_with(p, ".sh")) + if (!strcmp(p, ".") || !strcmp(p, "..")) continue; /* No pattern: match all */ From patchwork Wed Apr 24 19:14:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13642422 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7211375818 for ; Wed, 24 Apr 2024 19:14:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713986065; cv=none; b=tCGyXsS7bg0cLWBZ2Y29Vh1087STlLdonlKcUmoR1eFJFjmV6uLpssUSYhNBTqOunaQ6uEG9LKqF1Lcz093SQHwMUFHVu5hoia1oiaKQUoYpUk8Vt0SxZngjcOSP6XRIi/xp7HOIJCs/coe5moamHo21yETge7PsANnXRr//7fU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713986065; c=relaxed/simple; bh=yg46iEGvZBwXbfOKCDQhMMocQwlcjDeR+guuv1f81SQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=SrdikabHgOegG7BQBpLrC6Z4THV6EZQbSGB1jrrrHVcm+cknYuYS5yxAB2Bsc5IgRlQYhxo7efFMQsQYDMZNmeCPAhrQYNCm22dL/WPBJe7/tohL/AwgtPt02fvu+cvXM+BzLkxhwfhLiZT4xbYV81d3BcRlY4vzFIVo7fhXEkM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=MMSmTw6m; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="MMSmTw6m" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-de465caa098so429988276.2 for ; Wed, 24 Apr 2024 12:14:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713986063; x=1714590863; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=G64qEtS+3zmjGrrTEZef+ZAZnHDZCT8RCENC8Hj3u40=; b=MMSmTw6mQW6kQZatwopYcQGQ/EQPyEP+N5u1T3B+IUVrWqcWWQDYTDYI7qRXdK4mss bIwJWKd8UKF6XKnDAg9L/39uws0NYYGa2EqZiza/r4+9p/6Fg9d5xG5TAQrduUo9BU5E eJvlrT3RmrDxbYAc6FoaOyDPGh/wWOjbC++jUMRMGnQgNbqJWvQdXBMV/pEMPINGUEgv 8fAMrPwcqr+V0XUhSHre93hRi1ZLwOtfYLBAZY+wmrkTkXHHyrTS2/KHQVQsMXA8cDqP /c0IEAxdxlf41iUmcAM+pMJohWsHCJV1DxnxlOBavHDbFicuzJSHhPloBaApzBkuarQw BuzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713986063; x=1714590863; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=G64qEtS+3zmjGrrTEZef+ZAZnHDZCT8RCENC8Hj3u40=; b=ILo9exWyA611OCOrV9dKeO2KTEiY0SN62x0ZiA4oSS6cRljOUDml9ec8CsBSZgWGFi PmgEI7pPg58vNBBQw643rh4Dy0J1hCEmhdZG1ph8Q/j8bv6Ja+GSeqUkIO8TE9USAicU sXvj4w4qkSUVCTBE3DA+PJDyEXvCzsk0NecjmnKYjMY1/nQJouon36hZhC2ldyvMJ1TK q92byOUC+5kzjEAHBKzM7O8od+hiDuNh7cdipV2UI3ET/ozKgxcDJ4A7pOfL2khPjRkM HoDhaNvBpWFXj1pypsNOCJlPlUxZdJ2+Cbz2Jq7cso3APrOmBJ6MqmNQA/GNdZcnRfCh JRyA== X-Gm-Message-State: AOJu0YwtFkJBSqjFad7gTA7edmsx/R/bZbLT7/3iH88invuNxcaOgj15 sMwI3sroMkgFT+JpXfHqFSWg8eBFz8cj6tEQLf/GXd6GkVdKltEEL6xZ2evGBjMqxEkDa77H6Tm w+w9QEIgEDbNIIqWADPCNIUtMfT/K2KTK7uxuaL1TcZ1bpvKas5nQaUphI90qESdOKJrcn0oudu hWzPHVbbIJPUGxTCMINs97C7EUj31EAcGbgCGyYGU= X-Google-Smtp-Source: AGHT+IFh6kiPyBJwp4D0GppjYGEXjtLXwl0phEUdaysht4dlwkfR1Zyp5ZmGAU5hPur7XwFfMhLsj6o0ot0p0g== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:5580:46bd:98b:ecbb]) (user=steadmon job=sendgmr) by 2002:a05:6902:114d:b0:dda:c566:dadd with SMTP id p13-20020a056902114d00b00ddac566daddmr340769ybu.4.1713986063323; Wed, 24 Apr 2024 12:14:23 -0700 (PDT) Date: Wed, 24 Apr 2024 12:14:10 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.44.0.769.g3c40516874-goog Message-ID: Subject: [PATCH v4 4/7] test-tool run-command testsuite: support unit tests From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net Teach the testsuite runner in `test-tool run-command testsuite` how to run unit tests: if TEST_SHELL_PATH is not set, run the programs directly from CWD, rather than defaulting to "sh" as an interpreter. With this change, you can now use test-tool to run the unit tests: $ make $ cd t/unit-tests/bin $ ../../helper/test-tool run-command testsuite This should be helpful on Windows to allow running tests without requiring Perl (for `prove`), as discussed in [1] and [2]. This again breaks backwards compatibility, as it is now required to set TEST_SHELL_PATH properly for executing shell scripts, but again, as noted in [2], there are no longer any such invocations in our codebase. [1] https://lore.kernel.org/git/nycvar.QRO.7.76.6.2109091323150.59@tvgsbejvaqbjf.bet/ [2] https://lore.kernel.org/git/850ea42c-f103-68d5-896b-9120e2628686@gmx.de/ Signed-off-by: Josh Steadmon --- t/helper/test-run-command.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/t/helper/test-run-command.c b/t/helper/test-run-command.c index e6bd792274..61eb1175fe 100644 --- a/t/helper/test-run-command.c +++ b/t/helper/test-run-command.c @@ -158,6 +158,8 @@ static int testsuite(int argc, const char **argv) .task_finished = test_finished, .data = &suite, }; + struct strbuf progpath = STRBUF_INIT; + size_t path_prefix_len; argc = parse_options(argc, argv, NULL, options, testsuite_usage, PARSE_OPT_STOP_AT_NON_OPTION); @@ -165,9 +167,13 @@ static int testsuite(int argc, const char **argv) if (max_jobs <= 0) max_jobs = online_cpus(); + /* + * If we run without a shell, execute the programs directly from CWD. + */ suite.shell_path = getenv("TEST_SHELL_PATH"); if (!suite.shell_path) - suite.shell_path = "sh"; + strbuf_addstr(&progpath, "./"); + path_prefix_len = progpath.len; dir = opendir("."); if (!dir) @@ -180,13 +186,17 @@ static int testsuite(int argc, const char **argv) /* No pattern: match all */ if (!argc) { - string_list_append(&suite.tests, p); + strbuf_setlen(&progpath, path_prefix_len); + strbuf_addstr(&progpath, p); + string_list_append(&suite.tests, progpath.buf); continue; } for (i = 0; i < argc; i++) if (!wildmatch(argv[i], p, 0)) { - string_list_append(&suite.tests, p); + strbuf_setlen(&progpath, path_prefix_len); + strbuf_addstr(&progpath, p); + string_list_append(&suite.tests, progpath.buf); break; } } @@ -213,6 +223,7 @@ static int testsuite(int argc, const char **argv) string_list_clear(&suite.tests, 0); string_list_clear(&suite.failed, 0); + strbuf_release(&progpath); return ret; } From patchwork Wed Apr 24 19:14:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13642423 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5649D768EF for ; Wed, 24 Apr 2024 19:14:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713986067; cv=none; b=hqdLTk/HdL8wP/v/iXQbrlANRDvfXgQAoQCDL4H0sIkY3omvmV83IEtO3oQ0TgPq4je/apsYUi+DcRjPas68CRWbXacXsbDXDMP/jsL5IajzWzw6LH3t3FYKpUiEwn6bV7PI20LHOCMSelXKPxm666R9lGO8kH+8JNOSMfM8JDo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713986067; c=relaxed/simple; bh=0o0CxDfjGz2bjkuxxOsWJ4cKzUqvmEaqYobt7WRc7iw=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=FQ7cjykOEwexDwmYA3NIE7+SfYHD1diSd9MGyUinx8z/LQPhWg6RukzO9iDJokc/deGHdgqTtGmKdEAxmsYTh4Diq/gVmpXTFJo6mbB3VctqMSXbM9MCZb9zpLIy8WY02k6Wrv9fQCSp1ZaTpwiq0HQwb4cDvhLI8YXTQxP3wWk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=pqUFNlUI; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="pqUFNlUI" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-61ab00d8a2eso2553487b3.3 for ; Wed, 24 Apr 2024 12:14:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713986065; x=1714590865; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=KbD6bguqs6FaA9q79veX+5zGuNpubqTIT4TYQlUpYMU=; b=pqUFNlUIN5cEoVDdIwxMKckXb4Qhg4YR/U0YM154xn57BxstEpKUgeGs3F18FxbtzQ IrZHI83Rw0y4U1cNOtS9e10+10Yz41OEH7EHydb9R3QSnPa18fYaowE/aKLQYt9JBjED 23z3Ahb9SMbGhqwfGs5rJdOY6yBzRi+6c7+daQH69wANDxxG3em5yjRYs3YFGIe6sipU 8q1OXbx2WEdN+1NkgDbcnNs3nJY3T/avyT1GAuw9/FYCCYsIirv+0uxZdsJYLHiGnLma +zm4B0V7sXOgnLoURGfOWOsFp88be6yzI9pqKKnaxzKtV7ow+kmaD1Cd2KxVlN2GUsdb qMfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713986065; x=1714590865; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KbD6bguqs6FaA9q79veX+5zGuNpubqTIT4TYQlUpYMU=; b=UbEomKBVVLgc036hYSJbuAfz8eeGAxpSR+x649eygJpxLyPqXOIoEdEbqAR45dpBKK +yRYwaJDXiO7nYSoG+3pVEw8eH6dIYUeZ6K6ZPPUtshJQGASEy8B+ThsftoXl+klWk58 Y1AxFJVvKr6S8GYsCKz2gcRCfqZ4K3xpuU6pDz0yhJqJ1yU66xwYRQTdRmqJ4mXXLpAQ Bacx32k3heTIYgNTRNBcYIEXYMxZI34zraSwGCeywctfHbAJb/AlWRclBWq32oPV+TsY L7LU+US2+tfCH5n0x8wXwkiFWiFBuiGTlybA3i+mr+AQZ0eWexT67YoBmo0PdP1wPF6G xanA== X-Gm-Message-State: AOJu0YyJXeFJZQVSVAAIdqmXv4p2XMTzkgXvdRQZv6hN1O5MvvAymVFM qlOyngVIagAXsZFueGgGWwHU8FroARnZcqSNrSZXuRdUyvTvmWghoDiIytDgTCi8qiYTJsOWnTW dAa8JADMgFgkwJolBxbdGD2WUaf+lTuDMs6b1oHcVufhUBc2nNW6lwxauPx+qH5lbtEMXZaw5G3 ZdZYfABCjekVLoAdBSlTje4vc7C1Tr2lAEw5VMhlM= X-Google-Smtp-Source: AGHT+IHDHPbJNs+0IRnI7I3rb1l+RbMYjSzrfxPpLZiiQ+itCNCJOoSdOgmej+YEl3gX/1xuExSGpWy5zOtYRg== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:5580:46bd:98b:ecbb]) (user=steadmon job=sendgmr) by 2002:a05:6902:706:b0:dbe:30cd:8fcb with SMTP id k6-20020a056902070600b00dbe30cd8fcbmr336936ybt.0.1713986065321; Wed, 24 Apr 2024 12:14:25 -0700 (PDT) Date: Wed, 24 Apr 2024 12:14:11 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.44.0.769.g3c40516874-goog Message-ID: Subject: [PATCH v4 5/7] unit tests: add rule for running with test-tool From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net In the previous commit, we added support in test-tool for running collections of unit tests. Now, add rules in t/Makefile for running in this way. This new rule can be executed from the top-level Makefile via `make DEFAULT_UNIT_TEST_TARGET=unit-tests-test-tool unit-tests`, or by setting DEFAULT_UNIT_TEST_TARGET in config.mak. Signed-off-by: Josh Steadmon --- Makefile | 2 +- t/Makefile | 10 +++++++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index ba55d817ee..b0d1f04b4d 100644 --- a/Makefile +++ b/Makefile @@ -3870,5 +3870,5 @@ $(UNIT_TEST_PROGS): $(UNIT_TEST_BIN)/%$X: $(UNIT_TEST_DIR)/%.o $(UNIT_TEST_DIR)/ .PHONY: build-unit-tests unit-tests build-unit-tests: $(UNIT_TEST_PROGS) -unit-tests: $(UNIT_TEST_PROGS) +unit-tests: $(UNIT_TEST_PROGS) t/helper/test-tool$X $(MAKE) -C t/ unit-tests diff --git a/t/Makefile b/t/Makefile index 4861edafe6..0ae04f1e42 100644 --- a/t/Makefile +++ b/t/Makefile @@ -49,6 +49,7 @@ CHAINLINT = '$(PERL_PATH_SQ)' chainlint.pl UNIT_TEST_SOURCES = $(wildcard unit-tests/t-*.c) UNIT_TEST_PROGRAMS = $(patsubst unit-tests/%.c,unit-tests/bin/%$(X),$(UNIT_TEST_SOURCES)) UNIT_TESTS = $(sort $(UNIT_TEST_PROGRAMS)) +UNIT_TESTS_NO_DIR = $(notdir $(UNIT_TESTS)) # `test-chainlint` (which is a dependency of `test-lint`, `test` and `prove`) # checks all tests in all scripts via a single invocation, so tell individual @@ -76,7 +77,7 @@ $(T): $(UNIT_TESTS): @echo "*** $@ ***"; $@ -.PHONY: unit-tests unit-tests-raw unit-tests-prove +.PHONY: unit-tests unit-tests-raw unit-tests-prove unit-tests-test-tool unit-tests: $(DEFAULT_UNIT_TEST_TARGET) unit-tests-raw: $(UNIT_TESTS) @@ -84,6 +85,13 @@ unit-tests-raw: $(UNIT_TESTS) unit-tests-prove: @echo "*** prove - unit tests ***"; $(PROVE) $(GIT_PROVE_OPTS) $(UNIT_TESTS) +unit-tests-test-tool: + @echo "*** test-tool - unit tests **" + ( \ + cd unit-tests/bin && \ + ../../helper/test-tool$X run-command testsuite $(UNIT_TESTS_NO_DIR)\ + ) + pre-clean: $(RM) -r '$(TEST_RESULTS_DIRECTORY_SQ)' From patchwork Wed Apr 24 19:14:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13642424 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BCCCC2E652 for ; Wed, 24 Apr 2024 19:14:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713986069; cv=none; b=SUmHmv8athif5vQlSd8yiDUaSTVkv+90NcG3yEGU2HcAZlVFTfXYa9RYA5hosh5MXo1V8ajDdBsEbb4ZlpKVi3SfP0e1Qex96HVr14q+Y1jArFCIOOqIpFrCV4ZQbk4MlZ+wirIVPnPtONBkA7394UE6UGzXGJiDQwc0IqHKYhA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713986069; c=relaxed/simple; bh=OK/AK7OwRaxjhQuX69mk/1GRNWht2P/CGaHRI6gZEic=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=GmhF1rFoR2REUWBO50muKroSWldtyK1McNn2JNLK8VUyCg4WMlsVHd/i8i+ADe9kpzPgzg6ZDDapCXvR1aOHlcXXcFVqrQOaMcM66aniqSdmQrCJRi6VC9ySCSuUgfixnBDr+J1KTI4ptiMjPyVgiOdQ6/F4h6tPK+LVeO62r8Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=cdB9+GE3; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="cdB9+GE3" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-de58ab59821so80444276.1 for ; Wed, 24 Apr 2024 12:14:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713986067; x=1714590867; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=xmIf6E6awSbarOX/qVEWRtHpQQF7e9ghor3pb5DLyDU=; b=cdB9+GE3UuZfBr8RMKZSlULtbxQFLDjh78Mav4XZoqXOZ1LZZnHLDQsBYB4picrPW0 wkmwWuzV3XjahGjF3S59p6Op6P11ArhoJvFa/zIuj5HqcABjoEW7CP+tvN2jEdVztMn+ 6nZN3tBfbpzd3ltJ5P5m8h0Nq8NTfFz+URVg9D/AKxQVgnduuPpEwwh58YhFGfR/dEfi yvzFfqLLmktwMm8AYHS3+sqVjpa+Ua3iIyICrXLlAAb+LIliCL6PJ/0jrbe0LPrtihdx 0di/DZboKNBaP7SJhQaeIddTwNCWtnvfZESr9PuZQKvSKS9aKugXwEVzUtK2Dc9T9iCi IaUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713986067; x=1714590867; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=xmIf6E6awSbarOX/qVEWRtHpQQF7e9ghor3pb5DLyDU=; b=tKFg4ZrdySY7Ll3WVGSnJd/ZNmIbF/xVFx9lWyN5ZfOW7XDSxOh9lQgozHYZ5SQFGd d7M/mljlq39WHH4rS3IOkIdZtc1oHCh4wsKclCuWc4pQnvS3vfvKFiUzz3zfDF9KWCx3 uuH0VUHjMYMNJqA44oSTpvh8oh0MRvgTTfZ9qyxDLRY2s84QY7SKVP0Kagi6uIYE5Pxn LItmm8GXEYK4d1HEN/+z0V8DwhfoT11+VYicfXdl830FlR76aqny5PVlxk8q954xMYtH RqtMIKaTAdtGKHauZ9OXKtvM4Ny+YTdk1oinWU7ijAomyhpPeAdb8acOWbDEHTe8hkQw J6vg== X-Gm-Message-State: AOJu0YwoqTuI8RDO6k3Btjda4HiNL5glfEu1slHICkPS5zFZI74Fnu1b YnPjTJ7g5PpbG54xGydHPibEvwaZU7AA9QIWERl00X99zd6ErDnirsd5riAVE3wSIjo1lOqCnBl qkuVQYCRaMy6UkcI12H+hS/opM3Fz0e/h4wMwhnpvzNnD6bpOocibFauSYn3mPWwhdYNH7LCQnk rkR7RnzhHxM7UZcxDVsadQMZwqxmCwLGRw9ucXcmk= X-Google-Smtp-Source: AGHT+IE4DujGjh/6hRg7T2cxFcYJ8H/8eIRj2b+fUAPWP88a8nIWUa9GKDI/wohus2hwIxPrZb0E1zfoKfSVFA== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:5580:46bd:98b:ecbb]) (user=steadmon job=sendgmr) by 2002:a05:6902:1028:b0:dcb:e982:4e40 with SMTP id x8-20020a056902102800b00dcbe9824e40mr1093263ybt.12.1713986066839; Wed, 24 Apr 2024 12:14:26 -0700 (PDT) Date: Wed, 24 Apr 2024 12:14:12 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.44.0.769.g3c40516874-goog Message-ID: <0e32de1afe9cbab02c5d3476a0fc2a1ba0151dcf.1713985716.git.steadmon@google.com> Subject: [PATCH v4 6/7] t/Makefile: run unit tests alongside shell tests From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net From: Jeff King Add a wrapper script to allow `prove` to run both shell tests and unit tests from a single invocation. This avoids issues around running prove twice in CI, as discussed in [1]. Additionally, this moves the unit tests into the main dev workflow, so that errors can be spotted more quickly. Accordingly, we remove the separate unit tests step for Linux CI. (We leave the Windows CI unit-test step as-is, because the sharding scheme there involves selecting specific test files rather than running `make test`.) [1] https://lore.kernel.org/git/pull.1613.git.1699894837844.gitgitgadget@gmail.com/ Signed-off-by: Jeff King Signed-off-by: Josh Steadmon --- ci/run-build-and-tests.sh | 2 -- t/Makefile | 2 +- t/run-test.sh | 17 +++++++++++++++++ 3 files changed, 18 insertions(+), 3 deletions(-) create mode 100755 t/run-test.sh diff --git a/ci/run-build-and-tests.sh b/ci/run-build-and-tests.sh index 7a1466b868..2528f25e31 100755 --- a/ci/run-build-and-tests.sh +++ b/ci/run-build-and-tests.sh @@ -50,8 +50,6 @@ if test -n "$run_tests" then group "Run tests" make test || handle_failed_tests - group "Run unit tests" \ - make DEFAULT_UNIT_TEST_TARGET=unit-tests-prove unit-tests fi check_unignored_build_artifacts diff --git a/t/Makefile b/t/Makefile index 0ae04f1e42..b2eb9f770b 100644 --- a/t/Makefile +++ b/t/Makefile @@ -68,7 +68,7 @@ failed: test -z "$$failed" || $(MAKE) $$failed prove: pre-clean check-chainlint $(TEST_LINT) - @echo "*** prove ***"; $(CHAINLINTSUPPRESS) $(PROVE) --exec '$(TEST_SHELL_PATH_SQ)' $(GIT_PROVE_OPTS) $(T) :: $(GIT_TEST_OPTS) + @echo "*** prove (shell & unit tests) ***"; $(CHAINLINTSUPPRESS) TEST_SHELL_PATH='$(TEST_SHELL_PATH_SQ)' $(PROVE) --exec ./run-test.sh $(GIT_PROVE_OPTS) $(T) $(UNIT_TESTS) :: $(GIT_TEST_OPTS) $(MAKE) clean-except-prove-cache $(T): diff --git a/t/run-test.sh b/t/run-test.sh new file mode 100755 index 0000000000..a0e2dce5e0 --- /dev/null +++ b/t/run-test.sh @@ -0,0 +1,17 @@ +#!/bin/sh + +# A simple wrapper to run shell tests via TEST_SHELL_PATH, +# or exec unit tests directly. + +case "$1" in +*.sh) + if test -z "${TEST_SHELL_PATH+set}" ; then + echo "ERROR: TEST_SHELL_PATH is not set" >&2 + exit 1 + fi + exec ${TEST_SHELL_PATH} "$@" + ;; +*) + exec "$@" + ;; +esac From patchwork Wed Apr 24 19:14:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josh Steadmon X-Patchwork-Id: 13642425 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B39C478C98 for ; Wed, 24 Apr 2024 19:14:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713986071; cv=none; b=LU4yHiNh8cRS7kgM7wfGx7fLO2UO+GYFRSoxETQre1T7+1lHtMPbJgUZhn1Yt7DrP6GeY4Ba17UJ449JuY6OayNnXGD5RM+CrSebCSI2lZk8UkMucaaOItJbAYOXfSYdeUoSlJu7uC0r1T7GJvpXrZTPCnw9PWElTwEUXyPee54= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1713986071; c=relaxed/simple; bh=P2VMmYb2N92ifJDg/3RWAO/iSxL/5b85Gf+Xg4ygcdY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=QQsWVluQLYy1n8evq+z2x+lLdt5G04LYF/m5S+XjIj6S0c1zbnYgguet056kvL71nUnQCMXwN1+RZwnnODkWP+Qykm7uPh5U8Y7/KGiy0rqAOCao/12GhzldWFwamvcKD5UFKSh8Sn7TM+0LGt2COQWERcWuuDctTT3Q8WB+eZU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=2BMALEwT; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--steadmon.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="2BMALEwT" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-de549a4ea65so427950276.0 for ; Wed, 24 Apr 2024 12:14:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713986069; x=1714590869; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=o7MAdAleVDcOuxioqIMH/DlQEsXKWAG/gLCFJ1yj6s0=; b=2BMALEwTizvi5UejCunateaQrap56zj8TsskoQJJGl2DRapFQBEmePov9Bbrm1jvce FMBkgcqDytLK2h12CVjSg6HwhLDWTfrhcef5TZCmiMaSll2h/vYp7n7t2kSu8M1BXwUy 7O0Bj6EpAVwxR5LjnyuAowlnL1gm7Fwqfkj54odbhT1H3/b3TNvr9uvXCsnAoc7Hq5Ar PoCAE8TwuUYMcCx0BalaT1HofXfsHHom15wQX/UsmZfiA7B8RX1yUyqbX2v5cB+zT6v5 D+i9ccIeLjyhsS9PwvBPVm/ZPenpadIQrDeX4ihm318qqb3O/svCYF1ZgkXcE3hV94/Z yY2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713986069; x=1714590869; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=o7MAdAleVDcOuxioqIMH/DlQEsXKWAG/gLCFJ1yj6s0=; b=p2j+/P8JQwGgBNjCPvwW7P7nYAuTs4Qv0YY7PgCr/pcIsyNRvjYBgogLX1wmpsyPGu 5O0kUEzYz9sOE4dj7NE3LsEIyvyY/NLNcadYgG/zdlBd7EFGOj7wSgrr0d1rZa4NgJSN 8XHM+hFcxGfDmR8l2oH1RRyieqECBdtzql6Nkf0GODXHrJoiQRi2Ys3QrEdG0ZnYROsP 7TnGrj2b7zatwLKogyTDFUdFieB8uiZg5xinNjRIA/HZ3p71Tq4I0pCdoMp7/LEnbDiF kzhLRynSUu4htpDhcOKPTkVhIzrZ6rakPTs2JfcbICzaNM2ndCCI+xc/s3wgEKJ431BD Ybdg== X-Gm-Message-State: AOJu0Yy3BXBYE3U0eEKqxKOQFXWcONh69KI5GqNoWYTBTy2983U9/JZH vHpVhlB0K/J5LxYMez64S7IWNaA4Wb0aRXcDMDUq6/z8yv6ibuoGC+3iteKA7Ns6c8RniODgtvD qNNCO/nzaxm/Pid7UXifu2biPnNHglJKYLh7u+8KuHJhFGNgI6q8QRrVBcXH8LD2pzZC2KBhtrQ pdIQJrfa1YjNh3vrfC7W0ivMvO9ZZ6ZXCYple5CqE= X-Google-Smtp-Source: AGHT+IFChwla5Gn/DRLonfN9Qdi2oBjbQ1qleutAJk9D5dTvEP4QGJaJBSZjzcIUjEIH8GbJqLque/Yz28s3Vg== X-Received: from lunarfall.svl.corp.google.com ([2620:15c:2d3:204:5580:46bd:98b:ecbb]) (user=steadmon job=sendgmr) by 2002:a05:6902:260f:b0:de5:4ed6:d3f3 with SMTP id dw15-20020a056902260f00b00de54ed6d3f3mr1075444ybb.6.1713986068688; Wed, 24 Apr 2024 12:14:28 -0700 (PDT) Date: Wed, 24 Apr 2024 12:14:13 -0700 In-Reply-To: Precedence: bulk X-Mailing-List: git@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.44.0.769.g3c40516874-goog Message-ID: Subject: [PATCH v4 7/7] ci: use test-tool as unit test runner on Windows From: Josh Steadmon To: git@vger.kernel.org Cc: gitster@pobox.com, peff@peff.net Although the previous commit changed t/Makefile to run unit tests alongside shell tests, the Windows CI still needs a separate unit-tests step due to how the test sharding works. We want to avoid using `prove` as a test running on Windows due to performance issues [1], so use the new test-tool runner instead. [1] https://lore.kernel.org/git/850ea42c-f103-68d5-896b-9120e2628686@gmx.de/ Signed-off-by: Josh Steadmon --- ci/run-test-slice.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ci/run-test-slice.sh b/ci/run-test-slice.sh index ae8094382f..e167e646f7 100755 --- a/ci/run-test-slice.sh +++ b/ci/run-test-slice.sh @@ -17,7 +17,7 @@ handle_failed_tests # We only have one unit test at the moment, so run it in the first slice if [ "$1" == "0" ] ; then - group "Run unit tests" make --quiet -C t unit-tests-prove + group "Run unit tests" make --quiet -C t unit-tests-test-tool fi check_unignored_build_artifacts