diff mbox series

chainlint test failing on Linux sparc64

Message ID 8baa12f8d044265f1ddeabd64209e7ac0d3700ae.camel@physik.fu-berlin.de (mailing list archive)
State New, archived
Headers show
Series chainlint test failing on Linux sparc64 | expand

Commit Message

John Paul Adrian Glaubitz May 20, 2024, 7:56 a.m. UTC
Hello,

the chainlint test is failing on sparc64 (see below) and I'm wondering if anyone
can tell me how to run tests manually. I suspect there might be an unaligned access
which causes a crash on sparc64 only.

Thanks,
Adrian

===================================================================================

    SUBDIR git-gui
    SUBDIR gitk-git
    SUBDIR templates
make -C t/ all
make[1]: Entering directory '/home/glaubitz/git/t'
rm -f -r 'test-results'
make[1]: *** [Makefile:110: check-chainlint] Error 1
make[1]: Leaving directory '/home/glaubitz/git/t'
make: *** [Makefile:3229: test] Error 2

===================================================================================

Comments

Eric Sunshine May 20, 2024, 8:21 a.m. UTC | #1
On Mon, May 20, 2024 at 3:56 AM John Paul Adrian Glaubitz
<glaubitz@physik.fu-berlin.de> wrote:
> the chainlint test is failing on sparc64 (see below) and I'm wondering if anyone
> can tell me how to run tests manually. I suspect there might be an unaligned access
> which causes a crash on sparc64 only.
>
> rm -f -r 'test-results'
> --- chainlinttmp/expect 2024-05-19 12:26:50.051507198 +0000
> +++ chainlinttmp/actual 2024-05-19 12:26:50.051507198 +0000
> @@ -1,955 +0,0 @@
> -# chainlint: chainlinttmp/tests
> -# chainlint: arithmetic-expansion
> -(
> -       foo &&
> -       bar=$((42 + 1)) &&
> -       baz
> -) &&
> [...snip...]

The thing failing here is chainlint's own self-test, which you don't
actually need if you're merely building Git. You'd only care about
chainlint (let alone its self-test) if you're modifying tests or
creating new ones. You can bypass chainlint altogether by setting
environment variable GIT_TEST_CHAIN_LINT=0.

That said, chainlint is just a Perl script, and you can manually run
the self-test like this:

    cd t
    make check-chainlint

The output you posted is coming from this line in t/Makefile:

    diff -u '$(CHAINLINTTMP_SQ)'/expect '$(CHAINLINTTMP_SQ)'/actual

Based upon what you pasted, it looks like the "actual" file has no
content. That might suggest a problem with this line which immediately
precedes it:

    $(CHAINLINT) --emit-all '$(CHAINLINTTMP_SQ)'/tests | \
        sed -e 's/^[1-9][0-9]* //' >'$(CHAINLINTTMP_SQ)'/actual && \

"actual" might end up empty if the Perl script isn't emitting anything
for some reason, or if `sed` isn't emitting anything. Presumably you
have a working `sed` installed(?), but do you have Perl installed?
John Paul Adrian Glaubitz May 20, 2024, 8:24 a.m. UTC | #2
Hi Eric,

On Mon, 2024-05-20 at 04:21 -0400, Eric Sunshine wrote:
> The thing failing here is chainlint's own self-test, which you don't
> actually need if you're merely building Git. You'd only care about
> chainlint (let alone its self-test) if you're modifying tests or
> creating new ones. You can bypass chainlint altogether by setting
> environment variable GIT_TEST_CHAIN_LINT=0.
> 
> That said, chainlint is just a Perl script, and you can manually run
> the self-test like this:
> 
>     cd t
>     make check-chainlint
> 
> The output you posted is coming from this line in t/Makefile:
> 
>     diff -u '$(CHAINLINTTMP_SQ)'/expect '$(CHAINLINTTMP_SQ)'/actual
> 
> Based upon what you pasted, it looks like the "actual" file has no
> content. That might suggest a problem with this line which immediately
> precedes it:
> 
>     $(CHAINLINT) --emit-all '$(CHAINLINTTMP_SQ)'/tests | \
>         sed -e 's/^[1-9][0-9]* //' >'$(CHAINLINTTMP_SQ)'/actual && \
> 
> "actual" might end up empty if the Perl script isn't emitting anything
> for some reason, or if `sed` isn't emitting anything. Presumably you
> have a working `sed` installed(?), but do you have Perl installed?

Thanks a lot for the elaborate answer!

I think the problem seems to be that the testsuite is not properly invoked
at all. When I run the testsuite with v2.37.7, all tests are run and pass,
but starting with v2.38.0, it only runs the chainlint test and exists.

I am trying to bisect this now.

Adrian
John Paul Adrian Glaubitz May 20, 2024, 9:04 a.m. UTC | #3
Hi,

On Mon, 2024-05-20 at 10:24 +0200, John Paul Adrian Glaubitz wrote:
> I think the problem seems to be that the testsuite is not properly invoked
> at all. When I run the testsuite with v2.37.7, all tests are run and pass,
> but starting with v2.38.0, it only runs the chainlint test and exists.
> 
> I am trying to bisect this now.

Bisecting was successful and has lead me to this commit:

d00113ec3474a1652a73c11695c7e7b5182d80a7 is the first bad commit
commit d00113ec3474a1652a73c11695c7e7b5182d80a7
Author: Eric Sunshine <sunshine@sunshineco.com>
Date:   Thu Sep 1 00:29:46 2022 +0000

    t/Makefile: apply chainlint.pl to existing self-tests
    
    Now that chainlint.pl is functional, take advantage of the existing
    chainlint self-tests to validate its operation. (While at it, stop
    validating chainlint.sed against the self-tests since it will soon be
    retired.)
(...)

Adrian
Eric Sunshine May 20, 2024, 9:36 a.m. UTC | #4
On Mon, May 20, 2024 at 5:04 AM John Paul Adrian Glaubitz
<glaubitz@physik.fu-berlin.de> wrote:
> Bisecting was successful and has lead me to this commit:
>
> d00113ec3474a1652a73c11695c7e7b5182d80a7 is the first bad commit
> Author: Eric Sunshine <sunshine@sunshineco.com>
> Date:   Thu Sep 1 00:29:46 2022 +0000
>     t/Makefile: apply chainlint.pl to existing self-tests
>
>     Now that chainlint.pl is functional, take advantage of the existing
>     chainlint self-tests to validate its operation. (While at it, stop
>     validating chainlint.sed against the self-tests since it will soon be
>     retired.)

That's the point at which chainlint.sed was replaced with
chainlint.pl. It also leads to the same supposition expressed earlier
that the "actual" file seems to be empty for some reason, perhaps
because one of the commands which creates it is failing somehow. At
that point in time, the creation command for "actual" in t/Makefile
was:

    $(CHAINLINT) --emit-all '$(CHAINLINTTMP_SQ)'/tests | \
        grep -v '^[ ]*$$' >'$(CHAINLINTTMP_SQ)'/actual && \

which uses `grep` rather than the `sed` used in the present-day
t/Makefile. This may imply that Perl is somehow suspect or that
PERL_PATH or PERL_PATH_SQ in t/Makefile are incorrectly set, or that
t/chainlint.pl itself is not generating any output for some reason.

After running:

    cd t
    make check-chainlint

what is the output of:

    ls -l chainlinttmp

For me the output using the latest "master" branch is:

    % ls -l chainlinttmp
    -rw-------  1 me  me  14797 May 20 05:26 actual
    -rw-------  1 me  me  14797 May 20 05:26 expect
    -rw-------  1 me  me  15086 May 20 05:26 tests

My suspicion is that "actual" will have size 0 for you. If that's the
case, I'd suggest running the commands from the "check-chainlint"
target in t/Makefile manually one at a time to see if you can figure
out which is failing. For instance:

    % ./chainlint.pl --emit-all chainlinttmp/tests

If that produces output, then that's a good sign. Try capturing that
output and feeding it to the `sed` command:

    % ./chainlint.pl --emit-all chainlinttmp/tests >lint.out
    % sed -e 's/^[1-9][0-9]* //' lint.out

If that produces output, then something else is going wrong (such as
PERL_PATH or PERL_PATH_SQ being incorrect).
John Paul Adrian Glaubitz May 20, 2024, 9:56 a.m. UTC | #5
Hi Eric,

On Mon, 2024-05-20 at 05:36 -0400, Eric Sunshine wrote:
> That's the point at which chainlint.sed was replaced with
> chainlint.pl. It also leads to the same supposition expressed earlier
> that the "actual" file seems to be empty for some reason, perhaps
> because one of the commands which creates it is failing somehow. At
> that point in time, the creation command for "actual" in t/Makefile
> was:
> 
>     $(CHAINLINT) --emit-all '$(CHAINLINTTMP_SQ)'/tests | \
>         grep -v '^[ ]*$$' >'$(CHAINLINTTMP_SQ)'/actual && \
> 
> which uses `grep` rather than the `sed` used in the present-day
> t/Makefile. This may imply that Perl is somehow suspect or that
> PERL_PATH or PERL_PATH_SQ in t/Makefile are incorrectly set, or that
> t/chainlint.pl itself is not generating any output for some reason.
> 
> After running:
> 
>     cd t
>     make check-chainlint
> 
> what is the output of:
> 
>     ls -l chainlinttmp

(sid_sparc64-dchroot)glaubitz@stadler:~/git/t$ ls -l chainlinttmp
total 32
-rw-r--r-- 1 glaubitz glaubitz     0 May 20 09:54 actual
-rw-r--r-- 1 glaubitz glaubitz 14797 May 20 09:54 expect
-rw-r--r-- 1 glaubitz glaubitz 15086 May 20 09:54 tests
(sid_sparc64-dchroot)glaubitz@stadler:~/git/t$

> For me the output using the latest "master" branch is:
> 
>     % ls -l chainlinttmp
>     -rw-------  1 me  me  14797 May 20 05:26 actual
>     -rw-------  1 me  me  14797 May 20 05:26 expect
>     -rw-------  1 me  me  15086 May 20 05:26 tests
> 
> My suspicion is that "actual" will have size 0 for you. If that's the
> case, I'd suggest running the commands from the "check-chainlint"
> target in t/Makefile manually one at a time to see if you can figure
> out which is failing. For instance:
> 
>     % ./chainlint.pl --emit-all chainlinttmp/tests

It does not generate any output for me:

(sid_sparc64-dchroot)glaubitz@stadler:~/git/t$ ./chainlint.pl --emit-all chainlinttmp/tests
(sid_sparc64-dchroot)glaubitz@stadler:~/git/t$

> If that produces output, then that's a good sign. Try capturing that
> output and feeding it to the `sed` command:
> 
>     % ./chainlint.pl --emit-all chainlinttmp/tests >lint.out
>     % sed -e 's/^[1-9][0-9]* //' lint.out
> 
> If that produces output, then something else is going wrong (such as
> PERL_PATH or PERL_PATH_SQ being incorrect).

Adrian
Eric Sunshine May 20, 2024, 9:58 a.m. UTC | #6
On Mon, May 20, 2024 at 5:36 AM Eric Sunshine <sunshine@sunshineco.com> wrote:
> My suspicion is that "actual" will have size 0 for you. If that's the
> case, I'd suggest running the commands from the "check-chainlint"
> target in t/Makefile manually one at a time to see if you can figure
> out which is failing. For instance:
>
>     % ./chainlint.pl --emit-all chainlinttmp/tests

This reminded me of an earlier report in which chainlint.pl was
producing no output for "actual" on Linux for s390x processors. That
was due to /proc/cpuinfo producing results which the script wasn't
expecting. It was fixed by 1f51b77f4f (chainlint.pl: fix /proc/cpuinfo
regexp, 2022-11-22)[1]. At the time of that patch, I noted[2]:

    A separate problem is that chainlint.pl doesn't fall back to a
    sensible non-zero value if ncores() returns 0 (or some other
    nonsense value). That is, of course, outside the scope of the
    well-focused problem fix which this standalone patch addresses. I
    may end up submitting a fix separately to make it fall back
    sensibly.

which probably explains the behavior you're experiencing. To fix it,
we'll need to see the output you get from:

    cat /proc/cpuinfo

[1]: https://lore.kernel.org/git/pull.1385.git.git.1669148861635.gitgitgadget@gmail.com/
[2]: https://lore.kernel.org/git/CAPig+cQ6_7wf6C280Rqi7mcTCiQp-n5GiLWTPazfcUcGFeZi0g@mail.gmail.com/
John Paul Adrian Glaubitz May 20, 2024, 10:02 a.m. UTC | #7
Hi Eric,

On Mon, 2024-05-20 at 05:58 -0400, Eric Sunshine wrote:
> On Mon, May 20, 2024 at 5:36 AM Eric Sunshine <sunshine@sunshineco.com> wrote:
> > My suspicion is that "actual" will have size 0 for you. If that's the
> > case, I'd suggest running the commands from the "check-chainlint"
> > target in t/Makefile manually one at a time to see if you can figure
> > out which is failing. For instance:
> > 
> >     % ./chainlint.pl --emit-all chainlinttmp/tests
> 
> This reminded me of an earlier report in which chainlint.pl was
> producing no output for "actual" on Linux for s390x processors. That
> was due to /proc/cpuinfo producing results which the script wasn't
> expecting. It was fixed by 1f51b77f4f (chainlint.pl: fix /proc/cpuinfo
> regexp, 2022-11-22)[1]. At the time of that patch, I noted[2]:

Yeah, the output for /proc/cpuinfo is not standardized across architectures,
unfortunately.

>     A separate problem is that chainlint.pl doesn't fall back to a
>     sensible non-zero value if ncores() returns 0 (or some other
>     nonsense value). That is, of course, outside the scope of the
>     well-focused problem fix which this standalone patch addresses. I
>     may end up submitting a fix separately to make it fall back
>     sensibly.
> 
> which probably explains the behavior you're experiencing. To fix it,
> we'll need to see the output you get from:
> 
>     cat /proc/cpuinfo

Here you go:

glaubitz@stadler:~$ cat /proc/cpuinfo 
cpu             : UltraSparc T4 (Niagara4)
fpu             : UltraSparc T4 integrated FPU
pmu             : niagara4
prom            : OBP 4.38.16 2018/11/28 07:24
type            : sun4v
ncpus probed    : 48
ncpus active    : 48
D$ parity tl1   : 0
I$ parity tl1   : 0
cpucaps         :
flush,stbar,swap,muldiv,v9,blkinit,n2,mul32,div32,v8plus,popc,vis,vis2,ASIBlkInit,fmaf,vis3,hpc,ima,pause,cbcond,aes,des,kasumi,camellia,md5,sha1,sha256,sha512,mpmul,montmul,montsqr,crc32c
Cpu0ClkTck      : 00000000a9beeee4
Cpu1ClkTck      : 00000000a9beeee4
Cpu2ClkTck      : 00000000a9beeee4
Cpu3ClkTck      : 00000000a9beeee4
Cpu4ClkTck      : 00000000a9beeee4
Cpu5ClkTck      : 00000000a9beeee4
Cpu6ClkTck      : 00000000a9beeee4
Cpu7ClkTck      : 00000000a9beeee4
Cpu8ClkTck      : 00000000a9beeee4
Cpu9ClkTck      : 00000000a9beeee4
Cpu10ClkTck     : 00000000a9beeee4
Cpu11ClkTck     : 00000000a9beeee4
Cpu12ClkTck     : 00000000a9beeee4
Cpu13ClkTck     : 00000000a9beeee4
Cpu14ClkTck     : 00000000a9beeee4
Cpu15ClkTck     : 00000000a9beeee4
Cpu16ClkTck     : 00000000a9beeee4
Cpu17ClkTck     : 00000000a9beeee4
Cpu18ClkTck     : 00000000a9beeee4
Cpu19ClkTck     : 00000000a9beeee4
Cpu20ClkTck     : 00000000a9beeee4
Cpu21ClkTck     : 00000000a9beeee4
Cpu22ClkTck     : 00000000a9beeee4
Cpu23ClkTck     : 00000000a9beeee4
Cpu24ClkTck     : 00000000a9beeee4
Cpu25ClkTck     : 00000000a9beeee4
Cpu26ClkTck     : 00000000a9beeee4
Cpu27ClkTck     : 00000000a9beeee4
Cpu28ClkTck     : 00000000a9beeee4
Cpu29ClkTck     : 00000000a9beeee4
Cpu30ClkTck     : 00000000a9beeee4
Cpu31ClkTck     : 00000000a9beeee4
Cpu32ClkTck     : 00000000a9beeee4
Cpu33ClkTck     : 00000000a9beeee4
Cpu34ClkTck     : 00000000a9beeee4
Cpu35ClkTck     : 00000000a9beeee4
Cpu36ClkTck     : 00000000a9beeee4
Cpu37ClkTck     : 00000000a9beeee4
Cpu38ClkTck     : 00000000a9beeee4
Cpu39ClkTck     : 00000000a9beeee4
Cpu40ClkTck     : 00000000a9beeee4
Cpu41ClkTck     : 00000000a9beeee4
Cpu42ClkTck     : 00000000a9beeee4
Cpu43ClkTck     : 00000000a9beeee4
Cpu44ClkTck     : 00000000a9beeee4
Cpu45ClkTck     : 00000000a9beeee4
Cpu46ClkTck     : 00000000a9beeee4
Cpu47ClkTck     : 00000000a9beeee4
MMU Type        : Hypervisor (sun4v)
MMU PGSZs       : 8K,64K,4MB,256MB,2GB
State:
CPU0:           online
CPU1:           online
CPU2:           online
CPU3:           online
CPU4:           online
CPU5:           online
CPU6:           online
CPU7:           online
CPU8:           online
CPU9:           online
CPU10:          online
CPU11:          online
CPU12:          online
CPU13:          online
CPU14:          online
CPU15:          online
CPU16:          online
CPU17:          online
CPU18:          online
CPU19:          online
CPU20:          online
CPU21:          online
CPU22:          online
CPU23:          online
CPU24:          online
CPU25:          online
CPU26:          online
CPU27:          online
CPU28:          online
CPU29:          online
CPU30:          online
CPU31:          online
CPU32:          online
CPU33:          online
CPU34:          online
CPU35:          online
CPU36:          online
CPU37:          online
CPU38:          online
CPU39:          online
CPU40:          online
CPU41:          online
CPU42:          online
CPU43:          online
CPU44:          online
CPU45:          online
CPU46:          online
CPU47:          online
glaubitz@stadler:~$

Adrian
John Paul Adrian Glaubitz May 20, 2024, 11:02 a.m. UTC | #8
Hi Eric,

On Mon, 2024-05-20 at 12:02 +0200, John Paul Adrian Glaubitz wrote:
> > which probably explains the behavior you're experiencing. To fix it,
> > we'll need to see the output you get from:
> > 
> >     cat /proc/cpuinfo
> 
> Here you go:
> 
> glaubitz@stadler:~$ cat /proc/cpuinfo 
> cpu             : UltraSparc T4 (Niagara4)
> fpu             : UltraSparc T4 integrated FPU
> pmu             : niagara4
> prom            : OBP 4.38.16 2018/11/28 07:24
> type            : sun4v
> ncpus probed    : 48
> ncpus active    : 48
> D$ parity tl1   : 0
> I$ parity tl1   : 0
> cpucaps         :
> flush,stbar,swap,muldiv,v9,blkinit,n2,mul32,div32,v8plus,popc,vis,vis2,ASIBlkInit,fmaf,vis3,hpc,ima,pause,cbcond,aes,des,kasumi,camellia,md5,sha1,sha256,sha512,mpmul,montmul,montsqr,crc32c
> Cpu0ClkTck      : 00000000a9beeee4
> Cpu1ClkTck      : 00000000a9beeee4
> (...)
> State:
> CPU0:           online
> CPU1:           online
> CPU2:           online
> CPU3:           online

In order to verify this theory, I made the following temporary change:

diff --git a/t/chainlint.pl b/t/chainlint.pl
index 556ee91a15..63cac942ac 100755
--- a/t/chainlint.pl
+++ b/t/chainlint.pl
@@ -718,7 +718,7 @@ sub ncores {
        # Windows
        return $ENV{NUMBER_OF_PROCESSORS} if exists($ENV{NUMBER_OF_PROCESSORS});
        # Linux / MSYS2 / Cygwin / WSL
-       do { local @ARGV='/proc/cpuinfo'; return scalar(grep(/^processor[\s\d]*:/, <>)); } if -r '/proc/cpuinfo';
+       do { local @ARGV='/proc/cpuinfo'; return scalar(grep(/^processor[\s\d]*:||^CPU[\d]*:/, <>)); } if -r '/proc/cpuinfo';
        # macOS & BSD
        return qx/sysctl -n hw.ncpu/ if $^O =~ /(?:^darwin$|bsd)/;
        return 1;

and I confirm that this fixes the problem.

Let me whip up a patch and post it here.

Adrian
Junio C Hamano May 20, 2024, 4:04 p.m. UTC | #9
John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> writes:

> Hi Eric,
> ...
> In order to verify this theory, I made the following temporary change:
>
> diff --git a/t/chainlint.pl b/t/chainlint.pl
> index 556ee91a15..63cac942ac 100755
> --- a/t/chainlint.pl
> +++ b/t/chainlint.pl
> @@ -718,7 +718,7 @@ sub ncores {
>         # Windows
>         return $ENV{NUMBER_OF_PROCESSORS} if exists($ENV{NUMBER_OF_PROCESSORS});
>         # Linux / MSYS2 / Cygwin / WSL
> -       do { local @ARGV='/proc/cpuinfo'; return scalar(grep(/^processor[\s\d]*:/, <>)); } if -r '/proc/cpuinfo';
> +       do { local @ARGV='/proc/cpuinfo'; return scalar(grep(/^processor[\s\d]*:||^CPU[\d]*:/, <>)); } if -r '/proc/cpuinfo';
>         # macOS & BSD
>         return qx/sysctl -n hw.ncpu/ if $^O =~ /(?:^darwin$|bsd)/;
>         return 1;
>
> and I confirm that this fixes the problem.
>
> Let me whip up a patch and post it here.

Thanks for working so well together.  Very much appreciated.
John Paul Adrian Glaubitz May 20, 2024, 4:07 p.m. UTC | #10
On Mon, 2024-05-20 at 09:04 -0700, Junio C Hamano wrote:
> John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> writes:
> 
> > Hi Eric,
> > ...
> > In order to verify this theory, I made the following temporary change:
> > 
> > diff --git a/t/chainlint.pl b/t/chainlint.pl
> > index 556ee91a15..63cac942ac 100755
> > --- a/t/chainlint.pl
> > +++ b/t/chainlint.pl
> > @@ -718,7 +718,7 @@ sub ncores {
> >         # Windows
> >         return $ENV{NUMBER_OF_PROCESSORS} if exists($ENV{NUMBER_OF_PROCESSORS});
> >         # Linux / MSYS2 / Cygwin / WSL
> > -       do { local @ARGV='/proc/cpuinfo'; return scalar(grep(/^processor[\s\d]*:/, <>)); } if -r '/proc/cpuinfo';
> > +       do { local @ARGV='/proc/cpuinfo'; return scalar(grep(/^processor[\s\d]*:||^CPU[\d]*:/, <>)); } if -r '/proc/cpuinfo';
> >         # macOS & BSD
> >         return qx/sysctl -n hw.ncpu/ if $^O =~ /(?:^darwin$|bsd)/;
> >         return 1;
> > 
> > and I confirm that this fixes the problem.
> > 
> > Let me whip up a patch and post it here.
> 
> Thanks for working so well together.  Very much appreciated.

Thanks for the praise! I very much enjoyed it.

Adrian
diff mbox series

Patch

--- chainlinttmp/expect	2024-05-19 12:26:50.051507198 +0000
+++ chainlinttmp/actual	2024-05-19 12:26:50.051507198 +0000
@@ -1,955 +0,0 @@ 
-# chainlint: chainlinttmp/tests
-# chainlint: arithmetic-expansion
-(
-	foo &&
-	bar=$((42 + 1)) &&
-	baz
-) &&
-(
-	bar=$((42 + 1)) ?!AMP?!
-	baz
-)
-# chainlint: bash-array
-(
-	foo &&
-	bar=(gumbo stumbo wumbo) &&
-	baz
-) &&
-(
-	foo &&
-	bar=${#bar[@]} &&
-	baz
-)
-# chainlint: blank-line
-(
-
-	nothing &&
-
-	something
-
-
-)
-# chainlint: blank-line-before-esac
-test_done () {
-	case "$test_failure" in
-	0)
-		test_at_end_hook_
-
-		exit 0 ;;
-
-	*)
-		if test $test_external_has_tap -eq 0
-		then
-			say_color error "# failed $test_failure among $msg"
-			say "1..$test_count"
-		fi
-
-		exit 1 ;;
-
-	esac
-}
-# chainlint: block
-(
-	foo &&
-	{
-		echo a ?!AMP?!
-		echo b
-	} &&
-	bar &&
-	{
-		echo c
-	} ?!AMP?!
-	baz
-) &&
-
-{
-	echo a; ?!AMP?! echo b
-} &&
-{ echo a; ?!AMP?! echo b; } &&
-
-{
-	echo "${var}9" &&
-	echo "done"
-} &&
-finis
-# chainlint: block-comment
-(
-	{
-		# show a
-		echo a &&
-		# show b
-		echo b
-	}
-)
-# chainlint: broken-chain
-(
-	foo &&
-	bar ?!AMP?!
-	baz &&
-	wop
-)
-# chainlint: case
-(
-	case "$x" in
-	x) foo ;;
-	*) bar ;;
-	esac &&
-	foobar
-) &&
-(
-	case "$x" in
-	x) foo ;;
-	*) bar ;;
-	esac ?!AMP?!
-	foobar
-) &&
-(
-	case "$x" in 1) true;; esac &&
-	case "$y" in 2) false;; esac ?!AMP?!
-	foobar
-)
-# chainlint: case-comment
-(
-	case "$x" in
-	# found foo
-	x) foo ;;
-	# found other
-	*)
-		# treat it as bar
-		bar
-		;;
-	esac
-)
-# chainlint: chain-break-background
-JGIT_DAEMON_PID= &&
-git init --bare empty.git &&
->empty.git/git-daemon-export-ok &&
-mkfifo jgit_daemon_output &&
-{
-	jgit daemon --port="$JGIT_DAEMON_PORT" . >jgit_daemon_output &
-	JGIT_DAEMON_PID=$!
-} &&
-test_expect_code 2 git ls-remote --exit-code git://localhost:$JGIT_DAEMON_PORT/empty.git
-# chainlint: chain-break-continue
-git ls-tree --name-only -r refs/notes/many_notes |
-while read path
-do
-	test "$path" = "foobar/non-note.txt" && continue
-	test "$path" = "deadbeef" && continue
-	test "$path" = "de/adbeef" && continue
-
-	if test $(expr length "$path") -ne $hexsz
-	then
-		return 1
-	fi
-done
-# chainlint: chain-break-false
-if condition not satisified
-then
-	echo it did not work...
-	echo failed!
-	false
-else
-	echo it went okay ?!AMP?!
-	congratulate user
-fi
-# chainlint: chain-break-return-exit
-case "$(git ls-files)" in
-one) echo pass one ;;
-*) echo bad one; return 1 ;;
-esac &&
-(
-	case "$(git ls-files)" in
-	two) echo pass two ;;
-	*) echo bad two; exit 1 ;;
-	esac
-) &&
-case "$(git ls-files)" in
-dir/two"$LF"one) echo pass both ;;
-*) echo bad; return 1 ;;
-esac &&
-
-for i in 1 2 3 4 ; do
-	git checkout main -b $i || return $?
-	test_commit $i $i $i tag$i || return $?
-done
-# chainlint: chain-break-status
-OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 ) &&
-test_match_signal 13 "$OUT" &&
-
-{ test-tool sigchain >actual; ret=$?; } &&
-{
-	test_match_signal 15 "$ret" ||
-	test "$ret" = 3
-} &&
-test_cmp expect actual
-# chainlint: chained-block
-echo nobody home && {
-	test the doohicky ?!AMP?!
-	right now
-} &&
-
-GIT_EXTERNAL_DIFF=echo git diff | {
-	read path oldfile oldhex oldmode newfile newhex newmode &&
-	test "z$oh" = "z$oldhex"
-}
-# chainlint: chained-subshell
-mkdir sub && (
-	cd sub &&
-	foo the bar ?!AMP?!
-	nuff said
-) &&
-
-cut "-d " -f actual | (read s1 s2 s3 &&
-test -f $s1 ?!AMP?!
-test $(cat $s2) = tree2path1 &&
-test $(cat $s3) = tree3path1)
-# chainlint: close-nested-and-parent-together
-(cd foo &&
-	(bar &&
-		baz))
-# chainlint: close-subshell
-(
-	foo
-) &&
-(
-	bar
-) >out &&
-(
-	baz
-) 2>err &&
-(
-	boo
-) <input &&
-(
-	bip
-) | wuzzle &&
-(
-	bop
-) | fazz \
-	fozz &&
-(
-	bup
-) |
-fuzzle &&
-(
-	yop
-)
-# chainlint: command-substitution
-(
-	foo &&
-	bar=$(gobble) &&
-	baz
-) &&
-(
-	bar=$(gobble blocks) ?!AMP?!
-	baz
-)
-# chainlint: command-substitution-subsubshell
-OUT=$( ((large_git 1>&3) | :) 3>&1 ) &&
-test_match_signal 13 "$OUT"
-# chainlint: comment
-(
-	# comment 1
-	nothing &&
-	# comment 2
-	something
-	# comment 3
-	# comment 4
-)
-# chainlint: complex-if-in-cuddled-loop
-(for i in a b c; do
-   if test "$(echo $(waffle bat))" = "eleventeen" &&
-     test "$x" = "$y"; then
-     :
-   else
-     echo >file
-   fi ?!LOOP?!
- done) &&
-test ! -f file
-# chainlint: cuddled
-(cd foo &&
-	bar
-) &&
-
-(cd foo ?!AMP?!
-	bar
-) &&
-
-(
-	cd foo &&
-	bar) &&
-
-(cd foo &&
-	bar) &&
-
-(cd foo ?!AMP?!
-	bar)
-# chainlint: cuddled-if-then-else
-(if test -z ""; then
-    echo empty
- else
-    echo bizzy
- fi) &&
-echo foobar
-# chainlint: cuddled-loop
-( while read x
-  do foobar bop || exit 1
-  done <file ) &&
-outside subshell
-# chainlint: double-here-doc
-run_sub_test_lib_test_err run-inv-range-start \
-	"--run invalid range start" \
-	--run="a-5" <<-\EOF &&
-test_expect_success "passing test #1" "true"
-test_done
-EOF
-check_sub_test_lib_test_err run-inv-range-start \
-	<<-\EOF_OUT 3<<-EOF_ERR
-> FATAL: Unexpected exit with code 1
-EOF_OUT
-> error: --run: invalid non-numeric in range start: ${SQ}a-5${SQ}
-EOF_ERR
-# chainlint: dqstring-line-splice
-
-echo 'fatal: reword option of --fixup is mutually exclusive with'	'--patch/--interactive/--all/--include/--only' >expect &&
-test_must_fail git commit --fixup=reword:HEAD~ $1 2>actual &&
-test_cmp expect actual
-
-# chainlint: dqstring-no-interpolate
-grep "^ ! [rejected][ ]*$BRANCH -> $BRANCH (non-fast-forward)$" out &&
-
-grep "^\.git$" output.txt &&
-
-
-(
-	cd client$version &&
-	GIT_TEST_PROTOCOL_VERSION=$version git fetch-pack --no-progress .. $(cat ../input)
-) >output &&
-	cut -d ' ' -f 2 <output | sort >actual &&
-	test_cmp expect actual
-
-# chainlint: empty-here-doc
-git ls-tree $tree path >current &&
-cat >expected <<\EOF &&
-EOF
-test_output
-# chainlint: exclamation
-if ! condition; then echo nope; else yep; fi &&
-test_prerequisite !MINGW &&
-mail uucp!address &&
-echo !whatever!
-# chainlint: exit-loop
-(
-	for i in a b c
-	do
-		foo || exit 1
-		bar &&
-		baz
-	done
-) &&
-(
-	while true
-	do
-		foo || exit 1
-		bar &&
-		baz
-	done
-) &&
-(
-	i=0 &&
-	while test $i -lt 10
-	do
-		echo $i || exit
-		i=$(($i + 1))
-	done
-)
-# chainlint: exit-subshell
-(
-	foo || exit 1
-	bar &&
-	baz
-)
-# chainlint: for-loop
-(
-	for i in a b c
-	do
-		echo $i ?!AMP?!
-		cat <<-\EOF ?!LOOP?!
-		bar
-		EOF
-	done ?!AMP?!
-
-	for i in a b c; do
-		echo $i &&
-		cat $i ?!LOOP?!
-	done
-)
-# chainlint: for-loop-abbreviated
-for it
-do
-	path=$(expr "$it" : ([^:]*)) &&
-	git update-index --add "$path" || exit
-done
-# chainlint: function
-sha1_file() {
-	echo "$*" | sed "s#..#.git/objects/&/#"
-} &&
-
-remove_object() {
-	file=$(sha1_file "$*") &&
-	test -e "$file" ?!AMP?!
-	rm -f "$file"
-} ?!AMP?!
-
-sha1_file arg && remove_object arg
-# chainlint: here-doc
-boodle wobba \
-       gorgo snoot \
-       wafta snurb <<EOF &&
-quoth the raven,
-nevermore...
-EOF
-
-cat <<-Arbitrary_Tag_42 >foo &&
-snoz
-boz
-woz
-Arbitrary_Tag_42
-
-cat <<"zump" >boo &&
-snoz
-boz
-woz
-zump
-
-horticulture <<\EOF
-gomez
-morticia
-wednesday
-pugsly
-EOF
-# chainlint: here-doc-close-subshell
-(
-	cat <<-\INPUT)
-	fizz
-	INPUT
-# chainlint: here-doc-indent-operator
-cat >expect <<- EOF &&
-header: 43475048 1 $(test_oid oid_version) $NUM_CHUNKS 0
-num_commits: $1
-chunks: oid_fanout oid_lookup commit_metadata generation_data bloom_indexes bloom_data
-EOF
-
-cat >expect << -EOF ?!AMP?!
-this is not indented
--EOF
-
-cleanup
-# chainlint: here-doc-multi-line-command-subst
-(
-	x=$(bobble <<-\END &&
-		fossil
-		vegetable
-		END
-		wiffle) ?!AMP?!
-	echo $x
-)
-# chainlint: here-doc-multi-line-string
-(
-	cat <<-\TXT && echo "multi-line
-	string" ?!AMP?!
-	fizzle
-	TXT
-	bap
-)
-# chainlint: if-condition-split
-if bob &&
-   marcia ||
-   kevin
-then
-	echo "nomads" ?!AMP?!
-	echo "for sure"
-fi
-# chainlint: if-in-loop
-(
-	for i in a b c
-	do
-		if false
-		then
-			echo "err"
-			exit 1
-		fi ?!AMP?!
-		foo
-	done ?!AMP?!
-	bar
-)
-# chainlint: if-then-else
-(
-	if test -n ""
-	then
-		echo very ?!AMP?!
-		echo empty
-	elif test -z ""
-	then
-		echo foo
-	else
-		echo foo &&
-		cat <<-\EOF
-		bar
-		EOF
-	fi ?!AMP?!
-	echo poodle
-) &&
-(
-	if test -n ""; then
-		echo very &&
-		echo empty
-	fi
-)
-# chainlint: incomplete-line
-line 1 \
-line 2 \
-line 3 \
-line 4 &&
-(
-	line 5 \
-	line 6 \
-	line 7 \
-	line 8
-)
-# chainlint: inline-comment
-(
-	foobar && # comment 1
-	barfoo ?!AMP?! # wrong position for &&
-	flibble "not a # comment"
-) &&
-
-(cd foo &&
-	flibble "not a # comment")
-# chainlint: loop-detect-failure
-git init r1 &&
-for n in 1 2 3 4 5
-do
-	echo "This is file: $n" > r1/file.$n &&
-	git -C r1 add file.$n &&
-	git -C r1 commit -m "$n" || return 1
-done &&
-
-git init r2 &&
-for n in 1000 10000
-do
-	printf "%"$n"s" X > r2/large.$n &&
-	git -C r2 add large.$n &&
-	git -C r2 commit -m "$n" ?!LOOP?!
-done
-# chainlint: loop-detect-status
-(while test $i -le $blobcount
- do
-	printf "Generating blob $i/$blobcount\r" >&2 &&
-	printf "blob\nmark :$i\ndata $blobsize\n" &&
-	#test-tool genrandom $i $blobsize &&
-	printf "%-${blobsize}s" $i &&
-	echo "M 100644 :$i $i" >> commit &&
-	i=$(($i+1)) ||
-	echo $? > exit-status
- done &&
- echo "commit refs/heads/main" &&
- echo "author A U Thor <author@email.com> 123456789 +0000" &&
- echo "committer C O Mitter <committer@email.com> 123456789 +0000" &&
- echo "data 5" &&
- echo ">2gb" &&
- cat commit) |
-git fast-import --big-file-threshold=2 &&
-test ! -f exit-status
-# chainlint: loop-in-if
-(
-	if true
-	then
-		while true
-		do
-			echo "pop" ?!AMP?!
-			echo "glup" ?!LOOP?!
-		done ?!AMP?!
-		foo
-	fi ?!AMP?!
-	bar
-)
-# chainlint: loop-upstream-pipe
-(
-	git rev-list --objects --no-object-names base..loose |
-	while read oid
-	do
-		path="$objdir/$(test_oid_to_path "$oid")" &&
-		printf "%s %d\n" "$oid" "$(test-tool chmtime --get "$path")" ||
-		echo "object list generation failed for $oid"
-	done |
-	sort -k1
-) >expect &&
-# chainlint: multi-line-nested-command-substitution
-(
-	foo &&
-	x=$(
-		echo bar |
-		cat
-	) &&
-	echo ok
-) |
-sort &&
-(
-	bar &&
-	x=$(echo bar |
-		cat
-	) &&
-	y=$(echo baz |
-		fip) &&
-	echo fail
-)
-# chainlint: multi-line-string
-(
-	x="line 1
-		line 2
-		line 3" &&
-	y="line 1
-		line2" ?!AMP?!
-	foobar
-) &&
-(
-	echo "xyz" "abc
-		def
-		ghi" &&
-	barfoo
-)
-# chainlint: negated-one-liner
-! (foo && bar) &&
-! (foo && bar) >baz &&
-
-! (foo; ?!AMP?! bar) &&
-! (foo; ?!AMP?! bar) >baz
-# chainlint: nested-cuddled-subshell
-(
-	(cd foo &&
-		bar
-	) &&
-
-	(cd foo &&
-		bar
-	) ?!AMP?!
-
-	(
-		cd foo &&
-		bar) &&
-
-	(
-		cd foo &&
-		bar) ?!AMP?!
-
-	(cd foo &&
-		bar) &&
-
-	(cd foo &&
-		bar) ?!AMP?!
-
-	foobar
-)
-# chainlint: nested-here-doc
-cat <<ARBITRARY >foop &&
-naddle
-fub <<EOF
-	nozzle
-	noodle
-EOF
-formp
-ARBITRARY
-
-(
-	cat <<-\INPUT_END &&
-	fish are mice
-	but geese go slow
-	data <<EOF
-		perl is lerp
-		and nothing else
-	EOF
-	toink
-	INPUT_END
-
-	cat <<-\EOT ?!AMP?!
-	text goes here
-	data <<EOF
-		data goes here
-	EOF
-	more test here
-	EOT
-
-	foobar
-)
-# chainlint: nested-loop-detect-failure
-for i in 0 1 2 3 4 5 6 7 8 9;
-do
-	for j in 0 1 2 3 4 5 6 7 8 9;
-	do
-		echo "$i$j" >"path$i$j" ?!LOOP?!
-	done ?!LOOP?!
-done &&
-
-for i in 0 1 2 3 4 5 6 7 8 9;
-do
-	for j in 0 1 2 3 4 5 6 7 8 9;
-	do
-		echo "$i$j" >"path$i$j" || return 1
-	done
-done &&
-
-for i in 0 1 2 3 4 5 6 7 8 9;
-do
-	for j in 0 1 2 3 4 5 6 7 8 9;
-	do
-		echo "$i$j" >"path$i$j" ?!LOOP?!
-	done || return 1
-done &&
-
-for i in 0 1 2 3 4 5 6 7 8 9;
-do
-	for j in 0 1 2 3 4 5 6 7 8 9;
-	do
-		echo "$i$j" >"path$i$j" || return 1
-	done || return 1
-done
-# chainlint: nested-subshell
-(
-	cd foo &&
-	(
-		echo a &&
-		echo b
-	) >file &&
-
-	cd foo &&
-	(
-		echo a ?!AMP?!
-		echo b
-	) >file
-)
-# chainlint: nested-subshell-comment
-(
-	foo &&
-	(
-		bar &&
-		# bottles wobble while fiddles gobble
-		# minor numbers of cows (or do they?)
-		baz &&
-		snaff
-	) ?!AMP?!
-	fuzzy
-)
-# chainlint: not-heredoc
-echo "<<<<<<< ours" &&
-echo ourside &&
-echo "=======" &&
-echo theirside &&
-echo ">>>>>>> theirs" &&
-
-(
-	echo "<<<<<<< ours" &&
-	echo ourside &&
-	echo "=======" &&
-	echo theirside &&
-	echo ">>>>>>> theirs" ?!AMP?!
-	poodle
-) >merged
-# chainlint: one-liner
-(foo && bar) &&
-(foo && bar) |
-(foo && bar) >baz &&
-
-(foo; ?!AMP?! bar) &&
-(foo; ?!AMP?! bar) |
-(foo; ?!AMP?! bar) >baz &&
-
-(foo "bar; baz")
-# chainlint: one-liner-for-loop
-git init dir-rename-and-content &&
-(
-	cd dir-rename-and-content &&
-	test_write_lines 1 2 3 4 5 >foo &&
-	mkdir olddir &&
-	for i in a b c; do echo $i >olddir/$i; ?!LOOP?! done ?!AMP?!
-	git add foo olddir &&
-	git commit -m "original" &&
-)
-# chainlint: p4-filespec
-(
-	p4 print -1 //depot/fiddle#42 >file &&
-	foobar
-)
-# chainlint: pipe
-(
-	foo |
-	bar |
-	baz &&
-
-	fish |
-	cow ?!AMP?!
-
-	sunder
-)
-# chainlint: return-loop
-while test $i -lt $((num - 5))
-do
-	git notes add -m "notes for commit$i" HEAD~$i || return 1
-	i=$((i + 1))
-done
-# chainlint: semicolon
-(
-	cat foo ; ?!AMP?! echo bar ?!AMP?!
-	cat foo ; ?!AMP?! echo bar
-) &&
-(
-	cat foo ; ?!AMP?! echo bar &&
-	cat foo ; ?!AMP?! echo bar
-) &&
-(
-	echo "foo; bar" &&
-	cat foo; ?!AMP?! echo bar
-) &&
-(
-	foo;
-) &&
-(cd foo &&
-	for i in a b c; do
-		echo; ?!LOOP?!
-	done)
-# chainlint: sqstring-in-sqstring
-perl -e '
-	defined($_ = -s $_) or die for @ARGV;
-	exit 1 if $ARGV[0] <= $ARGV[1];
-' test-2-$packname_2.pack test-3-$packname_3.pack
-# chainlint: subshell-here-doc
-(
-	echo wobba \
-	       gorgo snoot \
-	       wafta snurb <<-EOF &&
-	quoth the raven,
-	nevermore...
-	EOF
-
-	cat <<EOF >bip ?!AMP?!
-	fish fly high
-EOF
-
-	echo <<-\EOF >bop
-	gomez
-	morticia
-	wednesday
-	pugsly
-	EOF
-) &&
-(
-	cat <<-\ARBITRARY >bup &&
-	glink
-	FIZZ
-	ARBITRARY
-	cat <<-"ARBITRARY3" >bup3 &&
-	glink
-	FIZZ
-	ARBITRARY3
-	meep
-)
-# chainlint: subshell-one-liner
-(
-	(foo && bar) &&
-	(foo && bar) |
-	(foo && bar) >baz &&
-
-	(foo; ?!AMP?! bar) &&
-	(foo; ?!AMP?! bar) |
-	(foo; ?!AMP?! bar) >baz &&
-
-	(foo || exit 1) &&
-	(foo || exit 1) |
-	(foo || exit 1) >baz &&
-
-	(foo && bar) ?!AMP?!
-
-	(foo && bar; ?!AMP?! baz) ?!AMP?!
-
-	foobar
-)
-# chainlint: t7900-subtree
-(
-	chks="sub1
-sub2
-sub3
-sub4" &&
-	chks_sub=$(cat <<TXT | sed "s,^,sub dir/,"
-$chks
-TXT
-) &&
-	chkms="main-sub1
-main-sub2
-main-sub3
-main-sub4" &&
-	chkms_sub=$(cat <<TXT | sed "s,^,sub dir/,"
-$chkms
-TXT
-) &&
-
-	subfiles=$(git ls-files) &&
-	check_equal "$subfiles" "$chkms
-$chks"
-)
-# chainlint: token-pasting
-git config filter.rot13.smudge ./rot13.sh &&
-git config filter.rot13.clean ./rot13.sh &&
-
-{
-    echo "*.t filter=rot13" ?!AMP?!
-    echo "*.i ident"
-} >.gitattributes &&
-
-{
-    echo a b c d e f g h i j k l m ?!AMP?!
-    echo n o p q r s t u v w x y z ?!AMP?!
-    echo '$Id$'
-} >test &&
-cat test >test.t &&
-cat test >test.o &&
-cat test >test.i &&
-git add test test.t test.i &&
-rm -f test test.t test.i &&
-git checkout -- test test.t test.i &&
-
-echo "content-test2" >test2.o &&
-echo "content-test3 - filename with special characters" >"test3 'sq',$x=.o" ?!AMP?!
-
-downstream_url_for_sed=$(
-	printf "%sn" "$downstream_url" |
-	sed -e 's/\/\\/g' -e 's/[[/.*^$]/\&/g'
-)
-# chainlint: unclosed-here-doc
-command_which_is_run &&
-cat >expect <<\EOF ?!UNCLOSED-HEREDOC?! &&
-	we try to end the here-doc below,
-	but the indentation throws us off
-	since the operator is not "<<-".
-	EOF
-command_which_is_gobbled
-# chainlint: unclosed-here-doc-indent
-command_which_is_run &&
-cat >expect <<-\EOF ?!UNCLOSED-HEREDOC?! &&
-we forget to end the here-doc
-command_which_is_gobbled
-# chainlint: while-loop
-(
-	while true
-	do
-		echo foo ?!AMP?!
-		cat <<-\EOF ?!LOOP?!
-		bar
-		EOF
-	done ?!AMP?!
-
-	while true; do
-		echo foo &&
-		cat bar ?!LOOP?!
-	done
-)