diff mbox series

t0025 flakey?

Message ID nycvar.QRO.7.76.6.1902061123410.41@tvgsbejvaqbjf.bet (mailing list archive)
State New, archived
Headers show
Series t0025 flakey? | expand

Commit Message

Johannes Schindelin Feb. 6, 2019, 10:25 a.m. UTC
Hi Torsten,

at first I thought that those intermittent test failures were limited to
Windows, but they are not: I can see it now in a build on 32-bit Linux.
Full logs here:

https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab

Excerpt from the failing test case:

-- snip --
not ok 2 - renormalize CRLF in repo
 expecting success: 
	echo "*.txt text=auto" >.gitattributes &&
	git add --renormalize "*.txt" &&
	cat >expect <<-\EOF &&
	i/lf w/crlf attr/text=auto CRLF.txt
	i/lf w/lf attr/text=auto LF.txt
	i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
	EOF
	git ls-files --eol |
	sed -e "s/	/ /g" -e "s/  */ /g" |
	sort >actual &&
	test_cmp expect actual

+ echo *.txt text=auto
+ git add --renormalize *.txt
+ cat
+ sort
+ sed -e s/	/ /g -e s/  */ /g
+ git ls-files --eol
+ test_cmp expect actual
+ diff -u expect actual
error: last command exited with $?=1
-- snap --

Any ideas?
Johannes

Comments

SZEDER Gábor Feb. 6, 2019, 10:42 a.m. UTC | #1
On Wed, Feb 06, 2019 at 11:25:38AM +0100, Johannes Schindelin wrote:
> Hi Torsten,
> 
> at first I thought that those intermittent test failures were limited to
> Windows, but they are not: I can see it now in a build on 32-bit Linux.
> Full logs here:
> 
> https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab
> 
> Excerpt from the failing test case:
> 
> -- snip --
> not ok 2 - renormalize CRLF in repo
>  expecting success: 
> 	echo "*.txt text=auto" >.gitattributes &&
> 	git add --renormalize "*.txt" &&
> 	cat >expect <<-\EOF &&
> 	i/lf w/crlf attr/text=auto CRLF.txt
> 	i/lf w/lf attr/text=auto LF.txt
> 	i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
> 	EOF
> 	git ls-files --eol |
> 	sed -e "s/	/ /g" -e "s/  */ /g" |
> 	sort >actual &&
> 	test_cmp expect actual
> 
> + echo *.txt text=auto
> + git add --renormalize *.txt
> + cat
> + sort
> + sed -e s/	/ /g -e s/  */ /g
> + git ls-files --eol
> + test_cmp expect actual
> + diff -u expect actual
> --- expect	2019-02-06 09:39:42.080733629 +0000
> +++ actual	2019-02-06 09:39:42.088733629 +0000
> @@ -1,3 +1,3 @@
> -i/lf w/crlf attr/text=auto CRLF.txt
> +i/crlf w/crlf attr/text=auto CRLF.txt
>  i/lf w/lf attr/text=auto LF.txt
> -i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
> +i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt
> error: last command exited with $?=1
> -- snap --
> 
> Any ideas?

I reported this and Peff looked into it on the way to Git Merge, but
not working solution yet.

https://public-inbox.org/git/20190129225121.GD1895@sigill.intra.peff.net/T/#u
Johannes Schindelin Feb. 6, 2019, 1:52 p.m. UTC | #2
Hi Gábor,

On Wed, 6 Feb 2019, SZEDER Gábor wrote:

> On Wed, Feb 06, 2019 at 11:25:38AM +0100, Johannes Schindelin wrote:
> 
> > at first I thought that those intermittent test failures were limited
> > to Windows, but they are not: I can see it now in a build on 32-bit
> > Linux.
> > Full logs here:
> > 
> > https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab
> > 
> > Excerpt from the failing test case:
> > 
> > -- snip --
> > not ok 2 - renormalize CRLF in repo
> >  expecting success: 
> > 	echo "*.txt text=auto" >.gitattributes &&
> > 	git add --renormalize "*.txt" &&
> > 	cat >expect <<-\EOF &&
> > 	i/lf w/crlf attr/text=auto CRLF.txt
> > 	i/lf w/lf attr/text=auto LF.txt
> > 	i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
> > 	EOF
> > 	git ls-files --eol |
> > 	sed -e "s/	/ /g" -e "s/  */ /g" |
> > 	sort >actual &&
> > 	test_cmp expect actual
> > 
> > + echo *.txt text=auto
> > + git add --renormalize *.txt
> > + cat
> > + sort
> > + sed -e s/	/ /g -e s/  */ /g
> > + git ls-files --eol
> > + test_cmp expect actual
> > + diff -u expect actual
> > --- expect	2019-02-06 09:39:42.080733629 +0000
> > +++ actual	2019-02-06 09:39:42.088733629 +0000
> > @@ -1,3 +1,3 @@
> > -i/lf w/crlf attr/text=auto CRLF.txt
> > +i/crlf w/crlf attr/text=auto CRLF.txt
> >  i/lf w/lf attr/text=auto LF.txt
> > -i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
> > +i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt
> > error: last command exited with $?=1
> > -- snap --
> > 
> > Any ideas?
> 
> I reported this and Peff looked into it on the way to Git Merge, but
> not working solution yet.
> 
> https://public-inbox.org/git/20190129225121.GD1895@sigill.intra.peff.net/T/#u

Thank you!
Dscho
Torsten Bögershausen Feb. 6, 2019, 5:15 p.m. UTC | #3
On Wed, Feb 06, 2019 at 02:52:53PM +0100, Johannes Schindelin wrote:
> Hi Gábor,
>
> On Wed, 6 Feb 2019, SZEDER Gábor wrote:
>
> > On Wed, Feb 06, 2019 at 11:25:38AM +0100, Johannes Schindelin wrote:
> >
> > > at first I thought that those intermittent test failures were limited
> > > to Windows, but they are not: I can see it now in a build on 32-bit
> > > Linux.
> > > Full logs here:
> > >
> > > https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab
> > >
> > > Excerpt from the failing test case:
> > >
> > > -- snip --
> > > not ok 2 - renormalize CRLF in repo
> > >  expecting success:
> > > 	echo "*.txt text=auto" >.gitattributes &&
> > > 	git add --renormalize "*.txt" &&
> > > 	cat >expect <<-\EOF &&
> > > 	i/lf w/crlf attr/text=auto CRLF.txt
> > > 	i/lf w/lf attr/text=auto LF.txt
> > > 	i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
> > > 	EOF
> > > 	git ls-files --eol |
> > > 	sed -e "s/	/ /g" -e "s/  */ /g" |
> > > 	sort >actual &&
> > > 	test_cmp expect actual
> > >
> > > + echo *.txt text=auto
> > > + git add --renormalize *.txt
> > > + cat
> > > + sort
> > > + sed -e s/	/ /g -e s/  */ /g
> > > + git ls-files --eol
> > > + test_cmp expect actual
> > > + diff -u expect actual
> > > --- expect	2019-02-06 09:39:42.080733629 +0000
> > > +++ actual	2019-02-06 09:39:42.088733629 +0000
> > > @@ -1,3 +1,3 @@
> > > -i/lf w/crlf attr/text=auto CRLF.txt
> > > +i/crlf w/crlf attr/text=auto CRLF.txt
> > >  i/lf w/lf attr/text=auto LF.txt
> > > -i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
> > > +i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt
> > > error: last command exited with $?=1
> > > -- snap --
> > >
> > > Any ideas?
> >
> > I reported this and Peff looked into it on the way to Git Merge, but
> > not working solution yet.
> >
> > https://public-inbox.org/git/20190129225121.GD1895@sigill.intra.peff.net/T/#u
>
> Thank you!
> Dscho

I shortly looked into the pointers here -
Is t0025 flaky after the fix from Peff:

[PATCH] add: use separate ADD_CACHE_RENORMALIZE flag

Or has it always been shaky ?
Does anybody know ?
SZEDER Gábor Feb. 6, 2019, 5:39 p.m. UTC | #4
On Wed, Feb 06, 2019 at 05:15:17PM +0000, Torsten Bögershausen wrote:
> On Wed, Feb 06, 2019 at 02:52:53PM +0100, Johannes Schindelin wrote:
> > Hi Gábor,
> >
> > On Wed, 6 Feb 2019, SZEDER Gábor wrote:
> >
> > > On Wed, Feb 06, 2019 at 11:25:38AM +0100, Johannes Schindelin wrote:
> > >
> > > > at first I thought that those intermittent test failures were limited
> > > > to Windows, but they are not: I can see it now in a build on 32-bit
> > > > Linux.
> > > > Full logs here:
> > > >
> > > > https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab
> > > >
> > > > Excerpt from the failing test case:
> > > >
> > > > -- snip --
> > > > not ok 2 - renormalize CRLF in repo
> > > >  expecting success:
> > > > 	echo "*.txt text=auto" >.gitattributes &&
> > > > 	git add --renormalize "*.txt" &&
> > > > 	cat >expect <<-\EOF &&
> > > > 	i/lf w/crlf attr/text=auto CRLF.txt
> > > > 	i/lf w/lf attr/text=auto LF.txt
> > > > 	i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
> > > > 	EOF
> > > > 	git ls-files --eol |
> > > > 	sed -e "s/	/ /g" -e "s/  */ /g" |
> > > > 	sort >actual &&
> > > > 	test_cmp expect actual
> > > >
> > > > + echo *.txt text=auto
> > > > + git add --renormalize *.txt
> > > > + cat
> > > > + sort
> > > > + sed -e s/	/ /g -e s/  */ /g
> > > > + git ls-files --eol
> > > > + test_cmp expect actual
> > > > + diff -u expect actual
> > > > --- expect	2019-02-06 09:39:42.080733629 +0000
> > > > +++ actual	2019-02-06 09:39:42.088733629 +0000
> > > > @@ -1,3 +1,3 @@
> > > > -i/lf w/crlf attr/text=auto CRLF.txt
> > > > +i/crlf w/crlf attr/text=auto CRLF.txt
> > > >  i/lf w/lf attr/text=auto LF.txt
> > > > -i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
> > > > +i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt
> > > > error: last command exited with $?=1
> > > > -- snap --
> > > >
> > > > Any ideas?
> > >
> > > I reported this and Peff looked into it on the way to Git Merge, but
> > > not working solution yet.
> > >
> > > https://public-inbox.org/git/20190129225121.GD1895@sigill.intra.peff.net/T/#u
> >
> > Thank you!
> > Dscho
> 
> I shortly looked into the pointers here -
> Is t0025 flaky after the fix from Peff:
> 
> [PATCH] add: use separate ADD_CACHE_RENORMALIZE flag
> 
> Or has it always been shaky ?
> Does anybody know ?

I sort-of bisected it, and it pointed to Peff's fix.

Running 't0025 --stress' in the merge of 'sg/stress-test' and
'jk/add-ignore-errors-bit-assignment-fix' results in a failure within
seconds or <20 repetitions.  In the merge with
'jk/add-ignore-errors-bit-assignment-fix^' it runs successfully for
thousands of repetitions.
Randall S. Becker Feb. 6, 2019, 6 p.m. UTC | #5
On February 6, 2019 12:15, Torsten Bögershausen wrote:
> To: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> Cc: SZEDER Gábor <szeder.dev@gmail.com>; Jeff King <peff@peff.net>;
> git@vger.kernel.org
> Subject: Re: t0025 flakey?
> 
> On Wed, Feb 06, 2019 at 02:52:53PM +0100, Johannes Schindelin wrote:
> > Hi Gábor,
> >
> > On Wed, 6 Feb 2019, SZEDER Gábor wrote:
> >
> > > On Wed, Feb 06, 2019 at 11:25:38AM +0100, Johannes Schindelin wrote:
> > >
> > > > at first I thought that those intermittent test failures were
> > > > limited to Windows, but they are not: I can see it now in a build
> > > > on 32-bit Linux.
> > > > Full logs here:
> > > >
> > > > https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032
> > > > &_a=summary&view=ms.vss-test-web.build-test-results-tab
> > > >
> > > > Excerpt from the failing test case:
> > > >
> > > > -- snip --
> > > > not ok 2 - renormalize CRLF in repo  expecting success:
> > > > 	echo "*.txt text=auto" >.gitattributes &&
> > > > 	git add --renormalize "*.txt" &&
> > > > 	cat >expect <<-\EOF &&
> > > > 	i/lf w/crlf attr/text=auto CRLF.txt
> > > > 	i/lf w/lf attr/text=auto LF.txt
> > > > 	i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
> > > > 	EOF
> > > > 	git ls-files --eol |
> > > > 	sed -e "s/	/ /g" -e "s/  */ /g" |
> > > > 	sort >actual &&
> > > > 	test_cmp expect actual
> > > >
> > > > + echo *.txt text=auto
> > > > + git add --renormalize *.txt
> > > > + cat
> > > > + sort
> > > > + sed -e s/	/ /g -e s/  */ /g
> > > > + git ls-files --eol
> > > > + test_cmp expect actual
> > > > + diff -u expect actual
> > > > --- expect	2019-02-06 09:39:42.080733629 +0000
> > > > +++ actual	2019-02-06 09:39:42.088733629 +0000
> > > > @@ -1,3 +1,3 @@
> > > > -i/lf w/crlf attr/text=auto CRLF.txt
> > > > +i/crlf w/crlf attr/text=auto CRLF.txt
> > > >  i/lf w/lf attr/text=auto LF.txt
> > > > -i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
> > > > +i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt
> > > > error: last command exited with $?=1
> > > > -- snap --
> > > >
> > > > Any ideas?
> > >
> > > I reported this and Peff looked into it on the way to Git Merge, but
> > > not working solution yet.
> > >
> > > https://public-inbox.org/git/20190129225121.GD1895@sigill.intra.peff
> > > .net/T/#u
> >
> > Thank you!
> > Dscho
> 
> I shortly looked into the pointers here - Is t0025 flaky after the fix
from Peff:
> 
> [PATCH] add: use separate ADD_CACHE_RENORMALIZE flag
> 
> Or has it always been shaky ?
> Does anybody know ?

The NonStop port has traditionally had issues with t0025, which we tended to
ignore because things did work. We wrote those off as bash issues in t0025
since they seemed to be corrected when we picked up a new bash version about
a year ago. I will keep monitoring this, particularly when 2.21 comes out.

Cheers,
Randall

-- Brief whoami:
 NonStop developer since approximately 211288444200000000
 UNIX developer since approximately 421664400
-- In my real life, I talk too much.
Randall S. Becker Feb. 7, 2019, 4:58 p.m. UTC | #6
On February 6, 2019 13:01, I wrote:
> On February 6, 2019 12:15, Torsten Bögershausen wrote:
> > To: Johannes Schindelin <Johannes.Schindelin@gmx.de>
> > Cc: SZEDER Gábor <szeder.dev@gmail.com>; Jeff King <peff@peff.net>;
> > git@vger.kernel.org
> > Subject: Re: t0025 flakey?
> >
> > On Wed, Feb 06, 2019 at 02:52:53PM +0100, Johannes Schindelin wrote:
> > > Hi Gábor,
> > >
> > > On Wed, 6 Feb 2019, SZEDER Gábor wrote:
> > >
> > > > On Wed, Feb 06, 2019 at 11:25:38AM +0100, Johannes Schindelin
> wrote:
> > > >
> > > > > at first I thought that those intermittent test failures were
> > > > > limited to Windows, but they are not: I can see it now in a
> > > > > build on 32-bit Linux.
> > > > > Full logs here:
> > > > >
> > > > > https://dev.azure.com/gitgitgadget/git/_build/results?buildId=10
> > > > > 32 &_a=summary&view=ms.vss-test-web.build-test-results-tab
> > > > >
> > > > > Excerpt from the failing test case:
> > > > >
> > > > > -- snip --
> > > > > not ok 2 - renormalize CRLF in repo  expecting success:
> > > > > 	echo "*.txt text=auto" >.gitattributes &&
> > > > > 	git add --renormalize "*.txt" &&
> > > > > 	cat >expect <<-\EOF &&
> > > > > 	i/lf w/crlf attr/text=auto CRLF.txt
> > > > > 	i/lf w/lf attr/text=auto LF.txt
> > > > > 	i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
> > > > > 	EOF
> > > > > 	git ls-files --eol |
> > > > > 	sed -e "s/	/ /g" -e "s/  */ /g" |
> > > > > 	sort >actual &&
> > > > > 	test_cmp expect actual
> > > > >
> > > > > + echo *.txt text=auto
> > > > > + git add --renormalize *.txt
> > > > > + cat
> > > > > + sort
> > > > > + sed -e s/	/ /g -e s/  */ /g
> > > > > + git ls-files --eol
> > > > > + test_cmp expect actual
> > > > > + diff -u expect actual
> > > > > --- expect	2019-02-06 09:39:42.080733629 +0000
> > > > > +++ actual	2019-02-06 09:39:42.088733629 +0000
> > > > > @@ -1,3 +1,3 @@
> > > > > -i/lf w/crlf attr/text=auto CRLF.txt
> > > > > +i/crlf w/crlf attr/text=auto CRLF.txt
> > > > >  i/lf w/lf attr/text=auto LF.txt -i/lf w/mixed attr/text=auto
> > > > > CRLF_mix_LF.txt
> > > > > +i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt
> > > > > error: last command exited with $?=1
> > > > > -- snap --
> > > > >
> > > > > Any ideas?
> > > >
> > > > I reported this and Peff looked into it on the way to Git Merge,
> > > > but not working solution yet.
> > > >
> > > > https://public-inbox.org/git/20190129225121.GD1895@sigill.intra.pe
> > > > ff
> > > > .net/T/#u
> > >
> > > Thank you!
> > > Dscho
> >
> > I shortly looked into the pointers here - Is t0025 flaky after the fix
from
> Peff:
> >
> > [PATCH] add: use separate ADD_CACHE_RENORMALIZE flag
> >
> > Or has it always been shaky ?
> > Does anybody know ?
> 
> The NonStop port has traditionally had issues with t0025, which we tended
> to ignore because things did work. We wrote those off as bash issues in
> t0025 since they seemed to be corrected when we picked up a new bash
> version about a year ago. I will keep monitoring this, particularly when
2.21
> comes out.

FYI: t0020-t0027 all passed on the NonStop port for 2.21.0-rc0 - so no
issues for us on this one.

Cheers,
Randall

-- Brief whoami:
 NonStop developer since approximately 211288444200000000
 UNIX developer since approximately 421664400
-- In my real life, I talk too much.
Junio C Hamano Feb. 7, 2019, 5:39 p.m. UTC | #7
"Randall S. Becker" <rsbecker@nexbridge.com> writes:

> On February 6, 2019 13:01, I wrote:
>> On February 6, 2019 12:15, Torsten Bögershausen wrote:
>> > To: Johannes Schindelin <Johannes.Schindelin@gmx.de>
>> > ...
>> > [PATCH] add: use separate ADD_CACHE_RENORMALIZE flag
>> >
>> > Or has it always been shaky ?
>> > Does anybody know ?
>> 
>> The NonStop port has traditionally had issues with t0025, which we tended
>> to ignore because things did work. We wrote those off as bash issues in
>> t0025 since they seemed to be corrected when we picked up a new bash
>> version about a year ago. I will keep monitoring this, particularly when
> 2.21
>> comes out.
>
> FYI: t0020-t0027 all passed on the NonStop port for 2.21.0-rc0 - so no
> issues for us on this one.

Yup, the preview has both the renormalize fix and another hotfix for
it Peff made.  Thanks all for quick fixes and confirmations.
SZEDER Gábor Feb. 7, 2019, 11:57 p.m. UTC | #8
On Thu, Feb 07, 2019 at 11:58:08AM -0500, Randall S. Becker wrote:
> > The NonStop port has traditionally had issues with t0025, which we tended
> > to ignore because things did work. We wrote those off as bash issues in
> > t0025 since they seemed to be corrected when we picked up a new bash
> > version about a year ago. I will keep monitoring this, particularly when
> 2.21
> > comes out.
> 
> FYI: t0020-t0027 all passed on the NonStop port for 2.21.0-rc0 - so no
> issues for us on this one.

Note that t0021 is very likely flaky on NonStop, too:

  https://public-inbox.org/git/20190111140408.GC840@szeder.dev/T/#u
Randall S. Becker Feb. 8, 2019, 10:21 a.m. UTC | #9
On February 7, 2019 18:57, SZEDER Gábor wrote:
> On Thu, Feb 07, 2019 at 11:58:08AM -0500, Randall S. Becker wrote:
> > > The NonStop port has traditionally had issues with t0025, which we
> > > tended to ignore because things did work. We wrote those off as bash
> > > issues in
> > > t0025 since they seemed to be corrected when we picked up a new bash
> > > version about a year ago. I will keep monitoring this, particularly
> > > when
> > 2.21
> > > comes out.
> >
> > FYI: t0020-t0027 all passed on the NonStop port for 2.21.0-rc0 - so no
> > issues for us on this one.
> 
> Note that t0021 is very likely flaky on NonStop, too:
> 
>   https://public-inbox.org/git/20190111140408.GC840@szeder.dev/T/#u

We will keep a watch on it, thanks. t0021 has been stable on this platform for at least a year and passes for 2.21.0-rc0 as well as 2.20.0.

Cheers,
Randall
diff mbox series

Patch

--- expect	2019-02-06 09:39:42.080733629 +0000
+++ actual	2019-02-06 09:39:42.088733629 +0000
@@ -1,3 +1,3 @@ 
-i/lf w/crlf attr/text=auto CRLF.txt
+i/crlf w/crlf attr/text=auto CRLF.txt
 i/lf w/lf attr/text=auto LF.txt
-i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
+i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt