| Message ID | b25ac1cc-8e77-17e6-602a-b289c1e1cfeb@web.de (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | p2000 failure due to empty reflog | expand |
On 10/2/2021 1:37 PM, René Scharfe wrote: > p2000 fails for me and reports: > > perf 18 - git checkout -f - (full-v3): > running: > ( > cd full-v3 && > echo >>f2/f4/a && > git checkout -f - > ) > > error: pathspec '-' did not match any file(s) known to git > > checkout fails because the reflog is empty, so the "-" can't be > resolved. The pathspec error message is confusing, though. > > The patch below adds a reflog entry and allows the script to > succeed. > > Before the "test_perf_on_all git commit -a -m A", there are two > reflog entries in each of the five clones, after it there are > none. How is that even possible? That is certainly confusing. Is there something about your global (or local to your test repo) GC settings that cause an auto-GC to prune the reflog aggressively? > @@ -109,6 +110,14 @@ test_perf_on_all git status > test_perf_on_all git add -A > test_perf_on_all git add . > test_perf_on_all git commit -a -m A > + > +test_expect_success 'add reflog entry' ' > + for repo in full-v3 full-v4 sparse-v3 sparse-v4 > + do > + git -C $repo checkout $OLD_COMMIT > + done > +' > + > test_perf_on_all git checkout -f - While I believe this will fix the situation, it might only be a band-aid on the real problem of losing the reflog during the test. Thanks, -Stolee
Am 04.10.21 um 21:55 schrieb Derrick Stolee: > On 10/2/2021 1:37 PM, René Scharfe wrote: >> p2000 fails for me and reports: >> >> perf 18 - git checkout -f - (full-v3): >> running: >> ( >> cd full-v3 && >> echo >>f2/f4/a && >> git checkout -f - >> ) >> >> error: pathspec '-' did not match any file(s) known to git >> >> checkout fails because the reflog is empty, so the "-" can't be >> resolved. The pathspec error message is confusing, though. >> >> The patch below adds a reflog entry and allows the script to >> succeed. >> >> Before the "test_perf_on_all git commit -a -m A", there are two >> reflog entries in each of the five clones, after it there are >> none. How is that even possible? > > That is certainly confusing. Is there something about your global > (or local to your test repo) GC settings that cause an auto-GC to > prune the reflog aggressively? Good point. I have gc.auto=0 in my config, but the tests use their own, empty config. A trace shows that "git gc --auto --no-quiet" is started. The following patch turns that off and allows the tests to succeed. Not doing maintenance in parallel to a performance test is a good idea anyway, but I still don't understand why it would empty the reflog -- that seems excessive, dangerous even. One of the maintenance commands from the trace is "git reflog expire --all". If I put that in before the "checkout -" test (on top of the patch below) then the reflog is emptied again and the test fails. René --- t/perf/p2000-sparse-operations.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh index 597626276f..9a06904247 100755 --- a/t/perf/p2000-sparse-operations.sh +++ b/t/perf/p2000-sparse-operations.sh @@ -105,6 +105,13 @@ test_perf_on_all () { done } +test_expect_success 'disable housekeeping' ' + for repo in full-v3 full-v4 sparse-v3 sparse-v4 + do + git -C $repo config gc.auto 0 + done +' + test_perf_on_all git status test_perf_on_all git add -A test_perf_on_all git add . -- 2.33.0
On Mon, Oct 04 2021, Derrick Stolee wrote: > On 10/2/2021 1:37 PM, René Scharfe wrote: >> p2000 fails for me and reports: >> >> perf 18 - git checkout -f - (full-v3): >> running: >> ( >> cd full-v3 && >> echo >>f2/f4/a && >> git checkout -f - >> ) >> >> error: pathspec '-' did not match any file(s) known to git >> >> checkout fails because the reflog is empty, so the "-" can't be >> resolved. The pathspec error message is confusing, though. >> >> The patch below adds a reflog entry and allows the script to >> succeed. >> >> Before the "test_perf_on_all git commit -a -m A", there are two >> reflog entries in each of the five clones, after it there are >> none. How is that even possible? > > That is certainly confusing. Is there something about your global > (or local to your test repo) GC settings that cause an auto-GC to > prune the reflog aggressively? Perhaps something to do with certain parts of the code doing a time(NULL), and other parts faking up times to be 2005-something? I have some WIP patches that run concurrent "git gc --auto" with a detach across the whole test suite, and IIRC I had to search/replace all time(NULL) with my own time() that faked up (looks it up) an epoch of 1113465166 to avoid this exact issue. If you try doing that across our tests I think you'll find that the @{-1} syntax (reflog-powered) will start failing. But that's from vague memory, but perhaps it'll point you in the right direction...
René Scharfe <l.s.r@web.de> writes: > Good point. I have gc.auto=0 in my config, but the tests use their own, > empty config. A trace shows that "git gc --auto --no-quiet" is started. > The following patch turns that off and allows the tests to succeed. > > Not doing maintenance in parallel to a performance test is a good idea > anyway, Sounds like a sensible analysis, and I agree that it is a good idea to by default disable the maintenance tasks in t/perf (possibly at a more centralized place). > but I still don't understand why it would empty the reflog -- > that seems excessive, dangerous even. > > One of the maintenance commands from the trace is "git reflog expire > --all". If I put that in before the "checkout -" test (on top of the > patch below) then the reflog is emptied again and the test fails. > > René > > --- > t/perf/p2000-sparse-operations.sh | 7 +++++++ > 1 file changed, 7 insertions(+) > > diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh > index 597626276f..9a06904247 100755 > --- a/t/perf/p2000-sparse-operations.sh > +++ b/t/perf/p2000-sparse-operations.sh > @@ -105,6 +105,13 @@ test_perf_on_all () { > done > } > > +test_expect_success 'disable housekeeping' ' > + for repo in full-v3 full-v4 sparse-v3 sparse-v4 > + do > + git -C $repo config gc.auto 0 > + done > +' > + > test_perf_on_all git status > test_perf_on_all git add -A > test_perf_on_all git add . > -- > 2.33.0
Am 05.10.21 um 23:38 schrieb Ævar Arnfjörð Bjarmason: > > On Mon, Oct 04 2021, Derrick Stolee wrote: > >> On 10/2/2021 1:37 PM, René Scharfe wrote: >>> p2000 fails for me and reports: >>> >>> perf 18 - git checkout -f - (full-v3): >>> running: >>> ( >>> cd full-v3 && >>> echo >>f2/f4/a && >>> git checkout -f - >>> ) >>> >>> error: pathspec '-' did not match any file(s) known to git >>> >>> checkout fails because the reflog is empty, so the "-" can't be >>> resolved. The pathspec error message is confusing, though. >>> >>> The patch below adds a reflog entry and allows the script to >>> succeed. >>> >>> Before the "test_perf_on_all git commit -a -m A", there are two >>> reflog entries in each of the five clones, after it there are >>> none. How is that even possible? >> >> That is certainly confusing. Is there something about your global >> (or local to your test repo) GC settings that cause an auto-GC to >> prune the reflog aggressively? > > Perhaps something to do with certain parts of the code doing a > time(NULL), and other parts faking up times to be 2005-something? > > I have some WIP patches that run concurrent "git gc --auto" with a > detach across the whole test suite, and IIRC I had to search/replace all > time(NULL) with my own time() that faked up (looks it up) an epoch of > 1113465166 to avoid this exact issue. > > If you try doing that across our tests I think you'll find that the > @{-1} syntax (reflog-powered) will start failing. > > But that's from vague memory, but perhaps it'll point you in the right > direction... Yes, that helped. The reflog entries have fake dates in 2005, and "git reflog expire --all" called by "git gc --auto" deletes entries older than 90 days based on the actual current time. René
diff --git a/t/perf/p2000-sparse-operations.sh b/t/perf/p2000-sparse-operations.sh index 597626276f..8529e3d295 100755 --- a/t/perf/p2000-sparse-operations.sh +++ b/t/perf/p2000-sparse-operations.sh @@ -25,6 +25,7 @@ test_expect_success 'setup repo and indexes' ' git commit -m "level 0" && BLOB=$(git rev-parse HEAD:a) && OLD_COMMIT=$(git rev-parse HEAD) && + test_export OLD_COMMIT && OLD_TREE=$(git rev-parse HEAD^{tree}) && for i in $(test_seq 1 3) @@ -109,6 +110,14 @@ test_perf_on_all git status test_perf_on_all git add -A test_perf_on_all git add . test_perf_on_all git commit -a -m A + +test_expect_success 'add reflog entry' ' + for repo in full-v3 full-v4 sparse-v3 sparse-v4 + do + git -C $repo checkout $OLD_COMMIT + done +' + test_perf_on_all git checkout -f - test_done