Message ID | 20250327-b4-pks-t-perlless-v3-0-b436de9da1b8@pks.im (mailing list archive) |
---|---|
Headers | show |
Series | t: drop Perl as a mandatory prerequisite | expand |
Hi Patrick On 27/03/2025 10:36, Patrick Steinhardt wrote: > > Changes in v3: > - Remove more useless indirections for sed(1). Thanks for removing these, the range-diff below looks good Best Wishes Phillip > Range-diff versus v2: > > 1: 8c98b24fe4c = 1: f2fe08ef0ff t: skip chain lint when PERL_PATH is unset > 2: f140153954c = 2: 9dd2edd0a1a t: refactor environment sanitization to not use Perl > 3: 94b5591f666 = 3: c77424e6907 t: adapt character translation helpers to not use Perl > 4: a5880fdb8ef = 4: 476d1b15932 t: adapt `test_copy_bytes()` to not use Perl > 5: 3b64c99c061 = 5: 14badee2551 t: adapt `test_readlink()` to not use Perl > 6: a3536260e4c = 6: 9a88a46bd10 t: introduce PERL_TEST_HELPERS prerequisite > 7: 98961b0e065 = 7: e7413bf28ae t: adapt existing PERL prerequisites > 8: bbdd1fe6c7c = 8: 581a9bedd22 meson: stop requiring Perl when tests are enabled > 9: bda7e7922ce = 9: cfe1797ae74 Makefile: stop requiring Perl when running tests > 10: d95d50c4b73 = 10: 99e678b83a6 t: refactor tests depending on Perl transliteration operator > 11: f5b30cc3f8f ! 11: 93a98d3e3cf t: refactor tests depending on Perl substitution operator > @@ t/t4029-diff-trailing-space.sh: test_expect_success PERL_TEST_HELPERS "$test_des > git diff f > actual && > test_cmp exp actual && > - perl -i.bak -p -e "s/^\$/ /" exp && > -+ sed "s/^\$/ /" <exp >exp.munged && > ++ sed "s/^\$/ /" exp >exp.munged && > + mv exp.munged exp && > git config --bool diff.suppressBlankEmpty false && > git diff f > actual && > @@ t/t4200-rerere.sh: test_expect_success 'activate rerere, old style (conflicting > test_must_fail git merge first && > > - sha1=$(perl -pe "s/ .*//" .git/MERGE_RR) && > -+ sha1=$(sed "s/ .*//" <.git/MERGE_RR) && > ++ sha1=$(sed "s/ .*//" .git/MERGE_RR) && > rr=.git/rr-cache/$sha1 && > grep "^=======\$" $rr/preimage && > ! test -f $rr/postimage && > @@ t/t4200-rerere.sh: test_expect_success 'rerere.enabled works, too' ' > test_must_fail git merge first && > > - sha1=$(perl -pe "s/ .*//" .git/MERGE_RR) && > -+ sha1=$(sed "s/ .*//" <.git/MERGE_RR) && > ++ sha1=$(sed "s/ .*//" .git/MERGE_RR) && > rr=.git/rr-cache/$sha1 && > grep ^=======$ $rr/preimage > ' > @@ t/t4200-rerere.sh: test_expect_success 'set up rr-cache' ' > git reset --hard && > test_must_fail git merge first && > - sha1=$(perl -pe "s/ .*//" .git/MERGE_RR) && > -+ sha1=$(sed "s/ .*//" <.git/MERGE_RR) && > ++ sha1=$(sed "s/ .*//" .git/MERGE_RR) && > rr=.git/rr-cache/$sha1 > ' > > @@ t/t5303-pack-corruption-resilience.sh: test_expect_success '... and loose copy o > git prune-packed && > chmod +w ${pack}.pack && > - perl -i.bak -pe "s/ base /abcdef/" ${pack}.pack && > -+ sed "s/ base /abcdef/" <${pack}.pack >${pack}.pack.munged && > ++ sed "s/ base /abcdef/" ${pack}.pack >${pack}.pack.munged && > + mv ${pack}.pack.munged ${pack}.pack && > test_must_fail git cat-file blob $blob_1 > /dev/null && > test_must_fail git cat-file blob $blob_2 > /dev/null && > @@ t/t5303-pack-corruption-resilience.sh: test_expect_success '... and then a repac > git prune-packed && > chmod +w ${pack}.pack && > - perl -i.bak -pe "s/ delta1 /abcdefgh/" ${pack}.pack && > -+ sed "s/ delta1 /abcdefgh/" <${pack}.pack >${pack}.pack.munged && > ++ sed "s/ delta1 /abcdefgh/" ${pack}.pack >${pack}.pack.munged && > + mv ${pack}.pack.munged ${pack}.pack && > git cat-file blob $blob_1 > /dev/null && > test_must_fail git cat-file blob $blob_2 > /dev/null && > @@ t/t5310-pack-bitmaps.sh: test_bitmap_cases () { > # mark the commits which did not receive bitmaps as preferred, > # and generate the bitmap again > - perl -pe "s{^}{create refs/tags/include/$. }" <before | > -+ sed "s|\(.*\)|create refs/tags/include/\1 \1|" <before | > ++ sed "s|\(.*\)|create refs/tags/include/\1 \1|" before | > git update-ref --stdin && > git -c pack.preferBitmapTips=refs/tags/include repack -adb && > > @@ t/t5534-push-signed.sh: test_expect_success GPG,PERL_TEST_HELPERS 'inconsistent > # different, then replay it on a fresh dst, checking that ff is not > # deleted. > - perl -pe "s/([^ ])bar/\$1baz/" push >push.tweak && > -+ sed "s/\([^ ]\)bar/\1baz/" <push >push.tweak && > ++ sed "s/\([^ ]\)bar/\1baz/" push >push.tweak && > prepare_dst && > git -C dst config receive.certnonceseed sekrit && > git -C dst config receive.advertisepushoptions 1 && > @@ t/t6011-rev-list-with-bad-commit.sh: test_expect_success 'verify number of revis > +test_expect_success 'corrupt second commit object' ' > + for p in .git/objects/pack/*.pack > + do > -+ sed "s/second commit/socond commit/" <"$p" >"$p.munged" && > ++ sed "s/second commit/socond commit/" "$p" >"$p.munged" && > + mv "$p.munged" "$p" || > + return 1 > + done && > @@ t/t7416-submodule-dash-url.sh: test_expect_success 'fsck accepts protected dash' > > test_expect_success 'remove ./ protection from .gitmodules url' ' > - perl -i -pe "s{\./}{}" .gitmodules && > -+ sed "s|\./||" <.gitmodules >.gitmodules.munged && > ++ sed "s|\./||" .gitmodules >.gitmodules.munged && > + mv .gitmodules.munged .gitmodules && > git commit -am "drop protection" > ' > @@ t/t8006-blame-textconv.sh: find_blame() { > #!/bin/sh > grep -q '^bin: ' "$1" || { echo "E: $1 is not \"binary\" file" 1>&2; exit 1; } > -"$PERL_PATH" -p -e 's/^bin: /converted: /' "$1" > -+sed 's/^bin: /converted: /' <"$1" > ++sed 's/^bin: /converted: /' "$1" > EOF > chmod +x helper > > @@ t/t9137-git-svn-dcommit-clobber-series.sh: test_expect_success 'initialize repo' > (cd tmp && > - perl -i.bak -p -e "s/^58$/5588/" file && > - perl -i.bak -p -e "s/^61$/6611/" file && > -+ sed -e "s/^58$/5588/" -e "s/^61$/6611/" <file >file.munged && > ++ sed -e "s/^58$/5588/" -e "s/^61$/6611/" file >file.munged && > + mv file.munged file && > poke file && > test x"$(sed -n -e 58p < file)" = x5588 && > @@ t/t9137-git-svn-dcommit-clobber-series.sh: test_expect_success 'some unrelated c > - perl -i.bak -p -e 's/^7\$/7777/' file && > + sed -e 's/^4\$/4444/' \ > + -e 's/^7\$/7777/' \ > -+ <file >file.munged && > ++ file >file.munged && > + mv file.munged file && > test x\"\$(sed -n -e 4p < file)\" = x4444 && > test x\"\$(sed -n -e 7p < file)\" = x7777 && > 12: e978d8ecfde ! 12: 17f862eaba3 t: refactor tests depending on Perl to print data > @@ t/t5300-pack-object.sh: test_expect_success 'pack-object <stdin parsing: --stdin > # e.g.: check_deltas stderr -gt 0 > check_deltas() { > - deltas=$(perl -lne '/delta (\d+)/ and print $1' "$1") && > -+ deltas=$(sed -n 's/Total [0-9][0-9]* (delta \([0-9][0-9]*\)).*/\1/p' <"$1") && > ++ deltas=$(sed -n 's/Total [0-9][0-9]* (delta \([0-9][0-9]*\)).*/\1/p' "$1") && > shift && > if ! test "$deltas" "$@" > then > @@ t/t5326-multi-pack-bitmaps.sh: test_midx_bitmap_cases () { > > - perl -ne "printf(\"create refs/tags/include/%d \", $.); print" \ > - <before | git update-ref --stdin && > -+ sed "s|\(.*\)|create refs/tags/include/\1 \1|" <before | > ++ sed "s|\(.*\)|create refs/tags/include/\1 \1|" before | > + git update-ref --stdin && > > rm -fr $midx-$(midx_checksum $objdir).bitmap && > @@ t/t5333-pseudo-merge-bitmaps.sh: test_pseudo_merges_reused () { > - perl -lne ' > - print "create refs/tags/" . $. . " " . $1 if /([0-9a-f]+)/ > - ' <in | git update-ref --stdin > -+ sed 's|\(.*\)|create refs/tags/\1 \1|' <in | > ++ sed 's|\(.*\)|create refs/tags/\1 \1|' in | > + git update-ref --stdin > } > > @@ t/t5333-pseudo-merge-bitmaps.sh: test_expect_success 'pseudo-merge pattern with > git rev-list HEAD~16.. >in && > - > - perl -lne "print \"create refs/remotes/$r/tags/\$. \$_\"" <in | > -+ sed "s|\(.*\)|create refs/remotes/$r/tags/\1 \1" <in | > ++ sed "s|\(.*\)|create refs/remotes/$r/tags/\1 \1" in | > git update-ref --stdin || return 1 > done && > > @@ t/t8002-blame.sh: test_expect_success 'set up abbrev tests' ' > echo $sha1 | cut -c 1-$expect >expect && > git blame "$@" abbrev.t >actual && > - perl -lne "/[0-9a-f]+/ and print \$&" <actual >actual.sha && > -+ sed -n "s/^[\^]\{0,1\}\([0-9a-f][0-9a-f]*\).*/\1/p" <actual >actual.sha && > ++ sed -n "s/^[\^]\{0,1\}\([0-9a-f][0-9a-f]*\).*/\1/p" actual >actual.sha && > test_cmp expect actual.sha > } > ' > 13: 905c25c9fb2 = 13: 7b03d096ccd t: refactor tests depending on Perl for textconv scripts > 14: 1fe67bba30f = 14: 195c0bf2445 t/lib-gpg: refactor `sanitize_pgp()` to not depend on Perl > 15: 9e572c3ba67 = 15: e92d178b96b t/lib-t6000: refactor `name_from_description()` to not depend on Perl > 16: 24abcffe96e ! 16: 0f2c9ad276b t/lib-httpd: refactor "one-time-perl" CGI script to not depend on Perl > @@ t/t5537-fetch-shallow.sh: test_expect_success PERL_TEST_HELPERS 'shallow fetches > - "$(git -C "$REPO" rev-parse HEAD^)" \ > - >"$HTTPD_ROOT_PATH/one-time-perl" && > + write_script "$HTTPD_ROOT_PATH/one-time-script" <<-EOF && > -+ sed "$(printf "$(test_oid perl)" "$(git -C "$REPO" rev-parse HEAD)" "$(git -C "$REPO" rev-parse HEAD^)")" <"\$1" > ++ sed "$(printf "$(test_oid perl)" "$(git -C "$REPO" rev-parse HEAD)" "$(git -C "$REPO" rev-parse HEAD^)")" "\$1" > + EOF > test_must_fail env GIT_TEST_SIDEBAND_ALL=0 git -C client \ > - fetch --depth=1 "$HTTPD_URL/one_time_perl/repo" \ > @@ t/t5616-partial-clone.sh: intersperse () { > + write_script "$HTTPD_ROOT_PATH/one-time-script" <<-EOF > + if grep packfile "\$1" >/dev/null > + then > -+ sed '/packfile/q' <"\$1" && > ++ sed '/packfile/q' "\$1" && > + # The protocol requires that the packfile be sent in sideband > + # 1, hence the extra \001 byte at the beginning. > + printf "%04x\001" \$((\$(wc -c <"$PWD/one-time-pack") + 5)) && > @@ t/t5702-protocol-v2.sh: test_expect_success PERL_TEST_HELPERS 'when server sends > - printf "\$ready = 1 if /ready/; \$ready && s/0001/0000/" \ > - >"$HTTPD_ROOT_PATH/one-time-perl" && > + write_script "$HTTPD_ROOT_PATH/one-time-script" <<-\EOF && > -+ sed "/ready/{n;s/0001/0000/;}" <"$1" > ++ sed "/ready/{n;s/0001/0000/;}" "$1" > + EOF > > test_must_fail git -C http_child -c protocol.version=2 \ > @@ t/t5702-protocol-v2.sh: test_expect_success PERL_TEST_HELPERS 'when server does > - printf "\$ack = 1 if /acknowledgments/; \$ack && s/0000/0001/" \ > - >"$HTTPD_ROOT_PATH/one-time-perl" && > + write_script "$HTTPD_ROOT_PATH/one-time-script" <<-\EOF && > -+ sed "/acknowledgments/,//{s/0000/0001/;}" <"$1" > ++ sed "/acknowledgments/,//{s/0000/0001/;}" "$1" > + EOF > > test_must_fail env GIT_TRACE_PACKET="$(pwd)/log" git -C http_child \ > @@ t/t5702-protocol-v2.sh: test_expect_success 'http:// --negotiate-only' ' > - echo "s/ wait-for-done/ xxxx-xxx-xxxx/" \ > - >"$HTTPD_ROOT_PATH/one-time-perl" && > + write_script "$HTTPD_ROOT_PATH/one-time-script" <<-\EOF && > -+ sed "s/ wait-for-done/ xxxx-xxx-xxxx/" <"$1" > ++ sed "s/ wait-for-done/ xxxx-xxx-xxxx/" "$1" > + EOF > > test_must_fail git -c protocol.version=2 -C client fetch \ > @@ t/t5703-upload-pack-ref-in-want.sh: inconsistency () { > oid2=$(git -C "$REPO" rev-parse $2) && > - echo "s/$oid1/$oid2/" >"$HTTPD_ROOT_PATH/one-time-perl" > + write_script "$HTTPD_ROOT_PATH/one-time-script" <<-EOF > -+ sed "s/$oid1/$oid2/" <"\$1" > ++ sed "s/$oid1/$oid2/" "\$1" > + EOF > } > > @@ t/t5703-upload-pack-ref-in-want.sh: test_expect_success 'server loses a ref - re > cp -r "$LOCAL_PRISTINE" local && > - echo "s/main/rain/" >"$HTTPD_ROOT_PATH/one-time-perl" && > + write_script "$HTTPD_ROOT_PATH/one-time-script" <<-\EOF && > -+ sed "s/main/rain/" <"$1" > ++ sed "s/main/rain/" "$1" > + EOF > test_must_fail git -C local fetch 2>err && > > 17: ce5adbd4818 = 17: 9857b461ed6 t0021: refactor `generate_random_characters()` to not depend on Perl > 18: e183c397da9 = 18: 7924b5bd9bf t0210: refactor trace2 scrubbing to not use Perl > 19: 156bdc4d62d = 19: 5d6996a1412 t5316: refactor `max_chain()` to not depend on Perl > 20: 3b181d0a203 = 20: 0c3afb70128 t5703: refactor test to not depend on Perl > > --- > base-commit: 683c54c999c301c2cd6f715c411407c413b1d84e > change-id: 20250317-b4-pks-t-perlless-138cf94696b8 > >
Hi Patrick, On Thu, 27 Mar 2025, Patrick Steinhardt wrote: > while Git was initially building on Perl quite a lot, the significance > of Perl has been dwindling over the years as more and more functionality > was converted into C builtins. Nowadays, an installation with Perl-based > features disabled is almost fully functional, only a handful of features > remain that require Perl: > > - gitweb, a read-only web interface. > > - A couple of scripts that allow importing repositories from GNU Arch, > CVS and Subversion. > > - git-send-email(1), which can be used to send mails. > > - Our Perl bindings for Git. > > - The netrc Git credential helper. > > None of these features really are critical for day-to-day usage of Git, > and most users probably wouldn't even notice if those features were not > installed. Perl is thus very much optional nowadays. > > There is one big exception though: it is impossible to run our test > suite without a Perl interpreter, so it is not easily possible to verify > that a Perl-less installation actually works as expected. For most of > the part though our test suite doesn't use all that much Perl, either. > It is present in a couple of critical paths, but those are easy to adapt > to not use Perl anymore. > > This is exactly what this patch series does: it refactors a couple of > central parts in our test suite to not use Perl anymore so that it > becomes possible to run most of our tests entirely without Perl. Tests > that still depend on Perl are marked with a new PERL_TEST_HELPERS prereq > so that they only execute when a Perl interpreter is available. > > With this patch series, 30342 out of 31358 tests pass, which is around > 97% of our tests. Thank you so much for working on this. I finally finished my review, I simply ran out of time yesterday. The patches look good to me, and the result even more so. From my perspective, it also sure looks like good timing, even so much as two years ago it would probably not have been realistically achievable to drop Git's test suite's dependency on Perl. On Git for Windows' side, I have opened a ticket [*1*] that even discusses the idea of dropping Perl from Git for Windows' installer altogether. The appeal of this is that Perl is quite a hefty dependency with all of those Perl modules that are part of the set that Perl users simply expect to be present. I did not investigate this fully yet, but would expect a reduction of at least 10MB, and Git for Windows carries this essentially for `git svn` and little else. So yes, I am quite happy about this direction, and could also imagine that at least the Windows test jobs (which use `prove` like the rest of the tests in Git's CI builds) could potentially switch away from `prove` to `test-tool run-command testsuite` and then drop Perl from the `git-sdk-x86_64-minimal` artifact that needs to be downloaded and installed for every single Windows job of Git's CI. (The `test-tool` may need to learn a few more tricks, but I'd be surprised if any non-trivial patches were needed there to essentially replace `prove` as far as Git's needs are concerned.) Ciao, Johannes Footnote *1*: https://github.com/git-for-windows/git/issues/5393