[v2,00/20] t: drop Perl as a mandatory prerequisite

Message

Patrick Steinhardt March 25, 2025, 1:14 p.m. UTC

Hi,

while Git was initially building on Perl quite a lot, the significance
of Perl has been dwindling over the years as more and more functionality
was converted into C builtins. Nowadays, an installation with Perl-based
features disabled is almost fully functional, only a handful of features
remain that require Perl:

  - gitweb, a read-only web interface.

  - A couple of scripts that allow importing repositories from GNU Arch,
    CVS and Subversion.

  - git-send-email(1), which can be used to send mails.

  - Our Perl bindings for Git.

  - The netrc Git credential helper.

None of these features really are critical for day-to-day usage of Git,
and most users probably wouldn't even notice if those features were not
installed. Perl is thus very much optional nowadays.

There is one big exception though: it is impossible to run our test
suite without a Perl interpreter, so it is not easily possible to verify
that a Perl-less installation actually works as expected. For most of
the part though our test suite doesn't use all that much Perl, either.
It is present in a couple of critical paths, but those are easy to adapt
to not use Perl anymore.

This is exactly what this patch series does: it refactors a couple of
central parts in our test suite to not use Perl anymore so that it
becomes possible to run most of our tests entirely without Perl. Tests
that still depend on Perl are marked with a new PERL_TEST_HELPERS prereq
so that they only execute when a Perl interpreter is available.

With this patch series, 30342 out of 31358 tests pass, which is around
97% of our tests.

Changes in v2:
  - Improve a couple of conversions based on feedback.
  - Clarify the commit message of the textconv conversion.
  - Fix a copy-paste error when it comes to skipping tests in t4103.
  - Link to v1: https://lore.kernel.org/r/[e-mail]

Thanks!

Patrick

---
Patrick Steinhardt (20):
      t: skip chain lint when PERL_PATH is unset
      t: refactor environment sanitization to not use Perl
      t: adapt character translation helpers to not use Perl
      t: adapt `test_copy_bytes()` to not use Perl
      t: adapt `test_readlink()` to not use Perl
      t: introduce PERL_TEST_HELPERS prerequisite
      t: adapt existing PERL prerequisites
      meson: stop requiring Perl when tests are enabled
      Makefile: stop requiring Perl when running tests
      t: refactor tests depending on Perl transliteration operator
      t: refactor tests depending on Perl substitution operator
      t: refactor tests depending on Perl to print data
      t: refactor tests depending on Perl for textconv scripts
      t/lib-gpg: refactor `sanitize_pgp()` to not depend on Perl
      t/lib-t6000: refactor `name_from_description()` to not depend on Perl
      t/lib-httpd: refactor "one-time-perl" CGI script to not depend on Perl
      t0021: refactor `generate_random_characters()` to not depend on Perl
      t0210: refactor trace2 scrubbing to not use Perl
      t5316: refactor `max_chain()` to not depend on Perl
      t5703: refactor test to not depend on Perl

 meson.build                               |  2 +-
 t/Makefile                                | 16 +++++++--
 t/helper/test-path-utils.c                | 13 ++++++++
 t/helper/test-sha1.sh                     |  4 +--
 t/lib-diff.sh                             |  4 +--
 t/lib-gpg.sh                              |  6 +---
 t/lib-httpd.sh                            |  2 +-
 t/lib-httpd/apache.conf                   |  6 ++--
 t/lib-httpd/apply-one-time-perl.sh        | 27 ---------------
 t/lib-httpd/apply-one-time-script.sh      | 26 +++++++++++++++
 t/lib-t6000.sh                            | 13 ++++----
 t/t0008-ignores.sh                        |  4 +--
 t/t0021-conversion.sh                     | 13 ++++----
 t/t0090-cache-tree.sh                     |  4 +--
 t/t0210-trace2-normal.sh                  | 55 ++++++++++++++++++++++++-------
 t/t0210/scrub_normal.perl                 | 54 ------------------------------
 t/t0211-trace2-perf.sh                    |  6 ++++
 t/t0610-reftable-basics.sh                |  5 ++-
 t/t0613-reftable-write-options.sh         |  2 +-
 t/t1006-cat-file.sh                       | 16 +++++----
 t/t1007-hash-object.sh                    |  6 ++--
 t/t1010-mktree.sh                         |  4 +--
 t/t1450-fsck.sh                           |  6 ++--
 t/t3300-funny-names.sh                    |  6 ++--
 t/t4013-diff-various.sh                   |  6 ++++
 t/t4014-format-patch.sh                   | 30 ++++++++---------
 t/t4020-diff-external.sh                  |  2 +-
 t/t4029-diff-trailing-space.sh            |  3 +-
 t/t4030-diff-textconv.sh                  |  9 ++---
 t/t4031-diff-rewrite-binary.sh            | 17 ++++------
 t/t4058-diff-duplicates.sh                |  6 ++++
 t/t4103-apply-binary.sh                   |  6 ++--
 t/t4116-apply-reverse.sh                  |  4 +--
 t/t4150-am.sh                             |  8 ++---
 t/t4200-rerere.sh                         |  8 ++---
 t/t4205-log-pretty-formats.sh             |  6 ++--
 t/t4216-log-bloom.sh                      |  8 ++---
 t/t5004-archive-corner-cases.sh           |  6 ++++
 t/t5300-pack-object.sh                    | 10 +++---
 t/t5303-pack-corruption-resilience.sh     |  6 ++--
 t/t5310-pack-bitmaps.sh                   |  2 +-
 t/t5316-pack-delta-depth.sh               | 10 +++---
 t/t5318-commit-graph.sh                   | 12 +++----
 t/t5319-multi-pack-index.sh               | 16 ++++-----
 t/t5324-split-commit-graph.sh             |  2 +-
 t/t5326-multi-pack-bitmaps.sh             |  4 +--
 t/t5328-commit-graph-64bit-time.sh        |  2 +-
 t/t5333-pseudo-merge-bitmaps.sh           | 12 +++----
 t/t5400-send-pack.sh                      |  2 +-
 t/t5410-receive-pack-alternates.sh        |  2 +-
 t/t5503-tagfollow.sh                      |  6 ++++
 t/t5504-fetch-receive-strict.sh           |  2 +-
 t/t5510-fetch.sh                          |  6 ++++
 t/t5532-fetch-proxy.sh                    |  6 ++++
 t/t5534-push-signed.sh                    |  2 +-
 t/t5537-fetch-shallow.sh                  | 15 ++++-----
 t/t5551-http-fetch-smart.sh               |  7 ++++
 t/t5562-http-backend-content-length.sh    |  6 ++++
 t/t5601-clone.sh                          |  4 +--
 t/t5616-partial-clone.sh                  | 46 ++++++++++++++------------
 t/t5701-git-serve.sh                      |  5 ++-
 t/t5702-protocol-v2.sh                    | 21 +++++++-----
 t/t5703-upload-pack-ref-in-want.sh        | 29 ++++++++--------
 t/t5710-promisor-remote-capability.sh     |  6 ++++
 t/t6011-rev-list-with-bad-commit.sh       | 14 +++++---
 t/t6013-rev-list-reverse-parents.sh       | 10 +++---
 t/t6102-rev-list-unexpected-objects.sh    |  6 ++++
 t/t6115-rev-list-du.sh                    |  2 +-
 t/t6300-for-each-ref.sh                   | 15 ++++++---
 t/t7006-pager.sh                          |  6 ++--
 t/t7416-submodule-dash-url.sh             |  3 +-
 t/t7501-commit-basic-functionality.sh     |  6 ++--
 t/t7508-status.sh                         |  2 +-
 t/t7815-grep-binary.sh                    |  9 ++---
 t/t8001-annotate.sh                       |  6 ++++
 t/t8002-blame.sh                          |  8 ++++-
 t/t8006-blame-textconv.sh                 |  2 +-
 t/t8011-blame-split-file.sh               |  6 ++--
 t/t8012-blame-colors.sh                   |  6 ++++
 t/t9137-git-svn-dcommit-clobber-series.sh | 10 +++---
 t/t9350-fast-export.sh                    |  2 +-
 t/t9850-shell.sh                          |  2 +-
 t/test-lib-functions.sh                   | 20 +++--------
 t/test-lib.sh                             | 49 +++++++++++++++++----------
 84 files changed, 471 insertions(+), 373 deletions(-)

Range-diff versus v1:

 1:  83596f2aa05 !  1:  658a45496a3 t: skip chain lint when PERL_PATH is unset
    @@ Metadata
      ## Commit message ##
         t: skip chain lint when PERL_PATH is unset
     
    -    Our chainlint scripts verify that test files have proper '&&' chains.
    -    These scripts are written in Perl and are executed for every test file
    -    before executing the test logic itself.
    +    Our chainlint script verifies that test files have proper '&&' chains.
    +    This script is written in Perl and executed for every test file before
    +    executing the test logic itself.
     
         In subsequent commits we're about to refactor our test suite so that
         Perl becomes an optional dependency, only. And while it is already
 2:  67e9859721d =  2:  12a618f141a t: refactor environment sanitization to not use Perl
 3:  6ebfa6a8581 =  3:  37b5ee501c7 t: adapt character translation helpers to not use Perl
 4:  01e9fce3751 =  4:  df0e1185b3b t: adapt `test_copy_bytes()` to not use Perl
 5:  1f291160713 =  5:  badfad15e0c t: adapt `test_readlink()` to not use Perl
 6:  ab15650ca76 !  6:  ae30c6a1524 t: introduce PERL_TEST_HELPERS prerequisite
    @@ t/t4103-apply-binary.sh: export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
      
     +if ! test_have_prereq PERL_TEST_HELPERS
     +then
    -+	skip_all='skipping ignores tests; Perl not available'
    ++	skip_all='skipping apply-binary tests; Perl not available'
     +	test_done
     +fi
     +
 7:  d278b3c6d90 =  7:  8084db771c1 t: adapt existing PERL prerequisites
 8:  bb8b13b2669 =  8:  87b58043c59 meson: stop requiring Perl when tests are enabled
 9:  6b2a77c2dc4 =  9:  b9831e97fca Makefile: stop requiring Perl when running tests
10:  0af780a491b ! 10:  4b3a4f5a1e5 t: refactor tests depending on Perl transliteration operator
    @@ t/t4103-apply-binary.sh: export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
      
     -if ! test_have_prereq PERL_TEST_HELPERS
     -then
    --	skip_all='skipping ignores tests; Perl not available'
    +-	skip_all='skipping apply-binary tests; Perl not available'
     -	test_done
     -fi
     -
11:  c57d1a1dc14 ! 11:  20ccdb34dc8 t: refactor tests depending on Perl substitution operator
    @@ t/t0008-ignores.sh: test_stderr () {
      
      broken_c_unquote () {
     -	"$PERL_PATH" -pe 's/^"//; s/\\//; s/"$//; tr/\n/\0/' "$@"
    -+	<"$1" sed -e 's/^"//' -e 's/\\//' -e 's/"$//' | tr '\n' '\0'
    ++	sed -e 's/^"//' -e 's/\\//' -e 's/"$//' "$1" | tr '\n' '\0'
      }
      
      broken_c_unquote_verbose () {
     -	"$PERL_PATH" -pe 's/	"/	/; s/\\//; s/"$//; tr/:\t\n/\0/' "$@"
    -+	<"$1" sed -e 's/	"/	/' -e 's/\\//' -e 's/"$//' | tr ':\t\n' '\000'
    ++	sed -e 's/	"/	/' -e 's/\\//' -e 's/"$//' "$1" | tr ':\t\n' '\000'
      }
      
      stderr_contains () {
12:  4cf1615b49a ! 12:  a6c15a4b619 t: refactor tests depending on Perl to print data
    @@ t/t0610-reftable-basics.sh: test_expect_success 'basic: commit and list refs' '
     -		print \"this is a long commit message\" x 50000
     -	" >commit-msg &&
     +
    -+	awk "BEGIN { for (i = 0; i < 50000; i++) print \"this is a long commit message\" }" >commit-msg &&
    ++	awk "BEGIN { for (i = 0; i < 50000; i++) printf \"%s\", \"this is a long commit message\" }" >commit-msg &&
      	git -C repo commit --allow-empty --file=../commit-msg
      '
      
13:  de36109ff9f ! 13:  4e6c4c88d4c t: refactor tests depending on Perl for textconv scripts
    @@ Commit message
         place: we only care that the textconv script does something, and that
         can be verified trivially without having a full-blown invocation of
         hexdump. So at times, the implementation of the textconv scripts is
    -    reduced to their bare minimum.
    +    reduced to their bare minimum and the expectations of those tests are
    +    adapted accordingly.
     
         Signed-off-by: Patrick Steinhardt <[e-mail]>
     
14:  ac0537394ab = 14:  28d6a862403 t/lib-gpg: refactor `sanitize_pgp()` to not depend on Perl
15:  96486f46150 ! 15:  17fbc84e23c t/lib-t6000: refactor `name_from_description()` to not depend on Perl
    @@ t/lib-t6000.sh: check_output () {
     +		-e 's/[^A-Za-z0-9.]/-/g' \
     +		-e 's/--*/-/g' \
     +		-e 's/-$//' \
    -+		-e 's/^-//' |
    -+	tr 'A-Z' 'a-z'
    ++		-e 's/^-//' \
    ++		-e 'y/A-Z/a-z/'
      }
      
      
16:  2b418ae8d8f = 16:  de58d23e7ab t/lib-httpd: refactor "one-time-perl" CGI script to not depend on Perl
17:  8eb9b127db6 = 17:  517094e41a6 t0021: refactor `generate_random_characters()` to not depend on Perl
18:  8d0e583f741 = 18:  0eecfc633bd t0210: refactor trace2 scrubbing to not use Perl
19:  7c5dc0d66b1 = 19:  8a2b080c949 t5316: refactor `max_chain()` to not depend on Perl
20:  8a35ab947b5 = 20:  118b00003ea t5703: refactor test to not depend on Perl

---
base-commit: 683c54c999c301c2cd6f715c411407c413b1d84e
change-id: 20250317-b4-pks-t-perlless-138cf94696b8