[v5,0/8] ci: add support for GitLab CI

Message

Patrick Steinhardt Nov. 1, 2023, 1:02 p.m. UTC

Hi,

this is the fifth version of my patch series that introduces support for
GitLab CI.

There are only minor changes compared to v4:

    - Patch 5: Fixed a typo in the commit message. Furthermore, I've
      dropped the note about `export VAR=value` being a Bashism.

    - Patch 5: Fixed the accidentally-dropped "windows_nt" case.

    - Patch 5: Introduce the JOBS variable right away so that there is
      less churn in the patch that introduces GitLab CI.

    - Patch 8: Moved `DEBIAN_FRONTEND` into the "linux-*" case to make
      clear that it shouldn't impact anything else.

    - Patch 8: Dropped the paragraph about GitLab CI not being a first
      class citizen based on Junio's feedback. This setup is going to be
      actively used (at least) by us at GitLab, and thus we're also
      going to maintain it.

After both Junio's and Victoria's feedback I've kept `.gitlab-ci.yml` in
its canonical location.

The pipeline for this version can be found at [1].

Thanks for all the discussion and feedback so far!

Patrick

[1]: https://gitlab.com/gitlab-org/git/-/pipelines/1057566736

Patrick Steinhardt (8):
  ci: reorder definitions for grouping functions
  ci: make grouping setup more generic
  ci: group installation of Docker dependencies
  ci: split out logic to set up failed test artifacts
  ci: unify setup of some environment variables
  ci: squelch warnings when testing with unusable Git repo
  ci: install test dependencies for linux-musl
  ci: add support for GitLab CI

 .gitlab-ci.yml                    |  53 +++++++++
 ci/install-docker-dependencies.sh |  23 +++-
 ci/lib.sh                         | 191 +++++++++++++++++++++---------
 ci/print-test-failures.sh         |   6 +
 t/lib-httpd.sh                    |  17 ++-
 5 files changed, 234 insertions(+), 56 deletions(-)
 create mode 100644 .gitlab-ci.yml

Range-diff against v4:
1:  8595fe5016a = 1:  0ba396f2a33 ci: reorder definitions for grouping functions
2:  7358a943392 = 2:  821cfcd6125 ci: make grouping setup more generic
3:  6d842592c6f = 3:  6e5bcf143c8 ci: group installation of Docker dependencies
4:  e15651b3f5d = 4:  2182acf5bfc ci: split out logic to set up failed test artifacts
5:  a64799b6e25 ! 5:  6078aea246d ci: unify setup of some environment variables
    @@ Metadata
      ## Commit message ##
         ci: unify setup of some environment variables
     
    -    Both GitHub Actions and Azue Pipelines set up the environment variables
    +    Both GitHub Actions and Azure Pipelines set up the environment variables
         GIT_TEST_OPTS, GIT_PROVE_OPTS and MAKEFLAGS. And while most values are
         actually the same, the setup is completely duplicate. With the upcoming
         support for GitLab CI this duplication would only extend even further.
    @@ Commit message
               `--no-bin-wrappers`. Interestingly though, we did so _after_
               already having exported the respective environment variables.
     
    -        - We stop using `export VAR=value` syntax, which is a Bashism. It's
    -          not quite worth it as we still use this syntax all over the place,
    -          but it doesn't hurt readability either.
    -
         Signed-off-by: Patrick Steinhardt <[e-mail]>
     
      ## ci/lib.sh ##
    @@ ci/lib.sh: then
     -	test windows_nt != "$CI_OS_NAME" ||
     -	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
     +	GIT_TEST_OPTS="--write-junit-xml"
    ++	JOBS=10
      elif test true = "$GITHUB_ACTIONS"
      then
      	CI_TYPE=github-actions
    @@ ci/lib.sh: then
     -	test windows != "$CI_OS_NAME" ||
     -	GIT_TEST_OPTS="--no-chain-lint --no-bin-wrappers $GIT_TEST_OPTS"
     +	GIT_TEST_OPTS="--github-workflow-markup"
    ++	JOBS=10
      else
      	echo "Could not identify CI type" >&2
      	env >&2
      	exit 1
      fi
      
    -+MAKEFLAGS="$MAKEFLAGS --jobs=10"
    -+GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
    ++MAKEFLAGS="$MAKEFLAGS --jobs=$JOBS"
    ++GIT_PROVE_OPTS="--timer --jobs $JOBS --state=failed,slow,save"
     +
     +GIT_TEST_OPTS="$GIT_TEST_OPTS --verbose-log -x"
    -+if test windows = "$CI_OS_NAME"
    -+then
    ++case "$CI_OS_NAME" in
    ++windows|windows_nt)
     +	GIT_TEST_OPTS="$GIT_TEST_OPTS --no-chain-lint --no-bin-wrappers"
    -+fi
    ++	;;
    ++esac
     +
     +export GIT_TEST_OPTS
     +export GIT_PROVE_OPTS
6:  f7d2a8666fe = 6:  d69bde92f2f ci: squelch warnings when testing with unusable Git repo
7:  9b43b0d90e3 = 7:  b911c005bae ci: install test dependencies for linux-musl
8:  f3f2c98a0dc ! 8:  5784d03a6f1 ci: add support for GitLab CI
    @@ Commit message
         that fail to compile or pass tests in GitHub Workflows. We would thus
         like to integrate the GitLab CI configuration into the Git project to
         help us send better patch series upstream and thus reduce overhead for
    -    the maintainer.
    -
    -    The integration does not necessarily have to be a first-class citizen,
    -    which would in practice only add to the fallout that pipeline failures
    -    have for the maintainer. That being said, we are happy to maintain this
    -    alternative CI setup for the Git project and will make test results
    -    available as part of our own mirror of the Git project at [1].
    +    the maintainer. Results of these pipeline runs will be made available
    +    (at least) in GitLab's mirror of the Git project at [1].
     
         This commit introduces the integration into our regular CI scripts so
         that most of the setup continues to be shared across all of the CI
    @@ .gitlab-ci.yml (new)
     +    when: on_failure
     
      ## ci/install-docker-dependencies.sh ##
    -@@
    - 
    - begin_group "Install dependencies"
    - 
    -+# Required so that apt doesn't wait for user input on certain packages.
    -+export DEBIAN_FRONTEND=noninteractive
    -+
    - case "$jobname" in
    - linux32)
    - 	linux32 --32bit i386 sh -c '
     @@ ci/install-docker-dependencies.sh: linux32)
      	'
      	;;
    @@ ci/install-docker-dependencies.sh: linux32)
      		bash cvs gnupg perl-cgi perl-dbd-sqlite >/dev/null
      	;;
     +linux-*)
    ++	# Required so that apt doesn't wait for user input on certain packages.
    ++	export DEBIAN_FRONTEND=noninteractive
    ++
     +	apt update -q &&
     +	apt install -q -y sudo git make language-pack-is libsvn-perl apache2 libssl-dev \
     +		libcurl4-openssl-dev libexpat-dev tcl tk gettext zlib1g-dev \
    @@ ci/lib.sh: then
      	begin_group () { :; }
      	end_group () { :; }
     @@ ci/lib.sh: then
    - 	cache_dir="$HOME/test-cache/$SYSTEM_PHASENAME"
    - 
    - 	GIT_TEST_OPTS="--write-junit-xml"
    -+	JOBS=10
    - elif test true = "$GITHUB_ACTIONS"
    - then
    - 	CI_TYPE=github-actions
    -@@ ci/lib.sh: then
    - 	cache_dir="$HOME/none"
      
      	GIT_TEST_OPTS="--github-workflow-markup"
    -+	JOBS=10
    + 	JOBS=10
     +elif test true = "$GITLAB_CI"
     +then
     +	CI_TYPE=gitlab-ci
    @@ ci/lib.sh: then
      else
      	echo "Could not identify CI type" >&2
      	env >&2
    - 	exit 1
    - fi
    - 
    --MAKEFLAGS="$MAKEFLAGS --jobs=10"
    --GIT_PROVE_OPTS="--timer --jobs 10 --state=failed,slow,save"
    -+MAKEFLAGS="$MAKEFLAGS --jobs=${JOBS}"
    -+GIT_PROVE_OPTS="--timer --jobs ${JOBS} --state=failed,slow,save"
    - 
    - GIT_TEST_OPTS="$GIT_TEST_OPTS --verbose-log -x"
    - if test windows = "$CI_OS_NAME"
     
      ## ci/print-test-failures.sh ##
     @@ ci/print-test-failures.sh: do

base-commit: 692be87cbba55e8488f805d236f2ad50483bd7d5