| Message ID | cover.1712896868.git.ps@pks.im (mailing list archive) |
|---|---|
| Headers | show |
| Series | t: exercise Git/JGit reftable compatibility | expand |
On 24/04/12 06:43AM, Patrick Steinhardt wrote: > Hi, > > here's the (hopefully last) version of my patch series that introduces > Git/JGit compatibility tests for the reftable format. Only a single > commit changed, where I fixed two typos and added a missing signoff. With 35e293e6 (Merge branch 'ps/ci-test-with-jgit' into next, 2024-05-01), I've noticed that a subset of the `test:linux` GitLab CI jobs have started failing on next. https://gitlab.com/gitlab-org/git/-/pipelines/1277877090 I'll take a look. -Justin
On Fri, May 03, 2024 at 01:42:32PM -0500, Justin Tobler wrote: > On 24/04/12 06:43AM, Patrick Steinhardt wrote: > > Hi, > > > > here's the (hopefully last) version of my patch series that introduces > > Git/JGit compatibility tests for the reftable format. Only a single > > commit changed, where I fixed two typos and added a missing signoff. > > With 35e293e6 (Merge branch 'ps/ci-test-with-jgit' into next, > 2024-05-01), I've noticed that a subset of the `test:linux` GitLab CI > jobs have started failing on next. > > https://gitlab.com/gitlab-org/git/-/pipelines/1277877090 > > I'll take a look. Are you sure it's related to this merge? All failures are on "ubuntu:latest", and the complaints are about the "python2" package being missing. Given the recent release of Ubuntu 24.04, maybe the root cause is that "python2" does not exist there anymore? Patrick
On Fri, May 03, 2024 at 08:48:03PM +0200, Patrick Steinhardt wrote: > On Fri, May 03, 2024 at 01:42:32PM -0500, Justin Tobler wrote: > > On 24/04/12 06:43AM, Patrick Steinhardt wrote: > > > Hi, > > > > > > here's the (hopefully last) version of my patch series that introduces > > > Git/JGit compatibility tests for the reftable format. Only a single > > > commit changed, where I fixed two typos and added a missing signoff. > > > > With 35e293e6 (Merge branch 'ps/ci-test-with-jgit' into next, > > 2024-05-01), I've noticed that a subset of the `test:linux` GitLab CI > > jobs have started failing on next. > > > > https://gitlab.com/gitlab-org/git/-/pipelines/1277877090 > > > > I'll take a look. > > Are you sure it's related to this merge? All failures are on > "ubuntu:latest", and the complaints are about the "python2" package > being missing. Given the recent release of Ubuntu 24.04, maybe the root > cause is that "python2" does not exist there anymore? > > Patrick Maybe we should do something like below patch. Basically, we start to acknowledge the fact that Python 2 is end of life and always use Python 3 on ubuntu:latest. We might go even further than that and only use Python 2 on ubuntu:20.04 and slowly phase out support for it. diff --git a/ci/lib.sh b/ci/lib.sh index 473a2d0348..3967a5af85 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -325,11 +325,18 @@ ubuntu-*) break fi - PYTHON_PACKAGE=python2 - if test "$jobname" = linux-gcc - then + case "$distro" in + ubuntu-latest) PYTHON_PACKAGE=python3 - fi + ;; + *) + PYTHON_PACKAGE=python2 + if test "$jobname" = linux-gcc + then + PYTHON_PACKAGE=python3 + fi + ;; + esac MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/$PYTHON_PACKAGE" export GIT_TEST_HTTPD=true Patrick
On 24/05/03 08:57PM, Patrick Steinhardt wrote: > On Fri, May 03, 2024 at 08:48:03PM +0200, Patrick Steinhardt wrote: > > On Fri, May 03, 2024 at 01:42:32PM -0500, Justin Tobler wrote: > > > On 24/04/12 06:43AM, Patrick Steinhardt wrote: > > > > Hi, > > > > > > > > here's the (hopefully last) version of my patch series that introduces > > > > Git/JGit compatibility tests for the reftable format. Only a single > > > > commit changed, where I fixed two typos and added a missing signoff. > > > > > > With 35e293e6 (Merge branch 'ps/ci-test-with-jgit' into next, > > > 2024-05-01), I've noticed that a subset of the `test:linux` GitLab CI > > > jobs have started failing on next. > > > > > > https://gitlab.com/gitlab-org/git/-/pipelines/1277877090 > > > > > > I'll take a look. > > > > Are you sure it's related to this merge? All failures are on > > "ubuntu:latest", and the complaints are about the "python2" package > > being missing. Given the recent release of Ubuntu 24.04, maybe the root > > cause is that "python2" does not exist there anymore? Ya, you are right. That appears to be the problem here. > Maybe we should do something like below patch. Basically, we start to > acknowledge the fact that Python 2 is end of life and always use Python > 3 on ubuntu:latest. We might go even further than that and only use > Python 2 on ubuntu:20.04 and slowly phase out support for it. > > diff --git a/ci/lib.sh b/ci/lib.sh > index 473a2d0348..3967a5af85 100755 > --- a/ci/lib.sh > +++ b/ci/lib.sh > @@ -325,11 +325,18 @@ ubuntu-*) > break > fi > > - PYTHON_PACKAGE=python2 > - if test "$jobname" = linux-gcc > - then > + case "$distro" in > + ubuntu-latest) > PYTHON_PACKAGE=python3 > - fi > + ;; > + *) > + PYTHON_PACKAGE=python2 > + if test "$jobname" = linux-gcc > + then > + PYTHON_PACKAGE=python3 > + fi > + ;; > + esac > MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/$PYTHON_PACKAGE" > > export GIT_TEST_HTTPD=true This seems reasonable to me :) -Justin
On Fri, May 03, 2024 at 02:35:46PM -0500, Justin Tobler wrote: > On 24/05/03 08:57PM, Patrick Steinhardt wrote: > > On Fri, May 03, 2024 at 08:48:03PM +0200, Patrick Steinhardt wrote: > > > On Fri, May 03, 2024 at 01:42:32PM -0500, Justin Tobler wrote: > > > > On 24/04/12 06:43AM, Patrick Steinhardt wrote: [snip] > > Maybe we should do something like below patch. Basically, we start to > > acknowledge the fact that Python 2 is end of life and always use Python > > 3 on ubuntu:latest. We might go even further than that and only use > > Python 2 on ubuntu:20.04 and slowly phase out support for it. > > > > diff --git a/ci/lib.sh b/ci/lib.sh > > index 473a2d0348..3967a5af85 100755 > > --- a/ci/lib.sh > > +++ b/ci/lib.sh > > @@ -325,11 +325,18 @@ ubuntu-*) > > break > > fi > > > > - PYTHON_PACKAGE=python2 > > - if test "$jobname" = linux-gcc > > - then > > + case "$distro" in > > + ubuntu-latest) > > PYTHON_PACKAGE=python3 > > - fi > > + ;; > > + *) > > + PYTHON_PACKAGE=python2 > > + if test "$jobname" = linux-gcc > > + then > > + PYTHON_PACKAGE=python3 > > + fi > > + ;; > > + esac > > MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/$PYTHON_PACKAGE" > > > > export GIT_TEST_HTTPD=true > > This seems reasonable to me :) Please feel free to adopt and adapt this fix. I probably shouldn't be reading mails at this time of the day in the first place :) Patrick
On 24/05/03 09:49PM, Patrick Steinhardt wrote: > On Fri, May 03, 2024 at 02:35:46PM -0500, Justin Tobler wrote: > > On 24/05/03 08:57PM, Patrick Steinhardt wrote: > > > On Fri, May 03, 2024 at 08:48:03PM +0200, Patrick Steinhardt wrote: > > > > On Fri, May 03, 2024 at 01:42:32PM -0500, Justin Tobler wrote: > > > > > On 24/04/12 06:43AM, Patrick Steinhardt wrote: > [snip] > > > Maybe we should do something like below patch. Basically, we start to > > > acknowledge the fact that Python 2 is end of life and always use Python > > > 3 on ubuntu:latest. We might go even further than that and only use > > > Python 2 on ubuntu:20.04 and slowly phase out support for it. > > > > > > diff --git a/ci/lib.sh b/ci/lib.sh > > > index 473a2d0348..3967a5af85 100755 > > > --- a/ci/lib.sh > > > +++ b/ci/lib.sh > > > @@ -325,11 +325,18 @@ ubuntu-*) > > > break > > > fi > > > > > > - PYTHON_PACKAGE=python2 > > > - if test "$jobname" = linux-gcc > > > - then > > > + case "$distro" in > > > + ubuntu-latest) > > > PYTHON_PACKAGE=python3 > > > - fi > > > + ;; > > > + *) > > > + PYTHON_PACKAGE=python2 > > > + if test "$jobname" = linux-gcc > > > + then > > > + PYTHON_PACKAGE=python3 > > > + fi > > > + ;; > > > + esac > > > MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/$PYTHON_PACKAGE" > > > > > > export GIT_TEST_HTTPD=true > > > > This seems reasonable to me :) > > Please feel free to adopt and adapt this fix. I probably shouldn't be > reading mails at this time of the day in the first place :) Thanks, I'll give it a go! -Justin
On Sat, May 04, 2024 at 12:32:31PM -0500, Justin Tobler wrote: > On 24/05/03 09:49PM, Patrick Steinhardt wrote: > > On Fri, May 03, 2024 at 02:35:46PM -0500, Justin Tobler wrote: > > > On 24/05/03 08:57PM, Patrick Steinhardt wrote: > > > > On Fri, May 03, 2024 at 08:48:03PM +0200, Patrick Steinhardt wrote: > > > > > On Fri, May 03, 2024 at 01:42:32PM -0500, Justin Tobler wrote: > > > > > > On 24/04/12 06:43AM, Patrick Steinhardt wrote: > > [snip] > > > > Maybe we should do something like below patch. Basically, we start to > > > > acknowledge the fact that Python 2 is end of life and always use Python > > > > 3 on ubuntu:latest. We might go even further than that and only use > > > > Python 2 on ubuntu:20.04 and slowly phase out support for it. > > > > > > > > diff --git a/ci/lib.sh b/ci/lib.sh > > > > index 473a2d0348..3967a5af85 100755 > > > > --- a/ci/lib.sh > > > > +++ b/ci/lib.sh > > > > @@ -325,11 +325,18 @@ ubuntu-*) > > > > break > > > > fi > > > > > > > > - PYTHON_PACKAGE=python2 > > > > - if test "$jobname" = linux-gcc > > > > - then > > > > + case "$distro" in > > > > + ubuntu-latest) > > > > PYTHON_PACKAGE=python3 > > > > - fi > > > > + ;; > > > > + *) > > > > + PYTHON_PACKAGE=python2 > > > > + if test "$jobname" = linux-gcc > > > > + then > > > > + PYTHON_PACKAGE=python3 > > > > + fi > > > > + ;; > > > > + esac > > > > MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/$PYTHON_PACKAGE" > > > > > > > > export GIT_TEST_HTTPD=true > > > > > > This seems reasonable to me :) > > > > Please feel free to adopt and adapt this fix. I probably shouldn't be > > reading mails at this time of the day in the first place :) > > Thanks, I'll give it a go! > > -Justin I've sent out a fix for this via [1] now so that we can hopefully fast-track this. Patrick [1]: https://lore.kernel.org/git/cb8cefc20f373a3516695e7cbee975132553ea95.1714973381.git.ps@pks.im/T/#u
Hi, here's the (hopefully last) version of my patch series that introduces Git/JGit compatibility tests for the reftable format. Only a single commit changed, where I fixed two typos and added a missing signoff. Thanks! Patrick Patrick Steinhardt (13): ci: rename "runs_on_pool" to "distro" ci: expose distro name in dockerized GitHub jobs ci: skip sudo when we are already root ci: drop duplicate package installation for "linux-gcc-default" ci: convert "install-dependencies.sh" to use "/bin/sh" ci: merge custom PATH directories ci: fix setup of custom path for GitLab CI ci: merge scripts which install dependencies ci: make Perforce binaries executable for all users ci: install JGit dependency t06xx: always execute backend-specific tests t0610: fix non-portable variable assignment t0612: add tests to exercise Git/JGit reftable compatibility .github/workflows/main.yml | 8 +- .gitlab-ci.yml | 6 +- ci/install-dependencies.sh | 101 +++++++++++++------ ci/install-docker-dependencies.sh | 46 --------- ci/lib.sh | 14 +-- t/t0600-reffiles-backend.sh | 8 +- t/t0601-reffiles-pack-refs.sh | 9 +- t/t0610-reftable-basics.sh | 15 ++- t/t0612-reftable-jgit-compatibility.sh | 132 +++++++++++++++++++++++++ 9 files changed, 229 insertions(+), 110 deletions(-) delete mode 100755 ci/install-docker-dependencies.sh create mode 100755 t/t0612-reftable-jgit-compatibility.sh Range-diff against v3: 1: 46502bbe22 = 1: 99b0b60359 ci: rename "runs_on_pool" to "distro" 2: d076ed9857 = 2: e1d4e1320d ci: expose distro name in dockerized GitHub jobs 3: cc0c29052f = 3: 44a0e07600 ci: skip sudo when we are already root 4: 803f5020e0 = 4: 8e58ce38d9 ci: drop duplicate package installation for "linux-gcc-default" 5: d2745e9438 = 5: b8f56a5e67 ci: convert "install-dependencies.sh" to use "/bin/sh" 6: 61f108d954 = 6: ce4f0c766c ci: merge custom PATH directories 7: ef61b578da ! 7: 9fc462b578 ci: fix setup of custom path for GitLab CI @@ Commit message root user, but runs tests as a separate, unprivileged user. As their respective home directories are different, we will end up using two different custom path directories. Consequently, the unprivileged user - will not be able eto find the binaries that were setu up as root user. + will not be able to find the binaries that were set up as root user. Fix this issue by allowing CI to override the custom path, which allows GitLab to set up a constant value that isn't derived from "$HOME". + Signed-off-by: Patrick Steinhardt <ps@pks.im> + ## .gitlab-ci.yml ## @@ .gitlab-ci.yml: workflow: 8: 7748f87f8c = 8: e7a17d57e7 ci: merge scripts which install dependencies 9: f7399382f2 = 9: 720d5a4682 ci: make Perforce binaries executable for all users 10: b835ff8b78 = 10: 77f6d6ecaa ci: install JGit dependency 11: 7136c8b6c2 = 11: acf0c28550 t06xx: always execute backend-specific tests 12: cf4ee9c427 = 12: a9b71e8eea t0610: fix non-portable variable assignment 13: a9cd20eebc = 13: 218c694d2e t0612: add tests to exercise Git/JGit reftable compatibility base-commit: 436d4e5b14df49870a897f64fe92c0ddc7017e4c