| Message ID | cb8cefc20f373a3516695e7cbee975132553ea95.1714973381.git.ps@pks.im (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | 5ca0c455f1d020156dc352a209fb1a3a6e548e3d |
| Headers | show |
| Series | ci: fix Python dependency on Ubuntu 24.04 | expand |
On 24/05/06 07:35AM, Patrick Steinhardt wrote: > Newer versions of Ubuntu have dropped Python 2 starting with Ubuntu > 23.04. By default though, our CI setups will try to use that Python > version on all Ubuntu-based jobs except for the "linux-gcc" one. Naive question, why were the "linux-gcc" jobs the only ones using Python 3? > > We didn't notice this issue due to two reasons: > > - The "ubuntu:latest" tag always points to the latest LTS release. > Until a few weeks ago this was Ubuntu 22.04, which still had Python > 2. > > - Our Docker-based CI jobs had their own script to install > dependencies until 9cdeb34b96 (ci: merge scripts which install > dependencies, 2024-04-12), where we didn't even try to install > Python at all for many of them. > > Since the CI refactorings have originally been implemented, Ubuntu > 24.04 was released, and it being an LTS versions means that the "latest" > tag now points to that Python-2-less version. Consequently, those jobs > that use "ubuntu:latest" broke. > > Address this by using Python 2 on Ubuntu 20.04, only, whereas we use > Python 3 on all other Ubuntu jobs. Eventually, we should think about > dropping support for Python 2 completely. > > Reported-by: Justin Tobler <jtobler@gmail.com> > Signed-off-by: Patrick Steinhardt <ps@pks.im> > --- > > Note: this topic depends on ps/ci-test-with-jgit at 70b81fbf3c (t0612: > add tests to exercise Git/JGit reftable compatibility, 2024-04-12). > > ci/lib.sh | 8 ++++++-- > 1 file changed, 6 insertions(+), 2 deletions(-) > > diff --git a/ci/lib.sh b/ci/lib.sh > index 473a2d0348..273f3540a6 100755 > --- a/ci/lib.sh > +++ b/ci/lib.sh > @@ -325,9 +325,13 @@ ubuntu-*) > break > fi > > - PYTHON_PACKAGE=python2 > - if test "$jobname" = linux-gcc With this change, "linux-gcc" jobs using ubuntu 20.04 will now use Python 2. Is that what we want? -Justin > + # Python 2 is end of life, and Ubuntu 23.04 and newer don't actually > + # have it anymore. We thus only test with Python 2 on older LTS > + # releases. > + if "$distro" = "ubuntu-20.04" > then > + PYTHON_PACKAGE=python2 > + else > PYTHON_PACKAGE=python3 > fi > MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/$PYTHON_PACKAGE" > > base-commit: b6db6b1598946fbf777e55ff0d187b11ff3bd21f > -- > 2.45.0 >
On Mon, May 06, 2024 at 07:55:25AM -0500, Justin Tobler wrote: > On 24/05/06 07:35AM, Patrick Steinhardt wrote: > > Newer versions of Ubuntu have dropped Python 2 starting with Ubuntu > > 23.04. By default though, our CI setups will try to use that Python > > version on all Ubuntu-based jobs except for the "linux-gcc" one. > > Naive question, why were the "linux-gcc" jobs the only ones using > Python 3? This has been introduced in 6bb40ed20a (ci: use python3 in linux-gcc and osx-gcc and python2 elsewhere, 2020-01-23). It's been a first step towards exercising Python 3 in our CI pipeline. The jobs were basically picked at random, there is no inherent reason. So ultimately, it does not matter which jobs use what. But what does matter is whether the image even supports Python 2. And as we nowadays do have a fair split of jobs using "ubuntu:20.04" and "ubuntu:latest", I'd just use that the the criterium to pick the Python version. Once we drop support for Ubuntu 20.04, we can then more forward and stop supporting Python 2 altogether. Or maybe even earlier than that -- I do not think it is reasonable to ask Git to maintain compatibility with a project that is end of life already. [snip] > > diff --git a/ci/lib.sh b/ci/lib.sh > > index 473a2d0348..273f3540a6 100755 > > --- a/ci/lib.sh > > +++ b/ci/lib.sh > > @@ -325,9 +325,13 @@ ubuntu-*) > > break > > fi > > > > - PYTHON_PACKAGE=python2 > > - if test "$jobname" = linux-gcc > > With this change, "linux-gcc" jobs using ubuntu 20.04 will now use > Python 2. Is that what we want? Well, as explained above, the selection was arbitrary in the first place. Now it's less so as the choice is mandated by what is actually supported. Patrick
Patrick Steinhardt <ps@pks.im> writes: > So ultimately, it does not matter which jobs use what. But what does > matter is whether the image even supports Python 2. And as we nowadays > do have a fair split of jobs using "ubuntu:20.04" and "ubuntu:latest", > I'd just use that the the criterium to pick the Python version. > > Once we drop support for Ubuntu 20.04, we can then more forward and stop > supporting Python 2 altogether. Or maybe even earlier than that -- I do > not think it is reasonable to ask Git to maintain compatibility with a > project that is end of life already. Yup. If an distro sticks to Python 2 for whatever reason, they may want to donate and dedicate CI resources, but otherwise it is not really our job to keep compatibility on our side. >> With this change, "linux-gcc" jobs using ubuntu 20.04 will now use >> Python 2. Is that what we want? > > Well, as explained above, the selection was arbitrary in the first > place. Now it's less so as the choice is mandated by what is actually > supported. Yup, as long as there is one that supports Python 2 among the distros we happen to use for our CIs, we can pick that one. Otherwise we should drop. Dropping Python 2 support does not have to wait for Git 3.0, I would say. Thanks.
On 24/05/06 07:35AM, Patrick Steinhardt wrote: > Newer versions of Ubuntu have dropped Python 2 starting with Ubuntu > 23.04. By default though, our CI setups will try to use that Python > version on all Ubuntu-based jobs except for the "linux-gcc" one. > > We didn't notice this issue due to two reasons: > > - The "ubuntu:latest" tag always points to the latest LTS release. > Until a few weeks ago this was Ubuntu 22.04, which still had Python > 2. > > - Our Docker-based CI jobs had their own script to install > dependencies until 9cdeb34b96 (ci: merge scripts which install > dependencies, 2024-04-12), where we didn't even try to install > Python at all for many of them. > > Since the CI refactorings have originally been implemented, Ubuntu > 24.04 was released, and it being an LTS versions means that the "latest" > tag now points to that Python-2-less version. Consequently, those jobs > that use "ubuntu:latest" broke. > > Address this by using Python 2 on Ubuntu 20.04, only, whereas we use > Python 3 on all other Ubuntu jobs. Eventually, we should think about > dropping support for Python 2 completely. > > Reported-by: Justin Tobler <jtobler@gmail.com> Not a big deal, but the email is slightly off. Should be: <jltobler@gmail.com> Otherwise this patch looks good to me. :) -Justin > Signed-off-by: Patrick Steinhardt <ps@pks.im> ...
Justin Tobler <jltobler@gmail.com> writes: >> Reported-by: Justin Tobler <jtobler@gmail.com> > > Not a big deal, but the email is slightly off. Should be: > <jltobler@gmail.com> > > Otherwise this patch looks good to me. :) The patch e-mail also went to that "slight off" address, which is funny. I've amended the Reported-by: line locally while queuing. Thanks, both of you.
On Mon, May 06, 2024 at 02:06:04PM -0500, Justin Tobler wrote: > On 24/05/06 07:35AM, Patrick Steinhardt wrote: > > Newer versions of Ubuntu have dropped Python 2 starting with Ubuntu > > 23.04. By default though, our CI setups will try to use that Python > > version on all Ubuntu-based jobs except for the "linux-gcc" one. > > > > We didn't notice this issue due to two reasons: > > > > - The "ubuntu:latest" tag always points to the latest LTS release. > > Until a few weeks ago this was Ubuntu 22.04, which still had Python > > 2. > > > > - Our Docker-based CI jobs had their own script to install > > dependencies until 9cdeb34b96 (ci: merge scripts which install > > dependencies, 2024-04-12), where we didn't even try to install > > Python at all for many of them. > > > > Since the CI refactorings have originally been implemented, Ubuntu > > 24.04 was released, and it being an LTS versions means that the "latest" > > tag now points to that Python-2-less version. Consequently, those jobs > > that use "ubuntu:latest" broke. > > > > Address this by using Python 2 on Ubuntu 20.04, only, whereas we use > > Python 3 on all other Ubuntu jobs. Eventually, we should think about > > dropping support for Python 2 completely. > > > > Reported-by: Justin Tobler <jtobler@gmail.com> > > Not a big deal, but the email is slightly off. Should be: > <jltobler@gmail.com> > > Otherwise this patch looks good to me. :) You know, every single time I copy over addresses. This one time I didn't and of course I immediately mistype it. Well, lesson learned, copy & paste it is. Patrick
diff --git a/ci/lib.sh b/ci/lib.sh index 473a2d0348..273f3540a6 100755 --- a/ci/lib.sh +++ b/ci/lib.sh @@ -325,9 +325,13 @@ ubuntu-*) break fi - PYTHON_PACKAGE=python2 - if test "$jobname" = linux-gcc + # Python 2 is end of life, and Ubuntu 23.04 and newer don't actually + # have it anymore. We thus only test with Python 2 on older LTS + # releases. + if "$distro" = "ubuntu-20.04" then + PYTHON_PACKAGE=python2 + else PYTHON_PACKAGE=python3 fi MAKEFLAGS="$MAKEFLAGS PYTHON_PATH=/usr/bin/$PYTHON_PACKAGE"
Newer versions of Ubuntu have dropped Python 2 starting with Ubuntu 23.04. By default though, our CI setups will try to use that Python version on all Ubuntu-based jobs except for the "linux-gcc" one. We didn't notice this issue due to two reasons: - The "ubuntu:latest" tag always points to the latest LTS release. Until a few weeks ago this was Ubuntu 22.04, which still had Python 2. - Our Docker-based CI jobs had their own script to install dependencies until 9cdeb34b96 (ci: merge scripts which install dependencies, 2024-04-12), where we didn't even try to install Python at all for many of them. Since the CI refactorings have originally been implemented, Ubuntu 24.04 was released, and it being an LTS versions means that the "latest" tag now points to that Python-2-less version. Consequently, those jobs that use "ubuntu:latest" broke. Address this by using Python 2 on Ubuntu 20.04, only, whereas we use Python 3 on all other Ubuntu jobs. Eventually, we should think about dropping support for Python 2 completely. Reported-by: Justin Tobler <jtobler@gmail.com> Signed-off-by: Patrick Steinhardt <ps@pks.im> --- Note: this topic depends on ps/ci-test-with-jgit at 70b81fbf3c (t0612: add tests to exercise Git/JGit reftable compatibility, 2024-04-12). ci/lib.sh | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) base-commit: b6db6b1598946fbf777e55ff0d187b11ff3bd21f