| Message ID | 20241004172723.2391026-1-andrew.cooper3@citrix.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | CI: Fix builds following qemu-xen update | expand |
On Fri, Oct 04, 2024 at 06:27:23PM +0100, Andrew Cooper wrote: > A recent update to qemu-xen has bumped the build requirements, with Python 3.8 > being the new baseline but also needing the 'ensurepip' and 'tomllib/tomli' > packages. > > * Ubuntu/Debian package 'ensurepip' separately, but it can be obtained by > installing the python3-venv package. > > * 'tomllib' was added to the python standard library in Python 3.11, but > previously it was a separate package named 'tomli'. > > In terms of changes required to build QEMU: > > * Ubuntu 24.04 (Noble) has Python 3.12 so only needs python3-venv > > * Ubuntu 22.04 (Jammy) has Python 3.10 but does have a python3-tomli package > that QEMU is happy with. > > * FreeBSD has Python 3.9, but Python 3.11 is available. > > In terms of exclusions: > > * Ubuntu 20.04 (Focal) has Python 3.8, but lacks any kind of tomli package. > > * Fedora 29 (Python 3.7), OpenSUSE Leap 15.6 (Python 3.6), and Ubuntu > 18.04/Bionic (Python 3.6) are now too old. > > Detecting tomllib/tomli is more than can fit in build's oneliner, so break it > out into a proper script. > > Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> The changes on the gitlab side look fine. I don't know if the changes on the cirrus side are ok, but at least the seems to work, after looking at the build logs. So: Reviewed-by: Anthony PERARD <anthony.perard@vates.tech> Thanks,
On 04/10/2024 7:14 pm, Anthony PERARD wrote: > On Fri, Oct 04, 2024 at 06:27:23PM +0100, Andrew Cooper wrote: >> A recent update to qemu-xen has bumped the build requirements, with Python 3.8 >> being the new baseline but also needing the 'ensurepip' and 'tomllib/tomli' >> packages. >> >> * Ubuntu/Debian package 'ensurepip' separately, but it can be obtained by >> installing the python3-venv package. >> >> * 'tomllib' was added to the python standard library in Python 3.11, but >> previously it was a separate package named 'tomli'. >> >> In terms of changes required to build QEMU: >> >> * Ubuntu 24.04 (Noble) has Python 3.12 so only needs python3-venv >> >> * Ubuntu 22.04 (Jammy) has Python 3.10 but does have a python3-tomli package >> that QEMU is happy with. >> >> * FreeBSD has Python 3.9, but Python 3.11 is available. >> >> In terms of exclusions: >> >> * Ubuntu 20.04 (Focal) has Python 3.8, but lacks any kind of tomli package. >> >> * Fedora 29 (Python 3.7), OpenSUSE Leap 15.6 (Python 3.6), and Ubuntu >> 18.04/Bionic (Python 3.6) are now too old. >> >> Detecting tomllib/tomli is more than can fit in build's oneliner, so break it >> out into a proper script. >> >> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> > The changes on the gitlab side look fine. I don't know if the changes on > the cirrus side are ok, but at least the seems to work, after looking at > the build logs. So: > > Reviewed-by: Anthony PERARD <anthony.perard@vates.tech> Thanks. Roger has given his ack on Matrix, so I'll get this committed right away to unblock things. ~Andrew
diff --git a/.cirrus.yml b/.cirrus.yml index 1c2a6cb8120e..00e4c57678c2 100644 --- a/.cirrus.yml +++ b/.cirrus.yml @@ -7,10 +7,11 @@ freebsd_template: &FREEBSD_TEMPLATE install_script: pkg install -y seabios gmake ninja bash pkgconf python bison perl5 yajl lzo2 pixman argp-standalone - libxml2 glib git + libxml2 glib git python311 build_script: - cc --version + - export PYTHON=/usr/local/bin/python3.11 - ./configure --with-system-seabios=/usr/local/share/seabios/bios.bin - gmake -j`sysctl -n hw.ncpu` clang=y diff --git a/automation/build/ubuntu/22.04-x86_64.dockerfile b/automation/build/ubuntu/22.04-x86_64.dockerfile index 230903f624d9..6aa3c4d1881d 100644 --- a/automation/build/ubuntu/22.04-x86_64.dockerfile +++ b/automation/build/ubuntu/22.04-x86_64.dockerfile @@ -62,6 +62,8 @@ RUN <<EOF meson ninja-build python3-packaging + python3-tomli + python3-venv ) apt-get -y --no-install-recommends install "${DEPS[@]}" diff --git a/automation/build/ubuntu/24.04-x86_64.dockerfile b/automation/build/ubuntu/24.04-x86_64.dockerfile index 277f92facfd9..c46d152abf10 100644 --- a/automation/build/ubuntu/24.04-x86_64.dockerfile +++ b/automation/build/ubuntu/24.04-x86_64.dockerfile @@ -62,6 +62,7 @@ RUN <<EOF meson ninja-build python3-packaging + python3-venv ) apt-get -y --no-install-recommends install "${DEPS[@]}" diff --git a/automation/scripts/build b/automation/scripts/build index 1879c1db6d0d..34416297a4b7 100755 --- a/automation/scripts/build +++ b/automation/scripts/build @@ -91,9 +91,9 @@ else cfgargs+=("--with-extra-qemuu-configure-args=\"--disable-werror\"") fi - # Qemu requires Python 3.5 or later, and ninja + # Qemu requires Python 3.8 or later, and ninja # and Clang 10 or later - if ! type python3 || python3 -c "import sys; res = sys.version_info < (3, 5); exit(not(res))" \ + if ! type python3 || ! python3 automation/scripts/qemu-deps-check.py \ || [[ "$cc_is_clang" == y && "$cc_ver" -lt 0x0a0000 ]] \ || ! type ninja; then cfgargs+=("--with-system-qemu=/bin/false") diff --git a/automation/scripts/qemu-deps-check.py b/automation/scripts/qemu-deps-check.py new file mode 100755 index 000000000000..f6188afb3e2a --- /dev/null +++ b/automation/scripts/qemu-deps-check.py @@ -0,0 +1,19 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +import sys + +if sys.version_info < (3, 8): + print("Python %d.%d.%d too old" % + (sys.version_info.major, + sys.version_info.minor, + sys.version_info.micro)) + exit(1) + +try: + import tomllib +except ImportError: + try: + import tomli + except ImportError: + print("No tomli") + exit(1)
A recent update to qemu-xen has bumped the build requirements, with Python 3.8 being the new baseline but also needing the 'ensurepip' and 'tomllib/tomli' packages. * Ubuntu/Debian package 'ensurepip' separately, but it can be obtained by installing the python3-venv package. * 'tomllib' was added to the python standard library in Python 3.11, but previously it was a separate package named 'tomli'. In terms of changes required to build QEMU: * Ubuntu 24.04 (Noble) has Python 3.12 so only needs python3-venv * Ubuntu 22.04 (Jammy) has Python 3.10 but does have a python3-tomli package that QEMU is happy with. * FreeBSD has Python 3.9, but Python 3.11 is available. In terms of exclusions: * Ubuntu 20.04 (Focal) has Python 3.8, but lacks any kind of tomli package. * Fedora 29 (Python 3.7), OpenSUSE Leap 15.6 (Python 3.6), and Ubuntu 18.04/Bionic (Python 3.6) are now too old. Detecting tomllib/tomli is more than can fit in build's oneliner, so break it out into a proper script. Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> --- CC: Anthony PERARD <anthony.perard@vates.tech> CC: Stefano Stabellini <sstabellini@kernel.org> CC: Michal Orzel <michal.orzel@amd.com> CC: Doug Goldstein <cardoe@cardoe.com> CC: Roger Pau Monné <roger.pau@citrix.com> Using local container fixes: https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1482180312 FreeBSD (13,14): https://cirrus-ci.com/build/5553798175784960 FreeBSD 15 is broken generally again so I can't confirm the fix there. --- .cirrus.yml | 3 ++- .../build/ubuntu/22.04-x86_64.dockerfile | 2 ++ .../build/ubuntu/24.04-x86_64.dockerfile | 1 + automation/scripts/build | 4 ++-- automation/scripts/qemu-deps-check.py | 19 +++++++++++++++++++ 5 files changed, 26 insertions(+), 3 deletions(-) create mode 100755 automation/scripts/qemu-deps-check.py