| Message ID | 20220306231753.50277-23-philippe.mathieu.daude@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | host: Support macOS 12 | expand |
Hi Alex, Thomas, Daniel, Could you ack this patch? On 7/3/22 00:17, Philippe Mathieu-Daudé wrote: > From: Philippe Mathieu-Daudé <f4bug@amsat.org> > > Add support for macOS 12 build on Cirrus-CI, similarly to commit > 0e103a65ba1 ("gitlab: support for ... macOS 11 via cirrus-run"), > but with the following differences: > - Enable modules (configure --enable-modules) > - Do not run softfloat3 tests (make check-softfloat) > - Run Aarch64 qtests instead of x86_64 ones > > Generate the vars file by calling 'make lcitool-refresh'. > > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> > --- > .gitlab-ci.d/cirrus.yml | 16 ++++++++++++++++ > .gitlab-ci.d/cirrus/macos-12.vars | 16 ++++++++++++++++ > tests/lcitool/refresh | 1 + > 3 files changed, 33 insertions(+) > create mode 100644 .gitlab-ci.d/cirrus/macos-12.vars > > diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml > index b96b22e269..be1dce5d4e 100644 > --- a/.gitlab-ci.d/cirrus.yml > +++ b/.gitlab-ci.d/cirrus.yml > @@ -87,6 +87,22 @@ x64-macos-11-base-build: > PKG_CONFIG_PATH: /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig > TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat check-qtest-x86_64 > > +x64-macos-12-base-build: > + extends: .cirrus_build_job > + variables: > + NAME: macos-12 > + CIRRUS_VM_INSTANCE_TYPE: osx_instance > + CIRRUS_VM_IMAGE_SELECTOR: image > + CIRRUS_VM_IMAGE_NAME: monterey-base > + CIRRUS_VM_CPUS: 12 > + CIRRUS_VM_RAM: 24G > + UPDATE_COMMAND: brew update > + INSTALL_COMMAND: brew install > + PATH_EXTRA: /usr/local/opt/ccache/libexec:/usr/local/opt/gettext/bin > + PKG_CONFIG_PATH: /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig > + CONFIGURE_ARGS: --enable-modules > + TEST_TARGETS: check-unit check-block check-qapi-schema check-qtest-aarch64 > + > > # The following jobs run VM-based tests via KVM on a Linux-based Cirrus-CI job > .cirrus_kvm_job: > diff --git a/.gitlab-ci.d/cirrus/macos-12.vars b/.gitlab-ci.d/cirrus/macos-12.vars > new file mode 100644 > index 0000000000..a793258c64 > --- /dev/null > +++ b/.gitlab-ci.d/cirrus/macos-12.vars > @@ -0,0 +1,16 @@ > +# THIS FILE WAS AUTO-GENERATED > +# > +# $ lcitool variables macos-12 qemu > +# > +# https://gitlab.com/libvirt/libvirt-ci > + > +CCACHE='/usr/local/bin/ccache' > +CPAN_PKGS='Test::Harness' > +CROSS_PKGS='' > +MAKE='/usr/local/bin/gmake' > +NINJA='/usr/local/bin/ninja' > +PACKAGING_COMMAND='brew' > +PIP3='/usr/local/bin/pip3' > +PKGS='bash bc bzip2 capstone ccache cpanminus ctags curl dbus diffutils dtc gcovr gettext git glib gnu-sed gnutls gtk+3 jemalloc jpeg-turbo libepoxy libffi libgcrypt libiscsi libnfs libpng libslirp libssh libtasn1 libusb llvm lzo make meson ncurses nettle ninja perl pixman pkg-config python3 rpm2cpio sdl2 sdl2_image snappy sparse spice-protocol tesseract texinfo usbredir vde vte3 zlib zstd' > +PYPI_PKGS='PyYAML numpy pillow sphinx sphinx-rtd-theme virtualenv' > +PYTHON='/usr/local/bin/python3' > diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh > index 1f00281b44..e2f0bbd1b1 100755 > --- a/tests/lcitool/refresh > +++ b/tests/lcitool/refresh > @@ -105,6 +105,7 @@ try: > generate_cirrus("freebsd-12") > generate_cirrus("freebsd-13") > generate_cirrus("macos-11") > + generate_cirrus("macos-12") > > sys.exit(0) > except Exception as ex:
On Mon, Mar 07, 2022 at 12:17:53AM +0100, Philippe Mathieu-Daudé wrote: > From: Philippe Mathieu-Daudé <f4bug@amsat.org> > > Add support for macOS 12 build on Cirrus-CI, similarly to commit > 0e103a65ba1 ("gitlab: support for ... macOS 11 via cirrus-run"), > but with the following differences: > - Enable modules (configure --enable-modules) > - Do not run softfloat3 tests (make check-softfloat) > - Run Aarch64 qtests instead of x86_64 ones > > Generate the vars file by calling 'make lcitool-refresh'. > > Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> > --- > .gitlab-ci.d/cirrus.yml | 16 ++++++++++++++++ > .gitlab-ci.d/cirrus/macos-12.vars | 16 ++++++++++++++++ > tests/lcitool/refresh | 1 + > 3 files changed, 33 insertions(+) > create mode 100644 .gitlab-ci.d/cirrus/macos-12.vars Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Regards, Daniel
On 09/03/2022 11.24, Philippe Mathieu-Daudé wrote: > Hi Alex, Thomas, Daniel, > > Could you ack this patch? Basically fine for me, but can we really run additional cirrus-ci jobs by default? IIRC the parallel execution of those were quite limited for the free tier, so did you look close that we don't run into additional timeouts yet, due to delayed cirrus-ci jobs? Thomas > On 7/3/22 00:17, Philippe Mathieu-Daudé wrote: >> From: Philippe Mathieu-Daudé <f4bug@amsat.org> >> >> Add support for macOS 12 build on Cirrus-CI, similarly to commit >> 0e103a65ba1 ("gitlab: support for ... macOS 11 via cirrus-run"), >> but with the following differences: >> - Enable modules (configure --enable-modules) >> - Do not run softfloat3 tests (make check-softfloat) >> - Run Aarch64 qtests instead of x86_64 ones >> >> Generate the vars file by calling 'make lcitool-refresh'. >> >> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> >> --- >> .gitlab-ci.d/cirrus.yml | 16 ++++++++++++++++ >> .gitlab-ci.d/cirrus/macos-12.vars | 16 ++++++++++++++++ >> tests/lcitool/refresh | 1 + >> 3 files changed, 33 insertions(+) >> create mode 100644 .gitlab-ci.d/cirrus/macos-12.vars
On Wed, Mar 09, 2022 at 12:58:42PM +0100, Thomas Huth wrote: > On 09/03/2022 11.24, Philippe Mathieu-Daudé wrote: > > Hi Alex, Thomas, Daniel, > > > > Could you ack this patch? > > Basically fine for me, but can we really run additional cirrus-ci jobs by > default? IIRC the parallel execution of those were quite limited for the > free tier, so did you look close that we don't run into additional timeouts > yet, due to delayed cirrus-ci jobs? You can run 2 jobs in parallel in Cirrus. Beyond that they get queued/serialized We have a 1 hour job timeout. We have to expect jobs will sometimes run slower than normal. IOW if we have a job on Cirrus taking 30 minutes normally, we expect it will sometimes take 45 minutes. All this means that if we want Cirrus to be reliable and not time out, we can really only have 2 jobs by default, unless we can get the job execution time down to around 20 minutes to allow for serialization. We used to have terrible problems with cirrus timeouts when we were running 4 jobs concurrently (2 on staging and 2 on master). We addressed that in 9968de0a4a5470bd7b98dcd2fae5d5269908f16b by disabling the jobs on master. IOW, we really need to choose 1 macOS job and 1 FreeBSD job and others need to be marked manual. Regards, Daniel
On 9/3/22 13:33, Daniel P. Berrangé wrote: > On Wed, Mar 09, 2022 at 12:58:42PM +0100, Thomas Huth wrote: >> Basically fine for me, but can we really run additional cirrus-ci jobs by >> default? IIRC the parallel execution of those were quite limited for the >> free tier, so did you look close that we don't run into additional timeouts >> yet, due to delayed cirrus-ci jobs? > > You can run 2 jobs in parallel in Cirrus. Beyond that they > get queued/serialized > > We have a 1 hour job timeout. > > We have to expect jobs will sometimes run slower than normal. > > IOW if we have a job on Cirrus taking 30 minutes normally, we > expect it will sometimes take 45 minutes. > > All this means that if we want Cirrus to be reliable and not > time out, we can really only have 2 jobs by default, unless > we can get the job execution time down to around 20 minutes > to allow for serialization. > > We used to have terrible problems with cirrus timeouts when > we were running 4 jobs concurrently (2 on staging and 2 on > master). We addressed that in 9968de0a4a5470bd7b98dcd2fae5d5269908f16b > by disabling the jobs on master. > > IOW, we really need to choose 1 macOS job and 1 FreeBSD job > and others need to be marked manual. Not sure which job to choose yet. Per the first google hits we still want to cover Catalina first: https://www.statista.com/statistics/944559/worldwide-macos-version-market-share/ Would it be beneficial to have a 1 per OS job during PR, and other jobs run nightly (once a day, not per PR)?
On 09/03/2022 13.50, Philippe Mathieu-Daudé wrote: > Would it be beneficial to have a 1 per OS job during PR, and > other jobs run nightly (once a day, not per PR)? Is there a way to trigger nightly runs in gitlab? Thomas
On Wed, Mar 09, 2022 at 01:52:15PM +0100, Thomas Huth wrote: > On 09/03/2022 13.50, Philippe Mathieu-Daudé wrote: > > > Would it be beneficial to have a 1 per OS job during PR, and > > other jobs run nightly (once a day, not per PR)? > > Is there a way to trigger nightly runs in gitlab? Yes, in the repo CI settings create a scheduled job, using cron syntax. Relies on the job "rules" in the config file not excluding scheduled execution of course. Regards, Daniel
On Wed, Mar 09, 2022 at 01:50:34PM +0100, Philippe Mathieu-Daudé wrote: > On 9/3/22 13:33, Daniel P. Berrangé wrote: > > On Wed, Mar 09, 2022 at 12:58:42PM +0100, Thomas Huth wrote: > > > > Basically fine for me, but can we really run additional cirrus-ci jobs by > > > default? IIRC the parallel execution of those were quite limited for the > > > free tier, so did you look close that we don't run into additional timeouts > > > yet, due to delayed cirrus-ci jobs? > > > > You can run 2 jobs in parallel in Cirrus. Beyond that they > > get queued/serialized > > > > We have a 1 hour job timeout. > > > > We have to expect jobs will sometimes run slower than normal. > > > > IOW if we have a job on Cirrus taking 30 minutes normally, we > > expect it will sometimes take 45 minutes. > > > > All this means that if we want Cirrus to be reliable and not > > time out, we can really only have 2 jobs by default, unless > > we can get the job execution time down to around 20 minutes > > to allow for serialization. > > > > We used to have terrible problems with cirrus timeouts when > > we were running 4 jobs concurrently (2 on staging and 2 on > > master). We addressed that in 9968de0a4a5470bd7b98dcd2fae5d5269908f16b > > by disabling the jobs on master. > > > > IOW, we really need to choose 1 macOS job and 1 FreeBSD job > > and others need to be marked manual. > > Not sure which job to choose yet. Per the first google hits we > still want to cover Catalina first: > https://www.statista.com/statistics/944559/worldwide-macos-version-market-share/ My general gut feeling is usually to prioritize testing older versions as they tend to be more widely used, and we want to avoid regresions on stuff that has been around the longest. Compatibility problems with new releases tend to get reported by users/maintainers and would not be regressions, but rather enhancements to support the new platform. But it is not really an easy decision, as clearly we would prefer to test both old and new if we had the ability. Having one as a nightly job is a reasonable compromise. With regards, Daniel
On Wed, 9 Mar 2022 at 13:02, Daniel P. Berrangé <berrange@redhat.com> wrote: > > On Wed, Mar 09, 2022 at 01:50:34PM +0100, Philippe Mathieu-Daudé wrote: > > Not sure which job to choose yet. Per the first google hits we > > still want to cover Catalina first: > > https://www.statista.com/statistics/944559/worldwide-macos-version-market-share/ ...that page says that the figures for Catalina are "incorrectly high", so not a very useful set of data for these purposes I think. > My general gut feeling is usually to prioritize testing older versions > as they tend to be more widely used, and we want to avoid regresions > on stuff that has been around the longest. Compatibility problems with > new releases tend to get reported by users/maintainers and would not > be regressions, but rather enhancements to support the new platform. Note that technically speaking Catalina (10.15) has already fallen off the end of our support policy (which currently requires Big Sur (11) and Monterey (12) support, as the two most recent versions). My personal macos x86 laptop is still running Catalina, though, because Big Sur dropped support for that hardware :-/ That suggests that maybe Big Sur would be what we should go for now if we have to pick just one for the CI. thanks -- PMM
On 9/3/22 14:09, Peter Maydell wrote: > On Wed, 9 Mar 2022 at 13:02, Daniel P. Berrangé <berrange@redhat.com> wrote: >> >> On Wed, Mar 09, 2022 at 01:50:34PM +0100, Philippe Mathieu-Daudé wrote: >>> Not sure which job to choose yet. Per the first google hits we >>> still want to cover Catalina first: >>> https://www.statista.com/statistics/944559/worldwide-macos-version-market-share/ > > ...that page says that the figures for Catalina are "incorrectly > high", so not a very useful set of data for these purposes I think. > >> My general gut feeling is usually to prioritize testing older versions >> as they tend to be more widely used, and we want to avoid regresions >> on stuff that has been around the longest. Compatibility problems with >> new releases tend to get reported by users/maintainers and would not >> be regressions, but rather enhancements to support the new platform. > > Note that technically speaking Catalina (10.15) has already fallen off > the end of our support policy (which currently requires Big Sur (11) > and Monterey (12) support, as the two most recent versions). My > personal macos x86 laptop is still running Catalina, though, > because Big Sur dropped support for that hardware :-/ > That suggests that maybe Big Sur would be what we should go for > now if we have to pick just one for the CI. OK. Since I'm running Monterey (12) on my workstation, I'll make this job manual / nightly, and keep the Big Sur (11) on CI. I'll let Daniel / Thomas / Alex decide which FreeBSD job to demote to manual. Regards, Phil.
diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml index b96b22e269..be1dce5d4e 100644 --- a/.gitlab-ci.d/cirrus.yml +++ b/.gitlab-ci.d/cirrus.yml @@ -87,6 +87,22 @@ x64-macos-11-base-build: PKG_CONFIG_PATH: /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat check-qtest-x86_64 +x64-macos-12-base-build: + extends: .cirrus_build_job + variables: + NAME: macos-12 + CIRRUS_VM_INSTANCE_TYPE: osx_instance + CIRRUS_VM_IMAGE_SELECTOR: image + CIRRUS_VM_IMAGE_NAME: monterey-base + CIRRUS_VM_CPUS: 12 + CIRRUS_VM_RAM: 24G + UPDATE_COMMAND: brew update + INSTALL_COMMAND: brew install + PATH_EXTRA: /usr/local/opt/ccache/libexec:/usr/local/opt/gettext/bin + PKG_CONFIG_PATH: /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig + CONFIGURE_ARGS: --enable-modules + TEST_TARGETS: check-unit check-block check-qapi-schema check-qtest-aarch64 + # The following jobs run VM-based tests via KVM on a Linux-based Cirrus-CI job .cirrus_kvm_job: diff --git a/.gitlab-ci.d/cirrus/macos-12.vars b/.gitlab-ci.d/cirrus/macos-12.vars new file mode 100644 index 0000000000..a793258c64 --- /dev/null +++ b/.gitlab-ci.d/cirrus/macos-12.vars @@ -0,0 +1,16 @@ +# THIS FILE WAS AUTO-GENERATED +# +# $ lcitool variables macos-12 qemu +# +# https://gitlab.com/libvirt/libvirt-ci + +CCACHE='/usr/local/bin/ccache' +CPAN_PKGS='Test::Harness' +CROSS_PKGS='' +MAKE='/usr/local/bin/gmake' +NINJA='/usr/local/bin/ninja' +PACKAGING_COMMAND='brew' +PIP3='/usr/local/bin/pip3' +PKGS='bash bc bzip2 capstone ccache cpanminus ctags curl dbus diffutils dtc gcovr gettext git glib gnu-sed gnutls gtk+3 jemalloc jpeg-turbo libepoxy libffi libgcrypt libiscsi libnfs libpng libslirp libssh libtasn1 libusb llvm lzo make meson ncurses nettle ninja perl pixman pkg-config python3 rpm2cpio sdl2 sdl2_image snappy sparse spice-protocol tesseract texinfo usbredir vde vte3 zlib zstd' +PYPI_PKGS='PyYAML numpy pillow sphinx sphinx-rtd-theme virtualenv' +PYTHON='/usr/local/bin/python3' diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh index 1f00281b44..e2f0bbd1b1 100755 --- a/tests/lcitool/refresh +++ b/tests/lcitool/refresh @@ -105,6 +105,7 @@ try: generate_cirrus("freebsd-12") generate_cirrus("freebsd-13") generate_cirrus("macos-11") + generate_cirrus("macos-12") sys.exit(0) except Exception as ex: