diff mbox series

[v7,22/22] gitlab-ci: Support macOS 12 via cirrus-run

Message ID 20220306231753.50277-23-philippe.mathieu.daude@gmail.com (mailing list archive)
State New, archived
Headers show
Series host: Support macOS 12 | expand

Commit Message

Philippe Mathieu-Daudé March 6, 2022, 11:17 p.m. UTC
From: Philippe Mathieu-Daudé <f4bug@amsat.org>

Add support for macOS 12 build on Cirrus-CI, similarly to commit
0e103a65ba1 ("gitlab: support for ... macOS 11 via cirrus-run"),
but with the following differences:
 - Enable modules (configure --enable-modules)
 - Do not run softfloat3 tests (make check-softfloat)
 - Run Aarch64 qtests instead of x86_64 ones

Generate the vars file by calling 'make lcitool-refresh'.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
---
 .gitlab-ci.d/cirrus.yml           | 16 ++++++++++++++++
 .gitlab-ci.d/cirrus/macos-12.vars | 16 ++++++++++++++++
 tests/lcitool/refresh             |  1 +
 3 files changed, 33 insertions(+)
 create mode 100644 .gitlab-ci.d/cirrus/macos-12.vars

Comments

Philippe Mathieu-Daudé March 9, 2022, 10:24 a.m. UTC | #1
Hi Alex, Thomas, Daniel,

Could you ack this patch?

On 7/3/22 00:17, Philippe Mathieu-Daudé wrote:
> From: Philippe Mathieu-Daudé <f4bug@amsat.org>
> 
> Add support for macOS 12 build on Cirrus-CI, similarly to commit
> 0e103a65ba1 ("gitlab: support for ... macOS 11 via cirrus-run"),
> but with the following differences:
>   - Enable modules (configure --enable-modules)
>   - Do not run softfloat3 tests (make check-softfloat)
>   - Run Aarch64 qtests instead of x86_64 ones
> 
> Generate the vars file by calling 'make lcitool-refresh'.
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>   .gitlab-ci.d/cirrus.yml           | 16 ++++++++++++++++
>   .gitlab-ci.d/cirrus/macos-12.vars | 16 ++++++++++++++++
>   tests/lcitool/refresh             |  1 +
>   3 files changed, 33 insertions(+)
>   create mode 100644 .gitlab-ci.d/cirrus/macos-12.vars
> 
> diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
> index b96b22e269..be1dce5d4e 100644
> --- a/.gitlab-ci.d/cirrus.yml
> +++ b/.gitlab-ci.d/cirrus.yml
> @@ -87,6 +87,22 @@ x64-macos-11-base-build:
>       PKG_CONFIG_PATH: /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig
>       TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat check-qtest-x86_64
>   
> +x64-macos-12-base-build:
> +  extends: .cirrus_build_job
> +  variables:
> +    NAME: macos-12
> +    CIRRUS_VM_INSTANCE_TYPE: osx_instance
> +    CIRRUS_VM_IMAGE_SELECTOR: image
> +    CIRRUS_VM_IMAGE_NAME: monterey-base
> +    CIRRUS_VM_CPUS: 12
> +    CIRRUS_VM_RAM: 24G
> +    UPDATE_COMMAND: brew update
> +    INSTALL_COMMAND: brew install
> +    PATH_EXTRA: /usr/local/opt/ccache/libexec:/usr/local/opt/gettext/bin
> +    PKG_CONFIG_PATH: /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig
> +    CONFIGURE_ARGS: --enable-modules
> +    TEST_TARGETS: check-unit check-block check-qapi-schema check-qtest-aarch64
> +
>   
>   # The following jobs run VM-based tests via KVM on a Linux-based Cirrus-CI job
>   .cirrus_kvm_job:
> diff --git a/.gitlab-ci.d/cirrus/macos-12.vars b/.gitlab-ci.d/cirrus/macos-12.vars
> new file mode 100644
> index 0000000000..a793258c64
> --- /dev/null
> +++ b/.gitlab-ci.d/cirrus/macos-12.vars
> @@ -0,0 +1,16 @@
> +# THIS FILE WAS AUTO-GENERATED
> +#
> +#  $ lcitool variables macos-12 qemu
> +#
> +# https://gitlab.com/libvirt/libvirt-ci
> +
> +CCACHE='/usr/local/bin/ccache'
> +CPAN_PKGS='Test::Harness'
> +CROSS_PKGS=''
> +MAKE='/usr/local/bin/gmake'
> +NINJA='/usr/local/bin/ninja'
> +PACKAGING_COMMAND='brew'
> +PIP3='/usr/local/bin/pip3'
> +PKGS='bash bc bzip2 capstone ccache cpanminus ctags curl dbus diffutils dtc gcovr gettext git glib gnu-sed gnutls gtk+3 jemalloc jpeg-turbo libepoxy libffi libgcrypt libiscsi libnfs libpng libslirp libssh libtasn1 libusb llvm lzo make meson ncurses nettle ninja perl pixman pkg-config python3 rpm2cpio sdl2 sdl2_image snappy sparse spice-protocol tesseract texinfo usbredir vde vte3 zlib zstd'
> +PYPI_PKGS='PyYAML numpy pillow sphinx sphinx-rtd-theme virtualenv'
> +PYTHON='/usr/local/bin/python3'
> diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
> index 1f00281b44..e2f0bbd1b1 100755
> --- a/tests/lcitool/refresh
> +++ b/tests/lcitool/refresh
> @@ -105,6 +105,7 @@ try:
>      generate_cirrus("freebsd-12")
>      generate_cirrus("freebsd-13")
>      generate_cirrus("macos-11")
> +   generate_cirrus("macos-12")
>   
>      sys.exit(0)
>   except Exception as ex:
Daniel P. Berrangé March 9, 2022, 10:28 a.m. UTC | #2
On Mon, Mar 07, 2022 at 12:17:53AM +0100, Philippe Mathieu-Daudé wrote:
> From: Philippe Mathieu-Daudé <f4bug@amsat.org>
> 
> Add support for macOS 12 build on Cirrus-CI, similarly to commit
> 0e103a65ba1 ("gitlab: support for ... macOS 11 via cirrus-run"),
> but with the following differences:
>  - Enable modules (configure --enable-modules)
>  - Do not run softfloat3 tests (make check-softfloat)
>  - Run Aarch64 qtests instead of x86_64 ones
> 
> Generate the vars file by calling 'make lcitool-refresh'.
> 
> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> ---
>  .gitlab-ci.d/cirrus.yml           | 16 ++++++++++++++++
>  .gitlab-ci.d/cirrus/macos-12.vars | 16 ++++++++++++++++
>  tests/lcitool/refresh             |  1 +
>  3 files changed, 33 insertions(+)
>  create mode 100644 .gitlab-ci.d/cirrus/macos-12.vars

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


Regards,
Daniel
Thomas Huth March 9, 2022, 11:58 a.m. UTC | #3
On 09/03/2022 11.24, Philippe Mathieu-Daudé wrote:
> Hi Alex, Thomas, Daniel,
> 
> Could you ack this patch?

Basically fine for me, but can we really run additional cirrus-ci jobs by 
default? IIRC the parallel execution of those were quite limited for the 
free tier, so did you look close that we don't run into additional timeouts 
yet, due to delayed cirrus-ci jobs?

  Thomas


> On 7/3/22 00:17, Philippe Mathieu-Daudé wrote:
>> From: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>
>> Add support for macOS 12 build on Cirrus-CI, similarly to commit
>> 0e103a65ba1 ("gitlab: support for ... macOS 11 via cirrus-run"),
>> but with the following differences:
>>   - Enable modules (configure --enable-modules)
>>   - Do not run softfloat3 tests (make check-softfloat)
>>   - Run Aarch64 qtests instead of x86_64 ones
>>
>> Generate the vars file by calling 'make lcitool-refresh'.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>> ---
>>   .gitlab-ci.d/cirrus.yml           | 16 ++++++++++++++++
>>   .gitlab-ci.d/cirrus/macos-12.vars | 16 ++++++++++++++++
>>   tests/lcitool/refresh             |  1 +
>>   3 files changed, 33 insertions(+)
>>   create mode 100644 .gitlab-ci.d/cirrus/macos-12.vars
Daniel P. Berrangé March 9, 2022, 12:33 p.m. UTC | #4
On Wed, Mar 09, 2022 at 12:58:42PM +0100, Thomas Huth wrote:
> On 09/03/2022 11.24, Philippe Mathieu-Daudé wrote:
> > Hi Alex, Thomas, Daniel,
> > 
> > Could you ack this patch?
>
> Basically fine for me, but can we really run additional cirrus-ci jobs by
> default? IIRC the parallel execution of those were quite limited for the
> free tier, so did you look close that we don't run into additional timeouts
> yet, due to delayed cirrus-ci jobs?

You can run 2 jobs in parallel in Cirrus. Beyond that they
get queued/serialized

We have a 1 hour job timeout.

We have to expect jobs will sometimes run slower than normal.

IOW if we have a job on Cirrus taking 30 minutes normally, we
expect it will sometimes take 45 minutes.

All this means that if we want Cirrus to be reliable and not
time out, we can really only have 2 jobs by default, unless
we can get the job execution time down to around 20 minutes
to allow for serialization.

We used to have terrible problems with cirrus timeouts when
we were running 4 jobs concurrently (2 on staging and 2 on
master). We addressed that in 9968de0a4a5470bd7b98dcd2fae5d5269908f16b
by disabling the jobs on master.

IOW, we really need to choose 1 macOS job and 1 FreeBSD job
and others need to be marked manual.

Regards,
Daniel
Philippe Mathieu-Daudé March 9, 2022, 12:50 p.m. UTC | #5
On 9/3/22 13:33, Daniel P. Berrangé wrote:
> On Wed, Mar 09, 2022 at 12:58:42PM +0100, Thomas Huth wrote:

>> Basically fine for me, but can we really run additional cirrus-ci jobs by
>> default? IIRC the parallel execution of those were quite limited for the
>> free tier, so did you look close that we don't run into additional timeouts
>> yet, due to delayed cirrus-ci jobs?
> 
> You can run 2 jobs in parallel in Cirrus. Beyond that they
> get queued/serialized
> 
> We have a 1 hour job timeout.
> 
> We have to expect jobs will sometimes run slower than normal.
> 
> IOW if we have a job on Cirrus taking 30 minutes normally, we
> expect it will sometimes take 45 minutes.
> 
> All this means that if we want Cirrus to be reliable and not
> time out, we can really only have 2 jobs by default, unless
> we can get the job execution time down to around 20 minutes
> to allow for serialization.
> 
> We used to have terrible problems with cirrus timeouts when
> we were running 4 jobs concurrently (2 on staging and 2 on
> master). We addressed that in 9968de0a4a5470bd7b98dcd2fae5d5269908f16b
> by disabling the jobs on master.
> 
> IOW, we really need to choose 1 macOS job and 1 FreeBSD job
> and others need to be marked manual.

Not sure which job to choose yet. Per the first google hits we
still want to cover Catalina first:
https://www.statista.com/statistics/944559/worldwide-macos-version-market-share/

Would it be beneficial to have a 1 per OS job during PR, and
other jobs run nightly (once a day, not per PR)?
Thomas Huth March 9, 2022, 12:52 p.m. UTC | #6
On 09/03/2022 13.50, Philippe Mathieu-Daudé wrote:

> Would it be beneficial to have a 1 per OS job during PR, and
> other jobs run nightly (once a day, not per PR)?

Is there a way to trigger nightly runs in gitlab?

  Thomas
Daniel P. Berrangé March 9, 2022, 12:55 p.m. UTC | #7
On Wed, Mar 09, 2022 at 01:52:15PM +0100, Thomas Huth wrote:
> On 09/03/2022 13.50, Philippe Mathieu-Daudé wrote:
> 
> > Would it be beneficial to have a 1 per OS job during PR, and
> > other jobs run nightly (once a day, not per PR)?
> 
> Is there a way to trigger nightly runs in gitlab?

Yes, in the repo CI settings create a scheduled job, using cron syntax.
Relies on the job "rules" in the config file not excluding scheduled
execution of course.

Regards,
Daniel
Daniel P. Berrangé March 9, 2022, 1:02 p.m. UTC | #8
On Wed, Mar 09, 2022 at 01:50:34PM +0100, Philippe Mathieu-Daudé wrote:
> On 9/3/22 13:33, Daniel P. Berrangé wrote:
> > On Wed, Mar 09, 2022 at 12:58:42PM +0100, Thomas Huth wrote:
> 
> > > Basically fine for me, but can we really run additional cirrus-ci jobs by
> > > default? IIRC the parallel execution of those were quite limited for the
> > > free tier, so did you look close that we don't run into additional timeouts
> > > yet, due to delayed cirrus-ci jobs?
> > 
> > You can run 2 jobs in parallel in Cirrus. Beyond that they
> > get queued/serialized
> > 
> > We have a 1 hour job timeout.
> > 
> > We have to expect jobs will sometimes run slower than normal.
> > 
> > IOW if we have a job on Cirrus taking 30 minutes normally, we
> > expect it will sometimes take 45 minutes.
> > 
> > All this means that if we want Cirrus to be reliable and not
> > time out, we can really only have 2 jobs by default, unless
> > we can get the job execution time down to around 20 minutes
> > to allow for serialization.
> > 
> > We used to have terrible problems with cirrus timeouts when
> > we were running 4 jobs concurrently (2 on staging and 2 on
> > master). We addressed that in 9968de0a4a5470bd7b98dcd2fae5d5269908f16b
> > by disabling the jobs on master.
> > 
> > IOW, we really need to choose 1 macOS job and 1 FreeBSD job
> > and others need to be marked manual.
> 
> Not sure which job to choose yet. Per the first google hits we
> still want to cover Catalina first:
> https://www.statista.com/statistics/944559/worldwide-macos-version-market-share/

My general gut feeling is usually to prioritize testing older versions
as they tend to be more widely used, and we want to avoid regresions
on stuff that has been around the longest. Compatibility problems with
new releases tend to get reported by users/maintainers and would not
be regressions, but rather enhancements to support the new platform.

But it is not really an easy decision, as clearly we would prefer to
test both old and new if we had the ability.

Having one as a nightly job is a reasonable compromise. 

With regards,
Daniel
Peter Maydell March 9, 2022, 1:09 p.m. UTC | #9
On Wed, 9 Mar 2022 at 13:02, Daniel P. Berrangé <berrange@redhat.com> wrote:
>
> On Wed, Mar 09, 2022 at 01:50:34PM +0100, Philippe Mathieu-Daudé wrote:
> > Not sure which job to choose yet. Per the first google hits we
> > still want to cover Catalina first:
> > https://www.statista.com/statistics/944559/worldwide-macos-version-market-share/

...that page says that the figures for Catalina are "incorrectly
high", so not a very useful set of data for these purposes I think.

> My general gut feeling is usually to prioritize testing older versions
> as they tend to be more widely used, and we want to avoid regresions
> on stuff that has been around the longest. Compatibility problems with
> new releases tend to get reported by users/maintainers and would not
> be regressions, but rather enhancements to support the new platform.

Note that technically speaking Catalina (10.15) has already fallen off
the end of our support policy (which currently requires Big Sur (11)
and Monterey (12) support, as the two most recent versions). My
personal macos x86 laptop is still running Catalina, though,
because Big Sur dropped support for that hardware :-/
That suggests that maybe Big Sur would be what we should go for
now if we have to pick just one for the CI.

thanks
-- PMM
Philippe Mathieu-Daudé March 9, 2022, 1:40 p.m. UTC | #10
On 9/3/22 14:09, Peter Maydell wrote:
> On Wed, 9 Mar 2022 at 13:02, Daniel P. Berrangé <berrange@redhat.com> wrote:
>>
>> On Wed, Mar 09, 2022 at 01:50:34PM +0100, Philippe Mathieu-Daudé wrote:
>>> Not sure which job to choose yet. Per the first google hits we
>>> still want to cover Catalina first:
>>> https://www.statista.com/statistics/944559/worldwide-macos-version-market-share/
> 
> ...that page says that the figures for Catalina are "incorrectly
> high", so not a very useful set of data for these purposes I think.
> 
>> My general gut feeling is usually to prioritize testing older versions
>> as they tend to be more widely used, and we want to avoid regresions
>> on stuff that has been around the longest. Compatibility problems with
>> new releases tend to get reported by users/maintainers and would not
>> be regressions, but rather enhancements to support the new platform.
> 
> Note that technically speaking Catalina (10.15) has already fallen off
> the end of our support policy (which currently requires Big Sur (11)
> and Monterey (12) support, as the two most recent versions). My
> personal macos x86 laptop is still running Catalina, though,
> because Big Sur dropped support for that hardware :-/
> That suggests that maybe Big Sur would be what we should go for
> now if we have to pick just one for the CI.

OK. Since I'm running Monterey (12) on my workstation, I'll make this
job manual / nightly, and keep the Big Sur (11) on CI.

I'll let Daniel / Thomas / Alex decide which FreeBSD job to demote to
manual.

Regards,

Phil.
diff mbox series

Patch

diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
index b96b22e269..be1dce5d4e 100644
--- a/.gitlab-ci.d/cirrus.yml
+++ b/.gitlab-ci.d/cirrus.yml
@@ -87,6 +87,22 @@  x64-macos-11-base-build:
     PKG_CONFIG_PATH: /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig
     TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat check-qtest-x86_64
 
+x64-macos-12-base-build:
+  extends: .cirrus_build_job
+  variables:
+    NAME: macos-12
+    CIRRUS_VM_INSTANCE_TYPE: osx_instance
+    CIRRUS_VM_IMAGE_SELECTOR: image
+    CIRRUS_VM_IMAGE_NAME: monterey-base
+    CIRRUS_VM_CPUS: 12
+    CIRRUS_VM_RAM: 24G
+    UPDATE_COMMAND: brew update
+    INSTALL_COMMAND: brew install
+    PATH_EXTRA: /usr/local/opt/ccache/libexec:/usr/local/opt/gettext/bin
+    PKG_CONFIG_PATH: /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig
+    CONFIGURE_ARGS: --enable-modules
+    TEST_TARGETS: check-unit check-block check-qapi-schema check-qtest-aarch64
+
 
 # The following jobs run VM-based tests via KVM on a Linux-based Cirrus-CI job
 .cirrus_kvm_job:
diff --git a/.gitlab-ci.d/cirrus/macos-12.vars b/.gitlab-ci.d/cirrus/macos-12.vars
new file mode 100644
index 0000000000..a793258c64
--- /dev/null
+++ b/.gitlab-ci.d/cirrus/macos-12.vars
@@ -0,0 +1,16 @@ 
+# THIS FILE WAS AUTO-GENERATED
+#
+#  $ lcitool variables macos-12 qemu
+#
+# https://gitlab.com/libvirt/libvirt-ci
+
+CCACHE='/usr/local/bin/ccache'
+CPAN_PKGS='Test::Harness'
+CROSS_PKGS=''
+MAKE='/usr/local/bin/gmake'
+NINJA='/usr/local/bin/ninja'
+PACKAGING_COMMAND='brew'
+PIP3='/usr/local/bin/pip3'
+PKGS='bash bc bzip2 capstone ccache cpanminus ctags curl dbus diffutils dtc gcovr gettext git glib gnu-sed gnutls gtk+3 jemalloc jpeg-turbo libepoxy libffi libgcrypt libiscsi libnfs libpng libslirp libssh libtasn1 libusb llvm lzo make meson ncurses nettle ninja perl pixman pkg-config python3 rpm2cpio sdl2 sdl2_image snappy sparse spice-protocol tesseract texinfo usbredir vde vte3 zlib zstd'
+PYPI_PKGS='PyYAML numpy pillow sphinx sphinx-rtd-theme virtualenv'
+PYTHON='/usr/local/bin/python3'
diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh
index 1f00281b44..e2f0bbd1b1 100755
--- a/tests/lcitool/refresh
+++ b/tests/lcitool/refresh
@@ -105,6 +105,7 @@  try:
    generate_cirrus("freebsd-12")
    generate_cirrus("freebsd-13")
    generate_cirrus("macos-11")
+   generate_cirrus("macos-12")
 
    sys.exit(0)
 except Exception as ex: