diff mbox series

[18/12] CI: Add Ubuntu 22.04 (Jammy) and 24.04 (Noble) testing

Message ID 20240712104907.3238026-1-andrew.cooper3@citrix.com (mailing list archive)
State New
Headers show
Series CI: part 3 (slimline, and PPC/RISCV fixes) | expand

Commit Message

Andrew Cooper July 12, 2024, 10:49 a.m. UTC 
The containers are exactly as per 20.04 (Focal).  However, this now brings us
to 5 releases * 4 build jobs worth of Ubuntu testing, which is overkill.

The oldest and newest toolchains are the most likely to find problems with new
code, so reduce the middle 3 releases (18/20/22) to just a single smoke test
each.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Anthony PERARD <anthony.perard@vates.tech>
CC: Juergen Gross <jgross@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Julien Grall <julien@xen.org>
CC: Oleksii Kurochko <oleksii.kurochko@gmail.com>
CC: Shawn Anastasio <sanastasio@raptorengineering.com>

Pipeline run showing all Ubuntu testing on staging:

  https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1371106799

There are 11 jobs in total.
---
 .../build/ubuntu/22.04-x86_64.dockerfile      | 72 +++++++++++++++++++
 .../build/ubuntu/24.04-x86_64.dockerfile      | 72 +++++++++++++++++++
 automation/gitlab-ci/build.yaml               | 39 +++++-----
 automation/scripts/containerize               |  2 +
 4 files changed, 163 insertions(+), 22 deletions(-)
 create mode 100644 automation/build/ubuntu/22.04-x86_64.dockerfile
 create mode 100644 automation/build/ubuntu/24.04-x86_64.dockerfile


base-commit: d46e4264c3fa68974fbb9b1bce514ea44af6d945
prerequisite-patch-id: b6875ad0bbd279067fdef31b9adf67e2a5abdf8a
prerequisite-patch-id: 5d4a01a02e7694939cdbf620a2e7509fee49dd92
prerequisite-patch-id: 65ecea93b1a320219deb8670264c3d464e00384e
prerequisite-patch-id: eb7042acce82d2bc324f8560fed3ffd57025d248
prerequisite-patch-id: 91c6663df2cfa139497e2d8b4dc17f1697f0c9f6
prerequisite-patch-id: 84c3345d580461986a3a9560735279baff21234f
prerequisite-patch-id: 5d732fc5f039fb25f7177279f63019f59acae4b1
prerequisite-patch-id: dd5006da276abc0e4da1ecb2c84d699c5bd98124
prerequisite-patch-id: 7e816655cd1066b42be5fa57a832c777c9867fc9
prerequisite-patch-id: fc25cefbee4cb69703df5c1f5b3eb317f03952af
prerequisite-patch-id: 531b3db898f54c05bff5fbcb69e0f7f62811c744
prerequisite-patch-id: ddb51b78aeaa2a6efe8035c39a128810c01aa822
prerequisite-patch-id: b4894b8120682429e1a74369c6b36bbbdc09d749
prerequisite-patch-id: c002530cb86a08b38980779a3d06cf491fdb9fae

Comments

Anthony PERARD July 12, 2024, 4:44 p.m. UTC | #1 
On Fri, Jul 12, 2024 at 11:49:07AM +0100, Andrew Cooper wrote:
> The containers are exactly as per 20.04 (Focal).  However, this now brings us
> to 5 releases * 4 build jobs worth of Ubuntu testing, which is overkill.
> 
> The oldest and newest toolchains are the most likely to find problems with new
> code, so reduce the middle 3 releases (18/20/22) to just a single smoke test
> each.

That would mean a bit less `clang` build-test, but I guess that would be
mostly covered by FreeBSD testing on GitHub.

I tried to find out which version of clang were been used with this
patch applied:
    Alpine clang version 16.0.6
    Debian clang version 14.0.6
    clang version 17.0.6
    clang version 18.1.8
    clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
    Ubuntu clang version 18.1.3 (1ubuntu1)
and the versions that won't be tested anymore:
    clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
    clang version 10.0.0-4ubuntu1
    Ubuntu clang version 14.0.0-1ubuntu1.1

> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Anyway:
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>

Thanks,
Andrew Cooper July 12, 2024, 5:01 p.m. UTC | #2 
On 12/07/2024 5:44 pm, Anthony PERARD wrote:
> On Fri, Jul 12, 2024 at 11:49:07AM +0100, Andrew Cooper wrote:
>> The containers are exactly as per 20.04 (Focal).  However, this now brings us
>> to 5 releases * 4 build jobs worth of Ubuntu testing, which is overkill.
>>
>> The oldest and newest toolchains are the most likely to find problems with new
>> code, so reduce the middle 3 releases (18/20/22) to just a single smoke test
>> each.
> That would mean a bit less `clang` build-test, but I guess that would be
> mostly covered by FreeBSD testing on GitHub.
>
> I tried to find out which version of clang were been used with this
> patch applied:
>     Alpine clang version 16.0.6
>     Debian clang version 14.0.6
>     clang version 17.0.6
>     clang version 18.1.8
>     clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
>     Ubuntu clang version 18.1.3 (1ubuntu1)
> and the versions that won't be tested anymore:
>     clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
>     clang version 10.0.0-4ubuntu1
>     Ubuntu clang version 14.0.0-1ubuntu1.1
>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Anyway:
> Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>

Thanks.

We could intermix Clang/GCC if we really want, but tbh (still!) the only
thing you want to use Clang for is stricter/better diagnostics.  GCC is
still a better optimising compiler for C, particularly for kernel
activities.

Also, those clang versions are entirely obsolete.  If we wanted to do
sensible testing, we'd need to be using the nightly builds and keeping
within ~18mo of the release which I think is their support window.

Also also, this testing is CC=Clang, LD=ld, which is subtly different to
the FreeBSD testing which is full-stack LLVM (so LD=lld).

There is definitely room for improvement in our Clang/LLVM testing.

~Andrew
diff mbox series

Patch

diff --git a/automation/build/ubuntu/22.04-x86_64.dockerfile b/automation/build/ubuntu/22.04-x86_64.dockerfile
new file mode 100644
index 000000000000..230903f624d9
--- /dev/null
+++ b/automation/build/ubuntu/22.04-x86_64.dockerfile
@@ -0,0 +1,72 @@ 
+# syntax=docker/dockerfile:1
+FROM --platform=linux/amd64 ubuntu:22.04
+LABEL maintainer.name="The Xen Project"
+LABEL maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+RUN <<EOF
+#!/bin/bash
+    set -e
+
+    useradd --create-home user
+
+    apt-get -y update
+
+    DEPS=(
+        # Xen
+        bison
+        build-essential
+        checkpolicy
+        clang
+        flex
+        python3-minimal
+
+        # Tools (general)
+        ca-certificates
+        git-core
+        gzip
+        patch
+        perl
+        pkg-config
+        wget
+        # libxenguest dombuilder
+        libbz2-dev
+        libzstd-dev
+        liblzo2-dev
+        liblzma-dev
+        zlib1g-dev
+        # libacpi
+        acpica-tools
+        # libxl
+        uuid-dev
+        libyajl-dev
+        # RomBIOS
+        bcc
+        bin86
+        # xentop
+        libncurses5-dev
+        # Python bindings
+        python3-dev
+        python3-setuptools
+        # Ocaml bindings/oxenstored
+        ocaml-nox
+        ocaml-findlib
+
+        # Stubdom download/extract
+        bzip2
+
+        # Qemu build
+        libglib2.0-dev
+        libpixman-1-dev
+        meson
+        ninja-build
+        python3-packaging
+    )
+
+    apt-get -y --no-install-recommends install "${DEPS[@]}"
+    rm -rf /var/lib/apt/lists/*
+EOF
+
+USER user
+WORKDIR /build
diff --git a/automation/build/ubuntu/24.04-x86_64.dockerfile b/automation/build/ubuntu/24.04-x86_64.dockerfile
new file mode 100644
index 000000000000..277f92facfd9
--- /dev/null
+++ b/automation/build/ubuntu/24.04-x86_64.dockerfile
@@ -0,0 +1,72 @@ 
+# syntax=docker/dockerfile:1
+FROM --platform=linux/amd64 ubuntu:24.04
+LABEL maintainer.name="The Xen Project"
+LABEL maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+RUN <<EOF
+#!/bin/bash
+    set -e
+
+    useradd --create-home user
+
+    apt-get -y update
+
+    DEPS=(
+        # Xen
+        bison
+        build-essential
+        checkpolicy
+        clang
+        flex
+        python3-minimal
+
+        # Tools (general)
+        ca-certificates
+        git-core
+        gzip
+        patch
+        perl
+        pkg-config
+        wget
+        # libxenguest dombuilder
+        libbz2-dev
+        libzstd-dev
+        liblzo2-dev
+        liblzma-dev
+        zlib1g-dev
+        # libacpi
+        acpica-tools
+        # libxl
+        uuid-dev
+        libyajl-dev
+        # RomBIOS
+        bcc
+        bin86
+        # xentop
+        libncurses5-dev
+        # Python bindings
+        python3-dev
+        python3-setuptools
+        # Ocaml bindings/oxenstored
+        ocaml-nox
+        ocaml-findlib
+
+        # Stubdom download/extract
+        bzip2
+
+        # Qemu build
+        libglib2.0-dev
+        libpixman-1-dev
+        meson
+        ninja-build
+        python3-packaging
+    )
+
+    apt-get -y --no-install-recommends install "${DEPS[@]}"
+    rm -rf /var/lib/apt/lists/*
+EOF
+
+USER user
+WORKDIR /build
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 6a2d796959a1..379cf17d65cc 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -608,45 +608,40 @@  ubuntu-16.04-x86_64-gcc-debug:
   variables:
     CONTAINER: ubuntu:16.04-x86_64
 
-ubuntu-18.04-x86_64-clang:
-  extends: .clang-x86-64-build
-  variables:
-    CONTAINER: ubuntu:18.04-x86_64
-
-ubuntu-18.04-x86_64-clang-debug:
-  extends: .clang-x86-64-build-debug
-  variables:
-    CONTAINER: ubuntu:18.04-x86_64
-
 ubuntu-18.04-x86_64-gcc:
   extends: .gcc-x86-64-build
   variables:
     CONTAINER: ubuntu:18.04-x86_64
 
-ubuntu-18.04-x86_64-gcc-debug:
-  extends: .gcc-x86-64-build-debug
-  variables:
-    CONTAINER: ubuntu:18.04-x86_64
-
 ubuntu-20.04-x86_64-gcc:
   extends: .gcc-x86-64-build
   variables:
     CONTAINER: ubuntu:20.04-x86_64
 
-ubuntu-20.04-x86_64-gcc-debug:
-  extends: .gcc-x86-64-build-debug
+ubuntu-22.04-x86_64-gcc:
+  extends: .gcc-x86-64-build
   variables:
-    CONTAINER: ubuntu:20.04-x86_64
+    CONTAINER: ubuntu:22.04-x86_64
 
-ubuntu-20.04-x86_64-clang:
+ubuntu-24.04-x86_64-clang:
   extends: .clang-x86-64-build
   variables:
-    CONTAINER: ubuntu:20.04-x86_64
+    CONTAINER: ubuntu:24.04-x86_64
 
-ubuntu-20.04-x86_64-clang-debug:
+ubuntu-24.04-x86_64-clang-debug:
   extends: .clang-x86-64-build-debug
   variables:
-    CONTAINER: ubuntu:20.04-x86_64
+    CONTAINER: ubuntu:24.04-x86_64
+
+ubuntu-24.04-x86_64-gcc:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: ubuntu:24.04-x86_64
+
+ubuntu-24.04-x86_64-gcc-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: ubuntu:24.04-x86_64
 
 opensuse-leap-15.6-clang:
   extends: .clang-x86-64-build
diff --git a/automation/scripts/containerize b/automation/scripts/containerize
index f27932dd4561..9844133aff63 100755
--- a/automation/scripts/containerize
+++ b/automation/scripts/containerize
@@ -55,6 +55,8 @@  case "_${CONTAINER}" in
     _xenial) CONTAINER="${BASE}/ubuntu:16.04-x86_64" ;;
     _bionic) CONTAINER="${BASE}/ubuntu:18.04-x86_64" ;;
     _focal)  CONTAINER="${BASE}/ubuntu:20.04-x86_64" ;;
+    _jammy)  CONTAINER="${BASE}/ubuntu:22.04-x86_64" ;;
+    _noble)  CONTAINER="${BASE}/ubuntu:24.04-x86_64" ;;
 
     *) guess_container ;;
 esac