| Message ID | 20240711111517.3064810-3-andrew.cooper3@citrix.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | CI: part 3 (slimline, and PPC/RISCV fixes) | expand |
On Thu, Jul 11, 2024 at 12:15:07PM +0100, Andrew Cooper wrote: > Randconfig builds pick CONFIG_DEBUG with 50% probability. Therefore > $foo{,-debug}-randconfig are two identical jobs with misleading names. > > Furthermore, arm64 has a randconfig jobs for both the alpine and bookworm > build environments. Both use GCC 12.2, so we have 4 identical jobs. Delete 3 > of them. > > This leaves us with 4 primary randconfig jobs: > > gitlab-ci/build.yaml:381:debian-bookworm-gcc-arm32-randconfig: > gitlab-ci/build.yaml:429:alpine-3.18-gcc-arm64-randconfig: > gitlab-ci/build.yaml:495:archlinux-current-gcc-riscv64-randconfig: > gitlab-ci/build.yaml:640:debian-bookworm-gcc-randconfig: > > as PPC64 doesn't want randconfig right now, and buster-gcc-ibt is a special > job with a custom compiler. > > Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Reviewed-by: Anthony PERARD <anthony.perard@vates.tech> Thanks, > If we want to run multiple identical randconfig jobs, that's spelt > > parallel: 5 > > in the configuration, and here is an example of what such a run looks like: > https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1368824041 > > Notice the randconfig jobs have a 5 in place of a retry button, and show a > submenu when clicked on. We already have "parallel: 2" on the whole pipeline, give or take, but those are quick jobs so it might be worth it to run them a few more time on a pipeline. Cheers,
On 11/07/2024 2:09 pm, Anthony PERARD wrote: > On Thu, Jul 11, 2024 at 12:15:07PM +0100, Andrew Cooper wrote: >> Randconfig builds pick CONFIG_DEBUG with 50% probability. Therefore >> $foo{,-debug}-randconfig are two identical jobs with misleading names. >> >> Furthermore, arm64 has a randconfig jobs for both the alpine and bookworm >> build environments. Both use GCC 12.2, so we have 4 identical jobs. Delete 3 >> of them. >> >> This leaves us with 4 primary randconfig jobs: >> >> gitlab-ci/build.yaml:381:debian-bookworm-gcc-arm32-randconfig: >> gitlab-ci/build.yaml:429:alpine-3.18-gcc-arm64-randconfig: >> gitlab-ci/build.yaml:495:archlinux-current-gcc-riscv64-randconfig: >> gitlab-ci/build.yaml:640:debian-bookworm-gcc-randconfig: >> >> as PPC64 doesn't want randconfig right now, and buster-gcc-ibt is a special >> job with a custom compiler. >> >> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> > Reviewed-by: Anthony PERARD <anthony.perard@vates.tech> Thanks. > > Thanks, > >> If we want to run multiple identical randconfig jobs, that's spelt >> >> parallel: 5 >> >> in the configuration, and here is an example of what such a run looks like: >> https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1368824041 >> >> Notice the randconfig jobs have a 5 in place of a retry button, and show a >> submenu when clicked on. > We already have "parallel: 2" on the whole pipeline, give or take, but > those are quick jobs so it might be worth it to run them a few more time > on a pipeline. I'm afraid I don't follow. The runners themselves are set up to run concurrent jobs. I could well believe that's spelt parallel: in the runner config files. Here, its really jobs:parallel which is the way of saying "I want 5 copies of this job to run in the pipeline". Obviously, 5 is arbitrary. I was thinking of having a scheduled (once per day, at 3AM or whenever) running randconfig only with a parallel:20 or so, so we can make good use of otherwise-idle time. ~Andrew
On Thu, Jul 11, 2024 at 04:16:29PM +0100, Andrew Cooper wrote: > On 11/07/2024 2:09 pm, Anthony PERARD wrote: > > We already have "parallel: 2" on the whole pipeline, give or take, but > > those are quick jobs so it might be worth it to run them a few more time > > on a pipeline. > > I'm afraid I don't follow. Sorry, I just meant that the whole test suite is run twice, once in https://gitlab.com/xen-project/hardware/xen/-/pipelines and once in https://gitlab.com/xen-project/xen/-/pipelines (with just a few tests that are differents) > Obviously, 5 is arbitrary. I was thinking of having a scheduled (once > per day, at 3AM or whenever) running randconfig only with a parallel:20 > or so, so we can make good use of otherwise-idle time. That sounds like a good idea. Cheers,
On 11/07/2024 4:50 pm, Anthony PERARD wrote: > On Thu, Jul 11, 2024 at 04:16:29PM +0100, Andrew Cooper wrote: >> On 11/07/2024 2:09 pm, Anthony PERARD wrote: >>> We already have "parallel: 2" on the whole pipeline, give or take, but >>> those are quick jobs so it might be worth it to run them a few more time >>> on a pipeline. >> I'm afraid I don't follow. > Sorry, I just meant that the whole test suite is run twice, once in > https://gitlab.com/xen-project/hardware/xen/-/pipelines > and once in > https://gitlab.com/xen-project/xen/-/pipelines > (with just a few tests that are differents) And the rest. Every time I re-sync my tree, and probably the same for the other maintainers. However, these have nothing on the overhead from patchew, which would be far more palatable if we didn't essentially throw away the result by not sending it anywhere useful. ~Andrew
diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml index ff5c9055d1f0..da7b8885aa30 100644 --- a/automation/gitlab-ci/build.yaml +++ b/automation/gitlab-ci/build.yaml @@ -385,13 +385,6 @@ debian-bookworm-gcc-arm32-randconfig: HYPERVISOR_ONLY: y RANDCONFIG: y -debian-bookworm-gcc-arm32-debug-randconfig: - extends: .gcc-arm32-cross-build-debug - variables: - CONTAINER: debian:bookworm-arm64v8-arm32-gcc - HYPERVISOR_ONLY: y - RANDCONFIG: y - debian-bookworm-gcc-arm32-debug-staticmem: extends: .gcc-arm32-cross-build-debug variables: @@ -423,18 +416,6 @@ debian-bookworm-gcc-debug-arm64: variables: CONTAINER: debian:bookworm-arm64v8 -debian-bookworm-gcc-arm64-randconfig: - extends: .gcc-arm64-build - variables: - CONTAINER: debian:bookworm-arm64v8 - RANDCONFIG: y - -debian-bookworm-gcc-debug-arm64-randconfig: - extends: .gcc-arm64-build-debug - variables: - CONTAINER: debian:bookworm-arm64v8 - RANDCONFIG: y - alpine-3.18-gcc-arm64: extends: .gcc-arm64-build variables: @@ -451,12 +432,6 @@ alpine-3.18-gcc-arm64-randconfig: CONTAINER: alpine:3.18-arm64v8 RANDCONFIG: y -alpine-3.18-gcc-debug-arm64-randconfig: - extends: .gcc-arm64-build-debug - variables: - CONTAINER: alpine:3.18-arm64v8 - RANDCONFIG: y - alpine-3.18-gcc-debug-arm64-staticmem: extends: .gcc-arm64-build-debug variables: @@ -525,14 +500,6 @@ archlinux-current-gcc-riscv64-randconfig: RANDCONFIG: y <<: *riscv-fixed-randconfig -archlinux-current-gcc-riscv64-debug-randconfig: - extends: .gcc-riscv64-cross-build-debug - variables: - CONTAINER: archlinux:current-riscv64 - KBUILD_DEFCONFIG: tiny64_defconfig - RANDCONFIG: y - <<: *riscv-fixed-randconfig - # Power cross-build debian-bullseye-gcc-ppc64le: extends: .gcc-ppc64le-cross-build @@ -676,12 +643,6 @@ debian-bookworm-gcc-randconfig: CONTAINER: debian:bookworm RANDCONFIG: y -debian-bookworm-gcc-debug-randconfig: - extends: .gcc-x86-64-build-debug - variables: - CONTAINER: debian:bookworm - RANDCONFIG: y - debian-bookworm-32-clang-debug: extends: .clang-x86-32-build-debug variables:
Randconfig builds pick CONFIG_DEBUG with 50% probability. Therefore $foo{,-debug}-randconfig are two identical jobs with misleading names. Furthermore, arm64 has a randconfig jobs for both the alpine and bookworm build environments. Both use GCC 12.2, so we have 4 identical jobs. Delete 3 of them. This leaves us with 4 primary randconfig jobs: gitlab-ci/build.yaml:381:debian-bookworm-gcc-arm32-randconfig: gitlab-ci/build.yaml:429:alpine-3.18-gcc-arm64-randconfig: gitlab-ci/build.yaml:495:archlinux-current-gcc-riscv64-randconfig: gitlab-ci/build.yaml:640:debian-bookworm-gcc-randconfig: as PPC64 doesn't want randconfig right now, and buster-gcc-ibt is a special job with a custom compiler. Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> --- CC: Anthony PERARD <anthony.perard@vates.tech> CC: Juergen Gross <jgross@suse.com> CC: Roger Pau Monné <roger.pau@citrix.com> CC: Jan Beulich <JBeulich@suse.com> CC: Stefano Stabellini <sstabellini@kernel.org> CC: Julien Grall <julien@xen.org> CC: Oleksii Kurochko <oleksii.kurochko@gmail.com> CC: Shawn Anastasio <sanastasio@raptorengineering.com> Here is an example debug-randconfig with a release builds: https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/7312848876 If we want to run multiple identical randconfig jobs, that's spelt parallel: 5 in the configuration, and here is an example of what such a run looks like: https://gitlab.com/xen-project/people/andyhhp/xen/-/pipelines/1368824041 Notice the randconfig jobs have a 5 in place of a retry button, and show a submenu when clicked on. --- automation/gitlab-ci/build.yaml | 39 --------------------------------- 1 file changed, 39 deletions(-)