| Message ID | 95088c06171ce140caf48029118dcb6fd2ac8d99.1739933790.git-series.marmarek@invisiblethingslab.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | Few CI improvements | expand |
On Wed, 19 Feb 2025, Marek Marczykowski-Górecki wrote: > Debugging sometimes involves running specific jobs on different > versions. It's useful to easily avoid running all of the not interesting > ones (for given case) to save both time and CI resources. Doing so used > to require changing the yaml files, usually in several places. > Ease this step by adding SELECTED_JOBS_ONLY variable that takes a regex. > Note that one needs to satisfy job dependencies on their own (for > example if a test job needs a build job, that specific build job > needs to be included too). > > The variable can be specified via Gitlab web UI when scheduling a > pipeline, but it can be also set when doing git push directly: > > git push -o ci.variable=SELECTED_JOBS_ONLY="/job1|job2/" > > More details at https://docs.gitlab.co.jp/ee/user/project/push_options.html > > The variable needs to include regex for selecting jobs, including > enclosing slashes. > A coma/space separated list of jobs to select would be friendlier UX, > but unfortunately that is not supported: > https://gitlab.com/gitlab-org/gitlab/-/issues/209904 (note the proposed > workaround doesn't work for job-level CI_JOB_NAME). > On the other hand, the regex is more flexible (one can select for > example all arm32 jobs). > > Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> While I am not super happy about this (it is not your fault, it is due to the limitations of the Gitlab YAML languange) and I don't think I would use SELECTED_JOBS_ONLY probably, if you find it useful maybe others will too. It is not a big maintenance burden. So: Acked-by: Stefano Stabellini <sstabellini@kernel.org> > --- > Changes in v3: > - include variable in Web UI for starting pipeline > --- > .gitlab-ci.yml | 2 ++ > automation/gitlab-ci/build.yaml | 6 ++++++ > automation/gitlab-ci/test.yaml | 14 ++++++++++++++ > 3 files changed, 22 insertions(+) > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index 5a9b8b722838..b3beb2ff9ddf 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -1,5 +1,7 @@ > variables: > XEN_REGISTRY: registry.gitlab.com/xen-project/xen > + SELECTED_JOBS_ONLY: > + description: "Regex to select only some jobs, must be enclosed with /. For example /job1|job2/" > > workflow: > rules: > diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml > index 35e224366f62..f12de00a164a 100644 > --- a/automation/gitlab-ci/build.yaml > +++ b/automation/gitlab-ci/build.yaml > @@ -12,6 +12,12 @@ > - '*/*.log' > when: always > needs: [] > + rules: > + - if: $SELECTED_JOBS_ONLY && $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY > + when: always > + - if: $SELECTED_JOBS_ONLY > + when: never > + - when: on_success > > .gcc-tmpl: > variables: &gcc > diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml > index c21a37933881..93632f1f9204 100644 > --- a/automation/gitlab-ci/test.yaml > +++ b/automation/gitlab-ci/test.yaml > @@ -1,6 +1,11 @@ > .test-jobs-common: > stage: test > image: ${XEN_REGISTRY}/${CONTAINER} > + rules: > + - if: $SELECTED_JOBS_ONLY && $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY > + - if: $SELECTED_JOBS_ONLY > + when: never > + - when: on_success > > .arm64-test-needs: &arm64-test-needs > - alpine-3.18-arm64-rootfs-export > @@ -99,6 +104,9 @@ > - '*.dtb' > when: always > rules: > + - if: $SELECTED_JOBS_ONLY && $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY > + - if: $SELECTED_JOBS_ONLY > + when: never > - if: $XILINX_JOBS == "true" && $CI_COMMIT_REF_PROTECTED == "true" > tags: > - xilinx > @@ -117,6 +125,9 @@ > - '*.log' > when: always > rules: > + - if: $SELECTED_JOBS_ONLY && $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY > + - if: $SELECTED_JOBS_ONLY > + when: never > - if: $XILINX_JOBS == "true" && $CI_COMMIT_REF_PROTECTED == "true" > tags: > - xilinx > @@ -137,6 +148,9 @@ > - '*.log' > when: always > rules: > + - if: $SELECTED_JOBS_ONLY && $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY > + - if: $SELECTED_JOBS_ONLY > + when: never > - if: $QUBES_JOBS == "true" && $CI_COMMIT_REF_PROTECTED == "true" > tags: > - qubes-hw2 > -- > git-series 0.9.1 >
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 5a9b8b722838..b3beb2ff9ddf 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,5 +1,7 @@ variables: XEN_REGISTRY: registry.gitlab.com/xen-project/xen + SELECTED_JOBS_ONLY: + description: "Regex to select only some jobs, must be enclosed with /. For example /job1|job2/" workflow: rules: diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml index 35e224366f62..f12de00a164a 100644 --- a/automation/gitlab-ci/build.yaml +++ b/automation/gitlab-ci/build.yaml @@ -12,6 +12,12 @@ - '*/*.log' when: always needs: [] + rules: + - if: $SELECTED_JOBS_ONLY && $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY + when: always + - if: $SELECTED_JOBS_ONLY + when: never + - when: on_success .gcc-tmpl: variables: &gcc diff --git a/automation/gitlab-ci/test.yaml b/automation/gitlab-ci/test.yaml index c21a37933881..93632f1f9204 100644 --- a/automation/gitlab-ci/test.yaml +++ b/automation/gitlab-ci/test.yaml @@ -1,6 +1,11 @@ .test-jobs-common: stage: test image: ${XEN_REGISTRY}/${CONTAINER} + rules: + - if: $SELECTED_JOBS_ONLY && $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY + - if: $SELECTED_JOBS_ONLY + when: never + - when: on_success .arm64-test-needs: &arm64-test-needs - alpine-3.18-arm64-rootfs-export @@ -99,6 +104,9 @@ - '*.dtb' when: always rules: + - if: $SELECTED_JOBS_ONLY && $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY + - if: $SELECTED_JOBS_ONLY + when: never - if: $XILINX_JOBS == "true" && $CI_COMMIT_REF_PROTECTED == "true" tags: - xilinx @@ -117,6 +125,9 @@ - '*.log' when: always rules: + - if: $SELECTED_JOBS_ONLY && $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY + - if: $SELECTED_JOBS_ONLY + when: never - if: $XILINX_JOBS == "true" && $CI_COMMIT_REF_PROTECTED == "true" tags: - xilinx @@ -137,6 +148,9 @@ - '*.log' when: always rules: + - if: $SELECTED_JOBS_ONLY && $CI_JOB_NAME =~ $SELECTED_JOBS_ONLY + - if: $SELECTED_JOBS_ONLY + when: never - if: $QUBES_JOBS == "true" && $CI_COMMIT_REF_PROTECTED == "true" tags: - qubes-hw2
Debugging sometimes involves running specific jobs on different versions. It's useful to easily avoid running all of the not interesting ones (for given case) to save both time and CI resources. Doing so used to require changing the yaml files, usually in several places. Ease this step by adding SELECTED_JOBS_ONLY variable that takes a regex. Note that one needs to satisfy job dependencies on their own (for example if a test job needs a build job, that specific build job needs to be included too). The variable can be specified via Gitlab web UI when scheduling a pipeline, but it can be also set when doing git push directly: git push -o ci.variable=SELECTED_JOBS_ONLY="/job1|job2/" More details at https://docs.gitlab.co.jp/ee/user/project/push_options.html The variable needs to include regex for selecting jobs, including enclosing slashes. A coma/space separated list of jobs to select would be friendlier UX, but unfortunately that is not supported: https://gitlab.com/gitlab-org/gitlab/-/issues/209904 (note the proposed workaround doesn't work for job-level CI_JOB_NAME). On the other hand, the regex is more flexible (one can select for example all arm32 jobs). Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> --- Changes in v3: - include variable in Web UI for starting pipeline --- .gitlab-ci.yml | 2 ++ automation/gitlab-ci/build.yaml | 6 ++++++ automation/gitlab-ci/test.yaml | 14 ++++++++++++++ 3 files changed, 22 insertions(+)