| Message ID | eb5fa852f306790160b967787592553d2fdf3de6.1689887842.git.simone.ballarin@bugseng.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [XEN] automation: add ECLAIR pipeline | expand |
On Thu, 20 Jul 2023, Simone Ballarin wrote: > Add two pipelines that analyze an ARM64 and a X86_64 build with the > ECLAIR static analyzer on the guidelines contained in Set1. > > The tool configuration is kept external to the xen repository for > practical reasons, it will be included in a subsequent phase. > > All commits on the xen-project/xen:staging branch will be analyzed > and their artifacts will be stored indefinitely; the integration will > report differential information with respect to the previous analysis. > > All commits on other branches or repositories will be analyzed and > only the last ten artifacts will be kept; the integration will report > differential information with respect to the analysis done on the common > ancestor with xen-project/xen:staging (if available). > > Currently the pipeline variable ENABLE_ECLAIR_BOT is set to "n". > Doing so disables the generation of comments with the analysis summary > on the commit threads. The variable can be set to "y" if the a masked > variable named ECLAIRIT_TOKEN is set with the impersonation token of > an account with enough privileges to write on all repositories. > > Additionaly any repository should be able to read a masked variable > named WTOKEN with the token provided by BUGSENG. > > Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com> Thanks for the patch! Patchew automatically picked it up from xen-devel and started a pipeline here: https://gitlab.com/xen-project/patchew/xen/-/pipelines/939440592 However the eclair-x86_64 job failed with: ERROR: Uploading artifacts as "archive" to coordinator... 413 Payload Too Large Also the eclair-ARM64 job failed but it is not clear to me why. I think at least initially we should mark the two Eclair jobs with: allow_failure: true until we are sure they work reliably all the time. Otherwise we end up blocking the whole Xen staging pipeline if we make any mistakes here. We can remove "allow_failure: true" once we are sure it works well all the time. The second thing I noticed is that the build phase didn't start until the analyze phase was concluded. This is not good because it would increase the overall time significantly. We need the build/test phases to start in parallel. To do that you need to add the following change to this patch: diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml index c401f62d61..f01e2c32bb 100644 --- a/automation/gitlab-ci/build.yaml +++ b/automation/gitlab-ci/build.yaml @@ -11,6 +11,7 @@ - '*.log' - '*/*.log' when: always + needs: [] except: - master - smoke > --- > .gitlab-ci.yml | 2 ++ > automation/gitlab-ci/analyze.yaml | 38 +++++++++++++++++++++++++++++++ > automation/scripts/eclair | 26 +++++++++++++++++++++ > 3 files changed, 66 insertions(+) > create mode 100644 automation/gitlab-ci/analyze.yaml > create mode 100755 automation/scripts/eclair > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index c8bd7519d5..ee5430b8b7 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -1,7 +1,9 @@ > stages: > + - analyze > - build > - test > > include: > + - 'automation/gitlab-ci/analyze.yaml' > - 'automation/gitlab-ci/build.yaml' > - 'automation/gitlab-ci/test.yaml' > diff --git a/automation/gitlab-ci/analyze.yaml b/automation/gitlab-ci/analyze.yaml > new file mode 100644 > index 0000000000..be96d96e71 > --- /dev/null > +++ b/automation/gitlab-ci/analyze.yaml > @@ -0,0 +1,38 @@ > +.eclair-analysis: > + stage: analyze > + tags: > + - eclair-analysis > + - eclair > + - misrac I would only use 1 tag, eclair-analysis or eclair, up to you > + variables: > + ECLAIR_OUTPUT_DIR: "ECLAIR_out" > + ANALYSIS_KIND: "normal" > + ECLAIR_REPORT_URL: "saas.eclairit.com" > + ENABLE_ECLAIR_BOT: "n" > + AUTOPRBRANCH: "staging" > + AUTOPRREPOSITORY: "xen-project/xen" > + artifacts: > + when: always > + paths: > + - "${ECLAIR_OUTPUT_DIR}" > + - '*.log' > + reports: > + codequality: gl-code-quality-report.json > + > +eclair-x86_64: > + extends: .eclair-analysis > + variables: > + LOGFILE: "eclair-x86_64.log" > + VARIANT: "X86_64" > + RULESET: "Set1" > + script: > + - ./automation/scripts/eclair 2>&1 | tee "${LOGFILE}" allow_failure: true > +eclair-ARM64: > + extends: .eclair-analysis > + variables: > + LOGFILE: "eclair-ARM64.log" > + VARIANT: "ARM64" > + RULESET: "Set1" > + script: > + - ./automation/scripts/eclair 2>&1 | tee "${LOGFILE}" allow_failure: true > diff --git a/automation/scripts/eclair b/automation/scripts/eclair > new file mode 100755 > index 0000000000..d7f0845aec > --- /dev/null > +++ b/automation/scripts/eclair > @@ -0,0 +1,26 @@ > +#!/bin/bash -eu > + > +# ECLAIR configuration files are maintened by BUGSENG > +export GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no" > +[ -d ECLAIR_scripts ] || git clone ssh://git@git.bugseng.com/eclair/scripts/XEN ECLAIR_scripts > +(cd ECLAIR_scripts; git pull --rebase) > + > +ECLAIR_DIR=ECLAIR_scripts/ECLAIR > +ECLAIR_OUTPUT_DIR=$(realpath "${ECLAIR_OUTPUT_DIR}") > + > +ECLAIR_scripts/prepare.sh "${VARIANT}" > + > +ex=0 > +"${ECLAIR_DIR}/analyze.sh" "${VARIANT}" "${RULESET}" || ex=$? > +"${ECLAIR_DIR}/action_log.sh" ANALYSIS_LOG \ > + "ECLAIR analysis log" \ > + "${ECLAIR_OUTPUT_DIR}/ANALYSIS.log" \ > + "${ex}" > +"${ECLAIR_DIR}/action_log.sh" REPORT_LOG \ > + "ECLAIR report log" \ > + "${ECLAIR_OUTPUT_DIR}/REPORT.log" \ > + "${ex}" > +[ "${ex}" = 0 ] || exit "${ex}" > +"${ECLAIR_DIR}/action_push.sh" "${WTOKEN}" "${ECLAIR_OUTPUT_DIR}" > + > +rm -rf "${ECLAIR_OUTPUT_DIR}/.data" > -- > 2.34.1 >
On Thu, Jul 20, 2023 at 11:20:29PM +0200, Simone Ballarin wrote: > +# ECLAIR configuration files are maintened by BUGSENG > +export GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no" > +[ -d ECLAIR_scripts ] || git clone ssh://git@git.bugseng.com/eclair/scripts/XEN ECLAIR_scripts > +(cd ECLAIR_scripts; git pull --rebase) I'd suggest to print here commit id of the scripts repo, so the logs will keep that information.
Il giorno ven 21 lug 2023 alle ore 00:29 Stefano Stabellini < sstabellini@kernel.org> ha scritto: > > Patchew automatically picked it up from xen-devel and started a pipeline > here: > > https://gitlab.com/xen-project/patchew/xen/-/pipelines/939440592 > > However the eclair-x86_64 job failed with: > > ERROR: Uploading artifacts as "archive" to coordinator... 413 Payload > Too Large > In the next patch, I will avoid uploading the ECLAIR database in GitLab artifacts. Of course, the databases can still be accessed via the links in the pipeline logs. > > Also the eclair-ARM64 job failed but it is not clear to me why. > The reason is that the WTOKEN variable is missing. This masked variable should be set and visible to *all* repositories (xen-project/xen, xen-project/people/.*). > I think at least initially we should mark the two Eclair jobs with: > > allow_failure: true > > until we are sure they work reliably all the time. Otherwise we end up > blocking the whole Xen staging pipeline if we make any mistakes here. We > can remove "allow_failure: true" once we are sure it works well all the > time. > Ok. > > The second thing I noticed is that the build phase didn't start until > the analyze phase was concluded. This is not good because it would > increase the overall time significantly. We need the build/test phases > to start in parallel. To do that you need to add the following change to > this patch: > > > diff --git a/automation/gitlab-ci/build.yaml > b/automation/gitlab-ci/build.yaml > index c401f62d61..f01e2c32bb 100644 > --- a/automation/gitlab-ci/build.yaml > +++ b/automation/gitlab-ci/build.yaml > @@ -11,6 +11,7 @@ > - '*.log' > - '*/*.log' > when: always > + needs: [] > except: > - master > - smoke > > Thanks. > > > --- > > .gitlab-ci.yml | 2 ++ > > automation/gitlab-ci/analyze.yaml | 38 +++++++++++++++++++++++++++++++ > > automation/scripts/eclair | 26 +++++++++++++++++++++ > > 3 files changed, 66 insertions(+) > > create mode 100644 automation/gitlab-ci/analyze.yaml > > create mode 100755 automation/scripts/eclair > > > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > > index c8bd7519d5..ee5430b8b7 100644 > > --- a/.gitlab-ci.yml > > +++ b/.gitlab-ci.yml > > @@ -1,7 +1,9 @@ > > stages: > > + - analyze > > - build > > - test > > > > include: > > + - 'automation/gitlab-ci/analyze.yaml' > > - 'automation/gitlab-ci/build.yaml' > > - 'automation/gitlab-ci/test.yaml' > > diff --git a/automation/gitlab-ci/analyze.yaml > b/automation/gitlab-ci/analyze.yaml > > new file mode 100644 > > index 0000000000..be96d96e71 > > --- /dev/null > > +++ b/automation/gitlab-ci/analyze.yaml > > @@ -0,0 +1,38 @@ > > +.eclair-analysis: > > + stage: analyze > > + tags: > > + - eclair-analysis > > + - eclair > > + - misrac > > I would only use 1 tag, eclair-analysis or eclair, up to you > > Ok. > > + variables: > > + ECLAIR_OUTPUT_DIR: "ECLAIR_out" > > + ANALYSIS_KIND: "normal" > > + ECLAIR_REPORT_URL: "saas.eclairit.com" > > + ENABLE_ECLAIR_BOT: "n" > > + AUTOPRBRANCH: "staging" > > + AUTOPRREPOSITORY: "xen-project/xen" > > + artifacts: > > + when: always > > + paths: > > + - "${ECLAIR_OUTPUT_DIR}" > > + - '*.log' > > + reports: > > + codequality: gl-code-quality-report.json > > + > > +eclair-x86_64: > > + extends: .eclair-analysis > > + variables: > > + LOGFILE: "eclair-x86_64.log" > > + VARIANT: "X86_64" > > + RULESET: "Set1" > > + script: > > + - ./automation/scripts/eclair 2>&1 | tee "${LOGFILE}" > > allow_failure: true > > > > +eclair-ARM64: > > + extends: .eclair-analysis > > + variables: > > + LOGFILE: "eclair-ARM64.log" > > + VARIANT: "ARM64" > > + RULESET: "Set1" > > + script: > > + - ./automation/scripts/eclair 2>&1 | tee "${LOGFILE}" > > allow_failure: true > > > > diff --git a/automation/scripts/eclair b/automation/scripts/eclair > > new file mode 100755 > > index 0000000000..d7f0845aec > > --- /dev/null > > +++ b/automation/scripts/eclair > > @@ -0,0 +1,26 @@ > > +#!/bin/bash -eu > > + > > +# ECLAIR configuration files are maintened by BUGSENG > > +export GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no" > > +[ -d ECLAIR_scripts ] || git clone ssh:// > git@git.bugseng.com/eclair/scripts/XEN ECLAIR_scripts > > +(cd ECLAIR_scripts; git pull --rebase) > > + > > +ECLAIR_DIR=ECLAIR_scripts/ECLAIR > > +ECLAIR_OUTPUT_DIR=$(realpath "${ECLAIR_OUTPUT_DIR}") > > + > > +ECLAIR_scripts/prepare.sh "${VARIANT}" > > + > > +ex=0 > > +"${ECLAIR_DIR}/analyze.sh" "${VARIANT}" "${RULESET}" || ex=$? > > +"${ECLAIR_DIR}/action_log.sh" ANALYSIS_LOG \ > > + "ECLAIR analysis log" \ > > + "${ECLAIR_OUTPUT_DIR}/ANALYSIS.log" \ > > + "${ex}" > > +"${ECLAIR_DIR}/action_log.sh" REPORT_LOG \ > > + "ECLAIR report log" \ > > + "${ECLAIR_OUTPUT_DIR}/REPORT.log" \ > > + "${ex}" > > +[ "${ex}" = 0 ] || exit "${ex}" > > +"${ECLAIR_DIR}/action_push.sh" "${WTOKEN}" "${ECLAIR_OUTPUT_DIR}" > > + > > +rm -rf "${ECLAIR_OUTPUT_DIR}/.data" > > -- > > 2.34.1 > > >
On Fri, 21 Jul 2023, Simone Ballarin wrote: > Il giorno ven 21 lug 2023 alle ore 00:29 Stefano Stabellini <sstabellini@kernel.org> ha scritto: > > Patchew automatically picked it up from xen-devel and started a pipeline > here: > > https://gitlab.com/xen-project/patchew/xen/-/pipelines/939440592 > > However the eclair-x86_64 job failed with: > > ERROR: Uploading artifacts as "archive" to coordinator... 413 Payload > Too Large > > > In the next patch, I will avoid uploading the ECLAIR database in GitLab artifacts. > Of course, the databases can still be accessed via the links in the pipeline logs. > > > Also the eclair-ARM64 job failed but it is not clear to me why. > > > The reason is that the WTOKEN variable is missing. This masked variable should be set > and visible to *all* repositories (xen-project/xen, xen-project/people/.*). OK this is interesting. I added WTOKEN to https://gitlab.com/xen-project so I assumed it would propagate to all the subprojects like https://gitlab.com/xen-project/xen, https://gitlab.com/xen-project/people/sstabellini, etc. but it looks like it might have to be added manually at every instance.
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index c8bd7519d5..ee5430b8b7 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,7 +1,9 @@ stages: + - analyze - build - test include: + - 'automation/gitlab-ci/analyze.yaml' - 'automation/gitlab-ci/build.yaml' - 'automation/gitlab-ci/test.yaml' diff --git a/automation/gitlab-ci/analyze.yaml b/automation/gitlab-ci/analyze.yaml new file mode 100644 index 0000000000..be96d96e71 --- /dev/null +++ b/automation/gitlab-ci/analyze.yaml @@ -0,0 +1,38 @@ +.eclair-analysis: + stage: analyze + tags: + - eclair-analysis + - eclair + - misrac + variables: + ECLAIR_OUTPUT_DIR: "ECLAIR_out" + ANALYSIS_KIND: "normal" + ECLAIR_REPORT_URL: "saas.eclairit.com" + ENABLE_ECLAIR_BOT: "n" + AUTOPRBRANCH: "staging" + AUTOPRREPOSITORY: "xen-project/xen" + artifacts: + when: always + paths: + - "${ECLAIR_OUTPUT_DIR}" + - '*.log' + reports: + codequality: gl-code-quality-report.json + +eclair-x86_64: + extends: .eclair-analysis + variables: + LOGFILE: "eclair-x86_64.log" + VARIANT: "X86_64" + RULESET: "Set1" + script: + - ./automation/scripts/eclair 2>&1 | tee "${LOGFILE}" + +eclair-ARM64: + extends: .eclair-analysis + variables: + LOGFILE: "eclair-ARM64.log" + VARIANT: "ARM64" + RULESET: "Set1" + script: + - ./automation/scripts/eclair 2>&1 | tee "${LOGFILE}" diff --git a/automation/scripts/eclair b/automation/scripts/eclair new file mode 100755 index 0000000000..d7f0845aec --- /dev/null +++ b/automation/scripts/eclair @@ -0,0 +1,26 @@ +#!/bin/bash -eu + +# ECLAIR configuration files are maintened by BUGSENG +export GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no" +[ -d ECLAIR_scripts ] || git clone ssh://git@git.bugseng.com/eclair/scripts/XEN ECLAIR_scripts +(cd ECLAIR_scripts; git pull --rebase) + +ECLAIR_DIR=ECLAIR_scripts/ECLAIR +ECLAIR_OUTPUT_DIR=$(realpath "${ECLAIR_OUTPUT_DIR}") + +ECLAIR_scripts/prepare.sh "${VARIANT}" + +ex=0 +"${ECLAIR_DIR}/analyze.sh" "${VARIANT}" "${RULESET}" || ex=$? +"${ECLAIR_DIR}/action_log.sh" ANALYSIS_LOG \ + "ECLAIR analysis log" \ + "${ECLAIR_OUTPUT_DIR}/ANALYSIS.log" \ + "${ex}" +"${ECLAIR_DIR}/action_log.sh" REPORT_LOG \ + "ECLAIR report log" \ + "${ECLAIR_OUTPUT_DIR}/REPORT.log" \ + "${ex}" +[ "${ex}" = 0 ] || exit "${ex}" +"${ECLAIR_DIR}/action_push.sh" "${WTOKEN}" "${ECLAIR_OUTPUT_DIR}" + +rm -rf "${ECLAIR_OUTPUT_DIR}/.data"
Add two pipelines that analyze an ARM64 and a X86_64 build with the ECLAIR static analyzer on the guidelines contained in Set1. The tool configuration is kept external to the xen repository for practical reasons, it will be included in a subsequent phase. All commits on the xen-project/xen:staging branch will be analyzed and their artifacts will be stored indefinitely; the integration will report differential information with respect to the previous analysis. All commits on other branches or repositories will be analyzed and only the last ten artifacts will be kept; the integration will report differential information with respect to the analysis done on the common ancestor with xen-project/xen:staging (if available). Currently the pipeline variable ENABLE_ECLAIR_BOT is set to "n". Doing so disables the generation of comments with the analysis summary on the commit threads. The variable can be set to "y" if the a masked variable named ECLAIRIT_TOKEN is set with the impersonation token of an account with enough privileges to write on all repositories. Additionaly any repository should be able to read a masked variable named WTOKEN with the token provided by BUGSENG. Signed-off-by: Simone Ballarin <simone.ballarin@bugseng.com> --- .gitlab-ci.yml | 2 ++ automation/gitlab-ci/analyze.yaml | 38 +++++++++++++++++++++++++++++++ automation/scripts/eclair | 26 +++++++++++++++++++++ 3 files changed, 66 insertions(+) create mode 100644 automation/gitlab-ci/analyze.yaml create mode 100755 automation/scripts/eclair