diff mbox series

[isar-cip-core,v3,1/2] .gitlab-ci.yml: Set timeout value to run IEC layer tests for different targets.

Message ID 20250416145141.2678093-1-Sai.Sathujoda@toshiba-tsip.com (mailing list archive)
State New
Headers show
Series [isar-cip-core,v3,1/2] .gitlab-ci.yml: Set timeout value to run IEC layer tests for different targets. | expand

Commit Message

Sai.Sathujoda@toshiba-tsip.com April 16, 2025, 2:51 p.m. UTC 
From: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>

Signed-off-by: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>
---
 .gitlab-ci.yml                   | 6 +++++-
 scripts/submit_lava.sh           | 5 +++++
 tests/templates/IEC_template.yml | 2 +-
 3 files changed, 11 insertions(+), 2 deletions(-)

Comments

Jan Kiszka April 16, 2025, 3:58 p.m. UTC | #1 
On 16.04.25 16:51, Sai.Sathujoda@toshiba-tsip.com wrote:
> From: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>
> 
> Signed-off-by: sai ashrith sathujoda <sai.sathujoda@toshiba-tsip.com>
> ---
>  .gitlab-ci.yml                   | 6 +++++-
>  scripts/submit_lava.sh           | 5 +++++
>  tests/templates/IEC_template.yml | 2 +-
>  3 files changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 146eb33..2c1deaa 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -334,7 +334,7 @@ build:qemu-riscv64:
>    tags:
>      - small
>    script:
> -    - scripts/submit_lava.sh ${test_function} ${target} ${CI_COMMIT_SHORT_SHA} ${release} ${CI_COMMIT_REF_SLUG}
> +    - scripts/submit_lava.sh ${test_function} ${target} ${CI_COMMIT_SHORT_SHA} ${release} ${CI_COMMIT_REF_SLUG} ${iec_test_timeout}
>    artifacts:
>      name: "$CI_JOB_NAME"
>      when: always
> @@ -431,6 +431,7 @@ test:qemu-amd64-IEC:
>    variables:
>      target: qemu-amd64
>      test_function: IEC
> +    iec_test_timeout: 40
>  
>  test:qemu-arm64-IEC:
>    extends:
> @@ -439,6 +440,7 @@ test:qemu-arm64-IEC:
>    variables:
>      target: qemu-arm64
>      test_function: IEC
> +    iec_test_timeout: 40
>  
>  test:qemu-arm-IEC:
>    extends:
> @@ -447,6 +449,7 @@ test:qemu-arm-IEC:
>    variables:
>      target: qemu-arm
>      test_function: IEC
> +    iec_test_timeout: 40
>  
>  test:x86-uefi-IEC:
>    extends:
> @@ -455,6 +458,7 @@ test:x86-uefi-IEC:
>    variables:
>      target: x86-uefi
>      test_function: IEC
> +    iec_test_timeout: 60
>  
>  test:qemu-amd64-secure-boot-unsigned-kernel:
>    extends:
> diff --git a/scripts/submit_lava.sh b/scripts/submit_lava.sh
> index 9da083a..028bad8 100755
> --- a/scripts/submit_lava.sh
> +++ b/scripts/submit_lava.sh
> @@ -24,6 +24,7 @@ TARGET=$2
>  COMMIT_REF=$3
>  RELEASE=$4
>  COMMIT_BRANCH=$5
> +IEC_TEST_TIMEOUT_MINUTES=$6

You can export already here.

>  
>  if [ -z "$SUBMIT_ONLY" ]; then SUBMIT_ONLY=false; fi
>  
> @@ -55,6 +56,8 @@ add_firmware_artifacts () {
>  create_job_qemu () {
>  	if [ "$1" = "IEC" ]; then
>  		cp $LAVA_TEMPLATES/IEC_template.yml "${job_dir}/${1}_${2}.yml"
> +		export IEC_TEST_TIMEOUT_MINUTES
> +		envsubst < "${job_dir}/${1}_${2}.yml" > IEC.tmp && mv IEC.tmp "${job_dir}/${1}_${2}.yml"
>  
>  	elif [ "$1" = "swupdate" ]; then
>  		cp $LAVA_TEMPLATES/swupdate_template.yml "${job_dir}/${1}_${2}.yml"
> @@ -163,6 +166,8 @@ create_job_mcom () {
>  	cp $LAVA_TEMPLATES/M-COM-x86.yml "${job_dir}/${1}_${2}.yml"
>  	if [ "$1" = "IEC" ]; then
>  		grep -A 9 "# TEST_BLOCK" "$LAVA_TEMPLATES/$1_template.yml" >> "${job_dir}/${1}_${2}.yml"
> +		export IEC_TEST_TIMEOUT_MINUTES
> +		envsubst < "${job_dir}/${1}_${2}.yml" > IEC.tmp && mv IEC.tmp "${job_dir}/${1}_${2}.yml"

Can't we avoid this duplication? How about running envsubst
unconditionally on the final yml, just like xenomai-images does? That
will allow us to use this pattern not only for the IEC cases.

>  	elif [ "$1" = "secure-boot" ]; then
>  		grep -A 1 "parameters" "$LAVA_TEMPLATES/secureboot_template.yml" >> "${job_dir}/${1}_${2}.yml"
>  	else
> diff --git a/tests/templates/IEC_template.yml b/tests/templates/IEC_template.yml
> index 92944da..c8efe40 100644
> --- a/tests/templates/IEC_template.yml
> +++ b/tests/templates/IEC_template.yml
> @@ -60,7 +60,7 @@ actions:
>  # TEST_BLOCK
>  - test:
>       timeout:
> -       minutes: 40
> +       minutes: ${IEC_TEST_TIMEOUT_MINUTES}
>       definitions:
>       - repository: https://gitlab.com/cip-project/cip-testing/cip-security-tests.git
>         from: git

Jan
Sai.Sathujoda@toshiba-tsip.com April 16, 2025, 4:58 p.m. UTC | #2 
Hi Jan,

>> Can't we avoid this duplication? How about running envsubst
>> unconditionally on the final yml, just like xenomai-images does? That
>> will allow us to use this pattern not only for the IEC cases.

The timeout of 40 minutes and 60 minutes is only applicable for IEC tests on QEMU and M-COM x86. For swupdate, secureboot related test cases that amount of time is not required.

That is why I added envsubst is qemu and m-com job creation methods separately. But as you suggested to reduce handling various timeout values, we can set 60 minutes timeout to all the test jobs (IEC, swupdate) and we can just run envsubst directly on the final yaml as you said. Please confirm if you are OK with this. Then I will update and send v4. I will also include exporting timeout value only once when submit_lava.sh script received the value from .gitlab-ci.yml in v4.

Thanks and regards,
Sai Ashrith
Jan Kiszka April 17, 2025, 5:12 a.m. UTC | #3 
On 16.04.25 18:58, Sai.Sathujoda@toshiba-tsip.com wrote:
> Hi Jan,
> 
>>> Can't we avoid this duplication? How about running envsubst
>>> unconditionally on the final yml, just like xenomai-images does? That
>>> will allow us to use this pattern not only for the IEC cases.
> 
> The timeout of 40 minutes and 60 minutes is only applicable for IEC
> tests on QEMU and M-COM x86. For swupdate, secureboot related test cases
> that amount of time is not required.

I was not only referring to a timeout as potential use case for
variables in the templates.

> 
> That is why I added envsubst is qemu and m-com job creation methods
> separately. But as you suggested to reduce handling various timeout
> values, we can set 60 minutes timeout to all the test jobs (IEC,
> swupdate) and we can just run envsubst directly on the final yaml as you
> said. Please confirm if you are OK with this. Then I will update and
> send v4. I will also include exporting timeout value only once when
> submit_lava.sh script received the value from .gitlab-ci.yml in v4.

Please move envsubst to central place an run it only once. It will do no
harm if a template has no vars to substitute.

Jan
diff mbox series

Patch

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 146eb33..2c1deaa 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -334,7 +334,7 @@  build:qemu-riscv64:
   tags:
     - small
   script:
-    - scripts/submit_lava.sh ${test_function} ${target} ${CI_COMMIT_SHORT_SHA} ${release} ${CI_COMMIT_REF_SLUG}
+    - scripts/submit_lava.sh ${test_function} ${target} ${CI_COMMIT_SHORT_SHA} ${release} ${CI_COMMIT_REF_SLUG} ${iec_test_timeout}
   artifacts:
     name: "$CI_JOB_NAME"
     when: always
@@ -431,6 +431,7 @@  test:qemu-amd64-IEC:
   variables:
     target: qemu-amd64
     test_function: IEC
+    iec_test_timeout: 40
 
 test:qemu-arm64-IEC:
   extends:
@@ -439,6 +440,7 @@  test:qemu-arm64-IEC:
   variables:
     target: qemu-arm64
     test_function: IEC
+    iec_test_timeout: 40
 
 test:qemu-arm-IEC:
   extends:
@@ -447,6 +449,7 @@  test:qemu-arm-IEC:
   variables:
     target: qemu-arm
     test_function: IEC
+    iec_test_timeout: 40
 
 test:x86-uefi-IEC:
   extends:
@@ -455,6 +458,7 @@  test:x86-uefi-IEC:
   variables:
     target: x86-uefi
     test_function: IEC
+    iec_test_timeout: 60
 
 test:qemu-amd64-secure-boot-unsigned-kernel:
   extends:
diff --git a/scripts/submit_lava.sh b/scripts/submit_lava.sh
index 9da083a..028bad8 100755
--- a/scripts/submit_lava.sh
+++ b/scripts/submit_lava.sh
@@ -24,6 +24,7 @@  TARGET=$2
 COMMIT_REF=$3
 RELEASE=$4
 COMMIT_BRANCH=$5
+IEC_TEST_TIMEOUT_MINUTES=$6
 
 if [ -z "$SUBMIT_ONLY" ]; then SUBMIT_ONLY=false; fi
 
@@ -55,6 +56,8 @@  add_firmware_artifacts () {
 create_job_qemu () {
 	if [ "$1" = "IEC" ]; then
 		cp $LAVA_TEMPLATES/IEC_template.yml "${job_dir}/${1}_${2}.yml"
+		export IEC_TEST_TIMEOUT_MINUTES
+		envsubst < "${job_dir}/${1}_${2}.yml" > IEC.tmp && mv IEC.tmp "${job_dir}/${1}_${2}.yml"
 
 	elif [ "$1" = "swupdate" ]; then
 		cp $LAVA_TEMPLATES/swupdate_template.yml "${job_dir}/${1}_${2}.yml"
@@ -163,6 +166,8 @@  create_job_mcom () {
 	cp $LAVA_TEMPLATES/M-COM-x86.yml "${job_dir}/${1}_${2}.yml"
 	if [ "$1" = "IEC" ]; then
 		grep -A 9 "# TEST_BLOCK" "$LAVA_TEMPLATES/$1_template.yml" >> "${job_dir}/${1}_${2}.yml"
+		export IEC_TEST_TIMEOUT_MINUTES
+		envsubst < "${job_dir}/${1}_${2}.yml" > IEC.tmp && mv IEC.tmp "${job_dir}/${1}_${2}.yml"
 	elif [ "$1" = "secure-boot" ]; then
 		grep -A 1 "parameters" "$LAVA_TEMPLATES/secureboot_template.yml" >> "${job_dir}/${1}_${2}.yml"
 	else
diff --git a/tests/templates/IEC_template.yml b/tests/templates/IEC_template.yml
index 92944da..c8efe40 100644
--- a/tests/templates/IEC_template.yml
+++ b/tests/templates/IEC_template.yml
@@ -60,7 +60,7 @@  actions:
 # TEST_BLOCK
 - test:
      timeout:
-       minutes: 40
+       minutes: ${IEC_TEST_TIMEOUT_MINUTES}
      definitions:
      - repository: https://gitlab.com/cip-project/cip-testing/cip-security-tests.git
        from: git