| Message ID | 20220225210156.2032055-5-crosa@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Avocado tests: filter out tests using TCG booting full blown distros | expand |
On 25/02/2022 22.01, Cleber Rosa wrote: > Even though there have been a number of improvements (and some pretty > deep internal changes) since Avocado 88.1, only one change should > affect "make check-avocado". > > With the nrunner architecture, test execution happens in parallel by > default. But, tests may fail due to insufficient timeouts or similar > reasons when run under systems with limited or shared resources. To > avoid breakages, especially on CI, let's keep the serial execution > until proven that it won't impact the CI jobs. > > Signed-off-by: Cleber Rosa <crosa@redhat.com> > --- > tests/Makefile.include | 1 + > tests/requirements.txt | 2 +- > 2 files changed, 2 insertions(+), 1 deletion(-) > > diff --git a/tests/Makefile.include b/tests/Makefile.include > index e7153c8e91..676aa0d944 100644 > --- a/tests/Makefile.include > +++ b/tests/Makefile.include > @@ -135,6 +135,7 @@ check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images > $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \ > --filter-by-tags-include-empty-key) \ > $(AVOCADO_CMDLINE_TAGS) \ > + --nrunner-max-parallel-tasks=1 \ Could you maybe change it so that it uses the "-jX" setting from "make" instead? ... that way the users could decide whether they want to run the tests in parallel or not. I think you could get that value from the MAKEFLAGS env variable, see tests/check-block.sh : JOBS=$(echo "$MAKEFLAGS" | sed -n 's/\(^\|.* \)-j\([0-9]\+\)\( .*\|$\)/-j \2/p') Thomas
diff --git a/tests/Makefile.include b/tests/Makefile.include index e7153c8e91..676aa0d944 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -135,6 +135,7 @@ check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images $(if $(AVOCADO_TAGS),, --filter-by-tags-include-empty \ --filter-by-tags-include-empty-key) \ $(AVOCADO_CMDLINE_TAGS) \ + --nrunner-max-parallel-tasks=1 \ $(if $(GITLAB_CI),,--failfast) $(AVOCADO_TESTS), \ "AVOCADO", "tests/avocado") diff --git a/tests/requirements.txt b/tests/requirements.txt index a21b59b443..49aa0fd6f6 100644 --- a/tests/requirements.txt +++ b/tests/requirements.txt @@ -1,5 +1,5 @@ # Add Python module requirements, one per line, to be installed # in the tests/venv Python virtual environment. For more info, # refer to: https://pip.pypa.io/en/stable/user_guide/#id1 -avocado-framework==88.1 +avocado-framework==95.0 pycdlib==1.11.0
Even though there have been a number of improvements (and some pretty deep internal changes) since Avocado 88.1, only one change should affect "make check-avocado". With the nrunner architecture, test execution happens in parallel by default. But, tests may fail due to insufficient timeouts or similar reasons when run under systems with limited or shared resources. To avoid breakages, especially on CI, let's keep the serial execution until proven that it won't impact the CI jobs. Signed-off-by: Cleber Rosa <crosa@redhat.com> --- tests/Makefile.include | 1 + tests/requirements.txt | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)