diff mbox series

[v10,3/3] Acceptance tests: add make targets to download images

Message ID 20200317141654.29355-4-crosa@redhat.com (mailing list archive)
State New, archived
Headers show
Series Acceptance test: Add "boot_linux" acceptance test | expand

Commit Message

Cleber Rosa March 17, 2020, 2:16 p.m. UTC 
The newly introduced "boot linux" tests make use of Linux images that
are larger than usual, and fall into what Avocado calls "vmimages",
and can be referred to by name, version and architecture.

The images can be downloaded automatically during the test. But, to
make for more reliable test results, this introduces a target that
will download the vmimages for the architectures that have been
configured and are available for the currently used distro (Fedora
31).

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
---
 tests/Makefile.include | 19 +++++++++++++++++--
 1 file changed, 17 insertions(+), 2 deletions(-)

Comments

Alex Bennée March 17, 2020, 6:36 p.m. UTC | #1 
Cleber Rosa <crosa@redhat.com> writes:

> The newly introduced "boot linux" tests make use of Linux images that
> are larger than usual, and fall into what Avocado calls "vmimages",
> and can be referred to by name, version and architecture.
>
> The images can be downloaded automatically during the test. But, to
> make for more reliable test results, this introduces a target that
> will download the vmimages for the architectures that have been
> configured and are available for the currently used distro (Fedora
> 31).
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> Reviewed-by: Willian Rampazzo <willianr@redhat.com>
> ---
>  tests/Makefile.include | 19 +++++++++++++++++--
>  1 file changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 67e8fcddda..9c19229a06 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -20,6 +20,8 @@ check-help:
>  	@echo " $(MAKE) check-venv           Creates a Python venv for tests"
>  	@echo " $(MAKE) check-clean          Clean the tests and related data"
>  	@echo
> +	@echo " $(MAKE) get-vm-images        Downloads all images used by acceptance tests, according to configured targets (~350 MB each, 1.5 GB max)"
> +	@echo

I'm not overly enamoured with a super long line help for something that
is a dependency anyway but whatever...


>  	@echo
>  	@echo "The variable SPEED can be set to control the gtester speed setting."
>  	@echo "Default options are -k and (for $(MAKE) V=1) --verbose; they can be"
> @@ -889,7 +891,20 @@ $(TESTS_RESULTS_DIR):
>  
>  check-venv: $(TESTS_VENV_DIR)
>  
> -check-acceptance: check-venv $(TESTS_RESULTS_DIR)
> +FEDORA_31_ARCHES_CANDIDATES=$(patsubst ppc64,ppc64le,$(TARGETS))
> +FEDORA_31_ARCHES := x86_64 aarch64 ppc64le s390x
> +FEDORA_31_DOWNLOAD=$(filter $(FEDORA_31_ARCHES),$(FEDORA_31_ARCHES_CANDIDATES))
> +
> +# download one specific Fedora 31 image
> +get-vm-image-fedora-31-%: $(check-venv)

Why $(check-venv) instead of check-venv

> +	$(call quiet-command, \
> +             $(TESTS_VENV_DIR)/bin/python -m avocado vmimage get \
> +             --distro=fedora --distro-version=31 --arch=$*)

Some short text for the operation would be nice (DNLD acceptance images?)

> +
> +# download all vm images, according to defined targets
> +get-vm-images: $(check-venv) $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOWNLOAD))
> +
> +check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
>  	$(call quiet-command, \
>              $(TESTS_VENV_DIR)/bin/python -m avocado \
>              --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
> @@ -900,7 +915,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
>  
>  # Consolidated targets
>  
> -.PHONY: check-block check-qapi-schema check-qtest check-unit check check-clean
> +.PHONY: check-block check-qapi-schema check-qtest check-unit check check-clean get-vm-images
>  check-qapi-schema: check-tests/qapi-schema/frontend check-tests/qapi-schema/doc-good.texi
>  check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
>  ifeq ($(CONFIG_TOOLS),y)

Otherwise:

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Willian Rampazzo March 17, 2020, 9:49 p.m. UTC | #2 
On Tue, Mar 17, 2020 at 11:17 AM Cleber Rosa <crosa@redhat.com> wrote:
>
> The newly introduced "boot linux" tests make use of Linux images that
> are larger than usual, and fall into what Avocado calls "vmimages",
> and can be referred to by name, version and architecture.
>
> The images can be downloaded automatically during the test. But, to
> make for more reliable test results, this introduces a target that
> will download the vmimages for the architectures that have been
> configured and are available for the currently used distro (Fedora
> 31).
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> Reviewed-by: Willian Rampazzo <willianr@redhat.com>
> ---
>  tests/Makefile.include | 19 +++++++++++++++++--
>  1 file changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 67e8fcddda..9c19229a06 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -20,6 +20,8 @@ check-help:
>         @echo " $(MAKE) check-venv           Creates a Python venv for tests"
>         @echo " $(MAKE) check-clean          Clean the tests and related data"
>         @echo
> +       @echo " $(MAKE) get-vm-images        Downloads all images used by acceptance tests, according to configured targets (~350 MB each, 1.5 GB max)"
> +       @echo
>         @echo
>         @echo "The variable SPEED can be set to control the gtester speed setting."
>         @echo "Default options are -k and (for $(MAKE) V=1) --verbose; they can be"
> @@ -889,7 +891,20 @@ $(TESTS_RESULTS_DIR):
>
>  check-venv: $(TESTS_VENV_DIR)
>
> -check-acceptance: check-venv $(TESTS_RESULTS_DIR)
> +FEDORA_31_ARCHES_CANDIDATES=$(patsubst ppc64,ppc64le,$(TARGETS))
> +FEDORA_31_ARCHES := x86_64 aarch64 ppc64le s390x
> +FEDORA_31_DOWNLOAD=$(filter $(FEDORA_31_ARCHES),$(FEDORA_31_ARCHES_CANDIDATES))
> +
> +# download one specific Fedora 31 image
> +get-vm-image-fedora-31-%: $(check-venv)
> +       $(call quiet-command, \
> +             $(TESTS_VENV_DIR)/bin/python -m avocado vmimage get \
> +             --distro=fedora --distro-version=31 --arch=$*)
> +
> +# download all vm images, according to defined targets
> +get-vm-images: $(check-venv) $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOWNLOAD))

I tested this target and it works as expected. In this case, also a

Tested-by: Willian Rampazzo <willianr@redhat.com>

> +
> +check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
>         $(call quiet-command, \
>              $(TESTS_VENV_DIR)/bin/python -m avocado \
>              --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
> @@ -900,7 +915,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
>
>  # Consolidated targets
>
> -.PHONY: check-block check-qapi-schema check-qtest check-unit check check-clean
> +.PHONY: check-block check-qapi-schema check-qtest check-unit check check-clean get-vm-images
>  check-qapi-schema: check-tests/qapi-schema/frontend check-tests/qapi-schema/doc-good.texi
>  check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
>  ifeq ($(CONFIG_TOOLS),y)
> --
> 2.25.1
>
Cleber Rosa March 17, 2020, 11:10 p.m. UTC | #3 
On Tue, Mar 17, 2020 at 06:36:18PM +0000, Alex Bennée wrote:
> 
> Cleber Rosa <crosa@redhat.com> writes:
> 
> > The newly introduced "boot linux" tests make use of Linux images that
> > are larger than usual, and fall into what Avocado calls "vmimages",
> > and can be referred to by name, version and architecture.
> >
> > The images can be downloaded automatically during the test. But, to
> > make for more reliable test results, this introduces a target that
> > will download the vmimages for the architectures that have been
> > configured and are available for the currently used distro (Fedora
> > 31).
> >
> > Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > Reviewed-by: Willian Rampazzo <willianr@redhat.com>
> > ---
> >  tests/Makefile.include | 19 +++++++++++++++++--
> >  1 file changed, 17 insertions(+), 2 deletions(-)
> >
> > diff --git a/tests/Makefile.include b/tests/Makefile.include
> > index 67e8fcddda..9c19229a06 100644
> > --- a/tests/Makefile.include
> > +++ b/tests/Makefile.include
> > @@ -20,6 +20,8 @@ check-help:
> >  	@echo " $(MAKE) check-venv           Creates a Python venv for tests"
> >  	@echo " $(MAKE) check-clean          Clean the tests and related data"
> >  	@echo
> > +	@echo " $(MAKE) get-vm-images        Downloads all images used by acceptance tests, according to configured targets (~350 MB each, 1.5 GB max)"
> > +	@echo
> 
> I'm not overly enamoured with a super long line help for something that
> is a dependency anyway but whatever...
> 
>

I had a hard time choosing the right info to put there, and I didn't
want to break the style of the majority of help messages that use a
single line.  Anyway, we can always improve that.

> >  	@echo
> >  	@echo "The variable SPEED can be set to control the gtester speed setting."
> >  	@echo "Default options are -k and (for $(MAKE) V=1) --verbose; they can be"
> > @@ -889,7 +891,20 @@ $(TESTS_RESULTS_DIR):
> >  
> >  check-venv: $(TESTS_VENV_DIR)
> >  
> > -check-acceptance: check-venv $(TESTS_RESULTS_DIR)
> > +FEDORA_31_ARCHES_CANDIDATES=$(patsubst ppc64,ppc64le,$(TARGETS))
> > +FEDORA_31_ARCHES := x86_64 aarch64 ppc64le s390x
> > +FEDORA_31_DOWNLOAD=$(filter $(FEDORA_31_ARCHES),$(FEDORA_31_ARCHES_CANDIDATES))
> > +
> > +# download one specific Fedora 31 image
> > +get-vm-image-fedora-31-%: $(check-venv)
> 
> Why $(check-venv) instead of check-venv
>

Good point.

> > +	$(call quiet-command, \
> > +             $(TESTS_VENV_DIR)/bin/python -m avocado vmimage get \
> > +             --distro=fedora --distro-version=31 --arch=$*)
> 
> Some short text for the operation would be nice (DNLD acceptance images?)
>

Another good point.

> > +
> > +# download all vm images, according to defined targets
> > +get-vm-images: $(check-venv) $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOWNLOAD))
> > +
> > +check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
> >  	$(call quiet-command, \
> >              $(TESTS_VENV_DIR)/bin/python -m avocado \
> >              --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
> > @@ -900,7 +915,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR)
> >  
> >  # Consolidated targets
> >  
> > -.PHONY: check-block check-qapi-schema check-qtest check-unit check check-clean
> > +.PHONY: check-block check-qapi-schema check-qtest check-unit check check-clean get-vm-images
> >  check-qapi-schema: check-tests/qapi-schema/frontend check-tests/qapi-schema/doc-good.texi
> >  check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
> >  ifeq ($(CONFIG_TOOLS),y)
> 
> Otherwise:
> 
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> 
>

Thanks!
- Cleber.

> -- 
> Alex Bennée
>
diff mbox series

Patch

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 67e8fcddda..9c19229a06 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -20,6 +20,8 @@  check-help:
 	@echo " $(MAKE) check-venv           Creates a Python venv for tests"
 	@echo " $(MAKE) check-clean          Clean the tests and related data"
 	@echo
+	@echo " $(MAKE) get-vm-images        Downloads all images used by acceptance tests, according to configured targets (~350 MB each, 1.5 GB max)"
+	@echo
 	@echo
 	@echo "The variable SPEED can be set to control the gtester speed setting."
 	@echo "Default options are -k and (for $(MAKE) V=1) --verbose; they can be"
@@ -889,7 +891,20 @@  $(TESTS_RESULTS_DIR):
 
 check-venv: $(TESTS_VENV_DIR)
 
-check-acceptance: check-venv $(TESTS_RESULTS_DIR)
+FEDORA_31_ARCHES_CANDIDATES=$(patsubst ppc64,ppc64le,$(TARGETS))
+FEDORA_31_ARCHES := x86_64 aarch64 ppc64le s390x
+FEDORA_31_DOWNLOAD=$(filter $(FEDORA_31_ARCHES),$(FEDORA_31_ARCHES_CANDIDATES))
+
+# download one specific Fedora 31 image
+get-vm-image-fedora-31-%: $(check-venv)
+	$(call quiet-command, \
+             $(TESTS_VENV_DIR)/bin/python -m avocado vmimage get \
+             --distro=fedora --distro-version=31 --arch=$*)
+
+# download all vm images, according to defined targets
+get-vm-images: $(check-venv) $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_31_DOWNLOAD))
+
+check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
 	$(call quiet-command, \
             $(TESTS_VENV_DIR)/bin/python -m avocado \
             --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \
@@ -900,7 +915,7 @@  check-acceptance: check-venv $(TESTS_RESULTS_DIR)
 
 # Consolidated targets
 
-.PHONY: check-block check-qapi-schema check-qtest check-unit check check-clean
+.PHONY: check-block check-qapi-schema check-qtest check-unit check check-clean get-vm-images
 check-qapi-schema: check-tests/qapi-schema/frontend check-tests/qapi-schema/doc-good.texi
 check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
 ifeq ($(CONFIG_TOOLS),y)