| Message ID | 20181009232607.15521-8-crosa@redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Acceptance Tests: basic architecture support | expand |
On 10/10/2018 01:26, Cleber Rosa wrote: > With the introduction of a variants file that can run the same > tests on various architectures, it makes sense to make most tests > to be reusable on those environments. The exception should be > when a test is really testing a specific architecture feature. > > With the change proposed here, on a command line such as: > > $ avocado run \ > --json-variants-load=tests/acceptance/variants/arch.json \ > -- tests/acceptance/ > > The boot_linux_console.py tests will appear as "CANCELED: Currently > specific to the x86_64 arch", which is as a good thing when compared > to being ignored by tags because: > > * The architecture specific parts can be addressed > * It will be run on the matching architecture (as opposed to always > being filtered out by the tags mechanism) > * CANCELED tests do no influence negatively the overall job results, > they're not considered an error or failure > > Signed-off-by: Cleber Rosa <crosa@redhat.com> > --- > tests/acceptance/boot_linux_console.py | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py > index 58032f971c..ba3ac036da 100644 > --- a/tests/acceptance/boot_linux_console.py > +++ b/tests/acceptance/boot_linux_console.py > @@ -19,12 +19,13 @@ class BootLinuxConsole(Test): > and the kernel command line is properly passed from QEMU to the kernel > > :avocado: enable > - :avocado: tags=x86_64 > """ > > timeout = 60 > > def test(self): > + if self.arch != 'x86_64': > + self.cancel('Currently specific to the x86_64 target arch') > kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/' > 'Everything/x86_64/os/images/pxeboot/vmlinuz') > kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a' > For some reason this test run quicker on a aarch64 host than my x86 laptop: ThunderX 88XX (aarch64 Little Endian): (05/30) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test;x86_64: PASS (12.88 s) Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz: (05/30) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test;x86_64: PASS (31.13 s) Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
On 10/10/18 6:50 AM, Philippe Mathieu-Daudé wrote: > On 10/10/2018 01:26, Cleber Rosa wrote: >> With the introduction of a variants file that can run the same >> tests on various architectures, it makes sense to make most tests >> to be reusable on those environments. The exception should be >> when a test is really testing a specific architecture feature. >> >> With the change proposed here, on a command line such as: >> >> $ avocado run \ >> --json-variants-load=tests/acceptance/variants/arch.json \ >> -- tests/acceptance/ >> >> The boot_linux_console.py tests will appear as "CANCELED: Currently >> specific to the x86_64 arch", which is as a good thing when compared >> to being ignored by tags because: >> >> * The architecture specific parts can be addressed >> * It will be run on the matching architecture (as opposed to always >> being filtered out by the tags mechanism) >> * CANCELED tests do no influence negatively the overall job results, >> they're not considered an error or failure >> >> Signed-off-by: Cleber Rosa <crosa@redhat.com> >> --- >> tests/acceptance/boot_linux_console.py | 3 ++- >> 1 file changed, 2 insertions(+), 1 deletion(-) >> >> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py >> index 58032f971c..ba3ac036da 100644 >> --- a/tests/acceptance/boot_linux_console.py >> +++ b/tests/acceptance/boot_linux_console.py >> @@ -19,12 +19,13 @@ class BootLinuxConsole(Test): >> and the kernel command line is properly passed from QEMU to the kernel >> >> :avocado: enable >> - :avocado: tags=x86_64 >> """ >> >> timeout = 60 >> >> def test(self): >> + if self.arch != 'x86_64': >> + self.cancel('Currently specific to the x86_64 target arch') >> kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/' >> 'Everything/x86_64/os/images/pxeboot/vmlinuz') >> kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a' >> > > For some reason this test run quicker on a aarch64 host than my x86 laptop: > > ThunderX 88XX (aarch64 Little Endian): > > (05/30) > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test;x86_64: > PASS (12.88 s) > > Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz: > > (05/30) > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test;x86_64: > PASS (31.13 s) > The only reason I can think of is faster networking on the aarch64 machine. Are those numbers more or less the same on subsequent runs? Thanks for testing it! - Cleber. > Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> > Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> >
Hi Cleber, On 10/10/18 1:26 AM, Cleber Rosa wrote: > With the introduction of a variants file that can run the same > tests on various architectures, it makes sense to make most tests > to be reusable on those environments. The exception should be > when a test is really testing a specific architecture feature. > > With the change proposed here, on a command line such as: > > $ avocado run \ > --json-variants-load=tests/acceptance/variants/arch.json \ > -- tests/acceptance/ > > The boot_linux_console.py tests will appear as "CANCELED: Currently > specific to the x86_64 arch", which is as a good thing when compared > to being ignored by tags because: > > * The architecture specific parts can be addressed > * It will be run on the matching architecture (as opposed to always > being filtered out by the tags mechanism) > * CANCELED tests do no influence negatively the overall job results, > they're not considered an error or failure > > Signed-off-by: Cleber Rosa <crosa@redhat.com> > --- > tests/acceptance/boot_linux_console.py | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py > index 58032f971c..ba3ac036da 100644 > --- a/tests/acceptance/boot_linux_console.py > +++ b/tests/acceptance/boot_linux_console.py > @@ -19,12 +19,13 @@ class BootLinuxConsole(Test): > and the kernel command line is properly passed from QEMU to the kernel > > :avocado: enable > - :avocado: tags=x86_64 Can we keep a such tag (in format 'arch:x86_64') ... > """ > > timeout = 60 > > def test(self): > + if self.arch != 'x86_64': > + self.cancel('Currently specific to the x86_64 target arch') ... and have this check generic? (Eventually via a QemuTest(Test) parent class). > kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/' > 'Everything/x86_64/os/images/pxeboot/vmlinuz') > kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a' >
On 10/13/18 7:08 AM, Philippe Mathieu-Daudé wrote: > Hi Cleber, > > On 10/10/18 1:26 AM, Cleber Rosa wrote: >> With the introduction of a variants file that can run the same >> tests on various architectures, it makes sense to make most tests >> to be reusable on those environments. The exception should be >> when a test is really testing a specific architecture feature. >> >> With the change proposed here, on a command line such as: >> >> $ avocado run \ >> --json-variants-load=tests/acceptance/variants/arch.json \ >> -- tests/acceptance/ >> >> The boot_linux_console.py tests will appear as "CANCELED: Currently >> specific to the x86_64 arch", which is as a good thing when compared >> to being ignored by tags because: >> >> * The architecture specific parts can be addressed >> * It will be run on the matching architecture (as opposed to always >> being filtered out by the tags mechanism) >> * CANCELED tests do no influence negatively the overall job results, >> they're not considered an error or failure >> >> Signed-off-by: Cleber Rosa <crosa@redhat.com> >> --- >> tests/acceptance/boot_linux_console.py | 3 ++- >> 1 file changed, 2 insertions(+), 1 deletion(-) >> >> diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py >> index 58032f971c..ba3ac036da 100644 >> --- a/tests/acceptance/boot_linux_console.py >> +++ b/tests/acceptance/boot_linux_console.py >> @@ -19,12 +19,13 @@ class BootLinuxConsole(Test): >> and the kernel command line is properly passed from QEMU to the kernel >> >> :avocado: enable >> - :avocado: tags=x86_64 > > Can we keep a such tag (in format 'arch:x86_64') ... > Yes, sure. That's how the best "standards" are born ;) >> """ >> >> timeout = 60 >> >> def test(self): >> + if self.arch != 'x86_64': >> + self.cancel('Currently specific to the x86_64 target arch') > > ... and have this check generic? (Eventually via a QemuTest(Test) parent > class). > Yes, there's a lot of upcoming setup that can be made generic to the base class. Some of that is coming soon. Thanks! - Cleber. >> kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/' >> 'Everything/x86_64/os/images/pxeboot/vmlinuz') >> kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a' >>
diff --git a/tests/acceptance/boot_linux_console.py b/tests/acceptance/boot_linux_console.py index 58032f971c..ba3ac036da 100644 --- a/tests/acceptance/boot_linux_console.py +++ b/tests/acceptance/boot_linux_console.py @@ -19,12 +19,13 @@ class BootLinuxConsole(Test): and the kernel command line is properly passed from QEMU to the kernel :avocado: enable - :avocado: tags=x86_64 """ timeout = 60 def test(self): + if self.arch != 'x86_64': + self.cancel('Currently specific to the x86_64 target arch') kernel_url = ('https://mirrors.kernel.org/fedora/releases/28/' 'Everything/x86_64/os/images/pxeboot/vmlinuz') kernel_hash = '238e083e114c48200f80d889f7e32eeb2793e02a'
With the introduction of a variants file that can run the same tests on various architectures, it makes sense to make most tests to be reusable on those environments. The exception should be when a test is really testing a specific architecture feature. With the change proposed here, on a command line such as: $ avocado run \ --json-variants-load=tests/acceptance/variants/arch.json \ -- tests/acceptance/ The boot_linux_console.py tests will appear as "CANCELED: Currently specific to the x86_64 arch", which is as a good thing when compared to being ignored by tags because: * The architecture specific parts can be addressed * It will be run on the matching architecture (as opposed to always being filtered out by the tags mechanism) * CANCELED tests do no influence negatively the overall job results, they're not considered an error or failure Signed-off-by: Cleber Rosa <crosa@redhat.com> --- tests/acceptance/boot_linux_console.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)