| Message ID | 20250205155946.2811296-2-berrange@redhat.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | tests/functional: a few misc cleanups and fixes | expand |
On 05/02/2025 16.59, Daniel P. Berrangé wrote: > If QEMU_TEST_QEMU_BINARY is not set we currently assert in the setUp > function, resulting in a big traceback: > > TAP version 13 > Traceback (most recent call last): > File "/var/home/berrange/src/virt/qemu/tests/functional/qemu_test/testcase.py", line 280, in setUp > super().setUp('qemu-system-') > ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^ > File "/var/home/berrange/src/virt/qemu/tests/functional/qemu_test/testcase.py", line 196, in setUp > self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set') > ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > AssertionError: unexpectedly None : QEMU_TEST_QEMU_BINARY must be set > > not ok 1 test_ppc_405.Ppc405Machine.test_ppc_ref405ep > 1..1 > > For every other test pre-requisite that's missing we will mark the test > as skipped. This does the same for missing QEMU_TEST_QEMU_BINARY, such > that we get > > TAP version 13 > ok 1 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest # SKIP QEMU_TEST_QEMU_BINARY env variable is not set > ok 2 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_noapic_nomsi # SKIP QEMU_TEST_QEMU_BINARY env variable is not set > ok 3 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_nomsi # SKIP QEMU_TEST_QEMU_BINARY env variable is not set > ok 4 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_novector # SKIP QEMU_TEST_QEMU_BINARY env variable is not set > ok 5 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_novector_noapic # SKIP QEMU_TEST_QEMU_BINARY env variable is not set > ok 6 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_novector_nomsi # SKIP QEMU_TEST_QEMU_BINARY env variable is not set > ok 7 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_vapic # SKIP QEMU_TEST_QEMU_BINARY env variable is not set > 1..7 Not sure whether this is the right approach, since a missing QEMU_TEST_QEMU_BINARY is a real error, and if we just skip, then the problem might go unnoticed if the user does not look closely. But to ease the situation: We could maybe add some auto-detection logic that tries to guess the right qemu-system-$TARGET by looking at the file name of the test and/or the test function name? We already encode the target architecture in most of these... WDYT? Thomas
On Thu, Feb 06, 2025 at 10:01:45AM +0100, Thomas Huth wrote: > On 05/02/2025 16.59, Daniel P. Berrangé wrote: > > If QEMU_TEST_QEMU_BINARY is not set we currently assert in the setUp > > function, resulting in a big traceback: > > > > TAP version 13 > > Traceback (most recent call last): > > File "/var/home/berrange/src/virt/qemu/tests/functional/qemu_test/testcase.py", line 280, in setUp > > super().setUp('qemu-system-') > > ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^ > > File "/var/home/berrange/src/virt/qemu/tests/functional/qemu_test/testcase.py", line 196, in setUp > > self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set') > > ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > AssertionError: unexpectedly None : QEMU_TEST_QEMU_BINARY must be set > > > > not ok 1 test_ppc_405.Ppc405Machine.test_ppc_ref405ep > > 1..1 > > > > For every other test pre-requisite that's missing we will mark the test > > as skipped. This does the same for missing QEMU_TEST_QEMU_BINARY, such > > that we get > > > > TAP version 13 > > ok 1 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest # SKIP QEMU_TEST_QEMU_BINARY env variable is not set > > ok 2 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_noapic_nomsi # SKIP QEMU_TEST_QEMU_BINARY env variable is not set > > ok 3 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_nomsi # SKIP QEMU_TEST_QEMU_BINARY env variable is not set > > ok 4 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_novector # SKIP QEMU_TEST_QEMU_BINARY env variable is not set > > ok 5 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_novector_noapic # SKIP QEMU_TEST_QEMU_BINARY env variable is not set > > ok 6 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_novector_nomsi # SKIP QEMU_TEST_QEMU_BINARY env variable is not set > > ok 7 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_vapic # SKIP QEMU_TEST_QEMU_BINARY env variable is not set > > 1..7 > > Not sure whether this is the right approach, since a missing > QEMU_TEST_QEMU_BINARY is a real error, and if we just skip, then the problem > might go unnoticed if the user does not look closely. > > But to ease the situation: We could maybe add some auto-detection logic that > tries to guess the right qemu-system-$TARGET by looking at the file name of > the test and/or the test function name? We already encode the target > architecture in most of these... WDYT? When the user provides a binary, a single binary applies to all tests, so tests that don't match the binary get skipped. I think it would be a bit wierd to auto-select a different binary per test. With regards, Daniel
diff --git a/tests/functional/qemu_test/testcase.py b/tests/functional/qemu_test/testcase.py index 869f3949fe..94541e8bfb 100644 --- a/tests/functional/qemu_test/testcase.py +++ b/tests/functional/qemu_test/testcase.py @@ -193,7 +193,9 @@ def assets_available(self): return True def setUp(self, bin_prefix): - self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set') + if self.qemu_bin is None: + self.skipTest("QEMU_TEST_QEMU_BINARY env variable is not set") + self.arch = self.qemu_bin.split('-')[-1] self.socketdir = None
If QEMU_TEST_QEMU_BINARY is not set we currently assert in the setUp function, resulting in a big traceback: TAP version 13 Traceback (most recent call last): File "/var/home/berrange/src/virt/qemu/tests/functional/qemu_test/testcase.py", line 280, in setUp super().setUp('qemu-system-') ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^ File "/var/home/berrange/src/virt/qemu/tests/functional/qemu_test/testcase.py", line 196, in setUp self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be set') ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: unexpectedly None : QEMU_TEST_QEMU_BINARY must be set not ok 1 test_ppc_405.Ppc405Machine.test_ppc_ref405ep 1..1 For every other test pre-requisite that's missing we will mark the test as skipped. This does the same for missing QEMU_TEST_QEMU_BINARY, such that we get TAP version 13 ok 1 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest # SKIP QEMU_TEST_QEMU_BINARY env variable is not set ok 2 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_noapic_nomsi # SKIP QEMU_TEST_QEMU_BINARY env variable is not set ok 3 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_nomsi # SKIP QEMU_TEST_QEMU_BINARY env variable is not set ok 4 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_novector # SKIP QEMU_TEST_QEMU_BINARY env variable is not set ok 5 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_novector_noapic # SKIP QEMU_TEST_QEMU_BINARY env variable is not set ok 6 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_novector_nomsi # SKIP QEMU_TEST_QEMU_BINARY env variable is not set ok 7 test_x86_64_kvm_xen.KVMXenGuest.test_kvm_xen_guest_vapic # SKIP QEMU_TEST_QEMU_BINARY env variable is not set 1..7 Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> --- tests/functional/qemu_test/testcase.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)