@@ -2075,8 +2075,8 @@ ACPI/AVOCADO/BIOSBITS
M: Ani Sinha <anisinha@redhat.com>
M: Michael S. Tsirkin <mst@redhat.com>
S: Supported
-F: tests/avocado/acpi-bits/*
-F: tests/avocado/acpi-bits.py
+F: tests/functional/acpi-bits/*
+F: tests/functional/test_acpi_bits.py
F: docs/devel/acpi-bits.rst
ACPI/HEST/GHES
@@ -1,6 +1,6 @@
-=============================================================================
-ACPI/SMBIOS avocado tests using biosbits
-=============================================================================
+==================================
+ACPI/SMBIOS testing using biosbits
+==================================
************
Introduction
************
@@ -35,7 +35,7 @@ for developing biosbits and its real life uses can be found in [#a]_ and [#b]_.
For QEMU, we maintain a fork of bios bits in gitlab along with all the
dependent submodules `here <https://gitlab.com/qemu-project/biosbits-bits>`__.
This fork contains numerous fixes, a newer acpica and changes specific to
-running this avocado QEMU tests using bits. The author of this document
+running these functional QEMU tests using bits. The author of this document
is the sole maintainer of the QEMU fork of bios bits repository. For more
information, please see author's `FOSDEM talk on this bios-bits based test
framework <https://fosdem.org/2024/schedule/event/fosdem-2024-2262-exercising-qemu-generated-acpi-smbios-tables-using-biosbits-from-within-a-guest-vm-/>`__.
@@ -44,12 +44,12 @@ framework <https://fosdem.org/2024/schedule/event/fosdem-2024-2262-exercising-qe
Description of the test framework
*********************************
-Under the directory ``tests/avocado/``, ``acpi-bits.py`` is a QEMU avocado
-test that drives all this.
+Under the directory ``tests/functional/``, ``test_acpi_bits.py`` is a QEMU
+functional test that drives all this.
A brief description of the various test files follows.
-Under ``tests/avocado/`` as the root we have:
+Under ``tests/functional/`` as the root we have:
::
@@ -60,12 +60,12 @@ Under ``tests/avocado/`` as the root we have:
│ ├── smbios.py2
│ ├── testacpi.py2
│ └── testcpuid.py2
- ├── acpi-bits.py
+ ├── test_acpi_bits.py
-* ``tests/avocado``:
+* ``tests/functional``:
- ``acpi-bits.py``:
- This is the main python avocado test script that generates a
+ ``test_acpi_bits.py``:
+ This is the main python functional test script that generates a
biosbits iso. It then spawns a QEMU VM with it, collects the log and reports
test failures. This is the script one would be interested in if they wanted
to add or change some component of the log parsing, add a new command line
@@ -79,35 +79,22 @@ Under ``tests/avocado/`` as the root we have:
you to inspect and run the specific commands manually.
In order to run this test, please perform the following steps from the QEMU
- build directory:
+ build directory (assuming that the sources are in ".."):
::
- $ make check-venv (needed only the first time to create the venv)
- $ ./pyvenv/bin/avocado run -t acpi tests/avocado
+ $ export PYTHONPATH=../python:../tests/functional
+ $ export QEMU_TEST_QEMU_BINARY=$PWD/qemu-system-x86_64
+ $ python3 ../tests/functional/test_acpi_bits.py
- The above will run all acpi avocado tests including this one.
- In order to run the individual tests, perform the following:
- ::
-
- $ ./pyvenv/bin/avocado run tests/avocado/acpi-bits.py --tap -
-
- The above will produce output in tap format. You can omit "--tap -" in the
- end and it will produce output like the following:
- ::
-
- $ ./pyvenv/bin/avocado run tests/avocado/acpi-bits.py
- Fetching asset from tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
- JOB ID : eab225724da7b64c012c65705dc2fa14ab1defef
- JOB LOG : /home/anisinha/avocado/job-results/job-2022-10-10T17.58-eab2257/job.log
- (1/1) tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits: PASS (33.09 s)
- RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
- JOB TIME : 39.22 s
+ The above will run all acpi-bits functional tests (producing output in
+ tap format).
- You can inspect the log file for more information about the run or in order
- to diagnoze issues. If you pass V=1 in the environment, more diagnostic logs
- would be found in the test log.
+ You can inspect the log files in tests/functional/x86_64/test_acpi_bits.*/
+ for more information about the run or in order to diagnoze issues.
+ If you pass V=1 in the environment, more diagnostic logs will be put into
+ the test log.
-* ``tests/avocado/acpi-bits/bits-config``:
+* ``tests/functional/acpi-bits/bits-config``:
This location contains biosbits configuration files that determine how the
software runs the tests.
@@ -117,7 +104,7 @@ Under ``tests/avocado/`` as the root we have:
or actions are performed by bits. The description of the config options are
provided in the file itself.
-* ``tests/avocado/acpi-bits/bits-tests``:
+* ``tests/functional/acpi-bits/bits-tests``:
This directory contains biosbits python based tests that are run from within
the biosbits environment in the spawned VM. New additions of test cases can
@@ -155,7 +142,8 @@ Under ``tests/avocado/`` as the root we have:
(a) They are python2.7 based scripts and not python 3 scripts.
(b) They are run from within the bios bits VM and is not subjected to QEMU
build/test python script maintenance and dependency resolutions.
- (c) They need not be loaded by avocado framework when running tests.
+ (c) They need not be loaded by the test framework by accident when running
+ tests.
Author: Ani Sinha <anisinha@redhat.com>
similarity index 100%
rename from tests/avocado/acpi-bits/bits-config/bits-cfg.txt
rename to tests/functional/acpi-bits/bits-config/bits-cfg.txt
similarity index 100%
rename from tests/avocado/acpi-bits/bits-tests/smbios.py2
rename to tests/functional/acpi-bits/bits-tests/smbios.py2
similarity index 100%
rename from tests/avocado/acpi-bits/bits-tests/smilatency.py2
rename to tests/functional/acpi-bits/bits-tests/smilatency.py2
similarity index 100%
rename from tests/avocado/acpi-bits/bits-tests/testacpi.py2
rename to tests/functional/acpi-bits/bits-tests/testacpi.py2
similarity index 100%
rename from tests/avocado/acpi-bits/bits-tests/testcpuid.py2
rename to tests/functional/acpi-bits/bits-tests/testcpuid.py2
@@ -11,6 +11,7 @@ endif
# Timeouts for individual tests that can be slow e.g. with debugging enabled
test_timeouts = {
+ 'acpi_bits' : 240,
'netdev_ethtool' : 180,
'ppc_40p' : 240,
'ppc64_hv' : 1000,
@@ -99,6 +100,7 @@ tests_x86_64_quick = [
]
tests_x86_64_thorough = [
+ 'acpi_bits',
'netdev_ethtool',
'virtio_gpu',
]
old mode 100644
new mode 100755
similarity index 86%
rename from tests/avocado/acpi-bits.py
rename to tests/functional/test_acpi_bits.py
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
-# group: rw quick
+#
# Exercise QEMU generated ACPI/SMBIOS tables using biosbits,
# https://biosbits.org/
#
@@ -24,7 +24,7 @@
# pylint: disable=consider-using-f-string
"""
-This is QEMU ACPI/SMBIOS avocado tests using biosbits.
+This is QEMU ACPI/SMBIOS functional tests using biosbits.
Biosbits is available originally at https://biosbits.org/.
This test uses a fork of the upstream bits and has numerous fixes
including an upgraded acpica. The fork is located here:
@@ -41,15 +41,16 @@
import tempfile
import time
import zipfile
+
+from pathlib import Path
from typing import (
List,
Optional,
Sequence,
)
from qemu.machine import QEMUMachine
-from avocado import skipIf
-from avocado.utils import datadrainer as drainer
-from avocado_qemu import QemuBaseTest
+from unittest import skipIf
+from qemu_test import QemuBaseTest, Asset
deps = ["xorriso", "mformat"] # dependent tools needed in the test setup/box.
supported_platforms = ['x86_64'] # supported test platforms.
@@ -129,34 +130,32 @@ def base_args(self):
class AcpiBitsTest(QemuBaseTest): #pylint: disable=too-many-instance-attributes
"""
ACPI and SMBIOS tests using biosbits.
-
- :avocado: tags=arch:x86_64
- :avocado: tags=acpi
-
"""
# in slower systems the test can take as long as 3 minutes to complete.
timeout = BITS_TIMEOUT
+ # following are some standard configuration constants
+ # gitlab CI does shallow clones of depth 20
+ BITS_INTERNAL_VER = 2020
+ # commit hash must match the artifact tag below
+ BITS_COMMIT_HASH = 'c7920d2b'
+ # this is the latest bits release as of today.
+ BITS_TAG = "qemu-bits-10262023"
+
+ ASSET_BITS = Asset(("https://gitlab.com/qemu-project/"
+ "biosbits-bits/-/jobs/artifacts/%s/"
+ "download?job=qemu-bits-build" % BITS_TAG),
+ '1b8dd612c6831a6b491716a77acc486666aaa867051cdc34f7ce169c2e25f487')
+
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._vm = None
self._workDir = None
self._baseDir = None
- # following are some standard configuration constants
- self._bitsInternalVer = 2020 # gitlab CI does shallow clones of depth 20
- self._bitsCommitHash = 'c7920d2b' # commit hash must match
- # the artifact tag below
- self._bitsTag = "qemu-bits-10262023" # this is the latest bits
- # release as of today.
- self._bitsArtSHA1Hash = 'b22cdfcfc7453875297d06d626f5474ee36a343f'
- self._bitsArtURL = ("https://gitlab.com/qemu-project/"
- "biosbits-bits/-/jobs/artifacts/%s/"
- "download?job=qemu-bits-build" %self._bitsTag)
self._debugcon_addr = '0x403'
self._debugcon_log = 'debugcon-log.txt'
- logging.basicConfig(level=logging.INFO)
- self.logger = logging.getLogger('acpi-bits')
+ self.logger = self.log
def _print_log(self, log):
self.logger.info('\nlogs from biosbits follows:')
@@ -171,7 +170,7 @@ def copy_bits_config(self):
bits_config_dir = os.path.join(self._baseDir, 'acpi-bits',
'bits-config')
target_config_dir = os.path.join(self._workDir,
- 'bits-%d' %self._bitsInternalVer,
+ 'bits-%d' %self.BITS_INTERNAL_VER,
'boot')
self.assertTrue(os.path.exists(bits_config_dir))
self.assertTrue(os.path.exists(target_config_dir))
@@ -188,7 +187,7 @@ def copy_test_scripts(self):
bits_test_dir = os.path.join(self._baseDir, 'acpi-bits',
'bits-tests')
target_test_dir = os.path.join(self._workDir,
- 'bits-%d' %self._bitsInternalVer,
+ 'bits-%d' %self.BITS_INTERNAL_VER,
'boot', 'python')
self.assertTrue(os.path.exists(bits_test_dir))
@@ -248,9 +247,9 @@ def generate_bits_iso(self):
test scripts
"""
bits_dir = os.path.join(self._workDir,
- 'bits-%d' %self._bitsInternalVer)
+ 'bits-%d' %self.BITS_INTERNAL_VER)
iso_file = os.path.join(self._workDir,
- 'bits-%d.iso' %self._bitsInternalVer)
+ 'bits-%d.iso' %self.BITS_INTERNAL_VER)
mkrescue_script = os.path.join(self._workDir,
'grub-inst-x86_64-efi', 'bin',
'grub-mkrescue')
@@ -264,8 +263,12 @@ def generate_bits_iso(self):
try:
if os.getenv('V') or os.getenv('BITS_DEBUG'):
- subprocess.check_call([mkrescue_script, '-o', iso_file,
- bits_dir], stderr=subprocess.STDOUT)
+ proc = subprocess.run([mkrescue_script, '-o', iso_file,
+ bits_dir],
+ stdout=subprocess.PIPE,
+ stderr=subprocess.STDOUT,
+ check=True)
+ self.logger.info("grub-mkrescue output %s" % proc.stdout)
else:
subprocess.check_call([mkrescue_script, '-o',
iso_file, bits_dir],
@@ -282,8 +285,9 @@ def generate_bits_iso(self):
def setUp(self): # pylint: disable=arguments-differ
super().setUp('qemu-system-')
+ self.logger = self.log
- self._baseDir = os.getenv('AVOCADO_TEST_BASEDIR')
+ self._baseDir = Path(__file__).parent
# workdir could also be avocado's own workdir in self.workdir.
# At present, I prefer to maintain my own temporary working
@@ -300,15 +304,14 @@ def setUp(self): # pylint: disable=arguments-differ
os.mkdir(prebuiltDir, mode=0o775)
bits_zip_file = os.path.join(prebuiltDir, 'bits-%d-%s.zip'
- %(self._bitsInternalVer,
- self._bitsCommitHash))
+ %(self.BITS_INTERNAL_VER,
+ self.BITS_COMMIT_HASH))
grub_tar_file = os.path.join(prebuiltDir,
'bits-%d-%s-grub.tar.gz'
- %(self._bitsInternalVer,
- self._bitsCommitHash))
+ %(self.BITS_INTERNAL_VER,
+ self.BITS_COMMIT_HASH))
- bitsLocalArtLoc = self.fetch_asset(self._bitsArtURL,
- asset_hash=self._bitsArtSHA1Hash)
+ bitsLocalArtLoc = self.ASSET_BITS.fetch()
self.logger.info("downloaded bits artifacts to %s", bitsLocalArtLoc)
# extract the bits artifact in the temp working directory
@@ -369,7 +372,7 @@ def test_acpi_smbios_bits(self):
"""The main test case implementation."""
iso_file = os.path.join(self._workDir,
- 'bits-%d.iso' %self._bitsInternalVer)
+ 'bits-%d.iso' %self.BITS_INTERNAL_VER)
self.assertTrue(os.access(iso_file, os.R_OK))
@@ -393,12 +396,6 @@ def test_acpi_smbios_bits(self):
self._vm.set_console()
self._vm.launch()
- self.logger.debug("Console output from bits VM follows ...")
- c_drainer = drainer.LineLogger(self._vm.console_socket.fileno(),
- logger=self.logger.getChild("console"),
- stop_check=(lambda :
- not self._vm.is_running()))
- c_drainer.start()
# biosbits has been configured to run all the specified test suites
# in batch mode and then automatically initiate a vm shutdown.
@@ -406,4 +403,8 @@ def test_acpi_smbios_bits(self):
# with the avocado test timeout.
self._vm.event_wait('SHUTDOWN', timeout=BITS_TIMEOUT)
self._vm.wait(timeout=None)
+ self.logger.debug("Checking console output ...")
self.parse_log()
+
+if __name__ == '__main__':
+ QemuBaseTest.main()