| Message ID | 20200219163537.22098-1-robert.foley@linaro.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | tests/vm: Add support for aarch64 VMs | expand |
Robert Foley <robert.foley@linaro.org> writes: > This is version 2 of the patch series to > add support for aarch64 VMs. > - Ubuntu 18.04 aarch64 VM > - CentOS 8 aarch64 VM > > V1: https://lists.gnu.org/archive/html/qemu-devel/2020-02/msg01180.html > > Changes in version 2 > - Most changes relate to the patch: "Add workaround to consume console". > - We changed this patch to make it cleaner. > - We added a ConsoleSocket, which slots in for the current console socket > with the difference being that we drain this socket in the background on a > callback basis. > - We also made the logging of the console to file optional > - Relocated the log file path and name. For example: > ~/.cache/qemu-vm/ubuntu.aarch64.install.log > - Made one fix for a hang issue we were seeing. > - The issue was a timing problem around a reboot where the > ubuntu.aarch64 script assumed the reboot guaranteed that > the next successful command would occur after the reboot. > - The fix is to simply make it more deterministic by shutting down the > VM and restarting it instead of issuing the reboot. > - Made a few changes to CentOS VM to update its dependencies properly. > - We made a few changes related to latin1 vs utf-8. > We found in some cases the latin1 is needed for chars coming out of the i > socket which do not have a utf-8 equivalent. > > Robert Foley (14): > tests/vm: use $(PYTHON) consistently > tests/vm: Debug mode shows ssh output. > tests/vm: increased max timeout for vm boot. > tests/vm: give wait_ssh() option to wait for root > tests/vm: Added gen_cloud_init_iso() to basevm.py For now I've pulled the first 5 patches into testing/next as they are obvious clean-ups. > tests/vm: Add workaround to consume console I still have concerns about this approach but I'm going to give it some more testing. However I ran into problems testing on my aarch64 box because of a missing gen-iso-image which makes me think we need to add some gating via configure for tools and libraries we need. > tests/vm: Add configuration to basevm.py > tests/vm: Added configuration file support > tests/vm: add --boot-console switch > tests/vm: Add ability to select QEMU from current build. > tests/vm: allow wait_ssh() to specify command > tests/vm: Added a new script for ubuntu.aarch64. > tests/vm: Added a new script for centos.aarch64. > tests/vm: change scripts to use self._config > > python/qemu/console_socket.py | 162 ++++++++++++++++ > python/qemu/machine.py | 12 +- > tests/vm/Makefile.include | 20 +- > tests/vm/aarch64vm.py | 100 ++++++++++ > tests/vm/basevm.py | 294 +++++++++++++++++++++++++----- > tests/vm/centos | 33 +--- > tests/vm/centos-8-aarch64.ks | 51 ++++++ > tests/vm/centos.aarch64 | 224 +++++++++++++++++++++++ > tests/vm/conf_example_aarch64.yml | 51 ++++++ > tests/vm/conf_example_x86.yml | 50 +++++ > tests/vm/fedora | 17 +- > tests/vm/freebsd | 16 +- > tests/vm/netbsd | 19 +- > tests/vm/openbsd | 17 +- > tests/vm/ubuntu.aarch64 | 117 ++++++++++++ > tests/vm/ubuntu.i386 | 37 +--- > 16 files changed, 1069 insertions(+), 151 deletions(-) > create mode 100644 python/qemu/console_socket.py > create mode 100644 tests/vm/aarch64vm.py > create mode 100644 tests/vm/centos-8-aarch64.ks > create mode 100755 tests/vm/centos.aarch64 > create mode 100644 tests/vm/conf_example_aarch64.yml > create mode 100644 tests/vm/conf_example_x86.yml > create mode 100755 tests/vm/ubuntu.aarch64
On Mon, 2 Mar 2020 at 11:38, Alex Bennée <alex.bennee@linaro.org> wrote: > > > Robert Foley <robert.foley@linaro.org> writes: > > > This is version 2 of the patch series to > > add support for aarch64 VMs. > > - Ubuntu 18.04 aarch64 VM > > - CentOS 8 aarch64 VM > > For now I've pulled the first 5 patches into testing/next as they are > obvious clean-ups. > > > tests/vm: Add workaround to consume console > > I still have concerns about this approach but I'm going to give it some > more testing. We are happy to make any adjustments here. Our first set of refactoring here was aimed at making it more pythonic. Is this where the concerns are? > However I ran into problems testing on my aarch64 box > because of a missing gen-iso-image which makes me think we need to add > some gating via configure for tools and libraries we need. Should we error out in configure if the tools and libraries needed to build the VMs are not there? Or maybe tolerate these dependencies not being there at configure time and provide an error later when someone tries to vm-build these VMs? Just curious which approach we should pursue here. Thanks & Regards, -Rob
Robert Foley <robert.foley@linaro.org> writes: > On Mon, 2 Mar 2020 at 11:38, Alex Bennée <alex.bennee@linaro.org> wrote: >> >> >> Robert Foley <robert.foley@linaro.org> writes: >> >> > This is version 2 of the patch series to >> > add support for aarch64 VMs. >> > - Ubuntu 18.04 aarch64 VM >> > - CentOS 8 aarch64 VM >> >> For now I've pulled the first 5 patches into testing/next as they are >> obvious clean-ups. >> >> > tests/vm: Add workaround to consume console >> >> I still have concerns about this approach but I'm going to give it some >> more testing. > > We are happy to make any adjustments here. Our first set of > refactoring here was > aimed at making it more pythonic. Is this where the concerns are? I'm just worried about the fragility of multiple steps in the chain of io we are snooping on. That said Phillipe made a reasonable point that other tools could be used - QMP for example would be the way to check the status of the network connection before we trigger ssh rather than the current busy-timeout approach. However that would result in more complexity so if what works is stable...*shrug* >> However I ran into problems testing on my aarch64 box >> because of a missing gen-iso-image which makes me think we need to add >> some gating via configure for tools and libraries we need. > > Should we error out in configure if the tools and libraries needed to build the > VMs are not there? > Or maybe tolerate these dependencies not being there at configure time and > provide an error later when someone tries to vm-build these VMs? We currently do both ;-) When we can detect at configure time and skip in make we do - see tests/docker/Makefile.include and the compiler tests for tests/tcg. However the acceptance tests current use runtime decorators in python to detect as we go - but that test framework prints a summary and doesn't exit -1 to the rest of make if it skips something. > Just curious which approach we should pursue here. Have a look at: Subject: [PATCH v1 00/10] testing/next updates (tweaks and re-greening) Date: Mon, 2 Mar 2020 18:18:57 +0000 Message-Id: <20200302181907.32110-1-alex.bennee@linaro.org> and see what you think. > > > Thanks & Regards, > -Rob
On Tue, 3 Mar 2020 at 05:24, Alex Bennée <alex.bennee@linaro.org> wrote: > > We are happy to make any adjustments here. Our first set of > > refactoring here was > > aimed at making it more pythonic. Is this where the concerns are? > > I'm just worried about the fragility of multiple steps in the chain of > io we are snooping on. That said Phillipe made a reasonable point that > other tools could be used - QMP for example would be the way to check > the status of the network connection before we trigger ssh rather than > the current busy-timeout approach. However that would result in more > complexity so if what works is stable...*shrug* OK. Makes sense. Thanks for explaining. > > >> However I ran into problems testing on my aarch64 box > >> because of a missing gen-iso-image which makes me think we need to add > >> some gating via configure for tools and libraries we need. > > > > Should we error out in configure if the tools and libraries needed to build the > > VMs are not there? > > Or maybe tolerate these dependencies not being there at configure time and > > provide an error later when someone tries to vm-build these VMs? > > We currently do both ;-) > > When we can detect at configure time and skip in make we do - see > tests/docker/Makefile.include and the compiler tests for tests/tcg. > However the acceptance tests current use runtime decorators in python to > detect as we go - but that test framework prints a summary and doesn't > exit -1 to the rest of make if it skips something. > > > Just curious which approach we should pursue here. > > Have a look at: > > Subject: [PATCH v1 00/10] testing/next updates (tweaks and re-greening) > Date: Mon, 2 Mar 2020 18:18:57 +0000 > Message-Id: <20200302181907.32110-1-alex.bennee@linaro.org> > > and see what you think. Thanks for the details. It seems we should be able to use a similar approach of detecting in configure and then skip and inform in make, for the aarch64 VM dependencies, as well as the yaml python dependency. Will look into adding these. Thanks & Regards, -Rob