[v1,00/14] tests/vm: Add support for aarch64 VMs

Message

Robert Foley Feb. 5, 2020, 9:29 p.m. UTC

This is version 1 of the patch series to 
add support for 2 aarch64 VMs.  
 - Ubuntu 18.04 aarch64 VM
 - CentOS 8 aarch64 VM
 
Changes in version 1
- Added environment variable QEMU_LOCAL=1 which means
  use the appropriate QEMU binary from the current build.
- Improved support of aarch64 VMs to be backwards compatible
  by default with older versions of QEMU.
- Fixed a locale issue with ubuntu.aarch64 VM.
  The issue here was that we were not waiting for the cloud-init to finish,
  prior to rebooting the VM.
- Workaround of issue where the test hangs as QEMU is waiting for
  the consumption of character device data.  Workaround adds a 
  thread to the test framework to consume these characters and dump them
  to a file. The test then consumes these characters from the file instead of
  from the character device as it did previously.
  One advantage of this change is that there is now a log file which 
  contains the console output for debug.
- Change creation of aarch64 flash files to be in temp directory.
- Added changes to convert existing scripts to using self._config for
  SSH_KEY_PUB, GUEST_USER, GUEST_PASS, ROOT_PASS.
- Created example config file for x86, renamed existing 
  example file to aarch64.
- Make gen_cloud_init_iso() common to basevm.
- Updated wait_ssh() to only extend time when we are under TCG.
- Changed wait_ssh() to wait for root or wait for guest user, not both.
- Removed SSH_KEY global as it is no longer needed.
- Clarified use of ssh_key_file and ssh_pub_key_file,
  added tmp to the names to clarify, and added a comment to explain use here.
- Cleaned up handling of the boot_console option.
- Moved tweak to Makefile.include to a separate patch.
- Removed setting of tcg,thread=multi in aarch64 VMs.
- Moved the validation of ssh keys to a common location.
- Reworked the handling of the config file yaml to
  simply set the values into the config dictionary in one step.

More details on the patch in general:
In order to add support for the two new aarch64 VMs, we
generalized and parameterized basevm.py.  
We added a new concept of a configuration, which is really just a 
set of parameters which define how to configure the VM.  
Some examples of parameters are "machine", "memory" and "cpu".
We preserved current default parameters.
Current configuration of pre-existing VMs is supported by default
without need to override default parameters.
 
For example, previously only the 'pc' machine was supported in basevm.py. 
The new aarch64 VMs will override machine to use virt. 
There are a few other examples where we needed to add parameters 
in order to add support for these aarch64 VMs.
In some other cases we added parameters that we thought would be useful
in general, for example username/password, ssh keys, 

In the case of the aarch64 VMs, they override certain parameters
by default.  However, it is also of value to be able to 
dynamically specify other values for these parameters.
Take the case where you create a new VM using vm-build,
but then want to test it using various hardware configurations
such as for example different NUMA topologies. 
Or maybe you want to use a different amount of memory or a different cpu type.

In order to support these use cases we added support
for a configuration .yml file, which allows the user to
specify certain values dynamically such as:
 - machine
 - cpu
 - memory size
 - other qemu args, which allow configuring alternate
   hardware topologies such as NUMA nodes.
 - username, password
 - ssh keys
 For an example of a .yml file, see the included config_example.yml
 
The main use case for using this config.yml file is for where we
are testing/debugging with qemu (vm-build), and need to configure
the VM differently.  However, there is another use case we have
developed, which is a project called lisa-qemu 
(https://github.com/rf972/lisa-qemu).  
This project is an integration between the LISA tool and QEMU.  
This project uses the VMs created by
QEMU's vm-build scripts for use in testing with LISA.  
This use case is similar to the vm-build case in that,
the VM gets created once, and we want to launch the VM with different
configurations (memory, cpu, etc.).
 
As part of developing the scripts for these VMs, we implemented
a few enhancements to help with testing.
For example, we added support for allowing debug mode to
show the ssh output.
We also added support for a new --boot-console option which
will show the boot console as the VM boots up to aid in 
debugging problems during VM boot.

Robert Foley (14):
  tests/vm: use $(PYTHON) consistently
  tests/vm: Debug mode shows ssh output.
  tests/vm: increased max timeout for vm boot.
  tests/vm: give wait_ssh() option to wait for root
  tests/vm: Added gen_cloud_init_iso() to basevm.py
  tests/vm: Add logging of console to file.
  tests/vm: Add configuration to basevm.py
  tests/vm: Added configuration file support
  tests/vm: add --boot-console switch
  tests/vm: Add ability to select QEMU from current build.
  tests/vm: allow wait_ssh() to specify command
  tests/vm: Added a new script for ubuntu.aarch64.
  tests/vm: Added a new script for centos.aarch64.
  tests/vm: change scripts to use self._config

 tests/vm/Makefile.include         |  16 +-
 tests/vm/aarch64vm.py             | 100 +++++++++
 tests/vm/basevm.py                | 324 +++++++++++++++++++++++++-----
 tests/vm/centos                   |  33 +--
 tests/vm/centos-8-aarch64.ks      |  51 +++++
 tests/vm/centos.aarch64           | 221 ++++++++++++++++++++
 tests/vm/conf_example_aarch64.yml |  51 +++++
 tests/vm/conf_example_x86.yml     |  50 +++++
 tests/vm/fedora                   |  17 +-
 tests/vm/freebsd                  |  16 +-
 tests/vm/netbsd                   |  19 +-
 tests/vm/openbsd                  |  17 +-
 tests/vm/socket_thread.py         |  73 +++++++
 tests/vm/ubuntu.aarch64           | 113 +++++++++++
 tests/vm/ubuntu.i386              |  37 +---
 15 files changed, 987 insertions(+), 151 deletions(-)
 create mode 100644 tests/vm/aarch64vm.py
 create mode 100644 tests/vm/centos-8-aarch64.ks
 create mode 100755 tests/vm/centos.aarch64
 create mode 100644 tests/vm/conf_example_aarch64.yml
 create mode 100644 tests/vm/conf_example_x86.yml
 create mode 100644 tests/vm/socket_thread.py
 create mode 100755 tests/vm/ubuntu.aarch64

Comments

Alex Bennée Feb. 7, 2020, 4:50 p.m. UTC | #1

Robert Foley <[e-mail]> writes:

> This is version 1 of the patch series to 
> add support for 2 aarch64 VMs.  
>  - Ubuntu 18.04 aarch64 VM
>  - CentOS 8 aarch64 VM

Testing under TCG with:

  make vm-build-ubuntu.aarch64 V=1 QEMU=aarch64-softmmu/qemu-system-aarch64

I'm seeing it hang:

  DEBUG:root:ssh_cmd: ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=60 -p 37431 -i /home/alex.bennee/lsrc/qemu.git/builds/all/vm-test-xxrw1r36.tmp/id_rsa -o SendEnv=https_proxy -o SendEnv=http_proxy -o SendEnv=ftp_proxy -o SendEnv=no_proxy root@127.0.0.1 sed -ie 's/"1"/"0"/g' /etc/apt/apt.conf.d/20auto-upgrades
  Warning: Permanently added '[127.0.0.1]:37431' (ECDSA) to the list of known hosts.
  bash: warning: setlocale: LC_ALL: cannot change locale (en_GB.UTF-8)
  Connection to 127.0.0.1 closed.
  DEBUG:root:ssh_cmd: ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=60 -p 37431 -i /home/alex.bennee/lsrc/qemu.git/builds/all/vm-test-xxrw1r36.tmp/id_rsa -o SendEnv=https_proxy -o SendEnv=http_proxy -o SendEnv=ftp_proxy -o SendEnv=no_proxy root@127.0.0.1 sync && reboot
  Warning: Permanently added '[127.0.0.1]:37431' (ECDSA) to the list of known hosts.
  bash: warning: setlocale: LC_ALL: cannot change locale (en_GB.UTF-8)
  Connection to 127.0.0.1 closed by remote host.
  Connection to 127.0.0.1 closed.
  DEBUG:root:ssh_cmd: ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=60 -p 37431 -i /home/alex.bennee/lsrc/qemu.git/builds/all/vm-test-xxrw1r36.tmp/id_rsa -o SendEnv=https_proxy -o SendEnv=http_proxy -o SendEnv=ftp_proxy -o SendEnv=no_proxy root@127.0.0.1 exit 0
  Warning: Permanently added '[127.0.0.1]:37431' (ECDSA) to the list of known hosts.
  bash: warning: setlocale: LC_ALL: cannot change locale (en_GB.UTF-8)
  Connection to 127.0.0.1 closed.
  DEBUG:root:ssh_cmd: ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=60 -p 37431 -i /home/alex.bennee/lsrc/qemu.git/builds/all/vm-test-xxrw1r36.tmp/id_rsa -o SendEnv=https_proxy -o SendEnv=http_proxy -o SendEnv=ftp_proxy -o SendEnv=no_proxy root@127.0.0.1 locale
  ssh_exchange_identification: Connection closed by remote host
  DEBUG:root:ssh_cmd: ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=60 -p 37431 -i /home/alex.bennee/lsrc/qemu.git/builds/all/vm-test-xxrw1r36.tmp/id_rsa -o SendEnv=https_proxy -o SendEnv=http_proxy -o SendEnv=ftp_proxy -o SendEnv=no_proxy root@127.0.0.1 sed -ie s/^#\ deb-src/deb-src/g /etc/apt/sources.list
  ssh_exchange_identification: Connection closed by remote host
  DEBUG:root:ssh_cmd: ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=60 -p 37431 -i /home/alex.bennee/lsrc/qemu.git/builds/all/vm-test-xxrw1r36.tmp/id_rsa -o SendEnv=https_proxy -o SendEnv=http_proxy -o SendEnv=ftp_proxy -o SendEnv=no_proxy root@127.0.0.1 apt-get update
  ssh_exchange_identification: Connection closed by remote host
  DEBUG:root:ssh_cmd: ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=60 -p 37431 -i /home/alex.bennee/lsrc/qemu.git/builds/all/vm-test-xxrw1r36.tmp/id_rsa -o SendEnv=https_proxy -o SendEnv=http_proxy -o SendEnv=ftp_proxy -o SendEnv=no_proxy root@127.0.0.1 apt-get build-dep -y qemu
  ssh_exchange_identification: Connection closed by remote host
  DEBUG:root:ssh_cmd: ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=60 -p 37431 -i /home/alex.bennee/lsrc/qemu.git/builds/all/vm-test-xxrw1r36.tmp/id_rsa -o SendEnv=https_proxy -o SendEnv=http_proxy -o SendEnv=ftp_proxy -o SendEnv=no_proxy root@127.0.0.1 apt-get install -y libfdt-dev flex bison
  ssh_exchange_identification: Connection closed by remote host
  DEBUG:root:ssh_cmd: ssh -t -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=60 -p 37431 -i /home/alex.bennee/lsrc/qemu.git/builds/all/vm-test-xxrw1r36.tmp/id_rsa -o SendEnv=https_proxy -o SendEnv=http_proxy -o SendEnv=ftp_proxy -o SendEnv=no_proxy root@127.0.0.1 poweroff
  ssh_exchange_identification: Connection closed by remote host