[0/6] tests/boot_linux_console: add extra boot acceptance tests

Message

Liam Merwick Jan. 27, 2020, 4:36 p.m. UTC

Add acceptance tests for the microvm machine class, PVH, and the
new qboot BIOS.

In the case of the test to boot an uncompressed kernel there didn't
seem to be any suitable uncompressed kernel on https://archives.fedoraproject.org/
(there is a vmlinux in kernel-debuginfo but that RPM is 575M and
caused timeouts when populating the Avocado cache when first run)
so I chose an RPM with kernels for Kata that is 14M.
(there was a discussion in [1] regarding testing PVH boot but it focussed
more around building a vmlinux binary during testing).

[ What prompted these patches was the discovery that a 'pc' guest booting an
uncompressed kernel (PVH) with a PCI netdev hangs (before we even get guest
console output) when bios-microvm.bin (qboot) is supplied via -bios
(no issue when using 'q35' or 'microvm' machine classes).

E.g. adding the following line to test_x86_64_pc_qboot_pvh() is enough to
trigger a guest hang during startup:
self.vm.add_args('-netdev', 'user,id=n1', '-device', 'virtio-net-pci,netdev=n1')

I bisected that issue to 176d2cda0dee [2] in 4.1 but haven't worked out yet
how/why the "die-id" changes impact the qboot/PVH combination
(the boot succeeds with any subset of those boot variables).

Is booting the 'pc' machine class with bios-microvm.bin something that QEMU
officially supports or is qboot intended for microvm only? ]

Each test added here adds about 1.5s to the overall runtime.
I have run them through the Travis QEMU CI [3] and those acceptance tests pass.

My modifications to test_x86_64_pc() in Patch1 will conflict with Wainer's
patch in [4] - I'll rebase on top of that once that series is pulled and
and apply feedback for this series, etc.

[1] https://patchew.org/QEMU/[e-mail]/
[2] 176d2cda0dee ("i386/cpu: Consolidate die-id validity in smp context")
[3] https://travis-ci.org/merwick/qemu/jobs/641505543
[4] https://github.com/wainersm/qemu/commit/8f705e98df90b436b0f4946331d441309c437f7b


Liam Merwick (6):
  tests/boot_linux_console: add microvm acceptance test
  tests/boot_linux_console: add BIOS acceptance test
  tests/boot_linux_console: fix extract_from_deb() comment
  travis.yml: install rpm2cpio for acceptance tests
  tests/boot_linux_console: add extract_from_rpm method
  tests/boot_linux_console: add PVH acceptance tests

 .travis.yml                            |  1 +
 tests/acceptance/boot_linux_console.py | 91 +++++++++++++++++++++++++++++++---
 2 files changed, 84 insertions(+), 8 deletions(-)

Comments

Wainer dos Santos Moschetta Jan. 30, 2020, 5:57 p.m. UTC | #1

Hi Liam,

On 1/27/20 2:36 PM, Liam Merwick wrote:
> Add acceptance tests for the microvm machine class, PVH, and the
> new qboot BIOS.
>
> In the case of the test to boot an uncompressed kernel there didn't
> seem to be any suitable uncompressed kernel on https://archives.fedoraproject.org/
> (there is a vmlinux in kernel-debuginfo but that RPM is 575M and
> caused timeouts when populating the Avocado cache when first run)
> so I chose an RPM with kernels for Kata that is 14M.
> (there was a discussion in [1] regarding testing PVH boot but it focussed
> more around building a vmlinux binary during testing).

Yeah, my proposal [1] for building the vmlinux with PVH at test time was 
gently rejected. I was going to send a v2 where I would use a built 
kernel from somewhere. I'm glad you send this series before, so I 
discard mine in favor of yours.

>
> [ What prompted these patches was the discovery that a 'pc' guest booting an
> uncompressed kernel (PVH) with a PCI netdev hangs (before we even get guest
> console output) when bios-microvm.bin (qboot) is supplied via -bios
> (no issue when using 'q35' or 'microvm' machine classes).
>
> E.g. adding the following line to test_x86_64_pc_qboot_pvh() is enough to
> trigger a guest hang during startup:
> self.vm.add_args('-netdev', 'user,id=n1', '-device', 'virtio-net-pci,netdev=n1')
>
> I bisected that issue to 176d2cda0dee [2] in 4.1 but haven't worked out yet
> how/why the "die-id" changes impact the qboot/PVH combination
> (the boot succeeds with any subset of those boot variables).
>
> Is booting the 'pc' machine class with bios-microvm.bin something that QEMU
> officially supports or is qboot intended for microvm only? ]
>
> Each test added here adds about 1.5s to the overall runtime.
> I have run them through the Travis QEMU CI [3] and those acceptance tests pass.

Thanks for using Travis CI to check it. This way I've some minutes 
saved, otherwise I would have to test it manually. :)

>
> My modifications to test_x86_64_pc() in Patch1 will conflict with Wainer's
> patch in [4] - I'll rebase on top of that once that series is pulled and
> and apply feedback for this series, etc.

No problem, I can rebase mine patches in case this get in first.

>
> [1] https://patchew.org/QEMU/[e-mail]/
> [2] 176d2cda0dee ("i386/cpu: Consolidate die-id validity in smp context")
> [3] https://travis-ci.org/merwick/qemu/jobs/641505543
> [4] https://github.com/wainersm/qemu/commit/8f705e98df90b436b0f4946331d441309c437f7b
>
>
> Liam Merwick (6):
>    tests/boot_linux_console: add microvm acceptance test
>    tests/boot_linux_console: add BIOS acceptance test
>    tests/boot_linux_console: fix extract_from_deb() comment
>    travis.yml: install rpm2cpio for acceptance tests
>    tests/boot_linux_console: add extract_from_rpm method
>    tests/boot_linux_console: add PVH acceptance tests
>
>   .travis.yml                            |  1 +
>   tests/acceptance/boot_linux_console.py | 91 +++++++++++++++++++++++++++++++---
>   2 files changed, 84 insertions(+), 8 deletions(-)
>