| Message ID | 20230428120405.3770496-1-nikos.nikoleris@arm.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | EFI and ACPI support for arm64 | expand |
On Fri, Apr 28, 2023 at 01:03:36PM +0100, Nikos Nikoleris wrote: > Hello, > > This series adds initial support for building arm64 tests as EFI > apps and running them under QEMU. Much like x86_64, we import external > dependencies from gnu-efi and adapt them to work with types and other > assumptions from kvm-unit-tests. In addition, this series adds support > for enumerating parts of the system using ACPI. > > The first set of patches, moves the existing ACPI code to the common > lib path. Then, it extends definitions and functions to allow for more > robust discovery of ACPI tables. We add support for setting up the PSCI > conduit, discovering the UART, timers, GIC and cpus via ACPI. The code > retains existing behavior and gives priority to discovery through DT > when one has been provided. > > In the second set of patches, we add support for getting the command > line from the EFI shell. This is a requirement for many of the > existing arm64 tests. > > In the third set of patches, we import code from gnu-efi, make minor > changes and add an alternative setup sequence from arm64 systems that > boot through EFI. Finally, we add support in the build system and a > run script which is used to run an EFI app. > > After this set of patches one can build arm64 EFI tests: > > $> ./configure --enable-efi > $> make > > And use the run script to run an EFI tests: > > $> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256" > > Or all tests: > > $> ./run_tests.sh > > There are a few items that this series does not address but they would > be useful to have: > - Support for booting the system from EL2. Currently, we assume that a > test starts EL1. This will be required to run EFI tests on sytems > that implement EL2. > - Support for reading environment variables and populating __envp. > - Support for discovering the PCI subsystem using ACPI. > - Get rid of other assumptions (e.g., vmalloc area) that don't hold on > real HW. > - Various fixes related to cache maintaince to better support turn the > MMU off. > - Switch to a new stack and avoid relying on the one provided by EFI. > > git branch: https://github.com/relokin/kvm-unit-tests/pull/new/target-efi-upstream-v5 > > v4: https://lore.kernel.org/kvmarm/20230213101759.2577077-1-nikos.nikoleris@arm.com/ > v3: https://lore.kernel.org/all/20220630100324.3153655-1-nikos.nikoleris@arm.com/ > v2: https://lore.kernel.org/kvm/20220506205605.359830-1-nikos.nikoleris@arm.com/ > > Changes in v5: > - Minor style changes (thanks Shaoqin). > - Avoid including lib/acpi.o to cflatobjs twice (thanks Drew). > - Increase NR_INITIAL_MEM_REGIONS to avoid overflows and add check when > we run out of space (thanks Shaoqin). > > Changes in v4: > - Removed patch that reworks cache maintenance when turning the MMU > off. This is not strictly required for EFI tests running with tcg and > will be addressed in a separate series by Alex. > - Fix compilation for arm (Alex). > - Convert ACPI tables to Linux style (Alex). > > Changes in v3: > - Addressed feedback from Drew, Alex and Ricardo. Many thanks for the reviews! > - Added support for discovering the GIC through ACPI > - Added a missing header file (<elf.h>) > - Added support for correctly parsing the outcome of tests (./run_tests) > Thanks, Nikos! I'd like to get an ack from either Paolo or Sean on the changes to ACPI, as they're shared with x86, and there are also some x86 code changes. Also, 1) It'd be nice if this worked with DT, too. We can use UEFI with DT when adding '-no-acpi' to the QEMU command line. setup_efi() needs to learn how to find the dtb and most the '#ifdef CONFIG_EFI's would need to change to a new CONFIG_ACPI guard. 2) The debug bp and ss tests fail with EFI, but not without, for me. 3) The timer test runs (and succeeds) when run with './arm/efi/run ./arm/timer.efi', but not when run with './run_tests.sh -g timer'. This is because UEFI takes up all the given timeout time (10s), and then the test times out. The hackyish fix below resolves it for me. I'll consider posting it as a real patch Thanks, drew diff --git a/scripts/arch-run.bash b/scripts/arch-run.bash index 51e4b97b27d1..72ce718b1170 100644 --- a/scripts/arch-run.bash +++ b/scripts/arch-run.bash @@ -94,7 +94,17 @@ run_qemu_status () timeout_cmd () { + local s + if [ "$TIMEOUT" ] && [ "$TIMEOUT" != "0" ]; then + if [ "$CONFIG_EFI" = 'y' ]; then + s=${TIMEOUT: -1} + if [ "$s" = 's' ]; then + TIMEOUT=${TIMEOUT:0:-1} + ((TIMEOUT += 10)) # Add 10 seconds for booting UEFI + TIMEOUT="${TIMEOUT}s" + fi + fi echo "timeout -k 1s --foreground $TIMEOUT" fi }
On Sat, Apr 29, 2023 at 06:18:25PM +0200, Andrew Jones wrote: > On Fri, Apr 28, 2023 at 01:03:36PM +0100, Nikos Nikoleris wrote: > > Hello, > > > > This series adds initial support for building arm64 tests as EFI > > apps and running them under QEMU. Much like x86_64, we import external > > dependencies from gnu-efi and adapt them to work with types and other > > assumptions from kvm-unit-tests. In addition, this series adds support > > for enumerating parts of the system using ACPI. > > > > The first set of patches, moves the existing ACPI code to the common > > lib path. Then, it extends definitions and functions to allow for more > > robust discovery of ACPI tables. We add support for setting up the PSCI > > conduit, discovering the UART, timers, GIC and cpus via ACPI. The code > > retains existing behavior and gives priority to discovery through DT > > when one has been provided. > > > > In the second set of patches, we add support for getting the command > > line from the EFI shell. This is a requirement for many of the > > existing arm64 tests. > > > > In the third set of patches, we import code from gnu-efi, make minor > > changes and add an alternative setup sequence from arm64 systems that > > boot through EFI. Finally, we add support in the build system and a > > run script which is used to run an EFI app. > > > > After this set of patches one can build arm64 EFI tests: > > > > $> ./configure --enable-efi > > $> make > > > > And use the run script to run an EFI tests: > > > > $> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256" > > > > Or all tests: > > > > $> ./run_tests.sh > > > > There are a few items that this series does not address but they would > > be useful to have: > > - Support for booting the system from EL2. Currently, we assume that a > > test starts EL1. This will be required to run EFI tests on sytems > > that implement EL2. > > - Support for reading environment variables and populating __envp. > > - Support for discovering the PCI subsystem using ACPI. > > - Get rid of other assumptions (e.g., vmalloc area) that don't hold on > > real HW. > > - Various fixes related to cache maintaince to better support turn the > > MMU off. > > - Switch to a new stack and avoid relying on the one provided by EFI. > > > > git branch: https://github.com/relokin/kvm-unit-tests/pull/new/target-efi-upstream-v5 > > > > v4: https://lore.kernel.org/kvmarm/20230213101759.2577077-1-nikos.nikoleris@arm.com/ > > v3: https://lore.kernel.org/all/20220630100324.3153655-1-nikos.nikoleris@arm.com/ > > v2: https://lore.kernel.org/kvm/20220506205605.359830-1-nikos.nikoleris@arm.com/ > > > > Changes in v5: > > - Minor style changes (thanks Shaoqin). > > - Avoid including lib/acpi.o to cflatobjs twice (thanks Drew). > > - Increase NR_INITIAL_MEM_REGIONS to avoid overflows and add check when > > we run out of space (thanks Shaoqin). > > > > Changes in v4: > > - Removed patch that reworks cache maintenance when turning the MMU > > off. This is not strictly required for EFI tests running with tcg and > > will be addressed in a separate series by Alex. > > - Fix compilation for arm (Alex). > > - Convert ACPI tables to Linux style (Alex). > > > > Changes in v3: > > - Addressed feedback from Drew, Alex and Ricardo. Many thanks for the reviews! > > - Added support for discovering the GIC through ACPI > > - Added a missing header file (<elf.h>) > > - Added support for correctly parsing the outcome of tests (./run_tests) > > > > Thanks, Nikos! > > I'd like to get an ack from either Paolo or Sean on the changes to ACPI, > as they're shared with x86, and there are also some x86 code changes. Actually, there are two build pipeline failures with the new ACPI code. Please take a look at https://gitlab.com/jones-drew/kvm-unit-tests/-/pipelines/852864569 Thanks, drew > > Also, > > 1) It'd be nice if this worked with DT, too. We can use UEFI with DT > when adding '-no-acpi' to the QEMU command line. setup_efi() needs > to learn how to find the dtb and most the '#ifdef CONFIG_EFI's > would need to change to a new CONFIG_ACPI guard. > > 2) The debug bp and ss tests fail with EFI, but not without, for me. > > 3) The timer test runs (and succeeds) when run with > './arm/efi/run ./arm/timer.efi', but not when run with > './run_tests.sh -g timer'. This is because UEFI takes > up all the given timeout time (10s), and then the test times out. > The hackyish fix below resolves it for me. I'll consider posting it > as a real patch > > Thanks, > drew > > diff --git a/scripts/arch-run.bash b/scripts/arch-run.bash > index 51e4b97b27d1..72ce718b1170 100644 > --- a/scripts/arch-run.bash > +++ b/scripts/arch-run.bash > @@ -94,7 +94,17 @@ run_qemu_status () > > timeout_cmd () > { > + local s > + > if [ "$TIMEOUT" ] && [ "$TIMEOUT" != "0" ]; then > + if [ "$CONFIG_EFI" = 'y' ]; then > + s=${TIMEOUT: -1} > + if [ "$s" = 's' ]; then > + TIMEOUT=${TIMEOUT:0:-1} > + ((TIMEOUT += 10)) # Add 10 seconds for booting UEFI > + TIMEOUT="${TIMEOUT}s" > + fi > + fi > echo "timeout -k 1s --foreground $TIMEOUT" > fi > }
On 29/04/2023 17:21, Andrew Jones wrote: > On Sat, Apr 29, 2023 at 06:18:25PM +0200, Andrew Jones wrote: >> On Fri, Apr 28, 2023 at 01:03:36PM +0100, Nikos Nikoleris wrote: >>> Hello, >>> >>> This series adds initial support for building arm64 tests as EFI >>> apps and running them under QEMU. Much like x86_64, we import external >>> dependencies from gnu-efi and adapt them to work with types and other >>> assumptions from kvm-unit-tests. In addition, this series adds support >>> for enumerating parts of the system using ACPI. >>> >>> The first set of patches, moves the existing ACPI code to the common >>> lib path. Then, it extends definitions and functions to allow for more >>> robust discovery of ACPI tables. We add support for setting up the PSCI >>> conduit, discovering the UART, timers, GIC and cpus via ACPI. The code >>> retains existing behavior and gives priority to discovery through DT >>> when one has been provided. >>> >>> In the second set of patches, we add support for getting the command >>> line from the EFI shell. This is a requirement for many of the >>> existing arm64 tests. >>> >>> In the third set of patches, we import code from gnu-efi, make minor >>> changes and add an alternative setup sequence from arm64 systems that >>> boot through EFI. Finally, we add support in the build system and a >>> run script which is used to run an EFI app. >>> >>> After this set of patches one can build arm64 EFI tests: >>> >>> $> ./configure --enable-efi >>> $> make >>> >>> And use the run script to run an EFI tests: >>> >>> $> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256" >>> >>> Or all tests: >>> >>> $> ./run_tests.sh >>> >>> There are a few items that this series does not address but they would >>> be useful to have: >>> - Support for booting the system from EL2. Currently, we assume that a >>> test starts EL1. This will be required to run EFI tests on sytems >>> that implement EL2. >>> - Support for reading environment variables and populating __envp. >>> - Support for discovering the PCI subsystem using ACPI. >>> - Get rid of other assumptions (e.g., vmalloc area) that don't hold on >>> real HW. >>> - Various fixes related to cache maintaince to better support turn the >>> MMU off. >>> - Switch to a new stack and avoid relying on the one provided by EFI. >>> >>> git branch: https://github.com/relokin/kvm-unit-tests/pull/new/target-efi-upstream-v5 >>> >>> v4: https://lore.kernel.org/kvmarm/20230213101759.2577077-1-nikos.nikoleris@arm.com/ >>> v3: https://lore.kernel.org/all/20220630100324.3153655-1-nikos.nikoleris@arm.com/ >>> v2: https://lore.kernel.org/kvm/20220506205605.359830-1-nikos.nikoleris@arm.com/ >>> >>> Changes in v5: >>> - Minor style changes (thanks Shaoqin). >>> - Avoid including lib/acpi.o to cflatobjs twice (thanks Drew). >>> - Increase NR_INITIAL_MEM_REGIONS to avoid overflows and add check when >>> we run out of space (thanks Shaoqin). >>> >>> Changes in v4: >>> - Removed patch that reworks cache maintenance when turning the MMU >>> off. This is not strictly required for EFI tests running with tcg and >>> will be addressed in a separate series by Alex. >>> - Fix compilation for arm (Alex). >>> - Convert ACPI tables to Linux style (Alex). >>> >>> Changes in v3: >>> - Addressed feedback from Drew, Alex and Ricardo. Many thanks for the reviews! >>> - Added support for discovering the GIC through ACPI >>> - Added a missing header file (<elf.h>) >>> - Added support for correctly parsing the outcome of tests (./run_tests) >>> >> >> Thanks, Nikos! >> >> I'd like to get an ack from either Paolo or Sean on the changes to ACPI, >> as they're shared with x86, and there are also some x86 code changes. > > Actually, there are two build pipeline failures with the new ACPI code. > Please take a look at > > https://gitlab.com/jones-drew/kvm-unit-tests/-/pipelines/852864569 > Thanks for reviewing the series! I think this fixes the compilation issues: diff --git a/lib/acpi.h b/lib/acpi.h index 202d832e..c330c877 100644 --- a/lib/acpi.h +++ b/lib/acpi.h @@ -292,7 +292,8 @@ struct acpi_table_gtdt { u32 platform_timer_offset; }; -#pragma pack(0) +/* Reset to default packing */ +#pragma pack() void set_efi_rsdp(struct acpi_table_rsdp *rsdp); void *find_acpi_table_addr(u32 sig); diff --git a/lib/acpi.c b/lib/acpi.c index 760cd8b2..0440cddb 100644 --- a/lib/acpi.c +++ b/lib/acpi.c @@ -70,7 +70,7 @@ void *find_acpi_table_addr(u32 sig) return rsdt; if (rsdp->revision >= 2) { - xsdt = (void *)rsdp->xsdt_physical_address; + xsdt = (void *)(ulong) rsdp->xsdt_physical_address; if (xsdt && xsdt->signature != XSDT_SIGNATURE) xsdt = NULL; } > Thanks, > drew > >> >> Also, >> >> 1) It'd be nice if this worked with DT, too. We can use UEFI with DT >> when adding '-no-acpi' to the QEMU command line. setup_efi() needs >> to learn how to find the dtb and most the '#ifdef CONFIG_EFI's >> would need to change to a new CONFIG_ACPI guard. >> I had a quick look at it at some point and it didn't look straightforward but I'll check again. >> 2) The debug bp and ss tests fail with EFI, but not without, for me. >> I think, I've found the problem, the patch below fixes it for me. diff --git a/arm/debug.c b/arm/debug.c index b3e9749c..126fa267 100644 --- a/arm/debug.c +++ b/arm/debug.c @@ -292,11 +292,14 @@ static noinline void test_hw_bp(bool migrate) hw_bp_idx = 0; /* Trap on up to 16 debug exception unmask instructions. */ - asm volatile("hw_bp0:\n" - "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr daifclr, #8\n" - "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr daifclr, #8\n" - "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr daifclr, #8\n" - "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr daifclr, #8\n"); + asm volatile( + ".globl hw_bp0\n" + "hw_bp0:\n" + "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr daifclr, #8\n" + "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr daifclr, #8\n" + "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr daifclr, #8\n" + "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr daifclr, #8\n" + ); for (i = 0, addr = (uint64_t)&hw_bp0; i < num_bp; i++, addr += 4) report(hw_bp_addr[i] == addr, "hw breakpoint: %d", i); @@ -367,11 +370,14 @@ static noinline void test_ss(bool migrate) asm volatile("msr daifclr, #8"); - asm volatile("ss_start:\n" + asm volatile( + ".globl ss_start\n" + "ss_start:\n" "mrs x0, esr_el1\n" "add x0, x0, #1\n" "msr daifset, #8\n" - : : : "x0"); + : : : "x0" + ); report(ss_addr[0] == (uint64_t)&ss_start, "single step"); } >> 3) The timer test runs (and succeeds) when run with >> './arm/efi/run ./arm/timer.efi', but not when run with >> './run_tests.sh -g timer'. This is because UEFI takes >> up all the given timeout time (10s), and then the test times out. >> The hackyish fix below resolves it for me. I'll consider posting it >> as a real patch >> I see. I didn't hit the timeout on my test machine but I tried on a slower machine and I did. New branch with the fixups here: https://github.com/relokin/kvm-unit-tests/pull/new/target-efi-upstream-v5 Many thanks, Nikos >> Thanks, >> drew >> >> diff --git a/scripts/arch-run.bash b/scripts/arch-run.bash >> index 51e4b97b27d1..72ce718b1170 100644 >> --- a/scripts/arch-run.bash >> +++ b/scripts/arch-run.bash >> @@ -94,7 +94,17 @@ run_qemu_status () >> >> timeout_cmd () >> { >> + local s >> + >> if [ "$TIMEOUT" ] && [ "$TIMEOUT" != "0" ]; then >> + if [ "$CONFIG_EFI" = 'y' ]; then >> + s=${TIMEOUT: -1} >> + if [ "$s" = 's' ]; then >> + TIMEOUT=${TIMEOUT:0:-1} >> + ((TIMEOUT += 10)) # Add 10 seconds for booting UEFI >> + TIMEOUT="${TIMEOUT}s" >> + fi >> + fi >> echo "timeout -k 1s --foreground $TIMEOUT" >> fi >> }
On Sun, Apr 30, 2023 at 01:02:42AM +0100, Nikos Nikoleris wrote: > On 29/04/2023 17:21, Andrew Jones wrote: > > On Sat, Apr 29, 2023 at 06:18:25PM +0200, Andrew Jones wrote: > > > On Fri, Apr 28, 2023 at 01:03:36PM +0100, Nikos Nikoleris wrote: > > > > Hello, > > > > > > > > This series adds initial support for building arm64 tests as EFI > > > > apps and running them under QEMU. Much like x86_64, we import external > > > > dependencies from gnu-efi and adapt them to work with types and other > > > > assumptions from kvm-unit-tests. In addition, this series adds support > > > > for enumerating parts of the system using ACPI. > > > > > > > > The first set of patches, moves the existing ACPI code to the common > > > > lib path. Then, it extends definitions and functions to allow for more > > > > robust discovery of ACPI tables. We add support for setting up the PSCI > > > > conduit, discovering the UART, timers, GIC and cpus via ACPI. The code > > > > retains existing behavior and gives priority to discovery through DT > > > > when one has been provided. > > > > > > > > In the second set of patches, we add support for getting the command > > > > line from the EFI shell. This is a requirement for many of the > > > > existing arm64 tests. > > > > > > > > In the third set of patches, we import code from gnu-efi, make minor > > > > changes and add an alternative setup sequence from arm64 systems that > > > > boot through EFI. Finally, we add support in the build system and a > > > > run script which is used to run an EFI app. > > > > > > > > After this set of patches one can build arm64 EFI tests: > > > > > > > > $> ./configure --enable-efi > > > > $> make > > > > > > > > And use the run script to run an EFI tests: > > > > > > > > $> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256" > > > > > > > > Or all tests: > > > > > > > > $> ./run_tests.sh > > > > > > > > There are a few items that this series does not address but they would > > > > be useful to have: > > > > - Support for booting the system from EL2. Currently, we assume that a > > > > test starts EL1. This will be required to run EFI tests on sytems > > > > that implement EL2. > > > > - Support for reading environment variables and populating __envp. > > > > - Support for discovering the PCI subsystem using ACPI. > > > > - Get rid of other assumptions (e.g., vmalloc area) that don't hold on > > > > real HW. > > > > - Various fixes related to cache maintaince to better support turn the > > > > MMU off. > > > > - Switch to a new stack and avoid relying on the one provided by EFI. > > > > > > > > git branch: https://github.com/relokin/kvm-unit-tests/pull/new/target-efi-upstream-v5 > > > > > > > > v4: https://lore.kernel.org/kvmarm/20230213101759.2577077-1-nikos.nikoleris@arm.com/ > > > > v3: https://lore.kernel.org/all/20220630100324.3153655-1-nikos.nikoleris@arm.com/ > > > > v2: https://lore.kernel.org/kvm/20220506205605.359830-1-nikos.nikoleris@arm.com/ > > > > > > > > Changes in v5: > > > > - Minor style changes (thanks Shaoqin). > > > > - Avoid including lib/acpi.o to cflatobjs twice (thanks Drew). > > > > - Increase NR_INITIAL_MEM_REGIONS to avoid overflows and add check when > > > > we run out of space (thanks Shaoqin). > > > > > > > > Changes in v4: > > > > - Removed patch that reworks cache maintenance when turning the MMU > > > > off. This is not strictly required for EFI tests running with tcg and > > > > will be addressed in a separate series by Alex. > > > > - Fix compilation for arm (Alex). > > > > - Convert ACPI tables to Linux style (Alex). > > > > > > > > Changes in v3: > > > > - Addressed feedback from Drew, Alex and Ricardo. Many thanks for the reviews! > > > > - Added support for discovering the GIC through ACPI > > > > - Added a missing header file (<elf.h>) > > > > - Added support for correctly parsing the outcome of tests (./run_tests) > > > > > > > > > > Thanks, Nikos! > > > > > > I'd like to get an ack from either Paolo or Sean on the changes to ACPI, > > > as they're shared with x86, and there are also some x86 code changes. > > > > Actually, there are two build pipeline failures with the new ACPI code. > > Please take a look at > > > > https://gitlab.com/jones-drew/kvm-unit-tests/-/pipelines/852864569 > > > > Thanks for reviewing the series! > > I think this fixes the compilation issues: > > diff --git a/lib/acpi.h b/lib/acpi.h > index 202d832e..c330c877 100644 > --- a/lib/acpi.h > +++ b/lib/acpi.h > @@ -292,7 +292,8 @@ struct acpi_table_gtdt { > u32 platform_timer_offset; > }; > > -#pragma pack(0) > +/* Reset to default packing */ > +#pragma pack() > > void set_efi_rsdp(struct acpi_table_rsdp *rsdp); > void *find_acpi_table_addr(u32 sig); > diff --git a/lib/acpi.c b/lib/acpi.c > index 760cd8b2..0440cddb 100644 > --- a/lib/acpi.c > +++ b/lib/acpi.c > @@ -70,7 +70,7 @@ void *find_acpi_table_addr(u32 sig) > return rsdt; > > if (rsdp->revision >= 2) { > - xsdt = (void *)rsdp->xsdt_physical_address; > + xsdt = (void *)(ulong) rsdp->xsdt_physical_address; > if (xsdt && xsdt->signature != XSDT_SIGNATURE) > xsdt = NULL; > } > > > Thanks, > > drew > > > > > > > > Also, > > > > > > 1) It'd be nice if this worked with DT, too. We can use UEFI with DT > > > when adding '-no-acpi' to the QEMU command line. setup_efi() needs > > > to learn how to find the dtb and most the '#ifdef CONFIG_EFI's > > > would need to change to a new CONFIG_ACPI guard. > > > > > I had a quick look at it at some point and it didn't look straightforward > but I'll check again. > > > > 2) The debug bp and ss tests fail with EFI, but not without, for me. > > > > > I think, I've found the problem, the patch below fixes it for me. > > diff --git a/arm/debug.c b/arm/debug.c > index b3e9749c..126fa267 100644 > --- a/arm/debug.c > +++ b/arm/debug.c > @@ -292,11 +292,14 @@ static noinline void test_hw_bp(bool migrate) > hw_bp_idx = 0; > > /* Trap on up to 16 debug exception unmask instructions. */ > - asm volatile("hw_bp0:\n" > - "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr > daifclr, #8\n" > - "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr > daifclr, #8\n" > - "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr > daifclr, #8\n" > - "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr > daifclr, #8\n"); > + asm volatile( > + ".globl hw_bp0\n" > + "hw_bp0:\n" > + "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; > msr daifclr, #8\n" > + "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; > msr daifclr, #8\n" > + "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; > msr daifclr, #8\n" > + "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; > msr daifclr, #8\n" > + ); > > for (i = 0, addr = (uint64_t)&hw_bp0; i < num_bp; i++, addr += 4) > report(hw_bp_addr[i] == addr, "hw breakpoint: %d", i); > @@ -367,11 +370,14 @@ static noinline void test_ss(bool migrate) > > asm volatile("msr daifclr, #8"); > > - asm volatile("ss_start:\n" > + asm volatile( > + ".globl ss_start\n" > + "ss_start:\n" > "mrs x0, esr_el1\n" > "add x0, x0, #1\n" > "msr daifset, #8\n" > - : : : "x0"); > + : : : "x0" > + ); > > report(ss_addr[0] == (uint64_t)&ss_start, "single step"); > } > > > > 3) The timer test runs (and succeeds) when run with > > > './arm/efi/run ./arm/timer.efi', but not when run with > > > './run_tests.sh -g timer'. This is because UEFI takes > > > up all the given timeout time (10s), and then the test times out. > > > The hackyish fix below resolves it for me. I'll consider posting it > > > as a real patch > > > > > I see. I didn't hit the timeout on my test machine but I tried on a slower > machine and I did. > > New branch with the fixups here: > > https://github.com/relokin/kvm-unit-tests/pull/new/target-efi-upstream-v5 Thanks for the quick fixes. Can you update your tree and make an MR? I no longer use github.com/rhdrjones/kvm-unit-tests. I use https://gitlab.com/jones-drew/kvm-unit-tests.git Thanks, drew > > Many thanks, > > Nikos > > > > Thanks, > > > drew > > > > > > diff --git a/scripts/arch-run.bash b/scripts/arch-run.bash > > > index 51e4b97b27d1..72ce718b1170 100644 > > > --- a/scripts/arch-run.bash > > > +++ b/scripts/arch-run.bash > > > @@ -94,7 +94,17 @@ run_qemu_status () > > > timeout_cmd () > > > { > > > + local s > > > + > > > if [ "$TIMEOUT" ] && [ "$TIMEOUT" != "0" ]; then > > > + if [ "$CONFIG_EFI" = 'y' ]; then > > > + s=${TIMEOUT: -1} > > > + if [ "$s" = 's' ]; then > > > + TIMEOUT=${TIMEOUT:0:-1} > > > + ((TIMEOUT += 10)) # Add 10 seconds for booting UEFI > > > + TIMEOUT="${TIMEOUT}s" > > > + fi > > > + fi > > > echo "timeout -k 1s --foreground $TIMEOUT" > > > fi > > > }
Hi Nikos,
When I test this series by simply run ./run_tests.sh, some migration
test will hang up forever, like the `its-migration` test.
After have a quick look, when do the migration test, the guest can't be
migrated due to:
{"return": {"blocked-reasons": ["The qcow format used by node
'#block389' does not support live migration", "The vvfat (rw) format
used by node '#block085' does not support live migration"]}}
Although the guest will be timeout, but the script itself will not be
timeout. The infinite loop happened at here:
// script/arch-run.bash
151 # Wait for the migration to complete
152 migstatus=`qmp ${qmp1} '"query-migrate"' | grep return`
153 while ! grep -q '"completed"' <<<"$migstatus" ; do
154 sleep 1
155 migstatus=`qmp ${qmp1} '"query-migrate"' | grep
return`
156 if grep -q '"failed"' <<<"$migstatus" ; then
157 echo "ERROR: Migration failed." >&2
158 qmp ${qmp1} '"quit"'> ${qmpout1} 2>/dev/null
159 qmp ${qmp2} '"quit"'> ${qmpout2} 2>/dev/null
160 return 2
161 fi
162 done
Since the "query-migrate" here will never get "completed" or "failed",
so it will never exit.
Have you ever meet this problem?
Thanks,
Shaoqin
On 4/28/23 20:03, Nikos Nikoleris wrote:
> Hello,
>
> This series adds initial support for building arm64 tests as EFI
> apps and running them under QEMU. Much like x86_64, we import external
> dependencies from gnu-efi and adapt them to work with types and other
> assumptions from kvm-unit-tests. In addition, this series adds support
> for enumerating parts of the system using ACPI.
>
> The first set of patches, moves the existing ACPI code to the common
> lib path. Then, it extends definitions and functions to allow for more
> robust discovery of ACPI tables. We add support for setting up the PSCI
> conduit, discovering the UART, timers, GIC and cpus via ACPI. The code
> retains existing behavior and gives priority to discovery through DT
> when one has been provided.
>
> In the second set of patches, we add support for getting the command
> line from the EFI shell. This is a requirement for many of the
> existing arm64 tests.
>
> In the third set of patches, we import code from gnu-efi, make minor
> changes and add an alternative setup sequence from arm64 systems that
> boot through EFI. Finally, we add support in the build system and a
> run script which is used to run an EFI app.
>
> After this set of patches one can build arm64 EFI tests:
>
> $> ./configure --enable-efi
> $> make
>
> And use the run script to run an EFI tests:
>
> $> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256"
>
> Or all tests:
>
> $> ./run_tests.sh
>
> There are a few items that this series does not address but they would
> be useful to have:
> - Support for booting the system from EL2. Currently, we assume that a
> test starts EL1. This will be required to run EFI tests on sytems
> that implement EL2.
> - Support for reading environment variables and populating __envp.
> - Support for discovering the PCI subsystem using ACPI.
> - Get rid of other assumptions (e.g., vmalloc area) that don't hold on
> real HW.
> - Various fixes related to cache maintaince to better support turn the
> MMU off.
> - Switch to a new stack and avoid relying on the one provided by EFI.
>
> git branch: https://github.com/relokin/kvm-unit-tests/pull/new/target-efi-upstream-v5
>
> v4: https://lore.kernel.org/kvmarm/20230213101759.2577077-1-nikos.nikoleris@arm.com/
> v3: https://lore.kernel.org/all/20220630100324.3153655-1-nikos.nikoleris@arm.com/
> v2: https://lore.kernel.org/kvm/20220506205605.359830-1-nikos.nikoleris@arm.com/
>
> Changes in v5:
> - Minor style changes (thanks Shaoqin).
> - Avoid including lib/acpi.o to cflatobjs twice (thanks Drew).
> - Increase NR_INITIAL_MEM_REGIONS to avoid overflows and add check when
> we run out of space (thanks Shaoqin).
>
> Changes in v4:
> - Removed patch that reworks cache maintenance when turning the MMU
> off. This is not strictly required for EFI tests running with tcg and
> will be addressed in a separate series by Alex.
> - Fix compilation for arm (Alex).
> - Convert ACPI tables to Linux style (Alex).
>
> Changes in v3:
> - Addressed feedback from Drew, Alex and Ricardo. Many thanks for the reviews!
> - Added support for discovering the GIC through ACPI
> - Added a missing header file (<elf.h>)
> - Added support for correctly parsing the outcome of tests (./run_tests)
>
> Thanks,
>
> Nikos
>
> Alexandru Elisei (2):
> lib/acpi: Convert table names to Linux style
> lib: arm: Print test exit status
>
> Andrew Jones (2):
> arm/arm64: Rename etext to _etext
> arm64: Add a new type of memory type flag MR_F_RESERVED
>
> Nikos Nikoleris (25):
> lib: Move acpi header and implementation to lib
> x86: Move x86_64-specific EFI CFLAGS to x86_64 Makefile
> lib: Apply Lindent to acpi.{c,h}
> lib: Fix style for acpi.{c,h}
> x86: Avoid references to fields of ACPI tables
> lib/acpi: Ensure all struct definition for ACPI tables are packed
> lib/acpi: Add support for the XSDT table
> lib/acpi: Extend the definition of the FADT table
> devicetree: Check that fdt is not NULL in dt_available()
> arm64: Add support for setting up the PSCI conduit through ACPI
> arm64: Add support for discovering the UART through ACPI
> arm64: Add support for timer initialization through ACPI
> arm64: Add support for cpu initialization through ACPI
> arm64: Add support for gic initialization through ACPI
> lib/printf: Support for precision modifier in printf
> lib/printf: Add support for printing wide strings
> lib/efi: Add support for getting the cmdline
> lib: Avoid ms_abi for calls related to EFI on arm64
> arm64: Add a setup sequence for systems that boot through EFI
> arm64: Copy code from GNU-EFI
> arm64: Change GNU-EFI imported code to use defined types
> arm64: Use code from the gnu-efi when booting with EFI
> lib: Avoid external dependency in libelf
> arm64: Add support for efi in Makefile
> arm64: Add an efi/run script
>
> scripts/runtime.bash | 13 +-
> arm/efi/run | 61 ++++++++
> arm/run | 14 +-
> configure | 17 +-
> Makefile | 4 -
> arm/Makefile.arm | 6 +
> arm/Makefile.arm64 | 22 ++-
> arm/Makefile.common | 47 ++++--
> x86/Makefile.common | 2 +-
> x86/Makefile.x86_64 | 4 +
> lib/linux/efi.h | 25 +++
> lib/arm/asm/setup.h | 9 ++
> lib/arm/asm/timer.h | 2 +
> lib/x86/asm/setup.h | 2 +-
> lib/acpi.h | 301 ++++++++++++++++++++++++++++++++++++
> lib/argv.h | 1 +
> lib/elf.h | 57 +++++++
> lib/libcflat.h | 1 +
> lib/x86/acpi.h | 112 --------------
> lib/acpi.c | 129 ++++++++++++++++
> lib/argv.c | 2 +-
> lib/arm/gic.c | 139 ++++++++++++++++-
> lib/arm/io.c | 41 ++++-
> lib/arm/mmu.c | 4 +
> lib/arm/psci.c | 37 ++++-
> lib/arm/setup.c | 269 ++++++++++++++++++++++++++------
> lib/arm/timer.c | 92 +++++++++++
> lib/devicetree.c | 2 +-
> lib/efi.c | 102 ++++++++++++
> lib/printf.c | 194 +++++++++++++++++++++--
> lib/x86/acpi.c | 82 ----------
> lib/x86/setup.c | 2 +-
> arm/efi/elf_aarch64_efi.lds | 63 ++++++++
> arm/flat.lds | 2 +-
> arm/cstart.S | 1 +
> arm/cstart64.S | 7 +
> arm/efi/crt0-efi-aarch64.S | 141 +++++++++++++++++
> arm/dummy.c | 12 ++
> arm/efi/reloc_aarch64.c | 94 +++++++++++
> arm/micro-bench.c | 4 +-
> arm/timer.c | 10 +-
> x86/s3.c | 21 +--
> x86/vmexit.c | 4 +-
> 43 files changed, 1831 insertions(+), 323 deletions(-)
> create mode 100755 arm/efi/run
> create mode 100644 lib/acpi.h
> create mode 100644 lib/elf.h
> delete mode 100644 lib/x86/acpi.h
> create mode 100644 lib/acpi.c
> create mode 100644 lib/arm/timer.c
> delete mode 100644 lib/x86/acpi.c
> create mode 100644 arm/efi/elf_aarch64_efi.lds
> create mode 100644 arm/efi/crt0-efi-aarch64.S
> create mode 100644 arm/dummy.c
> create mode 100644 arm/efi/reloc_aarch64.c
>
On 01/05/2023 12:21, Andrew Jones wrote: > On Sun, Apr 30, 2023 at 01:02:42AM +0100, Nikos Nikoleris wrote: >> On 29/04/2023 17:21, Andrew Jones wrote: >>> On Sat, Apr 29, 2023 at 06:18:25PM +0200, Andrew Jones wrote: >>>> On Fri, Apr 28, 2023 at 01:03:36PM +0100, Nikos Nikoleris wrote: >>>>> Hello, >>>>> >>>>> This series adds initial support for building arm64 tests as EFI >>>>> apps and running them under QEMU. Much like x86_64, we import external >>>>> dependencies from gnu-efi and adapt them to work with types and other >>>>> assumptions from kvm-unit-tests. In addition, this series adds support >>>>> for enumerating parts of the system using ACPI. >>>>> >>>>> The first set of patches, moves the existing ACPI code to the common >>>>> lib path. Then, it extends definitions and functions to allow for more >>>>> robust discovery of ACPI tables. We add support for setting up the PSCI >>>>> conduit, discovering the UART, timers, GIC and cpus via ACPI. The code >>>>> retains existing behavior and gives priority to discovery through DT >>>>> when one has been provided. >>>>> >>>>> In the second set of patches, we add support for getting the command >>>>> line from the EFI shell. This is a requirement for many of the >>>>> existing arm64 tests. >>>>> >>>>> In the third set of patches, we import code from gnu-efi, make minor >>>>> changes and add an alternative setup sequence from arm64 systems that >>>>> boot through EFI. Finally, we add support in the build system and a >>>>> run script which is used to run an EFI app. >>>>> >>>>> After this set of patches one can build arm64 EFI tests: >>>>> >>>>> $> ./configure --enable-efi >>>>> $> make >>>>> >>>>> And use the run script to run an EFI tests: >>>>> >>>>> $> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256" >>>>> >>>>> Or all tests: >>>>> >>>>> $> ./run_tests.sh >>>>> >>>>> There are a few items that this series does not address but they would >>>>> be useful to have: >>>>> - Support for booting the system from EL2. Currently, we assume that a >>>>> test starts EL1. This will be required to run EFI tests on sytems >>>>> that implement EL2. >>>>> - Support for reading environment variables and populating __envp. >>>>> - Support for discovering the PCI subsystem using ACPI. >>>>> - Get rid of other assumptions (e.g., vmalloc area) that don't hold on >>>>> real HW. >>>>> - Various fixes related to cache maintaince to better support turn the >>>>> MMU off. >>>>> - Switch to a new stack and avoid relying on the one provided by EFI. >>>>> >>>>> git branch: https://github.com/relokin/kvm-unit-tests/pull/new/target-efi-upstream-v5 >>>>> >>>>> v4: https://lore.kernel.org/kvmarm/20230213101759.2577077-1-nikos.nikoleris@arm.com/ >>>>> v3: https://lore.kernel.org/all/20220630100324.3153655-1-nikos.nikoleris@arm.com/ >>>>> v2: https://lore.kernel.org/kvm/20220506205605.359830-1-nikos.nikoleris@arm.com/ >>>>> >>>>> Changes in v5: >>>>> - Minor style changes (thanks Shaoqin). >>>>> - Avoid including lib/acpi.o to cflatobjs twice (thanks Drew). >>>>> - Increase NR_INITIAL_MEM_REGIONS to avoid overflows and add check when >>>>> we run out of space (thanks Shaoqin). >>>>> >>>>> Changes in v4: >>>>> - Removed patch that reworks cache maintenance when turning the MMU >>>>> off. This is not strictly required for EFI tests running with tcg and >>>>> will be addressed in a separate series by Alex. >>>>> - Fix compilation for arm (Alex). >>>>> - Convert ACPI tables to Linux style (Alex). >>>>> >>>>> Changes in v3: >>>>> - Addressed feedback from Drew, Alex and Ricardo. Many thanks for the reviews! >>>>> - Added support for discovering the GIC through ACPI >>>>> - Added a missing header file (<elf.h>) >>>>> - Added support for correctly parsing the outcome of tests (./run_tests) >>>>> >>>> >>>> Thanks, Nikos! >>>> >>>> I'd like to get an ack from either Paolo or Sean on the changes to ACPI, >>>> as they're shared with x86, and there are also some x86 code changes. >>> >>> Actually, there are two build pipeline failures with the new ACPI code. >>> Please take a look at >>> >>> https://gitlab.com/jones-drew/kvm-unit-tests/-/pipelines/852864569 >>> >> >> Thanks for reviewing the series! >> >> I think this fixes the compilation issues: >> >> diff --git a/lib/acpi.h b/lib/acpi.h >> index 202d832e..c330c877 100644 >> --- a/lib/acpi.h >> +++ b/lib/acpi.h >> @@ -292,7 +292,8 @@ struct acpi_table_gtdt { >> u32 platform_timer_offset; >> }; >> >> -#pragma pack(0) >> +/* Reset to default packing */ >> +#pragma pack() >> >> void set_efi_rsdp(struct acpi_table_rsdp *rsdp); >> void *find_acpi_table_addr(u32 sig); >> diff --git a/lib/acpi.c b/lib/acpi.c >> index 760cd8b2..0440cddb 100644 >> --- a/lib/acpi.c >> +++ b/lib/acpi.c >> @@ -70,7 +70,7 @@ void *find_acpi_table_addr(u32 sig) >> return rsdt; >> >> if (rsdp->revision >= 2) { >> - xsdt = (void *)rsdp->xsdt_physical_address; >> + xsdt = (void *)(ulong) rsdp->xsdt_physical_address; >> if (xsdt && xsdt->signature != XSDT_SIGNATURE) >> xsdt = NULL; >> } >> >>> Thanks, >>> drew >>> >>>> >>>> Also, >>>> >>>> 1) It'd be nice if this worked with DT, too. We can use UEFI with DT >>>> when adding '-no-acpi' to the QEMU command line. setup_efi() needs >>>> to learn how to find the dtb and most the '#ifdef CONFIG_EFI's >>>> would need to change to a new CONFIG_ACPI guard. >>>> >> >> I had a quick look at it at some point and it didn't look straightforward >> but I'll check again. >> >>>> 2) The debug bp and ss tests fail with EFI, but not without, for me. >>>> >> >> I think, I've found the problem, the patch below fixes it for me. >> >> diff --git a/arm/debug.c b/arm/debug.c >> index b3e9749c..126fa267 100644 >> --- a/arm/debug.c >> +++ b/arm/debug.c >> @@ -292,11 +292,14 @@ static noinline void test_hw_bp(bool migrate) >> hw_bp_idx = 0; >> >> /* Trap on up to 16 debug exception unmask instructions. */ >> - asm volatile("hw_bp0:\n" >> - "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr >> daifclr, #8\n" >> - "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr >> daifclr, #8\n" >> - "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr >> daifclr, #8\n" >> - "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; msr >> daifclr, #8\n"); >> + asm volatile( >> + ".globl hw_bp0\n" >> + "hw_bp0:\n" >> + "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; >> msr daifclr, #8\n" >> + "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; >> msr daifclr, #8\n" >> + "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; >> msr daifclr, #8\n" >> + "msr daifclr, #8; msr daifclr, #8; msr daifclr, #8; >> msr daifclr, #8\n" >> + ); >> >> for (i = 0, addr = (uint64_t)&hw_bp0; i < num_bp; i++, addr += 4) >> report(hw_bp_addr[i] == addr, "hw breakpoint: %d", i); >> @@ -367,11 +370,14 @@ static noinline void test_ss(bool migrate) >> >> asm volatile("msr daifclr, #8"); >> >> - asm volatile("ss_start:\n" >> + asm volatile( >> + ".globl ss_start\n" >> + "ss_start:\n" >> "mrs x0, esr_el1\n" >> "add x0, x0, #1\n" >> "msr daifset, #8\n" >> - : : : "x0"); >> + : : : "x0" >> + ); >> >> report(ss_addr[0] == (uint64_t)&ss_start, "single step"); >> } >> >>>> 3) The timer test runs (and succeeds) when run with >>>> './arm/efi/run ./arm/timer.efi', but not when run with >>>> './run_tests.sh -g timer'. This is because UEFI takes >>>> up all the given timeout time (10s), and then the test times out. >>>> The hackyish fix below resolves it for me. I'll consider posting it >>>> as a real patch >>>> >> >> I see. I didn't hit the timeout on my test machine but I tried on a slower >> machine and I did. >> >> New branch with the fixups here: >> >> https://github.com/relokin/kvm-unit-tests/pull/new/target-efi-upstream-v5 > > Thanks for the quick fixes. Can you update your tree and make an MR? I > no longer use github.com/rhdrjones/kvm-unit-tests. I use > > https://gitlab.com/jones-drew/kvm-unit-tests.git > You mean a MR into your repo and to the branch arm/queue, don't you? I've tried doing that but I get an error: "Target branch "arm/queue" does not exist", but it allows me to create a MR into other repos. Am I doing something wrong? My branch on gitlab: https://gitlab.com/nnikoleris/kvm-unit-tests/-/tree/target-efi-upstream-v6-pre Has the new fixups squashed and I've added Shaoqin's Reviewed-by tags. And the branch with the fixups separately lives here: https://gitlab.com/nnikoleris/kvm-unit-tests/-/tree/target-efi-upstream-v5-fixups Do you want me to add your patch as well? Or are you going to add it afterwards? Meanwhile, I've tried adding support for booting using a fdt. I think it's not too hard, but I wouldn't mind merging this series first and looking at the patches for the fdt support later. Thanks, Nikos > Thanks, > drew > >> >> Many thanks, >> >> Nikos >> >>>> Thanks, >>>> drew >>>> >>>> diff --git a/scripts/arch-run.bash b/scripts/arch-run.bash >>>> index 51e4b97b27d1..72ce718b1170 100644 >>>> --- a/scripts/arch-run.bash >>>> +++ b/scripts/arch-run.bash >>>> @@ -94,7 +94,17 @@ run_qemu_status () >>>> timeout_cmd () >>>> { >>>> + local s >>>> + >>>> if [ "$TIMEOUT" ] && [ "$TIMEOUT" != "0" ]; then >>>> + if [ "$CONFIG_EFI" = 'y' ]; then >>>> + s=${TIMEOUT: -1} >>>> + if [ "$s" = 's' ]; then >>>> + TIMEOUT=${TIMEOUT:0:-1} >>>> + ((TIMEOUT += 10)) # Add 10 seconds for booting UEFI >>>> + TIMEOUT="${TIMEOUT}s" >>>> + fi >>>> + fi >>>> echo "timeout -k 1s --foreground $TIMEOUT" >>>> fi >>>> }
On Mon, May 01, 2023 at 11:27:23PM +0100, Nikos Nikoleris wrote: > On 01/05/2023 12:21, Andrew Jones wrote: ... > > > https://github.com/relokin/kvm-unit-tests/pull/new/target-efi-upstream-v5 > > > > Thanks for the quick fixes. Can you update your tree and make an MR? I > > no longer use github.com/rhdrjones/kvm-unit-tests. I use > > > > https://gitlab.com/jones-drew/kvm-unit-tests.git > > > > You mean a MR into your repo and to the branch arm/queue, don't you? I've > tried doing that but I get an error: "Target branch "arm/queue" does not > exist", but it allows me to create a MR into other repos. Am I doing > something wrong? I'm not sure what's happening unless it's looking for the arm/queue branch on the main gitlab kvm-unit-tests repo instead of mine. > > My branch on gitlab: > > https://gitlab.com/nnikoleris/kvm-unit-tests/-/tree/target-efi-upstream-v6-pre Thanks. I grabbed the patches and applied to arm/queue, https://gitlab.com/jones-drew/kvm-unit-tests/-/commits/arm/queue > Do you want me to add your patch as well? Or are you going to add it > afterwards? We can do the additional runtime stuff on top. I have a couple other script changes too and I think Shaoqin is looking at how to better cope with migration tests. > Meanwhile, I've tried adding support for booting using a fdt. I > think it's not too hard, but I wouldn't mind merging this series first and > looking at the patches for the fdt support later. I'm also anxious to get this merged, but I'm a little reluctant to take it without DT, since we may find other issues once we boot the tests with DT. I also know how it's easy, at least for me, to let stuff get dropped when their priorities reduce. If you have time now, how about we poke at it until the end of the week? With some luck, it'll be sorted out and we'll also have the x86 ack, which I'd like to get, by then. Thanks, drew
Hi Shaoqin, On 01/05/2023 13:09, Shaoqin Huang wrote: > Hi Nikos, > > When I test this series by simply run ./run_tests.sh, some migration > test will hang up forever, like the `its-migration` test. > > After have a quick look, when do the migration test, the guest can't be > migrated due to: > > {"return": {"blocked-reasons": ["The qcow format used by node > '#block389' does not support live migration", "The vvfat (rw) format > used by node '#block085' does not support live migration"]}} > > Although the guest will be timeout, but the script itself will not be > timeout. The infinite loop happened at here: > > // script/arch-run.bash > 151 # Wait for the migration to complete > 152 migstatus=`qmp ${qmp1} '"query-migrate"' | grep return` > 153 while ! grep -q '"completed"' <<<"$migstatus" ; do > 154 sleep 1 > 155 migstatus=`qmp ${qmp1} '"query-migrate"' | grep > return` > 156 if grep -q '"failed"' <<<"$migstatus" ; then > 157 echo "ERROR: Migration failed." >&2 > 158 qmp ${qmp1} '"quit"'> ${qmpout1} 2>/dev/null > 159 qmp ${qmp2} '"quit"'> ${qmpout2} 2>/dev/null > 160 return 2 > 161 fi > 162 done > > Since the "query-migrate" here will never get "completed" or "failed", > so it will never exit. > > Have you ever meet this problem? I haven't encountered any problems on Linux but I think I am running into something like this when I run on a MacOS. Unfortunately, I don't know much about the migration tests and I don't know how they are supposed to work in this case. Maybe the problem is that when we can run_tests.sh for each test we first run dummy.efi? Thanks, Nikos > > Thanks, > Shaoqin > > On 4/28/23 20:03, Nikos Nikoleris wrote: >> Hello, >> >> This series adds initial support for building arm64 tests as EFI >> apps and running them under QEMU. Much like x86_64, we import external >> dependencies from gnu-efi and adapt them to work with types and other >> assumptions from kvm-unit-tests. In addition, this series adds support >> for enumerating parts of the system using ACPI. >> >> The first set of patches, moves the existing ACPI code to the common >> lib path. Then, it extends definitions and functions to allow for more >> robust discovery of ACPI tables. We add support for setting up the PSCI >> conduit, discovering the UART, timers, GIC and cpus via ACPI. The code >> retains existing behavior and gives priority to discovery through DT >> when one has been provided. >> >> In the second set of patches, we add support for getting the command >> line from the EFI shell. This is a requirement for many of the >> existing arm64 tests. >> >> In the third set of patches, we import code from gnu-efi, make minor >> changes and add an alternative setup sequence from arm64 systems that >> boot through EFI. Finally, we add support in the build system and a >> run script which is used to run an EFI app. >> >> After this set of patches one can build arm64 EFI tests: >> >> $> ./configure --enable-efi >> $> make >> >> And use the run script to run an EFI tests: >> >> $> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256" >> >> Or all tests: >> >> $> ./run_tests.sh >> >> There are a few items that this series does not address but they would >> be useful to have: >> - Support for booting the system from EL2. Currently, we assume that a >> test starts EL1. This will be required to run EFI tests on sytems >> that implement EL2. >> - Support for reading environment variables and populating __envp. >> - Support for discovering the PCI subsystem using ACPI. >> - Get rid of other assumptions (e.g., vmalloc area) that don't hold on >> real HW. >> - Various fixes related to cache maintaince to better support turn the >> MMU off. >> - Switch to a new stack and avoid relying on the one provided by EFI. >> >> git branch: https://github.com/relokin/kvm-unit-tests/pull/new/target-efi-upstream-v5 >> >> v4: https://lore.kernel.org/kvmarm/20230213101759.2577077-1-nikos.nikoleris@arm.com/ >> v3: https://lore.kernel.org/all/20220630100324.3153655-1-nikos.nikoleris@arm.com/ >> v2: https://lore.kernel.org/kvm/20220506205605.359830-1-nikos.nikoleris@arm.com/ >> >> Changes in v5: >> - Minor style changes (thanks Shaoqin). >> - Avoid including lib/acpi.o to cflatobjs twice (thanks Drew). >> - Increase NR_INITIAL_MEM_REGIONS to avoid overflows and add check when >> we run out of space (thanks Shaoqin). >> >> Changes in v4: >> - Removed patch that reworks cache maintenance when turning the MMU >> off. This is not strictly required for EFI tests running with tcg and >> will be addressed in a separate series by Alex. >> - Fix compilation for arm (Alex). >> - Convert ACPI tables to Linux style (Alex). >> >> Changes in v3: >> - Addressed feedback from Drew, Alex and Ricardo. Many thanks for the reviews! >> - Added support for discovering the GIC through ACPI >> - Added a missing header file (<elf.h>) >> - Added support for correctly parsing the outcome of tests (./run_tests) >> >> Thanks, >> >> Nikos >> >> Alexandru Elisei (2): >> lib/acpi: Convert table names to Linux style >> lib: arm: Print test exit status >> >> Andrew Jones (2): >> arm/arm64: Rename etext to _etext >> arm64: Add a new type of memory type flag MR_F_RESERVED >> >> Nikos Nikoleris (25): >> lib: Move acpi header and implementation to lib >> x86: Move x86_64-specific EFI CFLAGS to x86_64 Makefile >> lib: Apply Lindent to acpi.{c,h} >> lib: Fix style for acpi.{c,h} >> x86: Avoid references to fields of ACPI tables >> lib/acpi: Ensure all struct definition for ACPI tables are packed >> lib/acpi: Add support for the XSDT table >> lib/acpi: Extend the definition of the FADT table >> devicetree: Check that fdt is not NULL in dt_available() >> arm64: Add support for setting up the PSCI conduit through ACPI >> arm64: Add support for discovering the UART through ACPI >> arm64: Add support for timer initialization through ACPI >> arm64: Add support for cpu initialization through ACPI >> arm64: Add support for gic initialization through ACPI >> lib/printf: Support for precision modifier in printf >> lib/printf: Add support for printing wide strings >> lib/efi: Add support for getting the cmdline >> lib: Avoid ms_abi for calls related to EFI on arm64 >> arm64: Add a setup sequence for systems that boot through EFI >> arm64: Copy code from GNU-EFI >> arm64: Change GNU-EFI imported code to use defined types >> arm64: Use code from the gnu-efi when booting with EFI >> lib: Avoid external dependency in libelf >> arm64: Add support for efi in Makefile >> arm64: Add an efi/run script >> >> scripts/runtime.bash | 13 +- >> arm/efi/run | 61 ++++++++ >> arm/run | 14 +- >> configure | 17 +- >> Makefile | 4 - >> arm/Makefile.arm | 6 + >> arm/Makefile.arm64 | 22 ++- >> arm/Makefile.common | 47 ++++-- >> x86/Makefile.common | 2 +- >> x86/Makefile.x86_64 | 4 + >> lib/linux/efi.h | 25 +++ >> lib/arm/asm/setup.h | 9 ++ >> lib/arm/asm/timer.h | 2 + >> lib/x86/asm/setup.h | 2 +- >> lib/acpi.h | 301 ++++++++++++++++++++++++++++++++++++ >> lib/argv.h | 1 + >> lib/elf.h | 57 +++++++ >> lib/libcflat.h | 1 + >> lib/x86/acpi.h | 112 -------------- >> lib/acpi.c | 129 ++++++++++++++++ >> lib/argv.c | 2 +- >> lib/arm/gic.c | 139 ++++++++++++++++- >> lib/arm/io.c | 41 ++++- >> lib/arm/mmu.c | 4 + >> lib/arm/psci.c | 37 ++++- >> lib/arm/setup.c | 269 ++++++++++++++++++++++++++------ >> lib/arm/timer.c | 92 +++++++++++ >> lib/devicetree.c | 2 +- >> lib/efi.c | 102 ++++++++++++ >> lib/printf.c | 194 +++++++++++++++++++++-- >> lib/x86/acpi.c | 82 ---------- >> lib/x86/setup.c | 2 +- >> arm/efi/elf_aarch64_efi.lds | 63 ++++++++ >> arm/flat.lds | 2 +- >> arm/cstart.S | 1 + >> arm/cstart64.S | 7 + >> arm/efi/crt0-efi-aarch64.S | 141 +++++++++++++++++ >> arm/dummy.c | 12 ++ >> arm/efi/reloc_aarch64.c | 94 +++++++++++ >> arm/micro-bench.c | 4 +- >> arm/timer.c | 10 +- >> x86/s3.c | 21 +-- >> x86/vmexit.c | 4 +- >> 43 files changed, 1831 insertions(+), 323 deletions(-) >> create mode 100755 arm/efi/run >> create mode 100644 lib/acpi.h >> create mode 100644 lib/elf.h >> delete mode 100644 lib/x86/acpi.h >> create mode 100644 lib/acpi.c >> create mode 100644 lib/arm/timer.c >> delete mode 100644 lib/x86/acpi.c >> create mode 100644 arm/efi/elf_aarch64_efi.lds >> create mode 100644 arm/efi/crt0-efi-aarch64.S >> create mode 100644 arm/dummy.c >> create mode 100644 arm/efi/reloc_aarch64.c >> >
On 02/05/2023 08:32, Andrew Jones wrote: > On Mon, May 01, 2023 at 11:27:23PM +0100, Nikos Nikoleris wrote: >> On 01/05/2023 12:21, Andrew Jones wrote: ... > > I'm also anxious to get this merged, but I'm a little reluctant to > take it without DT, since we may find other issues once we boot the > tests with DT. I also know how it's easy, at least for me, to let > stuff get dropped when their priorities reduce. If you have time now, > how about we poke at it until the end of the week? With some luck, > it'll be sorted out and we'll also have the x86 ack, which I'd like > to get, by then. > Thanks Drew! I've now implemented support for using an fdt when running tests as EFI apps. You can find the two additional commits here: https://gitlab.com/nnikoleris/kvm-unit-tests/-/tree/target-efi-upstream-v6-pre But I suppose, at some point, I should also send the new version of this series to the mailing list? What do you think? Thanks, Nikos
Hello, This series adds initial support for building arm64 tests as EFI apps and running them under QEMU. Much like x86_64, we import external dependencies from gnu-efi and adapt them to work with types and other assumptions from kvm-unit-tests. In addition, this series adds support for enumerating parts of the system using ACPI. The first set of patches, moves the existing ACPI code to the common lib path. Then, it extends definitions and functions to allow for more robust discovery of ACPI tables. We add support for setting up the PSCI conduit, discovering the UART, timers, GIC and cpus via ACPI. The code retains existing behavior and gives priority to discovery through DT when one has been provided. In the second set of patches, we add support for getting the command line from the EFI shell. This is a requirement for many of the existing arm64 tests. In the third set of patches, we import code from gnu-efi, make minor changes and add an alternative setup sequence from arm64 systems that boot through EFI. Finally, we add support in the build system and a run script which is used to run an EFI app. After this set of patches one can build arm64 EFI tests: $> ./configure --enable-efi $> make And use the run script to run an EFI tests: $> ./arm/efi/run ./arm/selftest.efi -smp 2 -m 256 -append "setup smp=2 mem=256" Or all tests: $> ./run_tests.sh There are a few items that this series does not address but they would be useful to have: - Support for booting the system from EL2. Currently, we assume that a test starts EL1. This will be required to run EFI tests on sytems that implement EL2. - Support for reading environment variables and populating __envp. - Support for discovering the PCI subsystem using ACPI. - Get rid of other assumptions (e.g., vmalloc area) that don't hold on real HW. - Various fixes related to cache maintaince to better support turn the MMU off. - Switch to a new stack and avoid relying on the one provided by EFI. git branch: https://github.com/relokin/kvm-unit-tests/pull/new/target-efi-upstream-v5 v4: https://lore.kernel.org/kvmarm/20230213101759.2577077-1-nikos.nikoleris@arm.com/ v3: https://lore.kernel.org/all/20220630100324.3153655-1-nikos.nikoleris@arm.com/ v2: https://lore.kernel.org/kvm/20220506205605.359830-1-nikos.nikoleris@arm.com/ Changes in v5: - Minor style changes (thanks Shaoqin). - Avoid including lib/acpi.o to cflatobjs twice (thanks Drew). - Increase NR_INITIAL_MEM_REGIONS to avoid overflows and add check when we run out of space (thanks Shaoqin). Changes in v4: - Removed patch that reworks cache maintenance when turning the MMU off. This is not strictly required for EFI tests running with tcg and will be addressed in a separate series by Alex. - Fix compilation for arm (Alex). - Convert ACPI tables to Linux style (Alex). Changes in v3: - Addressed feedback from Drew, Alex and Ricardo. Many thanks for the reviews! - Added support for discovering the GIC through ACPI - Added a missing header file (<elf.h>) - Added support for correctly parsing the outcome of tests (./run_tests) Thanks, Nikos Alexandru Elisei (2): lib/acpi: Convert table names to Linux style lib: arm: Print test exit status Andrew Jones (2): arm/arm64: Rename etext to _etext arm64: Add a new type of memory type flag MR_F_RESERVED Nikos Nikoleris (25): lib: Move acpi header and implementation to lib x86: Move x86_64-specific EFI CFLAGS to x86_64 Makefile lib: Apply Lindent to acpi.{c,h} lib: Fix style for acpi.{c,h} x86: Avoid references to fields of ACPI tables lib/acpi: Ensure all struct definition for ACPI tables are packed lib/acpi: Add support for the XSDT table lib/acpi: Extend the definition of the FADT table devicetree: Check that fdt is not NULL in dt_available() arm64: Add support for setting up the PSCI conduit through ACPI arm64: Add support for discovering the UART through ACPI arm64: Add support for timer initialization through ACPI arm64: Add support for cpu initialization through ACPI arm64: Add support for gic initialization through ACPI lib/printf: Support for precision modifier in printf lib/printf: Add support for printing wide strings lib/efi: Add support for getting the cmdline lib: Avoid ms_abi for calls related to EFI on arm64 arm64: Add a setup sequence for systems that boot through EFI arm64: Copy code from GNU-EFI arm64: Change GNU-EFI imported code to use defined types arm64: Use code from the gnu-efi when booting with EFI lib: Avoid external dependency in libelf arm64: Add support for efi in Makefile arm64: Add an efi/run script scripts/runtime.bash | 13 +- arm/efi/run | 61 ++++++++ arm/run | 14 +- configure | 17 +- Makefile | 4 - arm/Makefile.arm | 6 + arm/Makefile.arm64 | 22 ++- arm/Makefile.common | 47 ++++-- x86/Makefile.common | 2 +- x86/Makefile.x86_64 | 4 + lib/linux/efi.h | 25 +++ lib/arm/asm/setup.h | 9 ++ lib/arm/asm/timer.h | 2 + lib/x86/asm/setup.h | 2 +- lib/acpi.h | 301 ++++++++++++++++++++++++++++++++++++ lib/argv.h | 1 + lib/elf.h | 57 +++++++ lib/libcflat.h | 1 + lib/x86/acpi.h | 112 -------------- lib/acpi.c | 129 ++++++++++++++++ lib/argv.c | 2 +- lib/arm/gic.c | 139 ++++++++++++++++- lib/arm/io.c | 41 ++++- lib/arm/mmu.c | 4 + lib/arm/psci.c | 37 ++++- lib/arm/setup.c | 269 ++++++++++++++++++++++++++------ lib/arm/timer.c | 92 +++++++++++ lib/devicetree.c | 2 +- lib/efi.c | 102 ++++++++++++ lib/printf.c | 194 +++++++++++++++++++++-- lib/x86/acpi.c | 82 ---------- lib/x86/setup.c | 2 +- arm/efi/elf_aarch64_efi.lds | 63 ++++++++ arm/flat.lds | 2 +- arm/cstart.S | 1 + arm/cstart64.S | 7 + arm/efi/crt0-efi-aarch64.S | 141 +++++++++++++++++ arm/dummy.c | 12 ++ arm/efi/reloc_aarch64.c | 94 +++++++++++ arm/micro-bench.c | 4 +- arm/timer.c | 10 +- x86/s3.c | 21 +-- x86/vmexit.c | 4 +- 43 files changed, 1831 insertions(+), 323 deletions(-) create mode 100755 arm/efi/run create mode 100644 lib/acpi.h create mode 100644 lib/elf.h delete mode 100644 lib/x86/acpi.h create mode 100644 lib/acpi.c create mode 100644 lib/arm/timer.c delete mode 100644 lib/x86/acpi.c create mode 100644 arm/efi/elf_aarch64_efi.lds create mode 100644 arm/efi/crt0-efi-aarch64.S create mode 100644 arm/dummy.c create mode 100644 arm/efi/reloc_aarch64.c