| Message ID | 20230526221712.317287-1-oliver.upton@linux.dev (mailing list archive) |
|---|---|
| Headers | show |
| Series | arm64: Handle PSCI calls in userspace | expand |
Hi Oliver, On Fri, May 26, 2023 at 10:16:51PM +0000, Oliver Upton wrote: > The 6.4 kernel picks up support for a generalized SMCCC filter, allowing > userspace to select hypercall ranges that should be forwarded to > userspace. This is a shameless attempt of making future SMCCC interfaces > the responsibility of userspace :) > > As a starting point, let's move PSCI up into userspace. KVM already > leans on userspace for handling calls that have a system-wide effect. > > Tested on linux-next with a 64 vCPU VM. Additionally, I took a stab at > running kvm-unit-test's psci test, which passes. I applied these patches to kvmtool and tested 6.4, however they don't work with an SVE enabled kernel/device. # ./lkvm run -c 2 -m 4 -k psci.flat INFO: psci: PSCI version 1.0 PASS: psci: invalid-function PASS: psci: affinity-info-on PASS: psci: affinity-info-off Error: KVM_ARM_VCPU_FINALIZE: Operation not permitted Fatal: Unable to configure requested vcpu features `kvm_cpu__configure_features` in kvmtool is failing because Linux returns an error if SVE was already finalised (arch/arm64/kvm/reset.c): ``` int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature) { switch (feature) { case KVM_ARM_VCPU_SVE: if (!vcpu_has_sve(vcpu)) return -EINVAL; if (kvm_arm_vcpu_sve_finalized(vcpu)) return -EPERM; // <---- returns here return kvm_vcpu_finalize_sve(vcpu); } return -EINVAL; } ``` It's not immediately obvious to me why finalising SVE twice is an error. Changing that to `return 0;` gets the test passing, but not sure if there are other implications. I also booted with `arm64.nosve` and the test passed. Thanks, Joey
Hey Joey, Thanks for the review and taking the patches for a spin. On Wed, Jun 14, 2023 at 01:05:03PM +0100, Joey Gouly wrote: > `kvm_cpu__configure_features` in kvmtool is failing because Linux returns an > error if SVE was already finalised (arch/arm64/kvm/reset.c): > > ``` > int kvm_arm_vcpu_finalize(struct kvm_vcpu *vcpu, int feature) > { > switch (feature) { > case KVM_ARM_VCPU_SVE: > if (!vcpu_has_sve(vcpu)) > return -EINVAL; > > if (kvm_arm_vcpu_sve_finalized(vcpu)) > return -EPERM; // <---- returns here > > return kvm_vcpu_finalize_sve(vcpu); > } > > return -EINVAL; > } > ``` > > It's not immediately obvious to me why finalising SVE twice is an error. > Changing that to `return 0;` gets the test passing, but not sure if there > are other implications. This is utterly mindless on my part, apologies. The SVE feature shouldn't be finalised (again). I'll probably drop patch 8 altogether and replace its usage with a direct call to KVM_ARM_VCPU_INIT.
The 6.4 kernel picks up support for a generalized SMCCC filter, allowing userspace to select hypercall ranges that should be forwarded to userspace. This is a shameless attempt of making future SMCCC interfaces the responsibility of userspace :) As a starting point, let's move PSCI up into userspace. KVM already leans on userspace for handling calls that have a system-wide effect. Tested on linux-next with a 64 vCPU VM. Additionally, I took a stab at running kvm-unit-test's psci test, which passes. Apologies for some of the changelogs being a bit short. It's Friday, and I'm lazy. Oliver Upton (21): update_headers: Use a list for arch-generic headers update_headers: Add missing entries to list of headers to copy Copy 64-bit alignment attrtibutes from Linux 6.4-rc1 Update headers with Linux 6.4-rc1 Import arm-smccc.h from Linux 6.4-rc1 aarch64: Copy cputype.h from Linux 6.4-rc1 arm: Stash kvm_vcpu_init for later use arm: Add support for resetting a vCPU arm: Use KVM_SET_MP_STATE ioctl to power off non-boot vCPUs aarch64: Expose ARM64_CORE_REG() for general use arm: Generalize execution state specific VM initialization Add helpers to pause the VM from vCPU thread aarch64: Add support for finding vCPU for given MPIDR aarch64: Add skeleton implementation for PSCI aarch64: psci: Implement CPU_SUSPEND aarch64: psci: Implement CPU_OFF aarch64: psci: Implement CPU_ON aarch64: psci: Implement AFFINITY_INFO aarch64: psci: Implement MIGRATE_INFO_TYPE aarch64: psci: Implement SYSTEM_{OFF,RESET} aarch64: smccc: Start sending PSCI to userspace Makefile | 4 +- arm/aarch32/include/kvm/kvm-arch.h | 2 +- arm/aarch32/kvm-cpu.c | 5 + arm/aarch64/include/asm/cputype.h | 186 +++++ arm/aarch64/include/asm/kvm.h | 38 + arm/aarch64/include/asm/smccc.h | 65 ++ arm/aarch64/include/kvm/kvm-arch.h | 2 +- arm/aarch64/include/kvm/kvm-config-arch.h | 6 +- arm/aarch64/include/kvm/kvm-cpu-arch.h | 28 +- arm/aarch64/kvm-cpu.c | 48 +- arm/aarch64/kvm.c | 25 +- arm/aarch64/psci.c | 206 +++++ arm/aarch64/smccc.c | 82 ++ arm/include/arm-common/kvm-arch.h | 2 + arm/include/arm-common/kvm-config-arch.h | 1 + arm/include/arm-common/kvm-cpu-arch.h | 4 +- arm/kvm-cpu.c | 35 +- arm/kvm.c | 2 +- include/kvm/kvm-cpu.h | 3 + include/linux/arm-smccc.h | 240 ++++++ include/linux/kvm.h | 55 +- include/linux/psci.h | 47 ++ include/linux/types.h | 13 + include/linux/vfio.h | 920 +++++++++++++++++++++- include/linux/vhost.h | 186 ++--- include/linux/virtio_blk.h | 105 +++ include/linux/virtio_net.h | 4 + kvm-cpu.c | 15 + riscv/include/asm/kvm.h | 3 + util/update_headers.sh | 25 +- x86/include/asm/kvm.h | 50 +- 31 files changed, 2225 insertions(+), 182 deletions(-) create mode 100644 arm/aarch64/include/asm/cputype.h create mode 100644 arm/aarch64/include/asm/smccc.h create mode 100644 arm/aarch64/psci.c create mode 100644 arm/aarch64/smccc.c create mode 100644 include/linux/arm-smccc.h base-commit: 77b108c6a6f1c66fb7f60a80d17596bb80bda8ad