[0/4] KVM: arm64: Fix CPU resume/on with pKVM

Message

Quentin Perret Feb. 1, 2023, 10:37 a.m. UTC

When using pKVM, we do not reset the EL2 exception vectors back to the
stubs for e.g. Power Management or CPU hotplug as we normally do in KVM.
As consequence, the initialisation perfomed by __finalise_el2 is missing
on e.g. the CPU_RESUME path with pKVM, hence leaving certain registers
in an incorrect state.

One such example is ZCR_EL2 which remains configured with SVE traps
enabled. And so using SVE on a CPU that has gone through a hotplug
off/on cycle leads to a hyp panic. Not good.

This series fixes this by macroizing the first half of __finalise_el2
(that is, the part that is not specific to VHE) to allow its re-use
from pKVM's PSCI relay.

Quentin Perret (4):
  KVM: arm64: Provide sanitized SYS_ID_AA64SMFR0_EL1 to nVHE
  KVM: arm64: Introduce finalise_el2_state macro
  KVM: arm64: Use sanitized values in __check_override in nVHE
  KVM: arm64: Finalise EL2 state from pKVM PSCI relay

 arch/arm64/include/asm/el2_setup.h | 92 ++++++++++++++++++++++++++++++
 arch/arm64/include/asm/kvm_hyp.h   |  1 +
 arch/arm64/kernel/hyp-stub.S       | 79 +------------------------
 arch/arm64/kvm/arm.c               |  1 +
 arch/arm64/kvm/hyp/nvhe/hyp-init.S |  1 +
 arch/arm64/kvm/hyp/nvhe/sys_regs.c |  1 +
 6 files changed, 98 insertions(+), 77 deletions(-)

Comments

Marc Zyngier Feb. 1, 2023, 2:48 p.m. UTC | #1

On Wed, 01 Feb 2023 10:37:50 +0000,
Quentin Perret <[e-mail]> wrote:
> 
> When using pKVM, we do not reset the EL2 exception vectors back to the
> stubs for e.g. Power Management or CPU hotplug as we normally do in KVM.
> As consequence, the initialisation perfomed by __finalise_el2 is missing
> on e.g. the CPU_RESUME path with pKVM, hence leaving certain registers
> in an incorrect state.
> 
> One such example is ZCR_EL2 which remains configured with SVE traps
> enabled. And so using SVE on a CPU that has gone through a hotplug
> off/on cycle leads to a hyp panic. Not good.
> 
> This series fixes this by macroizing the first half of __finalise_el2
> (that is, the part that is not specific to VHE) to allow its re-use
> from pKVM's PSCI relay.

For the series:

Reviewed-by: Marc Zyngier <[e-mail]>

I think it might be a bit late to send this as fixes for 6.2 (my
latest pull request for kvmarm-fixes-6.2-3 is still in limbo), so I'd
suggest we take it for 6.3.

How do you want to deal with the backports? None of the patches have a
Cc: stable, and only the last one has a Fixes: tag, but cannot be
applied standalone.

Thanks,

	M.

Quentin Perret Feb. 1, 2023, 4:57 p.m. UTC | #2

On Wednesday 01 Feb 2023 at 14:48:03 (+0000), Marc Zyngier wrote:
> On Wed, 01 Feb 2023 10:37:50 +0000,
> Quentin Perret <[e-mail]> wrote:
> > 
> > When using pKVM, we do not reset the EL2 exception vectors back to the
> > stubs for e.g. Power Management or CPU hotplug as we normally do in KVM.
> > As consequence, the initialisation perfomed by __finalise_el2 is missing
> > on e.g. the CPU_RESUME path with pKVM, hence leaving certain registers
> > in an incorrect state.
> > 
> > One such example is ZCR_EL2 which remains configured with SVE traps
> > enabled. And so using SVE on a CPU that has gone through a hotplug
> > off/on cycle leads to a hyp panic. Not good.
> > 
> > This series fixes this by macroizing the first half of __finalise_el2
> > (that is, the part that is not specific to VHE) to allow its re-use
> > from pKVM's PSCI relay.
> 
> For the series:
> 
> Reviewed-by: Marc Zyngier <[e-mail]>
> 
> I think it might be a bit late to send this as fixes for 6.2 (my
> latest pull request for kvmarm-fixes-6.2-3 is still in limbo), so I'd
> suggest we take it for 6.3.

Works for me.

> How do you want to deal with the backports? None of the patches have a
> Cc: stable, and only the last one has a Fixes: tag, but cannot be
> applied standalone.

Right, I wasn't sure what was best -- the first patches aren't really
fixing anything per se but yeah, we kinda need them...

Happy to re-post a version with the same 'Fixes:' tag on all patches and
'Cc: stable' everywhere if that makes things easier. Wdyt?

Cheers,
Quentin

Oliver Upton Feb. 2, 2023, 9:43 p.m. UTC | #3

On Wed, Feb 01, 2023 at 04:57:09PM +0000, Quentin Perret wrote:
> On Wednesday 01 Feb 2023 at 14:48:03 (+0000), Marc Zyngier wrote:
> > How do you want to deal with the backports? None of the patches have a
> > Cc: stable, and only the last one has a Fixes: tag, but cannot be
> > applied standalone.
> 
> Right, I wasn't sure what was best -- the first patches aren't really
> fixing anything per se but yeah, we kinda need them...
> 
> Happy to re-post a version with the same 'Fixes:' tag on all patches and
> 'Cc: stable' everywhere if that makes things easier. Wdyt?

I'd like to get these patches cooking in -next so I'll probably just
take what you've posted.

Oliver Upton Feb. 6, 2023, 5:12 p.m. UTC | #4

On Wed, 1 Feb 2023 10:37:50 +0000, Quentin Perret wrote:
> When using pKVM, we do not reset the EL2 exception vectors back to the
> stubs for e.g. Power Management or CPU hotplug as we normally do in KVM.
> As consequence, the initialisation perfomed by __finalise_el2 is missing
> on e.g. the CPU_RESUME path with pKVM, hence leaving certain registers
> in an incorrect state.
> 
> One such example is ZCR_EL2 which remains configured with SVE traps
> enabled. And so using SVE on a CPU that has gone through a hotplug
> off/on cycle leads to a hyp panic. Not good.
> 
> [...]

Applied to kvmarm/next, thanks!

[1/4] KVM: arm64: Provide sanitized SYS_ID_AA64SMFR0_EL1 to nVHE
      https://git.kernel.org/kvmarm/kvmarm/c/8669651ce0d9
[2/4] KVM: arm64: Introduce finalise_el2_state macro
      https://git.kernel.org/kvmarm/kvmarm/c/e2d4f5ae1771
[3/4] KVM: arm64: Use sanitized values in __check_override in nVHE
      https://git.kernel.org/kvmarm/kvmarm/c/3c4cc31537ec
[4/4] KVM: arm64: Finalise EL2 state from pKVM PSCI relay
      https://git.kernel.org/kvmarm/kvmarm/c/6f10f2ec61c7

--
Best,
Oliver