From patchwork Thu Jun 20 16:46:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705906 Received: from out-189.mta1.migadu.com (out-189.mta1.migadu.com [95.215.58.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B59C1AD9D5 for ; Thu, 20 Jun 2024 16:47:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.189 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902033; cv=none; b=mHdZ6KUxJjLu6SiFF5iBoYxZjvX1qm2QeynJqXlEMVnKCvJox9ejSd+FRlc6Z2TVmSgKTKf8ndxxsPMNRFkna/I2rWlopDsDMy3+WfUU/E3/q8Jj/wZkq2heKSYuBAvREcEw/ElECYWmvcth3XZQ7YYpRaA/DnQJK8pMk3naJoI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902033; c=relaxed/simple; bh=8F+cL1uAYf9JGVDeoFp8WvE/ix2DTBKwTTvZ3EaCKD8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VLWpZzdw76Z+IgQpr4dMm5r6TvtqpvvVOztRzfcsovsZsWE0yAf4UrRhpMNhIXJr8BzP0iaSYUyavdg+jEX3JA35SrqotoZfvYqeVMvYTH6AIfLn9bQW04eTQ50Qhomt6cP/SsbTxV1AVTCxfIYqM4ehkk167jtwf7ErYERhVVQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=VmF6mshm; arc=none smtp.client-ip=95.215.58.189 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="VmF6mshm" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902025; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cRgjmZ9/UVonezZxApoYoSKYJAYeAK2PT3Zy3D03wwQ=; b=VmF6mshmzB67F73iqjv4kNAfje8VuxqK7ctdHZlT/HwkldBWf+0Wa3MsW15Zp9KwF2hqB5 nJUoXhnD6GCqT4fsd5TqFKMlgLEM4L2+2H2EsGF6ZhteNJNRAA2tQGn9QqV+27pVS8aIYB 5mYJMCV1iJYTa9u/nv+72jGdrKUE4A4= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: jintack.lim@linaro.org X-Envelope-To: christoffer.dall@arm.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Jintack Lim , Christoffer Dall , Oliver Upton Subject: [PATCH v3 01/15] KVM: arm64: nv: Forward FP/ASIMD traps to guest hypervisor Date: Thu, 20 Jun 2024 16:46:38 +0000 Message-ID: <20240620164653.1130714-2-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT From: Jintack Lim Give precedence to the guest hypervisor's trap configuration when routing an FP/ASIMD trap taken to EL2. Take advantage of the infrastructure for translating CPTR_EL2 into the VHE (i.e. EL1) format and base the trap decision solely on the VHE view of the register. The in-memory value of CPTR_EL2 will always be up to date for the guest hypervisor (more on that later), so just read it directly from memory. Bury all of this behind a macro keyed off of the CPTR bitfield in anticipation of supporting other traps (e.g. SVE). [maz: account for HCR_EL2.E2H when testing for TFP/FPEN, with all the hard work actually being done by Chase Conklin] [ oliver: translate nVHE->VHE format for testing traps; macro for reuse in other CPTR_EL2.xEN fields ] Signed-off-by: Jintack Lim Signed-off-by: Christoffer Dall Signed-off-by: Marc Zyngier Reviewed-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/include/asm/kvm_emulate.h | 43 +++++++++++++++++++++++++ arch/arm64/kvm/handle_exit.c | 16 ++++++--- arch/arm64/kvm/hyp/include/hyp/switch.h | 3 ++ 3 files changed, 58 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h index 501e3e019c93..c3c5a5999ed7 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -11,6 +11,7 @@ #ifndef __ARM64_KVM_EMULATE_H__ #define __ARM64_KVM_EMULATE_H__ +#include #include #include @@ -599,4 +600,46 @@ static __always_inline void kvm_reset_cptr_el2(struct kvm_vcpu *vcpu) kvm_write_cptr_el2(val); } + +/* + * Returns a 'sanitised' view of CPTR_EL2, translating from nVHE to the VHE + * format if E2H isn't set. + */ +static inline u64 vcpu_sanitised_cptr_el2(const struct kvm_vcpu *vcpu) +{ + u64 cptr = __vcpu_sys_reg(vcpu, CPTR_EL2); + + if (!vcpu_el2_e2h_is_set(vcpu)) + cptr = translate_cptr_el2_to_cpacr_el1(cptr); + + return cptr; +} + +static inline bool ____cptr_xen_trap_enabled(const struct kvm_vcpu *vcpu, + unsigned int xen) +{ + switch (xen) { + case 0b00: + case 0b10: + return true; + case 0b01: + return vcpu_el2_tge_is_set(vcpu) && !vcpu_is_el2(vcpu); + case 0b11: + default: + return false; + } +} + +#define __guest_hyp_cptr_xen_trap_enabled(vcpu, xen) \ + (!vcpu_has_nv(vcpu) ? false : \ + ____cptr_xen_trap_enabled(vcpu, \ + SYS_FIELD_GET(CPACR_ELx, xen, \ + vcpu_sanitised_cptr_el2(vcpu)))) + +static inline bool guest_hyp_fpsimd_traps_enabled(const struct kvm_vcpu *vcpu) +{ + return __guest_hyp_cptr_xen_trap_enabled(vcpu, FPEN); +} + + #endif /* __ARM64_KVM_EMULATE_H__ */ diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index b037f0a0e27e..59fe9b10a87a 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -94,11 +94,19 @@ static int handle_smc(struct kvm_vcpu *vcpu) } /* - * Guest access to FP/ASIMD registers are routed to this handler only - * when the system doesn't support FP/ASIMD. + * This handles the cases where the system does not support FP/ASIMD or when + * we are running nested virtualization and the guest hypervisor is trapping + * FP/ASIMD accesses by its guest guest. + * + * All other handling of guest vs. host FP/ASIMD register state is handled in + * fixup_guest_exit(). */ -static int handle_no_fpsimd(struct kvm_vcpu *vcpu) +static int kvm_handle_fpasimd(struct kvm_vcpu *vcpu) { + if (guest_hyp_fpsimd_traps_enabled(vcpu)) + return kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu)); + + /* This is the case when the system doesn't support FP/ASIMD. */ kvm_inject_undefined(vcpu); return 1; } @@ -304,7 +312,7 @@ static exit_handle_fn arm_exit_handlers[] = { [ESR_ELx_EC_BREAKPT_LOW]= kvm_handle_guest_debug, [ESR_ELx_EC_BKPT32] = kvm_handle_guest_debug, [ESR_ELx_EC_BRK64] = kvm_handle_guest_debug, - [ESR_ELx_EC_FP_ASIMD] = handle_no_fpsimd, + [ESR_ELx_EC_FP_ASIMD] = kvm_handle_fpasimd, [ESR_ELx_EC_PAC] = kvm_handle_ptrauth, }; diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index a92566f36022..b302d32f8326 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -341,6 +341,9 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code) /* Only handle traps the vCPU can support here: */ switch (esr_ec) { case ESR_ELx_EC_FP_ASIMD: + /* Forward traps to the guest hypervisor as required */ + if (guest_hyp_fpsimd_traps_enabled(vcpu)) + return false; break; case ESR_ELx_EC_SVE: if (!sve_guest) From patchwork Thu Jun 20 16:46:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705904 Received: from out-171.mta1.migadu.com (out-171.mta1.migadu.com [95.215.58.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59CE41B29AC for ; Thu, 20 Jun 2024 16:47:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902032; cv=none; b=RP2uoXXLn50+6//ojZ6UbVnUjbMknx8m10MxwZzDhuV5JZvCy597g54AmW3IcBAMQouTy+B6KGQAzQOv+3KBqTawTcGYgSja4Ba8cMju6/6AG+ngea+OrBytzsL6f1aQdB6oU4zr5E1bypJXLZr/O6JlbiXlCx+eMuvJVM/8HkA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902032; c=relaxed/simple; bh=ETX0LQ7kYSg7RqI/4kAd77n7iye+m5txnswDWv+x1iI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pA214epwtsHTDRAAfWlQUlcza3NvvaChtfN700a5pMNaYp29AnyKrfSYXOP6/Zj6PcAzQ6OblXhTl06NPcsGFrYYps9YBoRANxRGz5WidVXloHuCIk76Ti+Vd5gpxKreLgaC6RCbUkiun2uYrAtiXehsFDCSgoaBNN76UbfBRY4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=lcnSzIhx; arc=none smtp.client-ip=95.215.58.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="lcnSzIhx" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902026; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H05WUL/HhblgxcxQS07XCyaa2Ay55FYE9vwptiKKacE=; b=lcnSzIhxLamBw3LAgU4LdacXsVx2zETov7WpP36KoPfoeZh2NHgOtCXK6tSSIk/B9zTxtm lT2DBf6zX9yTYS3b3n4GvTk2KJ8vUNFcqNbx99K4dDAuWmYWi7NFiRBFJmrsvA29StLHLj gBQ1rd10TnXC29zOCTwkxF63Cyfdz7U= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Oliver Upton Subject: [PATCH v3 02/15] KVM: arm64: nv: Forward SVE traps to guest hypervisor Date: Thu, 20 Jun 2024 16:46:39 +0000 Message-ID: <20240620164653.1130714-3-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Similar to FPSIMD traps, don't load SVE state if the guest hypervisor has SVE traps enabled and forward the trap instead. Note that ZCR_EL2 will require some special handling, as it takes a sysreg trap to EL2 when HCR_EL2.NV = 1. Reviewed-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/include/asm/kvm_emulate.h | 4 ++++ arch/arm64/kvm/handle_exit.c | 3 +++ arch/arm64/kvm/hyp/include/hyp/switch.h | 2 ++ 3 files changed, 9 insertions(+) diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h index c3c5a5999ed7..cc5c93b46e6f 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -641,5 +641,9 @@ static inline bool guest_hyp_fpsimd_traps_enabled(const struct kvm_vcpu *vcpu) return __guest_hyp_cptr_xen_trap_enabled(vcpu, FPEN); } +static inline bool guest_hyp_sve_traps_enabled(const struct kvm_vcpu *vcpu) +{ + return __guest_hyp_cptr_xen_trap_enabled(vcpu, ZEN); +} #endif /* __ARM64_KVM_EMULATE_H__ */ diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 59fe9b10a87a..e4f74699f360 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -217,6 +217,9 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu) */ static int handle_sve(struct kvm_vcpu *vcpu) { + if (guest_hyp_sve_traps_enabled(vcpu)) + return kvm_inject_nested_sync(vcpu, kvm_vcpu_get_esr(vcpu)); + kvm_inject_undefined(vcpu); return 1; } diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index b302d32f8326..428ee15dd6ae 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -348,6 +348,8 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code) case ESR_ELx_EC_SVE: if (!sve_guest) return false; + if (guest_hyp_sve_traps_enabled(vcpu)) + return false; break; default: return false; From patchwork Thu Jun 20 16:46:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705905 Received: from out-185.mta1.migadu.com (out-185.mta1.migadu.com [95.215.58.185]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41A2F1B151D for ; Thu, 20 Jun 2024 16:47:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.185 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902032; cv=none; b=h5puwXEm4cLJqo9ZKqi/H9Jcn9Wyvm0n71eUZfQblNZYUklpG+4PlM9y6rxa/PYkhR2ei2aJV0D+Hg5VWUt8ncQ9Myujol8gXg0dwP5CVwZJ7OlGyjWUpuLHAR9va4NKs65SEytwZ2NBmmVDedU0wxR3EI438OJ4wCyBG2grPOQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902032; c=relaxed/simple; bh=SIQZqzQaFcBJrCKiOWUNha8OzuNiWrEQ8pZ8emdsv+s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TO/6PIx8Mz6GcRRtH9/3RcyWpswLGIHrmmETVQ9HqdT9qIxERVeBfjY33EK4R/Q9azu+tMSs3iFCInZOseIiptlJWpoEtVz7xOE/vDkGIh/n4Rm5FTbgA+apzsQgIBeIOANbwHFRLUhKRTVVeRbVD1L1k1Y/otchNY2StRuS5fs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=sr92Ybfe; arc=none smtp.client-ip=95.215.58.185 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="sr92Ybfe" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902028; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/3aRGFWE/WQp/6kn+do3vtR0T38O3nr9oCPqc1hpA8o=; b=sr92YbfeOPHxAhqEwftdGMmmW4Sg0XkJza3cBXO3oIEvAUQb/bO64CtRgcdDSd3wsthSyF 0CR0oIaXiguRVwDy7c15Bo6Z4lmh/i5ZEd+aTrx9JElKANZPQKlzpk3ErCIbQYjL3D1QrR hQqQ5WfsEI8B0iUjolPZj9UU9jhINqk= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Oliver Upton Subject: [PATCH v3 03/15] KVM: arm64: nv: Handle ZCR_EL2 traps Date: Thu, 20 Jun 2024 16:46:40 +0000 Message-ID: <20240620164653.1130714-4-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Unlike other SVE-related registers, ZCR_EL2 takes a sysreg trap to EL2 when HCR_EL2.NV = 1. KVM still needs to honor the guest hypervisor's trap configuration, which expects an SVE trap (i.e. ESR_EL2.EC = 0x19) when CPTR traps are enabled for the vCPU's current context. Otherwise, if the guest hypervisor has traps disabled, emulate the access by mapping the requested VL into ZCR_EL1. Reviewed-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/include/asm/kvm_emulate.h | 8 ++++++ arch/arm64/include/asm/kvm_host.h | 3 +++ arch/arm64/kvm/sys_regs.c | 38 ++++++++++++++++++++++++++++ 3 files changed, 49 insertions(+) diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h index cc5c93b46e6f..15c70f1ebeb3 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -56,6 +56,14 @@ void kvm_emulate_nested_eret(struct kvm_vcpu *vcpu); int kvm_inject_nested_sync(struct kvm_vcpu *vcpu, u64 esr_el2); int kvm_inject_nested_irq(struct kvm_vcpu *vcpu); +static inline void kvm_inject_nested_sve_trap(struct kvm_vcpu *vcpu) +{ + u64 esr = FIELD_PREP(ESR_ELx_EC_MASK, ESR_ELx_EC_SVE) | + ESR_ELx_IL; + + kvm_inject_nested_sync(vcpu, esr); +} + #if defined(__KVM_VHE_HYPERVISOR__) || defined(__KVM_NVHE_HYPERVISOR__) static __always_inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu) { diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 8170c04fde91..5a114720fd57 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -422,6 +422,7 @@ enum vcpu_sysreg { MDCR_EL2, /* Monitor Debug Configuration Register (EL2) */ CPTR_EL2, /* Architectural Feature Trap Register (EL2) */ HACR_EL2, /* Hypervisor Auxiliary Control Register */ + ZCR_EL2, /* SVE Control Register (EL2) */ TTBR0_EL2, /* Translation Table Base Register 0 (EL2) */ TTBR1_EL2, /* Translation Table Base Register 1 (EL2) */ TCR_EL2, /* Translation Control Register (EL2) */ @@ -968,6 +969,7 @@ static inline bool __vcpu_read_sys_reg_from_cpu(int reg, u64 *val) case DACR32_EL2: *val = read_sysreg_s(SYS_DACR32_EL2); break; case IFSR32_EL2: *val = read_sysreg_s(SYS_IFSR32_EL2); break; case DBGVCR32_EL2: *val = read_sysreg_s(SYS_DBGVCR32_EL2); break; + case ZCR_EL1: *val = read_sysreg_s(SYS_ZCR_EL12); break; default: return false; } @@ -1013,6 +1015,7 @@ static inline bool __vcpu_write_sys_reg_to_cpu(u64 val, int reg) case DACR32_EL2: write_sysreg_s(val, SYS_DACR32_EL2); break; case IFSR32_EL2: write_sysreg_s(val, SYS_IFSR32_EL2); break; case DBGVCR32_EL2: write_sysreg_s(val, SYS_DBGVCR32_EL2); break; + case ZCR_EL1: write_sysreg_s(val, SYS_ZCR_EL12); break; default: return false; } diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c index 22b45a15d068..80aa99a012ae 100644 --- a/arch/arm64/kvm/sys_regs.c +++ b/arch/arm64/kvm/sys_regs.c @@ -121,6 +121,7 @@ static bool get_el2_to_el1_mapping(unsigned int reg, MAPPED_EL2_SYSREG(AMAIR_EL2, AMAIR_EL1, NULL ); MAPPED_EL2_SYSREG(ELR_EL2, ELR_EL1, NULL ); MAPPED_EL2_SYSREG(SPSR_EL2, SPSR_EL1, NULL ); + MAPPED_EL2_SYSREG(ZCR_EL2, ZCR_EL1, NULL ); default: return false; } @@ -2199,6 +2200,40 @@ static u64 reset_hcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) return __vcpu_sys_reg(vcpu, r->reg) = val; } +static unsigned int sve_el2_visibility(const struct kvm_vcpu *vcpu, + const struct sys_reg_desc *rd) +{ + unsigned int r; + + r = el2_visibility(vcpu, rd); + if (r) + return r; + + return sve_visibility(vcpu, rd); +} + +static bool access_zcr_el2(struct kvm_vcpu *vcpu, + struct sys_reg_params *p, + const struct sys_reg_desc *r) +{ + unsigned int vq; + + if (guest_hyp_sve_traps_enabled(vcpu)) { + kvm_inject_nested_sve_trap(vcpu); + return true; + } + + if (!p->is_write) { + p->regval = vcpu_read_sys_reg(vcpu, ZCR_EL2); + return true; + } + + vq = SYS_FIELD_GET(ZCR_ELx, LEN, p->regval) + 1; + vq = min(vq, vcpu_sve_max_vq(vcpu)); + vcpu_write_sys_reg(vcpu, vq - 1, ZCR_EL2); + return true; +} + /* * Architected system registers. * Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2 @@ -2688,6 +2723,9 @@ static const struct sys_reg_desc sys_reg_descs[] = { EL2_REG_VNCR(HFGITR_EL2, reset_val, 0), EL2_REG_VNCR(HACR_EL2, reset_val, 0), + { SYS_DESC(SYS_ZCR_EL2), .access = access_zcr_el2, .reset = reset_val, + .visibility = sve_el2_visibility, .reg = ZCR_EL2 }, + EL2_REG_VNCR(HCRX_EL2, reset_val, 0), EL2_REG(TTBR0_EL2, access_rw, reset_val, 0), From patchwork Thu Jun 20 16:46:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705907 Received: from out-183.mta1.migadu.com (out-183.mta1.migadu.com [95.215.58.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2BF761B3740 for ; Thu, 20 Jun 2024 16:47:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.183 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902033; cv=none; b=Ctj/1ZzzeW5O84O/y1JPLBAZq8FDGgI3R2wdsRElcni1u8xLuIJt0z7qWCGNg6pk8anKMiZVkXWbZgwYSO7Z8sg0s6P6LISjwT/3E3+r0fTKaYgGYOyiJuYxB+HtJ4nTEC3+4ybi0F6QKdjIfKh/cIzU7V9xO9wDQcvaG1emIQU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902033; c=relaxed/simple; bh=IIVYWH3bq625xeSi28YWguqieG6uzj41WZKm6Iqx/pw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AGLzTk/BLXBdzYqgD6EraNAeCiDyjZmSpRZUzBFGZsMOmmizMRhb87JRM7/jwFWPG7H5aBEkWofNmuQwfxGKj+zs9InVPQlQ3nboghl40rs3QS2Max1FB1Q0/d5y6ViqHbJ0C3/vXqBIW1ur+maX2tSb2jzY44pGXmWgJGLnIM8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=i+bKUJ0z; arc=none smtp.client-ip=95.215.58.183 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="i+bKUJ0z" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902030; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tQHZtMFMJ+3kzDmdjymdOh/PCcuUm2OaBa1yJN/FLTk=; b=i+bKUJ0ztdk8E5r7Eks59pf5XL5wyakT5ELz7nzuc4DKYwVhL78LiE6ciYttyTZpVu5RBT qJhRzUHn/YC/m82aKr6BlgL7FIWUpKxazOzb4ehn+heXq2hGCuijyQSz1IVUS5QpanGYeP Izzn1vP+ALddmdA11DW9vl33cwWLn1U= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Oliver Upton Subject: [PATCH v3 04/15] KVM: arm64: nv: Load guest hyp's ZCR into EL1 state Date: Thu, 20 Jun 2024 16:46:41 +0000 Message-ID: <20240620164653.1130714-5-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Load the guest hypervisor's ZCR_EL2 into the corresponding EL1 register when restoring SVE state, as ZCR_EL2 affects the VL in the hypervisor context. Reviewed-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/include/asm/kvm_host.h | 3 +++ arch/arm64/kvm/hyp/include/hyp/switch.h | 3 ++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 5a114720fd57..2c3eff0031eb 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -845,6 +845,9 @@ struct kvm_vcpu_arch { #define vcpu_sve_max_vq(vcpu) sve_vq_from_vl((vcpu)->arch.sve_max_vl) +#define vcpu_sve_zcr_elx(vcpu) \ + (unlikely(is_hyp_ctxt(vcpu)) ? ZCR_EL2 : ZCR_EL1) + #define vcpu_sve_state_size(vcpu) ({ \ size_t __size_ret; \ unsigned int __vcpu_vq; \ diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 428ee15dd6ae..aeb08c2d926d 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -317,7 +317,8 @@ static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu) sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2); __sve_restore_state(vcpu_sve_pffr(vcpu), &vcpu->arch.ctxt.fp_regs.fpsr); - write_sysreg_el1(__vcpu_sys_reg(vcpu, ZCR_EL1), SYS_ZCR); + + write_sysreg_el1(__vcpu_sys_reg(vcpu, vcpu_sve_zcr_elx(vcpu)), SYS_ZCR); } /* From patchwork Thu Jun 20 16:46:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705908 Received: from out-171.mta1.migadu.com (out-171.mta1.migadu.com [95.215.58.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A849E1B29CE for ; Thu, 20 Jun 2024 16:47:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902036; cv=none; b=pgByH/2adJDfOlqv1eDoz85NGt1Id/RSMsMT5+gxqfN29ZxC3gweGcLM3kFl6v8tyf22pQRQGJTKXB/X9vLwld7e7fI5oaUgW+J6A6ImyaN2QsBkgJa+jEIZMdGvrGCjoKMz7+LG+H17uHFLX5xaFCrrBBSjmcUSRvXdrbLi5Sw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902036; c=relaxed/simple; bh=IGdiuoRGOLVZqyVbifezCELLTsi1tM/xNN8J4u4u/hU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FfjOt4zdr/NlNwvz/H5t53JIENyNlTqhPhvREEpphilGiV80ZnalsFpN2pZ5Uw0uDipIOm6CDgFwhhkSYtSzt8/gpUWNsn447pTDAmnjhxj9451fn2dJAeCUV3FVttwNGGbwY69wf/zbsiVXRDJV/BAhyTOQ/PaNpt30jV0YNFs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=ado6EpIT; arc=none smtp.client-ip=95.215.58.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="ado6EpIT" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902032; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gvxRT1OHplY0Ss1dvgabPecEDiEzUnsjUcQ/6TWlwjQ=; b=ado6EpIT/iosUGrm4uDA6ukqZAZe+WcD6VOhCwnqPYXP/dRA8ryVon7bkiAQKT/QQL6BiJ fcGytQXNSVh6NY4rPx8f9NaVstdmSBywxfVSfReCuq/U7oCxfi/50yWygi8IScDI+d9zSX nM+a7CDrP1S5tjvYvNxNqUZDnd/B5zQ= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Oliver Upton Subject: [PATCH v3 05/15] KVM: arm64: nv: Save guest's ZCR_EL2 when in hyp context Date: Thu, 20 Jun 2024 16:46:42 +0000 Message-ID: <20240620164653.1130714-6-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT When running a guest hypervisor, ZCR_EL2 is an alias for the counterpart EL1 state. Reviewed-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/kvm/fpsimd.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index 1807d3a79a8a..d948f1c684ca 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -173,7 +173,13 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu) if (guest_owns_fp_regs()) { if (vcpu_has_sve(vcpu)) { - __vcpu_sys_reg(vcpu, ZCR_EL1) = read_sysreg_el1(SYS_ZCR); + u64 zcr = read_sysreg_el1(SYS_ZCR); + + /* + * If the vCPU is in the hyp context then ZCR_EL1 is + * loaded with its vEL2 counterpart. + */ + __vcpu_sys_reg(vcpu, vcpu_sve_zcr_elx(vcpu)) = zcr; /* * Restore the VL that was saved when bound to the CPU, From patchwork Thu Jun 20 16:46:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705909 Received: from out-186.mta1.migadu.com (out-186.mta1.migadu.com [95.215.58.186]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 806A01B3F30 for ; Thu, 20 Jun 2024 16:47:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.186 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902038; cv=none; b=nlVNA/tcFgIbtdYAvm4T1nI1fCJA8d90cXIJ7hJnQcXw7ChEhQFYDqehnm0QL7BG4hxmwecNdLLB4wzdqyBnOJ9x8FGmzuRlx67DBjod6aOwExDFXg9sWiKPkjyXgiwIwlP3h+hYK4+DUsPwQtBXc8QDkdsS47lPKm6S4EZ0JzM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902038; c=relaxed/simple; bh=tJlPrqW1eWQjQsPRwIX2fZ7adL8zHKb6NOP8VUWDKd0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QcqCamP72RwM0GP72bnI9qSRY8YV5Q3Qx4uPjE/e5VA9fnXR3BwpeH40UyG+zsPTNOkqkPF8mR5LzmARs7sTmV6BNnBFrtG+d4Q9bUhwb3uBbeYxLclku7UCYVkYStHIl09u8wtZAb/Vy3J+ojHuk0WmwbOnxraKtEdI18nIReY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=hwd/bytR; arc=none smtp.client-ip=95.215.58.186 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="hwd/bytR" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902033; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Qa5SQ5ZGOqhKVPfOYLLgM23ZGqX6pCTYzsPBsDZvBQE=; b=hwd/bytRo8ec6YeZyg0F6NvCN4Pmk3oiItd6ypvdXipJV1VwWz9I4pN/WwcfZGaUaUZ/Oh GZgnIukCnp1hOSvOSTywQTfY4ZC2rGzskpAwzN/pzxa36EoD1y8D9vfN83+E6MhVnas3+i T+LEUH06hJl69UynvIbMcC/WJtJMttw= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Oliver Upton Subject: [PATCH v3 06/15] KVM: arm64: nv: Use guest hypervisor's max VL when running nested guest Date: Thu, 20 Jun 2024 16:46:43 +0000 Message-ID: <20240620164653.1130714-7-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT The max VL for nested guests is additionally constrained by the max VL selected by the guest hypervisor. Use that instead of KVM's max VL when running a nested guest. Note that the guest hypervisor's ZCR_EL2 is sanitised against the VM's max VL at the time of access, so there's no additional handling required at the time of use. Reviewed-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/kvm/hyp/include/hyp/switch.h | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index aeb08c2d926d..aaad4d68b6e7 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -314,10 +314,22 @@ static bool kvm_hyp_handle_mops(struct kvm_vcpu *vcpu, u64 *exit_code) static inline void __hyp_sve_restore_guest(struct kvm_vcpu *vcpu) { + /* + * The vCPU's saved SVE state layout always matches the max VL of the + * vCPU. Start off with the max VL so we can load the SVE state. + */ sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL2); __sve_restore_state(vcpu_sve_pffr(vcpu), &vcpu->arch.ctxt.fp_regs.fpsr); + /* + * The effective VL for a VM could differ from the max VL when running a + * nested guest, as the guest hypervisor could select a smaller VL. Slap + * that into hardware before wrapping up. + */ + if (vcpu_has_nv(vcpu) && !is_hyp_ctxt(vcpu)) + sve_cond_update_zcr_vq(__vcpu_sys_reg(vcpu, ZCR_EL2), SYS_ZCR_EL2); + write_sysreg_el1(__vcpu_sys_reg(vcpu, vcpu_sve_zcr_elx(vcpu)), SYS_ZCR); } From patchwork Thu Jun 20 16:46:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705910 Received: from out-178.mta1.migadu.com (out-178.mta1.migadu.com [95.215.58.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2E351B3F08 for ; Thu, 20 Jun 2024 16:47:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902038; cv=none; b=MBDYFUwJL0rhTAkKVgW1Nyyrfg0F9/etFI2elIgv/JmvChI8gEDiT1lnDKeZAXTZhvDmugoj0oA1FNicfVwWWpByqtf76p41ctmrD0DEvczShXCPK6IXOzonos80EYfHg4LTn3N34vlMaOUSlNsein2k+IsWk6d1q9bD5pMw76A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902038; c=relaxed/simple; bh=MNEsSbek93dEnKdVXcnjLptk0/WvLnHiuamXW0WR44Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Lldtu1zkoj9LLRBOuOnqju1BZWDiNcMVGdP/KxFRYi7rIKSYo9OpWhmBxGudlE/9hWLTNTaO5avE9aTzywI6rmbHoCJEieHj/zW3tMm4Atvm9PoHVt3h6IdZq06m2Hrw302F8vQvEwmpKbIpT5imPdbgn7jUikEuYUd90kqyeWk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=rsK+rQQ+; arc=none smtp.client-ip=95.215.58.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="rsK+rQQ+" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902035; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OrPYUpaXTuyZ913QZ9yyjlNewzvBDOPiUFlx9T7Nkio=; b=rsK+rQQ+9aA/3JKjxRutx3Pi7Aun4Y0Akv0UNklkxhdcuKVFgZjGzKCsO00McNlrT+/4xZ 0SON5H7smqXy31KH8a3mAcVSDz+JJ1osKXrW2/fENUzd1WLOvx8GfJasygRMsv7l2DAKBA wdIWNT8+x3Ewn3g2O5RSVLXTchla9mU= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Oliver Upton Subject: [PATCH v3 07/15] KVM: arm64: nv: Ensure correct VL is loaded before saving SVE state Date: Thu, 20 Jun 2024 16:46:44 +0000 Message-ID: <20240620164653.1130714-8-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT It is possible that the guest hypervisor has selected a smaller VL than the maximum for its nested guest. As such, ZCR_EL2 may be configured for a different VL when exiting a nested guest. Set ZCR_EL2 (via the EL1 alias) to the maximum VL for the VM before saving SVE state as the SVE save area is dimensioned by the max VL. Reviewed-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/kvm/fpsimd.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/fpsimd.c b/arch/arm64/kvm/fpsimd.c index d948f1c684ca..947486a111e1 100644 --- a/arch/arm64/kvm/fpsimd.c +++ b/arch/arm64/kvm/fpsimd.c @@ -190,11 +190,14 @@ void kvm_arch_vcpu_put_fp(struct kvm_vcpu *vcpu) * Note that this means that at guest exit ZCR_EL1 is * not necessarily the same as on guest entry. * - * Restoring the VL isn't needed in VHE mode since - * ZCR_EL2 (accessed via ZCR_EL1) would fulfill the same - * role when doing the save from EL2. + * ZCR_EL2 holds the guest hypervisor's VL when running + * a nested guest, which could be smaller than the + * max for the vCPU. Similar to above, we first need to + * switch to a VL consistent with the layout of the + * vCPU's SVE state. KVM support for NV implies VHE, so + * using the ZCR_EL1 alias is safe. */ - if (!has_vhe()) + if (!has_vhe() || (vcpu_has_nv(vcpu) && !is_hyp_ctxt(vcpu))) sve_cond_update_zcr_vq(vcpu_sve_max_vq(vcpu) - 1, SYS_ZCR_EL1); } From patchwork Thu Jun 20 16:46:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705911 Received: from out-186.mta1.migadu.com (out-186.mta1.migadu.com [95.215.58.186]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C20EC1B47B2 for ; Thu, 20 Jun 2024 16:47:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.186 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902040; cv=none; b=dyA0f3Rsb096W9vX9K9aUVuO5uysabnVMKF05VVfLKaC9osgCemnTqdtBgd7blXy3mG6t8Lbb0wSV1nA8FY0JNhzEy94Re8BGgjpJiprJ+X+ttADEvJErmurZNriHZJ8C4b79Ol2Py+7BF8WZVhXsp49/SJgDDKgXbLdr4oIbkY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902040; c=relaxed/simple; bh=eHFcSnx/FrqvvY+4VU8ze04GVmd2fbF4l7dDs6USX/U=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kjI58B2WSNpzO0sPuZ4t2a1PXCy88bKp6qiom73cbByxsP4nXGOskprZTt1exVYfIr99M2GmjPKR7tQWKgBbb9Czbb0gYVqfLoFeEwtvSdymL9NC0/4K+m3EK8lGeOYi+NbILrIBcGXVbTVZNK3yh9kfhUuuKhL0OCFQCgYfwxg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=xNNpJcci; arc=none smtp.client-ip=95.215.58.186 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="xNNpJcci" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902037; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cStQMLZGUlCbmuI60XeY0OY25KBZNZcrijUKFbj3MYQ=; b=xNNpJcci+IX/dKs2+h8LpwC8LyofKfTBiVRoLC0USlzPbHawItPjiRUAmyBNcwiOaz1SDt PRKwlY4AQMYs7DcF0S7Xg4eUt6hJEl+zkXrajRewuJ8m234VLszNqc6Lp7oDXiKdx1GePQ 1qNIX14kE/2+3pWBH6kjhI1uZOYI+aw= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Oliver Upton Subject: [PATCH v3 08/15] KVM: arm64: Spin off helper for programming CPTR traps Date: Thu, 20 Jun 2024 16:46:45 +0000 Message-ID: <20240620164653.1130714-9-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT A subsequent change to KVM will add preliminary support for merging a guest hypervisor's CPTR traps with that of KVM. Prepare by spinning off a new helper for managing CPTR traps. Avoid reading CPACR_EL1 for the baseline trap config, and start off with the most restrictive set of traps that is subsequently relaxed. Reviewed-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/kvm/hyp/vhe/switch.c | 49 ++++++++++++++++----------------- 1 file changed, 24 insertions(+), 25 deletions(-) diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index d7af5f46f22a..697253673d7b 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -65,6 +65,29 @@ static u64 __compute_hcr(struct kvm_vcpu *vcpu) return hcr | (__vcpu_sys_reg(vcpu, HCR_EL2) & ~NV_HCR_GUEST_EXCLUDE); } +static void __activate_cptr_traps(struct kvm_vcpu *vcpu) +{ + /* + * With VHE (HCR.E2H == 1), accesses to CPACR_EL1 are routed to + * CPTR_EL2. In general, CPACR_EL1 has the same layout as CPTR_EL2, + * except for some missing controls, such as TAM. + * In this case, CPTR_EL2.TAM has the same position with or without + * VHE (HCR.E2H == 1) which allows us to use here the CPTR_EL2.TAM + * shift value for trapping the AMU accesses. + */ + u64 val = CPACR_ELx_TTA | CPTR_EL2_TAM; + + if (guest_owns_fp_regs()) { + val |= CPACR_ELx_FPEN; + if (vcpu_has_sve(vcpu)) + val |= CPACR_ELx_ZEN; + } else { + __activate_traps_fpsimd32(vcpu); + } + + write_sysreg(val, cpacr_el1); +} + static void __activate_traps(struct kvm_vcpu *vcpu) { u64 val; @@ -91,31 +114,7 @@ static void __activate_traps(struct kvm_vcpu *vcpu) } } - val = read_sysreg(cpacr_el1); - val |= CPACR_ELx_TTA; - val &= ~(CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN | - CPACR_EL1_SMEN_EL0EN | CPACR_EL1_SMEN_EL1EN); - - /* - * With VHE (HCR.E2H == 1), accesses to CPACR_EL1 are routed to - * CPTR_EL2. In general, CPACR_EL1 has the same layout as CPTR_EL2, - * except for some missing controls, such as TAM. - * In this case, CPTR_EL2.TAM has the same position with or without - * VHE (HCR.E2H == 1) which allows us to use here the CPTR_EL2.TAM - * shift value for trapping the AMU accesses. - */ - - val |= CPTR_EL2_TAM; - - if (guest_owns_fp_regs()) { - if (vcpu_has_sve(vcpu)) - val |= CPACR_EL1_ZEN_EL0EN | CPACR_EL1_ZEN_EL1EN; - } else { - val &= ~(CPACR_EL1_FPEN_EL0EN | CPACR_EL1_FPEN_EL1EN); - __activate_traps_fpsimd32(vcpu); - } - - write_sysreg(val, cpacr_el1); + __activate_cptr_traps(vcpu); write_sysreg(__this_cpu_read(kvm_hyp_vector), vbar_el1); } From patchwork Thu Jun 20 16:46:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705912 Received: from out-182.mta1.migadu.com (out-182.mta1.migadu.com [95.215.58.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA8381B47CD for ; Thu, 20 Jun 2024 16:47:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902043; cv=none; b=kvaTtoly97q3wQWlRHN9VXjgls052Qlu0Cni87y8ZzmGbCKShk6RwLqF2yAtcU0rRlyoBhS16g3FvdzMFmP+WcQ8ADs1YlZvPL0Wwy9UfPVr8WlIgFr9f+ADajEFegJcrO3zCPrdkJf23sxOllQQO5TzwSUC0dBqrwDNaBe5mjI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902043; c=relaxed/simple; bh=VC3cAS1vVLoMoF5Tcsry/hVzapF0FahnZnOna4JN5yI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X9o0Tn1mMV6xhHKuNiy6+/94I6uU7QfJ6uLS6W2PSswKdfjH7ntXNgc77B0APWLmS8YsOA0zLQFasEouEcaJ7+qO34yoNyIhfPby1QpHZsMpi7Cx15h2p9gKjnAlJoCteLWUx0Zrn6kbiPWVGqnjJmcbWZnXBdqrAktO0TSH/4o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=a+ytgbh/; arc=none smtp.client-ip=95.215.58.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="a+ytgbh/" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902038; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VqLkG/Vh8Bov5AOtu1PNtbm0dwVwhXMlOkximXK5PoU=; b=a+ytgbh/ArUlnySWyK44RFFDYDICgO+ixGaZ5/s5rb7gCMaDmflmBL3kiDUaHM/osLw1gT +7v0IybqQ/Uc+lHhBVUKthykeRtkiBON81Mo9lWY4idY5/hjQvu9aHIAD8fQS7QIQCLtBs kGlFnxqlpRfQljowqXrnAKmLCecXdAQ= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Oliver Upton Subject: [PATCH v3 09/15] KVM: arm64: nv: Handle CPACR_EL1 traps Date: Thu, 20 Jun 2024 16:46:46 +0000 Message-ID: <20240620164653.1130714-10-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT From: Marc Zyngier Handle CPACR_EL1 accesses when running a VHE guest. In order to limit the cost of the emulation, implement it ass a shallow exit. In the other cases: - this is a nVHE L1 which will write to memory, and we don't trap - this is a L2 guest: * the L1 has CPTR_EL2.TCPAC==0, and the L2 has direct register access * the L1 has CPTR_EL2.TCPAC==1, and the L2 will trap, but the handling is defered to the general handling for forwarding Signed-off-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/kvm/hyp/vhe/switch.c | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 697253673d7b..564655277594 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -261,10 +261,40 @@ static bool kvm_hyp_handle_eret(struct kvm_vcpu *vcpu, u64 *exit_code) return true; } +static bool kvm_hyp_handle_cpacr_el1(struct kvm_vcpu *vcpu, u64 *exit_code) +{ + u64 esr = kvm_vcpu_get_esr(vcpu); + int rt; + + if (!is_hyp_ctxt(vcpu) || esr_sys64_to_sysreg(esr) != SYS_CPACR_EL1) + return false; + + rt = kvm_vcpu_sys_get_rt(vcpu); + + if ((esr & ESR_ELx_SYS64_ISS_DIR_MASK) == ESR_ELx_SYS64_ISS_DIR_READ) { + vcpu_set_reg(vcpu, rt, __vcpu_sys_reg(vcpu, CPTR_EL2)); + } else { + vcpu_write_sys_reg(vcpu, vcpu_get_reg(vcpu, rt), CPTR_EL2); + __activate_cptr_traps(vcpu); + } + + __kvm_skip_instr(vcpu); + + return true; +} + +static bool kvm_hyp_handle_sysreg_vhe(struct kvm_vcpu *vcpu, u64 *exit_code) +{ + if (kvm_hyp_handle_cpacr_el1(vcpu, exit_code)) + return true; + + return kvm_hyp_handle_sysreg(vcpu, exit_code); +} + static const exit_handler_fn hyp_exit_handlers[] = { [0 ... ESR_ELx_EC_MAX] = NULL, [ESR_ELx_EC_CP15_32] = kvm_hyp_handle_cp15_32, - [ESR_ELx_EC_SYS64] = kvm_hyp_handle_sysreg, + [ESR_ELx_EC_SYS64] = kvm_hyp_handle_sysreg_vhe, [ESR_ELx_EC_SVE] = kvm_hyp_handle_fpsimd, [ESR_ELx_EC_FP_ASIMD] = kvm_hyp_handle_fpsimd, [ESR_ELx_EC_IABT_LOW] = kvm_hyp_handle_iabt_low, From patchwork Thu Jun 20 16:46:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705913 Received: from out-189.mta1.migadu.com (out-189.mta1.migadu.com [95.215.58.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F6F41B3F08 for ; Thu, 20 Jun 2024 16:47:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.189 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902043; cv=none; b=pZaFxRUQFFj40KtXlfGVuI2601UxGrqEl8XAHYg1QJBYf3FIlupq+FawEjjX/KSZ2vS++30AovAsDLhk1hePdRbwFJu07f+5BTJBG/CjEgAgUd33+ttKZaiclOxtqfRIhwN2B3QAAqdNTVW2TlplmelzV6nUN0il4zXzplh+cU8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902043; c=relaxed/simple; bh=zkZovDalC9gFu65wm4cMj/WQvuSt5whQvWU2OVTRS5c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ki+w30Zh2MxUw2bQi/BDJdoqNU+hz2wxSbj4axhAROl2nPSW6idqf3rtuJSIzhlXyFMwvu9ZtS0Kv4Kk9nXoMHf4rdxxRojWpLI5122/eHG33f6J9hNbyHF+6liaPhSW3iY78ZClgzI52g9A7DxIvnK+wQVInmHmyr9boV/LlVM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=ecpOFhVw; arc=none smtp.client-ip=95.215.58.189 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="ecpOFhVw" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902040; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y6NlTN6bjYXUbwX0JsU1ZMZzb/6fyUage7s/Df7QDZA=; b=ecpOFhVw+w+ZaKG+veiSJgm6v1xMv/9BEjHa/xE66TjqSkiEX9w3tKqbFVrsp5Vepdc3TP FZnIPZHQGTNQy7voMWucvvmWBnL2lGcCeOFO1B5/Li8bvojB7HDD+WPnMNQpODy7v1O612 h8lECKZYiPTP/Dp8ve9uqxOUixWps9Q= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Oliver Upton Subject: [PATCH v3 10/15] KVM: arm64: nv: Load guest FP state for ZCR_EL2 trap Date: Thu, 20 Jun 2024 16:46:47 +0000 Message-ID: <20240620164653.1130714-11-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Round out the ZCR_EL2 gymnastics by loading SVE state in the fast path when the guest hypervisor tries to access SVE state. Reviewed-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/kvm/hyp/include/hyp/switch.h | 4 ++++ arch/arm64/kvm/hyp/vhe/switch.c | 27 +++++++++++++++++++++++++ 2 files changed, 31 insertions(+) diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index aaad4d68b6e7..ad8dec0b450b 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -358,6 +358,10 @@ static bool kvm_hyp_handle_fpsimd(struct kvm_vcpu *vcpu, u64 *exit_code) if (guest_hyp_fpsimd_traps_enabled(vcpu)) return false; break; + case ESR_ELx_EC_SYS64: + if (WARN_ON_ONCE(!is_hyp_ctxt(vcpu))) + return false; + fallthrough; case ESR_ELx_EC_SVE: if (!sve_guest) return false; diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 564655277594..925de4b4efd2 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -283,11 +283,38 @@ static bool kvm_hyp_handle_cpacr_el1(struct kvm_vcpu *vcpu, u64 *exit_code) return true; } +static bool kvm_hyp_handle_zcr_el2(struct kvm_vcpu *vcpu, u64 *exit_code) +{ + u32 sysreg = esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu)); + + if (!vcpu_has_nv(vcpu)) + return false; + + if (sysreg != SYS_ZCR_EL2) + return false; + + if (guest_owns_fp_regs()) + return false; + + /* + * ZCR_EL2 traps are handled in the slow path, with the expectation + * that the guest's FP context has already been loaded onto the CPU. + * + * Load the guest's FP context and unconditionally forward to the + * slow path for handling (i.e. return false). + */ + kvm_hyp_handle_fpsimd(vcpu, exit_code); + return false; +} + static bool kvm_hyp_handle_sysreg_vhe(struct kvm_vcpu *vcpu, u64 *exit_code) { if (kvm_hyp_handle_cpacr_el1(vcpu, exit_code)) return true; + if (kvm_hyp_handle_zcr_el2(vcpu, exit_code)) + return true; + return kvm_hyp_handle_sysreg(vcpu, exit_code); } From patchwork Thu Jun 20 16:46:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705914 Received: from out-187.mta1.migadu.com (out-187.mta1.migadu.com [95.215.58.187]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 320AA1B4C24 for ; Thu, 20 Jun 2024 16:47:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.187 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902046; cv=none; b=So6LSeHB0Th8NrM9kM+nUMHwV5K5FxlC2SsmIJyocc4nQwSbbxGlP7O/y56REn+/Zm1CkvNs/Qjum+OkMt/U/gSlqaF1Y3dzi23wTt82t4rIo+iAbcJg+F40/mNpHHMLWC25Z+wmvR6Tyv7nR0fUjkcPTH4vI+yFPGMtrKiSAuw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902046; c=relaxed/simple; bh=C5LEmrBiHT1xTeApGrYyneEYQCpGBT/Ty7lSjeiHStA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Zd6pcn9NDMXD3+IsD9ph5v4JDWDsvX7K/tgXqCjQWsaGm08WSeTotZK8jiDCRvtUHESAPsRuHts0PPUIOpudOs8CliM0jLV+1UPlpJ30EDyZLhtHBJAzacYFmzKUr9u03G41a1PlRHhuRsdcO0CxN0yJfrcf9Dqhi6t5/jQYuUQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=alz0b7Wp; arc=none smtp.client-ip=95.215.58.187 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="alz0b7Wp" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902042; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sHh2kb1Z2Qr4oSw9MwaKbtA4JNckM/+p9o6iCRwIjxg=; b=alz0b7WpkSSpddQOfhfzSGKS499wvF8Z7tQOLbIx1gKByOgmXA9Z6REO1oDwnpjcs/Uf2j EuEMumEnZRREdD+hJqOKp8NJhRsh/OckI2lFcCNi4Au4fb4Sz5Kt2sUT2NHlwB+/03aIG6 xkv1KZUD9bGtm5BnpveWuFQb8YLXwq0= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Oliver Upton Subject: [PATCH v3 11/15] KVM: arm64: nv: Honor guest hypervisor's FP/SVE traps in CPTR_EL2 Date: Thu, 20 Jun 2024 16:46:48 +0000 Message-ID: <20240620164653.1130714-12-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Start folding the guest hypervisor's FP/SVE traps into the value programmed in hardware. Note that as of writing this is dead code, since KVM does a full put() / load() for every nested exception boundary which saves + flushes the FP/SVE state. However, this will become useful when we can keep the guest's FP/SVE state alive across a nested exception boundary and the host no longer needs to conservatively program traps. Reviewed-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/kvm/hyp/vhe/switch.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 925de4b4efd2..b0b1935a3626 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -67,6 +67,8 @@ static u64 __compute_hcr(struct kvm_vcpu *vcpu) static void __activate_cptr_traps(struct kvm_vcpu *vcpu) { + u64 cptr; + /* * With VHE (HCR.E2H == 1), accesses to CPACR_EL1 are routed to * CPTR_EL2. In general, CPACR_EL1 has the same layout as CPTR_EL2, @@ -85,6 +87,35 @@ static void __activate_cptr_traps(struct kvm_vcpu *vcpu) __activate_traps_fpsimd32(vcpu); } + /* + * Layer the guest hypervisor's trap configuration on top of our own if + * we're in a nested context. + */ + if (!vcpu_has_nv(vcpu) || is_hyp_ctxt(vcpu)) + goto write; + + cptr = vcpu_sanitised_cptr_el2(vcpu); + + /* + * Pay attention, there's some interesting detail here. + * + * The CPTR_EL2.xEN fields are 2 bits wide, although there are only two + * meaningful trap states when HCR_EL2.TGE = 0 (running a nested guest): + * + * - CPTR_EL2.xEN = x0, traps are enabled + * - CPTR_EL2.xEN = x1, traps are disabled + * + * In other words, bit[0] determines if guest accesses trap or not. In + * the interest of simplicity, clear the entire field if the guest + * hypervisor has traps enabled to dispel any illusion of something more + * complicated taking place. + */ + if (!(SYS_FIELD_GET(CPACR_ELx, FPEN, cptr) & BIT(0))) + val &= ~CPACR_ELx_FPEN; + if (!(SYS_FIELD_GET(CPACR_ELx, ZEN, cptr) & BIT(0))) + val &= ~CPACR_ELx_ZEN; + +write: write_sysreg(val, cpacr_el1); } From patchwork Thu Jun 20 16:46:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705915 Received: from out-189.mta1.migadu.com (out-189.mta1.migadu.com [95.215.58.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B17621B4C2C for ; Thu, 20 Jun 2024 16:47:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.189 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902049; cv=none; b=kIf01BticrgkxRKLQUYR6ATGcEvCyECoXYiVcM+3bdSRgLtY3aahChAvsRNoHATJjKmkLJZBLgHWfLvvmXajf9x82yRXnHdWx3qX+jmXWlJdLu+8wZx2XqMhmy6Nndhq1J89T+rz23EOpuqUI9n5bMf9+MynuaKCH/ICaomqtfE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902049; c=relaxed/simple; bh=AB7fFKaZ4IzZn209XJyuszZvhI37LCYwB3JZOQnIwPc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EF+kvf/QIYwy+ExIwwIbELeMhypluA5uhTJM2GpcEqDokeEt3t26QYBSpn01D+/meb1jIxWOb8mVdyBuh5TYKTB7e/xf6a4GuNhauXnG44SdtFptfJoZMYdUXjcTUDQdcGcmZjrT+Jwzij9a6PPqbiY+C0YWoM36dCvm1TA2OpU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=skqt5qEU; arc=none smtp.client-ip=95.215.58.189 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="skqt5qEU" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902045; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NAN7p6f7rgTiinz1pWiC7BoeIE5/rA20oN+kCgrXMGk=; b=skqt5qEUKdD4gmmJdjkRja67McNVpEwCIsA8Mpx3yq8Pvz5TOzW8sjT049KFybDEC+tzgp W+dLXD1o9oaF5HLSgQf12XJUZB1ioNTovPQE6+5KIXBBgj/5jiQk9VF9/IDTF/pltYQLBe 8wMOhgqCzn8P2hoGsEb8iZP4jNHiaCU= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Oliver Upton Subject: [PATCH v3 12/15] KVM: arm64: nv: Add TCPAC/TTA to CPTR->CPACR conversion helper Date: Thu, 20 Jun 2024 16:46:49 +0000 Message-ID: <20240620164653.1130714-13-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT From: Marc Zyngier We are missing the propagation of CPTR_EL2.{TCPAC,TTA} into the CPACR format. Make sure we preserve these bits. Signed-off-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/include/asm/kvm_nested.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kvm_nested.h b/arch/arm64/include/asm/kvm_nested.h index 5e0ab0596246..a11ed921d4e0 100644 --- a/arch/arm64/include/asm/kvm_nested.h +++ b/arch/arm64/include/asm/kvm_nested.h @@ -32,7 +32,7 @@ static inline u64 translate_tcr_el2_to_tcr_el1(u64 tcr) static inline u64 translate_cptr_el2_to_cpacr_el1(u64 cptr_el2) { - u64 cpacr_el1 = 0; + u64 cpacr_el1 = CPACR_ELx_RES1; if (cptr_el2 & CPTR_EL2_TTA) cpacr_el1 |= CPACR_ELx_TTA; @@ -41,6 +41,8 @@ static inline u64 translate_cptr_el2_to_cpacr_el1(u64 cptr_el2) if (!(cptr_el2 & CPTR_EL2_TZ)) cpacr_el1 |= CPACR_ELx_ZEN; + cpacr_el1 |= cptr_el2 & (CPTR_EL2_TCPAC | CPTR_EL2_TAM); + return cpacr_el1; } From patchwork Thu Jun 20 16:46:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705916 Received: from out-183.mta1.migadu.com (out-183.mta1.migadu.com [95.215.58.183]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8094B1B29C0 for ; Thu, 20 Jun 2024 16:47:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.183 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902050; cv=none; b=Vzm3QaV8s6iSN1dCeHJvorujPc7/+ptw0ZJw0oLAHxLRi1Iwi2Q5OmZCS+MwzNszo79lwtae08nsVEFuY25RoaegsRb8AnWqBHgR3XogByHMnAwaGUeeIUtS/6LZsaq306/oMxPumibW0U+8poew3TPw/b42YU75pjKE+WMpap4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902050; c=relaxed/simple; bh=1yg5AYELubECKlo6Eo+bOKuSCabL7JbyyNuiHWaHuKs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oynqt1Wt64f/KAmoRSz0HxdvOwslDKG4gCjoszy4/q7A2zGaoIt+F5l1zPf8LUxTjmY0iXk20DI+9rZOjFWRBEwj10NL/81VDwhIE6xXFkKuEm4BH9ly4kfvRy3mrId+DzYPian3QeNy3XBAhTFy0FfQ1LNM6HJCuFYynXVtDuU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=KxgRQBMW; arc=none smtp.client-ip=95.215.58.183 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="KxgRQBMW" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902046; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pmtAAR+VAB2Y/FTza7jwIQlFzzrt1sW9kGZMKbx09nI=; b=KxgRQBMW5FNtkE0N7vAipJ+aFRuNn/hNERbLuTOyH3eyQ3m7nn0Lu/EN/LfhGCgu5CW7U5 aDrXklfGs8abRzRUbqttsoyNSy1EIaBcVRXx3qeNacOJDtq5iRM/u/DOgc6UNMnwdaz21p SXFBKMLVH1m6TynaTSmgaHiDymEO9kM= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Oliver Upton Subject: [PATCH v3 13/15] KVM: arm64: nv: Add trap description for CPTR_EL2 Date: Thu, 20 Jun 2024 16:46:50 +0000 Message-ID: <20240620164653.1130714-14-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT From: Marc Zyngier Add trap description for CPTR_EL2.{TCPAC,TAM,E0POE,TTA}. TTA is a bit annoying as it changes location depending on E2H. This forces us to add yet another "complex" trap condition. Signed-off-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/kvm/emulate-nested.c | 91 +++++++++++++++++++++++++++++++++ 1 file changed, 91 insertions(+) diff --git a/arch/arm64/kvm/emulate-nested.c b/arch/arm64/kvm/emulate-nested.c index 72d733c74a38..61e6b97c3e25 100644 --- a/arch/arm64/kvm/emulate-nested.c +++ b/arch/arm64/kvm/emulate-nested.c @@ -79,6 +79,10 @@ enum cgt_group_id { CGT_MDCR_E2TB, CGT_MDCR_TDCC, + CGT_CPACR_E0POE, + CGT_CPTR_TAM, + CGT_CPTR_TCPAC, + /* * Anything after this point is a combination of coarse trap * controls, which must all be evaluated to decide what to do. @@ -106,6 +110,8 @@ enum cgt_group_id { CGT_CNTHCTL_EL1PCTEN = __COMPLEX_CONDITIONS__, CGT_CNTHCTL_EL1PTEN, + CGT_CPTR_TTA, + /* Must be last */ __NR_CGT_GROUP_IDS__ }; @@ -345,6 +351,24 @@ static const struct trap_bits coarse_trap_bits[] = { .mask = MDCR_EL2_TDCC, .behaviour = BEHAVE_FORWARD_ANY, }, + [CGT_CPACR_E0POE] = { + .index = CPTR_EL2, + .value = CPACR_ELx_E0POE, + .mask = CPACR_ELx_E0POE, + .behaviour = BEHAVE_FORWARD_ANY, + }, + [CGT_CPTR_TAM] = { + .index = CPTR_EL2, + .value = CPTR_EL2_TAM, + .mask = CPTR_EL2_TAM, + .behaviour = BEHAVE_FORWARD_ANY, + }, + [CGT_CPTR_TCPAC] = { + .index = CPTR_EL2, + .value = CPTR_EL2_TCPAC, + .mask = CPTR_EL2_TCPAC, + .behaviour = BEHAVE_FORWARD_ANY, + }, }; #define MCB(id, ...) \ @@ -410,12 +434,26 @@ static enum trap_behaviour check_cnthctl_el1pten(struct kvm_vcpu *vcpu) return BEHAVE_FORWARD_ANY; } +static enum trap_behaviour check_cptr_tta(struct kvm_vcpu *vcpu) +{ + u64 val = __vcpu_sys_reg(vcpu, CPTR_EL2); + + if (!vcpu_el2_e2h_is_set(vcpu)) + val = translate_cptr_el2_to_cpacr_el1(val); + + if (val & CPACR_ELx_TTA) + return BEHAVE_FORWARD_ANY; + + return BEHAVE_HANDLE_LOCALLY; +} + #define CCC(id, fn) \ [id - __COMPLEX_CONDITIONS__] = fn static const complex_condition_check ccc[] = { CCC(CGT_CNTHCTL_EL1PCTEN, check_cnthctl_el1pcten), CCC(CGT_CNTHCTL_EL1PTEN, check_cnthctl_el1pten), + CCC(CGT_CPTR_TTA, check_cptr_tta), }; /* @@ -1000,6 +1038,59 @@ static const struct encoding_to_trap_config encoding_to_cgt[] __initconst = { SR_TRAP(SYS_TRBPTR_EL1, CGT_MDCR_E2TB), SR_TRAP(SYS_TRBSR_EL1, CGT_MDCR_E2TB), SR_TRAP(SYS_TRBTRG_EL1, CGT_MDCR_E2TB), + SR_TRAP(SYS_CPACR_EL1, CGT_CPTR_TCPAC), + SR_TRAP(SYS_AMUSERENR_EL0, CGT_CPTR_TAM), + SR_TRAP(SYS_AMCFGR_EL0, CGT_CPTR_TAM), + SR_TRAP(SYS_AMCGCR_EL0, CGT_CPTR_TAM), + SR_TRAP(SYS_AMCNTENCLR0_EL0, CGT_CPTR_TAM), + SR_TRAP(SYS_AMCNTENCLR1_EL0, CGT_CPTR_TAM), + SR_TRAP(SYS_AMCNTENSET0_EL0, CGT_CPTR_TAM), + SR_TRAP(SYS_AMCNTENSET1_EL0, CGT_CPTR_TAM), + SR_TRAP(SYS_AMCR_EL0, CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR0_EL0(0), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR0_EL0(1), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR0_EL0(2), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR0_EL0(3), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(0), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(1), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(2), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(3), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(4), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(5), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(6), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(7), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(8), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(9), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(10), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(11), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(12), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(13), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(14), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVCNTR1_EL0(15), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER0_EL0(0), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER0_EL0(1), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER0_EL0(2), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER0_EL0(3), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(0), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(1), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(2), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(3), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(4), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(5), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(6), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(7), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(8), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(9), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(10), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(11), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(12), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(13), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(14), CGT_CPTR_TAM), + SR_TRAP(SYS_AMEVTYPER1_EL0(15), CGT_CPTR_TAM), + SR_TRAP(SYS_POR_EL0, CGT_CPACR_E0POE), + /* op0=2, op1=1, and CRn<0b1000 */ + SR_RANGE_TRAP(sys_reg(2, 1, 0, 0, 0), + sys_reg(2, 1, 7, 15, 7), CGT_CPTR_TTA), SR_TRAP(SYS_CNTP_TVAL_EL0, CGT_CNTHCTL_EL1PTEN), SR_TRAP(SYS_CNTP_CVAL_EL0, CGT_CNTHCTL_EL1PTEN), SR_TRAP(SYS_CNTP_CTL_EL0, CGT_CNTHCTL_EL1PTEN), From patchwork Thu Jun 20 16:46:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705917 Received: from out-186.mta1.migadu.com (out-186.mta1.migadu.com [95.215.58.186]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C90E1B4C35 for ; Thu, 20 Jun 2024 16:47:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.186 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902051; cv=none; b=Za+naNRRNTj4fFAwFbWZQXdhxbCvXEvr16swSk1yNXe4uETKvja+IYM9umTwLrvbsL0upR+FXZnQ7j2DQ2sPiAB405CEN2df3esmnBX3v4xPtbCdB1e1urLt/o9ATfkAhUIPYG6mYAz/6arAEVVWW9xAti/2a9KlCDKh6GE9LKE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902051; c=relaxed/simple; bh=JhlcTE0mw0HU+iuzSo6ZzaFOPAAQYm8aEXGIEqANUU8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VQYHZUePMj5kgOUpAbv3/4NM4E1qoLTeAXTAK/4rw+ZqG+dg5mTBKqHiaow8Vy1P8c5TryDv9gpYfGhMtBUPdC8Hkq4TyxOsOO4IN5DMXFTRVlsZ0TPj5R42hPzFT436pDeIneofYN1UHy8cAL4mAwllMsaGKZkTfSnlKSzULQk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=mpVMlspE; arc=none smtp.client-ip=95.215.58.186 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="mpVMlspE" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902048; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XgzC9jsNkREHCUhV560SZO3nTwTBAoSv9nxx7koTnRo=; b=mpVMlspEn7MfQU+uw/7tjCUhzacdSE837YPJ122MIE681FjCo4rrYtdw9ngfsE1ljDQoir apP2T4uBARCJhs1EMp40Zng7RU0x2JccTQOVFLa1D40ZekkkjU//0YAw3BgZPywoNdaxRn DF2xsZdFxT5DnmSZH7tn+8slTdL842g= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Oliver Upton Subject: [PATCH v3 14/15] KVM: arm64: nv: Add additional trap setup for CPTR_EL2 Date: Thu, 20 Jun 2024 16:46:51 +0000 Message-ID: <20240620164653.1130714-15-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT From: Marc Zyngier We need to teach KVM a couple of new tricks. CPTR_EL2 and its VHE accessor CPACR_EL1 need to be handled specially: - CPACR_EL1 is trapped on VHE so that we can track the TCPAC and TTA bits - CPTR_EL2.{TCPAC,E0POE} are propagated from L1 to L2 Signed-off-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/kvm/hyp/vhe/switch.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index b0b1935a3626..d30fee61fb17 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -87,11 +87,23 @@ static void __activate_cptr_traps(struct kvm_vcpu *vcpu) __activate_traps_fpsimd32(vcpu); } + if (!vcpu_has_nv(vcpu)) + goto write; + + /* + * The architecture is a bit crap (what a surprise): an EL2 guest + * writing to CPTR_EL2 via CPACR_EL1 can't set any of TCPAC or TTA, + * as they are RES0 in the guest's view. To work around it, trap the + * sucker using the very same bit it can't set... + */ + if (vcpu_el2_e2h_is_set(vcpu) && is_hyp_ctxt(vcpu)) + val |= CPTR_EL2_TCPAC; + /* * Layer the guest hypervisor's trap configuration on top of our own if * we're in a nested context. */ - if (!vcpu_has_nv(vcpu) || is_hyp_ctxt(vcpu)) + if (is_hyp_ctxt(vcpu)) goto write; cptr = vcpu_sanitised_cptr_el2(vcpu); @@ -115,6 +127,11 @@ static void __activate_cptr_traps(struct kvm_vcpu *vcpu) if (!(SYS_FIELD_GET(CPACR_ELx, ZEN, cptr) & BIT(0))) val &= ~CPACR_ELx_ZEN; + if (kvm_has_feat(vcpu->kvm, ID_AA64MMFR3_EL1, S2POE, IMP)) + val |= cptr & CPACR_ELx_E0POE; + + val |= cptr & CPTR_EL2_TCPAC; + write: write_sysreg(val, cpacr_el1); } From patchwork Thu Jun 20 16:46:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 13705918 Received: from out-173.mta1.migadu.com (out-173.mta1.migadu.com [95.215.58.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 191D31B4C48 for ; Thu, 20 Jun 2024 16:47:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902053; cv=none; b=h8nidUxVUBcylCz93bvUOGNVrQsxPr9hLzQgjH7YfGD0wlRaZsFAWBtt3b2ja1PCwXMnorVCr/xenF/ldEaYTdt8ROlrdvhOnln3h049owoIfHLQBfF7iEKBPr3HGPIWOBEeRRQnyB9ckncVDjEsy/dNQBR0IUj9g94UgOnqM/Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718902053; c=relaxed/simple; bh=eEcShQr/X3WmlCmwaIzpGFc3Jk0ZwrbBaOMtiuwG4HA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gLUG4BU6PWeWblZ1QlcBK1Q9mhrCm7mCs2TgW2GC7sEZzM0WKhHxt7hsK0luhUXkn1mj0CHYkQ914R56IDrEoq5Qs9HibI8FaHSvGaiMe5p6wIzLODFKYbbk4q9V+1SNIVC6LXgTjZi6KBANvJvGnD8pAehvfRqSDsfde9KJ0XQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=U4nWGIaR; arc=none smtp.client-ip=95.215.58.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="U4nWGIaR" X-Envelope-To: kvmarm@lists.linux.dev DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718902050; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Bsfr5ty35fbLYRU7XMGH77z8ls/kA62MLRgj2uAwsLE=; b=U4nWGIaRXlgskXoWzOU8/MsxVGfsc4li1dHfFygLdbmZZkrBhXc17ioPZW5+0f/BZV288I wpCxnGZZn+BjihBi45bpzmo4353euOwzQjyCPqoItngHoj4rlXIdYfrTHurBI070ThsHjX IWFRlbNLgg5xe/oYtIvY/rMgZvRR6qY= X-Envelope-To: maz@kernel.org X-Envelope-To: james.morse@arm.com X-Envelope-To: suzuki.poulose@arm.com X-Envelope-To: yuzenghui@huawei.com X-Envelope-To: kvm@vger.kernel.org X-Envelope-To: tabba@google.com X-Envelope-To: oliver.upton@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: kvmarm@lists.linux.dev Cc: Marc Zyngier , James Morse , Suzuki K Poulose , Zenghui Yu , kvm@vger.kernel.org, Fuad Tabba , Oliver Upton Subject: [PATCH v3 15/15] KVM: arm64: Allow the use of SVE+NV Date: Thu, 20 Jun 2024 16:46:52 +0000 Message-ID: <20240620164653.1130714-16-oliver.upton@linux.dev> In-Reply-To: <20240620164653.1130714-1-oliver.upton@linux.dev> References: <20240620164653.1130714-1-oliver.upton@linux.dev> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT Allow SVE and NV to mix now that everything is in place to handle it correctly. Reviewed-by: Marc Zyngier Signed-off-by: Oliver Upton --- arch/arm64/kvm/arm.c | 5 ----- arch/arm64/kvm/nested.c | 3 +-- 2 files changed, 1 insertion(+), 7 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 9996a989b52e..e2c934728f73 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1419,11 +1419,6 @@ static int kvm_vcpu_init_check_features(struct kvm_vcpu *vcpu, test_bit(KVM_ARM_VCPU_PTRAUTH_GENERIC, &features)) return -EINVAL; - /* Disallow NV+SVE for the time being */ - if (test_bit(KVM_ARM_VCPU_HAS_EL2, &features) && - test_bit(KVM_ARM_VCPU_SVE, &features)) - return -EINVAL; - if (!test_bit(KVM_ARM_VCPU_EL1_32BIT, &features)) return 0; diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c index 6813c7c7f00a..0aefc3e1b9a7 100644 --- a/arch/arm64/kvm/nested.c +++ b/arch/arm64/kvm/nested.c @@ -41,13 +41,12 @@ static u64 limit_nv_id_reg(u32 id, u64 val) break; case SYS_ID_AA64PFR0_EL1: - /* No AMU, MPAM, S-EL2, RAS or SVE */ + /* No AMU, MPAM, S-EL2, or RAS */ val &= ~(GENMASK_ULL(55, 52) | NV_FTR(PFR0, AMU) | NV_FTR(PFR0, MPAM) | NV_FTR(PFR0, SEL2) | NV_FTR(PFR0, RAS) | - NV_FTR(PFR0, SVE) | NV_FTR(PFR0, EL3) | NV_FTR(PFR0, EL2) | NV_FTR(PFR0, EL1));