From patchwork Fri Jul 16 21:26:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Upton X-Patchwork-Id: 12382923 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14EF8C12002 for ; Fri, 16 Jul 2021 21:47:38 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C5AC9613EB for ; Fri, 16 Jul 2021 21:47:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C5AC9613EB Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Cf/WU+KX5NeNug48PKXph2ZO1fjtz1vCP6TbHqKang4=; b=jNn3kcuVlgvaZDfEkVfjkfTaaM 8XB9ZGxkDSkLKH6mYvHfBCZjr9+T+DXWHF4GWXIuDGCvAdGEz/mjUAyzcAbNYDGYXapkX3fS7DWly 3SSdGlU2Nh4fJ1Thi6RZIW+EtetJwqA8n7tCSbLVANosqrLj8Sb2v4JxQXlefTRD8Yvvmxr7T3PV5 w+kGYB1cwAa5tbDjCYAEp1PzdbwDrQ7NcpQJbX3V/o8dkq8pJKqKzqGJS/TVHGvIbXpa2DJxMmb39 ZbJXnXR+idjl+rOwCr6DawK0cTGXdO+jEazSWS1zyc3BQ+H6/i2ogyxQRK4wf6MZV16uKCnuDt6eF zZM+WXkw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m4Vdp-005PtJ-Ee; Fri, 16 Jul 2021 21:45:06 +0000 Received: from mail-io1-xd4a.google.com ([2607:f8b0:4864:20::d4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m4VMB-005Icz-64 for linux-arm-kernel@lists.infradead.org; Fri, 16 Jul 2021 21:26:53 +0000 Received: by mail-io1-xd4a.google.com with SMTP id c7-20020a5e8f070000b02905286f544a84so5871177iok.1 for ; Fri, 16 Jul 2021 14:26:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=XbTtxhZ4ROyo17V7q+xHcBeN3BFqrITDBTCQhNqMurw=; b=BAD5EG/FARsvUlfJo9jxJQm07diGtiugOw+Nnj7Il1fvTjIe1u7En0TVK+zlQMxOE7 OqNjQfourhwhvBPsx88wcNFp8sNWYX+h1k7Bcm3iSHaJQfnLtEu9tZo6Pd61N+mR6uuo wzRVhtyYvXKdCdOcfE7tR+pEBwvjkiM/AVW9H59tpzIITNyXNAkUn3xsrtONbOL19b6C cWkm2GaR1/ISzJklDoO3Y4u8ETemxqA/LmBs5P/ASYgTfDJImSD81uriAIb05a4TvVGq tNb8R3dzWDsesSkNSAdI5SBq8K/Fv+b61IEBWcAoF27uS+nBs0OZ54FclVuDgGmHs8v4 qhRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=XbTtxhZ4ROyo17V7q+xHcBeN3BFqrITDBTCQhNqMurw=; b=HFAk04iWUdtSDwtQudCtP4LRbxGt9Hrnm7ksbhE7/v95wPMehJM50582J8xFxP2K8P zI0AHwthJz/eToK9JUuGo9Tmnb5wub8CNxew71e0a9B0ABwZ6Y3TSL5cTwxVT6WDmBmX KcyEjYe8fkLVvLsKecuWP8g3GdEjZv9aKNS83bR2f1ZD++gs4wA6DIK2VLTngw/8TYcu 7mqZi06FrieXYLyynNdk2q5uvdzkDgrQPwLzckOEXK1RimU5MN8E3t66meUXFHz8qksX wlVRC+nQRHcn7G0RohVkp4EdHxXdUxeV8KpSvgrQr/UIoLNQJcS75XW8au8KS+1/u17f owTQ== X-Gm-Message-State: AOAM531sxmO5K+QG1jEXbOWm5r1sAzgGqOdST4vR9L26a85DySfU8RoI JNEU9wQIZvvEs1iXydiyaNwo4w5XgD4= X-Google-Smtp-Source: ABdhPJzrLwoLK3b1Cv6NPy2HjwvQxnldo54jBH5NTNwPpqTIwdi6o4Bn+8pBZE2ZRJbKUcNVCJwRjsTJ2VE= X-Received: from oupton.c.googlers.com ([fda3:e722:ac3:cc00:2b:ff92:c0a8:404]) (user=oupton job=sendgmr) by 2002:a02:8783:: with SMTP id t3mr10540629jai.45.1626470809588; Fri, 16 Jul 2021 14:26:49 -0700 (PDT) Date: Fri, 16 Jul 2021 21:26:27 +0000 In-Reply-To: <20210716212629.2232756-1-oupton@google.com> Message-Id: <20210716212629.2232756-11-oupton@google.com> Mime-Version: 1.0 References: <20210716212629.2232756-1-oupton@google.com> X-Mailer: git-send-email 2.32.0.402.g57bb445576-goog Subject: [PATCH v2 10/12] KVM: arm64: Provide userspace access to the physical counter offset From: Oliver Upton To: kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu Cc: Paolo Bonzini , Sean Christopherson , Marc Zyngier , Peter Shier , Jim Mattson , David Matlack , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , James Morse , Alexandru Elisei , Suzuki K Poulose , linux-arm-kernel@lists.infradead.org, Oliver Upton X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210716_142651_289750_262E6AF7 X-CRM114-Status: GOOD ( 29.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Presently, KVM provides no facilities for correctly migrating a guest that depends on the physical counter-timer. While most guests (barring NV, of course) should not depend on the physical counter-timer, an operator may still wish to provide a consistent view of the physical counter-timer across migrations. Provide userspace with a new vCPU attribute to modify the guest physical counter-timer offset. Since the base architecture doesn't provide a physical counter-timer offset register, emulate the correct behavior by trapping accesses to the physical counter-timer whenever the offset value is non-zero. Uphold the same behavior as CNTVOFF_EL2 and broadcast the physical offset to all vCPUs whenever written. This guarantees that the counter-timer we provide the guest remains architectural, wherein all views of the counter-timer are consistent across vCPUs. Reconfigure timer traps for VHE on every guest entry, as different VMs will now have different traps enabled. Enable physical counter traps for nVHE whenever the offset is nonzero (we already trap physical timer registers in nVHE). FEAT_ECV provides a guest physical counter-timer offset register (CNTPOFF_EL2), but ECV-enabled hardware is nonexistent at the time of writing so support for it was elided for the sake of the author :) Signed-off-by: Oliver Upton --- arch/arm64/include/asm/kvm_host.h | 1 + arch/arm64/include/asm/kvm_hyp.h | 2 - arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/include/uapi/asm/kvm.h | 1 + arch/arm64/kvm/arch_timer.c | 50 ++++++++++++++++++++--- arch/arm64/kvm/arm.c | 4 +- arch/arm64/kvm/hyp/include/hyp/switch.h | 23 +++++++++++ arch/arm64/kvm/hyp/include/hyp/timer-sr.h | 26 ++++++++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 2 - arch/arm64/kvm/hyp/nvhe/timer-sr.c | 21 +++++----- arch/arm64/kvm/hyp/vhe/timer-sr.c | 27 ++++++++++++ include/kvm/arm_arch_timer.h | 2 - 12 files changed, 136 insertions(+), 24 deletions(-) create mode 100644 arch/arm64/kvm/hyp/include/hyp/timer-sr.h diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 41911585ae0c..de92fa678924 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -204,6 +204,7 @@ enum vcpu_sysreg { SP_EL1, SPSR_EL1, + CNTPOFF_EL2, CNTVOFF_EL2, CNTV_CVAL_EL0, CNTV_CTL_EL0, diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 9d60b3006efc..01eb3864e50f 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -65,10 +65,8 @@ void __vgic_v3_save_aprs(struct vgic_v3_cpu_if *cpu_if); void __vgic_v3_restore_aprs(struct vgic_v3_cpu_if *cpu_if); int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu); -#ifdef __KVM_NVHE_HYPERVISOR__ void __timer_enable_traps(struct kvm_vcpu *vcpu); void __timer_disable_traps(struct kvm_vcpu *vcpu); -#endif #ifdef __KVM_NVHE_HYPERVISOR__ void __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt); diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 347ccac2341e..243e36c088e7 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -505,6 +505,7 @@ #define SYS_AMEVCNTR0_MEM_STALL SYS_AMEVCNTR0_EL0(3) #define SYS_CNTFRQ_EL0 sys_reg(3, 3, 14, 0, 0) +#define SYS_CNTPCT_EL0 sys_reg(3, 3, 14, 0, 1) #define SYS_CNTP_TVAL_EL0 sys_reg(3, 3, 14, 2, 0) #define SYS_CNTP_CTL_EL0 sys_reg(3, 3, 14, 2, 1) diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h index 008d0518d2b1..3e42c72d4c68 100644 --- a/arch/arm64/include/uapi/asm/kvm.h +++ b/arch/arm64/include/uapi/asm/kvm.h @@ -366,6 +366,7 @@ struct kvm_arm_copy_mte_tags { #define KVM_ARM_VCPU_TIMER_IRQ_VTIMER 0 #define KVM_ARM_VCPU_TIMER_IRQ_PTIMER 1 #define KVM_ARM_VCPU_TIMER_OFFSET_VTIMER 2 +#define KVM_ARM_VCPU_TIMER_OFFSET_PTIMER 3 #define KVM_ARM_VCPU_PVTIME_CTRL 2 #define KVM_ARM_VCPU_PVTIME_IPA 0 diff --git a/arch/arm64/kvm/arch_timer.c b/arch/arm64/kvm/arch_timer.c index d2b1b13af658..05ec385e26b5 100644 --- a/arch/arm64/kvm/arch_timer.c +++ b/arch/arm64/kvm/arch_timer.c @@ -89,7 +89,10 @@ static u64 timer_get_offset(struct arch_timer_context *ctxt) switch(arch_timer_ctx_index(ctxt)) { case TIMER_VTIMER: return __vcpu_sys_reg(vcpu, CNTVOFF_EL2); + case TIMER_PTIMER: + return __vcpu_sys_reg(vcpu, CNTPOFF_EL2); default: + WARN_ONCE(1, "unrecognized timer index %ld", arch_timer_ctx_index(ctxt)); return 0; } } @@ -134,6 +137,9 @@ static void timer_set_offset(struct arch_timer_context *ctxt, u64 offset) case TIMER_VTIMER: __vcpu_sys_reg(vcpu, CNTVOFF_EL2) = offset; break; + case TIMER_PTIMER: + __vcpu_sys_reg(vcpu, CNTPOFF_EL2) = offset; + break; default: WARN(offset, "timer %ld\n", arch_timer_ctx_index(ctxt)); } @@ -144,9 +150,24 @@ u64 kvm_phys_timer_read(void) return timecounter->cc->read(timecounter->cc); } +static bool timer_emulation_required(struct arch_timer_context *ctx) +{ + enum kvm_arch_timers timer = arch_timer_ctx_index(ctx); + + switch (timer) { + case TIMER_VTIMER: + return false; + case TIMER_PTIMER: + return timer_get_offset(ctx); + default: + WARN_ONCE(1, "unrecognized timer index %ld\n", timer); + return false; + } +} + static void get_timer_map(struct kvm_vcpu *vcpu, struct timer_map *map) { - if (has_vhe()) { + if (has_vhe() && !timer_emulation_required(vcpu_ptimer(vcpu))) { map->direct_vtimer = vcpu_vtimer(vcpu); map->direct_ptimer = vcpu_ptimer(vcpu); map->emul_ptimer = NULL; @@ -747,8 +768,9 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu) return 0; } -/* Make the updates of cntvoff for all vtimer contexts atomic */ -static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff) +/* Make the updates of offset for all timer contexts atomic */ +static void update_timer_offset(struct kvm_vcpu *vcpu, + enum kvm_arch_timers timer, u64 offset) { int i; struct kvm *kvm = vcpu->kvm; @@ -756,16 +778,26 @@ static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff) mutex_lock(&kvm->lock); kvm_for_each_vcpu(i, tmp, kvm) - timer_set_offset(vcpu_vtimer(tmp), cntvoff); + timer_set_offset(vcpu_get_timer(tmp, timer), offset); /* * When called from the vcpu create path, the CPU being created is not * included in the loop above, so we just set it here as well. */ - timer_set_offset(vcpu_vtimer(vcpu), cntvoff); + timer_set_offset(vcpu_get_timer(vcpu, timer), offset); mutex_unlock(&kvm->lock); } +static void update_vtimer_cntvoff(struct kvm_vcpu *vcpu, u64 cntvoff) +{ + update_timer_offset(vcpu, TIMER_VTIMER, cntvoff); +} + +static void update_ptimer_cntpoff(struct kvm_vcpu *vcpu, u64 cntpoff) +{ + update_timer_offset(vcpu, TIMER_PTIMER, cntpoff); +} + void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu) { struct arch_timer_cpu *timer = vcpu_timer(vcpu); @@ -1350,6 +1382,9 @@ int kvm_arm_timer_set_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER: update_vtimer_cntvoff(vcpu, offset); break; + case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER: + update_ptimer_cntpoff(vcpu, offset); + break; default: return -ENXIO; } @@ -1364,6 +1399,7 @@ int kvm_arm_timer_set_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) case KVM_ARM_VCPU_TIMER_IRQ_PTIMER: return kvm_arm_timer_set_attr_irq(vcpu, attr); case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER: + case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER: return kvm_arm_timer_set_attr_offset(vcpu, attr); } @@ -1400,6 +1436,8 @@ int kvm_arm_timer_get_attr_offset(struct kvm_vcpu *vcpu, struct kvm_device_attr switch (attr->attr) { case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER: timer = vcpu_vtimer(vcpu); + case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER: + timer = vcpu_ptimer(vcpu); break; default: return -ENXIO; @@ -1416,6 +1454,7 @@ int kvm_arm_timer_get_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) case KVM_ARM_VCPU_TIMER_IRQ_PTIMER: return kvm_arm_timer_get_attr_irq(vcpu, attr); case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER: + case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER: return kvm_arm_timer_get_attr_offset(vcpu, attr); } @@ -1428,6 +1467,7 @@ int kvm_arm_timer_has_attr(struct kvm_vcpu *vcpu, struct kvm_device_attr *attr) case KVM_ARM_VCPU_TIMER_IRQ_VTIMER: case KVM_ARM_VCPU_TIMER_IRQ_PTIMER: case KVM_ARM_VCPU_TIMER_OFFSET_VTIMER: + case KVM_ARM_VCPU_TIMER_OFFSET_PTIMER: return 0; } diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index e0b81870ff2a..217f4c6038e8 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1549,9 +1549,7 @@ static void cpu_hyp_reinit(void) cpu_hyp_reset(); - if (is_kernel_in_hyp_mode()) - kvm_timer_init_vhe(); - else + if (!is_kernel_in_hyp_mode()) cpu_init_hyp_mode(); cpu_set_hyp_vector(); diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index e4a2f295a394..c3ae1e0614a2 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -8,6 +8,7 @@ #define __ARM64_KVM_HYP_SWITCH_H__ #include +#include #include #include @@ -113,6 +114,8 @@ static inline void ___activate_traps(struct kvm_vcpu *vcpu) if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN) && (hcr & HCR_VSE)) write_sysreg_s(vcpu->arch.vsesr_el2, SYS_VSESR_EL2); + + __timer_enable_traps(vcpu); } static inline void ___deactivate_traps(struct kvm_vcpu *vcpu) @@ -127,6 +130,8 @@ static inline void ___deactivate_traps(struct kvm_vcpu *vcpu) vcpu->arch.hcr_el2 &= ~HCR_VSE; vcpu->arch.hcr_el2 |= read_sysreg(hcr_el2) & HCR_VSE; } + + __timer_disable_traps(vcpu); } static inline bool __translate_far_to_hpfar(u64 far, u64 *hpfar) @@ -405,6 +410,21 @@ static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu) return true; } +static inline bool __hyp_handle_counter(struct kvm_vcpu *vcpu) +{ + u32 sysreg = esr_sys64_to_sysreg(kvm_vcpu_get_esr(vcpu)); + int rt = kvm_vcpu_sys_get_rt(vcpu); + u64 rv; + + if (sysreg != SYS_CNTPCT_EL0) + return false; + + rv = __timer_read_cntpct(vcpu); + vcpu_set_reg(vcpu, rt, rv); + __kvm_skip_instr(vcpu); + return true; +} + /* * Return true when we were able to fixup the guest exit and should return to * the guest, false when we should restore the host state and return to the @@ -439,6 +459,9 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) if (*exit_code != ARM_EXCEPTION_TRAP) goto exit; + if (__hyp_handle_counter(vcpu)) + goto guest; + if (cpus_have_final_cap(ARM64_WORKAROUND_CAVIUM_TX2_219_TVM) && kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_SYS64 && handle_tx2_tvm(vcpu)) diff --git a/arch/arm64/kvm/hyp/include/hyp/timer-sr.h b/arch/arm64/kvm/hyp/include/hyp/timer-sr.h new file mode 100644 index 000000000000..c91fbf9e5170 --- /dev/null +++ b/arch/arm64/kvm/hyp/include/hyp/timer-sr.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2021 Google LLC + * Author: Oliver Upton + */ + +#ifndef __ARM64_KVM_HYP_TIMER_SR_H__ +#define __ARM64_KVM_HYP_TIMER_SR_H__ + +#include +#include + +#include +#include + +static inline bool __timer_physical_emulation_required(struct kvm_vcpu *vcpu) +{ + return __vcpu_sys_reg(vcpu, CNTPOFF_EL2); +} + +static inline u64 __timer_read_cntpct(struct kvm_vcpu *vcpu) +{ + return read_sysreg(cntpct_el0) - __vcpu_sys_reg(vcpu, CNTPOFF_EL2); +} + +#endif /* __ARM64_KVM_HYP_TIMER_SR_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index f7af9688c1f7..4a190c932f8c 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -217,7 +217,6 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) __activate_traps(vcpu); __hyp_vgic_restore_state(vcpu); - __timer_enable_traps(vcpu); __debug_switch_to_guest(vcpu); @@ -230,7 +229,6 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) __sysreg_save_state_nvhe(guest_ctxt); __sysreg32_save_state(vcpu); - __timer_disable_traps(vcpu); __hyp_vgic_save_state(vcpu); __deactivate_traps(vcpu); diff --git a/arch/arm64/kvm/hyp/nvhe/timer-sr.c b/arch/arm64/kvm/hyp/nvhe/timer-sr.c index 9072e71693ba..ebc3f0d0908d 100644 --- a/arch/arm64/kvm/hyp/nvhe/timer-sr.c +++ b/arch/arm64/kvm/hyp/nvhe/timer-sr.c @@ -9,16 +9,13 @@ #include #include +#include void __kvm_timer_set_cntvoff(u64 cntvoff) { write_sysreg(cntvoff, cntvoff_el2); } -/* - * Should only be called on non-VHE systems. - * VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe(). - */ void __timer_disable_traps(struct kvm_vcpu *vcpu) { u64 val; @@ -29,20 +26,24 @@ void __timer_disable_traps(struct kvm_vcpu *vcpu) write_sysreg(val, cnthctl_el2); } -/* - * Should only be called on non-VHE systems. - * VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe(). - */ void __timer_enable_traps(struct kvm_vcpu *vcpu) { u64 val; /* * Disallow physical timer access for the guest - * Physical counter access is allowed */ val = read_sysreg(cnthctl_el2); val &= ~CNTHCTL_EL1PCEN; - val |= CNTHCTL_EL1PCTEN; + + /* + * Disallow physical counter access for the guest if offsetting is + * requested. + */ + if (__timer_physical_emulation_required(vcpu)) + val &= ~CNTHCTL_EL1PCTEN; + else + val |= CNTHCTL_EL1PCTEN; + write_sysreg(val, cnthctl_el2); } diff --git a/arch/arm64/kvm/hyp/vhe/timer-sr.c b/arch/arm64/kvm/hyp/vhe/timer-sr.c index 4cda674a8be6..10506f3ce8a1 100644 --- a/arch/arm64/kvm/hyp/vhe/timer-sr.c +++ b/arch/arm64/kvm/hyp/vhe/timer-sr.c @@ -4,9 +4,36 @@ * Author: Marc Zyngier */ +#include #include +#include void __kvm_timer_set_cntvoff(u64 cntvoff) { write_sysreg(cntvoff, cntvoff_el2); } + +void __timer_enable_traps(struct kvm_vcpu *vcpu) +{ + /* When HCR_EL2.E2H == 1, EL1PCEN nad EL1PCTEN are shifted by 10 */ + u32 cnthctl_shift = 10; + u64 val, mask; + + mask = CNTHCTL_EL1PCEN << cnthctl_shift; + mask |= CNTHCTL_EL1PCTEN << cnthctl_shift; + + val = read_sysreg(cnthctl_el2); + + /* + * VHE systems allow the guest direct access to the EL1 physical + * timer/counter if offsetting isn't requested. + */ + if (__timer_physical_emulation_required(vcpu)) + val &= ~mask; + else + val |= mask; + + write_sysreg(val, cnthctl_el2); +} + +void __timer_disable_traps(struct kvm_vcpu *vcpu) {} diff --git a/include/kvm/arm_arch_timer.h b/include/kvm/arm_arch_timer.h index 51c19381108c..f24fc435c401 100644 --- a/include/kvm/arm_arch_timer.h +++ b/include/kvm/arm_arch_timer.h @@ -83,8 +83,6 @@ u64 kvm_phys_timer_read(void); void kvm_timer_vcpu_load(struct kvm_vcpu *vcpu); void kvm_timer_vcpu_put(struct kvm_vcpu *vcpu); -void kvm_timer_init_vhe(void); - bool kvm_arch_timer_get_input_level(int vintid); #define vcpu_timer(v) (&(v)->arch.timer_cpu)