From patchwork Sat Jun 3 02:50:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reiji Watanabe X-Patchwork-Id: 13266054 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4BD95C7EE2A for ; Sat, 3 Jun 2023 02:51:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=dbuddA/qN2oGcK5JOq1IrTWf4WjiMo0DP8ZkA6SEVGY=; b=kChoHNyfbJ4bNpLS9gwQisH1xd lfwxod6OR4cLtZTev11PXr3dTC5b8Hs+Ywa86GgMGwXm5uLZ+ieZBjLcovPTvXzcyvpO9dEdBEIRG 4vjNRCwehhvUCEZUqdWG21uhuylMVFYNXQFnlyyr7u0SLFcR7xR+a4YKOKjQsOMg0ahwDXpiE9rBW kwUrWgrSqRF8RHnciB4re71wK1Xxnssx/I696WISy83Xb5DmJsEHYN2Ipeh2mAvKOqTFUWFUJQMBm O8O8Zz/KITwvryI7Efmis9kkaMjWlopPTj8Qhw75g9GO4X5yWeOJoLL6bZZ+ONwaDNSuUL9Pm9aB3 HIiXZPVg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q5HMS-008aM6-2y; Sat, 03 Jun 2023 02:51:24 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q5HMR-008aLA-02 for linux-arm-kernel@lists.infradead.org; Sat, 03 Jun 2023 02:51:24 +0000 Received: by mail-pg1-x549.google.com with SMTP id 41be03b00d2f7-53f6f7d1881so2496065a12.3 for ; Fri, 02 Jun 2023 19:51:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1685760680; x=1688352680; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=2SBYW84L/m+RDCA19ekjkbKA/bg9Rb+RSkEF96qBDgs=; b=1BcDQfkzkfe0i2bUABNsoY/fv0w3MChr1jJqEU1n5t/4gQTnzoRWB7gRfTWMfu0/mq StQhrK4POlXN0TeajosvdJpgLIDh4y/h3gR+xD8ja8duhQea+WGjU8/WBf0nC/hI8xjz gp9KhTAp/Ox67+UgDbbucgvc1UU3yHRhe2N+1cnIf6IfyR4VlJ4SlEfgKgqzMEsjClqC W7OoNYZpeUqhnNmZGwpmqeH2+Xv6OHIEh8QTSoEqgZHKONTkBsh53tUOEsZSonpd6Ekv P4OP3+yr4H0UCyMlwnpABk0ceKHRM9J/c4j61GlTNidkXbvJf+dnWXsyxRQDlgjl2oFI WD+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685760680; x=1688352680; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=2SBYW84L/m+RDCA19ekjkbKA/bg9Rb+RSkEF96qBDgs=; b=RYgGwoESG8Yn+uHJRNNNnjN65SbuuAZhrBOPJSdffRpS0Av1TwOfnvoVF1lOyPMXKV V/1orxiKweTGBwrfPZq8zAWYgywhjmq5rZ7k91qSefAGDURM5zEU/tqqH5fQjk38JJf/ e7IHkEuv3dkvzBWDEbdaxXGwqAHHtMrFC289qYu32KaaWCiKViWKnj8YtUq4QDVX535F 4ivoTLGmwSXyOgINGyEvQ23s2TfyYojFen5vm3JWBWsNnDvJs+Hc96xlXPmrt5IjOmFf GCvHJEBBSovk9+ZSIbutV/XRxgrkLpo9+iaz5GNlRNiV3/PqIfUqVEZ550chHH6nGmw2 hIbg== X-Gm-Message-State: AC+VfDwgZeEjHqJruVNdtiMPNd1I+lpDfcyAdVzzRg0XZeVg6TPwY0ti XFzVcqDcjBNSVDMGIT3z7KXw11F2T+I= X-Google-Smtp-Source: ACHHUZ7F41GX4kI7XUmhvJo2WeUg7o8g2JM34l5sTdU0OOY63gxJLffhrx6c9AFBJ63aEnrO/XTy+PrLYBc= X-Received: from reijiw-west4.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:aa1]) (user=reijiw job=sendgmr) by 2002:a65:47c9:0:b0:53f:6f7c:554e with SMTP id f9-20020a6547c9000000b0053f6f7c554emr2728965pgs.12.1685760680506; Fri, 02 Jun 2023 19:51:20 -0700 (PDT) Date: Fri, 2 Jun 2023 19:50:34 -0700 In-Reply-To: <20230603025035.3781797-1-reijiw@google.com> Mime-Version: 1.0 References: <20230603025035.3781797-1-reijiw@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230603025035.3781797-2-reijiw@google.com> Subject: [PATCH v5 1/2] KVM: arm64: PMU: Restore the host's PMUSERENR_EL0 From: Reiji Watanabe To: Marc Zyngier , Mark Rutland , Oliver Upton , Will Deacon , Catalin Marinas , kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, James Morse , Alexandru Elisei , Zenghui Yu , Suzuki K Poulose , Paolo Bonzini , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , Shaoqin Huang , Rob Herring , Reiji Watanabe X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230602_195123_072408_81C74035 X-CRM114-Status: GOOD ( 12.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Restore the host's PMUSERENR_EL0 value instead of clearing it, before returning back to userspace, as the host's EL0 might have a direct access to PMU registers (some bits of PMUSERENR_EL0 for might not be zero for the host EL0). Fixes: 83a7a4d643d3 ("arm64: perf: Enable PMU counter userspace access for perf event") Signed-off-by: Reiji Watanabe --- arch/arm64/kvm/hyp/include/hyp/switch.h | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index e78a08a72a3c..51229d847103 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -82,7 +82,12 @@ static inline void __activate_traps_common(struct kvm_vcpu *vcpu) * EL1 instead of being trapped to EL2. */ if (kvm_arm_support_pmu_v3()) { + struct kvm_cpu_context *hctxt; + write_sysreg(0, pmselr_el0); + + hctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; + ctxt_sys_reg(hctxt, PMUSERENR_EL0) = read_sysreg(pmuserenr_el0); write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0); } @@ -106,8 +111,12 @@ static inline void __deactivate_traps_common(struct kvm_vcpu *vcpu) write_sysreg(vcpu->arch.mdcr_el2_host, mdcr_el2); write_sysreg(0, hstr_el2); - if (kvm_arm_support_pmu_v3()) - write_sysreg(0, pmuserenr_el0); + if (kvm_arm_support_pmu_v3()) { + struct kvm_cpu_context *hctxt; + + hctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; + write_sysreg(ctxt_sys_reg(hctxt, PMUSERENR_EL0), pmuserenr_el0); + } if (cpus_have_final_cap(ARM64_SME)) { sysreg_clear_set_s(SYS_HFGRTR_EL2, 0, From patchwork Sat Jun 3 02:50:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reiji Watanabe X-Patchwork-Id: 13266055 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AEE12C7EE2A for ; Sat, 3 Jun 2023 02:52:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=2XWX5rqiLGVLgm0w6efIW4rSYKegLk9h/x8tkDdjloQ=; b=E/ZPjJ4VPRqObEr+PvP5uVeJ7w vusqc9njuvwUzT6BqvAC8G1YsZLUQqiV50dwflnT7YE//F8BO6Cfp9as4A6Kn7Ne0jlaEq1U9Dibe LuPbGAFiN4gWYr1yxp5mlu8j6pLKlWsCqE+B/Lx9jOm/8E1Tph6WBhsEwTgJ853u3LmKRrCQI0vrf AzM6g/JP9ThSsGuGInqV2FgWxW3w/E1qt7bxK5JVo+qb6XD/dl5O1K09ocba8VfXgNDqPKB2ohXW+ X/m277al6raVeKLAAp6/Nd/C5/PVYSt6gjqyqwBT+Gtn/8V+kObrbKv5YFJdP/u39a0nTb7JpUVnG tLY95b0g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1q5HN9-008aWI-2e; Sat, 03 Jun 2023 02:52:07 +0000 Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1q5HN6-008aVL-1y for linux-arm-kernel@lists.infradead.org; Sat, 03 Jun 2023 02:52:06 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-568ab5c813eso41844277b3.2 for ; Fri, 02 Jun 2023 19:52:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1685760722; x=1688352722; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=KDFXlg40JZK7tD1QcINc3yvgddagMFe1ISddFDG7yUk=; b=jjrCMUfevaPYgb2kZwsK1yDw7Qlp8585C9teElxM/xHufCFV3l63n5NHNbiPffqLeV KmpleA/Kdn1JqTCpeOrLlqzCz0KhToR5k7o3gU0zyJlErZFHsvIKm4nm0Tj4RW6krX45 dx1KBykEOQpPDW5NccaI7csUq/WQkGCQps/wA1aq79jwWiyI9ThIgRSchtJuwSuVzKKa v3hGZ1WKVhKnMPrvuDnawLt+L2qyKD98LfwjhFUUEOqfFn4nFF09pYRWAq33+MtW/26j drcEZZuTyh16IOrbgy++GzYImwlu8KEsa8nvMvI3+LA1TFZow7KsYlTtArA61oWr4Yv9 jKMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685760722; x=1688352722; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KDFXlg40JZK7tD1QcINc3yvgddagMFe1ISddFDG7yUk=; b=iFxUcgUt0lvG3GyAyqQnNEvRyYmXXLlzJMizAscZp+5Dnl/YO925LqNw05nvP+qEJG TTYMyt9IED5itozQAxBvUnlIi2aegbOMXE7Cc54xln0f2E8ypzQnEq69tsz0A4gauFOL w0svd1s8KQAJeYOKsGaPm7Fg0s4qYoh3HeUnoLRJxoQ58Lrxql9AByG0rw+4Z21cSLTz Pft6CquBpa45NEojJV8mKVjb6E8GePtGxKv16D50nw1wAhkUc8D5V/aLKZjANfKa7/F/ an4zatOzlrSbR1WGF1/fe1VBat97fyMtjtVsyZ41631q0pN7OKD/AmbBRW1LyqenVv/p zfzQ== X-Gm-Message-State: AC+VfDyQJcP0tlKtyoPe80iYxmMHxnQf2nWThvn2DjdUPzEpwET19bKm zqW/ZMg9Ar2OMPEBkl43vCpyWePgt+c= X-Google-Smtp-Source: ACHHUZ4CDztfSVCjau0l24GyQAsrUl9AQhGkOIfEYDSBDcjmK8RGLjB3vZ/MQohARF1O0R5waBVGa8RCccg= X-Received: from reijiw-west4.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:aa1]) (user=reijiw job=sendgmr) by 2002:a81:ad06:0:b0:565:bf4b:de20 with SMTP id l6-20020a81ad06000000b00565bf4bde20mr926860ywh.2.1685760722413; Fri, 02 Jun 2023 19:52:02 -0700 (PDT) Date: Fri, 2 Jun 2023 19:50:35 -0700 In-Reply-To: <20230603025035.3781797-1-reijiw@google.com> Mime-Version: 1.0 References: <20230603025035.3781797-1-reijiw@google.com> X-Mailer: git-send-email 2.41.0.rc0.172.g3f132b7071-goog Message-ID: <20230603025035.3781797-3-reijiw@google.com> Subject: [PATCH v5 2/2] KVM: arm64: PMU: Don't overwrite PMUSERENR with vcpu loaded From: Reiji Watanabe To: Marc Zyngier , Mark Rutland , Oliver Upton , Will Deacon , Catalin Marinas , kvmarm@lists.linux.dev Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, James Morse , Alexandru Elisei , Zenghui Yu , Suzuki K Poulose , Paolo Bonzini , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , Shaoqin Huang , Rob Herring , Reiji Watanabe X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230602_195204_658325_56191A8F X-CRM114-Status: GOOD ( 23.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, with VHE, KVM sets ER, CR, SW and EN bits of PMUSERENR_EL0 to 1 on vcpu_load(), and saves and restores the register value for the host on vcpu_load() and vcpu_put(). If the value of those bits are cleared on a pCPU with a vCPU loaded (armv8pmu_start() would do that when PMU counters are programmed for the guest), PMU access from the guest EL0 might be trapped to the guest EL1 directly regardless of the current PMUSERENR_EL0 value of the vCPU. Fix this by not letting armv8pmu_start() overwrite PMUSERENR_EL0 on the pCPU where PMUSERENR_EL0 for the guest is loaded, and instead updating the saved shadow register value for the host so that the value can be restored on vcpu_put() later. While vcpu_{put,load}() are manipulating PMUSERENR_EL0, disable IRQs to prevent a race condition between these processes and IPIs that attempt to update PMUSERENR_EL0 for the host EL0. Suggested-by: Mark Rutland Suggested-by: Marc Zyngier Fixes: 83a7a4d643d3 ("arm64: perf: Enable PMU counter userspace access for perf event") Signed-off-by: Reiji Watanabe --- arch/arm/include/asm/arm_pmuv3.h | 5 +++++ arch/arm64/include/asm/kvm_host.h | 7 +++++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 2 ++ arch/arm64/kvm/hyp/vhe/switch.c | 14 +++++++++++++ arch/arm64/kvm/pmu.c | 27 +++++++++++++++++++++++++ drivers/perf/arm_pmuv3.c | 21 ++++++++++++++++--- 6 files changed, 73 insertions(+), 3 deletions(-) diff --git a/arch/arm/include/asm/arm_pmuv3.h b/arch/arm/include/asm/arm_pmuv3.h index f4db3e75d75f..f3cd04ff022d 100644 --- a/arch/arm/include/asm/arm_pmuv3.h +++ b/arch/arm/include/asm/arm_pmuv3.h @@ -222,6 +222,11 @@ static inline bool kvm_pmu_counter_deferred(struct perf_event_attr *attr) return false; } +static inline bool kvm_set_pmuserenr(u64 val) +{ + return false; +} + /* PMU Version in DFR Register */ #define ARMV8_PMU_DFR_VER_NI 0 #define ARMV8_PMU_DFR_VER_V3P4 0x5 diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 7e7e19ef6993..9787503ff43f 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -699,6 +699,8 @@ struct kvm_vcpu_arch { #define SYSREGS_ON_CPU __vcpu_single_flag(sflags, BIT(4)) /* Software step state is Active-pending */ #define DBG_SS_ACTIVE_PENDING __vcpu_single_flag(sflags, BIT(5)) +/* PMUSERENR for the guest EL0 is on physical CPU */ +#define PMUSERENR_ON_CPU __vcpu_single_flag(sflags, BIT(6)) /* Pointer to the vcpu's SVE FFR for sve_{save,load}_state() */ @@ -1065,9 +1067,14 @@ void kvm_arch_vcpu_put_debug_state_flags(struct kvm_vcpu *vcpu); #ifdef CONFIG_KVM void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr); void kvm_clr_pmu_events(u32 clr); +bool kvm_set_pmuserenr(u64 val); #else static inline void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) {} static inline void kvm_clr_pmu_events(u32 clr) {} +static inline bool kvm_set_pmuserenr(u64 val) +{ + return false; +} #endif void kvm_vcpu_load_sysregs_vhe(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 51229d847103..c530cf6b4bef 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -89,6 +89,7 @@ static inline void __activate_traps_common(struct kvm_vcpu *vcpu) hctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; ctxt_sys_reg(hctxt, PMUSERENR_EL0) = read_sysreg(pmuserenr_el0); write_sysreg(ARMV8_PMU_USERENR_MASK, pmuserenr_el0); + vcpu_set_flag(vcpu, PMUSERENR_ON_CPU); } vcpu->arch.mdcr_el2_host = read_sysreg(mdcr_el2); @@ -116,6 +117,7 @@ static inline void __deactivate_traps_common(struct kvm_vcpu *vcpu) hctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; write_sysreg(ctxt_sys_reg(hctxt, PMUSERENR_EL0), pmuserenr_el0); + vcpu_clear_flag(vcpu, PMUSERENR_ON_CPU); } if (cpus_have_final_cap(ARM64_SME)) { diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 3d868e84c7a0..3f0b5756c956 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -92,14 +92,28 @@ static void __deactivate_traps(struct kvm_vcpu *vcpu) } NOKPROBE_SYMBOL(__deactivate_traps); +/* + * Disable IRQs in {activate,deactivate}_traps_vhe_{load,put}() to + * prevent a race condition between context switching of PMUSERENR_EL0 + * in __{activate,deactivate}_traps_common() and IPIs that attempts to + * update PMUSERENR_EL0. See also kvm_set_pmuserenr(). + */ void activate_traps_vhe_load(struct kvm_vcpu *vcpu) { + unsigned long flags; + + local_irq_save(flags); __activate_traps_common(vcpu); + local_irq_restore(flags); } void deactivate_traps_vhe_put(struct kvm_vcpu *vcpu) { + unsigned long flags; + + local_irq_save(flags); __deactivate_traps_common(vcpu); + local_irq_restore(flags); } static const exit_handler_fn hyp_exit_handlers[] = { diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c index 7887133d15f0..121f1a14c829 100644 --- a/arch/arm64/kvm/pmu.c +++ b/arch/arm64/kvm/pmu.c @@ -209,3 +209,30 @@ void kvm_vcpu_pmu_restore_host(struct kvm_vcpu *vcpu) kvm_vcpu_pmu_enable_el0(events_host); kvm_vcpu_pmu_disable_el0(events_guest); } + +/* + * With VHE, keep track of the PMUSERENR_EL0 value for the host EL0 on the pCPU + * where PMUSERENR_EL0 for the guest is loaded, since PMUSERENR_EL0 is switched + * to the value for the guest on vcpu_load(). The value for the host EL0 + * will be restored on vcpu_put(), before returning to userspace. + * This isn't necessary for nVHE, as the register is context switched for + * every guest enter/exit. + * + * Return true if KVM takes care of the register. Otherwise return false. + */ +bool kvm_set_pmuserenr(u64 val) +{ + struct kvm_cpu_context *hctxt; + struct kvm_vcpu *vcpu; + + if (!kvm_arm_support_pmu_v3() || !has_vhe()) + return false; + + vcpu = kvm_get_running_vcpu(); + if (!vcpu || !vcpu_get_flag(vcpu, PMUSERENR_ON_CPU)) + return false; + + hctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; + ctxt_sys_reg(hctxt, PMUSERENR_EL0) = val; + return true; +} diff --git a/drivers/perf/arm_pmuv3.c b/drivers/perf/arm_pmuv3.c index c98e4039386d..93b7edb5f1e7 100644 --- a/drivers/perf/arm_pmuv3.c +++ b/drivers/perf/arm_pmuv3.c @@ -677,9 +677,25 @@ static inline u32 armv8pmu_getreset_flags(void) return value; } +static void update_pmuserenr(u64 val) +{ + lockdep_assert_irqs_disabled(); + + /* + * The current PMUSERENR_EL0 value might be the value for the guest. + * If that's the case, have KVM keep tracking of the register value + * for the host EL0 so that KVM can restore it before returning to + * the host EL0. Otherwise, update the register now. + */ + if (kvm_set_pmuserenr(val)) + return; + + write_pmuserenr(val); +} + static void armv8pmu_disable_user_access(void) { - write_pmuserenr(0); + update_pmuserenr(0); } static void armv8pmu_enable_user_access(struct arm_pmu *cpu_pmu) @@ -695,8 +711,7 @@ static void armv8pmu_enable_user_access(struct arm_pmu *cpu_pmu) armv8pmu_write_evcntr(i, 0); } - write_pmuserenr(0); - write_pmuserenr(ARMV8_PMU_USERENR_ER | ARMV8_PMU_USERENR_CR); + update_pmuserenr(ARMV8_PMU_USERENR_ER | ARMV8_PMU_USERENR_CR); } static void armv8pmu_enable_event(struct perf_event *event)