From patchwork Tue Apr 30 18:14:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Colton Lewis X-Patchwork-Id: 13649842 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 82504C10F16 for ; Tue, 30 Apr 2024 18:15:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: Mime-Version:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=/jz/F5tt9FJpV5i9Z+XZeOw6sy0t11uf0+51BdbqqFQ=; b=jYY AOTBn2SNoZDVyuAo7YS0B3eCECRx2+9o97P9FoFz+Odu13rE4I6iGfDhZcHKuCM2zkF3v5W+dTPjK DpzFF1xGSsXKs9Zr19FWqU6CAEdiyyEUyHDVbQzfl4ScCVI9dp8mkRq9OQO+7P3KF9bGDPrkHg2sB gjdLKYD9m8b4jMWEXrEsBSQVETJ6nftWyG2IvMSR2VWjZWv51Zbd5RCxKb9A4UI/WwMV5bRd+vFOU XxrUYtpFXGvr/iMGaU5ICGPz64DRaHerihvOKzkVkuy5O8b+Lyp41tOJtfGtocE2+xD4VM2oGf2QT Qys9M2U58IYSVLfgDi0ncgh+o8jy6gQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s1s0t-00000007YXD-2Fm0; Tue, 30 Apr 2024 18:15:35 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s1s0p-00000007YUh-3yus for linux-arm-kernel@lists.infradead.org; Tue, 30 Apr 2024 18:15:33 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 3f1490d57ef6-dcdc3db67f0so113233276.1 for ; Tue, 30 Apr 2024 11:15:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1714500927; x=1715105727; darn=lists.infradead.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=OOND0Wc6LHFDakdT/Gp/z0KSYn+8JMGA/w2gsPmNcgU=; b=dGVKsDFStD2qDg+I4aB/MZ+lyyG5JawWwLCc5IEx2EpiE9CvJU2mYlEJA4QEmqajSm fuqpN8bt3JRUo/3yFJmQvr5K+BDB9kZw9cJOS9UsKOEF++U4cjqVUCl7rt34bPEOdi6P 51bxXxV+RIubjwcA8FrM7vlG6XNrluy9QL95vHllWbKlHjwto6NeioK8wHnJ/EpEQ6vB ePzGqWWCdY17sGJYj8rD4sbvEWR2+tggaRfjGH9uxdREbfUvIb2QU2Z0T+/+/wiOLgly tzA5XN4pO/rPcSQiqrdACU81u2secZDp+dki54eWwG1NA2rVCX4AbOUywmFrtpJC8mcg zi4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714500927; x=1715105727; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=OOND0Wc6LHFDakdT/Gp/z0KSYn+8JMGA/w2gsPmNcgU=; b=qApCdM6+UzlfY2ZoypIo/OlvF3AOyt5bF6T4JkpFvn4+zZd2Tn6a+fJ76FzklOU0mY PNgTuj6S4hCQMQdZdXs+t9+TeruMv9/6owAp6MNYFubUXNidv0tCbHdUabaxr5UdgIal 5EcSEpwtZHqIJlXUxzPgeQkfkqyhnakatCJruaFntW6ENCD7SdwAQDggipBFbRi2dkst VV0GlYyQF+RSIvRCqSgfujncwkQILkvoq+75w6t++mm+ofvuWp1t5lKnLokibSPovzVJ ZfzcZcPnC0+PZbEPjuqicaWBOVefwDEvsJ4op1Tqkyh2Fi/sT/fTA4TZgUb1M2LJi/5I Kd8Q== X-Forwarded-Encrypted: i=1; AJvYcCVD1sJGky03IDHC42j11XCJL/GUYfSeBe2q8Cu8NpcSSwAPmvi6fkiryBLBrMuc78in8OY6CJPqz1DsP8MQti4tRlG4POppvriOJi5TMbLs+2SGt/Y= X-Gm-Message-State: AOJu0YwXsKAJ8S/+BEMJfWejPrAyUQkJFg3s/CHaNngqsWPFHXU3TYkt B8xiHV/6mPAZgs1sXEKilZr/6mVbT+jWqczQ35ojE1XRd41Dp0rGpY+diTMr7y5ucHrrfc65uBn NtuxTV+sDSfctunEwD4IswA== X-Google-Smtp-Source: AGHT+IFZX4OkaZT9f1Iz17d6NCT/rGoZ3ImWb/jbwJPJ+SyqpNN0LxB8nnl5a1tJ2nYlbr07zt95ivtf9PAzW7KOJQ== X-Received: from coltonlewis-kvm.c.googlers.com ([fda3:e722:ac3:cc00:2b:ff92:c0a8:14ce]) (user=coltonlewis job=sendgmr) by 2002:a05:6902:110c:b0:dda:c4ec:7db5 with SMTP id o12-20020a056902110c00b00ddac4ec7db5mr176675ybu.4.1714500926956; Tue, 30 Apr 2024 11:15:26 -0700 (PDT) Date: Tue, 30 Apr 2024 18:14:44 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.45.0.rc0.197.gbae5840b3b-goog Message-ID: <20240430181444.670773-1-coltonlewis@google.com> Subject: [PATCH v5] KVM: arm64: Add early_param to control WFx trapping From: Colton Lewis To: kvm@vger.kernel.org Cc: Jonathan Corbet , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Zenghui Yu , Catalin Marinas , Will Deacon , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, Colton Lewis X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240430_111532_022987_5180A3CE X-CRM114-Status: GOOD ( 20.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add an early_params to control WFI and WFE trapping. This is to control the degree guests can wait for interrupts on their own without being trapped by KVM. Options for each param are trap and notrap. trap enables the trap. notrap disables the trap. Absent an explicitly set policy, default to current behavior: disabling the trap if only a single task is running and enabling otherwise. Signed-off-by: Colton Lewis --- v5: * Move trap configuration to vcpu_reset_hcr(). This required moving kvm_emulate.h:vcpu_reset_hcr() to arm.c:kvm_vcpu_reset_hcr() to avoid needing to pull scheduler headers and my enums into kvm_emulate.h. I thought the function looked too bulky for that header anyway. * Delete vcpu_{set,clear}_vfx_traps helpers that are no longer used anywhere. * Remove documentation of explicit option for default behavior to avoid any implicit suggestion default behavior will stay that way. v4: https://lore.kernel.org/kvmarm/20240422181716.237284-1-coltonlewis@google.com/ v3: https://lore.kernel.org/kvmarm/20240410175437.793508-1-coltonlewis@google.com/ v2: https://lore.kernel.org/kvmarm/20240319164341.1674863-1-coltonlewis@google.com/ v1: https://lore.kernel.org/kvmarm/20240129213918.3124494-1-coltonlewis@google.com/ .../admin-guide/kernel-parameters.txt | 16 +++ arch/arm64/include/asm/kvm_emulate.h | 53 --------- arch/arm64/include/asm/kvm_host.h | 7 ++ arch/arm64/kvm/arm.c | 110 +++++++++++++++++- 4 files changed, 127 insertions(+), 59 deletions(-) -- 2.45.0.rc0.197.gbae5840b3b-goog diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 31b3a25680d0..a4d94d9abbe4 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -2653,6 +2653,22 @@ [KVM,ARM] Allow use of GICv4 for direct injection of LPIs. + kvm-arm.wfe_trap_policy= + [KVM,ARM] Control when to set WFE instruction trap for + KVM VMs. + + trap: set WFE instruction trap + + notrap: clear WFE instruction trap + + kvm-arm.wfi_trap_policy= + [KVM,ARM] Control when to set WFI instruction trap for + KVM VMs. + + trap: set WFI instruction trap + + notrap: clear WFI instruction trap + kvm_cma_resv_ratio=n [PPC] Reserves given percentage from system memory area for contiguous memory allocation for KVM hash pagetable diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h index b804fe832184..c2a9a409ebfe 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -67,64 +67,11 @@ static __always_inline bool vcpu_el1_is_32bit(struct kvm_vcpu *vcpu) } #endif -static inline void vcpu_reset_hcr(struct kvm_vcpu *vcpu) -{ - vcpu->arch.hcr_el2 = HCR_GUEST_FLAGS; - if (has_vhe() || has_hvhe()) - vcpu->arch.hcr_el2 |= HCR_E2H; - if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN)) { - /* route synchronous external abort exceptions to EL2 */ - vcpu->arch.hcr_el2 |= HCR_TEA; - /* trap error record accesses */ - vcpu->arch.hcr_el2 |= HCR_TERR; - } - - if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB)) { - vcpu->arch.hcr_el2 |= HCR_FWB; - } else { - /* - * For non-FWB CPUs, we trap VM ops (HCR_EL2.TVM) until M+C - * get set in SCTLR_EL1 such that we can detect when the guest - * MMU gets turned on and do the necessary cache maintenance - * then. - */ - vcpu->arch.hcr_el2 |= HCR_TVM; - } - - if (cpus_have_final_cap(ARM64_HAS_EVT) && - !cpus_have_final_cap(ARM64_MISMATCHED_CACHE_TYPE)) - vcpu->arch.hcr_el2 |= HCR_TID4; - else - vcpu->arch.hcr_el2 |= HCR_TID2; - - if (vcpu_el1_is_32bit(vcpu)) - vcpu->arch.hcr_el2 &= ~HCR_RW; - - if (kvm_has_mte(vcpu->kvm)) - vcpu->arch.hcr_el2 |= HCR_ATA; -} - static inline unsigned long *vcpu_hcr(struct kvm_vcpu *vcpu) { return (unsigned long *)&vcpu->arch.hcr_el2; } -static inline void vcpu_clear_wfx_traps(struct kvm_vcpu *vcpu) -{ - vcpu->arch.hcr_el2 &= ~HCR_TWE; - if (atomic_read(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe.vlpi_count) || - vcpu->kvm->arch.vgic.nassgireq) - vcpu->arch.hcr_el2 &= ~HCR_TWI; - else - vcpu->arch.hcr_el2 |= HCR_TWI; -} - -static inline void vcpu_set_wfx_traps(struct kvm_vcpu *vcpu) -{ - vcpu->arch.hcr_el2 |= HCR_TWE; - vcpu->arch.hcr_el2 |= HCR_TWI; -} - static inline void vcpu_ptrauth_enable(struct kvm_vcpu *vcpu) { vcpu->arch.hcr_el2 |= (HCR_API | HCR_APK); diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 21c57b812569..315ee7bfc1cb 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -67,6 +67,13 @@ enum kvm_mode { KVM_MODE_NV, KVM_MODE_NONE, }; + +enum kvm_wfx_trap_policy { + KVM_WFX_NOTRAP_SINGLE_TASK, /* Default option */ + KVM_WFX_NOTRAP, + KVM_WFX_TRAP, +}; + #ifdef CONFIG_KVM enum kvm_mode kvm_get_mode(void); #else diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index a25265aca432..5ec52333e042 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -46,6 +46,8 @@ #include static enum kvm_mode kvm_mode = KVM_MODE_DEFAULT; +static enum kvm_wfx_trap_policy kvm_wfi_trap_policy = KVM_WFX_NOTRAP_SINGLE_TASK; +static enum kvm_wfx_trap_policy kvm_wfe_trap_policy = KVM_WFX_NOTRAP_SINGLE_TASK; DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector); @@ -456,11 +458,6 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu) if (kvm_arm_is_pvtime_enabled(&vcpu->arch)) kvm_make_request(KVM_REQ_RECORD_STEAL, vcpu); - if (single_task_running()) - vcpu_clear_wfx_traps(vcpu); - else - vcpu_set_wfx_traps(vcpu); - if (vcpu_has_ptrauth(vcpu)) vcpu_ptrauth_disable(vcpu); kvm_arch_vcpu_load_debug_state_flags(vcpu); @@ -1391,6 +1388,72 @@ static int kvm_vcpu_set_target(struct kvm_vcpu *vcpu, return 0; } +static bool kvm_vcpu_should_clear_twi(struct kvm_vcpu *vcpu) +{ + if (likely(kvm_wfi_trap_policy == KVM_WFX_NOTRAP_SINGLE_TASK)) + return single_task_running() && + (atomic_read(&vcpu->arch.vgic_cpu.vgic_v3.its_vpe.vlpi_count) || + vcpu->kvm->arch.vgic.nassgireq); + + return kvm_wfi_trap_policy == KVM_WFX_NOTRAP; +} + +static bool kvm_vcpu_should_clear_twe(struct kvm_vcpu *vcpu) +{ + if (likely(kvm_wfe_trap_policy == KVM_WFX_NOTRAP_SINGLE_TASK)) + return single_task_running(); + + return kvm_wfe_trap_policy == KVM_WFX_NOTRAP; +} + +static inline void kvm_vcpu_reset_hcr(struct kvm_vcpu *vcpu) +{ + vcpu->arch.hcr_el2 = HCR_GUEST_FLAGS; + if (has_vhe() || has_hvhe()) + vcpu->arch.hcr_el2 |= HCR_E2H; + if (cpus_have_final_cap(ARM64_HAS_RAS_EXTN)) { + /* route synchronous external abort exceptions to EL2 */ + vcpu->arch.hcr_el2 |= HCR_TEA; + /* trap error record accesses */ + vcpu->arch.hcr_el2 |= HCR_TERR; + } + + if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB)) { + vcpu->arch.hcr_el2 |= HCR_FWB; + } else { + /* + * For non-FWB CPUs, we trap VM ops (HCR_EL2.TVM) until M+C + * get set in SCTLR_EL1 such that we can detect when the guest + * MMU gets turned on and do the necessary cache maintenance + * then. + */ + vcpu->arch.hcr_el2 |= HCR_TVM; + } + + if (cpus_have_final_cap(ARM64_HAS_EVT) && + !cpus_have_final_cap(ARM64_MISMATCHED_CACHE_TYPE)) + vcpu->arch.hcr_el2 |= HCR_TID4; + else + vcpu->arch.hcr_el2 |= HCR_TID2; + + if (vcpu_el1_is_32bit(vcpu)) + vcpu->arch.hcr_el2 &= ~HCR_RW; + + if (kvm_has_mte(vcpu->kvm)) + vcpu->arch.hcr_el2 |= HCR_ATA; + + + if (kvm_vcpu_should_clear_twe(vcpu)) + vcpu->arch.hcr_el2 &= ~HCR_TWE; + else + vcpu->arch.hcr_el2 |= HCR_TWE; + + if (kvm_vcpu_should_clear_twi(vcpu)) + vcpu->arch.hcr_el2 &= ~HCR_TWI; + else + vcpu->arch.hcr_el2 |= HCR_TWI; +} + static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu, struct kvm_vcpu_init *init) { @@ -1427,7 +1490,7 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu, icache_inval_all_pou(); } - vcpu_reset_hcr(vcpu); + kvm_vcpu_reset_hcr(vcpu); vcpu->arch.cptr_el2 = kvm_get_reset_cptr_el2(vcpu); /* @@ -2654,6 +2717,41 @@ static int __init early_kvm_mode_cfg(char *arg) } early_param("kvm-arm.mode", early_kvm_mode_cfg); +static int __init early_kvm_wfx_trap_policy_cfg(char *arg, enum kvm_wfx_trap_policy *p) +{ + if (!arg) + return -EINVAL; + + if (strcmp(arg, "trap") == 0) { + *p = KVM_WFX_TRAP; + return 0; + } + + if (strcmp(arg, "notrap") == 0) { + *p = KVM_WFX_NOTRAP; + return 0; + } + + if (strcmp(arg, "default") == 0) { + *p = KVM_WFX_NOTRAP_SINGLE_TASK; + return 0; + } + + return -EINVAL; +} + +static int __init early_kvm_wfi_trap_policy_cfg(char *arg) +{ + return early_kvm_wfx_trap_policy_cfg(arg, &kvm_wfi_trap_policy); +} +early_param("kvm-arm.wfi_trap_policy", early_kvm_wfi_trap_policy_cfg); + +static int __init early_kvm_wfe_trap_policy_cfg(char *arg) +{ + return early_kvm_wfx_trap_policy_cfg(arg, &kvm_wfe_trap_policy); +} +early_param("kvm-arm.wfe_trap_policy", early_kvm_wfe_trap_policy_cfg); + enum kvm_mode kvm_get_mode(void) { return kvm_mode;