From patchwork Tue Jan 16 19:00:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: David Woodhouse X-Patchwork-Id: 13521049 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC90C1D520 for ; Tue, 16 Jan 2024 19:00:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=casper.srs.infradead.org Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=90.155.50.34 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705431621; cv=none; b=lx+mCF6X+K7daBTyF/BKnu+bsd86o9hU2pg5V0TP7J3yLcZWN3KcCw1awzBsQzP+aD3eEg7+DUb5INb2zQAJApVi3hnWkg8/+/GMqHep/bscQ7RYZ0K5fgFryEVucY3uadqQcmC1rz0JbZCAeTPstV2RQhu0aK6pq3VexOVwkHo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1705431621; c=relaxed/simple; bh=I/B1Rb4QKBzj2FKjN8c30plCxufX4d/hoLPTQr5CT0w=; h=DKIM-Signature:Received:Message-ID:Subject:From:To:Cc:Date: Content-Type:User-Agent:MIME-Version:X-SRS-Rewrite; b=XNTZU9Tt8auF5xz8VK8IwE9VRZU7kQbMT9cY7KhKXcMiuxR49grSAWV5uw/hwTOK9ie+rrMbfxQW/C1bDa9lQidGv5tLKwl+Eb3D5K3i1m63WKSp5bieTZgbV/WlDTx7e5qHfYs/v1nc8fbvK6YPLDOw/+QwdDMytnE2MbdIo+0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=ipxoBMWE; arc=none smtp.client-ip=90.155.50.34 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="ipxoBMWE" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=MIME-Version:Content-Type:Date:Cc:To: From:Subject:Message-ID:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:In-Reply-To:References; bh=wSh6S/huFsWR3yb4sOLAdwF0qi/MuTayzbcZT74UPkg=; b=ipxoBMWEgg7EOkX1nTg8F4A0Uo wJbgQutiP3OybWYAuv0Japy1MfRkaVIrz3fngsfruaDshZ42qsxrVYLiZC3hu0GgARr9rw8hYcPf6 lc9UM/nGHyEiUZw/DoX3erK99+lyVVQSIlyZbbmO0rXCiEwm53NBW7S8V9Ljc6AWkUwY9G1VYfz0j VVSu2/ENIt+PFvG0UTVwFAaYQKcV3clWfGhIutOUvGSMwK/DNogd9jofDYyxSsqs5UdXGB731ZbeF W+ojFfhh+uWC2LSbUUxBakjTWkeibIKORDm2cQjmFnujfe74DcadKM29uUcDhKu8zjbbH/fJMa0C8 k9Y27UcA==; Received: from [54.239.6.188] (helo=u3832b3a9db3152.ant.amazon.com) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1rPofX-00Dw1A-Gz; Tue, 16 Jan 2024 19:00:16 +0000 Message-ID: <6150a0a8c3d911c6c2ada23c0b9c8b35991bd235.camel@infradead.org> Subject: [PATCH v4] KVM: x86/xen: Inject vCPU upcall vector when local APIC is enabled From: David Woodhouse To: kvm Cc: Sean Christopherson , Paolo Bonzini , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Paul Durrant Date: Tue, 16 Jan 2024 20:00:14 +0100 User-Agent: Evolution 3.44.4-0ubuntu2 Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html From: David Woodhouse Linux guests since commit b1c3497e604d ("x86/xen: Add support for HVMOP_set_evtchn_upcall_vector") in v6.0 onwards will use the per-vCPU upcall vector when it's advertised in the Xen CPUID leaves. This upcall is injected through the guest's local APIC as an MSI, unlike the older system vector which was merely injected by the hypervisor any time the CPU was able to receive an interrupt and the upcall_pending flags is set in its vcpu_info. Effectively, that makes the per-CPU upcall edge triggered instead of level triggered, which results in the upcall being lost if the MSI is delivered when the local APIC is *disabled*. Xen checks the vcpu_info->evtchn_upcall_pending flag when the local APIC for a vCPU is software enabled (in fact, on any write to the SPIV register which doesn't disable the APIC). Do the same in KVM since KVM doesn't provide a way for userspace to intervene and trap accesses to the SPIV register of a local APIC emulated by KVM. Astute reviewers may note that kvm_xen_inject_vcpu_vector() function has a WARN_ON_ONCE() in the case where kvm_irq_delivery_to_apic_fast() fails and returns false. In the case where the MSI is not delivered due to the local APIC being disabled, kvm_irq_delivery_to_apic_fast() still returns true but the value in *r is zero. So the WARN_ON_ONCE() remains correct, as that case should still never happen. Fixes: fde0451be8fb3 ("KVM: x86/xen: Support per-vCPU event channel upcall via local APIC") Signed-off-by: David Woodhouse Reviewed-by: Paul Durrant Cc: stable@vger.kernel.org --- v4: Reword commit message, rename kvm_xen_enable_lapic() → kvm_xen_sw_enable_lapic(). v3: Repost, add Cc:stable. v2: Add Fixes: tag.  arch/x86/kvm/lapic.c |  5 ++++-  arch/x86/kvm/xen.c   |  2 +-  arch/x86/kvm/xen.h   | 18 ++++++++++++++++++  3 files changed, 23 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 3242f3da2457..75bc7d3f0022 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -41,6 +41,7 @@  #include "ioapic.h"  #include "trace.h"  #include "x86.h" +#include "xen.h"  #include "cpuid.h"  #include "hyperv.h"  #include "smm.h" @@ -499,8 +500,10 @@ static inline void apic_set_spiv(struct kvm_lapic *apic, u32 val)         }           /* Check if there are APF page ready requests pending */ -       if (enabled) +       if (enabled) {                 kvm_make_request(KVM_REQ_APF_READY, apic->vcpu); +               kvm_xen_sw_enable_lapic(apic->vcpu); +       }  }    static inline void kvm_apic_set_xapic_id(struct kvm_lapic *apic, u8 id) diff --git a/arch/x86/kvm/xen.c b/arch/x86/kvm/xen.c index 8ef668922340..7c602037b596 100644 --- a/arch/x86/kvm/xen.c +++ b/arch/x86/kvm/xen.c @@ -568,7 +568,7 @@ void kvm_xen_update_runstate(struct kvm_vcpu *v, int state)                 kvm_xen_update_runstate_guest(v, state == RUNSTATE_runnable);  }   -static void kvm_xen_inject_vcpu_vector(struct kvm_vcpu *v) +void kvm_xen_inject_vcpu_vector(struct kvm_vcpu *v)  {         struct kvm_lapic_irq irq = { };         int r; diff --git a/arch/x86/kvm/xen.h b/arch/x86/kvm/xen.h index f8f1fe22d090..f5841d9000ae 100644 --- a/arch/x86/kvm/xen.h +++ b/arch/x86/kvm/xen.h @@ -18,6 +18,7 @@ extern struct static_key_false_deferred kvm_xen_enabled;    int __kvm_xen_has_interrupt(struct kvm_vcpu *vcpu);  void kvm_xen_inject_pending_events(struct kvm_vcpu *vcpu); +void kvm_xen_inject_vcpu_vector(struct kvm_vcpu *vcpu);  int kvm_xen_vcpu_set_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data);  int kvm_xen_vcpu_get_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data);  int kvm_xen_hvm_set_attr(struct kvm *kvm, struct kvm_xen_hvm_attr *data); @@ -36,6 +37,19 @@ int kvm_xen_setup_evtchn(struct kvm *kvm,                          const struct kvm_irq_routing_entry *ue);  void kvm_xen_update_tsc_info(struct kvm_vcpu *vcpu);   +static inline void kvm_xen_sw_enable_lapic(struct kvm_vcpu *vcpu) +{ +       /* +        * The local APIC is being enabled. If the per-vCPU upcall vector is +        * set and the vCPU's evtchn_upcall_pending flag is set, inject the +        * interrupt. +        */ +       if (static_branch_unlikely(&kvm_xen_enabled.key) && +           vcpu->arch.xen.vcpu_info_cache.active && +           vcpu->arch.xen.upcall_vector && __kvm_xen_has_interrupt(vcpu)) +               kvm_xen_inject_vcpu_vector(vcpu); +} +  static inline bool kvm_xen_msr_enabled(struct kvm *kvm)  {         return static_branch_unlikely(&kvm_xen_enabled.key) && @@ -101,6 +115,10 @@ static inline void kvm_xen_destroy_vcpu(struct kvm_vcpu *vcpu)  {  }   +static inline void kvm_xen_sw_enable_lapic(struct kvm_vcpu *vcpu) +{ +} +  static inline bool kvm_xen_msr_enabled(struct kvm *kvm)  {         return false;