From patchwork Wed Oct 9 09:05:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 11180859 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C66A517D4 for ; Wed, 9 Oct 2019 09:07:21 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 92AF72133F for ; Wed, 9 Oct 2019 09:07:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="XCmx+wfj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 92AF72133F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iI7v0-0002Vj-Lh; Wed, 09 Oct 2019 09:06:02 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iI7uz-0002Ve-Ts for xen-devel@lists.xenproject.org; Wed, 09 Oct 2019 09:06:01 +0000 X-Inumbo-ID: 034e1608-ea74-11e9-8c93-bc764e2007e4 Received: from esa5.hc3370-68.iphmx.com (unknown [216.71.155.168]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 034e1608-ea74-11e9-8c93-bc764e2007e4; Wed, 09 Oct 2019 09:06:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1570611960; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=KWOarQIqm0IcJJ0LDY6aQIWZiun98l7e0RuwTp9nj34=; b=XCmx+wfj3g8FIQe19djQM0xqAbRq1TjWsWi7GXrhFgm5wL621az/AhCm GUKGviVJokAoA3QdS4RkoAfyKPGiGdK9jOJgB4CKv6AHvWi+wgt+FJQB6 pBZzePLoYI6xyNZyryoTbasQHJaU9c96dKd4nN57gMIRN435YXFIG5sX7 E=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa5.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa5.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa5.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: 0zstpCwJp6egxKuZ54uTnletTW0PMUlQIwFV+6rizkqSZdbEeGpZBK/ruaCy7vgfJFEKsDiRiP RxAP7MKWXEBEr5k0Y+9l4VOsLYvYtd4uvAiybvlnDMf1o4wrZ6gDaimlsfn2RBsB50ZqxLUPTY lHb14voO1GjFKZKxA40ZuT8SndAghN8cbh0DMh/V9uYUYRbKYEZO0k46jEG++Y4PxF1IpXAKcJ 2imCw8WA862f5hDNd5FIafp8GSsADhfvWaMhEu6srBS8vVNbi4EYEjwoNcnJM/152NQ6Hd0Hdy 3xM= X-SBRS: 2.7 X-MesageID: 6910163 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.67,273,1566878400"; d="scan'208";a="6910163" From: Roger Pau Monne To: Date: Wed, 9 Oct 2019 11:05:39 +0200 Message-ID: <20191009090539.2141-1-roger.pau@citrix.com> X-Mailer: git-send-email 2.23.0 MIME-Version: 1.0 Subject: [Xen-devel] [PATCH for-4.13] x86/passthrough: fix migration of MSI when using posted interrupts X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Kevin Tian , Wei Liu , Andrew Cooper , Joe Jin , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" When using posted interrupts and the guest migrates MSI from vCPUs Xen needs to flush any pending PIRR vectors on the previous vCPU, or else those vectors could get wrongly injected at a later point when the MSI fields are already updated. Rename sync_pir_to_irr to vlapic_sync_pir_to_irr and export it so it can be called when updating the posted interrupt descriptor field in pi_update_irte. While there also remove the unlock_out from pi_update_irte, it's used in a single goto and removing it makes the function smaller. Note that PIRR is synced to IRR both in pt_irq_destroy_bind and pt_irq_create_bind when the interrupt delivery data is being updated. While there guard pi_update_irte with CONFIG_HVM since it's only used with HVM guests. Reported-by: Joe Jin Signed-off-by: Roger Pau Monné --- Cc: Joe Jin Cc: Juergen Gross --- I would like to see a bug fix for this issue in 4.13. The fix here only affects posted interrupts, hence I think the risk of breaking anything else is low. --- xen/arch/x86/hvm/vlapic.c | 6 +++--- xen/drivers/passthrough/io.c | 10 +++++++--- xen/drivers/passthrough/vtd/intremap.c | 15 ++++++++------- xen/include/asm-x86/hvm/vlapic.h | 2 ++ xen/include/asm-x86/iommu.h | 2 +- 5 files changed, 21 insertions(+), 14 deletions(-) diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c index 9466258d6f..d255ad8db7 100644 --- a/xen/arch/x86/hvm/vlapic.c +++ b/xen/arch/x86/hvm/vlapic.c @@ -106,7 +106,7 @@ static void vlapic_clear_irr(int vector, struct vlapic *vlapic) vlapic_clear_vector(vector, &vlapic->regs->data[APIC_IRR]); } -static void sync_pir_to_irr(struct vcpu *v) +void vlapic_sync_pir_to_irr(struct vcpu *v) { if ( hvm_funcs.sync_pir_to_irr ) alternative_vcall(hvm_funcs.sync_pir_to_irr, v); @@ -114,7 +114,7 @@ static void sync_pir_to_irr(struct vcpu *v) static int vlapic_find_highest_irr(struct vlapic *vlapic) { - sync_pir_to_irr(vlapic_vcpu(vlapic)); + vlapic_sync_pir_to_irr(vlapic_vcpu(vlapic)); return vlapic_find_highest_vector(&vlapic->regs->data[APIC_IRR]); } @@ -1493,7 +1493,7 @@ static int lapic_save_regs(struct vcpu *v, hvm_domain_context_t *h) if ( !has_vlapic(v->domain) ) return 0; - sync_pir_to_irr(v); + vlapic_sync_pir_to_irr(v); return hvm_save_entry(LAPIC_REGS, v->vcpu_id, h, vcpu_vlapic(v)->regs); } diff --git a/xen/drivers/passthrough/io.c b/xen/drivers/passthrough/io.c index b292e79382..88188d2d7f 100644 --- a/xen/drivers/passthrough/io.c +++ b/xen/drivers/passthrough/io.c @@ -341,7 +341,7 @@ int pt_irq_create_bind( { uint8_t dest, delivery_mode; bool dest_mode; - int dest_vcpu_id; + int dest_vcpu_id, prev_vcpu_id = -1; const struct vcpu *vcpu; uint32_t gflags = pt_irq_bind->u.msi.gflags & ~XEN_DOMCTL_VMSI_X86_UNMASKED; @@ -411,6 +411,7 @@ int pt_irq_create_bind( pirq_dpci->gmsi.gvec = pt_irq_bind->u.msi.gvec; pirq_dpci->gmsi.gflags = gflags; + prev_vcpu_id = pirq_dpci->gmsi.dest_vcpu_id; } } /* Calculate dest_vcpu_id for MSI-type pirq migration. */ @@ -440,7 +441,8 @@ int pt_irq_create_bind( /* Use interrupt posting if it is supported. */ if ( iommu_intpost ) pi_update_irte(vcpu ? &vcpu->arch.hvm.vmx.pi_desc : NULL, - info, pirq_dpci->gmsi.gvec); + info, pirq_dpci->gmsi.gvec, + prev_vcpu_id >= 0 ? d->vcpu[prev_vcpu_id] : NULL); if ( pt_irq_bind->u.msi.gflags & XEN_DOMCTL_VMSI_X86_UNMASKED ) { @@ -729,7 +731,9 @@ int pt_irq_destroy_bind( what = "bogus"; } else if ( pirq_dpci && pirq_dpci->gmsi.posted ) - pi_update_irte(NULL, pirq, 0); + pi_update_irte(NULL, pirq, 0, + pirq_dpci->gmsi.dest_vcpu_id >= 0 + ? d->vcpu[pirq_dpci->gmsi.dest_vcpu_id] : NULL); if ( pirq_dpci && (pirq_dpci->flags & HVM_IRQ_DPCI_MAPPED) && list_empty(&pirq_dpci->digl_list) ) diff --git a/xen/drivers/passthrough/vtd/intremap.c b/xen/drivers/passthrough/vtd/intremap.c index bf846195c4..0f364e81b0 100644 --- a/xen/drivers/passthrough/vtd/intremap.c +++ b/xen/drivers/passthrough/vtd/intremap.c @@ -946,12 +946,13 @@ void intel_iommu_disable_eim(void) disable_qinval(drhd->iommu); } +#if CONFIG_HVM /* * This function is used to update the IRTE for posted-interrupt * when guest changes MSI/MSI-X information. */ int pi_update_irte(const struct pi_desc *pi_desc, const struct pirq *pirq, - const uint8_t gvec) + const uint8_t gvec, struct vcpu *prev) { struct irq_desc *desc; struct msi_desc *msi_desc; @@ -964,8 +965,8 @@ int pi_update_irte(const struct pi_desc *pi_desc, const struct pirq *pirq, msi_desc = desc->msi_desc; if ( !msi_desc ) { - rc = -ENODEV; - goto unlock_out; + spin_unlock_irq(&desc->lock); + return -ENODEV; } msi_desc->pi_desc = pi_desc; msi_desc->gvec = gvec; @@ -974,10 +975,10 @@ int pi_update_irte(const struct pi_desc *pi_desc, const struct pirq *pirq, ASSERT(pcidevs_locked()); - return msi_msg_write_remap_rte(msi_desc, &msi_desc->msg); - - unlock_out: - spin_unlock_irq(&desc->lock); + rc = msi_msg_write_remap_rte(msi_desc, &msi_desc->msg); + if ( !rc && prev ) + vlapic_sync_pir_to_irr(prev); return rc; } +#endif diff --git a/xen/include/asm-x86/hvm/vlapic.h b/xen/include/asm-x86/hvm/vlapic.h index dde66b4f0f..b0017d1dae 100644 --- a/xen/include/asm-x86/hvm/vlapic.h +++ b/xen/include/asm-x86/hvm/vlapic.h @@ -150,4 +150,6 @@ bool_t vlapic_match_dest( const struct vlapic *target, const struct vlapic *source, int short_hand, uint32_t dest, bool_t dest_mode); +void vlapic_sync_pir_to_irr(struct vcpu *v); + #endif /* __ASM_X86_HVM_VLAPIC_H__ */ diff --git a/xen/include/asm-x86/iommu.h b/xen/include/asm-x86/iommu.h index 85741f7c96..314dcfbe47 100644 --- a/xen/include/asm-x86/iommu.h +++ b/xen/include/asm-x86/iommu.h @@ -119,7 +119,7 @@ static inline void iommu_disable_x2apic(void) extern bool untrusted_msi; int pi_update_irte(const struct pi_desc *pi_desc, const struct pirq *pirq, - const uint8_t gvec); + const uint8_t gvec, struct vcpu *prev); #endif /* !__ARCH_X86_IOMMU_H__ */ /*