From patchwork Mon Nov 18 10:16:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Roger_Pau_Monn=C3=A9?= X-Patchwork-Id: 11249281 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B2F88913 for ; Mon, 18 Nov 2019 10:17:42 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 84E1D20727 for ; Mon, 18 Nov 2019 10:17:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="b1dbxLY6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 84E1D20727 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iWe4o-0006wL-1X; Mon, 18 Nov 2019 10:16:10 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iWe4m-0006wG-KL for xen-devel@lists.xenproject.org; Mon, 18 Nov 2019 10:16:08 +0000 X-Inumbo-ID: 6f12876a-09ec-11ea-a2d6-12813bfff9fa Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 6f12876a-09ec-11ea-a2d6-12813bfff9fa; Mon, 18 Nov 2019 10:16:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1574072168; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=QOU/IDHRg9Q3sip9K6IbB60HrqlsuNqUtHrcNn9zOW8=; b=b1dbxLY6jy6+sGuYz9HfqARHLtx2PsL9ch/Om8GBWAWvXKFBPXfRyeNf ftR2Oc65grINXU3PJr7mIcAutZFXCrnUtN3slN6UnYTM7wFT0DZbjJCo0 YcZO1VMaPbwfZSjpiSHkNaaME8V8CrB7Bk2Xl90QWM8QtWRN3VvGWDbWJ Q=; Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa2.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: EhnM2k3gB/Y6HYdGHU6XsGx0E+e78oKBdI+MuIl2wRgAHEFNMb6ev4sWgLpBA/kh607Jb/uCCk GNJV0u4U6vMlaDCI/pzZ4R04ZD2oMzu3cXAoQ60QZFYn85rIUMWKWbOqlhhDfko5NnuTj90HPz rzzU8oHg3KRcriCpgap4YvQLl3H2v8V2xDWslos1ZO7JDXNWibLTaZY9528JkVhqRnS+MwTXym kk4sKC/Pc4k9T9OPLGnUKSAlRmUDZpqQXTpWudxKfryydtswiS25n6qXcWx1tjM6oeQ4PW5TO8 +XU= X-SBRS: 2.7 X-MesageID: 8456562 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.68,319,1569297600"; d="scan'208";a="8456562" From: Roger Pau Monne To: Date: Mon, 18 Nov 2019 11:16:00 +0100 Message-ID: <20191118101600.94645-1-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.0 MIME-Version: 1.0 Subject: [Xen-devel] [PATCH for-4.13] x86/vmx: always sync PIR to IRR before vmentry X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Kevin Tian , Jun Nakajima , Wei Liu , Andrew Cooper , Joe Jin , Jan Beulich , Roger Pau Monne Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" When using posted interrupts on Intel hardware it's possible that the vCPU resumes execution with a stale local APIC IRR register because depending on the interrupts to be injected vlapic_has_pending_irq might not be called, and thus PIR won't be synced into IRR. Fix this by making sure PIR is always synced to IRR in vmx_intr_assist regardless of what interrupts are pending. While there also simplify the code in __vmx_deliver_posted_interrupt: there's no need to raise a softirq if the destination vCPU is the one currently running on the pCPU. The sync of PIR into IRR will be performed by vmx_intr_assist regardless of whether there's a softirq pending. Reported-by: Joe Jin Signed-off-by: Roger Pau Monné Tested-by: Joe Jin --- xen/arch/x86/hvm/vmx/intr.c | 8 +++++ xen/arch/x86/hvm/vmx/vmx.c | 60 ++++++++++--------------------- xen/include/asm-x86/hvm/vmx/vmx.h | 1 + 3 files changed, 28 insertions(+), 41 deletions(-) diff --git a/xen/arch/x86/hvm/vmx/intr.c b/xen/arch/x86/hvm/vmx/intr.c index 0d097cf1f2..ce70f4bc75 100644 --- a/xen/arch/x86/hvm/vmx/intr.c +++ b/xen/arch/x86/hvm/vmx/intr.c @@ -232,6 +232,14 @@ void vmx_intr_assist(void) enum hvm_intblk intblk; int pt_vector; + if ( cpu_has_vmx_posted_intr_processing ) + /* + * Always force PIR to be synced to IRR before vmentry, this is also + * done by vlapic_has_pending_irq but it's possible other pending + * interrupts prevent the execution of that function. + */ + vmx_sync_pir_to_irr(v); + /* Block event injection when single step with MTF. */ if ( unlikely(v->arch.hvm.single_step) ) { diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index 33e68eaddf..82a1b972c5 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -1945,8 +1945,6 @@ static void vmx_process_isr(int isr, struct vcpu *v) static void __vmx_deliver_posted_interrupt(struct vcpu *v) { - bool_t running = v->is_running; - vcpu_unblock(v); /* * Just like vcpu_kick(), nothing is needed for the following two cases: @@ -1954,48 +1952,28 @@ static void __vmx_deliver_posted_interrupt(struct vcpu *v) * 2. The target vCPU is the current vCPU and we're in non-interrupt * context. */ - if ( running && (in_irq() || (v != current)) ) - { + if ( vcpu_runnable(v) && v != current ) /* - * Note: Only two cases will reach here: - * 1. The target vCPU is running on other pCPU. - * 2. The target vCPU is the current vCPU. + * If the vCPU is running on another pCPU send an IPI to the pCPU. When + * the IPI arrives, the target vCPU may be running in non-root mode, + * running in root mode, runnable or blocked. If the target vCPU is + * running in non-root mode, the hardware will sync PIR to vIRR for + * 'posted_intr_vector' is a special vector handled directly by the + * hardware. * - * Note2: Don't worry the v->processor may change. The vCPU being - * moved to another processor is guaranteed to sync PIR to vIRR, - * due to the involved scheduling cycle. + * If the target vCPU is running in root-mode, the interrupt handler + * starts to run. Considering an IPI may arrive in the window between + * the call to vmx_intr_assist() and interrupts getting disabled, the + * interrupt handler should raise a softirq to ensure events will be + * delivered in time. */ - unsigned int cpu = v->processor; + send_IPI_mask(cpumask_of(v->processor), posted_intr_vector); - /* - * For case 1, we send an IPI to the pCPU. When an IPI arrives, the - * target vCPU maybe is running in non-root mode, running in root - * mode, runnable or blocked. If the target vCPU is running in - * non-root mode, the hardware will sync PIR to vIRR for - * 'posted_intr_vector' is special to the pCPU. If the target vCPU is - * running in root-mode, the interrupt handler starts to run. - * Considering an IPI may arrive in the window between the call to - * vmx_intr_assist() and interrupts getting disabled, the interrupt - * handler should raise a softirq to ensure events will be delivered - * in time. If the target vCPU is runnable, it will sync PIR to - * vIRR next time it is chose to run. In this case, a IPI and a - * softirq is sent to a wrong vCPU which will not have any adverse - * effect. If the target vCPU is blocked, since vcpu_block() checks - * whether there is an event to be delivered through - * local_events_need_delivery() just after blocking, the vCPU must - * have synced PIR to vIRR. Similarly, there is a IPI and a softirq - * sent to a wrong vCPU. - */ - if ( cpu != smp_processor_id() ) - send_IPI_mask(cpumask_of(cpu), posted_intr_vector); - /* - * For case 2, raising a softirq ensures PIR will be synced to vIRR. - * As any softirq will do, as an optimization we only raise one if - * none is pending already. - */ - else if ( !softirq_pending(cpu) ) - raise_softirq(VCPU_KICK_SOFTIRQ); - } + /* + * If the vCPU is not runnable or if it's the one currently running in this + * pCPU there's nothing to do, the vmentry code will already sync the PIR + * to IRR when resuming execution. + */ } static void vmx_deliver_posted_intr(struct vcpu *v, u8 vector) @@ -2048,7 +2026,7 @@ static void vmx_deliver_posted_intr(struct vcpu *v, u8 vector) vcpu_kick(v); } -static void vmx_sync_pir_to_irr(struct vcpu *v) +void vmx_sync_pir_to_irr(struct vcpu *v) { struct vlapic *vlapic = vcpu_vlapic(v); unsigned int group, i; diff --git a/xen/include/asm-x86/hvm/vmx/vmx.h b/xen/include/asm-x86/hvm/vmx/vmx.h index ebaa74449b..c43fab7c4f 100644 --- a/xen/include/asm-x86/hvm/vmx/vmx.h +++ b/xen/include/asm-x86/hvm/vmx/vmx.h @@ -101,6 +101,7 @@ void vmx_update_debug_state(struct vcpu *v); void vmx_update_exception_bitmap(struct vcpu *v); void vmx_update_cpu_exec_control(struct vcpu *v); void vmx_update_secondary_exec_control(struct vcpu *v); +void vmx_sync_pir_to_irr(struct vcpu *v); #define POSTED_INTR_ON 0 #define POSTED_INTR_SN 1