From patchwork Fri Aug 1 08:12:07 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 4661011 Return-Path: X-Original-To: patchwork-kvm@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 58BF19F319 for ; Fri, 1 Aug 2014 08:12:49 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 6B8DA2020A for ; Fri, 1 Aug 2014 08:12:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 7061820115 for ; Fri, 1 Aug 2014 08:12:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754275AbaHAIMY (ORCPT ); Fri, 1 Aug 2014 04:12:24 -0400 Received: from mga02.intel.com ([134.134.136.20]:39958 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751602AbaHAIMT (ORCPT ); Fri, 1 Aug 2014 04:12:19 -0400 Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga101.jf.intel.com with ESMTP; 01 Aug 2014 01:12:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.01,778,1400050800"; d="scan'208";a="552253658" Received: from wanpengl-mobl.ccr.corp.intel.com (HELO localhost) ([10.238.130.35]) by orsmga001.jf.intel.com with ESMTP; 01 Aug 2014 01:12:16 -0700 From: Wanpeng Li To: Paolo Bonzini , Jan Kiszka Cc: Marcelo Tosatti , Gleb Natapov , Bandan Das , Zhang Yang , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Wanpeng Li Subject: [PATCH 2/2] KVM: nVMX: fix acknowledge interrupt on exit when APICv is in use Date: Fri, 1 Aug 2014 16:12:07 +0800 Message-Id: <1406880727-60001-2-git-send-email-wanpeng.li@linux.intel.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1406880727-60001-1-git-send-email-wanpeng.li@linux.intel.com> References: <1406880727-60001-1-git-send-email-wanpeng.li@linux.intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Spam-Status: No, score=-7.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP After commit 77b0f5d (KVM: nVMX: Ack and write vector info to intr_info if L1 asks us to), "Acknowledge interrupt on exit" behavior can be emulated. To do so, KVM will ask the APIC for the interrupt vector if during a nested vmexit if VM_EXIT_ACK_INTR_ON_EXIT is set. With APICv, kvm_get_apic_interrupt would return -1 and give the following WARNING: Call Trace: [] dump_stack+0x49/0x5e [] warn_slowpath_common+0x7c/0x96 [] ? nested_vmx_vmexit+0xa4/0x233 [kvm_intel] [] warn_slowpath_null+0x15/0x17 [] nested_vmx_vmexit+0xa4/0x233 [kvm_intel] [] ? nested_vmx_exit_handled+0x6a/0x39e [kvm_intel] [] ? kvm_apic_has_interrupt+0x80/0xd5 [kvm] [] vmx_check_nested_events+0xc3/0xd3 [kvm_intel] [] inject_pending_event+0xd0/0x16e [kvm] [] vcpu_enter_guest+0x319/0x704 [kvm] If enabling APIC-v, all interrupts to L1 are delivered through APIC-v. But when L2 is running, external interrupt will casue L1 vmexit with reason external interrupt. Then L1 will pick up the interrupt through vmcs12. when L1 ack the interrupt, since the APIC-v is enabled when L1 is running, so APIC-v hardware still will do vEOI updating. The problem is that the interrupt is delivered not through APIC-v hardware, this means SVI/RVI/vPPR are not setting, but hardware required them when doing vEOI updating. The solution is that, when L1 tried to pick up the interrupt from vmcs12, then hypervisor will help to update the SVI/RVI/vPPR to make sure the following vEOI updating and vPPR updating corrently. Also, since interrupt is delivered through vmcs12, so APIC-v hardware will not cleare vIRR and hypervisor need to clear it before L1 running. Suggested-by: Paolo Bonzini Suggested-by: "Zhang, Yang Z" Signed-off-by: Wanpeng Li Tested-by: Liu, RongrongX --- arch/x86/kvm/lapic.c | 18 ++++++++++++++++++ arch/x86/kvm/lapic.h | 1 + arch/x86/kvm/vmx.c | 10 ++++++++++ 3 files changed, 29 insertions(+) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 3855103..06942b9 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -534,6 +534,24 @@ static void apic_set_tpr(struct kvm_lapic *apic, u32 tpr) apic_update_ppr(apic); } +int kvm_lapic_ack_apicv(struct kvm_vcpu *vcpu) +{ + struct kvm_lapic *apic = vcpu->arch.apic; + int vec; + + vec = kvm_apic_has_interrupt(vcpu); + + if (vec == -1) + return vec; + + apic_set_vector(vec, apic->regs + APIC_ISR); + apic_update_ppr(apic); + apic_clear_vector(vec, apic->regs + APIC_IRR); + + return vec; +} +EXPORT_SYMBOL_GPL(kvm_lapic_ack_apicv); + int kvm_apic_match_physical_addr(struct kvm_lapic *apic, u16 dest) { return dest == 0xff || kvm_apic_id(apic) == dest; diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h index 6a11845..ead1392 100644 --- a/arch/x86/kvm/lapic.h +++ b/arch/x86/kvm/lapic.h @@ -169,5 +169,6 @@ static inline bool kvm_apic_has_events(struct kvm_vcpu *vcpu) } bool kvm_apic_pending_eoi(struct kvm_vcpu *vcpu, int vector); +int kvm_lapic_ack_apicv(struct kvm_vcpu *vcpu); #endif diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index b8122b3..c604f3c 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -8766,6 +8766,16 @@ static void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 exit_reason, if ((exit_reason == EXIT_REASON_EXTERNAL_INTERRUPT) && nested_exit_intr_ack_set(vcpu)) { int irq = kvm_cpu_get_interrupt(vcpu); + + if (irq < 0 && kvm_apic_vid_enabled(vcpu->kvm)) { + irq = kvm_lapic_ack_apicv(vcpu); + if (irq >= 0) { + vmx_hwapic_isr_update(vcpu->kvm, irq); + /* try to update RVI */ + kvm_make_request(KVM_REQ_EVENT, vcpu); + } + } + WARN_ON(irq < 0); vmcs12->vm_exit_intr_info = irq | INTR_INFO_VALID_MASK | INTR_TYPE_EXT_INTR;