From patchwork Mon Feb 27 11:35:20 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 9593037 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 10DEE604AB for ; Mon, 27 Feb 2017 12:01:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 06A4D28236 for ; Mon, 27 Feb 2017 12:01:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EFC0928488; Mon, 27 Feb 2017 12:01:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 921CC28236 for ; Mon, 27 Feb 2017 12:01:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751881AbdB0MBA (ORCPT ); Mon, 27 Feb 2017 07:01:00 -0500 Received: from mail-wr0-f194.google.com ([209.85.128.194]:36076 "EHLO mail-wr0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751550AbdB0MA6 (ORCPT ); Mon, 27 Feb 2017 07:00:58 -0500 Received: by mail-wr0-f194.google.com with SMTP id l37so1249620wrc.3; Mon, 27 Feb 2017 04:00:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=aVbqw3H1lx8v5dCBKTXPIZjwNPx9fxZNmmzzGkrj3cc=; b=tSHGIFBw3uZkS0ddmUYVCIh0KwR4DbVMxgsj60PavfzhvtL+nV6YNj4oCQ9W5PMy+Q QPsFUKShtCsZsDeLCspw1yaMlUtHD2Nt3T5IEUZTR1PSmY8AdCoTSy4/Iyu55J40KmLI 2Qj61Lqd9iO6jCOTC6FEaRBmBbXg7sb65EoXcdg9SkKzMgRkFe5nVk7ttZYIujipnwtm +bqK/Qe77GiRUNkQEUyPGAcxjVNGV8yvDQjxvQeMs9af0R/HeHkQeFPRcv40FoZT4k0L PAjkDookCA+e7cCv2r14Jn0c0vnxTeSkkqgVFG2uvQ1eY9hbWz0lkrXlV6EYJjX2W8SU WaGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=aVbqw3H1lx8v5dCBKTXPIZjwNPx9fxZNmmzzGkrj3cc=; b=Sd/sJIyaZ8bUAPmBaOmYohnXNFNohdyKBzG/hFntayjc9qI7Tmhbk/eVqlE9YW9P2n jVGgxuKNrQC4xFF9o/cVCt/XQnf9F4bbUnxvPEce4vqssiVlJU4I8tDbsJMea+uJtrl0 lT46w1lOkby3YZA/+9Z5W/F2xH9Gk+RD8/tqsK552QRINI36Vz7kPaKlgPNgF2FZycgZ aqX5BfskvGmlZfRu0uuigXpS1xxwrTSqeeF5s8YHtlJGTPYTxEP36uZI+g0oj3jG9voa dIV5q+mxq234cdtVxgN4+GqA0mSCUaGs4YvOB2Wk4mYWSbrgjmGXKYvegADon0OtvwSe 3zng== X-Gm-Message-State: AMke39nhQljiPmwmJYyKh89NYOhYcrKFGTDzpP2jRqLhiknlQSVmuLNbe3kQ6KunHxCgMRIYu1KJGppBUFWKxA== X-Received: by 10.223.128.231 with SMTP id 94mr13413021wrl.165.1488195321189; Mon, 27 Feb 2017 03:35:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.80.140.69 with HTTP; Mon, 27 Feb 2017 03:35:20 -0800 (PST) In-Reply-To: References: <1488098811-24953-1-git-send-email-wanpeng.li@hotmail.com> From: Wanpeng Li Date: Mon, 27 Feb 2017 19:35:20 +0800 Message-ID: Subject: Re: [PATCH] KVM: nVMX: Fix pending events injection To: Paolo Bonzini Cc: "linux-kernel@vger.kernel.org" , kvm , =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= , Wanpeng Li , Jan Kiszka Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP 2017-02-27 18:19 GMT+08:00 Paolo Bonzini : > > > On 26/02/2017 09:46, Wanpeng Li wrote: >> From: Wanpeng Li >> >> L2 fails to boot on a non-APICv box dues to 'commit 0ad3bed6c5ec >> ("kvm: nVMX: move nested events check to kvm_vcpu_running")' >> >> KVM internal error. Suberror: 3 >> extra data[0]: 800000ef >> extra data[1]: 1 >> RAX=0000000000000000 RBX=ffffffff81f36140 RCX=0000000000000000 RDX=0000000000000000 >> RSI=0000000000000000 RDI=0000000000000000 RBP=ffff88007c92fe90 RSP=ffff88007c92fe90 >> R8 =ffff88007fccdca0 R9 =0000000000000000 R10=00000000fffedb3d R11=0000000000000000 >> R12=0000000000000003 R13=0000000000000000 R14=0000000000000000 R15=ffff88007c92c000 >> RIP=ffffffff810645e6 RFL=00000246 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 >> ES =0000 0000000000000000 ffffffff 00c00000 >> CS =0010 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA] >> SS =0000 0000000000000000 ffffffff 00c00000 >> DS =0000 0000000000000000 ffffffff 00c00000 >> FS =0000 0000000000000000 ffffffff 00c00000 >> GS =0000 ffff88007fcc0000 ffffffff 00c00000 >> LDT=0000 0000000000000000 ffffffff 00c00000 >> TR =0040 ffff88007fcd4200 00002087 00008b00 DPL=0 TSS64-busy >> GDT= ffff88007fcc9000 0000007f >> IDT= ffffffffff578000 00000fff >> CR0=80050033 CR2=00000000ffffffff CR3=0000000001e0a000 CR4=003406e0 >> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 >> DR6=00000000fffe0ff0 DR7=0000000000000400 >> EFER=0000000000000d01 >> >> We should try to reinject previous events if any before trying to inject >> new event if pending. If vmexit is triggered by L2 guest and L0 interested >> in, we should reinject IDT-vectoring info to L2 through vmcs02 if any, >> otherwise, we can consider new IRQs/NMIs which can be injected and call >> nested events callback to switch from L2 to L1 if needed and inject the >> proper vmexit events. However, 'commit 0ad3bed6c5ec ("kvm: nVMX: move >> nested events check to kvm_vcpu_running")' results in the handle events >> order reversely on non-APICv box. This patch fixes it by checking nested >> events if there is no KVM_REQ_EVENT since APICv interrupt injection doesn't >> use KVM_REQ_EVENT any more. >> >> Cc: Paolo Bonzini >> Cc: Radim Krčmář >> Cc: Jan Kiszka >> Signed-off-by: Wanpeng Li > > I need to understand this better. I would hope that something like > > @@ -10668,7 +10598,8 @@ static int vmx_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr) > > if ((kvm_cpu_has_interrupt(vcpu) || external_intr) && > nested_exit_on_intr(vcpu)) { > - if (vmx->nested.nested_run_pending) > + if (vmx->nested.nested_run_pending || > + vcpu->arch.interrupt.pending) > return -EBUSY; > nested_vmx_vmexit(vcpu, EXIT_REASON_EXTERNAL_INTERRUPT, 0, 0); > return 0; > This is insufficient, L2 guest boot with some mitigation, but very slowly and finally stuck will the same crash as above. How about something like below: if (vmx->nested.nested_run_pending) Regards, Wanpeng Li diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index ef4ba71..d46af65 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -10642,6 +10642,11 @@ static int vmx_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr) { struct vcpu_vmx *vmx = to_vmx(vcpu); + if (vcpu->arch.exception.pending || + vcpu->arch.nmi_injected || + vcpu->arch.interrupt.pending) + return -EBUSY; + if (nested_cpu_has_preemption_timer(get_vmcs12(vcpu)) && vmx->nested.preemption_timer_expired) {