From patchwork Tue Apr 28 06:23:23 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11513887 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D609B14DD for ; Tue, 28 Apr 2020 06:23:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BD06A206B9 for ; Tue, 28 Apr 2020 06:23:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fECfBrBs" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726445AbgD1GXo (ORCPT ); Tue, 28 Apr 2020 02:23:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42662 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726430AbgD1GXn (ORCPT ); Tue, 28 Apr 2020 02:23:43 -0400 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BB202C03C1A9; Mon, 27 Apr 2020 23:23:42 -0700 (PDT) Received: by mail-pg1-x541.google.com with SMTP id o15so9829103pgi.1; Mon, 27 Apr 2020 23:23:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=pyGp7grSiZpObxZrTgZ2GmGoqZvme1lxk38SftFA0lc=; b=fECfBrBsZgxeek/QXB19YDGD3bP/vi62URcqoJCCJ5xd9hmkWUhcgCZoLFdi1Full4 AGxSqhCBJ2KL700zuXaAxmVO+kGsiTMsJmHMLc02bCz06rivuJ0liO7jnqcylYvn0Fe/ wtMANRLPEI36qRmGcYb3ZE9zGV5aVES+6GETcZt3qCIha0hf4c75VvGq5Ck6AspiLb4v dm+EB4lKPMUVv6eCoLY/IoLFXqAc1gpy2XChp5GOq48ZKaX6xhSQjXUqSk2IS7v/gfzo x3/HCRnQq7rRz7dD+3JqPvoD6xAO0DWBBWXF1oKulvsU1K3570Zxj1wOVg1P7fhKj8NJ j20Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=pyGp7grSiZpObxZrTgZ2GmGoqZvme1lxk38SftFA0lc=; b=XS3ScyHGkc9Iexm+R6U1OMUUd8GqvJyyzb9PGu/1XOIXFjOZrq6TMtAn2OoM55H5ys +9MT90OPeBenzbtNP5dmk73iLub3Fw6QEW5ewYnkLi32WoE7bXJgjRNXe099VTBOFCrr DWG9Z/vo2noxt0cQCwlUUa29KQbsfDuYSN8s5fK7nXFYMyQf7LpYbpyQE6Ax/jBB2IGW 3Jr476DWsfTW9MUvRSrhWDLM00DSofwf0NkIjiFJwkufGDX4FG3CaZ2t9kcFZyDTZDBN l0x8SjLPgHSXFgZL+ut1JyA8WqS73gl0AJK9YTII46XtiTyOM6JQTa2P3g3QcA778H2z VdMA== X-Gm-Message-State: AGi0PuZdp51mWWW9qcg8afCEtMNd2Pi9ASIbR8NZq6Sqbx6VtipvrckU Nd6psVRlLinVQv+J/kSL1MMYZneI X-Google-Smtp-Source: APiQypJqZ0BVbucsAJwdJGRtBft0EMYZR73UQsT9HphY+32jtWyJpChDnKfKU38CrAV0Ime3auGDfA== X-Received: by 2002:a63:e60a:: with SMTP id g10mr10237869pgh.51.1588055022101; Mon, 27 Apr 2020 23:23:42 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id u188sm14183071pfu.33.2020.04.27.23.23.39 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Apr 2020 23:23:41 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Haiwei Li Subject: [PATCH v4 1/7] KVM: VMX: Introduce generic fastpath handler Date: Tue, 28 Apr 2020 14:23:23 +0800 Message-Id: <1588055009-12677-2-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1588055009-12677-1-git-send-email-wanpengli@tencent.com> References: <1588055009-12677-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li Introduce generic fastpath handler to handle MSR fastpath, VMX-preemption timer fastpath etc, move it after vmx_complete_interrupts() in order that later patch can catch the case vmexit occurred while another event was being delivered to guest. There is no obversed performance difference for IPI fastpath testing after this move. Tested-by: Haiwei Li Cc: Haiwei Li Signed-off-by: Wanpeng Li Reviewed-by: Vitaly Kuznetsov --- arch/x86/kvm/vmx/vmx.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 3ab6ca6..9b5adb4 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6583,6 +6583,20 @@ void vmx_update_host_rsp(struct vcpu_vmx *vmx, unsigned long host_rsp) } } +static enum exit_fastpath_completion vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu) +{ + if (!is_guest_mode(vcpu)) { + switch (to_vmx(vcpu)->exit_reason) { + case EXIT_REASON_MSR_WRITE: + return handle_fastpath_set_msr_irqoff(vcpu); + default: + return EXIT_FASTPATH_NONE; + } + } + + return EXIT_FASTPATH_NONE; +} + bool __vmx_vcpu_run(struct vcpu_vmx *vmx, unsigned long *regs, bool launched); static enum exit_fastpath_completion vmx_vcpu_run(struct kvm_vcpu *vcpu) @@ -6757,17 +6771,14 @@ static enum exit_fastpath_completion vmx_vcpu_run(struct kvm_vcpu *vcpu) if (unlikely(vmx->exit_reason & VMX_EXIT_REASONS_FAILED_VMENTRY)) return EXIT_FASTPATH_NONE; - if (!is_guest_mode(vcpu) && vmx->exit_reason == EXIT_REASON_MSR_WRITE) - exit_fastpath = handle_fastpath_set_msr_irqoff(vcpu); - else - exit_fastpath = EXIT_FASTPATH_NONE; - vmx->loaded_vmcs->launched = 1; vmx->idt_vectoring_info = vmcs_read32(IDT_VECTORING_INFO_FIELD); vmx_recover_nmi_blocking(vmx); vmx_complete_interrupts(vmx); + exit_fastpath = vmx_exit_handlers_fastpath(vcpu); + return exit_fastpath; } From patchwork Tue Apr 28 06:23:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11513899 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 61A7614DD for ; Tue, 28 Apr 2020 06:24:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 44D59206D7 for ; Tue, 28 Apr 2020 06:24:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="na1g+X5k" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726495AbgD1GXu (ORCPT ); Tue, 28 Apr 2020 02:23:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42670 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726463AbgD1GXq (ORCPT ); Tue, 28 Apr 2020 02:23:46 -0400 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A3C3EC03C1A9; Mon, 27 Apr 2020 23:23:45 -0700 (PDT) Received: by mail-pg1-x542.google.com with SMTP id l20so1886467pgb.11; Mon, 27 Apr 2020 23:23:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=oQfltYNxiPjEH77FfOHpysAJQ9qhxIF+HiwY5HDOYEg=; b=na1g+X5kfYLb3TSDeKHaz9nHOucdv8OBXFIYXvDOTe0/LDDQpP7S+ngzWqWkMFF8P2 +EEtmI+zlu5C5KBfm+/htgzONU+jf+GKEevYCdSstpf4OdbGsOkKxQkllYcIo4O48fP1 ImqWa7uJzyJnKm2g5QXpUIULaxlE3wFTK6lsqHvXD8fteSAD7W7Qfl4/1M9lfTTWT1se 7qX+eMYgrSEjkAq2N7m1qtZ2l9t4RxG23M19y5H/vtsW0y3+8QD7anTOgi265fWt0NsD bI3FDiKz3OVg9gMj93babGkkJ/vsUXNABFShyI4SwL3OGZNLuEj6gKOIjtd1iK8eM6is wSZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=oQfltYNxiPjEH77FfOHpysAJQ9qhxIF+HiwY5HDOYEg=; b=XPNdUzPZ3PI3/lY1dpfOTZ7vIp2LXMoK1DUVeXngFo0zkoaL3CEIkZdQTJcSYM45FZ 3BJ9zPXHeycVC4tYJSifv9jT8vbFl6L3xVLALHL15eYennZ3U+tYtU3I6hXWqdcL0KjL XWoahoRJlwd9bk5kggWwRvKbcCKs2a9V3/G4beHERx3Jl8dJxg6cbZRQhfSCte0u1ETi Vd4QawSKBpOX1VeGQHMQt4YUL1eO8PdaEBQZXYViqEC7FwW99VUJybtMhoo1Zrs0rQFf 8q32Mnh46xNvmLwxpJYee1tPeK7N7z5CRlUKPM803ZLtXJcaXeunk/hHdjBEYyJbHLMG xliQ== X-Gm-Message-State: AGi0PubsgvZbKZGXRy8/JXhgZt++gGMQVbtq648ov1W3oKwC5e7SI3d2 pPDOah5/4TaOCl11bAy2rwYlQvOR X-Google-Smtp-Source: APiQypKp1hI4VWMsNDTx5ZJ2YDneopaWSbNacQZzEDnVuNIoRpzvPLBXOQoCT+NgPVoa0SdmA4aqAg== X-Received: by 2002:a63:d806:: with SMTP id b6mr2196905pgh.72.1588055025038; Mon, 27 Apr 2020 23:23:45 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id u188sm14183071pfu.33.2020.04.27.23.23.42 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Apr 2020 23:23:44 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Haiwei Li Subject: [PATCH v4 2/7] KVM: X86: Enable fastpath when APICv is enabled Date: Tue, 28 Apr 2020 14:23:24 +0800 Message-Id: <1588055009-12677-3-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1588055009-12677-1-git-send-email-wanpengli@tencent.com> References: <1588055009-12677-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li We can't observe benefit from single target IPI fastpath when APICv is disabled, let's just enable IPI and Timer fastpath when APICv is enabled for now. Tested-by: Haiwei Li Cc: Haiwei Li Signed-off-by: Wanpeng Li --- arch/x86/kvm/svm/svm.c | 2 +- arch/x86/kvm/vmx/vmx.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 8f8fc65..1e7220e 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -3344,7 +3344,7 @@ static void svm_cancel_injection(struct kvm_vcpu *vcpu) static enum exit_fastpath_completion svm_exit_handlers_fastpath(struct kvm_vcpu *vcpu) { - if (!is_guest_mode(vcpu) && + if (!is_guest_mode(vcpu) && vcpu->arch.apicv_active && to_svm(vcpu)->vmcb->control.exit_code == SVM_EXIT_MSR && to_svm(vcpu)->vmcb->control.exit_info_1) return handle_fastpath_set_msr_irqoff(vcpu); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 9b5adb4..f207004 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6585,7 +6585,7 @@ void vmx_update_host_rsp(struct vcpu_vmx *vmx, unsigned long host_rsp) static enum exit_fastpath_completion vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu) { - if (!is_guest_mode(vcpu)) { + if (!is_guest_mode(vcpu) && vcpu->arch.apicv_active) { switch (to_vmx(vcpu)->exit_reason) { case EXIT_REASON_MSR_WRITE: return handle_fastpath_set_msr_irqoff(vcpu); From patchwork Tue Apr 28 06:23:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11513889 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CFFA614B4 for ; Tue, 28 Apr 2020 06:23:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B2191206B9 for ; Tue, 28 Apr 2020 06:23:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="owxHs7l+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726486AbgD1GXu (ORCPT ); Tue, 28 Apr 2020 02:23:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726476AbgD1GXs (ORCPT ); Tue, 28 Apr 2020 02:23:48 -0400 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F22DC03C1AC; Mon, 27 Apr 2020 23:23:48 -0700 (PDT) Received: by mail-pg1-x544.google.com with SMTP id o185so9831090pgo.3; Mon, 27 Apr 2020 23:23:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Z/xnjPHU7knLcRgYbx+btpyqMQDl/s3pOWwSvfXVlgY=; b=owxHs7l+7A+uG8W0bVjbMfuIkY0quDeo1nvLcYHFdKmpo5XJQfHvtmZ1jd5MWPRsw6 +OjGdofLDy52FXwA5IGP2fVFuq1gMV5exS4ccAieDFsgCBwguKlGq82Z1Fu2J7rPIiYH T1guNkdrkigC2+nHLPL1DxkS19lQL+hP7aUYP/AcXwYV8mQAPAUSj50Yx+csLe/SM7dK TueH9pYExYqnHMGep4s9vDm1jXcv2/m46bzvnuRvbvrmfzyZVXUy5stPOd2Im/707YNe Fx2/zaieEEVTuzum3ABd0HDM9tLGHktp7QXzJts05AObtRn/hMDXr31vIGn9Y6shMGED 1F8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Z/xnjPHU7knLcRgYbx+btpyqMQDl/s3pOWwSvfXVlgY=; b=Uh0lG7hjvgCB5dioWZmfoc7l7FfY+J2KIo2MNPJIqoyj3azG2C422mjO2t6bhrkGvB aii/4YBCr2pXHeuMQ3D7cRJgMHx0r5yafGJBEwkCGW8NIBz/aReWYle7MICUS5ZBAO5m YU1fBSHKPYHPmU9Fnxo1hVwBejPHWm3vu7SJE7clSrpTE3D3+bz90lz2kA7+dYM3hcQs 9E2rJQ9iBviSt8msD90b9Swa0XT0MbdtIlWpu+Z/rcsa4aNbmIfA3zdvHjo5JJ6p+K+H EstYnhU8BV61LA94p7BtP0oalrOgsqFYxzqTckvFQ0ZlokWUrQY/Vo6KXQHtfyALJedp Potw== X-Gm-Message-State: AGi0PuYpNLCVBdjjfHcums6oIYKfqUe+CkXCRqOEk2cRTEWS79SDvIVn rDNT82vGPI1OoAmk0E98UZidwiv8 X-Google-Smtp-Source: APiQypLOFFWPsFWGiJ0Efux7mcQE7puOEXXe+W6xOaW1QMsCNAY5wPE6v6hyQDcnjFZuJv2R9e7pQQ== X-Received: by 2002:a62:3607:: with SMTP id d7mr27935128pfa.245.1588055027847; Mon, 27 Apr 2020 23:23:47 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id u188sm14183071pfu.33.2020.04.27.23.23.45 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Apr 2020 23:23:47 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Haiwei Li Subject: [PATCH v4 3/7] KVM: X86: Introduce more exit_fastpath_completion enum values Date: Tue, 28 Apr 2020 14:23:25 +0800 Message-Id: <1588055009-12677-4-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1588055009-12677-1-git-send-email-wanpengli@tencent.com> References: <1588055009-12677-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li Introduce another two exit_fastpath_completion enum values. - EXIT_FASTPATH_REENTER_GUEST complete fastpath and there is no other stuff prevents enter guest again immediately. - EXIT_FASTPATH_NOP kvm will still go through it's full run loop, but it would skip invoking the exit handler. They will be used by later patch, in addition, adds a fastpath_t typedef since enum lines are a bit long. Tested-by: Haiwei Li Cc: Haiwei Li Signed-off-by: Wanpeng Li --- arch/x86/include/asm/kvm_host.h | 3 +++ arch/x86/kvm/svm/svm.c | 9 ++++----- arch/x86/kvm/vmx/vmx.c | 9 ++++----- arch/x86/kvm/x86.c | 4 ++-- arch/x86/kvm/x86.h | 2 +- 5 files changed, 14 insertions(+), 13 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 7cd68d1..1535484 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -188,7 +188,10 @@ enum { enum exit_fastpath_completion { EXIT_FASTPATH_NONE, EXIT_FASTPATH_SKIP_EMUL_INS, + EXIT_FASTPATH_REENTER_GUEST, + EXIT_FASTPATH_NOP, }; +typedef enum exit_fastpath_completion fastpath_t; struct x86_emulate_ctxt; struct x86_exception; diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 1e7220e..26f623f 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -2911,8 +2911,7 @@ static void svm_get_exit_info(struct kvm_vcpu *vcpu, u64 *info1, u64 *info2) *info2 = control->exit_info_2; } -static int handle_exit(struct kvm_vcpu *vcpu, - enum exit_fastpath_completion exit_fastpath) +static int handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath) { struct vcpu_svm *svm = to_svm(vcpu); struct kvm_run *kvm_run = vcpu->run; @@ -3342,7 +3341,7 @@ static void svm_cancel_injection(struct kvm_vcpu *vcpu) svm_complete_interrupts(svm); } -static enum exit_fastpath_completion svm_exit_handlers_fastpath(struct kvm_vcpu *vcpu) +static fastpath_t svm_exit_handlers_fastpath(struct kvm_vcpu *vcpu) { if (!is_guest_mode(vcpu) && vcpu->arch.apicv_active && to_svm(vcpu)->vmcb->control.exit_code == SVM_EXIT_MSR && @@ -3354,9 +3353,9 @@ static enum exit_fastpath_completion svm_exit_handlers_fastpath(struct kvm_vcpu void __svm_vcpu_run(unsigned long vmcb_pa, unsigned long *regs); -static enum exit_fastpath_completion svm_vcpu_run(struct kvm_vcpu *vcpu) +static fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) { - enum exit_fastpath_completion exit_fastpath; + fastpath_t exit_fastpath; struct vcpu_svm *svm = to_svm(vcpu); svm->vmcb->save.rax = vcpu->arch.regs[VCPU_REGS_RAX]; diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index f207004..e12a42e 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -5881,8 +5881,7 @@ void dump_vmcs(void) * The guest has exited. See if we can fix it or if we need userspace * assistance. */ -static int vmx_handle_exit(struct kvm_vcpu *vcpu, - enum exit_fastpath_completion exit_fastpath) +static int vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath) { struct vcpu_vmx *vmx = to_vmx(vcpu); u32 exit_reason = vmx->exit_reason; @@ -6583,7 +6582,7 @@ void vmx_update_host_rsp(struct vcpu_vmx *vmx, unsigned long host_rsp) } } -static enum exit_fastpath_completion vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu) +static fastpath_t vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu) { if (!is_guest_mode(vcpu) && vcpu->arch.apicv_active) { switch (to_vmx(vcpu)->exit_reason) { @@ -6599,9 +6598,9 @@ static enum exit_fastpath_completion vmx_exit_handlers_fastpath(struct kvm_vcpu bool __vmx_vcpu_run(struct vcpu_vmx *vmx, unsigned long *regs, bool launched); -static enum exit_fastpath_completion vmx_vcpu_run(struct kvm_vcpu *vcpu) +static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu) { - enum exit_fastpath_completion exit_fastpath; + fastpath_t exit_fastpath; struct vcpu_vmx *vmx = to_vmx(vcpu); unsigned long cr3, cr4; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 856b6fc2..df38b40 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1609,7 +1609,7 @@ static int handle_fastpath_set_x2apic_icr_irqoff(struct kvm_vcpu *vcpu, u64 data return 1; } -enum exit_fastpath_completion handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu) +fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu) { u32 msr = kvm_rcx_read(vcpu); u64 data; @@ -8168,7 +8168,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) bool req_int_win = dm_request_for_irq_injection(vcpu) && kvm_cpu_accept_dm_intr(vcpu); - enum exit_fastpath_completion exit_fastpath; + fastpath_t exit_fastpath; bool req_immediate_exit = false; diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 7b5ed8e..2f02dc0 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -274,7 +274,7 @@ bool kvm_mtrr_check_gfn_range_consistency(struct kvm_vcpu *vcpu, gfn_t gfn, bool kvm_vector_hashing_enabled(void); int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, int emulation_type, void *insn, int insn_len); -enum exit_fastpath_completion handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu); +fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu); extern u64 host_xcr0; extern u64 supported_xcr0; From patchwork Tue Apr 28 06:23:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11513891 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F193914DD for ; Tue, 28 Apr 2020 06:23:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D5678206B9 for ; Tue, 28 Apr 2020 06:23:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="V4vrhTp+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726511AbgD1GXz (ORCPT ); Tue, 28 Apr 2020 02:23:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42708 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726355AbgD1GXz (ORCPT ); Tue, 28 Apr 2020 02:23:55 -0400 Received: from mail-pl1-x644.google.com (mail-pl1-x644.google.com [IPv6:2607:f8b0:4864:20::644]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E713DC03C1AA; Mon, 27 Apr 2020 23:23:54 -0700 (PDT) Received: by mail-pl1-x644.google.com with SMTP id d24so7940090pll.8; Mon, 27 Apr 2020 23:23:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dASjRXsAg6WzAae5CXACCO5RVvFv8dsS43RJHmTgzTs=; b=V4vrhTp+G2YuztWgbTcOG0kzgrR8Rjq27TRZj4WM+JH7ENf17cEXehRD3QYP8/7Io9 TRzTzUTcGwsFd8wED53Al5Lf1e9d6fEci2seRTv/bsTzredAB3ZKNrNTPkD1jkw6TRwg Pj9e5lD/VwwVjGqPTWTa+6FvuJ8NZC2vT/DQW8Sp6NPERTXP8XtybizKH9Jh5sIxGy7I mj0Gak2SH8YCpk3XP2AeT6W606c8NNb5UgBZpLOj+uCk7r2eTebi8COcq2ybxhTvX5GR nMvZdgTkhKo64RckR88JvNc6mkMCmOywxBrDdTARontdbMxvHjbNWblRrPkMYdOTgq0U GH0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dASjRXsAg6WzAae5CXACCO5RVvFv8dsS43RJHmTgzTs=; b=ZqMtS/8E8KHXDwr9xGPiD+kpmCWK2riqIhbhOWrBsPPkCZlMDvZW+g1SllLRhM/vLz XoMTDEg9opTYcPwIUzdVevDtYGVJhI6kfzsQz+vg3qNjCzTiFsyZmz9vxRHH+tZ/y/Rt 4KumQUOkLZq3lfP/MXjj9wRx3GxdAy4j4/STWZk8sbapr9XB6XV09IQ/sQkvGLoFFPOC a3safBPn7TsyG35CiC+AezmbriiwHnOB3ccdBs6R6B2XrihFd9qqDHXXNO7M/soXfp34 4D2c4Kxd1A+CR41IKZJ/tVdfZ35eVSBUyNz/E5sRfKx91X93Sla1Dpx578TrP2wghq4a e9Jw== X-Gm-Message-State: AGi0PuYFTAU36ZegTlSLiVkgjKz3WqkfGjrv/TAcl06uqLbuFpL57R8F bVf4/tBnyh4Fo1xbE9G06hz3DVFp X-Google-Smtp-Source: APiQypJ2dyHcYY2mJa3PGa2d76ad4Tri/9ZR7Ip0tE6jflFVGWTKZMBwmKayZxw5sbVUvfCPgkM9ng== X-Received: by 2002:a17:90a:b293:: with SMTP id c19mr3093890pjr.22.1588055030740; Mon, 27 Apr 2020 23:23:50 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id u188sm14183071pfu.33.2020.04.27.23.23.48 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Apr 2020 23:23:50 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Haiwei Li Subject: [PATCH v4 4/7] KVM: X86: Introduce kvm_vcpu_exit_request() helper Date: Tue, 28 Apr 2020 14:23:26 +0800 Message-Id: <1588055009-12677-5-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1588055009-12677-1-git-send-email-wanpengli@tencent.com> References: <1588055009-12677-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li Introduce kvm_vcpu_exit_request() helper, we need to check some conditions before enter guest again immediately, we skip invoking the exit handler and go through full run loop if complete fastpath but there is stuff preventing we enter guest again immediately. Tested-by: Haiwei Li Cc: Haiwei Li Signed-off-by: Wanpeng Li --- arch/x86/kvm/vmx/vmx.c | 3 +++ arch/x86/kvm/x86.c | 10 ++++++++-- arch/x86/kvm/x86.h | 1 + 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index e12a42e..24cadf4 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -6777,6 +6777,9 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu) vmx_complete_interrupts(vmx); exit_fastpath = vmx_exit_handlers_fastpath(vcpu); + if (exit_fastpath == EXIT_FASTPATH_REENTER_GUEST && + kvm_vcpu_exit_request(vcpu)) + exit_fastpath = EXIT_FASTPATH_NOP; return exit_fastpath; } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index df38b40..afe052c 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1581,6 +1581,13 @@ int kvm_emulate_wrmsr(struct kvm_vcpu *vcpu) } EXPORT_SYMBOL_GPL(kvm_emulate_wrmsr); +bool kvm_vcpu_exit_request(struct kvm_vcpu *vcpu) +{ + return vcpu->mode == EXITING_GUEST_MODE || kvm_request_pending(vcpu) || + need_resched() || signal_pending(current); +} +EXPORT_SYMBOL_GPL(kvm_vcpu_exit_request); + /* * The fast path for frequent and performance sensitive wrmsr emulation, * i.e. the sending of IPI, sending IPI early in the VM-Exit flow reduces @@ -8366,8 +8373,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) if (kvm_lapic_enabled(vcpu) && vcpu->arch.apicv_active) kvm_x86_ops.sync_pir_to_irr(vcpu); - if (vcpu->mode == EXITING_GUEST_MODE || kvm_request_pending(vcpu) - || need_resched() || signal_pending(current)) { + if (kvm_vcpu_exit_request(vcpu)) { vcpu->mode = OUTSIDE_GUEST_MODE; smp_wmb(); local_irq_enable(); diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 2f02dc0..6eb62e9 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -364,5 +364,6 @@ static inline bool kvm_dr7_valid(u64 data) void kvm_load_guest_xsave_state(struct kvm_vcpu *vcpu); void kvm_load_host_xsave_state(struct kvm_vcpu *vcpu); u64 kvm_spec_ctrl_valid_bits(struct kvm_vcpu *vcpu); +bool kvm_vcpu_exit_request(struct kvm_vcpu *vcpu); #endif From patchwork Tue Apr 28 06:23:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11513897 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A39EA14B4 for ; Tue, 28 Apr 2020 06:24:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 88960206D8 for ; Tue, 28 Apr 2020 06:24:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="sFxQQzq7" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726540AbgD1GYI (ORCPT ); Tue, 28 Apr 2020 02:24:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42706 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726474AbgD1GXz (ORCPT ); Tue, 28 Apr 2020 02:23:55 -0400 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80887C03C1A9; Mon, 27 Apr 2020 23:23:54 -0700 (PDT) Received: by mail-pg1-x544.google.com with SMTP id d17so9841238pgo.0; Mon, 27 Apr 2020 23:23:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=c4J23pfnjE51eiU8sg/gmUkqucxcbnJvyvaM93YyDy8=; b=sFxQQzq7Mg0a9LVrigkvsGpOutFd3UmHoo7YxqEc2e/CIwse9vEG3VCe7dlSVMd1L4 Nx6zQZnhL6FYfDXT/LCne37szpqNofdNRrF4qmXnkj5chX5NTOffEyRfGNOOXVHXZevb ORWL45so9Q2uygWze4TlTfFtszIpZdZvshtJ2KfNTBif1kUgfo++tH58p4v0GANSsxzQ zY2CvFKWMse6vlPklFz/Im0M6bQtEih/bf/3rLadg6+MjlecQ34l4bS2EZMiBQgxgBn1 CGgCupke1j5ye0TR7e6x+Toq+OyzFAPeSnlltPImbAPfPqNN2LqGlUnpIfdSMcOvzCuO R2Qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=c4J23pfnjE51eiU8sg/gmUkqucxcbnJvyvaM93YyDy8=; b=VhnrXwqa+nduBNewuI71cg6Bi81E33uWdGxvgja1i8SxgFmzQu56cA5JeLO9lcW+GM FjJooivYrdjJIE6J495LCL4cb/mwKoCS3c5xXNwV/iprfgEphHKAL6FCkNlq+sY7z+Gt YqvRCX1UIeY5pfy1X4EZ7rBopty7EYMl4Virv5WzeH1ptnUrvRvdl3zpliQKYnXug+UT BnF9/pN60riVNoJYnOl2tv3Nj65H8mbjpfd3yIAoz2hiLvaUIgtZM2imX/S+6sw6mBqh Skz024W+fsFoRZMqihNrwMff/GGYU2XMPhTdoZ222qUHTvkABSaqUnXJPgm0v1f8EUgE kyuA== X-Gm-Message-State: AGi0PuZuBhkKGC71fQL4y0bOphoYh+IB9JJFnIdbZmJqRpxaAtLQ5I4G GZ/kVqA4PlAuP0B1TyySID3KrhIl X-Google-Smtp-Source: APiQypLQMmro+DQJ5c5SblXXuHsVr5QFrpUVoA0Jh34UQ6ckiRrL94Euwh7qSKyy+M/FsCUJccvn9g== X-Received: by 2002:a62:4d43:: with SMTP id a64mr30288672pfb.156.1588055033511; Mon, 27 Apr 2020 23:23:53 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id u188sm14183071pfu.33.2020.04.27.23.23.51 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Apr 2020 23:23:53 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Haiwei Li Subject: [PATCH v4 5/7] KVM: VMX: Optimize posted-interrupt delivery for timer fastpath Date: Tue, 28 Apr 2020 14:23:27 +0800 Message-Id: <1588055009-12677-6-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1588055009-12677-1-git-send-email-wanpengli@tencent.com> References: <1588055009-12677-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li Optimizing posted-interrupt delivery especially for the timer fastpath scenario, I observe kvm_x86_ops.deliver_posted_interrupt() has more latency then vmx_sync_pir_to_irr() in the case of timer fastpath scenario, since it needs to wait vmentry, after that it can handle external interrupt, ack the notification vector, read posted-interrupt descriptor etc, it is slower than evaluate and delivery during vmentry immediately approach. Let's skip sending interrupt to notify target pCPU and replace by vmx_sync_pir_to_irr() before each reenter guest. Tested-by: Haiwei Li Cc: Haiwei Li Signed-off-by: Wanpeng Li --- arch/x86/kvm/vmx/vmx.c | 12 ++++++++---- virt/kvm/kvm_main.c | 1 + 2 files changed, 9 insertions(+), 4 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 24cadf4..ce19b0e 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -3909,7 +3909,8 @@ static int vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector) if (pi_test_and_set_on(&vmx->pi_desc)) return 0; - if (!kvm_vcpu_trigger_posted_interrupt(vcpu, false)) + if (vcpu != kvm_get_running_vcpu() && + !kvm_vcpu_trigger_posted_interrupt(vcpu, false)) kvm_vcpu_kick(vcpu); return 0; @@ -6777,9 +6778,12 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu) vmx_complete_interrupts(vmx); exit_fastpath = vmx_exit_handlers_fastpath(vcpu); - if (exit_fastpath == EXIT_FASTPATH_REENTER_GUEST && - kvm_vcpu_exit_request(vcpu)) - exit_fastpath = EXIT_FASTPATH_NOP; + if (exit_fastpath == EXIT_FASTPATH_REENTER_GUEST) { + if (!kvm_vcpu_exit_request(vcpu)) + vmx_sync_pir_to_irr(vcpu); + else + exit_fastpath = EXIT_FASTPATH_NOP; + } return exit_fastpath; } diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index 33e1eee..2482f3c 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -4644,6 +4644,7 @@ struct kvm_vcpu *kvm_get_running_vcpu(void) return vcpu; } +EXPORT_SYMBOL_GPL(kvm_get_running_vcpu); /** * kvm_get_running_vcpus - get the per-CPU array of currently running vcpus. From patchwork Tue Apr 28 06:23:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11513895 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8191414B4 for ; Tue, 28 Apr 2020 06:24:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 63031206B9 for ; Tue, 28 Apr 2020 06:24:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="XF8PTyEC" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726534AbgD1GX7 (ORCPT ); Tue, 28 Apr 2020 02:23:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726526AbgD1GX6 (ORCPT ); Tue, 28 Apr 2020 02:23:58 -0400 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E54B7C03C1A9; Mon, 27 Apr 2020 23:23:56 -0700 (PDT) Received: by mail-pg1-x541.google.com with SMTP id n16so9811206pgb.7; Mon, 27 Apr 2020 23:23:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=0Bu4Zd3dsV7URxWqUsHu8lketvap8gr0AXXVtw9yqAM=; b=XF8PTyECj50vdI++0IB6Q2n9T+9XCqbbOBSfyM7RQe3689NG0EoGuFSqnApTfrMMBG 8bGwENVwfGEcNeF8hp1jHeNqI33Iz6bimDgf/ROynVfOMFe/DGDXn9wVDbo/lCVFe8tk Ty/z3EoHNQ+zySNTj/HbZYBqjAxZOnnSngmQN6GXGqcxgOTzA5nOZqQeNakHvSRqV+wO w2QrCru40ViOo1czLFWttsgyo+qP2hstxO4Nb8iqqOD1yIgsajVitrgXUf/W+1e/ybHe oJ9nTu7o8TGXczjGz/ql0Or6YfnbHqqp6H5ZcUyXqJFwaDnbJ77onGNuU0Q+EVCW93zr wS0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=0Bu4Zd3dsV7URxWqUsHu8lketvap8gr0AXXVtw9yqAM=; b=LRfAj0/1Od3WF7W1y34Pv2qUc8tqkR6Hhprvv+nJBXwNHHHw7PHDO/GJ4UIMeElR55 lIVcx0xrHDvQ3evdaJntazCKcuheS2sTZp1w8dd67N2IT2abBd1S/K3jtdA/HlNydX/l DxIiFApcXth4t+cyoHMsCNgXMO5f31ZW2p7LeGUvZkY4mQAl4CW/zaMVDEBIewgAgUWH 4ep+qhdnjekOL4rsXTz8uzpD7AzknmCljKf4RQULSQwVw3Rbo2mehfUeZDNrQ4yhwjRo G/O1IoVAdA4h7dHn2Hjgn8HT59tBJwmCqzIQOtF0U2ozgrAsvEc3bKrsb/92mn+rLWRT rpsg== X-Gm-Message-State: AGi0Pub89AcFYfjB0IE6fUuPFw3lJxQcCqNH+o8ybBVP6YCVbx2zmv5n D1PzKavXbC/IniLPu1aQtdzU72fn X-Google-Smtp-Source: APiQypLSqmV3V1oAkhuUWHZMJgVrTwPZnAvh5rL2Y5gXYLMNs4ozWDv5FaFSrgenCITB9WfzTFPyiA== X-Received: by 2002:a63:7050:: with SMTP id a16mr15041579pgn.105.1588055036254; Mon, 27 Apr 2020 23:23:56 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id u188sm14183071pfu.33.2020.04.27.23.23.53 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Apr 2020 23:23:55 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Haiwei Li Subject: [PATCH v4 6/7] KVM: X86: TSCDEADLINE MSR emulation fastpath Date: Tue, 28 Apr 2020 14:23:28 +0800 Message-Id: <1588055009-12677-7-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1588055009-12677-1-git-send-email-wanpengli@tencent.com> References: <1588055009-12677-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li This patch implements tscdealine msr emulation fastpath, after wrmsr tscdeadline vmexit, handle it as soon as possible and vmentry immediately without checking various kvm stuff when possible. Tested-by: Haiwei Li Cc: Haiwei Li Signed-off-by: Wanpeng Li --- arch/x86/kvm/lapic.c | 18 ++++++++++++------ arch/x86/kvm/vmx/vmx.c | 12 ++++++++---- arch/x86/kvm/x86.c | 30 ++++++++++++++++++++++++------ 3 files changed, 44 insertions(+), 16 deletions(-) diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 38f7dc9..3589237 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -1593,7 +1593,7 @@ static void kvm_apic_inject_pending_timer_irqs(struct kvm_lapic *apic) } } -static void apic_timer_expired(struct kvm_lapic *apic) +static void apic_timer_expired(struct kvm_lapic *apic, bool from_timer_fn) { struct kvm_vcpu *vcpu = apic->vcpu; struct kvm_timer *ktimer = &apic->lapic_timer; @@ -1604,6 +1604,12 @@ static void apic_timer_expired(struct kvm_lapic *apic) if (apic_lvtt_tscdeadline(apic) || ktimer->hv_timer_in_use) ktimer->expired_tscdeadline = ktimer->tscdeadline; + if (!from_timer_fn && vcpu->arch.apicv_active) { + WARN_ON(kvm_get_running_vcpu() != vcpu); + kvm_apic_inject_pending_timer_irqs(apic); + return; + } + if (kvm_use_posted_timer_interrupt(apic->vcpu)) { if (apic->lapic_timer.timer_advance_ns) __kvm_wait_lapic_expire(vcpu); @@ -1643,7 +1649,7 @@ static void start_sw_tscdeadline(struct kvm_lapic *apic) expire = ktime_sub_ns(expire, ktimer->timer_advance_ns); hrtimer_start(&ktimer->timer, expire, HRTIMER_MODE_ABS_HARD); } else - apic_timer_expired(apic); + apic_timer_expired(apic, false); local_irq_restore(flags); } @@ -1751,7 +1757,7 @@ static void start_sw_period(struct kvm_lapic *apic) if (ktime_after(ktime_get(), apic->lapic_timer.target_expiration)) { - apic_timer_expired(apic); + apic_timer_expired(apic, false); if (apic_lvtt_oneshot(apic)) return; @@ -1813,7 +1819,7 @@ static bool start_hv_timer(struct kvm_lapic *apic) if (atomic_read(&ktimer->pending)) { cancel_hv_timer(apic); } else if (expired) { - apic_timer_expired(apic); + apic_timer_expired(apic, false); cancel_hv_timer(apic); } } @@ -1863,7 +1869,7 @@ void kvm_lapic_expired_hv_timer(struct kvm_vcpu *vcpu) goto out; WARN_ON(swait_active(&vcpu->wq)); cancel_hv_timer(apic); - apic_timer_expired(apic); + apic_timer_expired(apic, false); if (apic_lvtt_period(apic) && apic->lapic_timer.period) { advance_periodic_target_expiration(apic); @@ -2369,7 +2375,7 @@ static enum hrtimer_restart apic_timer_fn(struct hrtimer *data) struct kvm_timer *ktimer = container_of(data, struct kvm_timer, timer); struct kvm_lapic *apic = container_of(ktimer, struct kvm_lapic, lapic_timer); - apic_timer_expired(apic); + apic_timer_expired(apic, true); if (lapic_is_periodic(apic)) { advance_periodic_target_expiration(apic); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index ce19b0e..bb5c4f1 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -5994,7 +5994,8 @@ static int vmx_handle_exit(struct kvm_vcpu *vcpu, fastpath_t exit_fastpath) if (exit_fastpath == EXIT_FASTPATH_SKIP_EMUL_INS) { kvm_skip_emulated_instruction(vcpu); return 1; - } + } else if (exit_fastpath == EXIT_FASTPATH_NOP) + return 1; if (exit_reason >= kvm_vmx_max_exit_handlers) goto unexpected_vmexit; @@ -6605,6 +6606,7 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu) struct vcpu_vmx *vmx = to_vmx(vcpu); unsigned long cr3, cr4; +REENTER_GUEST: /* Record the guest's net vcpu time for enforced NMI injections. */ if (unlikely(!enable_vnmi && vmx->loaded_vmcs->soft_vnmi_blocked)) @@ -6779,10 +6781,12 @@ static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu) exit_fastpath = vmx_exit_handlers_fastpath(vcpu); if (exit_fastpath == EXIT_FASTPATH_REENTER_GUEST) { - if (!kvm_vcpu_exit_request(vcpu)) + if (!kvm_vcpu_exit_request(vcpu)) { vmx_sync_pir_to_irr(vcpu); - else - exit_fastpath = EXIT_FASTPATH_NOP; + /* static call is better with retpolines */ + goto REENTER_GUEST; + } + exit_fastpath = EXIT_FASTPATH_NOP; } return exit_fastpath; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index afe052c..f3a5fe4 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1616,27 +1616,45 @@ static int handle_fastpath_set_x2apic_icr_irqoff(struct kvm_vcpu *vcpu, u64 data return 1; } +static int handle_fastpath_set_tscdeadline(struct kvm_vcpu *vcpu, u64 data) +{ + if (!kvm_x86_ops.set_hv_timer || + kvm_mwait_in_guest(vcpu->kvm) || + kvm_can_post_timer_interrupt(vcpu)) + return 1; + + kvm_set_lapic_tscdeadline_msr(vcpu, data); + return 0; +} + fastpath_t handle_fastpath_set_msr_irqoff(struct kvm_vcpu *vcpu) { u32 msr = kvm_rcx_read(vcpu); u64 data; - int ret = 0; + int ret = EXIT_FASTPATH_NONE; switch (msr) { case APIC_BASE_MSR + (APIC_ICR >> 4): data = kvm_read_edx_eax(vcpu); - ret = handle_fastpath_set_x2apic_icr_irqoff(vcpu, data); + if (!handle_fastpath_set_x2apic_icr_irqoff(vcpu, data)) + ret = EXIT_FASTPATH_SKIP_EMUL_INS; + break; + case MSR_IA32_TSCDEADLINE: + data = kvm_read_edx_eax(vcpu); + if (!handle_fastpath_set_tscdeadline(vcpu, data)) + ret = EXIT_FASTPATH_REENTER_GUEST; break; default: - return EXIT_FASTPATH_NONE; + ret = EXIT_FASTPATH_NONE; } - if (!ret) { + if (ret != EXIT_FASTPATH_NONE) { trace_kvm_msr_write(msr, data); - return EXIT_FASTPATH_SKIP_EMUL_INS; + if (ret == EXIT_FASTPATH_REENTER_GUEST) + kvm_skip_emulated_instruction(vcpu); } - return EXIT_FASTPATH_NONE; + return ret; } EXPORT_SYMBOL_GPL(handle_fastpath_set_msr_irqoff); From patchwork Tue Apr 28 06:23:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wanpeng Li X-Patchwork-Id: 11513893 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7BDDD14DD for ; Tue, 28 Apr 2020 06:24:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 630B82073C for ; Tue, 28 Apr 2020 06:24:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YH7nqxE1" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726552AbgD1GYA (ORCPT ); Tue, 28 Apr 2020 02:24:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42724 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726545AbgD1GX7 (ORCPT ); Tue, 28 Apr 2020 02:23:59 -0400 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B559C03C1A9; Mon, 27 Apr 2020 23:23:59 -0700 (PDT) Received: by mail-pf1-x444.google.com with SMTP id y25so10190490pfn.5; Mon, 27 Apr 2020 23:23:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=R8HPAyGqQD6jSi5KIg2alMZG6asgvoHzeT449ZMc+4o=; b=YH7nqxE1vb+ew04V4qQwG7m9DKgHwiXOjL11fNRLi7t3iEo1A1Cfdn4hoGn+79v9Iu kSolvU1zIRhhmhslXZ5jGryUhE5kOcZb2pImIH13HqWCNtSKyJ+OOBU0VQum4sYJc403 3ytnmnOV6AqhB2Cf6Ml6HbbaMWw2EbdVc6ODMBjdY33d/P9O6lb2sEnA3GGTakNoc6BL 6d6yWIKWLaudaYxZK7e6YgAyCN5+7v1dRX1A2OCI9h1PLg8JPLWRF+M8Ji6cXMbgCD/W hEd3SbDTeKNgrKp9fVm2g+PwZMXXmSpPgX8KDwrGbHoWaLh+Jux6lIrje/DWKvlLF68y 99jA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=R8HPAyGqQD6jSi5KIg2alMZG6asgvoHzeT449ZMc+4o=; b=PfF9OlS3PtpyP9PpFFC1Uw5eAqV1c8GcT87AU8KGvEdZp8+XigL9TXI6JKEeg6SV2c T0r7XPyg9imy3YUun3y9e/F7ODlNJxGYL1p+58a33h6lsQsmjxB9KbQaM75MmCygqhbY REZ1ng3jpktRrdxciMRXucBmBViAVe7Ilsg0r3QRsCt181EJcrXFXvNGTYk4AESKZ4g3 DPlgoyQevi+Sfx4VuuyOrgdWCPxcpsXAuHx0RjooO7KLV/Nyy3uMlb2ET1v7ab1N00hv /+uy+2ySqO9vxBfjskYB6+3WHgU/GiJjomGLB92agGrJTmFEo53xp5f9CNOu2Dv0CENe pRiw== X-Gm-Message-State: AGi0PubC/olQkV5cCQeVrb1g0tXZ92EDdHxiLwqnm/WiJpQ0F41N3u1H KYEOrHd8r91RPyXxx10AcnkaCq0q X-Google-Smtp-Source: APiQypK+aStJd2TrI3rBfVtB8W+7HTndD9+ZYTPuTAOznY/9PLTEH7pYXQILNl0yj+iQF4rgGR4EVw== X-Received: by 2002:a65:62ce:: with SMTP id m14mr26427472pgv.174.1588055038933; Mon, 27 Apr 2020 23:23:58 -0700 (PDT) Received: from localhost.localdomain ([103.7.29.6]) by smtp.googlemail.com with ESMTPSA id u188sm14183071pfu.33.2020.04.27.23.23.56 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Apr 2020 23:23:58 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , Haiwei Li Subject: [PATCH v4 7/7] KVM: VMX: Handle preemption timer fastpath Date: Tue, 28 Apr 2020 14:23:29 +0800 Message-Id: <1588055009-12677-8-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1588055009-12677-1-git-send-email-wanpengli@tencent.com> References: <1588055009-12677-1-git-send-email-wanpengli@tencent.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Wanpeng Li This patch implements handle preemption timer fastpath, after timer fire due to VMX-preemption timer counts down to zero, handle it as soon as possible and vmentry immediately without checking various kvm stuff when possible. Testing on SKX Server. cyclictest in guest(w/o mwait exposed, adaptive advance lapic timer is default -1): 5540.5ns -> 4602ns 17% kvm-unit-test/vmexit.flat: w/o avanced timer: tscdeadline_immed: 3028.5 -> 2494.75 17.6% tscdeadline: 5765.7 -> 5285 8.3% w/ adaptive advance timer default -1: tscdeadline_immed: 3123.75 -> 2583 17.3% tscdeadline: 4663.75 -> 4537 2.7% Tested-by: Haiwei Li Cc: Haiwei Li Signed-off-by: Wanpeng Li --- arch/x86/kvm/vmx/vmx.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index bb5c4f1..7dcc99f 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -5556,17 +5556,34 @@ static int handle_pml_full(struct kvm_vcpu *vcpu) return 1; } -static int handle_preemption_timer(struct kvm_vcpu *vcpu) +static int __handle_preemption_timer(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); if (!vmx->req_immediate_exit && - !unlikely(vmx->loaded_vmcs->hv_timer_soft_disabled)) + !unlikely(vmx->loaded_vmcs->hv_timer_soft_disabled)) { kvm_lapic_expired_hv_timer(vcpu); + return 1; + } + return 0; +} + +static int handle_preemption_timer(struct kvm_vcpu *vcpu) +{ + __handle_preemption_timer(vcpu); return 1; } +static fastpath_t handle_fastpath_preemption_timer(struct kvm_vcpu *vcpu) +{ + if (__handle_preemption_timer(vcpu)) { + trace_kvm_exit(EXIT_REASON_PREEMPTION_TIMER, vcpu, KVM_ISA_VMX); + return EXIT_FASTPATH_REENTER_GUEST; + } + return EXIT_FASTPATH_NONE; +} + /* * When nested=0, all VMX instruction VM Exits filter here. The handlers * are overwritten by nested_vmx_setup() when nested=1. @@ -6590,6 +6607,8 @@ static fastpath_t vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu) switch (to_vmx(vcpu)->exit_reason) { case EXIT_REASON_MSR_WRITE: return handle_fastpath_set_msr_irqoff(vcpu); + case EXIT_REASON_PREEMPTION_TIMER: + return handle_fastpath_preemption_timer(vcpu); default: return EXIT_FASTPATH_NONE; }