From patchwork Thu Feb 1 17:54:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luiz Capitulino X-Patchwork-Id: 10195913 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id F33F560380 for ; Thu, 1 Feb 2018 18:56:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CF6A0289B9 for ; Thu, 1 Feb 2018 18:56:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C3FE228BCD; Thu, 1 Feb 2018 18:56:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 62308289B9 for ; Thu, 1 Feb 2018 18:56:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754501AbeBAS4W (ORCPT ); Thu, 1 Feb 2018 13:56:22 -0500 Received: from mx1.redhat.com ([209.132.183.28]:39922 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753562AbeBAS4V (ORCPT ); Thu, 1 Feb 2018 13:56:21 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4074C49024 for ; Thu, 1 Feb 2018 18:56:21 +0000 (UTC) Received: from localhost (ovpn-116-199.phx2.redhat.com [10.3.116.199]) by smtp.corp.redhat.com (Postfix) with ESMTP id DD7CC60BF3; Thu, 1 Feb 2018 18:55:46 +0000 (UTC) Date: Thu, 1 Feb 2018 12:54:41 -0500 From: Luiz Capitulino To: kvm@vger.kernel.org Cc: pbonzini@redhat.com, rkrcmar@redhat.com, berrange@redhat.com, ehabkost@redhat.com Subject: [RFC] kvm: x86: export vCPU halted state to sysfs Message-ID: <20180201125441.2f5b4fdd@redhat.com> Organization: Red Hat MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 01 Feb 2018 18:56:21 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Libvirt needs to know when a vCPU is halted. To get this information, libvirt has started using the query-cpus command from QEMU. However, if in kernel irqchip is in use, query-cpus will force all vCPUs to user-space since they have to issue the KVM_GET_MP_STATE ioctl. This has catastrophic implications to low-latency workloads like KVM-RT and zero packet loss with DPDK. To make matters worse, there's an OpenStack service called ceilometer that causes libvirt to issue query-cpus every few minutes. The solution proposed in this patch is to export the vCPU halted state in the already existing vcpu directory in sysfs. This way, libvirt can read the vCPU halted state from sysfs and avoid using the query-cpus command. This solution seems to be sufficient for libvirt needs, but it has the following cons: * vcpu information in sysfs lives in a debug directory, so libvirt would be basing its API on debug info * Currently, only x86 supports the vcpu dir in sysfs, so we'd have to expand this to other archs (should be doable) If we agree that this solution is feasible, I'll work on extending the vcpu debug information to other archs for my next posting. Signed-off-by: Luiz Capitulino --- arch/x86/kvm/debugfs.c | 15 +++++++++++++++ arch/x86/kvm/x86.c | 2 ++ include/linux/kvm_host.h | 11 +++++++++++ 3 files changed, 28 insertions(+) diff --git a/arch/x86/kvm/debugfs.c b/arch/x86/kvm/debugfs.c index c19c7ede9bd6..056dd1c787bc 100644 --- a/arch/x86/kvm/debugfs.c +++ b/arch/x86/kvm/debugfs.c @@ -15,6 +15,15 @@ bool kvm_arch_has_vcpu_debugfs(void) return true; } +static int vcpu_get_halted(void *data, u64 *val) +{ + struct kvm_vcpu *vcpu = (struct kvm_vcpu *) data; + *val = vcpu->halted; + return 0; +} + +DEFINE_SIMPLE_ATTRIBUTE(vcpu_halted_fops, vcpu_get_halted, NULL, "%lld\n"); + static int vcpu_get_tsc_offset(void *data, u64 *val) { struct kvm_vcpu *vcpu = (struct kvm_vcpu *) data; @@ -51,6 +60,12 @@ int kvm_arch_create_vcpu_debugfs(struct kvm_vcpu *vcpu) if (!ret) return -ENOMEM; + ret = debugfs_create_file("halted", 0444, + vcpu->debugfs_dentry, + vcpu, &vcpu_halted_fops); + if (!ret) + return -ENOMEM; + if (kvm_has_tsc_control) { ret = debugfs_create_file("tsc-scaling-ratio", 0444, vcpu->debugfs_dentry, diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index c13cd14c4780..9841841d186b 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -6273,6 +6273,7 @@ void kvm_arch_exit(void) int kvm_vcpu_halt(struct kvm_vcpu *vcpu) { + kvm_vcpu_set_halted(vcpu); ++vcpu->stat.halt_exits; if (lapic_in_kernel(vcpu)) { vcpu->arch.mp_state = KVM_MP_STATE_HALTED; @@ -7204,6 +7205,7 @@ static int vcpu_run(struct kvm_vcpu *vcpu) for (;;) { if (kvm_vcpu_running(vcpu)) { + kvm_vcpu_set_running(vcpu); r = vcpu_enter_guest(vcpu); } else { r = vcpu_block(kvm, vcpu); diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index ac0062b74aed..430a4d06b0fb 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -272,10 +272,21 @@ struct kvm_vcpu { } spin_loop; #endif bool preempted; + bool halted; struct kvm_vcpu_arch arch; struct dentry *debugfs_dentry; }; +static inline void kvm_vcpu_set_running(struct kvm_vcpu *vcpu) +{ + vcpu->halted = 0; +} + +static inline void kvm_vcpu_set_halted(struct kvm_vcpu *vcpu) +{ + vcpu->halted = 1; +} + static inline int kvm_vcpu_exiting_guest_mode(struct kvm_vcpu *vcpu) { /*