From patchwork Mon Apr 29 15:57:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Jimenez X-Patchwork-Id: 13647308 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0A4082D7C; Mon, 29 Apr 2024 15:57:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714406270; cv=none; b=eWmeVhvm2q6zva9rWTYTrf7Du1f+cluEIcr7GneyKBOqlKn+oB9uVTkYv4lIdn01pcSsbyNjM7/t9umQYrI5gRm/daMTBKPoJo24U0I4TCexgbpw3yJowYupWmb4wq0wBIGjcHgHlb0QzDpqTgcznylVRi1/h2EpXWJFKrh0348= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714406270; c=relaxed/simple; bh=kD+D/2NFmo6Jf6tcxV53dZCI5gWFzqlfjYjOfIed1X4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=DiDDw7B9ZCvWrVUYvIIdX9ZxN2qRIJnNUTqSD/WokH7cmLDvWPFJ9kmzWPsUlYvjpyjdghumAO3jvVGBRaOOrZL3SecuHNplyw9zoQ4jsaPIRV02oaWZL41aw9TK6Lub+SUw0EOvqw3WZe3LJMw8xoWIioo2NPjD//4q/7Fnbxc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=SpITK5K/; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="SpITK5K/" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43TFmkww006744; Mon, 29 Apr 2024 15:57:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=PBi5rXGW3IONrVECLMotjHdxT/VjBv5UCgvhocP4s54=; b=SpITK5K/JsROmI9bVB0HI5RoKX+46Qor5WJMoJIVK6KbJ9LCbjmjX63XQCZ3VqdwXT7s ttxsNhJksOoshpIrfdIM5pEtqRM/QSYvok0VMdGP87vLWtxubnWF7QUV+VAz7morwqYl ca2NZ+LXZC3fBwoY1yC/sWEEL/F+jTus58sd+KTqPO9Tmqne0ZQgqtyRaOJ21FKUjTmS fIMpsw5YKC3dHjQZH5RVFY1YlkqBHOm4pmkrMOKUutcI73Mf7tVStsXnYk+PWMiPsctz tSH/EyQ3cDlLdYd0064QBA+Q7wvFL46Bn8spMZGf6ZxVzO8LLvp8mZm4JgFX6k6E2AMR ew== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xrsdejwup-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Apr 2024 15:57:43 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43TEcwQ6011335; Mon, 29 Apr 2024 15:57:42 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xrqt6j95m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Apr 2024 15:57:42 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43TFuxjM040299; Mon, 29 Apr 2024 15:57:42 GMT Received: from alaljime-dev-e4flex-vm.osdevelopmeniad.oraclevcn.com (alaljime-dev-e4flex-vm.allregionaliads.osdevelopmeniad.oraclevcn.com [100.100.249.106]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xrqt6j91w-2; Mon, 29 Apr 2024 15:57:42 +0000 From: Alejandro Jimenez To: kvm@vger.kernel.org Cc: seanjc@google.com, pbonzini@redhat.com, linux-kernel@vger.kernel.org, suravee.suthikulpanit@amd.com, vashegde@amd.com, mlevitsk@redhat.com, joao.m.martins@oracle.com, boris.ostrovsky@oracle.com, mark.kanda@oracle.com, alejandro.j.jimenez@oracle.com Subject: [PATCH 1/4] KVM: x86: Expose per-vCPU APICv status Date: Mon, 29 Apr 2024 15:57:35 +0000 Message-Id: <20240429155738.990025-2-alejandro.j.jimenez@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240429155738.990025-1-alejandro.j.jimenez@oracle.com> References: <20240429155738.990025-1-alejandro.j.jimenez@oracle.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-29_14,2024-04-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0 adultscore=0 mlxscore=0 suspectscore=0 phishscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404290101 X-Proofpoint-GUID: faF3Eo7RghRW60RkookFeY1oCaAqZqld X-Proofpoint-ORIG-GUID: faF3Eo7RghRW60RkookFeY1oCaAqZqld Expose the APICv activation status of individual vCPUs via the stats subsystem. In special cases a vCPU's APICv can be deactivated/disabled even though there are no VM-wide inhibition reasons. The only current example of this is AVIC for a vCPU running in nested mode. This type of inhibition is not recorded in the VM inhibit reasons or visible in current tracepoints. Signed-off-by: Alejandro Jimenez Reviewed-by: Vasant Hegde --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/lapic.c | 1 + arch/x86/kvm/x86.c | 2 ++ 3 files changed, 4 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 1d13e3cd1dc5..12f30cb5c842 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1573,6 +1573,7 @@ struct kvm_vcpu_stat { u64 preemption_other; u64 guest_mode; u64 notify_window_exits; + u64 apicv_active; }; struct x86_instruction_info; diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index cf37586f0466..37fe75a5db8c 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -2872,6 +2872,7 @@ int kvm_create_lapic(struct kvm_vcpu *vcpu, int timer_advance_ns) */ if (enable_apicv) { apic->apicv_active = true; + vcpu->stat.apicv_active = apic->apicv_active; kvm_make_request(KVM_REQ_APICV_UPDATE, vcpu); } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index e9ef1fa4b90b..0451c4c8d731 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -304,6 +304,7 @@ const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { STATS_DESC_COUNTER(VCPU, preemption_other), STATS_DESC_IBOOLEAN(VCPU, guest_mode), STATS_DESC_COUNTER(VCPU, notify_window_exits), + STATS_DESC_IBOOLEAN(VCPU, apicv_active), }; const struct kvm_stats_header kvm_vcpu_stats_header = { @@ -10625,6 +10626,7 @@ void __kvm_vcpu_update_apicv(struct kvm_vcpu *vcpu) goto out; apic->apicv_active = activate; + vcpu->stat.apicv_active = apic->apicv_active; kvm_apic_update_apicv(vcpu); static_call(kvm_x86_refresh_apicv_exec_ctrl)(vcpu); From patchwork Mon Apr 29 15:57:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Jimenez X-Patchwork-Id: 13647309 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9CC3084A2B; Mon, 29 Apr 2024 15:57:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714406272; cv=none; b=bp/Zp87dSbPryYukMO+mdLbv6gQbzOj/xRyN9kcuuHAvHhw6/tPJrlkvPmOqErSYTMZxIKs4yPr3bQ2R31CGOLW0B3vANRcoB1BI0K4b6/TWNdbvDhECEmb0iAhO45R3DWQtlxwvqRjdZTDTp3/fXqYNckFjVNd+dSm8k6krtDQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714406272; c=relaxed/simple; bh=Yx10yQlU7fLLx3U/PfdEEvcwLQHEP15tZTFMbY6ANHo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JdLhXiKfge1+kTT739tH8dA0BDIwqSplka1DJJbEK0JCfJs3Tmg3RzWRIH93AVqhlT+cEPUaonJJ5vN7N5jRkauzLtRNCDArK6whPckRp8/rSiOTS/rHFqPxyi8+JeHkuLu3uTM6VCW2HE2+hNB6dh2X9TSRjr+5DN7TltHloTI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=JDxUlc1c; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="JDxUlc1c" Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43TFo22o003852; Mon, 29 Apr 2024 15:57:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=9dxkp9qWdEPqQUS7q3J/EVb+1tSnlOWVbYhXQ6eDerA=; b=JDxUlc1cz/imfKZBHIpT0cX+v3nTPdQ+vnMqmliweJ6/HxNdVdQ7zcX8AhQlM5StHSlH 8xrg+8+rgMl375fPNJCY8nM3hwYLptFmhVeaZENoom2sbBVffFfxJkw87OE199N4wAeH tWF12yJwdDpUGrdzqerkf1YK/8bbPn0232b/Zx0RE7b5vo84CkpGOEpaCRWdGG/7OBqs 0H812eFmkz7rDwN8iwn35K3I0RQG+FsjQpXVDbufo6JCdpBNsVSOSqQplQsWTAvId0oI f5p/z2HHyl4hpnEgshxj2WDems+EJPSB98mnVlowNs91lMlNK69YZOA+A9L1qzuxdJDU 3g== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xrr9ck157-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Apr 2024 15:57:45 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43TF0qFq011412; Mon, 29 Apr 2024 15:57:44 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xrqt6j97g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Apr 2024 15:57:44 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43TFuxjO040299; Mon, 29 Apr 2024 15:57:43 GMT Received: from alaljime-dev-e4flex-vm.osdevelopmeniad.oraclevcn.com (alaljime-dev-e4flex-vm.allregionaliads.osdevelopmeniad.oraclevcn.com [100.100.249.106]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xrqt6j91w-3; Mon, 29 Apr 2024 15:57:43 +0000 From: Alejandro Jimenez To: kvm@vger.kernel.org Cc: seanjc@google.com, pbonzini@redhat.com, linux-kernel@vger.kernel.org, suravee.suthikulpanit@amd.com, vashegde@amd.com, mlevitsk@redhat.com, joao.m.martins@oracle.com, boris.ostrovsky@oracle.com, mark.kanda@oracle.com, alejandro.j.jimenez@oracle.com Subject: [PATCH 2/4] KVM: x86: Add a VM stat exposing when SynIC AutoEOI is in use Date: Mon, 29 Apr 2024 15:57:36 +0000 Message-Id: <20240429155738.990025-3-alejandro.j.jimenez@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240429155738.990025-1-alejandro.j.jimenez@oracle.com> References: <20240429155738.990025-1-alejandro.j.jimenez@oracle.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-29_14,2024-04-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0 adultscore=0 mlxscore=0 suspectscore=0 phishscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404290101 X-Proofpoint-GUID: iCIGZ6ZatbAlm23u8jcAY9IZx6XV8q_8 X-Proofpoint-ORIG-GUID: iCIGZ6ZatbAlm23u8jcAY9IZx6XV8q_8 APICv/AVIC is inhibited for guests using Hyper-V's synthetic interrupt controller (SynIC) enlightenment, and specifically SynIC's AutoEOI feature. It is recommended that guests do not enable AutoEOI (see flag HV_DEPRECATING_AEOI_RECOMMENDED and commit 0f250a646382 ("KVM: x86: hyper-v: Deactivate APICv only when AutoEOI feature is in use")), but it can still be used in legacy configurations. Suggested-by: Paolo Bonzini Signed-off-by: Alejandro Jimenez --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/hyperv.c | 2 ++ arch/x86/kvm/x86.c | 3 ++- 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 12f30cb5c842..f3b40cfebec4 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1534,6 +1534,7 @@ struct kvm_vm_stat { u64 nx_lpage_splits; u64 max_mmu_page_hash_collisions; u64 max_mmu_rmap_size; + u64 synic_auto_eoi_used; }; struct kvm_vcpu_stat { diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index 8a47f8541eab..4263b4ad71c5 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -149,6 +149,8 @@ static void synic_update_vector(struct kvm_vcpu_hv_synic *synic, APICV_INHIBIT_REASON_HYPERV, !!hv->synic_auto_eoi_used); + vcpu->kvm->stat.synic_auto_eoi_used = !!hv->synic_auto_eoi_used; + up_write(&vcpu->kvm->arch.apicv_update_lock); } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 0451c4c8d731..27e339133068 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -256,7 +256,8 @@ const struct _kvm_stats_desc kvm_vm_stats_desc[] = { STATS_DESC_ICOUNTER(VM, pages_1g), STATS_DESC_ICOUNTER(VM, nx_lpage_splits), STATS_DESC_PCOUNTER(VM, max_mmu_rmap_size), - STATS_DESC_PCOUNTER(VM, max_mmu_page_hash_collisions) + STATS_DESC_PCOUNTER(VM, max_mmu_page_hash_collisions), + STATS_DESC_IBOOLEAN(VM, synic_auto_eoi_used) }; const struct kvm_stats_header kvm_vm_stats_header = { From patchwork Mon Apr 29 15:57:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Jimenez X-Patchwork-Id: 13647310 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE58884E0F; Mon, 29 Apr 2024 15:57:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.165.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714406273; cv=none; b=kNcZVPlkH4cs1msNooN5rYYEVxdZ6+aeG/D5ePinqOe7SOaLODxc5FPDmAcBuejoQk6p6LMsPSISEQU+cfXmhjJlPWBlLTRoNH3T4RL7Z9upvT0gaTgHNVdqUAxVWkGpmB/GxrOJKFG+awP/yEUxaByrbYxJiRTdMB0ZZgTW/6Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714406273; c=relaxed/simple; bh=YgyQmLaloPcIUzIg3IrVssDo28Rrg23MlIk6U5amRAM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=suGjVXrmp9iKDtxktaVL0X6OlJdGita09w97sxJ8m9nU0YmKvgY9GJ+IIUAHje8H1tsEbbBWFHFh84acUtGEqxRX61UN3Ot1PRrl8Kt/qMPRNbC6gxv6Gj5jVxyH6NPh+LBvrFmLUgDgOgKKCzYsvFcd/VsxnokR0YMY6NMEIFc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=nAAe3m7S; arc=none smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="nAAe3m7S" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43TFnbYd017526; Mon, 29 Apr 2024 15:57:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=iQ8G9aJXvaHwNv5eTGivSBJiLsTXuxQiiW7NahnRb6s=; b=nAAe3m7SiZ43Fh5imak9evRgG5iYRNh0WV4qEGpuYdKblAyEVcasDF6wnpdiu7tE1Goa s9KJSKTWferbLIxgHZyjszNq8zh++d4wCi3/st2bm1LlGSsUX2mv9WF69HF80Lq/sGEg T84jXNfx6BzSZtev66dtQrsCf/n9HiZsDZMxg5nA01B49QesELZlnPshWwFhJ0gUe+63 teSW4djhPKiM0deKTbVItYlD36GGRFEg4CulBv9wvU14fPFRw3SkzfhRlEQWlN8jqVae 73zOlRKdpp+zwKnzhJwQ4Xu+KfRUryXT7Yiu9ht2F+hRLpgam2HyLWWQy5U3/J/3f6lV 1A== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xrs8cjxhs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Apr 2024 15:57:46 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43TEWetA011415; Mon, 29 Apr 2024 15:57:46 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xrqt6j98j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Apr 2024 15:57:46 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43TFuxjQ040299; Mon, 29 Apr 2024 15:57:45 GMT Received: from alaljime-dev-e4flex-vm.osdevelopmeniad.oraclevcn.com (alaljime-dev-e4flex-vm.allregionaliads.osdevelopmeniad.oraclevcn.com [100.100.249.106]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xrqt6j91w-4; Mon, 29 Apr 2024 15:57:45 +0000 From: Alejandro Jimenez To: kvm@vger.kernel.org Cc: seanjc@google.com, pbonzini@redhat.com, linux-kernel@vger.kernel.org, suravee.suthikulpanit@amd.com, vashegde@amd.com, mlevitsk@redhat.com, joao.m.martins@oracle.com, boris.ostrovsky@oracle.com, mark.kanda@oracle.com, alejandro.j.jimenez@oracle.com Subject: [PATCH 3/4] KVM: x86: Add a VM stat exposing when KVM PIT is set to reinject mode Date: Mon, 29 Apr 2024 15:57:37 +0000 Message-Id: <20240429155738.990025-4-alejandro.j.jimenez@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240429155738.990025-1-alejandro.j.jimenez@oracle.com> References: <20240429155738.990025-1-alejandro.j.jimenez@oracle.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-29_14,2024-04-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0 adultscore=0 mlxscore=0 suspectscore=0 phishscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404290101 X-Proofpoint-GUID: A8Pgz33DZ_3gpG3wfSelnF-CjYwMBvEO X-Proofpoint-ORIG-GUID: A8Pgz33DZ_3gpG3wfSelnF-CjYwMBvEO Add a stat to query when PIT is in reinject mode, which can have a large performance impact due to disabling SVM AVIC. When using in-kernel irqchip, QEMU and KVM default to creating a PIT in reinject mode, since this is necessary for old guest operating systems that use the PIT for timing. Unfortunately, reinject mode relies on EOI interception and so SVM AVIC must be inhibited when the PIT is set up using this mode. Signed-off-by: Alejandro Jimenez Reviewed-by: Vasant Hegde --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/i8254.c | 2 ++ arch/x86/kvm/x86.c | 3 ++- 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index f3b40cfebec4..e7e3213cefae 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1535,6 +1535,7 @@ struct kvm_vm_stat { u64 max_mmu_page_hash_collisions; u64 max_mmu_rmap_size; u64 synic_auto_eoi_used; + u64 pit_reinject_mode; }; struct kvm_vcpu_stat { diff --git a/arch/x86/kvm/i8254.c b/arch/x86/kvm/i8254.c index cd57a517d04a..44e593e909a1 100644 --- a/arch/x86/kvm/i8254.c +++ b/arch/x86/kvm/i8254.c @@ -316,6 +316,8 @@ void kvm_pit_set_reinject(struct kvm_pit *pit, bool reinject) kvm_unregister_irq_mask_notifier(kvm, 0, &pit->mask_notifier); } + kvm->stat.pit_reinject_mode = reinject; + atomic_set(&ps->reinject, reinject); } diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 27e339133068..03cb933920cb 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -257,7 +257,8 @@ const struct _kvm_stats_desc kvm_vm_stats_desc[] = { STATS_DESC_ICOUNTER(VM, nx_lpage_splits), STATS_DESC_PCOUNTER(VM, max_mmu_rmap_size), STATS_DESC_PCOUNTER(VM, max_mmu_page_hash_collisions), - STATS_DESC_IBOOLEAN(VM, synic_auto_eoi_used) + STATS_DESC_IBOOLEAN(VM, synic_auto_eoi_used), + STATS_DESC_IBOOLEAN(VM, pit_reinject_mode) }; const struct kvm_stats_header kvm_vm_stats_header = { From patchwork Mon Apr 29 15:57:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Jimenez X-Patchwork-Id: 13647311 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A2BB83A08; Mon, 29 Apr 2024 15:57:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=205.220.177.32 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714406278; cv=none; b=jlxIDYQMGKXoxXvn6bf6aP2y3KQ4LcSbEj8YxAF5Fm1bHfU9eH/UU2kE9PQjY3l4T1eYjPWPJeWpYe2CBBiIf5sefI5uaesTx6HS2Th1+QxKvKwS5cKEOX5agfN8VM0CpqKKv5q8S3cgm3TLApUOldSnpNJUgWdqgzly5OV4bUE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714406278; c=relaxed/simple; bh=R1npkU7NnhMaEAo02aQmqCLNDfuXn+An7TzBkaPxtlg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=JyD8xt7EtM1UMqO3nJNtOHz0lFB1/FOgeJcIatGBHSqFols/nxbJ1yiWxmZ4n3fzNo+QoOzTVu4X6xoC/6SphXa1D/HlIlrP1oB2dSrYJMckPfCTucTLABYKX33vY3ZfIerOUUd595s5JMOQtCK8jRBw84KmW1wEjs9NEj++hiI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=FjuVDr57; arc=none smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="FjuVDr57" Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 43TFmj9k006699; Mon, 29 Apr 2024 15:57:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-11-20; bh=7uCCKhplZr7mxOWs1OV2xLVt93NnadxTgAHX9zBzeUg=; b=FjuVDr57cVSzcGoZdM8Qv6BAvTVsKQYS+aR8llsLONHIspb1YGoxh6FivV7vLC+KzTxP um6PjlwebAXOmw6898HRX8FEQwzE6XWjp2WCYBnC6BYzp85HI+dYWNa/d3j5vP59QUYl /fEffKP2hi6kdo9CY34LX5KXzYZSluDaEX0dGmT/M3jp2qiJVFKPntrf0ZgfpPFmgwIq EYd7gxAWvGLXJEEUps4Usf8/T+64997XIXH+elczVtfipAkjCzi/Ii8NfieERXJH2yLE vcvD2gLx83socBWth/DV5eRZLtgXHAzUae3R/2OVcWlrJSQP8FX+7IvPyHTiLbCVeuFm ww== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3xrsdejwuu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Apr 2024 15:57:49 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 43TFdB04011361; Mon, 29 Apr 2024 15:57:48 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3xrqt6j99w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 29 Apr 2024 15:57:48 +0000 Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 43TFuxjS040299; Mon, 29 Apr 2024 15:57:47 GMT Received: from alaljime-dev-e4flex-vm.osdevelopmeniad.oraclevcn.com (alaljime-dev-e4flex-vm.allregionaliads.osdevelopmeniad.oraclevcn.com [100.100.249.106]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3xrqt6j91w-5; Mon, 29 Apr 2024 15:57:47 +0000 From: Alejandro Jimenez To: kvm@vger.kernel.org Cc: seanjc@google.com, pbonzini@redhat.com, linux-kernel@vger.kernel.org, suravee.suthikulpanit@amd.com, vashegde@amd.com, mlevitsk@redhat.com, joao.m.martins@oracle.com, boris.ostrovsky@oracle.com, mark.kanda@oracle.com, alejandro.j.jimenez@oracle.com Subject: [PATCH 4/4] KVM: x86: Add vCPU stat for APICv interrupt injections causing #VMEXIT Date: Mon, 29 Apr 2024 15:57:38 +0000 Message-Id: <20240429155738.990025-5-alejandro.j.jimenez@oracle.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20240429155738.990025-1-alejandro.j.jimenez@oracle.com> References: <20240429155738.990025-1-alejandro.j.jimenez@oracle.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26 definitions=2024-04-29_14,2024-04-29_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 malwarescore=0 adultscore=0 mlxscore=0 suspectscore=0 phishscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000 definitions=main-2404290101 X-Proofpoint-GUID: xg8gd8qHGZgxX_M2IfeAxc9sVgG0zQgs X-Proofpoint-ORIG-GUID: xg8gd8qHGZgxX_M2IfeAxc9sVgG0zQgs Even when APICv/AVIC is active, certain guest accesses to its local APIC(s) cannot be fully accelerated, and cause a #VMEXIT to allow the VMM to emulate the behavior and side effects. Expose a counter stat for these specific #VMEXIT types. Suggested-by: Paolo Bonzini Signed-off-by: Alejandro Jimenez Reviewed-by: Vasant Hegde # AMD --- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/svm/avic.c | 7 +++++++ arch/x86/kvm/vmx/vmx.c | 2 ++ arch/x86/kvm/x86.c | 1 + 4 files changed, 11 insertions(+) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index e7e3213cefae..388979dfe9f3 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1576,6 +1576,7 @@ struct kvm_vcpu_stat { u64 guest_mode; u64 notify_window_exits; u64 apicv_active; + u64 apicv_unaccelerated_inj; }; struct x86_instruction_info; diff --git a/arch/x86/kvm/svm/avic.c b/arch/x86/kvm/svm/avic.c index 4b74ea91f4e6..274041d3cf66 100644 --- a/arch/x86/kvm/svm/avic.c +++ b/arch/x86/kvm/svm/avic.c @@ -517,6 +517,8 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu) kvm_apic_write_nodecode(vcpu, APIC_ICR); else kvm_apic_send_ipi(apic, icrl, icrh); + + ++vcpu->stat.apicv_unaccelerated_inj; break; case AVIC_IPI_FAILURE_TARGET_NOT_RUNNING: /* @@ -525,6 +527,8 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu) * vcpus. So, we just need to kick the appropriate vcpu. */ avic_kick_target_vcpus(vcpu->kvm, apic, icrl, icrh, index); + + ++vcpu->stat.apicv_unaccelerated_inj; break; case AVIC_IPI_FAILURE_INVALID_BACKING_PAGE: WARN_ONCE(1, "Invalid backing page\n"); @@ -704,6 +708,9 @@ int avic_unaccelerated_access_interception(struct kvm_vcpu *vcpu) trace_kvm_avic_unaccelerated_access(vcpu->vcpu_id, offset, trap, write, vector); + + ++vcpu->stat.apicv_unaccelerated_inj; + if (trap) { /* Handling Trap */ WARN_ONCE(!write, "svm: Handling trap read.\n"); diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index f10b5f8f364b..a7487f12ded1 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -5657,6 +5657,8 @@ static int handle_apic_write(struct kvm_vcpu *vcpu) { unsigned long exit_qualification = vmx_get_exit_qual(vcpu); + ++vcpu->stat.apicv_unaccelerated_inj; + /* * APIC-write VM-Exit is trap-like, KVM doesn't need to advance RIP and * hardware has done any necessary aliasing, offset adjustments, etc... diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 03cb933920cb..c8730b0fac87 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -307,6 +307,7 @@ const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = { STATS_DESC_IBOOLEAN(VCPU, guest_mode), STATS_DESC_COUNTER(VCPU, notify_window_exits), STATS_DESC_IBOOLEAN(VCPU, apicv_active), + STATS_DESC_COUNTER(VCPU, apicv_unaccelerated_inj), }; const struct kvm_stats_header kvm_vcpu_stats_header = {