From patchwork Mon Nov 4 09:40:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dongli Zhang X-Patchwork-Id: 13861138 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 273DDD132A2 for ; Mon, 4 Nov 2024 09:42:57 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t7tbJ-0004gj-6b; Mon, 04 Nov 2024 04:42:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t7tbB-0004es-W1 for qemu-devel@nongnu.org; Mon, 04 Nov 2024 04:42:15 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t7tb8-0000l1-Ra for qemu-devel@nongnu.org; Mon, 04 Nov 2024 04:42:13 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A48fj4H012809; Mon, 4 Nov 2024 09:41:56 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=corp-2023-11-20; bh=8dj9+ Fj2Y7+fcJzsYZdCfnhnZ64eJxfakPqpai/l9Oc=; b=UsR1EPUSgMA/8T3Uj4kdO IOMuRDOvkNtmec6hplYuqak/t05nN03z2SzN7ywRj462Mmg63oZ3g7VWZDBwtG3E fFNgIyI0SNSM5X5RjlrWki5kWOLZEDfMIsaDlityqtvuJoamNgBYvmJSvv6Nqkew 6AEiuTWyABosn7lJ+mXuhOKiEXLgK+2W+hx5Cb6ePcnN8Jtm4CLuQxm+6Y1lihu8 72BnNJe1ctP1YUXea5AmSXlgRUv2sweKIARO6AjWrIIAmJmSVLrp9jNS68HKD4cR EyzZ5AsU7KMp1SkB+R9VLNIn0tIGgVH2wVndFLkYFyBeWsBasnS9XmbwKFMVWpjz A== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42nanytbxg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 Nov 2024 09:41:55 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A48fZlX009108; Mon, 4 Nov 2024 09:41:54 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42nahbt090-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 Nov 2024 09:41:54 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A49fqfe018519; Mon, 4 Nov 2024 09:41:54 GMT Received: from localhost.localdomain (ca-dev80.us.oracle.com [10.211.9.80]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 42nahbt06k-2; Mon, 04 Nov 2024 09:41:53 +0000 From: Dongli Zhang To: qemu-devel@nongnu.org, kvm@vger.kernel.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, sandipan.das@amd.com, babu.moger@amd.com, zhao1.liu@intel.com, likexu@tencent.com, like.xu.linux@gmail.com, zhenyuw@linux.intel.com, groug@kaod.org, lyan@digitalocean.com, khorenko@virtuozzo.com, alexander.ivanov@virtuozzo.com, den@virtuozzo.com, joe.jin@oracle.com, davydov-max@yandex-team.ru Subject: [PATCH 1/7] target/i386: disable PerfMonV2 when PERFCORE unavailable Date: Mon, 4 Nov 2024 01:40:16 -0800 Message-ID: <20241104094119.4131-2-dongli.zhang@oracle.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241104094119.4131-1-dongli.zhang@oracle.com> References: <20241104094119.4131-1-dongli.zhang@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-04_07,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 suspectscore=0 bulkscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411040085 X-Proofpoint-ORIG-GUID: Q3sRATBTcJgcGAcqpnTcqUfqFtcCcV9- X-Proofpoint-GUID: Q3sRATBTcJgcGAcqpnTcqUfqFtcCcV9- Received-SPF: pass client-ip=205.220.165.32; envelope-from=dongli.zhang@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org When the PERFCORE is disabled with "-cpu host,-perfctr-core", it is reflected in in guest dmesg. [ 0.285136] Performance Events: AMD PMU driver. However, the guest cpuid indicates the PerfMonV2 is still available. CPU: Extended Performance Monitoring and Debugging (0x80000022): AMD performance monitoring V2 = true AMD LBR V2 = false AMD LBR stack & PMC freezing = false number of core perf ctrs = 0x6 (6) number of LBR stack entries = 0x0 (0) number of avail Northbridge perf ctrs = 0x0 (0) number of available UMC PMCs = 0x0 (0) active UMCs bitmask = 0x0 Disable PerfMonV2 in cpuid when PERFCORE is disabled. Fixes: 209b0ac12074 ("target/i386: Add PerfMonV2 feature bit") Signed-off-by: Dongli Zhang --- target/i386/cpu.c | 1 + 1 file changed, 1 insertion(+) diff --git a/target/i386/cpu.c b/target/i386/cpu.c index 3baa95481f..4490a7a8d6 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -7103,6 +7103,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, *eax = *ebx = *ecx = *edx = 0; /* AMD Extended Performance Monitoring and Debug */ if (kvm_enabled() && cpu->enable_pmu && + (env->features[FEAT_8000_0001_ECX] & CPUID_EXT3_PERFCORE) && (env->features[FEAT_8000_0022_EAX] & CPUID_8000_0022_EAX_PERFMON_V2)) { *eax |= CPUID_8000_0022_EAX_PERFMON_V2; *ebx |= kvm_arch_get_supported_cpuid(cs->kvm_state, index, count, From patchwork Mon Nov 4 09:40:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dongli Zhang X-Patchwork-Id: 13861126 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6932FD132A3 for ; Mon, 4 Nov 2024 09:42:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t7tbB-0004eU-KR; Mon, 04 Nov 2024 04:42:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t7tbA-0004dY-PW for qemu-devel@nongnu.org; Mon, 04 Nov 2024 04:42:12 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t7tb5-0000j2-87 for qemu-devel@nongnu.org; Mon, 04 Nov 2024 04:42:12 -0500 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A48gHuK004180; Mon, 4 Nov 2024 09:41:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=corp-2023-11-20; bh=bfLnj bxcSPm6lnfdVUJ780ERZUAYKYTQqq6FUng196c=; b=dxuYMhMH5AjRDPqLOREeT hmt78JhXZJXgnn5T/Ymq+vGMJeCfOKIR/y09h4S0u4U8SHkOiT0H1dzITAilUavs gPIQJETp55L1fL7KeQ0u2StgztyWY7edfvnf2c7HMKENiO+e11f/bBxAIs86+49H dlHzVqSvUmtXZIfiY84hcdk0iPRS4vOFttlCF2HEUFvK1EclKCIiNZubBZ5rBT6w 4Atn1aDd4qIjGIlEm/lTaSpTyBIagXXQlz/Own3MNXL2FIG/TEf9EcTmlQLpXQS1 Hg+KB75aly7BVbbB41EL0D5QRK3qRQEYolBhVfx2oaM2rRAEL5bRmXf53uah5kX3 g== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42nav22acy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 Nov 2024 09:41:57 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A47NvkB009709; Mon, 4 Nov 2024 09:41:56 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42nahbt0a0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 Nov 2024 09:41:56 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A49fqfg018519; Mon, 4 Nov 2024 09:41:55 GMT Received: from localhost.localdomain (ca-dev80.us.oracle.com [10.211.9.80]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 42nahbt06k-3; Mon, 04 Nov 2024 09:41:55 +0000 From: Dongli Zhang To: qemu-devel@nongnu.org, kvm@vger.kernel.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, sandipan.das@amd.com, babu.moger@amd.com, zhao1.liu@intel.com, likexu@tencent.com, like.xu.linux@gmail.com, zhenyuw@linux.intel.com, groug@kaod.org, lyan@digitalocean.com, khorenko@virtuozzo.com, alexander.ivanov@virtuozzo.com, den@virtuozzo.com, joe.jin@oracle.com, davydov-max@yandex-team.ru Subject: [PATCH 2/7] target/i386/kvm: introduce 'pmu-cap-disabled' to set KVM_PMU_CAP_DISABLE Date: Mon, 4 Nov 2024 01:40:17 -0800 Message-ID: <20241104094119.4131-3-dongli.zhang@oracle.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241104094119.4131-1-dongli.zhang@oracle.com> References: <20241104094119.4131-1-dongli.zhang@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-04_07,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 suspectscore=0 bulkscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411040085 X-Proofpoint-GUID: akal7AZncT7cVfEwlijF__SDIscOosvs X-Proofpoint-ORIG-GUID: akal7AZncT7cVfEwlijF__SDIscOosvs Received-SPF: pass client-ip=205.220.177.32; envelope-from=dongli.zhang@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The AMD PMU virtualization is not disabled when configuring "-cpu host,-pmu" in the QEMU command line on an AMD server. Neither "-cpu host,-pmu" nor "-cpu EPYC" effectively disables AMD PMU virtualization in such an environment. As a result, VM logs typically show: [ 0.510611] Performance Events: Fam17h+ core perfctr, AMD PMU driver. whereas the expected logs should be: [ 0.596381] Performance Events: PMU not available due to virtualization, using software events only. [ 0.600972] NMI watchdog: Perf NMI watchdog permanently disabled This discrepancy occurs because AMD PMU does not use CPUID to determine whether PMU virtualization is supported. To address this, we introduce a new property, 'pmu-cap-disabled', for KVM acceleration. This property sets KVM_PMU_CAP_DISABLE if KVM_CAP_PMU_CAPABILITY is supported. Note that this feature currently supports only x86 hosts, as KVM_CAP_PMU_CAPABILITY is used exclusively for x86 systems. Signed-off-by: Dongli Zhang --- Another previous solution to re-use '-cpu host,-pmu': https://lore.kernel.org/all/20221119122901.2469-1-dongli.zhang@oracle.com/ accel/kvm/kvm-all.c | 1 + include/sysemu/kvm_int.h | 1 + qemu-options.hx | 9 ++++++- target/i386/cpu.c | 2 +- target/i386/kvm/kvm.c | 52 ++++++++++++++++++++++++++++++++++++++ target/i386/kvm/kvm_i386.h | 2 ++ 6 files changed, 65 insertions(+), 2 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 801cff16a5..8b5ba45cf7 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -3933,6 +3933,7 @@ static void kvm_accel_instance_init(Object *obj) s->xen_evtchn_max_pirq = 256; s->device = NULL; s->msr_energy.enable = false; + s->pmu_cap_disabled = false; } /** diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h index a1e72763da..cdee1a481c 100644 --- a/include/sysemu/kvm_int.h +++ b/include/sysemu/kvm_int.h @@ -166,6 +166,7 @@ struct KVMState uint16_t xen_gnttab_max_frames; uint16_t xen_evtchn_max_pirq; char *device; + bool pmu_cap_disabled; }; void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml, diff --git a/qemu-options.hx b/qemu-options.hx index dacc9790a4..9684424835 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -191,7 +191,8 @@ DEF("accel", HAS_ARG, QEMU_OPTION_accel, " eager-split-size=n (KVM Eager Page Split chunk size, default 0, disabled. ARM only)\n" " notify-vmexit=run|internal-error|disable,notify-window=n (enable notify VM exit and set notify window, x86 only)\n" " thread=single|multi (enable multi-threaded TCG)\n" - " device=path (KVM device path, default /dev/kvm)\n", QEMU_ARCH_ALL) + " device=path (KVM device path, default /dev/kvm)\n" + " pmu-cap-disabled=true|false (disable KVM_CAP_PMU_CAPABILITY, x86 only, default false)\n", QEMU_ARCH_ALL) SRST ``-accel name[,prop=value[,...]]`` This is used to enable an accelerator. Depending on the target @@ -277,6 +278,12 @@ SRST option can be used to pass the KVM device to use via a file descriptor by setting the value to ``/dev/fdset/NN``. + ``pmu-cap-disabled=true|false`` + When using the KVM accelerator, it controls the disabling of + KVM_CAP_PMU_CAPABILITY via KVM_PMU_CAP_DISABLE. When this capability is + disabled, PMU virtualization is turned off at the KVM module level. + Note that this functionality is supported for x86 hosts only. + ERST DEF("smp", HAS_ARG, QEMU_OPTION_smp, diff --git a/target/i386/cpu.c b/target/i386/cpu.c index 4490a7a8d6..c97ed74a47 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -7102,7 +7102,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, case 0x80000022: *eax = *ebx = *ecx = *edx = 0; /* AMD Extended Performance Monitoring and Debug */ - if (kvm_enabled() && cpu->enable_pmu && + if (kvm_enabled() && cpu->enable_pmu && !kvm_pmu_cap_disabled() && (env->features[FEAT_8000_0001_ECX] & CPUID_EXT3_PERFCORE) && (env->features[FEAT_8000_0022_EAX] & CPUID_8000_0022_EAX_PERFMON_V2)) { *eax |= CPUID_8000_0022_EAX_PERFMON_V2; diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c index 8e17942c3b..ed73b1e7e0 100644 --- a/target/i386/kvm/kvm.c +++ b/target/i386/kvm/kvm.c @@ -166,6 +166,7 @@ static bool has_msr_vmx_procbased_ctls2; static bool has_msr_perf_capabs; static bool has_msr_pkrs; static bool has_msr_hwcr; +static bool has_pmu_cap; static uint32_t has_architectural_pmu_version; static uint32_t num_architectural_pmu_gp_counters; @@ -3196,6 +3197,11 @@ static void kvm_vm_enable_energy_msrs(KVMState *s) return; } +bool kvm_pmu_cap_disabled(void) +{ + return kvm_state->pmu_cap_disabled; +} + int kvm_arch_init(MachineState *ms, KVMState *s) { int ret; @@ -3335,6 +3341,23 @@ int kvm_arch_init(MachineState *ms, KVMState *s) } } + has_pmu_cap = kvm_check_extension(s, KVM_CAP_PMU_CAPABILITY); + + if (s->pmu_cap_disabled) { + if (has_pmu_cap) { + ret = kvm_vm_enable_cap(s, KVM_CAP_PMU_CAPABILITY, 0, + KVM_PMU_CAP_DISABLE); + if (ret < 0) { + s->pmu_cap_disabled = false; + error_report("kvm: Failed to disable pmu cap: %s", + strerror(-ret)); + } + } else { + s->pmu_cap_disabled = false; + error_report("kvm: KVM_CAP_PMU_CAPABILITY is not supported"); + } + } + return 0; } @@ -6525,6 +6548,29 @@ static void kvm_arch_set_xen_evtchn_max_pirq(Object *obj, Visitor *v, s->xen_evtchn_max_pirq = value; } +static void kvm_set_pmu_cap_disabled(Object *obj, Visitor *v, + const char *name, void *opaque, + Error **errp) +{ + KVMState *s = KVM_STATE(obj); + bool pmu_cap_disabled; + Error *error = NULL; + + if (s->fd != -1) { + error_setg(errp, "Cannot set properties after the accelerator has " + "been initialized"); + return; + } + + visit_type_bool(v, name, &pmu_cap_disabled, &error); + if (error) { + error_propagate(errp, error); + return; + } + + s->pmu_cap_disabled = pmu_cap_disabled; +} + void kvm_arch_accel_class_init(ObjectClass *oc) { object_class_property_add_enum(oc, "notify-vmexit", "NotifyVMexitOption", @@ -6564,6 +6610,12 @@ void kvm_arch_accel_class_init(ObjectClass *oc) NULL, NULL); object_class_property_set_description(oc, "xen-evtchn-max-pirq", "Maximum number of Xen PIRQs"); + + object_class_property_add(oc, "pmu-cap-disabled", "bool", + NULL, kvm_set_pmu_cap_disabled, + NULL, NULL); + object_class_property_set_description(oc, "pmu-cap-disabled", + "Disable KVM_CAP_PMU_CAPABILITY"); } void kvm_set_max_apic_id(uint32_t max_apic_id) diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h index 9de9c0d303..03522de9d1 100644 --- a/target/i386/kvm/kvm_i386.h +++ b/target/i386/kvm/kvm_i386.h @@ -49,6 +49,8 @@ uint64_t kvm_arch_get_supported_msr_feature(KVMState *s, uint32_t index); void kvm_set_max_apic_id(uint32_t max_apic_id); void kvm_request_xsave_components(X86CPU *cpu, uint64_t mask); +bool kvm_pmu_cap_disabled(void); + #ifdef CONFIG_KVM bool kvm_is_vm_type_supported(int type); From patchwork Mon Nov 4 09:40:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dongli Zhang X-Patchwork-Id: 13861142 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 19D3AD132A3 for ; Mon, 4 Nov 2024 09:43:34 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t7tbI-0004gV-DO; Mon, 04 Nov 2024 04:42:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t7tbC-0004ex-PM for qemu-devel@nongnu.org; Mon, 04 Nov 2024 04:42:15 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t7tbA-0000lJ-I0 for qemu-devel@nongnu.org; Mon, 04 Nov 2024 04:42:14 -0500 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A48g0Op009185; Mon, 4 Nov 2024 09:41:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=corp-2023-11-20; bh=c/gUe N/1UHuvwH/MPvRLZ4qA0zjIgO5JFJ7PpI3xCiU=; b=CqpSVp6tJFzhiySQe760a LJaRs9dfROgHmYpwwrbC5S4f3gSLEQCT3NkXb57FoIdlW6fsYGG81AtZ/07fBlAm ljObIvQGrN3ZLwTVJRf0YYNB44iJGGmLchgu7vcGrH0tan46yZCDHXR9tBQ7Ivg4 zUDy9BtaESA0ybFIEcCDiW2rvP4Nejm1/8zG00BeG4fZEMKxg0OeU9rPQ23EnJ3E rb6J8P8eDQR/4GtYN+6v6EzspfANE5W/S7h10oglG7YePdyVTerFA8Q8etI4ramX Kufg0irESMwtCrOBqgwn81elzinqxR6bQxm25yX1YFrns5gM+TdDidtAEIoDwsQ9 w== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42nbpsj98e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 Nov 2024 09:41:58 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A49YcLD009089; Mon, 4 Nov 2024 09:41:58 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42nahbt0ar-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 Nov 2024 09:41:58 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A49fqfi018519; Mon, 4 Nov 2024 09:41:57 GMT Received: from localhost.localdomain (ca-dev80.us.oracle.com [10.211.9.80]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 42nahbt06k-4; Mon, 04 Nov 2024 09:41:57 +0000 From: Dongli Zhang To: qemu-devel@nongnu.org, kvm@vger.kernel.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, sandipan.das@amd.com, babu.moger@amd.com, zhao1.liu@intel.com, likexu@tencent.com, like.xu.linux@gmail.com, zhenyuw@linux.intel.com, groug@kaod.org, lyan@digitalocean.com, khorenko@virtuozzo.com, alexander.ivanov@virtuozzo.com, den@virtuozzo.com, joe.jin@oracle.com, davydov-max@yandex-team.ru Subject: [PATCH 3/7] target/i386/kvm: init PMU information only once Date: Mon, 4 Nov 2024 01:40:18 -0800 Message-ID: <20241104094119.4131-4-dongli.zhang@oracle.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241104094119.4131-1-dongli.zhang@oracle.com> References: <20241104094119.4131-1-dongli.zhang@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-04_07,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 suspectscore=0 bulkscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411040085 X-Proofpoint-ORIG-GUID: pqWGiGQkLfgBDNyXutvfFe4Epf2g77F1 X-Proofpoint-GUID: pqWGiGQkLfgBDNyXutvfFe4Epf2g77F1 Received-SPF: pass client-ip=205.220.177.32; envelope-from=dongli.zhang@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Currently, the 'has_architectural_pmu_version', 'num_architectural_pmu_gp_counters', and 'num_architectural_pmu_fixed_counters' are shared globally across all vCPUs and are initialized during the setup of each vCPU. To optimize, initialize PMU information only once for the first vCPU. Additionally, the code extracted from kvm_x86_build_cpuid() is unrelated to the process of building the CPUID. Signed-off-by: Dongli Zhang --- target/i386/kvm/kvm.c | 71 +++++++++++++++++++++++++++---------------- 1 file changed, 44 insertions(+), 27 deletions(-) diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c index ed73b1e7e0..5c0276f889 100644 --- a/target/i386/kvm/kvm.c +++ b/target/i386/kvm/kvm.c @@ -1961,33 +1961,6 @@ static uint32_t kvm_x86_build_cpuid(CPUX86State *env, } } - if (limit >= 0x0a) { - uint32_t eax, edx; - - cpu_x86_cpuid(env, 0x0a, 0, &eax, &unused, &unused, &edx); - - has_architectural_pmu_version = eax & 0xff; - if (has_architectural_pmu_version > 0) { - num_architectural_pmu_gp_counters = (eax & 0xff00) >> 8; - - /* Shouldn't be more than 32, since that's the number of bits - * available in EBX to tell us _which_ counters are available. - * Play it safe. - */ - if (num_architectural_pmu_gp_counters > MAX_GP_COUNTERS) { - num_architectural_pmu_gp_counters = MAX_GP_COUNTERS; - } - - if (has_architectural_pmu_version > 1) { - num_architectural_pmu_fixed_counters = edx & 0x1f; - - if (num_architectural_pmu_fixed_counters > MAX_FIXED_COUNTERS) { - num_architectural_pmu_fixed_counters = MAX_FIXED_COUNTERS; - } - } - } - } - cpu_x86_cpuid(env, 0x80000000, 0, &limit, &unused, &unused, &unused); for (i = 0x80000000; i <= limit; i++) { @@ -2055,6 +2028,43 @@ full: abort(); } +static void kvm_init_pmu_info(CPUX86State *env) +{ + uint32_t eax, edx; + uint32_t unused; + uint32_t limit; + + cpu_x86_cpuid(env, 0, 0, &limit, &unused, &unused, &unused); + + if (limit < 0x0a) { + return; + } + + cpu_x86_cpuid(env, 0x0a, 0, &eax, &unused, &unused, &edx); + + has_architectural_pmu_version = eax & 0xff; + if (has_architectural_pmu_version > 0) { + num_architectural_pmu_gp_counters = (eax & 0xff00) >> 8; + + /* + * Shouldn't be more than 32, since that's the number of bits + * available in EBX to tell us _which_ counters are available. + * Play it safe. + */ + if (num_architectural_pmu_gp_counters > MAX_GP_COUNTERS) { + num_architectural_pmu_gp_counters = MAX_GP_COUNTERS; + } + + if (has_architectural_pmu_version > 1) { + num_architectural_pmu_fixed_counters = edx & 0x1f; + + if (num_architectural_pmu_fixed_counters > MAX_FIXED_COUNTERS) { + num_architectural_pmu_fixed_counters = MAX_FIXED_COUNTERS; + } + } + } +} + int kvm_arch_init_vcpu(CPUState *cs) { struct { @@ -2237,6 +2247,13 @@ int kvm_arch_init_vcpu(CPUState *cs) cpuid_i = kvm_x86_build_cpuid(env, cpuid_data.entries, cpuid_i); cpuid_data.cpuid.nent = cpuid_i; + /* + * Initialize PMU information only once for the first vCPU. + */ + if (cs == first_cpu) { + kvm_init_pmu_info(env); + } + if (((env->cpuid_version >> 8)&0xF) >= 6 && (env->features[FEAT_1_EDX] & (CPUID_MCE | CPUID_MCA)) == (CPUID_MCE | CPUID_MCA)) { From patchwork Mon Nov 4 09:40:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dongli Zhang X-Patchwork-Id: 13861143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 534C0D132A2 for ; Mon, 4 Nov 2024 09:43:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t7tbL-0004h3-2k; Mon, 04 Nov 2024 04:42:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t7tbE-0004fc-Nk for qemu-devel@nongnu.org; Mon, 04 Nov 2024 04:42:17 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t7tbB-0000lh-Cc for qemu-devel@nongnu.org; Mon, 04 Nov 2024 04:42:15 -0500 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A48ferU019278; Mon, 4 Nov 2024 09:42:01 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=corp-2023-11-20; bh=aw8Da KbTFy541Imzo6l3juUlheHXjuaW/lcnjuRi9i8=; b=KQzh8vUimbrwlWWWRQ0bQ Wr2Sa1pd+x5EDv7de0eGJmN7gJ84M+8+oNTQQ5MC9fxVf98M1mmD9zPZPCJ3dP4v zwW4NhdTk1KM36Ijw1ykS6CG51DDB3fXtGGF495iX9SRayQKVZTDLUslP6r9QUL7 qrLW0BRjBSuXIVz1oQnZFaHSPnz3pHExOwX1Ohate/CpqE4UJdjHo1N1qhKil2rs CXf4uU05SabHWvc2Puu5cc7ocxU/d8FHq2DsjTJNVTlntLk8811hKK4QAGqKXNaZ /+/BiNBIddJ4GedmqsF4nsXtQUUBwEW2Eof5oFuGp4LyW0e0ePO2AWhOQfJmWGUO A== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42nagc2a5x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 Nov 2024 09:42:00 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A49GxM0009703; Mon, 4 Nov 2024 09:42:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42nahbt0bu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 Nov 2024 09:42:00 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A49fqfk018519; Mon, 4 Nov 2024 09:41:59 GMT Received: from localhost.localdomain (ca-dev80.us.oracle.com [10.211.9.80]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 42nahbt06k-5; Mon, 04 Nov 2024 09:41:59 +0000 From: Dongli Zhang To: qemu-devel@nongnu.org, kvm@vger.kernel.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, sandipan.das@amd.com, babu.moger@amd.com, zhao1.liu@intel.com, likexu@tencent.com, like.xu.linux@gmail.com, zhenyuw@linux.intel.com, groug@kaod.org, lyan@digitalocean.com, khorenko@virtuozzo.com, alexander.ivanov@virtuozzo.com, den@virtuozzo.com, joe.jin@oracle.com, davydov-max@yandex-team.ru Subject: [PATCH 4/7] target/i386/kvm: rename architectural PMU variables Date: Mon, 4 Nov 2024 01:40:19 -0800 Message-ID: <20241104094119.4131-5-dongli.zhang@oracle.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241104094119.4131-1-dongli.zhang@oracle.com> References: <20241104094119.4131-1-dongli.zhang@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-04_07,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 suspectscore=0 bulkscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411040085 X-Proofpoint-ORIG-GUID: G2G7tkFTicVLrnW9tFGXNacg0DRyZo-U X-Proofpoint-GUID: G2G7tkFTicVLrnW9tFGXNacg0DRyZo-U Received-SPF: pass client-ip=205.220.177.32; envelope-from=dongli.zhang@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org AMD does not have what is commonly referred to as an architectural PMU. Therefore, we need to rename the following variables to be applicable for both Intel and AMD: - has_architectural_pmu_version - num_architectural_pmu_gp_counters - num_architectural_pmu_fixed_counters For Intel processors, the meaning of has_pmu_version remains unchanged. For AMD processors: has_pmu_version == 1 corresponds to versions before AMD PerfMonV2. has_pmu_version == 2 corresponds to AMD PerfMonV2. Signed-off-by: Dongli Zhang --- target/i386/kvm/kvm.c | 49 ++++++++++++++++++++++++------------------- 1 file changed, 28 insertions(+), 21 deletions(-) diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c index 5c0276f889..ca2b644e2c 100644 --- a/target/i386/kvm/kvm.c +++ b/target/i386/kvm/kvm.c @@ -168,9 +168,16 @@ static bool has_msr_pkrs; static bool has_msr_hwcr; static bool has_pmu_cap; -static uint32_t has_architectural_pmu_version; -static uint32_t num_architectural_pmu_gp_counters; -static uint32_t num_architectural_pmu_fixed_counters; +/* + * For Intel processors, the meaning is the architectural PMU version + * number. + * + * For AMD processors: 1 corresponds to the prior versions, and 2 + * corresponds to AMD PerfMonV2. + */ +static uint32_t has_pmu_version; +static uint32_t num_pmu_gp_counters; +static uint32_t num_pmu_fixed_counters; static int has_xsave2; static int has_xcrs; @@ -2042,24 +2049,24 @@ static void kvm_init_pmu_info(CPUX86State *env) cpu_x86_cpuid(env, 0x0a, 0, &eax, &unused, &unused, &edx); - has_architectural_pmu_version = eax & 0xff; - if (has_architectural_pmu_version > 0) { - num_architectural_pmu_gp_counters = (eax & 0xff00) >> 8; + has_pmu_version = eax & 0xff; + if (has_pmu_version > 0) { + num_pmu_gp_counters = (eax & 0xff00) >> 8; /* * Shouldn't be more than 32, since that's the number of bits * available in EBX to tell us _which_ counters are available. * Play it safe. */ - if (num_architectural_pmu_gp_counters > MAX_GP_COUNTERS) { - num_architectural_pmu_gp_counters = MAX_GP_COUNTERS; + if (num_pmu_gp_counters > MAX_GP_COUNTERS) { + num_pmu_gp_counters = MAX_GP_COUNTERS; } - if (has_architectural_pmu_version > 1) { - num_architectural_pmu_fixed_counters = edx & 0x1f; + if (has_pmu_version > 1) { + num_pmu_fixed_counters = edx & 0x1f; - if (num_architectural_pmu_fixed_counters > MAX_FIXED_COUNTERS) { - num_architectural_pmu_fixed_counters = MAX_FIXED_COUNTERS; + if (num_pmu_fixed_counters > MAX_FIXED_COUNTERS) { + num_pmu_fixed_counters = MAX_FIXED_COUNTERS; } } } @@ -4020,25 +4027,25 @@ static int kvm_put_msrs(X86CPU *cpu, int level) kvm_msr_entry_add(cpu, MSR_KVM_POLL_CONTROL, env->poll_control_msr); } - if (has_architectural_pmu_version > 0) { - if (has_architectural_pmu_version > 1) { + if (has_pmu_version > 0) { + if (has_pmu_version > 1) { /* Stop the counter. */ kvm_msr_entry_add(cpu, MSR_CORE_PERF_FIXED_CTR_CTRL, 0); kvm_msr_entry_add(cpu, MSR_CORE_PERF_GLOBAL_CTRL, 0); } /* Set the counter values. */ - for (i = 0; i < num_architectural_pmu_fixed_counters; i++) { + for (i = 0; i < num_pmu_fixed_counters; i++) { kvm_msr_entry_add(cpu, MSR_CORE_PERF_FIXED_CTR0 + i, env->msr_fixed_counters[i]); } - for (i = 0; i < num_architectural_pmu_gp_counters; i++) { + for (i = 0; i < num_pmu_gp_counters; i++) { kvm_msr_entry_add(cpu, MSR_P6_PERFCTR0 + i, env->msr_gp_counters[i]); kvm_msr_entry_add(cpu, MSR_P6_EVNTSEL0 + i, env->msr_gp_evtsel[i]); } - if (has_architectural_pmu_version > 1) { + if (has_pmu_version > 1) { kvm_msr_entry_add(cpu, MSR_CORE_PERF_GLOBAL_STATUS, env->msr_global_status); kvm_msr_entry_add(cpu, MSR_CORE_PERF_GLOBAL_OVF_CTRL, @@ -4496,17 +4503,17 @@ static int kvm_get_msrs(X86CPU *cpu) if (env->features[FEAT_KVM] & (1 << KVM_FEATURE_POLL_CONTROL)) { kvm_msr_entry_add(cpu, MSR_KVM_POLL_CONTROL, 1); } - if (has_architectural_pmu_version > 0) { - if (has_architectural_pmu_version > 1) { + if (has_pmu_version > 0) { + if (has_pmu_version > 1) { kvm_msr_entry_add(cpu, MSR_CORE_PERF_FIXED_CTR_CTRL, 0); kvm_msr_entry_add(cpu, MSR_CORE_PERF_GLOBAL_CTRL, 0); kvm_msr_entry_add(cpu, MSR_CORE_PERF_GLOBAL_STATUS, 0); kvm_msr_entry_add(cpu, MSR_CORE_PERF_GLOBAL_OVF_CTRL, 0); } - for (i = 0; i < num_architectural_pmu_fixed_counters; i++) { + for (i = 0; i < num_pmu_fixed_counters; i++) { kvm_msr_entry_add(cpu, MSR_CORE_PERF_FIXED_CTR0 + i, 0); } - for (i = 0; i < num_architectural_pmu_gp_counters; i++) { + for (i = 0; i < num_pmu_gp_counters; i++) { kvm_msr_entry_add(cpu, MSR_P6_PERFCTR0 + i, 0); kvm_msr_entry_add(cpu, MSR_P6_EVNTSEL0 + i, 0); } From patchwork Mon Nov 4 09:40:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dongli Zhang X-Patchwork-Id: 13861144 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2BD05D132A6 for ; Mon, 4 Nov 2024 09:43:34 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t7tbM-0004hB-SN; Mon, 04 Nov 2024 04:42:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t7tbG-0004gD-Iq for qemu-devel@nongnu.org; Mon, 04 Nov 2024 04:42:18 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t7tbE-0000mo-JH for qemu-devel@nongnu.org; Mon, 04 Nov 2024 04:42:18 -0500 Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A48fcC0018200; Mon, 4 Nov 2024 09:42:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=corp-2023-11-20; bh=Iclc0 zYVet4nx97YIS/BP9FtQpieaSXD0nnk1nSLYTE=; b=mhhDRL664Gemhe8XLOnwp y3kTWMYkGSJI+MiKWwFekrw1xofWdDq4XhuB7kEYIUGkSMUMIdjo5tgYsAzhU/1V J43KiAjRc5aUn3o/WRWbyyI2ESAEf2OSDunXQruP1FapEfKsbANLbrTMKx1Jr9L7 ZOkm0mFpT1GPwk7rJnm3k6jS3XtFbHaLdDTaJUUZBzKTX5YcwOP7Ki9NPGL8CSl+ sksldJHzNXmSn5jLRsaKWw7cY95kkLyavvUJDdQe9zGO0Q+Elsb4v0h8GnIaV9KL s/t3QqBTlr0LwEdeXJd8GqVG1oXD+CdipWzkreEIVB3Qy6Zot0jpl9p3eZHoZJNX g== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42nc4bt9m6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 Nov 2024 09:42:02 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A47aUTJ009324; Mon, 4 Nov 2024 09:42:02 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42nahbt0d1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 Nov 2024 09:42:02 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A49fqfm018519; Mon, 4 Nov 2024 09:42:01 GMT Received: from localhost.localdomain (ca-dev80.us.oracle.com [10.211.9.80]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 42nahbt06k-6; Mon, 04 Nov 2024 09:42:01 +0000 From: Dongli Zhang To: qemu-devel@nongnu.org, kvm@vger.kernel.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, sandipan.das@amd.com, babu.moger@amd.com, zhao1.liu@intel.com, likexu@tencent.com, like.xu.linux@gmail.com, zhenyuw@linux.intel.com, groug@kaod.org, lyan@digitalocean.com, khorenko@virtuozzo.com, alexander.ivanov@virtuozzo.com, den@virtuozzo.com, joe.jin@oracle.com, davydov-max@yandex-team.ru Subject: [PATCH 5/7] target/i386/kvm: reset AMD PMU registers during VM reset Date: Mon, 4 Nov 2024 01:40:20 -0800 Message-ID: <20241104094119.4131-6-dongli.zhang@oracle.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241104094119.4131-1-dongli.zhang@oracle.com> References: <20241104094119.4131-1-dongli.zhang@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-04_07,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 suspectscore=0 bulkscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411040085 X-Proofpoint-ORIG-GUID: LGLvVXHmiIkrbiWy_YSBI4nhmM2P3gqK X-Proofpoint-GUID: LGLvVXHmiIkrbiWy_YSBI4nhmM2P3gqK Received-SPF: pass client-ip=205.220.177.32; envelope-from=dongli.zhang@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org QEMU uses the kvm_get_msrs() function to save Intel PMU registers from KVM and kvm_put_msrs() to restore them to KVM. However, there is no support for AMD PMU registers. Currently, has_pmu_version and num_pmu_gp_counters are initialized based on cpuid(0xa), which does not apply to AMD processors. For AMD CPUs, prior to PerfMonV2, the number of general-purpose registers is determined based on the CPU version. To address this issue, we need to add support for AMD PMU registers. Without this support, the following problems can arise: 1. If the VM is reset (e.g., via QEMU system_reset or VM kdump/kexec) while running "perf top", the PMU registers are not disabled properly. 2. Despite x86_cpu_reset() resetting many registers to zero, kvm_put_msrs() does not handle AMD PMU registers, causing some PMU events to remain enabled in KVM. 3. The KVM kvm_pmc_speculative_in_use() function consistently returns true, preventing the reclamation of these events. Consequently, the kvm_pmc->perf_event remains active. 4. After a reboot, the VM kernel may report the following error: [ 0.092011] Performance Events: Fam17h+ core perfctr, Broken BIOS detected, complain to your hardware vendor. [ 0.092023] [Firmware Bug]: the BIOS has corrupted hw-PMU resources (MSR c0010200 is 530076) 5. In the worst case, the active kvm_pmc->perf_event may inject unknown NMIs randomly into the VM kernel: [...] Uhhuh. NMI received for unknown reason 30 on CPU 0. To resolve these issues, we propose resetting AMD PMU registers during the VM reset process. Signed-off-by: Dongli Zhang --- target/i386/cpu.h | 8 +++ target/i386/kvm/kvm.c | 156 +++++++++++++++++++++++++++++++++++++++++- 2 files changed, 161 insertions(+), 3 deletions(-) diff --git a/target/i386/cpu.h b/target/i386/cpu.h index 59959b8b7a..0505eb3b08 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -488,6 +488,14 @@ typedef enum X86Seg { #define MSR_CORE_PERF_GLOBAL_CTRL 0x38f #define MSR_CORE_PERF_GLOBAL_OVF_CTRL 0x390 +#define MSR_K7_EVNTSEL0 0xc0010000 +#define MSR_K7_PERFCTR0 0xc0010004 +#define MSR_F15H_PERF_CTL0 0xc0010200 +#define MSR_F15H_PERF_CTR0 0xc0010201 + +#define AMD64_NUM_COUNTERS 4 +#define AMD64_NUM_COUNTERS_CORE 6 + #define MSR_MC0_CTL 0x400 #define MSR_MC0_STATUS 0x401 #define MSR_MC0_ADDR 0x402 diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c index ca2b644e2c..83ec85a9b9 100644 --- a/target/i386/kvm/kvm.c +++ b/target/i386/kvm/kvm.c @@ -2035,7 +2035,7 @@ full: abort(); } -static void kvm_init_pmu_info(CPUX86State *env) +static void kvm_init_pmu_info_intel(CPUX86State *env) { uint32_t eax, edx; uint32_t unused; @@ -2072,6 +2072,80 @@ static void kvm_init_pmu_info(CPUX86State *env) } } +static void kvm_init_pmu_info_amd(CPUX86State *env) +{ + int64_t family; + + has_pmu_version = 0; + + /* + * To determine the CPU family, the following code is derived from + * x86_cpuid_version_get_family(). + */ + family = (env->cpuid_version >> 8) & 0xf; + if (family == 0xf) { + family += (env->cpuid_version >> 20) & 0xff; + } + + /* + * Performance-monitoring supported from K7 and later. + */ + if (family < 6) { + return; + } + + has_pmu_version = 1; + + if (!(env->features[FEAT_8000_0001_ECX] & CPUID_EXT3_PERFCORE)) { + num_pmu_gp_counters = AMD64_NUM_COUNTERS; + return; + } + + num_pmu_gp_counters = AMD64_NUM_COUNTERS_CORE; +} + +static bool is_same_vendor(CPUX86State *env) +{ + static uint32_t host_cpuid_vendor1; + static uint32_t host_cpuid_vendor2; + static uint32_t host_cpuid_vendor3; + + host_cpuid(0x0, 0, NULL, &host_cpuid_vendor1, &host_cpuid_vendor3, + &host_cpuid_vendor2); + + return env->cpuid_vendor1 == host_cpuid_vendor1 && + env->cpuid_vendor2 == host_cpuid_vendor2 && + env->cpuid_vendor3 == host_cpuid_vendor3; +} + +static void kvm_init_pmu_info(CPUX86State *env) +{ + /* + * It is not supported to virtualize AMD PMU registers on Intel + * processors, nor to virtualize Intel PMU registers on AMD processors. + */ + if (!is_same_vendor(env)) { + return; + } + + /* + * If KVM_CAP_PMU_CAPABILITY is not supported, there is no way to + * disable the AMD pmu virtualization. + * + * If KVM_CAP_PMU_CAPABILITY is supported, kvm_state->pmu_cap_disabled + * indicates the KVM has already disabled the pmu virtualization. + */ + if (kvm_state->pmu_cap_disabled) { + return; + } + + if (IS_INTEL_CPU(env)) { + kvm_init_pmu_info_intel(env); + } else if (IS_AMD_CPU(env)) { + kvm_init_pmu_info_amd(env); + } +} + int kvm_arch_init_vcpu(CPUState *cs) { struct { @@ -4027,7 +4101,7 @@ static int kvm_put_msrs(X86CPU *cpu, int level) kvm_msr_entry_add(cpu, MSR_KVM_POLL_CONTROL, env->poll_control_msr); } - if (has_pmu_version > 0) { + if (IS_INTEL_CPU(env) && has_pmu_version > 0) { if (has_pmu_version > 1) { /* Stop the counter. */ kvm_msr_entry_add(cpu, MSR_CORE_PERF_FIXED_CTR_CTRL, 0); @@ -4058,6 +4132,38 @@ static int kvm_put_msrs(X86CPU *cpu, int level) env->msr_global_ctrl); } } + + if (IS_AMD_CPU(env) && has_pmu_version > 0) { + uint32_t sel_base = MSR_K7_EVNTSEL0; + uint32_t ctr_base = MSR_K7_PERFCTR0; + /* + * The address of the next selector or counter register is + * obtained by incrementing the address of the current selector + * or counter register by one. + */ + uint32_t step = 1; + + /* + * When PERFCORE is enabled, AMD PMU uses a separate set of + * addresses for the selector and counter registers. + * Additionally, the address of the next selector or counter + * register is determined by incrementing the address of the + * current register by two. + */ + if (num_pmu_gp_counters == AMD64_NUM_COUNTERS_CORE) { + sel_base = MSR_F15H_PERF_CTL0; + ctr_base = MSR_F15H_PERF_CTR0; + step = 2; + } + + for (i = 0; i < num_pmu_gp_counters; i++) { + kvm_msr_entry_add(cpu, ctr_base + i * step, + env->msr_gp_counters[i]); + kvm_msr_entry_add(cpu, sel_base + i * step, + env->msr_gp_evtsel[i]); + } + } + /* * Hyper-V partition-wide MSRs: to avoid clearing them on cpu hot-add, * only sync them to KVM on the first cpu @@ -4503,7 +4609,8 @@ static int kvm_get_msrs(X86CPU *cpu) if (env->features[FEAT_KVM] & (1 << KVM_FEATURE_POLL_CONTROL)) { kvm_msr_entry_add(cpu, MSR_KVM_POLL_CONTROL, 1); } - if (has_pmu_version > 0) { + + if (IS_INTEL_CPU(env) && has_pmu_version > 0) { if (has_pmu_version > 1) { kvm_msr_entry_add(cpu, MSR_CORE_PERF_FIXED_CTR_CTRL, 0); kvm_msr_entry_add(cpu, MSR_CORE_PERF_GLOBAL_CTRL, 0); @@ -4519,6 +4626,35 @@ static int kvm_get_msrs(X86CPU *cpu) } } + if (IS_AMD_CPU(env) && has_pmu_version > 0) { + uint32_t sel_base = MSR_K7_EVNTSEL0; + uint32_t ctr_base = MSR_K7_PERFCTR0; + /* + * The address of the next selector or counter register is + * obtained by incrementing the address of the current selector + * or counter register by one. + */ + uint32_t step = 1; + + /* + * When PERFCORE is enabled, AMD PMU uses a separate set of + * addresses for the selector and counter registers. + * Additionally, the address of the next selector or counter + * register is determined by incrementing the address of the + * current register by two. + */ + if (num_pmu_gp_counters == AMD64_NUM_COUNTERS_CORE) { + sel_base = MSR_F15H_PERF_CTL0; + ctr_base = MSR_F15H_PERF_CTR0; + step = 2; + } + + for (i = 0; i < num_pmu_gp_counters; i++) { + kvm_msr_entry_add(cpu, ctr_base + i * step, 0); + kvm_msr_entry_add(cpu, sel_base + i * step, 0); + } + } + if (env->mcg_cap) { kvm_msr_entry_add(cpu, MSR_MCG_STATUS, 0); kvm_msr_entry_add(cpu, MSR_MCG_CTL, 0); @@ -4830,6 +4966,20 @@ static int kvm_get_msrs(X86CPU *cpu) case MSR_P6_EVNTSEL0 ... MSR_P6_EVNTSEL0 + MAX_GP_COUNTERS - 1: env->msr_gp_evtsel[index - MSR_P6_EVNTSEL0] = msrs[i].data; break; + case MSR_K7_EVNTSEL0 ... MSR_K7_EVNTSEL0 + 3: + env->msr_gp_evtsel[index - MSR_K7_EVNTSEL0] = msrs[i].data; + break; + case MSR_K7_PERFCTR0 ... MSR_K7_PERFCTR0 + 3: + env->msr_gp_counters[index - MSR_K7_PERFCTR0] = msrs[i].data; + break; + case MSR_F15H_PERF_CTL0 ... MSR_F15H_PERF_CTL0 + 0xb: + index = index - MSR_F15H_PERF_CTL0; + if (index & 0x1) { + env->msr_gp_counters[index] = msrs[i].data; + } else { + env->msr_gp_evtsel[index] = msrs[i].data; + } + break; case HV_X64_MSR_HYPERCALL: env->msr_hv_hypercall = msrs[i].data; break; From patchwork Mon Nov 4 09:40:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dongli Zhang X-Patchwork-Id: 13861140 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 74340D132A3 for ; Mon, 4 Nov 2024 09:43:17 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t7tbR-0004nz-OC; Mon, 04 Nov 2024 04:42:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t7tbJ-0004gm-Sl for qemu-devel@nongnu.org; Mon, 04 Nov 2024 04:42:22 -0500 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t7tbH-0000nd-7I for qemu-devel@nongnu.org; Mon, 04 Nov 2024 04:42:20 -0500 Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A48feVg012731; Mon, 4 Nov 2024 09:42:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=corp-2023-11-20; bh=iByb0 xSBywgeMzBO1BPHrpu5g4pFiXx4tglO6zRirAE=; b=d4AdwlHNEss5xFlavlNK3 wXUowNkTIInE0Le83Wpgd/XDY9pwNf72p5uLncYOUh9a2+BhGG3gAanYaCDHeH+/ PIKmVKMzFdP4cVkFX3IZmlQiyqCL5oiel8woZVLq4KMGJYmhBk7JPXsR5hI/nMMR QQiCn1QyWtVHWuP5dldTefFiceevvOKbhPD7kvyYL9uOiYp+LRAdSqqLaQcR66f5 rSla0MNDgfsphrxLspmVQQgE3TPO+5gGBRSncyvqWCfjXjaDzoffkl+pl67T3jHn T3f9BD40WW2iVy9v3E0A2r7+wMsmoQmHmYasm2cJPE4VAD5ld4vxt5Zan1H0pkXb w== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42nanytbxt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 Nov 2024 09:42:05 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A47XMQq009856; Mon, 4 Nov 2024 09:42:04 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42nahbt0ep-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 Nov 2024 09:42:03 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A49fqfo018519; Mon, 4 Nov 2024 09:42:03 GMT Received: from localhost.localdomain (ca-dev80.us.oracle.com [10.211.9.80]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 42nahbt06k-7; Mon, 04 Nov 2024 09:42:03 +0000 From: Dongli Zhang To: qemu-devel@nongnu.org, kvm@vger.kernel.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, sandipan.das@amd.com, babu.moger@amd.com, zhao1.liu@intel.com, likexu@tencent.com, like.xu.linux@gmail.com, zhenyuw@linux.intel.com, groug@kaod.org, lyan@digitalocean.com, khorenko@virtuozzo.com, alexander.ivanov@virtuozzo.com, den@virtuozzo.com, joe.jin@oracle.com, davydov-max@yandex-team.ru Subject: [PATCH 6/7] target/i386/kvm: support perfmon-v2 for reset Date: Mon, 4 Nov 2024 01:40:21 -0800 Message-ID: <20241104094119.4131-7-dongli.zhang@oracle.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241104094119.4131-1-dongli.zhang@oracle.com> References: <20241104094119.4131-1-dongli.zhang@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-04_07,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 suspectscore=0 bulkscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411040085 X-Proofpoint-ORIG-GUID: nmP97gL8Vk16FKAIIa4yfRAMlu4iMFBe X-Proofpoint-GUID: nmP97gL8Vk16FKAIIa4yfRAMlu4iMFBe Received-SPF: pass client-ip=205.220.165.32; envelope-from=dongli.zhang@oracle.com; helo=mx0a-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Since perfmon-v2, the AMD PMU supports additional registers. This update includes get/put functionality for these extra registers. Similar to the implementation in KVM: - MSR_CORE_PERF_GLOBAL_STATUS and MSR_AMD64_PERF_CNTR_GLOBAL_STATUS both use env->msr_global_status. - MSR_CORE_PERF_GLOBAL_CTRL and MSR_AMD64_PERF_CNTR_GLOBAL_CTL both use env->msr_global_ctrl. - MSR_CORE_PERF_GLOBAL_OVF_CTRL and MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR both use env->msr_global_ovf_ctrl. No changes are needed for vmstate_msr_architectural_pmu or pmu_enable_needed(). Signed-off-by: Dongli Zhang --- target/i386/cpu.h | 4 ++++ target/i386/kvm/kvm.c | 47 ++++++++++++++++++++++++++++++++++--------- 2 files changed, 42 insertions(+), 9 deletions(-) diff --git a/target/i386/cpu.h b/target/i386/cpu.h index 0505eb3b08..68ed798808 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -488,6 +488,10 @@ typedef enum X86Seg { #define MSR_CORE_PERF_GLOBAL_CTRL 0x38f #define MSR_CORE_PERF_GLOBAL_OVF_CTRL 0x390 +#define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS 0xc0000300 +#define MSR_AMD64_PERF_CNTR_GLOBAL_CTL 0xc0000301 +#define MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR 0xc0000302 + #define MSR_K7_EVNTSEL0 0xc0010000 #define MSR_K7_PERFCTR0 0xc0010004 #define MSR_F15H_PERF_CTL0 0xc0010200 diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c index 83ec85a9b9..918dcb61fe 100644 --- a/target/i386/kvm/kvm.c +++ b/target/i386/kvm/kvm.c @@ -2074,6 +2074,8 @@ static void kvm_init_pmu_info_intel(CPUX86State *env) static void kvm_init_pmu_info_amd(CPUX86State *env) { + uint32_t eax, ebx; + uint32_t unused; int64_t family; has_pmu_version = 0; @@ -2102,6 +2104,13 @@ static void kvm_init_pmu_info_amd(CPUX86State *env) } num_pmu_gp_counters = AMD64_NUM_COUNTERS_CORE; + + cpu_x86_cpuid(env, 0x80000022, 0, &eax, &ebx, &unused, &unused); + + if (eax & CPUID_8000_0022_EAX_PERFMON_V2) { + has_pmu_version = 2; + num_pmu_gp_counters = ebx & 0xf; + } } static bool is_same_vendor(CPUX86State *env) @@ -4144,13 +4153,14 @@ static int kvm_put_msrs(X86CPU *cpu, int level) uint32_t step = 1; /* - * When PERFCORE is enabled, AMD PMU uses a separate set of - * addresses for the selector and counter registers. - * Additionally, the address of the next selector or counter - * register is determined by incrementing the address of the - * current register by two. + * When PERFCORE or PerfMonV2 is enabled, AMD PMU uses a + * separate set of addresses for the selector and counter + * registers. Additionally, the address of the next selector or + * counter register is determined by incrementing the address + * of the current register by two. */ - if (num_pmu_gp_counters == AMD64_NUM_COUNTERS_CORE) { + if (num_pmu_gp_counters == AMD64_NUM_COUNTERS_CORE || + has_pmu_version == 2) { sel_base = MSR_F15H_PERF_CTL0; ctr_base = MSR_F15H_PERF_CTR0; step = 2; @@ -4162,6 +4172,15 @@ static int kvm_put_msrs(X86CPU *cpu, int level) kvm_msr_entry_add(cpu, sel_base + i * step, env->msr_gp_evtsel[i]); } + + if (has_pmu_version == 2) { + kvm_msr_entry_add(cpu, MSR_AMD64_PERF_CNTR_GLOBAL_STATUS, + env->msr_global_status); + kvm_msr_entry_add(cpu, MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR, + env->msr_global_ovf_ctrl); + kvm_msr_entry_add(cpu, MSR_AMD64_PERF_CNTR_GLOBAL_CTL, + env->msr_global_ctrl); + } } /* @@ -4637,13 +4656,14 @@ static int kvm_get_msrs(X86CPU *cpu) uint32_t step = 1; /* - * When PERFCORE is enabled, AMD PMU uses a separate set of - * addresses for the selector and counter registers. + * When PERFCORE or PerfMonV2 is enabled, AMD PMU uses a separate + * set of addresses for the selector and counter registers. * Additionally, the address of the next selector or counter * register is determined by incrementing the address of the * current register by two. */ - if (num_pmu_gp_counters == AMD64_NUM_COUNTERS_CORE) { + if (num_pmu_gp_counters == AMD64_NUM_COUNTERS_CORE || + has_pmu_version == 2) { sel_base = MSR_F15H_PERF_CTL0; ctr_base = MSR_F15H_PERF_CTR0; step = 2; @@ -4653,6 +4673,12 @@ static int kvm_get_msrs(X86CPU *cpu) kvm_msr_entry_add(cpu, ctr_base + i * step, 0); kvm_msr_entry_add(cpu, sel_base + i * step, 0); } + + if (has_pmu_version == 2) { + kvm_msr_entry_add(cpu, MSR_AMD64_PERF_CNTR_GLOBAL_CTL, 0); + kvm_msr_entry_add(cpu, MSR_AMD64_PERF_CNTR_GLOBAL_STATUS, 0); + kvm_msr_entry_add(cpu, MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR, 0); + } } if (env->mcg_cap) { @@ -4949,12 +4975,15 @@ static int kvm_get_msrs(X86CPU *cpu) env->msr_fixed_ctr_ctrl = msrs[i].data; break; case MSR_CORE_PERF_GLOBAL_CTRL: + case MSR_AMD64_PERF_CNTR_GLOBAL_CTL: env->msr_global_ctrl = msrs[i].data; break; case MSR_CORE_PERF_GLOBAL_STATUS: + case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS: env->msr_global_status = msrs[i].data; break; case MSR_CORE_PERF_GLOBAL_OVF_CTRL: + case MSR_AMD64_PERF_CNTR_GLOBAL_STATUS_CLR: env->msr_global_ovf_ctrl = msrs[i].data; break; case MSR_CORE_PERF_FIXED_CTR0 ... MSR_CORE_PERF_FIXED_CTR0 + MAX_FIXED_COUNTERS - 1: From patchwork Mon Nov 4 09:40:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dongli Zhang X-Patchwork-Id: 13861139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1F14BD132A3 for ; Mon, 4 Nov 2024 09:43:14 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t7tbS-0004qt-Sq; Mon, 04 Nov 2024 04:42:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t7tbJ-0004gl-Sl for qemu-devel@nongnu.org; Mon, 04 Nov 2024 04:42:22 -0500 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t7tbH-0000nu-AP for qemu-devel@nongnu.org; Mon, 04 Nov 2024 04:42:20 -0500 Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4A48fdl3008884; Mon, 4 Nov 2024 09:42:06 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=corp-2023-11-20; bh=Mtp7c F2I5l+mbvT9VpcvvHwkRan/n1Y5vkihKlaMSj8=; b=eO4YEgkTJt/vUl+KFvawb Zqn141sWKxt2CWtY6q/lhve6vKPipqDgRVEVAwhY1CiRmhELErXnHWuUa8rltHrl RwUprTlhNgpeSmewqV1JCkPHYv2WVFvqZF1mdIb+rUW7rDDKbysPLf66qOkqqGU9 M83qxVLEmRBijKpz3+RojPsDKS+qpnqfv71uMIvZ93PzKzHgwX2toBbge3VX1W5j KbjFRIYpDVCVgG8r+gdXCkUAjhpg1gIIi3JCc2FTGSJSH4x0vkhyRsWR9nPh1uhv kZVzI+PH5cpI2rHvH/7hFdbvS+PDOGQtOzjh6NMwaShHNhU9U2x//Z6b2okWIKGL g== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42nbpsj991-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 Nov 2024 09:42:06 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4A48xrOO009864; Mon, 4 Nov 2024 09:42:05 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42nahbt0fh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 04 Nov 2024 09:42:05 +0000 Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 4A49fqfq018519; Mon, 4 Nov 2024 09:42:05 GMT Received: from localhost.localdomain (ca-dev80.us.oracle.com [10.211.9.80]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 42nahbt06k-8; Mon, 04 Nov 2024 09:42:04 +0000 From: Dongli Zhang To: qemu-devel@nongnu.org, kvm@vger.kernel.org Cc: pbonzini@redhat.com, mtosatti@redhat.com, sandipan.das@amd.com, babu.moger@amd.com, zhao1.liu@intel.com, likexu@tencent.com, like.xu.linux@gmail.com, zhenyuw@linux.intel.com, groug@kaod.org, lyan@digitalocean.com, khorenko@virtuozzo.com, alexander.ivanov@virtuozzo.com, den@virtuozzo.com, joe.jin@oracle.com, davydov-max@yandex-team.ru Subject: [PATCH 7/7] target/i386/kvm: don't stop Intel PMU counters Date: Mon, 4 Nov 2024 01:40:22 -0800 Message-ID: <20241104094119.4131-8-dongli.zhang@oracle.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241104094119.4131-1-dongli.zhang@oracle.com> References: <20241104094119.4131-1-dongli.zhang@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-11-04_07,2024-11-01_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 suspectscore=0 bulkscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2411040085 X-Proofpoint-ORIG-GUID: 6e8ieF02pb_kFg_RZwb-GTQwlJliHIY0 X-Proofpoint-GUID: 6e8ieF02pb_kFg_RZwb-GTQwlJliHIY0 Received-SPF: pass client-ip=205.220.177.32; envelope-from=dongli.zhang@oracle.com; helo=mx0b-00069f02.pphosted.com X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The kvm_put_msrs() sets the MSRs using KVM_SET_MSRS. The x86 KVM processes these MSRs one by one in a loop, only saving the config and triggering the KVM_REQ_PMU request. This approach does not immediately stop the event before updating PMC. In additional, PMU MSRs are set only at levels >= KVM_PUT_RESET_STATE, excluding runtime. Therefore, updating these MSRs without stopping events should be acceptable. Finally, KVM creates kernel perf events with host mode excluded (exclude_host = 1). While the events remain active, they don't increment the counter during QEMU vCPU userspace mode. No Fixed tag is going to be added for the commit 0d89436786b0 ("kvm: migrate vPMU state"), because this isn't a bugfix. Signed-off-by: Dongli Zhang --- target/i386/kvm/kvm.c | 9 --------- 1 file changed, 9 deletions(-) diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c index 918dcb61fe..c4a11c2e80 100644 --- a/target/i386/kvm/kvm.c +++ b/target/i386/kvm/kvm.c @@ -4111,13 +4111,6 @@ static int kvm_put_msrs(X86CPU *cpu, int level) } if (IS_INTEL_CPU(env) && has_pmu_version > 0) { - if (has_pmu_version > 1) { - /* Stop the counter. */ - kvm_msr_entry_add(cpu, MSR_CORE_PERF_FIXED_CTR_CTRL, 0); - kvm_msr_entry_add(cpu, MSR_CORE_PERF_GLOBAL_CTRL, 0); - } - - /* Set the counter values. */ for (i = 0; i < num_pmu_fixed_counters; i++) { kvm_msr_entry_add(cpu, MSR_CORE_PERF_FIXED_CTR0 + i, env->msr_fixed_counters[i]); @@ -4133,8 +4126,6 @@ static int kvm_put_msrs(X86CPU *cpu, int level) env->msr_global_status); kvm_msr_entry_add(cpu, MSR_CORE_PERF_GLOBAL_OVF_CTRL, env->msr_global_ovf_ctrl); - - /* Now start the PMU. */ kvm_msr_entry_add(cpu, MSR_CORE_PERF_FIXED_CTR_CTRL, env->msr_fixed_ctr_ctrl); kvm_msr_entry_add(cpu, MSR_CORE_PERF_GLOBAL_CTRL,