From patchwork Wed Apr 20 10:49:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Shan X-Patchwork-Id: 12820077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 30127C433F5 for ; Wed, 20 Apr 2022 11:18:38 +0000 (UTC) Received: from localhost ([::1]:53018 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nh8M1-0001SK-7A for qemu-devel@archiver.kernel.org; Wed, 20 Apr 2022 07:18:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33730) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nh7wN-00058V-9V for qemu-devel@nongnu.org; Wed, 20 Apr 2022 06:52:07 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:24256) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nh7wL-0001KS-31 for qemu-devel@nongnu.org; Wed, 20 Apr 2022 06:52:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1650451922; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GZUKCpp7xxGagrGtExZTsFbrYVMPiRchl0pqtU9wEI0=; b=Ak8b1NAJT5AgtnVQHYAfdgB6OVxPxxmsJJOmfdQVvak9SB4/VpDccKof08nUWz6a6JeOkQ 5Xa2+mPcHmMQkHchMR4YTjwU/W7IrHbO9wn+UN9YMIAQkJv/Ci85DGLiP4aNBJqOevflXq iJ+MySZlIYWwe6p+BRMb/TrYVqf8/Pc= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-475-nUODevN2PxS5JzuvUcXqVA-1; Wed, 20 Apr 2022 06:51:58 -0400 X-MC-Unique: nUODevN2PxS5JzuvUcXqVA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 326A129AA3B1; Wed, 20 Apr 2022 10:51:58 +0000 (UTC) Received: from gshan.redhat.com (ovpn-13-230.pek2.redhat.com [10.72.13.230]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9C8024047293; Wed, 20 Apr 2022 10:51:49 +0000 (UTC) From: Gavin Shan To: qemu-arm@nongnu.org Subject: [PATCH v7 1/4] qapi/machine.json: Add cluster-id Date: Wed, 20 Apr 2022 18:49:06 +0800 Message-Id: <20220420104909.233058-2-gshan@redhat.com> In-Reply-To: <20220420104909.233058-1-gshan@redhat.com> References: <20220420104909.233058-1-gshan@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, eduardo@habkost.net, thuth@redhat.com, berrange@redhat.com, shan.gavin@gmail.com, peter.maydell@linaro.org, Jonathan.Cameron@Huawei.com, zhenyzha@redhat.com, mst@redhat.com, qemu-devel@nongnu.org, armbru@redhat.com, wangyanan55@huawei.com, imammedo@redhat.com, ani@anisinha.ca, pbonzini@redhat.com, drjones@redhat.com, eblake@redhat.com, f4bug@amsat.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" This adds cluster-id in CPU instance properties, which will be used by arm/virt machine. Besides, the cluster-id is also verified or dumped in various spots: * hw/core/machine.c::machine_set_cpu_numa_node() to associate CPU with its NUMA node. * hw/core/machine.c::machine_numa_finish_cpu_init() to associate CPU with NUMA node when no default association isn't provided. * hw/core/machine-hmp-cmds.c::hmp_hotpluggable_cpus() to dump cluster-id. Signed-off-by: Gavin Shan Reviewed-by: Yanan Wang --- hw/core/machine-hmp-cmds.c | 4 ++++ hw/core/machine.c | 16 ++++++++++++++++ qapi/machine.json | 6 ++++-- 3 files changed, 24 insertions(+), 2 deletions(-) diff --git a/hw/core/machine-hmp-cmds.c b/hw/core/machine-hmp-cmds.c index 4e2f319aeb..5cb5eecbfc 100644 --- a/hw/core/machine-hmp-cmds.c +++ b/hw/core/machine-hmp-cmds.c @@ -77,6 +77,10 @@ void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict) if (c->has_die_id) { monitor_printf(mon, " die-id: \"%" PRIu64 "\"\n", c->die_id); } + if (c->has_cluster_id) { + monitor_printf(mon, " cluster-id: \"%" PRIu64 "\"\n", + c->cluster_id); + } if (c->has_core_id) { monitor_printf(mon, " core-id: \"%" PRIu64 "\"\n", c->core_id); } diff --git a/hw/core/machine.c b/hw/core/machine.c index 1e23fdc14b..ac91dfd834 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -679,6 +679,11 @@ void machine_set_cpu_numa_node(MachineState *machine, return; } + if (props->has_cluster_id && !slot->props.has_cluster_id) { + error_setg(errp, "cluster-id is not supported"); + return; + } + if (props->has_socket_id && !slot->props.has_socket_id) { error_setg(errp, "socket-id is not supported"); return; @@ -698,6 +703,11 @@ void machine_set_cpu_numa_node(MachineState *machine, continue; } + if (props->has_cluster_id && + props->cluster_id != slot->props.cluster_id) { + continue; + } + if (props->has_die_id && props->die_id != slot->props.die_id) { continue; } @@ -992,6 +1002,12 @@ static char *cpu_slot_to_string(const CPUArchId *cpu) } g_string_append_printf(s, "die-id: %"PRId64, cpu->props.die_id); } + if (cpu->props.has_cluster_id) { + if (s->len) { + g_string_append_printf(s, ", "); + } + g_string_append_printf(s, "cluster-id: %"PRId64, cpu->props.cluster_id); + } if (cpu->props.has_core_id) { if (s->len) { g_string_append_printf(s, ", "); diff --git a/qapi/machine.json b/qapi/machine.json index d25a481ce4..4c417e32a5 100644 --- a/qapi/machine.json +++ b/qapi/machine.json @@ -868,10 +868,11 @@ # @node-id: NUMA node ID the CPU belongs to # @socket-id: socket number within node/board the CPU belongs to # @die-id: die number within socket the CPU belongs to (since 4.1) -# @core-id: core number within die the CPU belongs to +# @cluster-id: cluster number within die the CPU belongs to (since 7.1) +# @core-id: core number within cluster the CPU belongs to # @thread-id: thread number within core the CPU belongs to # -# Note: currently there are 5 properties that could be present +# Note: currently there are 6 properties that could be present # but management should be prepared to pass through other # properties with device_add command to allow for future # interface extension. This also requires the filed names to be kept in @@ -883,6 +884,7 @@ 'data': { '*node-id': 'int', '*socket-id': 'int', '*die-id': 'int', + '*cluster-id': 'int', '*core-id': 'int', '*thread-id': 'int' } From patchwork Wed Apr 20 10:49:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Shan X-Patchwork-Id: 12820061 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 14110C433F5 for ; Wed, 20 Apr 2022 10:53:58 +0000 (UTC) Received: from localhost ([::1]:38172 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nh7y9-0007e4-73 for qemu-devel@archiver.kernel.org; Wed, 20 Apr 2022 06:53:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33770) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nh7wV-0005OJ-50 for qemu-devel@nongnu.org; Wed, 20 Apr 2022 06:52:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:57484) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nh7wT-0001LV-8f for qemu-devel@nongnu.org; Wed, 20 Apr 2022 06:52:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1650451932; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5pl/2vowPkUpsJ6JfH5IYPAqiwdXxCj0iCkTfGRGjl4=; b=PwGTWIJKbhdtaOfsfNRcAw5cT9n6u9/E9OVC1Rmzy2SdNqtom85FY7OxgAOCngtoNutwsW VvfqYv8u8PP/5JgRKD4h/sdA3WBaVW3WkhFonookkgREdar8AKuSe7V8e4k7ndicVgHIOu lsI/jDGJPUEUljt8C+jw5/1yOKIAwFU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-330-kDmi8u0CNzSW9MN4fBWpmg-1; Wed, 20 Apr 2022 06:52:08 -0400 X-MC-Unique: kDmi8u0CNzSW9MN4fBWpmg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A5DA5101AA42; Wed, 20 Apr 2022 10:52:07 +0000 (UTC) Received: from gshan.redhat.com (ovpn-13-230.pek2.redhat.com [10.72.13.230]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0251F40D1B9B; Wed, 20 Apr 2022 10:51:58 +0000 (UTC) From: Gavin Shan To: qemu-arm@nongnu.org Subject: [PATCH v7 2/4] hw/arm/virt: Consider SMP configuration in CPU topology Date: Wed, 20 Apr 2022 18:49:07 +0800 Message-Id: <20220420104909.233058-3-gshan@redhat.com> In-Reply-To: <20220420104909.233058-1-gshan@redhat.com> References: <20220420104909.233058-1-gshan@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, eduardo@habkost.net, thuth@redhat.com, berrange@redhat.com, shan.gavin@gmail.com, peter.maydell@linaro.org, Jonathan.Cameron@Huawei.com, zhenyzha@redhat.com, mst@redhat.com, qemu-devel@nongnu.org, armbru@redhat.com, wangyanan55@huawei.com, imammedo@redhat.com, ani@anisinha.ca, pbonzini@redhat.com, drjones@redhat.com, eblake@redhat.com, f4bug@amsat.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Currently, the SMP configuration isn't considered when the CPU topology is populated. In this case, it's impossible to provide the default CPU-to-NUMA mapping or association based on the socket ID of the given CPU. This takes account of SMP configuration when the CPU topology is populated. The die ID for the given CPU isn't assigned since it's not supported on arm/virt machine. Besides, the used SMP configuration in qtest/numa-test/aarch64_numa_cpu() is corrcted to avoid testing failure Signed-off-by: Gavin Shan Reviewed-by: Yanan Wang --- hw/arm/virt.c | 15 ++++++++++++++- tests/qtest/numa-test.c | 3 ++- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/hw/arm/virt.c b/hw/arm/virt.c index d2e5ecd234..5443ecae92 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -2505,6 +2505,7 @@ static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms) int n; unsigned int max_cpus = ms->smp.max_cpus; VirtMachineState *vms = VIRT_MACHINE(ms); + MachineClass *mc = MACHINE_GET_CLASS(vms); if (ms->possible_cpus) { assert(ms->possible_cpus->len == max_cpus); @@ -2518,8 +2519,20 @@ static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms) ms->possible_cpus->cpus[n].type = ms->cpu_type; ms->possible_cpus->cpus[n].arch_id = virt_cpu_mp_affinity(vms, n); + + assert(!mc->smp_props.dies_supported); + ms->possible_cpus->cpus[n].props.has_socket_id = true; + ms->possible_cpus->cpus[n].props.socket_id = + (n / (ms->smp.clusters * ms->smp.cores * ms->smp.threads)); + ms->possible_cpus->cpus[n].props.has_cluster_id = true; + ms->possible_cpus->cpus[n].props.cluster_id = + (n / (ms->smp.cores * ms->smp.threads)) % ms->smp.clusters; + ms->possible_cpus->cpus[n].props.has_core_id = true; + ms->possible_cpus->cpus[n].props.core_id = + (n / ms->smp.threads) % ms->smp.cores; ms->possible_cpus->cpus[n].props.has_thread_id = true; - ms->possible_cpus->cpus[n].props.thread_id = n; + ms->possible_cpus->cpus[n].props.thread_id = + n % ms->smp.threads; } return ms->possible_cpus; } diff --git a/tests/qtest/numa-test.c b/tests/qtest/numa-test.c index 90bf68a5b3..aeda8c774c 100644 --- a/tests/qtest/numa-test.c +++ b/tests/qtest/numa-test.c @@ -223,7 +223,8 @@ static void aarch64_numa_cpu(const void *data) QTestState *qts; g_autofree char *cli = NULL; - cli = make_cli(data, "-machine smp.cpus=2 " + cli = make_cli(data, "-machine " + "smp.cpus=2,smp.sockets=1,smp.clusters=1,smp.cores=1,smp.threads=2 " "-numa node,nodeid=0,memdev=ram -numa node,nodeid=1 " "-numa cpu,node-id=1,thread-id=0 " "-numa cpu,node-id=0,thread-id=1"); From patchwork Wed Apr 20 10:49:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Shan X-Patchwork-Id: 12820078 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0D83FC433EF for ; Wed, 20 Apr 2022 11:20:59 +0000 (UTC) Received: from localhost ([::1]:58852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nh8OI-0004zE-3N for qemu-devel@archiver.kernel.org; Wed, 20 Apr 2022 07:20:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33798) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nh7we-0005Uc-BV for qemu-devel@nongnu.org; Wed, 20 Apr 2022 06:52:24 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:35096) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nh7wb-0001Lp-If for qemu-devel@nongnu.org; Wed, 20 Apr 2022 06:52:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1650451941; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qfJEE3/IG/WPlZJ9pG2XcS77Ozmlz/hB9OcChPelWtg=; b=aLTDhJesNUDct9xJ+nsPh6CJba+GL+hVKen+ZC6d2iw16cH9SEOpP96A2CoakL6+LWzG/t 1sfBsOk1HOSl799Jx8mJ90iKkMVXaSvVGrLyKQX8Ccn3Q6LJ101c4yM1pxn0/j4aFhdXaD Dx7B3vpDDdepR7SzvLPIjXjNruPMrJg= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-546-X91ww2HNNbm-H-edAX1goQ-1; Wed, 20 Apr 2022 06:52:17 -0400 X-MC-Unique: X91ww2HNNbm-H-edAX1goQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 24E0286B8A3; Wed, 20 Apr 2022 10:52:17 +0000 (UTC) Received: from gshan.redhat.com (ovpn-13-230.pek2.redhat.com [10.72.13.230]) by smtp.corp.redhat.com (Postfix) with ESMTP id 74AA940D1B9B; Wed, 20 Apr 2022 10:52:08 +0000 (UTC) From: Gavin Shan To: qemu-arm@nongnu.org Subject: [PATCH v7 3/4] hw/arm/virt: Fix CPU's default NUMA node ID Date: Wed, 20 Apr 2022 18:49:08 +0800 Message-Id: <20220420104909.233058-4-gshan@redhat.com> In-Reply-To: <20220420104909.233058-1-gshan@redhat.com> References: <20220420104909.233058-1-gshan@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, eduardo@habkost.net, thuth@redhat.com, berrange@redhat.com, shan.gavin@gmail.com, peter.maydell@linaro.org, Jonathan.Cameron@Huawei.com, zhenyzha@redhat.com, mst@redhat.com, qemu-devel@nongnu.org, armbru@redhat.com, wangyanan55@huawei.com, imammedo@redhat.com, ani@anisinha.ca, pbonzini@redhat.com, drjones@redhat.com, eblake@redhat.com, f4bug@amsat.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" When CPU-to-NUMA association isn't explicitly provided by users, the default one is given by mc->get_default_cpu_node_id(). However, the CPU topology isn't fully considered in the default association and this causes CPU topology broken warnings on booting Linux guest. For example, the following warning messages are observed when the Linux guest is booted with the following command lines. /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64 \ -accel kvm -machine virt,gic-version=host \ -cpu host \ -smp 6,sockets=2,cores=3,threads=1 \ -m 1024M,slots=16,maxmem=64G \ -object memory-backend-ram,id=mem0,size=128M \ -object memory-backend-ram,id=mem1,size=128M \ -object memory-backend-ram,id=mem2,size=128M \ -object memory-backend-ram,id=mem3,size=128M \ -object memory-backend-ram,id=mem4,size=128M \ -object memory-backend-ram,id=mem4,size=384M \ -numa node,nodeid=0,memdev=mem0 \ -numa node,nodeid=1,memdev=mem1 \ -numa node,nodeid=2,memdev=mem2 \ -numa node,nodeid=3,memdev=mem3 \ -numa node,nodeid=4,memdev=mem4 \ -numa node,nodeid=5,memdev=mem5 : alternatives: patching kernel code BUG: arch topology borken the CLS domain not a subset of the MC domain BUG: arch topology borken the DIE domain not a subset of the NODE domain With current implementation of mc->get_default_cpu_node_id(), CPU#0 to CPU#5 are associated with NODE#0 to NODE#5 separately. That's incorrect because CPU#0/1/2 should be associated with same NUMA node because they're seated in same socket. This fixes the issue by considering the socket ID when the default CPU-to-NUMA association is provided in virt_possible_cpu_arch_ids(). With this applied, no more CPU topology broken warnings are seen from the Linux guest. The 6 CPUs are associated with NODE#0/1, but there are no CPUs associated with NODE#2/3/4/5. Signed-off-by: Gavin Shan Reviewed-by: Igor Mammedov Reviewed-by: Yanan Wang --- hw/arm/virt.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 5443ecae92..c3477a8a17 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -2497,7 +2497,9 @@ virt_cpu_index_to_props(MachineState *ms, unsigned cpu_index) static int64_t virt_get_default_cpu_node_id(const MachineState *ms, int idx) { - return idx % ms->numa_state->num_nodes; + int64_t socket_id = ms->possible_cpus->cpus[idx].props.socket_id; + + return socket_id % ms->numa_state->num_nodes; } static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms) From patchwork Wed Apr 20 10:49:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Shan X-Patchwork-Id: 12820066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E8863C433EF for ; Wed, 20 Apr 2022 11:06:39 +0000 (UTC) Received: from localhost ([::1]:33342 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nh8AR-0005Kz-0G for qemu-devel@archiver.kernel.org; Wed, 20 Apr 2022 07:06:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:33828) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nh7wn-0005i0-Vp for qemu-devel@nongnu.org; Wed, 20 Apr 2022 06:52:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:22005) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nh7wl-0001NU-9N for qemu-devel@nongnu.org; Wed, 20 Apr 2022 06:52:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1650451950; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YBDf492ElLhkh/EEmJKRz3bQzpMcRsw7lDRtLxNhPJ8=; b=FGdz4205+h1aPZrzDJoxQobqzkOO+J59JWHJkhBMxIdQGywaoXJgt11jobPvONwncnboS4 8ELUcY3rJpCvx5Q8I9XvfBETZgq3usqipeeiIf0RXng8QgufbCon6LIw73OEAyfEnS2shm qQqRpiVDxfetYaTksKea3GoXa5SgRw0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-141-GTvjL7I7Ow232l3ZGXFn5Q-1; Wed, 20 Apr 2022 06:52:27 -0400 X-MC-Unique: GTvjL7I7Ow232l3ZGXFn5Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D703F802C16; Wed, 20 Apr 2022 10:52:26 +0000 (UTC) Received: from gshan.redhat.com (ovpn-13-230.pek2.redhat.com [10.72.13.230]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2AAC840470C2; Wed, 20 Apr 2022 10:52:17 +0000 (UTC) From: Gavin Shan To: qemu-arm@nongnu.org Subject: [PATCH v7 4/4] hw/acpi/aml-build: Use existing CPU topology to build PPTT table Date: Wed, 20 Apr 2022 18:49:09 +0800 Message-Id: <20220420104909.233058-5-gshan@redhat.com> In-Reply-To: <20220420104909.233058-1-gshan@redhat.com> References: <20220420104909.233058-1-gshan@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lvivier@redhat.com, eduardo@habkost.net, thuth@redhat.com, berrange@redhat.com, shan.gavin@gmail.com, peter.maydell@linaro.org, Jonathan.Cameron@Huawei.com, zhenyzha@redhat.com, mst@redhat.com, qemu-devel@nongnu.org, armbru@redhat.com, wangyanan55@huawei.com, imammedo@redhat.com, ani@anisinha.ca, pbonzini@redhat.com, drjones@redhat.com, eblake@redhat.com, f4bug@amsat.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" When the PPTT table is built, the CPU topology is re-calculated, but it's unecessary because the CPU topology has been populated in virt_possible_cpu_arch_ids() on arm/virt machine. This reworks build_pptt() to avoid by reusing the existing IDs in ms->possible_cpus. Currently, the only user of build_pptt() is arm/virt machine. Signed-off-by: Gavin Shan Acked-by: Igor Mammedov Tested-by: Yanan Wang Reviewed-by: Yanan Wang --- hw/acpi/aml-build.c | 109 +++++++++++++++++++------------------------- 1 file changed, 47 insertions(+), 62 deletions(-) diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c index 4086879ebf..73f4e69c29 100644 --- a/hw/acpi/aml-build.c +++ b/hw/acpi/aml-build.c @@ -2002,86 +2002,71 @@ void build_pptt(GArray *table_data, BIOSLinker *linker, MachineState *ms, const char *oem_id, const char *oem_table_id) { MachineClass *mc = MACHINE_GET_CLASS(ms); - GQueue *list = g_queue_new(); - guint pptt_start = table_data->len; - guint parent_offset; - guint length, i; - int uid = 0; - int socket; + CPUArchIdList *cpus = ms->possible_cpus; + int64_t socket_id = -1, cluster_id = -1, core_id = -1; + uint32_t socket_offset = 0, cluster_offset = 0, core_offset = 0; + uint32_t pptt_start = table_data->len; + int n; AcpiTable table = { .sig = "PPTT", .rev = 2, .oem_id = oem_id, .oem_table_id = oem_table_id }; acpi_table_begin(&table, table_data); - for (socket = 0; socket < ms->smp.sockets; socket++) { - g_queue_push_tail(list, - GUINT_TO_POINTER(table_data->len - pptt_start)); - build_processor_hierarchy_node( - table_data, - /* - * Physical package - represents the boundary - * of a physical package - */ - (1 << 0), - 0, socket, NULL, 0); - } + /* + * This works with the assumption that cpus[n].props.*_id has been + * sorted from top to down levels in mc->possible_cpu_arch_ids(). + * Otherwise, the unexpected and duplicate containers will be + * created. + */ + for (n = 0; n < cpus->len; n++) { + if (cpus->cpus[n].props.socket_id != socket_id) { + assert(cpus->cpus[n].props.socket_id > socket_id); + socket_id = cpus->cpus[n].props.socket_id; + cluster_id = -1; + core_id = -1; + socket_offset = table_data->len - pptt_start; + build_processor_hierarchy_node(table_data, + (1 << 0), /* Physical package */ + 0, socket_id, NULL, 0); + } - if (mc->smp_props.clusters_supported) { - length = g_queue_get_length(list); - for (i = 0; i < length; i++) { - int cluster; - - parent_offset = GPOINTER_TO_UINT(g_queue_pop_head(list)); - for (cluster = 0; cluster < ms->smp.clusters; cluster++) { - g_queue_push_tail(list, - GUINT_TO_POINTER(table_data->len - pptt_start)); - build_processor_hierarchy_node( - table_data, - (0 << 0), /* not a physical package */ - parent_offset, cluster, NULL, 0); + if (mc->smp_props.clusters_supported) { + if (cpus->cpus[n].props.cluster_id != cluster_id) { + assert(cpus->cpus[n].props.cluster_id > cluster_id); + cluster_id = cpus->cpus[n].props.cluster_id; + core_id = -1; + cluster_offset = table_data->len - pptt_start; + build_processor_hierarchy_node(table_data, + (0 << 0), /* Not a physical package */ + socket_offset, cluster_id, NULL, 0); } + } else { + cluster_offset = socket_offset; } - } - length = g_queue_get_length(list); - for (i = 0; i < length; i++) { - int core; - - parent_offset = GPOINTER_TO_UINT(g_queue_pop_head(list)); - for (core = 0; core < ms->smp.cores; core++) { - if (ms->smp.threads > 1) { - g_queue_push_tail(list, - GUINT_TO_POINTER(table_data->len - pptt_start)); - build_processor_hierarchy_node( - table_data, + if (ms->smp.threads == 1) { + build_processor_hierarchy_node(table_data, + (1 << 1) | /* ACPI Processor ID valid */ + (1 << 3), /* Node is a Leaf */ + cluster_offset, n, NULL, 0); + } else { + if (cpus->cpus[n].props.core_id != core_id) { + assert(cpus->cpus[n].props.core_id > core_id); + core_id = cpus->cpus[n].props.core_id; + core_offset = table_data->len - pptt_start; + build_processor_hierarchy_node(table_data, (0 << 0), /* not a physical package */ - parent_offset, core, NULL, 0); - } else { - build_processor_hierarchy_node( - table_data, - (1 << 1) | /* ACPI Processor ID valid */ - (1 << 3), /* Node is a Leaf */ - parent_offset, uid++, NULL, 0); + cluster_offset, core_id, NULL, 0); } - } - } - - length = g_queue_get_length(list); - for (i = 0; i < length; i++) { - int thread; - parent_offset = GPOINTER_TO_UINT(g_queue_pop_head(list)); - for (thread = 0; thread < ms->smp.threads; thread++) { - build_processor_hierarchy_node( - table_data, + build_processor_hierarchy_node(table_data, (1 << 1) | /* ACPI Processor ID valid */ (1 << 2) | /* Processor is a Thread */ (1 << 3), /* Node is a Leaf */ - parent_offset, uid++, NULL, 0); + core_offset, n, NULL, 0); } } - g_queue_free(list); acpi_table_end(linker, &table); }