From patchwork Tue Oct 12 06:36:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Shan X-Patchwork-Id: 12551559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45A29C433F5 for ; Tue, 12 Oct 2021 06:38:33 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D5B6060E94 for ; Tue, 12 Oct 2021 06:38:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D5B6060E94 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:52968 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1maBQl-0005MG-S4 for qemu-devel@archiver.kernel.org; Tue, 12 Oct 2021 02:38:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34400) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1maBOn-0003Ru-7O for qemu-devel@nongnu.org; Tue, 12 Oct 2021 02:36:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:38995) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1maBOk-0007bB-GO for qemu-devel@nongnu.org; Tue, 12 Oct 2021 02:36:28 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1634020584; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TOl2eQ++G0oJvCl9LgkWNdIW4Qb9p0BWXTN7Dw6BYXQ=; b=NIQhMwcF7G+2qZbvj5j7MOls8ubGqXXa6ZrfUhGhGgp234TtpD0TptqYeotBhS20P+eGcG fVPTQYVCZJs2CV5yJfMH2+BuwPg50rmN2R2jfSLfpB21xhjXWl4FPynGrnUCFYDda8586Y MO3kcz5NVpGgSwr5C6gaKqH6JMB4WQY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-424-J86Cg2SANC2oEbvAA895yg-1; Tue, 12 Oct 2021 02:36:21 -0400 X-MC-Unique: J86Cg2SANC2oEbvAA895yg-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 270DA79EEF; Tue, 12 Oct 2021 06:36:20 +0000 (UTC) Received: from gshan.redhat.com (vpn2-54-56.bne.redhat.com [10.64.54.56]) by smtp.corp.redhat.com (Postfix) with ESMTP id 61FCF19C79; Tue, 12 Oct 2021 06:36:16 +0000 (UTC) From: Gavin Shan To: qemu-arm@nongnu.org Subject: [PATCH v2 1/2] numa: Set default distance map if needed Date: Tue, 12 Oct 2021 14:36:02 +0800 Message-Id: <20211012063603.166255-2-gshan@redhat.com> In-Reply-To: <20211012063603.166255-1-gshan@redhat.com> References: <20211012063603.166255-1-gshan@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=gshan@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=170.10.133.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.049, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, drjones@redhat.com, qemu-devel@nongnu.org, shan.gavin@gmail.com, ehabkost@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The following option is used to specify the distance map. It's possible the option isn't provided by user. In this case, the distance map isn't populated and exposed to platform. On the other hand, the empty NUMA node, where no memory resides, is allowed on ARM64 virt platform. For these empty NUMA nodes, their corresponding device-tree nodes aren't populated, but their NUMA IDs should be included in the "/distance-map" device-tree node, so that kernel can probe them properly if device-tree is used. -numa,dist,src=,dst=,val= So when user doesn't specify distance map, we need to generate the default distance map, where the local and remote distances are 10 and 20 separately. However, this is going to change the hardware description of the guest in this particular scenario. It's fine as the guest should be tolerant to ignore the distance map completely or parse it properly by following the device-tree specification. This introduces an extra parameter to the exiting function complete_init_numa_distance() to generate the default distance map when no node pair distances are provided by user. Signed-off-by: Gavin Shan Reviewed-by: Andrew Jones --- hw/core/numa.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/hw/core/numa.c b/hw/core/numa.c index 510d096a88..c508d857a0 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -594,7 +594,7 @@ static void validate_numa_distance(MachineState *ms) } } -static void complete_init_numa_distance(MachineState *ms) +static void complete_init_numa_distance(MachineState *ms, bool is_default) { int src, dst; NodeInfo *numa_info = ms->numa_state->nodes; @@ -609,6 +609,8 @@ static void complete_init_numa_distance(MachineState *ms) if (numa_info[src].distance[dst] == 0) { if (src == dst) { numa_info[src].distance[dst] = NUMA_DISTANCE_MIN; + } else if (is_default) { + numa_info[src].distance[dst] = NUMA_DISTANCE_DEFAULT; } else { numa_info[src].distance[dst] = numa_info[dst].distance[src]; } @@ -716,13 +718,20 @@ void numa_complete_configuration(MachineState *ms) * A->B != distance B->A, then that means the distance table is * asymmetric. In this case, the distances for both directions * of all node pairs are required. + * + * The default node pair distances, which are 10 and 20 for the + * local and remote nodes separately, are provided if user doesn't + * specify any node pair distances. */ if (ms->numa_state->have_numa_distance) { /* Validate enough NUMA distance information was provided. */ validate_numa_distance(ms); /* Validation succeeded, now fill in any missing distances. */ - complete_init_numa_distance(ms); + complete_init_numa_distance(ms, false); + } else { + complete_init_numa_distance(ms, true); + ms->numa_state->have_numa_distance = true; } } } From patchwork Tue Oct 12 06:36:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gavin Shan X-Patchwork-Id: 12551561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50D73C433EF for ; Tue, 12 Oct 2021 06:40:10 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1306560E94 for ; Tue, 12 Oct 2021 06:40:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1306560E94 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:56480 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1maBSL-0007j0-0p for qemu-devel@archiver.kernel.org; Tue, 12 Oct 2021 02:40:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:34444) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1maBOq-0003UX-Lv for qemu-devel@nongnu.org; Tue, 12 Oct 2021 02:36:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:45816) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1maBOp-0007fV-22 for qemu-devel@nongnu.org; Tue, 12 Oct 2021 02:36:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1634020590; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aD8nzljPsEKJ570RR+jldn/7/HXpuLhVnorn+6ome1Q=; b=XZW7Xj+BEsAlySomqxIH4vzqcYHDpSG+UK1dswTnt3P8S0eFSaoa00yxa05x28eF/NvimF 2iwgfzuEJSKRN8c+x5rR8wCrb+HI/M3+9pomve7VBRRJF5MfKOIquk2OZgFjm6+3pMggnE aN0P1U4ZJDXWH/wcBsvvJyVeeE+FmM0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-171-yrLFVuLzP76FmpPU8aJX-g-1; Tue, 12 Oct 2021 02:36:26 -0400 X-MC-Unique: yrLFVuLzP76FmpPU8aJX-g-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8C5DF91109; Tue, 12 Oct 2021 06:36:25 +0000 (UTC) Received: from gshan.redhat.com (vpn2-54-56.bne.redhat.com [10.64.54.56]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9115019C79; Tue, 12 Oct 2021 06:36:21 +0000 (UTC) From: Gavin Shan To: qemu-arm@nongnu.org Subject: [PATCH v2 2/2] hw/arm/virt: Don't create device-tree node for empty NUMA node Date: Tue, 12 Oct 2021 14:36:03 +0800 Message-Id: <20211012063603.166255-3-gshan@redhat.com> In-Reply-To: <20211012063603.166255-1-gshan@redhat.com> References: <20211012063603.166255-1-gshan@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=gshan@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=216.205.24.124; envelope-from=gshan@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.049, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, drjones@redhat.com, qemu-devel@nongnu.org, shan.gavin@gmail.com, ehabkost@redhat.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" The empty NUMA node, where no memory resides, are allowed. For example, the following command line specifies two empty NUMA nodes. With this, QEMU fails to boot because of the conflicting device-tree node names, as the following error message indicates. /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64 \ -accel kvm -machine virt,gic-version=host \ -cpu host -smp 4,sockets=2,cores=2,threads=1 \ -m 1024M,slots=16,maxmem=64G \ -object memory-backend-ram,id=mem0,size=512M \ -object memory-backend-ram,id=mem1,size=512M \ -numa node,nodeid=0,cpus=0-1,memdev=mem0 \ -numa node,nodeid=1,cpus=2-3,memdev=mem1 \ -numa node,nodeid=2 \ -numa node,nodeid=3 : qemu-system-aarch64: FDT: Failed to create subnode /memory@80000000: FDT_ERR_EXISTS As specified by linux device-tree binding document, the device-tree nodes for these empty NUMA nodes shouldn't be generated. However, the corresponding NUMA node IDs should be included in the distance map device-tree node. This skips populating the device-tree nodes for these empty NUMA nodes to avoid the error, so that QEMU can be started successfully. Signed-off-by: Gavin Shan Reviewed-by: Andrew Jones --- hw/arm/boot.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/hw/arm/boot.c b/hw/arm/boot.c index 57efb61ee4..4e5898fcdc 100644 --- a/hw/arm/boot.c +++ b/hw/arm/boot.c @@ -603,6 +603,10 @@ int arm_load_dtb(hwaddr addr, const struct arm_boot_info *binfo, mem_base = binfo->loader_start; for (i = 0; i < ms->numa_state->num_nodes; i++) { mem_len = ms->numa_state->nodes[i].node_mem; + if (!mem_len) { + continue; + } + rc = fdt_add_memory_node(fdt, acells, mem_base, scells, mem_len, i); if (rc < 0) {