From patchwork Mon Nov 4 21:06:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 13862025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8FE1AD1BDF2 for ; Mon, 4 Nov 2024 21:07:41 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t84HQ-0004aq-QP; Mon, 04 Nov 2024 16:06:33 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t84HN-0004Ul-W1 for qemu-devel@nongnu.org; Mon, 04 Nov 2024 16:06:30 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t84HL-0005JX-P0 for qemu-devel@nongnu.org; Mon, 04 Nov 2024 16:06:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1730754386; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=JBy/Nl5xx97LKS0DNOehVSO7MOxTahNCY7DAG+OBNKA=; b=L84JYH8m91ZkzPtMrWxqb+bIEHI+VRhPODA2liTz2mYpmkSCeGyQTRgT8KNCnQud4vK7W3 Bu4dWN8Dt339ZTv2HvPA8hHj66XaeQ/W5LtkD2Xuhh5Ygj9q98tBMIOi8XzngMl7w5qz17 E5FwnJffqkrl75Fdt0bjqLOC+6Of8ws= Received: from mail-lj1-f198.google.com (mail-lj1-f198.google.com [209.85.208.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-498-bziknDFzMTCheoIe_Fy-iw-1; Mon, 04 Nov 2024 16:06:25 -0500 X-MC-Unique: bziknDFzMTCheoIe_Fy-iw-1 Received: by mail-lj1-f198.google.com with SMTP id 38308e7fff4ca-2fb58d1da8eso26884991fa.0 for ; Mon, 04 Nov 2024 13:06:24 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730754383; x=1731359183; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=JBy/Nl5xx97LKS0DNOehVSO7MOxTahNCY7DAG+OBNKA=; b=Rnu34lxIxQTyaLsLIsLxXvj8Y3lmieJidABVnUc+f3qwj4LsNZijVeQIPTBKGOX+Re MY5pR0sRZx87/SGPZoTFsyYRRcCQ+COPuscH8DXhYu1YKevTi7LYn1v2B8zQEcH9fuPM mCKhuuI4MrrJm1mbPah2yKj0tEDtJx0dapQGy9Ot9pJvhfutoDAklaU6xF5j0foXWoJh NHRgtGgajJITH68zx7GRMcH6sXlg1+bs1QGY8iv2ndnNx6WX9OZnhv7lXWVl/1ecuHTn 3Q+pJdrgY2U2Yl3TqXmQN+3vAh30c6RbBId9Y7O+XjXYgEAPQrOnGzY2VIFd51VJhgAY ME1Q== X-Gm-Message-State: AOJu0YwZhSmMgnH2slukqb+JQwNzWy+Y2pON45rkJJBvxv4tTO8EUpfB 6NbUsoAsVnL/mFnpzbEw0X7YuInV4cjQs8EVaDkNUd93FJ45wXCkB/XHjFXzgrFO86GQ+rUVBIY lDfCCx+OHeQBytJ2l4pQofHiwLiICCxNdWlMEAHpcsqdMRiPM2N7rw/szFQhlCiJ+O0ZCCM7b+s HWZ6pA0lDAJFusRIutNNd8MTFxSyNstw== X-Received: by 2002:a05:651c:54a:b0:2f6:4f17:aeaf with SMTP id 38308e7fff4ca-2fedb7ed786mr58722791fa.45.1730754382938; Mon, 04 Nov 2024 13:06:22 -0800 (PST) X-Google-Smtp-Source: AGHT+IF2E26KbrvPpnh9uLsGNSnQK+tO72+I/zAhoIQCOBTPPBX/gfbkX+gkZjnoyplQ+3TSwvE9UA== X-Received: by 2002:a05:651c:54a:b0:2f6:4f17:aeaf with SMTP id 38308e7fff4ca-2fedb7ed786mr58721771fa.45.1730754380805; Mon, 04 Nov 2024 13:06:20 -0800 (PST) Received: from redhat.com ([2.52.14.134]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4327d6983c2sm168460605e9.42.2024.11.04.13.06.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Nov 2024 13:06:20 -0800 (PST) Date: Mon, 4 Nov 2024 16:06:16 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Jonathan Cameron , Markus Armbruster , "Huang, Ying" , Igor Mammedov , Ani Sinha , Shannon Zhao , Marcel Apfelbaum , Paolo Bonzini , Richard Henderson , Eduardo Habkost , Eric Blake , Daniel =?utf-8?b?UC4gQmVycmFuZ8Op?= , qemu-arm@nongnu.org Subject: [PULL 12/65] hw/acpi: Generic Port Affinity Structure support Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.133.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.34, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Jonathan Cameron These are very similar to the recently added Generic Initiators but instead of representing an initiator of memory traffic they represent an edge point beyond which may lie either targets or initiators. Here we add these ports such that they may be targets of hmat_lb records to describe the latency and bandwidth from host side initiators to the port. A discoverable mechanism such as UEFI CDAT read from CXL devices and switches is used to discover the remainder of the path, and the OS can build up full latency and bandwidth numbers as need for work and data placement decisions. Acked-by: Markus Armbruster Tested-by: "Huang, Ying" Signed-off-by: Jonathan Cameron Message-Id: <20240916174122.1843197-1-Jonathan.Cameron@huawei.com> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- qapi/qom.json | 41 ++++++++++ include/hw/acpi/aml-build.h | 3 + include/hw/acpi/pci.h | 2 +- include/hw/pci/pci_bridge.h | 1 + hw/acpi/aml-build.c | 39 ++++++++++ hw/acpi/pci.c | 116 +++++++++++++++++++++++++++- hw/arm/virt-acpi-build.c | 2 +- hw/i386/acpi-build.c | 2 +- hw/pci-bridge/pci_expander_bridge.c | 1 - 9 files changed, 202 insertions(+), 5 deletions(-) diff --git a/qapi/qom.json b/qapi/qom.json index 321ccd708a..a8beeabf1f 100644 --- a/qapi/qom.json +++ b/qapi/qom.json @@ -844,6 +844,45 @@ 'data': { 'pci-dev': 'str', 'node': 'uint32' } } +## +# @AcpiGenericPortProperties: +# +# Properties for acpi-generic-port objects. +# +# @pci-bus: QOM path of the PCI bus of the hostbridge associated with +# this SRAT Generic Port Affinity Structure. This is the same as +# the bus parameter for the root ports attached to this host +# bridge. The resulting SRAT Generic Port Affinity Structure will +# refer to the ACPI object in DSDT that represents the host bridge +# (e.g. ACPI0016 for CXL host bridges). See ACPI 6.5 Section +# 5.2.16.7 for more information. +# +# @node: Similar to a NUMA node ID, but instead of providing a +# reference point used for defining NUMA distances and access +# characteristics to memory or from an initiator (e.g. CPU), this +# node defines the boundary point between non-discoverable system +# buses which must be described by firmware, and a discoverable +# bus. NUMA distances and access characteristics are defined to +# and from that point. For system software to establish full +# initiator to target characteristics this information must be +# combined with information retrieved from the discoverable part +# of the path. An example would use CDAT (see UEFI.org) +# information read from devices and switches in conjunction with +# link characteristics read from PCIe Configuration space. +# To get the full path latency from CPU to CXL attached DRAM +# CXL device: Add the latency from CPU to Generic Port (from +# HMAT indexed via the the node ID in this SRAT structure) to +# that for CXL bus links, the latency across intermediate switches +# and from the EP port to the actual memory. Bandwidth is more +# complex as there may be interleaving across multiple devices +# and shared links in the path. +# +# Since: 9.1 +## +{ 'struct': 'AcpiGenericPortProperties', + 'data': { 'pci-bus': 'str', + 'node': 'uint32' } } + ## # @RngProperties: # @@ -1043,6 +1082,7 @@ { 'enum': 'ObjectType', 'data': [ 'acpi-generic-initiator', + 'acpi-generic-port', 'authz-list', 'authz-listfile', 'authz-pam', @@ -1118,6 +1158,7 @@ 'discriminator': 'qom-type', 'data': { 'acpi-generic-initiator': 'AcpiGenericInitiatorProperties', + 'acpi-generic-port': 'AcpiGenericPortProperties', 'authz-list': 'AuthZListProperties', 'authz-listfile': 'AuthZListFileProperties', 'authz-pam': 'AuthZPAMProperties', diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h index 33eef85791..47a4692a7d 100644 --- a/include/hw/acpi/aml-build.h +++ b/include/hw/acpi/aml-build.h @@ -490,6 +490,9 @@ void build_srat_pci_generic_initiator(GArray *table_data, int node, uint16_t segment, uint8_t bus, uint8_t devfn); +void build_srat_acpi_generic_port(GArray *table_data, uint32_t node, + const char *hid, uint32_t uid); + void build_slit(GArray *table_data, BIOSLinker *linker, MachineState *ms, const char *oem_id, const char *oem_table_id); diff --git a/include/hw/acpi/pci.h b/include/hw/acpi/pci.h index 3015a8171c..6359d574fd 100644 --- a/include/hw/acpi/pci.h +++ b/include/hw/acpi/pci.h @@ -41,6 +41,6 @@ Aml *aml_pci_device_dsm(void); void build_append_pci_bus_devices(Aml *parent_scope, PCIBus *bus); void build_pci_bridge_aml(AcpiDevAmlIf *adev, Aml *scope); -void build_srat_generic_pci_initiator(GArray *table_data); +void build_srat_generic_affinity_structures(GArray *table_data); #endif diff --git a/include/hw/pci/pci_bridge.h b/include/hw/pci/pci_bridge.h index 5cd452115a..5456e24883 100644 --- a/include/hw/pci/pci_bridge.h +++ b/include/hw/pci/pci_bridge.h @@ -102,6 +102,7 @@ typedef struct PXBPCIEDev { PXBDev parent_obj; } PXBPCIEDev; +#define TYPE_PXB_CXL_BUS "pxb-cxl-bus" #define TYPE_PXB_DEV "pxb" OBJECT_DECLARE_SIMPLE_TYPE(PXBDev, PXB_DEV) diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c index aeb4cb94e8..7a06ffec5a 100644 --- a/hw/acpi/aml-build.c +++ b/hw/acpi/aml-build.c @@ -1954,6 +1954,19 @@ static void build_append_srat_pci_device_handle(GArray *table_data, build_append_int_noprefix(table_data, 0, 12); } +static void build_append_srat_acpi_device_handle(GArray *table_data, + const char *hid, + uint32_t uid) +{ + assert(strlen(hid) == 8); + /* Device Handle - ACPI */ + for (int i = 0; i < sizeof(hid); i++) { + build_append_int_noprefix(table_data, hid[i], 1); + } + build_append_int_noprefix(table_data, uid, 4); + build_append_int_noprefix(table_data, 0, 4); +} + /* * ACPI spec, Revision 6.3 * 5.2.16.6 Generic Initiator Affinity Structure @@ -1981,6 +1994,32 @@ void build_srat_pci_generic_initiator(GArray *table_data, int node, build_append_int_noprefix(table_data, 0, 4); } +/* + * ACPI spec, Revision 6.5 + * 5.2.16.7 Generic Port Affinity Structure + * With ACPI Device Handle. + */ +void build_srat_acpi_generic_port(GArray *table_data, uint32_t node, + const char *hid, uint32_t uid) +{ + /* Type */ + build_append_int_noprefix(table_data, 6, 1); + /* Length */ + build_append_int_noprefix(table_data, 32, 1); + /* Reserved */ + build_append_int_noprefix(table_data, 0, 1); + /* Device Handle Type: ACPI */ + build_append_int_noprefix(table_data, 0, 1); + /* Proximity Domain */ + build_append_int_noprefix(table_data, node, 4); + /* Device Handle */ + build_append_srat_acpi_device_handle(table_data, hid, uid); + /* Flags - GP Enabled */ + build_append_int_noprefix(table_data, 1, 4); + /* Reserved */ + build_append_int_noprefix(table_data, 0, 4); +} + /* * ACPI spec 5.2.17 System Locality Distance Information Table * (Revision 2.0 or later) diff --git a/hw/acpi/pci.c b/hw/acpi/pci.c index 3e1db161cc..d7a0e91f01 100644 --- a/hw/acpi/pci.c +++ b/hw/acpi/pci.c @@ -30,6 +30,7 @@ #include "hw/boards.h" #include "hw/acpi/aml-build.h" #include "hw/acpi/pci.h" +#include "hw/pci/pci_bridge.h" #include "hw/pci/pci_device.h" #include "hw/pci/pcie_host.h" @@ -177,9 +178,122 @@ static int build_acpi_generic_initiator(Object *obj, void *opaque) return 0; } -void build_srat_generic_pci_initiator(GArray *table_data) +typedef struct AcpiGenericPort { + /* private */ + Object parent; + + /* public */ + char *pci_bus; + uint32_t node; +} AcpiGenericPort; + +typedef struct AcpiGenericPortClass { + ObjectClass parent_class; +} AcpiGenericPortClass; + +#define TYPE_ACPI_GENERIC_PORT "acpi-generic-port" + +OBJECT_DEFINE_TYPE_WITH_INTERFACES(AcpiGenericPort, acpi_generic_port, + ACPI_GENERIC_PORT, OBJECT, + { TYPE_USER_CREATABLE }, + { NULL }) + +OBJECT_DECLARE_SIMPLE_TYPE(AcpiGenericPort, ACPI_GENERIC_PORT) + +static void acpi_generic_port_init(Object *obj) +{ + AcpiGenericPort *gp = ACPI_GENERIC_PORT(obj); + + gp->node = MAX_NODES; + gp->pci_bus = NULL; +} + +static void acpi_generic_port_finalize(Object *obj) +{ + AcpiGenericPort *gp = ACPI_GENERIC_PORT(obj); + + g_free(gp->pci_bus); +} + +static void acpi_generic_port_set_pci_bus(Object *obj, const char *val, + Error **errp) +{ + AcpiGenericPort *gp = ACPI_GENERIC_PORT(obj); + + gp->pci_bus = g_strdup(val); +} + +static void acpi_generic_port_set_node(Object *obj, Visitor *v, + const char *name, void *opaque, + Error **errp) +{ + AcpiGenericPort *gp = ACPI_GENERIC_PORT(obj); + uint32_t value; + + if (!visit_type_uint32(v, name, &value, errp)) { + return; + } + + if (value >= MAX_NODES) { + error_printf("%s: Invalid NUMA node specified\n", + TYPE_ACPI_GENERIC_INITIATOR); + exit(1); + } + + gp->node = value; +} + +static void acpi_generic_port_class_init(ObjectClass *oc, void *data) +{ + object_class_property_add_str(oc, "pci-bus", NULL, + acpi_generic_port_set_pci_bus); + object_class_property_set_description(oc, "pci-bus", + "PCI Bus of the host bridge associated with this GP affinity structure"); + object_class_property_add(oc, "node", "int", NULL, + acpi_generic_port_set_node, NULL, NULL); + object_class_property_set_description(oc, "node", + "The NUMA node like ID to index HMAT/SLIT NUMA properties involving GP"); +} + +static int build_acpi_generic_port(Object *obj, void *opaque) +{ + MachineState *ms = MACHINE(qdev_get_machine()); + const char *hid = "ACPI0016"; + GArray *table_data = opaque; + AcpiGenericPort *gp; + uint32_t uid; + Object *o; + + if (!object_dynamic_cast(obj, TYPE_ACPI_GENERIC_PORT)) { + return 0; + } + + gp = ACPI_GENERIC_PORT(obj); + + if (gp->node >= ms->numa_state->num_nodes) { + error_printf("%s: node %d is invalid.\n", + TYPE_ACPI_GENERIC_PORT, gp->node); + exit(1); + } + + o = object_resolve_path_type(gp->pci_bus, TYPE_PXB_CXL_BUS, NULL); + if (!o) { + error_printf("%s: device must be a CXL host bridge.\n", + TYPE_ACPI_GENERIC_PORT); + exit(1); + } + + uid = object_property_get_uint(o, "acpi_uid", &error_fatal); + build_srat_acpi_generic_port(table_data, gp->node, hid, uid); + + return 0; +} + +void build_srat_generic_affinity_structures(GArray *table_data) { object_child_foreach_recursive(object_get_root(), build_acpi_generic_initiator, table_data); + object_child_foreach_recursive(object_get_root(), build_acpi_generic_port, + table_data); } diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index b5973c9148..620992c92c 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -510,7 +510,7 @@ build_srat(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) } } - build_srat_generic_pci_initiator(table_data); + build_srat_generic_affinity_structures(table_data); if (ms->nvdimms_state->is_enabled) { nvdimm_build_srat(table_data); diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c index 88227e343e..d01e704162 100644 --- a/hw/i386/acpi-build.c +++ b/hw/i386/acpi-build.c @@ -1973,7 +1973,7 @@ build_srat(GArray *table_data, BIOSLinker *linker, MachineState *machine) build_srat_memory(table_data, 0, 0, 0, MEM_AFFINITY_NOFLAGS); } - build_srat_generic_pci_initiator(table_data); + build_srat_generic_affinity_structures(table_data); /* * Entry is required for Windows to enable memory hotplug in OS diff --git a/hw/pci-bridge/pci_expander_bridge.c b/hw/pci-bridge/pci_expander_bridge.c index 3d52ea5867..4578e03024 100644 --- a/hw/pci-bridge/pci_expander_bridge.c +++ b/hw/pci-bridge/pci_expander_bridge.c @@ -38,7 +38,6 @@ DECLARE_INSTANCE_CHECKER(PXBBus, PXB_BUS, DECLARE_INSTANCE_CHECKER(PXBBus, PXB_PCIE_BUS, TYPE_PXB_PCIE_BUS) -#define TYPE_PXB_CXL_BUS "pxb-cxl-bus" DECLARE_INSTANCE_CHECKER(PXBBus, PXB_CXL_BUS, TYPE_PXB_CXL_BUS)