From patchwork Tue Jan 7 03:57:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Ming X-Patchwork-Id: 13928200 Received: from sender4-pp-o94.zoho.com (sender4-pp-o94.zoho.com [136.143.188.94]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 884641428E7; Tue, 7 Jan 2025 03:57:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.94 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222268; cv=pass; b=oyz0GRkm9N0xTVUDm+3HPeqXPazxQ59XmtKrqOvIg5wX58Z+IThnCA9P+UYz0VPpAIpf0JhoD2/hxoWaxa8vpIH/L3lVFH7F4lByHzMp/HNzpXPPsHVc0X2n2SVxagJIj18zEXg/nuqBpnPGSPeJn5sq6a5rPeERVDjMTnn2nV0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222268; c=relaxed/simple; bh=0Mwsm2U+HxFCFxxFksT1mRHsc/C4xXgqogE5S3Mzu3E=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kJ12KiAl78WDjZAMRp+PdsUIprECs5qAjLTSKbgCNBkPhkWdVTpQjxC10T/p9/XMbB4XEJpp2PxGRIAtwhjRBJ94kQqXwuQQgm6K9tDhByqYsnwB90rf3nyaPCyGfsozpJ5qTpFN5BZy0CmKtlEYnhVc9IedgWSvHizbQT7uJ5o= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com; spf=pass smtp.mailfrom=zohomail.com; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b=EokBLaPQ; arc=pass smtp.client-ip=136.143.188.94 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zohomail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b="EokBLaPQ" ARC-Seal: i=1; a=rsa-sha256; t=1736222249; cv=none; d=zohomail.com; s=zohoarc; b=gzmKvJe9miKZepezJE0GAdwGGB8RrH+CRBnFQsqk/KH+h4P/08xv0Nt6axhgMIgW6YlBURGmZcLYTtcCjD90l/WZ5tIw2W/TqJqOTpkczwUsic3A2j+g32QjE9G6TzB6Z3Yqn5kRRyknO0u7FiaOW0GI72yY+tEUQqsQD0LRXA8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1736222249; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=3fHPNLQX3+/YAAB44PMrBbiNoska5PAw4ZhbhhYu9nU=; b=N3mtXWeHqbfHPqSwYoofDFStaYx4ya5NzgxEmksWfc7I9BcNBJafw1WV7G+/6YLL23eufC9ZfV8GYOfLt5JCacu8qj75ErUi7ytx5kO2ecWFXRI8Ia4pO30HYx7SclsxGQLDBFxmEyQVmA9ux39ng+Wlu6HT90ikTEfeHU4eNFI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zohomail.com; spf=pass smtp.mailfrom=ming.li@zohomail.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1736222249; s=zm2022; d=zohomail.com; i=ming.li@zohomail.com; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-Id:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Feedback-ID:Reply-To; bh=3fHPNLQX3+/YAAB44PMrBbiNoska5PAw4ZhbhhYu9nU=; b=EokBLaPQP3EEAKr4nSWIRPH4eFDPaHAQHn6bA/Q8Wnbv6yKnsjntHaICzoxRPSE1 CR1eA2/0yNDgFjtKDAuk+GAtD4mZ1TEKXjzVp8YFs+2xDV/r1STTswsQqhCPNVnKjEb 7nE1dNRCCG7zWyN9W+lRUSHor8KczZWBKusT0mNI= Received: by mx.zohomail.com with SMTPS id 1736222247707626.4086841253875; Mon, 6 Jan 2025 19:57:27 -0800 (PST) From: Li Ming To: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Li Ming Subject: [RFC PATCH 1/8] cxl/port: Enumerate port component regs when endpoint attaching Date: Tue, 7 Jan 2025 11:57:01 +0800 Message-Id: <20250107035708.1134954-2-ming.li@zohomail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250107035708.1134954-1-ming.li@zohomail.com> References: <20250107035708.1134954-1-ming.li@zohomail.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Feedback-ID: rr080112273ce3ae1c79c562c7b4114c090000d8d6538b819bddcb63b73e67e4d892b9be91ff787d9def1ca1:zu08011227aa2caf80c888f70197d00983000097fce6194b4c70446ae487d46e9404bd25a58777cf0681ae31:rf0801122deafdd4e088c58c70bd45aa690000c5bb6e66483d2230c935b45c2f3cd4f0c2daa8495a6126ea21923e7f68a883:ZohoMail X-ZohoMailClient: External In some hardware platform, CXL host bridge does not expose its component registers if no CXL device/switch is under it. in this case, CXL subsystem will fail to enumerate component registers on such CXL host bridge during CXL host bridge port attaching, because a CXL port component registers are enumerated during adding the port to the hierarchy. To solve above issue, CXL port component registers enumeration should be delayed until the first endpoint attaching, the implementations is moving port component registers enumeration from devm_cxl_add_port() to devm_cxl_add_endpoint(). Signed-off-by: Li Ming --- drivers/cxl/core/port.c | 102 ++++++++++++++++++++++++++-------------- drivers/cxl/cxl.h | 4 ++ drivers/cxl/port.c | 19 ++++++++ 3 files changed, 91 insertions(+), 34 deletions(-) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 78a5c2c25982..1022c0775daa 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -596,6 +596,12 @@ struct cxl_port *to_cxl_port(const struct device *dev) } EXPORT_SYMBOL_NS_GPL(to_cxl_port, "CXL"); +static void cxl_register_map_reset(struct cxl_register_map *map) +{ + *map = (struct cxl_register_map){ 0 }; + map->resource = CXL_RESOURCE_NONE; +} + static void unregister_port(void *_port) { struct cxl_port *port = _port; @@ -735,6 +741,7 @@ static struct cxl_port *cxl_port_alloc(struct device *uport_dev, ida_init(&port->decoder_ida); port->hdm_end = -1; port->commit_end = -1; + cxl_register_map_reset(&port->reg_map); xa_init(&port->dports); xa_init(&port->endpoints); xa_init(&port->regions); @@ -766,14 +773,52 @@ static int cxl_setup_comp_regs(struct device *host, struct cxl_register_map *map return cxl_setup_regs(map); } -static int cxl_port_setup_regs(struct cxl_port *port, - resource_size_t component_reg_phys) +static resource_size_t find_component_registers(struct device *dev) { + struct cxl_register_map map; + struct pci_dev *pdev; + + /* + * Theoretically, CXL component registers can be hosted on a + * non-PCI device, in practice, only cxl_test hits this case. + */ + if (!dev_is_pci(dev)) + return CXL_RESOURCE_NONE; + + pdev = to_pci_dev(dev); + + cxl_find_regblock(pdev, CXL_REGLOC_RBI_COMPONENT, &map); + return map.resource; +} + +/** + * cxl_port_setup_regs - probe all component registers of a cxl port + * @port: target cxl port + */ +int cxl_port_setup_regs(struct cxl_port *port) +{ + resource_size_t component_reg_phys; + if (dev_is_platform(port->uport_dev)) return 0; + /* component registers have been set up */ + if (port->reg_map.resource != CXL_RESOURCE_NONE) + return 0; + + if (is_cxl_root(to_cxl_port(port->dev.parent))) + component_reg_phys = port->chbcr; + else + component_reg_phys = find_component_registers(port->uport_dev); + + if (component_reg_phys == CXL_RESOURCE_NONE) { + dev_warn(&port->dev, "Invalid Component Registers"); + return -ENXIO; + } + return cxl_setup_comp_regs(&port->dev, &port->reg_map, component_reg_phys); } +EXPORT_SYMBOL_NS_GPL(cxl_port_setup_regs, "CXL"); static int cxl_dport_setup_regs(struct device *host, struct cxl_dport *dport, resource_size_t component_reg_phys) @@ -856,9 +901,8 @@ static int cxl_port_add(struct cxl_port *port, if (rc) return rc; - rc = cxl_port_setup_regs(port, component_reg_phys); - if (rc) - return rc; + if (is_cxl_root(parent_dport->port)) + port->chbcr = component_reg_phys; } else { rc = dev_set_name(dev, "root%d", port->id); if (rc) @@ -1499,16 +1543,26 @@ static void cxl_detach_ep(void *data) dev_dbg(&cxlmd->dev, "disconnect %s from %s\n", ep ? dev_name(ep->ep) : "", dev_name(&port->dev)); cxl_ep_remove(port, ep); - if (ep && !port->dead && xa_empty(&port->endpoints) && - !is_cxl_root(parent_port) && parent_port->dev.driver) { + if (ep && xa_empty(&port->endpoints)) { /* - * This was the last ep attached to a dynamically - * enumerated port. Block new cxl_add_ep() and garbage - * collect the port. + * Reset component registers information on the port + * during the last ep detaching. So that the next ep + * attaching can trigger component registers probing + * again. */ - died = true; - port->dead = true; - reap_dports(port); + cxl_register_map_reset(&port->reg_map); + + if (!port->dead && !is_cxl_root(parent_port) && + parent_port->dev.driver) { + /* + * This was the last ep attached to a dynamically + * enumerated port. Block new cxl_add_ep() and garbage + * collect the port. + */ + died = true; + port->dead = true; + reap_dports(port); + } } device_unlock(&port->dev); @@ -1521,31 +1575,12 @@ static void cxl_detach_ep(void *data) } } -static resource_size_t find_component_registers(struct device *dev) -{ - struct cxl_register_map map; - struct pci_dev *pdev; - - /* - * Theoretically, CXL component registers can be hosted on a - * non-PCI device, in practice, only cxl_test hits this case. - */ - if (!dev_is_pci(dev)) - return CXL_RESOURCE_NONE; - - pdev = to_pci_dev(dev); - - cxl_find_regblock(pdev, CXL_REGLOC_RBI_COMPONENT, &map); - return map.resource; -} - static int add_port_attach_ep(struct cxl_memdev *cxlmd, struct device *uport_dev, struct device *dport_dev) { struct device *dparent = grandparent(dport_dev); struct cxl_dport *dport, *parent_dport; - resource_size_t component_reg_phys; int rc; if (!dparent) { @@ -1581,9 +1616,8 @@ static int add_port_attach_ep(struct cxl_memdev *cxlmd, port = find_cxl_port_at(parent_port, dport_dev, &dport); if (!port) { - component_reg_phys = find_component_registers(uport_dev); port = devm_cxl_add_port(&parent_port->dev, uport_dev, - component_reg_phys, parent_dport); + CXL_RESOURCE_NONE, parent_dport); if (IS_ERR(port)) return PTR_ERR(port); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index f6015f24ad38..9877a0ddb5e9 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -601,6 +601,7 @@ struct cxl_dax_region { * @parent_dport: dport that points to this port in the parent * @decoder_ida: allocator for decoder ids * @reg_map: component and ras register mapping parameters + * @chbcr: physical address of CXL host bridge component registers * @nr_dports: number of entries in @dports * @hdm_end: track last allocated HDM decoder instance for allocation ordering * @commit_end: cursor to track highest committed decoder for commit ordering @@ -621,6 +622,7 @@ struct cxl_port { struct cxl_dport *parent_dport; struct ida decoder_ida; struct cxl_register_map reg_map; + resource_size_t chbcr; int nr_dports; int hdm_end; int commit_end; @@ -771,6 +773,8 @@ struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_port *port, struct device *dport_dev, int port_id, resource_size_t rcrb); +int cxl_port_setup_regs(struct cxl_port *port); + #ifdef CONFIG_PCIEAER_CXL void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport); void cxl_dport_init_ras_reporting(struct cxl_dport *dport, struct device *host); diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 4c83f6a22e58..ae09bbbe0a17 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -91,6 +91,7 @@ static int cxl_switch_port_probe(struct cxl_port *port) static int cxl_endpoint_port_probe(struct cxl_port *port) { + struct cxl_port *iter, *parent_port = to_cxl_port(port->dev.parent); struct cxl_endpoint_dvsec_info info = { .port = port }; struct cxl_memdev *cxlmd = to_cxl_memdev(port->uport_dev); struct cxl_dev_state *cxlds = cxlmd->cxlds; @@ -98,6 +99,24 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) struct cxl_port *root; int rc; + for (iter = parent_port; !is_cxl_root(iter); + iter = to_cxl_port(iter->dev.parent)) { + /* + * The parent port of endpoint has been locked + * during endpoint port attaching. + * Holding the device lock of port to make sure + * the setup not invoked in parallel. + */ + if (iter != parent_port) { + guard(device)(&iter->dev); + rc = cxl_port_setup_regs(iter); + } else { + rc = cxl_port_setup_regs(iter); + } + if (rc) + return rc; + } + rc = cxl_dvsec_rr_decode(cxlds->dev, port, &info); if (rc < 0) return rc; From patchwork Tue Jan 7 03:57:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Ming X-Patchwork-Id: 13928201 Received: from sender4-pp-o94.zoho.com (sender4-pp-o94.zoho.com [136.143.188.94]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC2888632B; Tue, 7 Jan 2025 03:57:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.94 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222280; cv=pass; b=gGO4BC4i0oZwMn3reHeCf3dKUfAi7lmCBeipF+9TxEicZdjrZBvkGDTsTXz6zZWRqFEhLrsgCHo8u9TJ/c1cMRvaPI+iFdZtwerw+VNkAuRZ281BvdQP19ihIx+hnZkNKRyZC7S/fkvutAsgQTYlNiKWAjlWzEFcVxi5XEHYxTc= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222280; c=relaxed/simple; bh=ZJ2npi9TOUOUlJvV1ovQxH+NzqKD2NmODV5k8lPVwLQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=PdiRl2zoB94Yci7xMEYaR4JGJcdIbaBIxYB+kYKnuKYFtCspS3uepwbm0gZT0U0k4tX9bitJ4tLUHZaFJKUaJaMdNfZBA6ta5/alAU+MbN49fEvilruAaGPspIhJrdI6FM4eXthHLG+k9kfzAmqUkOZo3HMdH0hKeRPWq+cYEeU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com; spf=pass smtp.mailfrom=zohomail.com; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b=C7KHlKTP; arc=pass smtp.client-ip=136.143.188.94 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zohomail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b="C7KHlKTP" ARC-Seal: i=1; a=rsa-sha256; t=1736222257; cv=none; d=zohomail.com; s=zohoarc; b=eBtDz4OOvfmx8bhkyUHe1XIEQZhEacWP+ZrrZ83ypDazUZdMkhpjVhb2AMAKbAULGFu8P/13f+pP72ohT5vtmqdJzvpopKhPA1/sgiSrN4PRBHZsH4FYgvyly1ItOOX5I26z4iYnlzMkB3MWhXuF7kbQ6z50quk3dHuVzkzP2dk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1736222257; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=71Vfq/lr3HI1J1zvQTZQRh2XzJDy5bIV5/bnF/fmRUI=; b=lQ+pQrPSiuFjkrJglJEvvFYaaXthXumZTmeWmW7x88KFyLqdthCuxgVu/Gvw+U1Dunc+Eof2+IQ3n7H4IOVUfRZPgeUKX//4M+m6FOeIV0WXmJcnCvc09GUQ0ff9h/1ZuxiX+vaOb9mdp6pNUNVPIDL7rPCAdogQSYF81VMN71M= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zohomail.com; spf=pass smtp.mailfrom=ming.li@zohomail.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1736222257; s=zm2022; d=zohomail.com; i=ming.li@zohomail.com; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-Id:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Feedback-ID:Reply-To; bh=71Vfq/lr3HI1J1zvQTZQRh2XzJDy5bIV5/bnF/fmRUI=; b=C7KHlKTPCfpfLekYLgpmJj7BcQP6YP+UdJxdzWTSubtTwU+XQEa1ZuJP9yhjj7sA cXrU/YIeNXo/zor2EmacIgRzR9cJeUSyeBroKAcSTQt4BlPdpn/oLbkeWSNKWjkMfxI /xZaANbkWZudQTHnj0y/iCQ2DqomlxVzms3NVick= Received: by mx.zohomail.com with SMTPS id 1736222256697604.2342245130915; Mon, 6 Jan 2025 19:57:36 -0800 (PST) From: Li Ming To: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Li Ming Subject: [RFC PATCH 2/8] cxl/port: Delay port HDM setup until port component regs setup done Date: Tue, 7 Jan 2025 11:57:02 +0800 Message-Id: <20250107035708.1134954-3-ming.li@zohomail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250107035708.1134954-1-ming.li@zohomail.com> References: <20250107035708.1134954-1-ming.li@zohomail.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Feedback-ID: rr0801122721bf7b18384845fc227a7653000038dcea3b84bf001be3ee9689bc35dd43083f9e8465c96a9383:zu08011227af8fc1d34e5b4178c22d399e0000435e35c5b2af67e7dd3065942f9b4e2c38e93afbd25b8149a2:rf0801122d17b229b88663d808fed0c7470000d6ea91fa81cc05c594e745b3aae6a57a60b2c8a6e9fb226e0fd2d52f6156ee:ZohoMail X-ZohoMailClient: External Currently, devm_cxl_add_endpoint() is responsible for all CXL ports component registers enumeration. so HDM setup for the port of host bridge/switch also needs to be delayed until componeng registers setup finished. So moving CXL host bridge/switch HDM setup implementation out of cxl_switch_port_probe(), and let devm_cxl_add_endpoint() help to setup HDM on such ports. Signed-off-by: Li Ming --- drivers/cxl/port.c | 28 +++++++++++++++++++++++++--- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index ae09bbbe0a17..c59f198f6cb0 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -59,7 +59,6 @@ static int discover_region(struct device *dev, void *root) static int cxl_switch_port_probe(struct cxl_port *port) { - struct cxl_hdm *cxlhdm; int rc; /* Cache the data early to ensure is_visible() works */ @@ -71,6 +70,18 @@ static int cxl_switch_port_probe(struct cxl_port *port) cxl_switch_parse_cdat(port); + return 0; +} + +static int cxl_switch_port_setup_hdm(struct cxl_port *port) +{ + struct cxl_hdm *cxlhdm; + int rc; + + /* Skip hdm setup if there is a cxlhdm on the port */ + if (dev_get_drvdata(&port->dev)) + return 0; + cxlhdm = devm_cxl_setup_hdm(port, NULL); if (!IS_ERR(cxlhdm)) return devm_cxl_enumerate_decoders(cxlhdm, NULL); @@ -89,6 +100,17 @@ static int cxl_switch_port_probe(struct cxl_port *port) return -ENXIO; } +static int cxl_switch_port_setup(struct cxl_port *port) +{ + int rc; + + rc = cxl_port_setup_regs(port); + if (rc) + return rc; + + return cxl_switch_port_setup_hdm(port); +} + static int cxl_endpoint_port_probe(struct cxl_port *port) { struct cxl_port *iter, *parent_port = to_cxl_port(port->dev.parent); @@ -109,9 +131,9 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) */ if (iter != parent_port) { guard(device)(&iter->dev); - rc = cxl_port_setup_regs(iter); + rc = cxl_switch_port_setup(iter); } else { - rc = cxl_port_setup_regs(iter); + rc = cxl_switch_port_setup(iter); } if (rc) return rc; From patchwork Tue Jan 7 03:57:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Ming X-Patchwork-Id: 13928202 Received: from sender4-pp-o94.zoho.com (sender4-pp-o94.zoho.com [136.143.188.94]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D11768632B; Tue, 7 Jan 2025 03:58:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.94 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222292; cv=pass; b=sSADyGNsu7u9H5+HoV+ZRb2RipXhjrWb3/jcQiIJRDEzcwKd/9KvzJpNLTco5BRD0WBNVDuzDDtmrhWFC7q/WT+ud+Ewny11yeBZQs6Lpu6N+l3GR4fdbhxaqsG0bdNIqdPwXGkgz/PAbY6DailzOzuwWzc9FiiNlhgsI5G/3D4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222292; c=relaxed/simple; bh=zqSHsziad0iLZs6QqxuWE6HhpLK0IvOWGwH5UtPLgQ0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GV51yp0RQED5wpaIUsTjtgxXNaJopFXjktzXOc6pt6eRqhkGKr3y1cLqIvGFBJJIo4CZ5TnQMt5GsoKk2PCQCYAZhEyows5wdaQXc8NquJeERRpXH/DsD65KLTKTXxNmtCqHtwDne1PUoXbDHFvznoibwMNJInkbQz8PwS2az20= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com; spf=pass smtp.mailfrom=zohomail.com; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b=Bw7hL/7a; arc=pass smtp.client-ip=136.143.188.94 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zohomail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b="Bw7hL/7a" ARC-Seal: i=1; a=rsa-sha256; t=1736222267; cv=none; d=zohomail.com; s=zohoarc; b=D/tbmD1/1pPU9jGIBH/5COPQtHH1/iXr+e9MpXpJ7lqYaFVuFGty82RSsD5Rz/nYJS5fASAJFmKffpELPtRIQsWhlx773eeNFq09Q3vFHPJuv3qdGVPQn6JGuWxlxfyhuSmGX62Bxh3j3ltJyGsC8Ph2jnk6PdTHi5ejfFIhsIA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1736222267; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=uSVefGGKaTk1sDvzueyHqzcDxxx1WJNakriqtp4cVcY=; b=SBiODlx+qHS9k5NzkHluW5cnd1FTC7vrWT1hcyP0LPlqqL7kiJVWg9X6P14vbMatJBPmT4uDs623j8N/KvFjDUlsNtGFreZFU17I9NqelmOMX0jhCJOjDGgRNmu6p3KZudr725plCC/PJLIctpTvqUiEl8/Ls6vUCSc8DmX5yug= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zohomail.com; spf=pass smtp.mailfrom=ming.li@zohomail.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1736222267; s=zm2022; d=zohomail.com; i=ming.li@zohomail.com; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-Id:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Feedback-ID:Reply-To; bh=uSVefGGKaTk1sDvzueyHqzcDxxx1WJNakriqtp4cVcY=; b=Bw7hL/7aly7BLTjGq0mzEJ7G9ADvwBxfjGU94bZ/a7bvYH5fVqWNOXH/WGYGzPDT Bbx+EMYLH+9wLbMWWT4EQk8YdiLEMshtDyoZ0I7uNodW9P5uyUcoJglddYBtB+JSy6w rTuYBzmDEBHVE971jDW4yHHj6+6569JAE5oXHJ7Q= Received: by mx.zohomail.com with SMTPS id 1736222265459480.52893116612825; Mon, 6 Jan 2025 19:57:45 -0800 (PST) From: Li Ming To: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Li Ming Subject: [RFC PATCH 3/8] cxl/region: Check if dev is a decoder in check_commit_order() Date: Tue, 7 Jan 2025 11:57:03 +0800 Message-Id: <20250107035708.1134954-4-ming.li@zohomail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250107035708.1134954-1-ming.li@zohomail.com> References: <20250107035708.1134954-1-ming.li@zohomail.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Feedback-ID: rr08011227af502f1a8495808710c3aa39000015bf6d3ef7ba5e42951c0b9c32923616f4a1c640c9d8b583c2:zu080112270161c2dbca105bd02a72bbf90000eed5e76b7e96e1683d62d3b9ad0fdee4bb017b85b11fdd900c:rf0801122d8316552c81ef98b053220e860000b9b7009b32778fb0271da8f4050fe098a0f38548e1dbbedbc950ccbd57cd7a:ZohoMail X-ZohoMailClient: External Currently, endpoint port probing helps to enumerate decoders on its ancestor ports. it will impact the sequence of port's device child list. Before that, port's decoders are added to port's device child list firstly, then all ports under the port will be added to the port's device child list. But now, all ports under the port will be added to the port's device child list firstly, then the port's decoders will be added. So when using device_for_each_child_reverse_from() to check decoders commit order, needs to check if the child device is decoder, if not, that means the first decoder was walked. Returning a positive in such case to stop iterator. Signed-off-by: Li Ming --- drivers/cxl/core/region.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index b98b1ccffd1c..946fca0fe0fe 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -780,8 +780,12 @@ static size_t show_targetN(struct cxl_region *cxlr, char *buf, int pos) static int check_commit_order(struct device *dev, const void *data) { - struct cxl_decoder *cxld = to_cxl_decoder(dev); + struct cxl_decoder *cxld; + if (!is_switch_decoder(dev)) + return 1; + + cxld = to_cxl_decoder(dev); /* * if port->commit_end is not the only free decoder, then out of * order shutdown has occurred, block further allocations until @@ -815,7 +819,7 @@ static int match_free_decoder(struct device *dev, void *data) rc = device_for_each_child_reverse_from(dev->parent, dev, NULL, check_commit_order); - if (rc) { + if (rc < 0) { dev_dbg(dev->parent, "unable to allocate %s due to out of order shutdown\n", dev_name(dev)); From patchwork Tue Jan 7 03:57:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Ming X-Patchwork-Id: 13928203 Received: from sender4-pp-o94.zoho.com (sender4-pp-o94.zoho.com [136.143.188.94]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED4231531DB; Tue, 7 Jan 2025 03:58:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.94 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222304; cv=pass; b=rwT4aCeMUasXnwS/kp9CrALS/kJKR8pfl28FdfaOovGm03K6yQbKI+0ix76isgl53JdMy7koTZz0u/yy5RPYk/ZGS5IuxNgrZ1ZYhtyBFrlk/ukLRegdavxRXFzwZNi5f7f13JTiNQ/creLk+MVlJkTb1TxCqHWd/LzE7qpQZh8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222304; c=relaxed/simple; bh=7nFkkVqN7/D/o3xkRFgG0U8I1TMzENNOEsOlI33ucbQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YMCNclM7vN8TUzh24SC6ZO9X/NC/oMndzz3QhUosZ5p/2Fbe3XUyg1jj44il7VVQCpQUeF4y3L3qJu7qcfCkUUw9CkhFJvMSpwpsa1O+atWVT4/9Q2PD58sxdDtw8k5Q5YiLsVRKgOOLu6ruZb4npnq1lgCi1u85x/E0UcCVNC8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com; spf=pass smtp.mailfrom=zohomail.com; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b=epH7HLy9; arc=pass smtp.client-ip=136.143.188.94 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zohomail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b="epH7HLy9" ARC-Seal: i=1; a=rsa-sha256; t=1736222275; cv=none; d=zohomail.com; s=zohoarc; b=JnM2kxPJgo+cR61XE11ailDe3ukFGeFFXgyOKpviJAe38IXlvrS0bXRqTwgvGz87YKlfuED/g4OdmPZj97s1T4jd4Oz15zZuFuXl5wwcg+c8ZBMMGEAV6FWJMzWoZU3rX2NhE8AKaQVm6Dl9QMOG0rP+Pdq1BiyqLACrWDxfj2I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1736222275; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=rycCUr+DUm84Bi0eVFTDSNBuZn0sgVt0Mem0dR4C/+E=; b=RjApRVAk9AHQCWQUzXnSrVq76xkryPQs+0V+3+TDLnMwrsaAEQEu8w10Z+v52S0t0+cLctaCLZChhEodmJngXnYPgUs16ILeGElpwE6290Tc6+CxvfMEzYZv2d+v2HBKlBrcISg81Z4xLrzQYfRymiaOSrpryflE+2jSw3BRu4g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zohomail.com; spf=pass smtp.mailfrom=ming.li@zohomail.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1736222275; s=zm2022; d=zohomail.com; i=ming.li@zohomail.com; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-Id:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Feedback-ID:Reply-To; bh=rycCUr+DUm84Bi0eVFTDSNBuZn0sgVt0Mem0dR4C/+E=; b=epH7HLy9+liGyyXjdXx1cSsgQsimoagl9IYcLS9UEhEf7YqHZWy8db7qFI1w+GSU VfSV9BJEW5iwJ5pHjMk5CzaJacxO4zYLnM5CIzhsCSt0VaFvuCzygWT/rBuDVnqe6vw iT831Cbcec5wQ4jQWwhh+xahSLvPxcIbdJwCPxUE= Received: by mx.zohomail.com with SMTPS id 1736222274358752.5936388453697; Mon, 6 Jan 2025 19:57:54 -0800 (PST) From: Li Ming To: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Li Ming Subject: [RFC PATCH 4/8] cxl/port: Remove port HDM during the last endpoint detaching Date: Tue, 7 Jan 2025 11:57:04 +0800 Message-Id: <20250107035708.1134954-5-ming.li@zohomail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250107035708.1134954-1-ming.li@zohomail.com> References: <20250107035708.1134954-1-ming.li@zohomail.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Feedback-ID: rr08011227cb067eccf104b7fbeecfb58500007e962f4cf63ec338833a60be77648fcf007fc2cb62fb01d2d2:zu08011227eb3d0c92053a338e93b936d30000fb90b355ce885dec466a2a1b1e52786889740f6e2e8ce92ec4:rf0801122db7c2c356ce55061d7ce5b64400007cd11908feb53dbc9273b6d560028fab3e9c29ec039d67483b5209076e341f:ZohoMail X-ZohoMailClient: External HDM is set up on a CXL port when the first endpoint under the port attaching. The HDM should be released when no endpoint is under the port so that the first endpoint attaching can trigger HDM setup on the port again next time. So remove HDM and decoders on a CXL port if no endpoint is working under it. Signed-off-by: Li Ming --- drivers/cxl/core/port.c | 60 +++++++++++++++++++++++++++++++---------- drivers/cxl/core/regs.c | 9 ++++++- 2 files changed, 54 insertions(+), 15 deletions(-) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 1022c0775daa..57ed152d96d7 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1517,6 +1517,49 @@ static int port_has_memdev(struct device *dev, const void *data) return !!cxl_ep_load(port, ctx->cxlmd); } +static void cxld_unregister(void *dev) +{ + struct cxl_endpoint_decoder *cxled; + + if (is_endpoint_decoder(dev)) { + cxled = to_cxl_endpoint_decoder(dev); + cxl_decoder_kill_region(cxled); + } + + device_unregister(dev); +} + +static int cxl_decoder_remove(struct device *dev, void *data) +{ + struct cxl_port *port = (struct cxl_port *)data; + + if (!is_switch_decoder(dev) && !is_endpoint_decoder(dev)) + return 0; + + devm_release_action(&port->dev, cxld_unregister, dev); + return 0; +} + +static void cxl_port_remove_hdm(struct cxl_port *port) +{ + const struct cxl_reg_map *rmap = &port->reg_map.component_map.hdm_decoder; + struct cxl_hdm *cxlhdm = dev_get_drvdata(&port->dev); + resource_size_t addr, length; + + if (!cxlhdm) + return; + + device_for_each_child(&port->dev, port, cxl_decoder_remove); + if (cxlhdm->regs.hdm_decoder) { + devm_iounmap(&port->dev, cxlhdm->regs.hdm_decoder); + addr = port->reg_map.resource + rmap->offset; + length = rmap->size; + devm_release_mem_region(&port->dev, addr, length); + } + devm_kfree(&port->dev, cxlhdm); + dev_set_drvdata(&port->dev, NULL); +} + static void cxl_detach_ep(void *data) { struct cxl_memdev *cxlmd = data; @@ -1545,11 +1588,12 @@ static void cxl_detach_ep(void *data) cxl_ep_remove(port, ep); if (ep && xa_empty(&port->endpoints)) { /* - * Reset component registers information on the port + * Reset HDM and component registers information on the port * during the last ep detaching. So that the next ep * attaching can trigger component registers probing - * again. + * and HDM setup again. */ + cxl_port_remove_hdm(port); cxl_register_map_reset(&port->reg_map); if (!port->dead && !is_cxl_root(parent_port) && @@ -2031,18 +2075,6 @@ int cxl_decoder_add(struct cxl_decoder *cxld, int *target_map) } EXPORT_SYMBOL_NS_GPL(cxl_decoder_add, "CXL"); -static void cxld_unregister(void *dev) -{ - struct cxl_endpoint_decoder *cxled; - - if (is_endpoint_decoder(dev)) { - cxled = to_cxl_endpoint_decoder(dev); - cxl_decoder_kill_region(cxled); - } - - device_unregister(dev); -} - int cxl_decoder_autoremove(struct device *host, struct cxl_decoder *cxld) { return devm_add_action_or_reset(host, cxld_unregister, &cxld->dev); diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c index 59cb35b40c7e..7aba68785287 100644 --- a/drivers/cxl/core/regs.c +++ b/drivers/cxl/core/regs.c @@ -194,8 +194,15 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr, } ret_val = devm_ioremap(dev, addr, length); - if (!ret_val) + if (!ret_val) { dev_err(dev, "Failed to map region %pr\n", res); + /* + * Explicitly release mem region in failure case + * so that no need to consider the case of devm_request_mem_region() + * success but devm_ioremap() failure in cxl_port_remove_hdm(). + */ + devm_release_mem_region(dev, addr, length); + } return ret_val; } From patchwork Tue Jan 7 03:57:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Ming X-Patchwork-Id: 13928204 Received: from sender4-pp-o94.zoho.com (sender4-pp-o94.zoho.com [136.143.188.94]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7274E14F9FD; Tue, 7 Jan 2025 03:58:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.94 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222313; cv=pass; b=a00rfZy7lQpB3pB3YVdmRQIAlnCl8CkU2RGC405TpyonJxxO+Ah5U76BK9HSIXdA6yR5kG3IF2SgJgZpQSkKw7aKdaRAqUpXCEHQB+zGx63q9JQImi3sFCR5v4j4MiWCB/RLeXMkMOLP1Z/yEJWTywxbr9VCq7NU7P/0NzNbiDw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222313; c=relaxed/simple; bh=yGQY2jobpoPyyWyv8ALwgoLQF/nXPZikai9+tl3z6lY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Ztcpypy+OOtvUWmV1iARozXvKz9SFC7rWv0kexSlVn9MGkqf/8AnayhpU9ESbHeoXIcNTFOuSbYRL88OkYU3TfLJvypChl4yrT0w8jZSokh7tzOOOR4HqZh0/QRFXd0rD1eA3b1MD/k0MJRu/YfsVUeZcsIOkqStOAXLRTnO7eA= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com; spf=pass smtp.mailfrom=zohomail.com; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b=d77w07JD; arc=pass smtp.client-ip=136.143.188.94 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zohomail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b="d77w07JD" ARC-Seal: i=1; a=rsa-sha256; t=1736222285; cv=none; d=zohomail.com; s=zohoarc; b=Og0Ir0nOyeVSICdIAc5/mQINztxiedCRzmYOyKj08uJdUr2IC4JJZv9FdCT5PFPRv/pRqyahd5IbhTffS1Ww2XzLU/H3DO0bSSU1t71c27GJRpzhOrWiNJUB4phkPcBGSZfrZoEb6j/Y8d/y7hvdZ82lGbxg0PFgl36CJCmfytk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1736222285; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=xtx1Y0qNloiBiG48S0dYwcxuoZj69hd4Fnz1ee2aAyc=; b=CdErRd/mwawhDHH7cWIsEy7v6N0Sl/R8RT01PTgNi3T6pKxkM9EUpL0HwvlbOq6YiJn8RwXsflt3PnMd4TPGOAF+X0DeyMfADumq34c2UNFZFswPJZ1Psnoc6P38iZAjZT1FvPgQTUU14nFzpMAlyukTiuYHsXgeCMcJjFUUByA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zohomail.com; spf=pass smtp.mailfrom=ming.li@zohomail.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1736222285; s=zm2022; d=zohomail.com; i=ming.li@zohomail.com; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-Id:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Feedback-ID:Reply-To; bh=xtx1Y0qNloiBiG48S0dYwcxuoZj69hd4Fnz1ee2aAyc=; b=d77w07JDH5X+pacisYVtP5V4GKCz9LhbUujfbBsFkTC7E5wB/vWsSMTpWc6GTsHJ 1dcmhI8h6R8YT1X16xd1pGA8uXNC4f2b1xGPQx4+YqVZ3/6+dilpHILNeWj7rmXrC+6 Jyc9bW6y43BeIR2aLZUNP84tXPVpCeykEvbcNTQc= Received: by mx.zohomail.com with SMTPS id 1736222283105600.1208529234427; Mon, 6 Jan 2025 19:58:03 -0800 (PST) From: Li Ming To: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Li Ming Subject: [RFC PATCH 5/8] cxl/port: Remove component_reg_phys parameter from devm_cxl_add_port() Date: Tue, 7 Jan 2025 11:57:05 +0800 Message-Id: <20250107035708.1134954-6-ming.li@zohomail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250107035708.1134954-1-ming.li@zohomail.com> References: <20250107035708.1134954-1-ming.li@zohomail.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Feedback-ID: rr08011227af989cad0c073ab114caf994000043d42f35344d37d822569302e4839a7b8a32b9a893ed199608:zu08011227629b8e546cd1d307c8ed576200000225a6d961be14539e59c266feed87628be8dc63dc4dce1ca3:rf0801122dc633b6006627fa3cb8f6e194000025b11bb1f6ece12df4fb97d45fadc650442744261412d03e8436f9f7cf0e6f:ZohoMail X-ZohoMailClient: External devm_cxl_add_port() is not responsible for port component registers setup anymore, the component_reg_phys parameter is unnecessary for devm_cxl_add_port(). Signed-off-by: Li Ming --- drivers/cxl/acpi.c | 3 ++- drivers/cxl/core/port.c | 19 +++++-------------- drivers/cxl/cxl.h | 1 - drivers/cxl/mem.c | 3 +-- 4 files changed, 8 insertions(+), 18 deletions(-) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index cb14829bb9be..b754c2c55659 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -662,9 +662,10 @@ static int add_host_bridge_uport(struct device *match, void *arg) if (rc) return rc; - port = devm_cxl_add_port(host, bridge, component_reg_phys, dport); + port = devm_cxl_add_port(host, bridge, dport); if (IS_ERR(port)) return PTR_ERR(port); + port->chbcr = component_reg_phys; dev_info(bridge, "host supports CXL\n"); diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 57ed152d96d7..b406ba64f6bc 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -873,9 +873,7 @@ static void cxl_debugfs_create_dport_dir(struct cxl_dport *dport) &cxl_einj_inject_fops); } -static int cxl_port_add(struct cxl_port *port, - resource_size_t component_reg_phys, - struct cxl_dport *parent_dport) +static int cxl_port_add(struct cxl_port *port, struct cxl_dport *parent_dport) { struct device *dev __free(put_device) = &port->dev; int rc; @@ -900,9 +898,6 @@ static int cxl_port_add(struct cxl_port *port, rc = dev_set_name(dev, "port%d", port->id); if (rc) return rc; - - if (is_cxl_root(parent_dport->port)) - port->chbcr = component_reg_phys; } else { rc = dev_set_name(dev, "root%d", port->id); if (rc) @@ -920,7 +915,6 @@ static int cxl_port_add(struct cxl_port *port, static struct cxl_port *__devm_cxl_add_port(struct device *host, struct device *uport_dev, - resource_size_t component_reg_phys, struct cxl_dport *parent_dport) { struct cxl_port *port; @@ -930,7 +924,7 @@ static struct cxl_port *__devm_cxl_add_port(struct device *host, if (IS_ERR(port)) return port; - rc = cxl_port_add(port, component_reg_phys, parent_dport); + rc = cxl_port_add(port, parent_dport); if (rc) return ERR_PTR(rc); @@ -956,18 +950,15 @@ static struct cxl_port *__devm_cxl_add_port(struct device *host, * devm_cxl_add_port - register a cxl_port in CXL memory decode hierarchy * @host: host device for devm operations * @uport_dev: "physical" device implementing this upstream port - * @component_reg_phys: (optional) for configurable cxl_port instances * @parent_dport: next hop up in the CXL memory decode hierarchy */ struct cxl_port *devm_cxl_add_port(struct device *host, struct device *uport_dev, - resource_size_t component_reg_phys, struct cxl_dport *parent_dport) { struct cxl_port *port, *parent_port; - port = __devm_cxl_add_port(host, uport_dev, component_reg_phys, - parent_dport); + port = __devm_cxl_add_port(host, uport_dev, parent_dport); parent_port = parent_dport ? parent_dport->port : NULL; if (IS_ERR(port)) { @@ -994,7 +985,7 @@ struct cxl_root *devm_cxl_add_root(struct device *host, struct cxl_root *cxl_root; struct cxl_port *port; - port = devm_cxl_add_port(host, host, CXL_RESOURCE_NONE, NULL); + port = devm_cxl_add_port(host, host, NULL); if (IS_ERR(port)) return ERR_CAST(port); @@ -1661,7 +1652,7 @@ static int add_port_attach_ep(struct cxl_memdev *cxlmd, port = find_cxl_port_at(parent_port, dport_dev, &dport); if (!port) { port = devm_cxl_add_port(&parent_port->dev, uport_dev, - CXL_RESOURCE_NONE, parent_dport); + parent_dport); if (IS_ERR(port)) return PTR_ERR(port); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 9877a0ddb5e9..7763be02ef81 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -748,7 +748,6 @@ int devm_cxl_register_pci_bus(struct device *host, struct device *uport_dev, struct pci_bus *cxl_port_to_pci_bus(struct cxl_port *port); struct cxl_port *devm_cxl_add_port(struct device *host, struct device *uport_dev, - resource_size_t component_reg_phys, struct cxl_dport *parent_dport); struct cxl_root *devm_cxl_add_root(struct device *host, const struct cxl_root_ops *ops); diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 2f03a4d5606e..39da99a4bf4d 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -65,8 +65,7 @@ static int devm_cxl_add_endpoint(struct device *host, struct cxl_memdev *cxlmd, } /* Note: endpoint port component registers are derived from @cxlds */ - endpoint = devm_cxl_add_port(host, &cxlmd->dev, CXL_RESOURCE_NONE, - parent_dport); + endpoint = devm_cxl_add_port(host, &cxlmd->dev, parent_dport); if (IS_ERR(endpoint)) return PTR_ERR(endpoint); From patchwork Tue Jan 7 03:57:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Ming X-Patchwork-Id: 13928205 Received: from sender4-pp-o94.zoho.com (sender4-pp-o94.zoho.com [136.143.188.94]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 57615145B16; Tue, 7 Jan 2025 03:58:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.94 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222326; cv=pass; b=AUvGVeiqdmKaJFtsrrDda7Ba7jAEtdYr/qfVPIWnTFDSFYsNre0iNWijMZPP1wk3kFRV/ZcOvwQ3k5tQUNp+wG74ugy8BXyr8lrXFoeBsBQ9Tkl18fpCRH8adREExdqYeC30Gniq8gqFuBzhoeA4fbUDtsYO8pmyYoEOr1Kjkro= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222326; c=relaxed/simple; bh=Cm/w/MPqgdPfaPmPB3glkVTDu6hhJaaWkEc7imgxwt8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=YMc3aHltgJ7ZRwRAyasoSGLN0aewXOgJ4EEKM2wOgQqn1AF1sDhY5qpussNNeCuVLVilF/Etukxub77mQ4PStUkDRHmWINHMxcAPRCa8cHoee7UkWt1SHMSrJw8J/9ryyNbaF47s5jbXHHVusEOWj0PoYiRnoSBRZjY6ocxXptY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com; spf=pass smtp.mailfrom=zohomail.com; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b=CNwG1+Pc; arc=pass smtp.client-ip=136.143.188.94 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zohomail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b="CNwG1+Pc" ARC-Seal: i=1; a=rsa-sha256; t=1736222294; cv=none; d=zohomail.com; s=zohoarc; b=YNH7rIVXhbqKTZQECqbazmnfRtdR8Y/94bt4zUWiRiaTw8QQZJu9mR+yfvDRdcXXAdpiE7fscbIhvD33GRJ1vnMP869s2hr8QLsIigthrHCZCA4x5uzsIJPN4IZB8lEQQI+MUjbNpTlCy84eex9KrTB1V/t/8XqsqD8/GxswyYQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1736222294; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=q3AkgWCWMELI0m2SkQkpfDpVb4TefEBAixjvr5k18nc=; b=WWo7L4GnbCzQLoUFeONabKx4Z4MOUsiFWT8nxYICi6knwj6C9jAm2MGDhjLJTl1qYkGeTiQtR/KuTXtNhBcdw7mubBu1EPTTwsUM1IU7fqsJMf4pcnzexbolJ2Zun/FW20dNP+V+1/9j3M+b56pdeOdxltpTzppLzaVnbAH2j5I= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zohomail.com; spf=pass smtp.mailfrom=ming.li@zohomail.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1736222294; s=zm2022; d=zohomail.com; i=ming.li@zohomail.com; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-Id:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Feedback-ID:Reply-To; bh=q3AkgWCWMELI0m2SkQkpfDpVb4TefEBAixjvr5k18nc=; b=CNwG1+PcjtuP2x526pdbWbAz9PYUI2k2Lpy3dhYIuqmE5XEDYOQsiZ/mnJlGB1zd keEMqvqf+o1S1SYG9U1OjRJe8jqjqgP4goyB/Q7FvldueYlmISIMWVDPDaCoeEAkTWg gnu3UP35a5KVGH2sg4zHOg+IhNsEcI0v6ncBuq/w= Received: by mx.zohomail.com with SMTPS id 1736222291628441.67571042337863; Mon, 6 Jan 2025 19:58:11 -0800 (PST) From: Li Ming To: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Li Ming Subject: [RFC PATCH 6/8] cxl/port: Enumerate dport component registers when endpoint attaching Date: Tue, 7 Jan 2025 11:57:06 +0800 Message-Id: <20250107035708.1134954-7-ming.li@zohomail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250107035708.1134954-1-ming.li@zohomail.com> References: <20250107035708.1134954-1-ming.li@zohomail.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Feedback-ID: rr080112274e0e11f59bd0ec48e93449720000aa66435feda7b2a1038ae9172849cdeaaf5f62f6dfa1d9a620:zu080112271002f11d785326de5d5c87d2000076fdf23c5a46dc93a629c470e23ac110fdab4495299ddeb427:rf0801122dd287280cd9382e1a4a759415000030698c35451985f9e315c2a6653fa208b7e55076aed61bdfea22715f050075:ZohoMail X-ZohoMailClient: External In some hardware platform, dport's component registers are exposed only when the dport is working on CXL mode and connecting to a CXL switch/CXL device. Currently, CXL subsystem will probe a dport's component registers when the dport added to its port. In VH case, a root port's component registers are probed during the host bridge the root port belongs to attaching. As mentioned above, if no CXL/switch/CXL device connected to the root port, root port component registers setup will failed. The solution is delaying all CXL non-RCH dport component registers enumeration until the first endpoint attaching. Signed-off-by: Li Ming --- drivers/cxl/core/pci.c | 8 +------ drivers/cxl/core/port.c | 53 ++++++++++++++++++++++++++--------------- drivers/cxl/cxl.h | 1 + drivers/cxl/port.c | 20 ++++++++++------ 4 files changed, 49 insertions(+), 33 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 9d58ab9d33c5..3b22f3033484 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -37,10 +37,8 @@ static int match_add_dports(struct pci_dev *pdev, void *data) struct cxl_walk_context *ctx = data; struct cxl_port *port = ctx->port; int type = pci_pcie_type(pdev); - struct cxl_register_map map; struct cxl_dport *dport; u32 lnkcap, port_num; - int rc; if (pdev->bus != ctx->bus) return 0; @@ -52,12 +50,8 @@ static int match_add_dports(struct pci_dev *pdev, void *data) &lnkcap)) return 0; - rc = cxl_find_regblock(pdev, CXL_REGLOC_RBI_COMPONENT, &map); - if (rc) - dev_dbg(&port->dev, "failed to find component registers\n"); - port_num = FIELD_GET(PCI_EXP_LNKCAP_PN, lnkcap); - dport = devm_cxl_add_dport(port, &pdev->dev, port_num, map.resource); + dport = devm_cxl_add_dport(port, &pdev->dev, port_num, CXL_RESOURCE_NONE); if (IS_ERR(dport)) { ctx->error = PTR_ERR(dport); return PTR_ERR(dport); diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index b406ba64f6bc..04617ab825ee 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -626,6 +626,8 @@ static void unregister_port(void *_port) lock_dev = &parent->dev; device_lock_assert(lock_dev); + if (port->parent_dport && !port->parent_dport->rch) + cxl_register_map_reset(&port->parent_dport->reg_map); port->dead = true; device_unregister(&port->dev); } @@ -820,13 +822,39 @@ int cxl_port_setup_regs(struct cxl_port *port) } EXPORT_SYMBOL_NS_GPL(cxl_port_setup_regs, "CXL"); -static int cxl_dport_setup_regs(struct device *host, struct cxl_dport *dport, - resource_size_t component_reg_phys) +/** + * cxl_dport_setup_regs - probe all component registers of a cxl dport + * @dport: target cxl dport + */ +int cxl_dport_setup_regs(struct cxl_dport *dport) { + resource_size_t component_reg_phys; + struct device *host; int rc; if (dev_is_platform(dport->dport_dev)) return 0; + /* component registers have been set up */ + if (dport->reg_map.resource != CXL_RESOURCE_NONE) + return 0; + + if (dport->rcrb.base) { + component_reg_phys = __rcrb_to_component(dport->dport_dev, &dport->rcrb, + CXL_RCRB_DOWNSTREAM); + host = NULL; + } else { + component_reg_phys = find_component_registers(dport->dport_dev); + host = &dport->port->dev; + } + + if (component_reg_phys == CXL_RESOURCE_NONE) { + dev_warn(dport->dport_dev, "Invalid Component Registers%s", + dport->rcrb.base ? " in RCRB" : ""); + return -ENXIO; + } + + dev_dbg(dport->dport_dev, "Component Registers found for dport: %pa\n", + &component_reg_phys); /* * use @dport->dport_dev for the context for error messages during @@ -838,6 +866,7 @@ static int cxl_dport_setup_regs(struct device *host, struct cxl_dport *dport, dport->reg_map.host = host; return rc; } +EXPORT_SYMBOL_NS_GPL(cxl_dport_setup_regs, "CXL"); DEFINE_SHOW_ATTRIBUTE(einj_cxl_available_error_type); @@ -1176,39 +1205,25 @@ __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev, if (!dport) return ERR_PTR(-ENOMEM); + cxl_register_map_reset(&dport->reg_map); dport->dport_dev = dport_dev; dport->port_id = port_id; dport->port = port; - if (rcrb == CXL_RESOURCE_NONE) { - rc = cxl_dport_setup_regs(&port->dev, dport, - component_reg_phys); - if (rc) - return ERR_PTR(rc); - } else { + if (rcrb != CXL_RESOURCE_NONE) { dport->rcrb.base = rcrb; - component_reg_phys = __rcrb_to_component(dport_dev, &dport->rcrb, - CXL_RCRB_DOWNSTREAM); - if (component_reg_phys == CXL_RESOURCE_NONE) { - dev_warn(dport_dev, "Invalid Component Registers in RCRB"); - return ERR_PTR(-ENXIO); - } /* * RCH @dport is not ready to map until associated with its * memdev */ - rc = cxl_dport_setup_regs(NULL, dport, component_reg_phys); + rc = cxl_dport_setup_regs(dport); if (rc) return ERR_PTR(rc); dport->rch = true; } - if (component_reg_phys != CXL_RESOURCE_NONE) - dev_dbg(dport_dev, "Component Registers found for dport: %pa\n", - &component_reg_phys); - cond_cxl_root_lock(port); rc = add_dport(port, dport); cond_cxl_root_unlock(port); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 7763be02ef81..601818861441 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -773,6 +773,7 @@ struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_port *port, resource_size_t rcrb); int cxl_port_setup_regs(struct cxl_port *port); +int cxl_dport_setup_regs(struct cxl_dport *dport); #ifdef CONFIG_PCIEAER_CXL void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport); diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index c59f198f6cb0..ec135628edcb 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -100,7 +100,7 @@ static int cxl_switch_port_setup_hdm(struct cxl_port *port) return -ENXIO; } -static int cxl_switch_port_setup(struct cxl_port *port) +static int cxl_switch_port_setup(struct cxl_port *port, struct cxl_dport *dport) { int rc; @@ -108,12 +108,16 @@ static int cxl_switch_port_setup(struct cxl_port *port) if (rc) return rc; - return cxl_switch_port_setup_hdm(port); + rc = cxl_switch_port_setup_hdm(port); + if (rc) + return rc; + + return cxl_dport_setup_regs(dport); } static int cxl_endpoint_port_probe(struct cxl_port *port) { - struct cxl_port *iter, *parent_port = to_cxl_port(port->dev.parent); + struct cxl_port *iter, *prev, *parent_port = to_cxl_port(port->dev.parent); struct cxl_endpoint_dvsec_info info = { .port = port }; struct cxl_memdev *cxlmd = to_cxl_memdev(port->uport_dev); struct cxl_dev_state *cxlds = cxlmd->cxlds; @@ -121,8 +125,10 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) struct cxl_port *root; int rc; - for (iter = parent_port; !is_cxl_root(iter); - iter = to_cxl_port(iter->dev.parent)) { + for (iter = parent_port, prev = NULL; !is_cxl_root(iter); + prev = iter, iter = to_cxl_port(iter->dev.parent)) { + struct cxl_dport *dport = prev ? prev->parent_dport : + port->parent_dport; /* * The parent port of endpoint has been locked * during endpoint port attaching. @@ -131,9 +137,9 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) */ if (iter != parent_port) { guard(device)(&iter->dev); - rc = cxl_switch_port_setup(iter); + rc = cxl_switch_port_setup(iter, dport); } else { - rc = cxl_switch_port_setup(iter); + rc = cxl_switch_port_setup(iter, dport); } if (rc) return rc; From patchwork Tue Jan 7 03:57:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Ming X-Patchwork-Id: 13928206 Received: from sender4-pp-o94.zoho.com (sender4-pp-o94.zoho.com [136.143.188.94]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 228318248D; Tue, 7 Jan 2025 03:58:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.94 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222337; cv=pass; b=Tatxv38odgC6/3Xbrf01xVoGliF8FwHw/17TdbI6MZZZGsQ4Tca52MEHwpJB+xv16LmPyB5Fb4lBKEVcaG0J3TFdAKof4hvCIcUZDOXzrde9Sr4POsfM3KWT2lMEVUZ0829Grk+hsA54y/neiavwIkvVa/2C+00Drc0dDta1h3Q= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222337; c=relaxed/simple; bh=vmE2HGufo81XaUlVPpvMfRseeHPjcv3rSy6iL5FRr1w=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Yfm4kHgqRxsTXKFFRq91mdZ0F3t3Opn0kPz/1rSU6S9TBtwDz6G4ChrG6NxM1vyCYqc+lQcfmMQQX0+Y2vstk3kCdGKfeWOwlRnDo79zKYGackjckRExoib4gcWAFyCrb/afN3IfPW7w1T999iCa3y4Iiieflr0OvhSiIPuPMUg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com; spf=pass smtp.mailfrom=zohomail.com; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b=LV9drbLw; arc=pass smtp.client-ip=136.143.188.94 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zohomail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b="LV9drbLw" ARC-Seal: i=1; a=rsa-sha256; t=1736222302; cv=none; d=zohomail.com; s=zohoarc; b=hqJS6wcFaBynaaAZ/bQ+b2P0CoM2okL9QSeebROyOHbU2UmH0JTKBxgEsS9kEPwNVwZje8K9nEtzlZSx6e4SHcHHSZR+TgBqOKMxS4EWk54bNNOX4qHQ4mUbOS+URz3460pFrADwujShHpUzFDAdpaHQ2k+HbIiSPnON4jfin2w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1736222302; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=QsagTF1sqpb9JvFymmJIf9sxdVQ5YelFSbEClzxoVbY=; b=WXM7/gcrpgjlkBBaoVEeVZQs7vDIagb0FhC8VvkROfBB4igK5RlNf/0i9cK1QN1LuQ3Vxj97ACSOmRpo+PW3fI7KF4xrwUcrGMmivI+GDlSAsoeQ+UufozFgWt4zTqlYyxqYrhaaDjjxlbPB4oTqnlegYLcoOnUCtq3NXmayqFg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zohomail.com; spf=pass smtp.mailfrom=ming.li@zohomail.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1736222302; s=zm2022; d=zohomail.com; i=ming.li@zohomail.com; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-Id:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Feedback-ID:Reply-To; bh=QsagTF1sqpb9JvFymmJIf9sxdVQ5YelFSbEClzxoVbY=; b=LV9drbLwNlE29pTAnnjIP6DVztIOqAM++eSoQr18neIw5MNaU18u9H0nctGPwIRQ OG9v9mkbxSHHqDHn8dJ5H9laFIPSDckJJnWo6FfLjUm9xAArQ9gBER8kVlWTpnZJhaH nEGUOmIdq2+92jG1+2Z9yBnMevzBfmK0fS1I8gSQ= Received: by mx.zohomail.com with SMTPS id 173622230031819.636274695121756; Mon, 6 Jan 2025 19:58:20 -0800 (PST) From: Li Ming To: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Li Ming Subject: [RFC PATCH 7/8] cxl/port: Remove component_reg_phys parameter from devm_cxl_add_dport() Date: Tue, 7 Jan 2025 11:57:07 +0800 Message-Id: <20250107035708.1134954-8-ming.li@zohomail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250107035708.1134954-1-ming.li@zohomail.com> References: <20250107035708.1134954-1-ming.li@zohomail.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Feedback-ID: rr080112272d67b17ffbb4aedb913b19aa000075ee5274852ffe2fd7bb652e400f7ecc65588584397b8c7665:zu0801122786aa572f67f866847da1255e0000260cd114d33cac91e36c3a900115d37d0469cdcb51fb098caa:rf0801122d131082eb8a0061df18c9ab1100003c3db1610d0a5096f96390658d24bf686a00f62481b89c541b5a8feadbe96d:ZohoMail X-ZohoMailClient: External dport component registers setup will be triggered in cxl_endpoint_port_probe() when the first endpoint attaching. devm_cxl_add_dport() is not responsible for dport component registers setup anymore, so the component_reg_phys parameter of devm_cxl_add_dport() is unnecessary. Signed-off-by: Li Ming --- drivers/cxl/acpi.c | 3 +-- drivers/cxl/core/pci.c | 2 +- drivers/cxl/core/port.c | 13 +++++-------- drivers/cxl/cxl.h | 4 ++-- tools/testing/cxl/test/cxl.c | 4 +--- tools/testing/cxl/test/mock.c | 3 +-- 6 files changed, 11 insertions(+), 18 deletions(-) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index b754c2c55659..7efd650ab23f 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -596,8 +596,7 @@ static int add_host_bridge_dport(struct device *match, void *arg) dport = devm_cxl_add_rch_dport(root_port, bridge, ctx.uid, ctx.base); } else { - dport = devm_cxl_add_dport(root_port, bridge, ctx.uid, - CXL_RESOURCE_NONE); + dport = devm_cxl_add_dport(root_port, bridge, ctx.uid); } if (IS_ERR(dport)) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 3b22f3033484..7824f05713a6 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -51,7 +51,7 @@ static int match_add_dports(struct pci_dev *pdev, void *data) return 0; port_num = FIELD_GET(PCI_EXP_LNKCAP_PN, lnkcap); - dport = devm_cxl_add_dport(port, &pdev->dev, port_num, CXL_RESOURCE_NONE); + dport = devm_cxl_add_dport(port, &pdev->dev, port_num); if (IS_ERR(dport)) { ctx->error = PTR_ERR(dport); return PTR_ERR(dport); diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 04617ab825ee..b879520ef3cd 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1178,8 +1178,7 @@ static void cxl_dport_unlink(void *data) static struct cxl_dport * __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev, - int port_id, resource_size_t component_reg_phys, - resource_size_t rcrb) + int port_id, resource_size_t rcrb) { char link_name[CXL_TARGET_STRLEN]; struct cxl_dport *dport; @@ -1256,20 +1255,19 @@ __devm_cxl_add_dport(struct cxl_port *port, struct device *dport_dev, * @port: the cxl_port that references this dport * @dport_dev: firmware or PCI device representing the dport * @port_id: identifier for this dport in a decoder's target list - * @component_reg_phys: optional location of CXL component registers * * Note that dports are appended to the devm release action's of the * either the port's host (for root ports), or the port itself (for * switch ports) */ struct cxl_dport *devm_cxl_add_dport(struct cxl_port *port, - struct device *dport_dev, int port_id, - resource_size_t component_reg_phys) + struct device *dport_dev, + int port_id) { struct cxl_dport *dport; dport = __devm_cxl_add_dport(port, dport_dev, port_id, - component_reg_phys, CXL_RESOURCE_NONE); + CXL_RESOURCE_NONE); if (IS_ERR(dport)) { dev_dbg(dport_dev, "failed to add dport to %s: %ld\n", dev_name(&port->dev), PTR_ERR(dport)); @@ -1302,8 +1300,7 @@ struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_port *port, return ERR_PTR(-EINVAL); } - dport = __devm_cxl_add_dport(port, dport_dev, port_id, - CXL_RESOURCE_NONE, rcrb); + dport = __devm_cxl_add_dport(port, dport_dev, port_id, rcrb); if (IS_ERR(dport)) { dev_dbg(dport_dev, "failed to add RCH dport to %s: %ld\n", dev_name(&port->dev), PTR_ERR(dport)); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 601818861441..64a1199786f2 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -766,8 +766,8 @@ struct cxl_port *cxl_mem_find_port(struct cxl_memdev *cxlmd, bool schedule_cxl_memdev_detach(struct cxl_memdev *cxlmd); struct cxl_dport *devm_cxl_add_dport(struct cxl_port *port, - struct device *dport, int port_id, - resource_size_t component_reg_phys); + struct device *dport, + int port_id); struct cxl_dport *devm_cxl_add_rch_dport(struct cxl_port *port, struct device *dport_dev, int port_id, resource_size_t rcrb); diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c index d0337c11f9ee..4639ee990e6e 100644 --- a/tools/testing/cxl/test/cxl.c +++ b/tools/testing/cxl/test/cxl.c @@ -968,9 +968,7 @@ static int mock_cxl_port_enumerate_dports(struct cxl_port *port) continue; } - dport = devm_cxl_add_dport(port, &pdev->dev, pdev->id, - CXL_RESOURCE_NONE); - + dport = devm_cxl_add_dport(port, &pdev->dev, pdev->id); if (IS_ERR(dport)) return PTR_ERR(dport); } diff --git a/tools/testing/cxl/test/mock.c b/tools/testing/cxl/test/mock.c index 450c7566c33f..bcaab0150087 100644 --- a/tools/testing/cxl/test/mock.c +++ b/tools/testing/cxl/test/mock.c @@ -254,8 +254,7 @@ struct cxl_dport *__wrap_devm_cxl_add_rch_dport(struct cxl_port *port, struct cxl_mock_ops *ops = get_cxl_mock_ops(&index); if (ops && ops->is_mock_port(dport_dev)) { - dport = devm_cxl_add_dport(port, dport_dev, port_id, - CXL_RESOURCE_NONE); + dport = devm_cxl_add_dport(port, dport_dev, port_id); if (!IS_ERR(dport)) { dport->rcrb.base = rcrb; dport->rch = true; From patchwork Tue Jan 7 03:57:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Ming X-Patchwork-Id: 13928207 Received: from sender4-pp-o94.zoho.com (sender4-pp-o94.zoho.com [136.143.188.94]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB8E913B5AE; Tue, 7 Jan 2025 03:59:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.94 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222345; cv=pass; b=TdIVpT4WVnkIt2SWzs2VE1vjFFT2XPziinpq43K+33nCTDXs4cI7ugN8wFAHlXXJaE5O0C4JxYkYUtZ48z8j2ykNhFe3Q9hvhFZsNIxr7AyzaODMJC1ZfOsBGpvE/oQj10CuYq6tKubg+K5hzci/au+ue6/n8tCGaYL4tgxj7/U= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736222345; c=relaxed/simple; bh=XV1Sm6zlbGQZLKWsYSFwTAf2+p//RWPpGqQk/f5Gac4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BqzC45MBTmdJdi72G3enaOLo+7Aq7mRA6qoUZLIbbnuI+t0NghnQjzZyBPkaxb4Ea+aM/VCzsfJ7Quw5EZbWkJ4ZmYuDukBALLVWsEsM7mRi7L8NtBjKAM06K2feT01ze3cjiuQ8a+OVGrQ0YIHWdx+2aC3nUkw19sIO0J7M6RM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com; spf=pass smtp.mailfrom=zohomail.com; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b=Wd4LHNkW; arc=pass smtp.client-ip=136.143.188.94 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zohomail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b="Wd4LHNkW" ARC-Seal: i=1; a=rsa-sha256; t=1736222312; cv=none; d=zohomail.com; s=zohoarc; b=PelTFagZT8Ce0BvQapzhgwy5dCmgNwERabRVwa0vwcdIugWdLQsSSAi7BALqUUoY53V/4UWutskbrOSKcM7OQi6awiR4xs2djg6LLKD1VQHidjdHE9v51gpmdQseoctdGSvYbB2LyXEKaagVx4u4GOqCeyHSmMz23gXptOvNq7A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1736222312; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=M2mXgobPS3ES8nLPgPspoAhdDoBwUuAj8Zp/1t1ZuSI=; b=RPIgqXY/SghI8jUj8bCdFEkxoYj9zGviLY+33S8yBpcxLSIde6KM8EtnHflq557uafPEKlXjy4PucucH5jTbqTqgLLMJwBU9CbgJjhW0oYDjXIMnatmrRycsZGFs/u484hRE3+ZNwYFpWUwlGGaN84xrLePQmOk0PK8UpkSiijE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zohomail.com; spf=pass smtp.mailfrom=ming.li@zohomail.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1736222312; s=zm2022; d=zohomail.com; i=ming.li@zohomail.com; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-Id:Message-Id:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Feedback-ID:Reply-To; bh=M2mXgobPS3ES8nLPgPspoAhdDoBwUuAj8Zp/1t1ZuSI=; b=Wd4LHNkW+vQOdHo3p9QHZ+QR1PZGqQMAOR3jBpNhVCoYXyZgtAi0SMcmx5cQPbTH YisDB3iYNW+oU7BZHT+LTLrXVVBzySIrRlbk0e3adxYhglQlH4/z1LCYQDNb6A/0OxX TKg5mLC2nG4z20XFe6F2VSbQgtO6pNSN9Jtnjdgg= Received: by mx.zohomail.com with SMTPS id 1736222310497623.7130777416584; Mon, 6 Jan 2025 19:58:30 -0800 (PST) From: Li Ming To: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Li Ming Subject: [RFC PATCH 8/8] cxl/mem: Adjust cxl_dport_init_ras_reporting() invoked position Date: Tue, 7 Jan 2025 11:57:08 +0800 Message-Id: <20250107035708.1134954-9-ming.li@zohomail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250107035708.1134954-1-ming.li@zohomail.com> References: <20250107035708.1134954-1-ming.li@zohomail.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Feedback-ID: rr080112273613dc2fa3c07da19fe930d70000a3c615016a20dea8f05d19ef791ed4dac6877394e8be151f91:zu0801122744e16cdb5a5e17798d1f6d5000001d34f13629b2506cdd426258d2e3c1cc7f01454deb8cea5765:rf0801122d850d40504611565227a893d80000869a4f3b15f442da3b1e631244bbad6e2e0d8bdb1c25c989bd7ecdde68bbac:ZohoMail X-ZohoMailClient: External component registers of endpoint's parent dport is set up in devm_cxl_add_endpoint(), so cxl_dport_init_ras_reporting() should be only invoked after that. Signed-off-by: Li Ming --- drivers/cxl/mem.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 39da99a4bf4d..73b5a28db8cb 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -79,6 +79,8 @@ static int devm_cxl_add_endpoint(struct device *host, struct cxl_memdev *cxlmd, return -ENXIO; } + cxl_dport_init_ras_reporting(parent_dport, &cxlmd->dev); + return 0; } @@ -165,8 +167,6 @@ static int cxl_mem_probe(struct device *dev) else endpoint_parent = &parent_port->dev; - cxl_dport_init_ras_reporting(dport, dev); - scoped_guard(device, endpoint_parent) { if (!endpoint_parent->driver) { dev_err(dev, "CXL port topology %s not enabled\n",