From patchwork Wed Feb 8 19:20:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13133618 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87528C636D3 for ; Wed, 8 Feb 2023 19:20:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231770AbjBHTU4 (ORCPT ); Wed, 8 Feb 2023 14:20:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230337AbjBHTU4 (ORCPT ); Wed, 8 Feb 2023 14:20:56 -0500 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2FBDF51C60 for ; Wed, 8 Feb 2023 11:20:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675884055; x=1707420055; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KEQoi+p6qL7LGo9+vJjFVHdrgXYd/XM/Zv401xq5arQ=; b=eIEAbD/LGurwvVFw5XHj3+Detzl55PpiYabrKZj/v1+ezFlKQ2ISOWti 0lZZU65ussdrvOzci8cBLULedcRp+34WyXw5uTcyIu9Y0YhNDAfoHvtyx tOAPT7UIOiA96cctL01ZxB3xCUJskgrB07o7O8qbV1o0prmSUHztF/Cti iBZKlYaMCgcbSlQIlQ6Wpkw5fMR3BDB0xlGg+DXgEJKctPzxm84a7XnIb CBx3hncHse/3V8Yi2pnd4MJU0BSLn3ZatI7MCark2xJzEm7qzHRQ0ufCc 9ciP7pimNT5u1/59dq0YZaxg5A4qubHujHmikN2ItHdWB3OFEGVX+FeQC Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10615"; a="329925982" X-IronPort-AV: E=Sophos;i="5.97,281,1669104000"; d="scan'208";a="329925982" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2023 11:20:54 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10615"; a="776164923" X-IronPort-AV: E=Sophos;i="5.97,281,1669104000"; d="scan'208";a="776164923" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.48.215]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2023 11:20:54 -0800 Subject: [PATCH v4 1/7] cxl: break out range register decoding from cxl_hdm_decode_init() From: Dave Jiang To: dan.j.williams@intel.com Cc: Jonathan Cameron , linux-cxl@vger.kernel.org Date: Wed, 08 Feb 2023 12:20:52 -0700 Message-ID: <167588405143.1155956.12227802555070949872.stgit@djiang5-mobl3.local> In-Reply-To: <167588394236.1155956.8466475582138210344.stgit@djiang5-mobl3.local> References: <167588394236.1155956.8466475582138210344.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org There are 2 scenarios that requires additional handling. 1. A device that has active ranges in DVSEC range registers (RR) but no HDM decoder register block. 2. A device that has both RR active and HDM, but the HDM decoders are not programmed. The goal is to create emulated decoder software structs based on the RR. Move the CXL DVSEC range register decoding code block from cxl_hdm_decode_init() to its own function. Refactor code in preparation for the HDM decoder emulation. There is no functionality change to the code. Name the new function to cxl_dvsec_rr_decode(). The only change is to set range->start and range->end to CXL_RESOURCE_NONE and skipping the reading of base registers if the range size is 0, which equates to range not active. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- v4: - Init invalid range start to 0. (Dan) v2: - Refactor to continue when size is 0. (Jonathan) --- drivers/cxl/core/pci.c | 63 ++++++++++++++++++++++++++++++------------------ 1 file changed, 40 insertions(+), 23 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 57764e9cd19d..2ef03161a8bb 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -141,11 +141,10 @@ int cxl_await_media_ready(struct cxl_dev_state *cxlds) } EXPORT_SYMBOL_NS_GPL(cxl_await_media_ready, CXL); -static int wait_for_valid(struct cxl_dev_state *cxlds) +static int wait_for_valid(struct pci_dev *pdev, int d) { - struct pci_dev *pdev = to_pci_dev(cxlds->dev); - int d = cxlds->cxl_dvsec, rc; u32 val; + int rc; /* * Memory_Info_Valid: When set, indicates that the CXL Range 1 Size high @@ -334,20 +333,11 @@ static bool __cxl_hdm_decode_init(struct cxl_dev_state *cxlds, return true; } -/** - * cxl_hdm_decode_init() - Setup HDM decoding for the endpoint - * @cxlds: Device state - * @cxlhdm: Mapped HDM decoder Capability - * - * Try to enable the endpoint's HDM Decoder Capability - */ -int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm) +static int cxl_dvsec_rr_decode(struct pci_dev *pdev, int d, + struct cxl_endpoint_dvsec_info *info) { - struct pci_dev *pdev = to_pci_dev(cxlds->dev); - struct cxl_endpoint_dvsec_info info = { 0 }; int hdm_count, rc, i, ranges = 0; struct device *dev = &pdev->dev; - int d = cxlds->cxl_dvsec; u16 cap, ctrl; if (!d) { @@ -378,7 +368,7 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm) if (!hdm_count || hdm_count > 2) return -EINVAL; - rc = wait_for_valid(cxlds); + rc = wait_for_valid(pdev, d); if (rc) { dev_dbg(dev, "Failure awaiting MEM_INFO_VALID (%d)\n", rc); return rc; @@ -389,9 +379,9 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm) * disabled, and they will remain moot after the HDM Decoder * capability is enabled. */ - info.mem_enabled = FIELD_GET(CXL_DVSEC_MEM_ENABLE, ctrl); - if (!info.mem_enabled) - goto hdm_init; + info->mem_enabled = FIELD_GET(CXL_DVSEC_MEM_ENABLE, ctrl); + if (!info->mem_enabled) + return 0; for (i = 0; i < hdm_count; i++) { u64 base, size; @@ -410,6 +400,13 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm) return rc; size |= temp & CXL_DVSEC_MEM_SIZE_LOW_MASK; + if (!size) { + info->dvsec_range[i] = (struct range) { + .start = 0, + .end = CXL_RESOURCE_NONE, + }; + continue; + } rc = pci_read_config_dword( pdev, d + CXL_DVSEC_RANGE_BASE_HIGH(i), &temp); @@ -425,22 +422,42 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm) base |= temp & CXL_DVSEC_MEM_BASE_LOW_MASK; - info.dvsec_range[i] = (struct range) { + info->dvsec_range[i] = (struct range) { .start = base, .end = base + size - 1 }; - if (size) - ranges++; + ranges++; } - info.ranges = ranges; + info->ranges = ranges; + + return 0; +} + +/** + * cxl_hdm_decode_init() - Setup HDM decoding for the endpoint + * @cxlds: Device state + * @cxlhdm: Mapped HDM decoder Capability + * + * Try to enable the endpoint's HDM Decoder Capability + */ +int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm) +{ + struct pci_dev *pdev = to_pci_dev(cxlds->dev); + struct cxl_endpoint_dvsec_info info = { 0 }; + struct device *dev = &pdev->dev; + int d = cxlds->cxl_dvsec; + int rc; + + rc = cxl_dvsec_rr_decode(pdev, d, &info); + if (rc < 0) + return rc; /* * If DVSEC ranges are being used instead of HDM decoder registers there * is no use in trying to manage those. */ -hdm_init: if (!__cxl_hdm_decode_init(cxlds, cxlhdm, &info)) { dev_err(dev, "Legacy range registers configuration prevents HDM operation.\n"); From patchwork Wed Feb 8 19:21:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13133620 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1FFC2C636D3 for ; Wed, 8 Feb 2023 19:21:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229953AbjBHTVJ (ORCPT ); Wed, 8 Feb 2023 14:21:09 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36738 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231789AbjBHTVI (ORCPT ); Wed, 8 Feb 2023 14:21:08 -0500 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 83C5551C5C for ; Wed, 8 Feb 2023 11:21:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675884067; x=1707420067; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=rioNFLBlxGEFX9dgoav8UzHt+a6WGcx7DMURF3IgWxI=; b=VcKl/uj780D+poCJYzhzunsfKkz7/Iy2+n4LHiJGUDzYRmwrLZq7T7mP YyuFNE+PlM4bizQeNWMnGuYXklwK6JP1S8NbBIIRE3VMRGLCVK0oXyA2B Vzi+BGIxpkN3hrbT7MLApNGX3B6ZwUkos41FUvgzG8lyFkW4jC7p3gzhz 8nY2FmktsMrARAFWGBfOB0vAWJrOHuPd6dqx3XKp899AtKM5Pbu0c4srB UxKEao/aDdV/0dSTc2aNEPUQWTSQkUMhAiWB85q779uZkXvhKQy10IoAY SBwZhI4N9cKNbkzTZeSAfrqSyMbmXfw64QkYtkakm0CW08125CEO0sOxI A==; X-IronPort-AV: E=McAfee;i="6500,9779,10615"; a="329926054" X-IronPort-AV: E=Sophos;i="5.97,281,1669104000"; d="scan'208";a="329926054" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2023 11:21:07 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10615"; a="776164995" X-IronPort-AV: E=Sophos;i="5.97,281,1669104000"; d="scan'208";a="776164995" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.48.215]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2023 11:21:05 -0800 Subject: [PATCH v4 2/7] cxl: export cxl_dvsec_rr_decode() to cxl_port From: Dave Jiang To: dan.j.williams@intel.com Cc: Jonathan Cameron , linux-cxl@vger.kernel.org Date: Wed, 08 Feb 2023 12:21:01 -0700 Message-ID: <167588405993.1155956.13692778892491820148.stgit@djiang5-mobl3.local> In-Reply-To: <167588394236.1155956.8466475582138210344.stgit@djiang5-mobl3.local> References: <167588394236.1155956.8466475582138210344.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Call cxl_dvsec_rr_decode() in the beginning of cxl_port_probe() and preserve the decoded information in a local 'struct cxl_endpoint_dvsec_info'. This info can be passed to various functions later on in order to support the HDM decoder emulation. The invocation of cxl_dvsec_rr_decode() in cxl_hdm_decode_init() is removed and a pointer to the 'struct cxl_endpoint_dvsec_info' is passed in. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- v2: - Update kdoc comments (Jonathan) - Use a bool for is_cxl_endpoint() to make it easier for static analysis (Jonathan) --- drivers/cxl/core/pci.c | 18 +++++++----------- drivers/cxl/cxl.h | 14 ++++++++++++++ drivers/cxl/cxlmem.h | 12 ------------ drivers/cxl/cxlpci.h | 3 ++- drivers/cxl/port.c | 21 ++++++++++++++------- 5 files changed, 37 insertions(+), 31 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 2ef03161a8bb..a76735393b38 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -333,8 +333,8 @@ static bool __cxl_hdm_decode_init(struct cxl_dev_state *cxlds, return true; } -static int cxl_dvsec_rr_decode(struct pci_dev *pdev, int d, - struct cxl_endpoint_dvsec_info *info) +int cxl_dvsec_rr_decode(struct pci_dev *pdev, int d, + struct cxl_endpoint_dvsec_info *info) { int hdm_count, rc, i, ranges = 0; struct device *dev = &pdev->dev; @@ -434,31 +434,27 @@ static int cxl_dvsec_rr_decode(struct pci_dev *pdev, int d, return 0; } +EXPORT_SYMBOL_NS_GPL(cxl_dvsec_rr_decode, CXL); /** * cxl_hdm_decode_init() - Setup HDM decoding for the endpoint * @cxlds: Device state * @cxlhdm: Mapped HDM decoder Capability + * @info: Cached DVSEC range registers info * * Try to enable the endpoint's HDM Decoder Capability */ -int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm) +int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, + struct cxl_endpoint_dvsec_info *info) { struct pci_dev *pdev = to_pci_dev(cxlds->dev); - struct cxl_endpoint_dvsec_info info = { 0 }; struct device *dev = &pdev->dev; - int d = cxlds->cxl_dvsec; - int rc; - - rc = cxl_dvsec_rr_decode(pdev, d, &info); - if (rc < 0) - return rc; /* * If DVSEC ranges are being used instead of HDM decoder registers there * is no use in trying to manage those. */ - if (!__cxl_hdm_decode_init(cxlds, cxlhdm, &info)) { + if (!__cxl_hdm_decode_init(cxlds, cxlhdm, info)) { dev_err(dev, "Legacy range registers configuration prevents HDM operation.\n"); return -EBUSY; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 1b1cf459ac77..1057affb2db0 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -630,10 +630,24 @@ int cxl_decoder_add_locked(struct cxl_decoder *cxld, int *target_map); int cxl_decoder_autoremove(struct device *host, struct cxl_decoder *cxld); int cxl_endpoint_autoremove(struct cxl_memdev *cxlmd, struct cxl_port *endpoint); +/** + * struct cxl_endpoint_dvsec_info - Cached DVSEC info + * @mem_enabled: cached value of mem_enabled in the DVSEC, PCIE_DEVICE + * @ranges: Number of active HDM ranges this device uses. + * @dvsec_range: cached attributes of the ranges in the DVSEC, PCIE_DEVICE + */ +struct cxl_endpoint_dvsec_info { + bool mem_enabled; + int ranges; + struct range dvsec_range[2]; +}; + struct cxl_hdm; struct cxl_hdm *devm_cxl_setup_hdm(struct cxl_port *port); int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm); int devm_cxl_add_passthrough_decoder(struct cxl_port *port); +int cxl_dvsec_rr_decode(struct pci_dev *pdev, int dvsec, + struct cxl_endpoint_dvsec_info *info); bool is_cxl_region(struct device *dev); diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index ab138004f644..187a310780a9 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -181,18 +181,6 @@ static inline int cxl_mbox_cmd_rc2errno(struct cxl_mbox_cmd *mbox_cmd) */ #define CXL_CAPACITY_MULTIPLIER SZ_256M -/** - * struct cxl_endpoint_dvsec_info - Cached DVSEC info - * @mem_enabled: cached value of mem_enabled in the DVSEC, PCIE_DEVICE - * @ranges: Number of active HDM ranges this device uses. - * @dvsec_range: cached attributes of the ranges in the DVSEC, PCIE_DEVICE - */ -struct cxl_endpoint_dvsec_info { - bool mem_enabled; - int ranges; - struct range dvsec_range[2]; -}; - /** * struct cxl_dev_state - The driver device state * diff --git a/drivers/cxl/cxlpci.h b/drivers/cxl/cxlpci.h index 920909791bb9..430e23345a16 100644 --- a/drivers/cxl/cxlpci.h +++ b/drivers/cxl/cxlpci.h @@ -64,6 +64,7 @@ enum cxl_regloc_type { int devm_cxl_port_enumerate_dports(struct cxl_port *port); struct cxl_dev_state; -int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm); +int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, + struct cxl_endpoint_dvsec_info *info); void read_cdat_data(struct cxl_port *port); #endif /* __CXL_PCI_H__ */ diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 5453771bf330..404639a1c3d0 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -32,12 +32,22 @@ static void schedule_detach(void *cxlmd) static int cxl_port_probe(struct device *dev) { + struct cxl_endpoint_dvsec_info info = { 0 }; struct cxl_port *port = to_cxl_port(dev); + bool is_ep = is_cxl_endpoint(port); + struct cxl_dev_state *cxlds; + struct cxl_memdev *cxlmd; struct cxl_hdm *cxlhdm; int rc; - - if (!is_cxl_endpoint(port)) { + if (is_ep) { + cxlmd = to_cxl_memdev(port->uport); + cxlds = cxlmd->cxlds; + rc = cxl_dvsec_rr_decode(to_pci_dev(cxlds->dev), + cxlds->cxl_dvsec, &info); + if (rc < 0) + return rc; + } else { rc = devm_cxl_port_enumerate_dports(port); if (rc < 0) return rc; @@ -49,10 +59,7 @@ static int cxl_port_probe(struct device *dev) if (IS_ERR(cxlhdm)) return PTR_ERR(cxlhdm); - if (is_cxl_endpoint(port)) { - struct cxl_memdev *cxlmd = to_cxl_memdev(port->uport); - struct cxl_dev_state *cxlds = cxlmd->cxlds; - + if (is_ep) { /* Cache the data early to ensure is_visible() works */ read_cdat_data(port); @@ -61,7 +68,7 @@ static int cxl_port_probe(struct device *dev) if (rc) return rc; - rc = cxl_hdm_decode_init(cxlds, cxlhdm); + rc = cxl_hdm_decode_init(cxlds, cxlhdm, &info); if (rc) return rc; From patchwork Wed Feb 8 19:21:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13133622 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCC13C636D6 for ; Wed, 8 Feb 2023 19:21:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230337AbjBHTVV (ORCPT ); Wed, 8 Feb 2023 14:21:21 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36982 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231800AbjBHTVU (ORCPT ); Wed, 8 Feb 2023 14:21:20 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1FFA35828F for ; Wed, 8 Feb 2023 11:21:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675884076; x=1707420076; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fdJO9PK91eaVvybNKx35iF36fc78DSsqi9jbCkewzfM=; b=lBBJRj61OTIEiSCh0ussNPWa//Iaqe0c/N0/HzggpKV8e5Uy/W6TOz5i WUuk4R6Gqky6akUQruCsEcRdBya/XwXF+iUb1CXv94lbRX526PJzTje/Y mLf5u9Be0hT7rw/6tjq+uGqnz1FW+iPAJuhd8NUcNUxWrsqB5Pr/Bl+Bg nvN27nudRnbMC2fvGmGPm9BW0G5k+eqK+YW+8Fpr0bLewfJvM0wVHGIZH GuoSFyb9Jp6N9MLmTnmG4Wm3pOlOg1Gr6ZXax161nlcn6ntF2FbBuoGrH kMhr2c18GEa6obKtSjG+ZYMq40g8mLcP04uqGzw+qV5tyW2Oj4hcb6M9K g==; X-IronPort-AV: E=McAfee;i="6500,9779,10615"; a="357298466" X-IronPort-AV: E=Sophos;i="5.97,281,1669104000"; d="scan'208";a="357298466" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2023 11:21:15 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10615"; a="730998477" X-IronPort-AV: E=Sophos;i="5.97,281,1669104000"; d="scan'208";a="730998477" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.48.215]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2023 11:21:15 -0800 Subject: [PATCH v4 3/7] cxl: refactor cxl_hdm_decode_init() From: Dave Jiang To: dan.j.williams@intel.com Cc: Jonathan Cameron , linux-cxl@vger.kernel.org Date: Wed, 08 Feb 2023 12:21:12 -0700 Message-ID: <167588407080.1155956.5454204009189141313.stgit@djiang5-mobl3.local> In-Reply-To: <167588394236.1155956.8466475582138210344.stgit@djiang5-mobl3.local> References: <167588394236.1155956.8466475582138210344.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org With the previous refactoring of DVSEC range registers out of cxl_hdm_decode_init(), it basically becomes a skeleton function. Squash __cxl_hdm_decode_init() with cxl_hdm_decode_init() to simplify the code. cxl_hdm_decode_init() now returns more error codes than just -EBUSY. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- v3: - Simplify to directly return devm_cxl_enable_mem() in cxl_hdm_decode_init(). (Jonathan) v2: - Update commit log to indicate cxl_hdm_decode_init() return additional error codes after change. (Jonathan) --- drivers/cxl/core/pci.c | 135 +++++++++++++++++++----------------------------- 1 file changed, 54 insertions(+), 81 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index a76735393b38..2e86c209e24d 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -259,80 +259,6 @@ static int devm_cxl_enable_hdm(struct device *host, struct cxl_hdm *cxlhdm) return devm_add_action_or_reset(host, disable_hdm, cxlhdm); } -static bool __cxl_hdm_decode_init(struct cxl_dev_state *cxlds, - struct cxl_hdm *cxlhdm, - struct cxl_endpoint_dvsec_info *info) -{ - void __iomem *hdm = cxlhdm->regs.hdm_decoder; - struct cxl_port *port = cxlhdm->port; - struct device *dev = cxlds->dev; - struct cxl_port *root; - int i, rc, allowed; - u32 global_ctrl; - - global_ctrl = readl(hdm + CXL_HDM_DECODER_CTRL_OFFSET); - - /* - * If the HDM Decoder Capability is already enabled then assume - * that some other agent like platform firmware set it up. - */ - if (global_ctrl & CXL_HDM_DECODER_ENABLE) { - rc = devm_cxl_enable_mem(&port->dev, cxlds); - if (rc) - return false; - return true; - } - - root = to_cxl_port(port->dev.parent); - while (!is_cxl_root(root) && is_cxl_port(root->dev.parent)) - root = to_cxl_port(root->dev.parent); - if (!is_cxl_root(root)) { - dev_err(dev, "Failed to acquire root port for HDM enable\n"); - return false; - } - - for (i = 0, allowed = 0; info->mem_enabled && i < info->ranges; i++) { - struct device *cxld_dev; - - cxld_dev = device_find_child(&root->dev, &info->dvsec_range[i], - dvsec_range_allowed); - if (!cxld_dev) { - dev_dbg(dev, "DVSEC Range%d denied by platform\n", i); - continue; - } - dev_dbg(dev, "DVSEC Range%d allowed by platform\n", i); - put_device(cxld_dev); - allowed++; - } - - if (!allowed) { - cxl_set_mem_enable(cxlds, 0); - info->mem_enabled = 0; - } - - /* - * Per CXL 2.0 Section 8.1.3.8.3 and 8.1.3.8.4 DVSEC CXL Range 1 Base - * [High,Low] when HDM operation is enabled the range register values - * are ignored by the device, but the spec also recommends matching the - * DVSEC Range 1,2 to HDM Decoder Range 0,1. So, non-zero info->ranges - * are expected even though Linux does not require or maintain that - * match. If at least one DVSEC range is enabled and allowed, skip HDM - * Decoder Capability Enable. - */ - if (info->mem_enabled) - return false; - - rc = devm_cxl_enable_hdm(&port->dev, cxlhdm); - if (rc) - return false; - - rc = devm_cxl_enable_mem(&port->dev, cxlds); - if (rc) - return false; - - return true; -} - int cxl_dvsec_rr_decode(struct pci_dev *pdev, int d, struct cxl_endpoint_dvsec_info *info) { @@ -449,18 +375,65 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, { struct pci_dev *pdev = to_pci_dev(cxlds->dev); struct device *dev = &pdev->dev; + void __iomem *hdm = cxlhdm->regs.hdm_decoder; + struct cxl_port *port = cxlhdm->port; + struct cxl_port *root; + int i, rc, allowed; + u32 global_ctrl; + + global_ctrl = readl(hdm + CXL_HDM_DECODER_CTRL_OFFSET); /* - * If DVSEC ranges are being used instead of HDM decoder registers there - * is no use in trying to manage those. + * If the HDM Decoder Capability is already enabled then assume + * that some other agent like platform firmware set it up. */ - if (!__cxl_hdm_decode_init(cxlds, cxlhdm, info)) { - dev_err(dev, - "Legacy range registers configuration prevents HDM operation.\n"); - return -EBUSY; + if (global_ctrl & CXL_HDM_DECODER_ENABLE) + return devm_cxl_enable_mem(&port->dev, cxlds); + + root = to_cxl_port(port->dev.parent); + while (!is_cxl_root(root) && is_cxl_port(root->dev.parent)) + root = to_cxl_port(root->dev.parent); + if (!is_cxl_root(root)) { + dev_err(dev, "Failed to acquire root port for HDM enable\n"); + return -ENODEV; } - return 0; + for (i = 0, allowed = 0; info->mem_enabled && i < info->ranges; i++) { + struct device *cxld_dev; + + cxld_dev = device_find_child(&root->dev, &info->dvsec_range[i], + dvsec_range_allowed); + if (!cxld_dev) { + dev_dbg(dev, "DVSEC Range%d denied by platform\n", i); + continue; + } + dev_dbg(dev, "DVSEC Range%d allowed by platform\n", i); + put_device(cxld_dev); + allowed++; + } + + if (!allowed) { + cxl_set_mem_enable(cxlds, 0); + info->mem_enabled = 0; + } + + /* + * Per CXL 2.0 Section 8.1.3.8.3 and 8.1.3.8.4 DVSEC CXL Range 1 Base + * [High,Low] when HDM operation is enabled the range register values + * are ignored by the device, but the spec also recommends matching the + * DVSEC Range 1,2 to HDM Decoder Range 0,1. So, non-zero info->ranges + * are expected even though Linux does not require or maintain that + * match. If at least one DVSEC range is enabled and allowed, skip HDM + * Decoder Capability Enable. + */ + if (info->mem_enabled) + return -EBUSY; + + rc = devm_cxl_enable_hdm(&port->dev, cxlhdm); + if (rc) + return rc; + + return devm_cxl_enable_mem(&port->dev, cxlds); } EXPORT_SYMBOL_NS_GPL(cxl_hdm_decode_init, CXL); From patchwork Wed Feb 8 19:21:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13133621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABEDCC05027 for ; Wed, 8 Feb 2023 19:21:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230393AbjBHTV3 (ORCPT ); Wed, 8 Feb 2023 14:21:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230463AbjBHTV2 (ORCPT ); Wed, 8 Feb 2023 14:21:28 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D1EAA58283 for ; Wed, 8 Feb 2023 11:21:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675884084; x=1707420084; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aoEfltS4GgF62g2DKrrfcBDhh11UB+LhYQwIo4p6yoo=; b=SwtRtm9ukAHltHHUU0UebvtcsiY39R5gzm4JhtY/dupnJRFXRU17zToR dX38gsWHbXoln0EDuNPCuT0gldXZutLrOpWlbftLhPFYC5XDUbzVrhNWm ItVq+bLvVX65TBV+Jp4oJkywrRLtMptwx+pp5gjqzvxvb93jMhAZThnmb ptaMZ8ObA1Zv1gHwvnN6esf77JIFR7OVRtJn0wEH3uGDrHKuDRZmh5P1S jBMLuB7KNaQjNbU7FzpIlfL/4Vv1Zq7t3f4Ha8X5yJNBYX4O0LySE0FP2 Z96oy+6+AA8BoZV6bYFGY/v6Mt9SZjsuk8R018uVZiQvlhjeDy7G+asxn g==; X-IronPort-AV: E=McAfee;i="6500,9779,10615"; a="357298505" X-IronPort-AV: E=Sophos;i="5.97,281,1669104000"; d="scan'208";a="357298505" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2023 11:21:24 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10615"; a="730998596" X-IronPort-AV: E=Sophos;i="5.97,281,1669104000"; d="scan'208";a="730998596" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.48.215]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2023 11:21:23 -0800 Subject: [PATCH v4 4/7] cxl: emulate HDM decoder from DVSEC range registers From: Dave Jiang To: dan.j.williams@intel.com Cc: Jonathan Cameron , linux-cxl@vger.kernel.org Date: Wed, 08 Feb 2023 12:21:21 -0700 Message-ID: <167588408051.1155956.11059438224070853644.stgit@djiang5-mobl3.local> In-Reply-To: <167588394236.1155956.8466475582138210344.stgit@djiang5-mobl3.local> References: <167588394236.1155956.8466475582138210344.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org In the case where HDM decoder register block exists but is not programmed and at the same time the DVSEC range register range is active, populate the CXL decoder object 'cxl_decoder' with info from DVSEC range registers. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- v4: - Direct assign range to range. (Dan) - Add comment for why emulated decoder is locked. (Dan) - Check range_len() instead of content of .start. (Dan) - Fix too long line formatting. (Dan) v2: - Set target_type to CXL_DECODER_EXPANDER (type 3). (Jonathan) - Skip HDM enabling if DVSEC range is active. (Jonathan) --- drivers/cxl/core/hdm.c | 36 +++++++++++++++++++++++++++++++++--- drivers/cxl/core/pci.c | 2 +- drivers/cxl/cxl.h | 3 ++- drivers/cxl/port.c | 2 +- 4 files changed, 37 insertions(+), 6 deletions(-) diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index dcc16d7cb8f3..c0f224454447 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -679,9 +679,34 @@ static int cxl_decoder_reset(struct cxl_decoder *cxld) return 0; } +static int cxl_setup_hdm_decoder_from_dvsec(struct cxl_port *port, + struct cxl_decoder *cxld, int which, + struct cxl_endpoint_dvsec_info *info) +{ + if (!is_cxl_endpoint(port)) + return -EOPNOTSUPP; + + if (!range_len(&info->dvsec_range[which])) + return -ENOENT; + + cxld->target_type = CXL_DECODER_EXPANDER; + cxld->commit = NULL; + cxld->reset = NULL; + cxld->hpa_range = info->dvsec_range[which]; + + /* + * Set the emulated decoder as locked pending additional support to + * change the range registers at run time. + */ + cxld->flags |= CXL_DECODER_F_ENABLE | CXL_DECODER_F_LOCK; + port->commit_end = cxld->id; + + return 0; +} + static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, int *target_map, void __iomem *hdm, int which, - u64 *dpa_base) + u64 *dpa_base, struct cxl_endpoint_dvsec_info *info) { struct cxl_endpoint_decoder *cxled = NULL; u64 size, base, skip, dpa_size; @@ -717,6 +742,9 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, .end = base + size - 1, }; + if (cxled && !committed && range_len(&info->dvsec_range[which])) + return cxl_setup_hdm_decoder_from_dvsec(port, cxld, which, info); + /* decoders are enabled if committed */ if (committed) { cxld->flags |= CXL_DECODER_F_ENABLE; @@ -790,7 +818,8 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, * devm_cxl_enumerate_decoders - add decoder objects per HDM register set * @cxlhdm: Structure to populate with HDM capabilities */ -int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm) +int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, + struct cxl_endpoint_dvsec_info *info) { void __iomem *hdm = cxlhdm->regs.hdm_decoder; struct cxl_port *port = cxlhdm->port; @@ -842,7 +871,8 @@ int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm) cxld = &cxlsd->cxld; } - rc = init_hdm_decoder(port, cxld, target_map, hdm, i, &dpa_base); + rc = init_hdm_decoder(port, cxld, target_map, hdm, i, + &dpa_base, info); if (rc) { put_device(&cxld->dev); return rc; diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 2e86c209e24d..2e0976446053 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -427,7 +427,7 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, * Decoder Capability Enable. */ if (info->mem_enabled) - return -EBUSY; + return 0; rc = devm_cxl_enable_hdm(&port->dev, cxlhdm); if (rc) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 1057affb2db0..ea9548cbc7eb 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -644,7 +644,8 @@ struct cxl_endpoint_dvsec_info { struct cxl_hdm; struct cxl_hdm *devm_cxl_setup_hdm(struct cxl_port *port); -int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm); +int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, + struct cxl_endpoint_dvsec_info *info); int devm_cxl_add_passthrough_decoder(struct cxl_port *port); int cxl_dvsec_rr_decode(struct pci_dev *pdev, int dvsec, struct cxl_endpoint_dvsec_info *info); diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 404639a1c3d0..7f1b71c5cf15 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -79,7 +79,7 @@ static int cxl_port_probe(struct device *dev) } } - rc = devm_cxl_enumerate_decoders(cxlhdm); + rc = devm_cxl_enumerate_decoders(cxlhdm, &info); if (rc) { dev_err(dev, "Couldn't enumerate decoders (%d)\n", rc); return rc; From patchwork Wed Feb 8 19:21:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13133623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7FCAC636D3 for ; Wed, 8 Feb 2023 19:21:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230080AbjBHTVg (ORCPT ); Wed, 8 Feb 2023 14:21:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230463AbjBHTVf (ORCPT ); Wed, 8 Feb 2023 14:21:35 -0500 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B893D51C5C for ; Wed, 8 Feb 2023 11:21:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675884092; x=1707420092; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=DqoLt8NKnIgkwrg4LAFgJX1TeX6OOaYnSK1ZabAA1zA=; b=NA160S5FWNDWLT6ukdEVzbDn/q47siSfuUU5wqbMhIdmHsXYAJBencxf NRFicaw82+KUTG69OGtjV5B9Fq3oEikLxh3oVWK8qv5mSoo8NXe23G+pK oSzq19TtRE1jG510p+2QRLVOca6Ou66vvgLlbi2aOAT6DuZrwj6f550nA j0IZz0E7y/jXBcYx9yEslKPUYj6Dr/PiHCKO8+wv1xYSXUoTBTZS1my1v N2oqp3V/dzGIhiX7TCbXkM72Hqoy9gIQkbmo3WK8cO/mw+5FNEw3Js6DM JXquveCbEM9qr1ANGOOMYlg++T57tsSMgtk/ZPb1ZcDVCoKuY7YT5V49w w==; X-IronPort-AV: E=McAfee;i="6500,9779,10615"; a="357298529" X-IronPort-AV: E=Sophos;i="5.97,281,1669104000"; d="scan'208";a="357298529" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2023 11:21:32 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10615"; a="730998659" X-IronPort-AV: E=Sophos;i="5.97,281,1669104000"; d="scan'208";a="730998659" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.48.215]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2023 11:21:32 -0800 Subject: [PATCH v4 5/7] cxl: create emulated cxl_hdm for devices that do not have HDM decoders From: Dave Jiang To: dan.j.williams@intel.com Cc: Jonathan Cameron , linux-cxl@vger.kernel.org Date: Wed, 08 Feb 2023 12:21:30 -0700 Message-ID: <167588408897.1155956.17261006145412620005.stgit@djiang5-mobl3.local> In-Reply-To: <167588394236.1155956.8466475582138210344.stgit@djiang5-mobl3.local> References: <167588394236.1155956.8466475582138210344.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org CXL rev3 spec 8.1.3 RCDs may not have HDM register blocks. Create a fake HDM with information from the CXL PCIe DVSEC registers. The decoder count will be set to the HDM count retrieved from the DVSEC cap register. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- v3: - Move relevant changes from next patch in series. (Jonathan) - Add kernel doc update. (Jonathan) v2: - Set target_count to same as number of ranges. (Jonathan) --- drivers/cxl/core/hdm.c | 58 ++++++++++++++++++++++++++++++++++++++++-------- drivers/cxl/core/pci.c | 9 +++++-- drivers/cxl/cxl.h | 3 ++ drivers/cxl/port.c | 2 +- 4 files changed, 57 insertions(+), 15 deletions(-) diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index c0f224454447..a49543f22dca 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -101,11 +101,34 @@ static int map_hdm_decoder_regs(struct cxl_port *port, void __iomem *crb, BIT(CXL_CM_CAP_CAP_ID_HDM)); } +static struct cxl_hdm *devm_cxl_setup_emulated_hdm(struct cxl_port *port, + struct cxl_endpoint_dvsec_info *info) +{ + struct device *dev = &port->dev; + struct cxl_hdm *cxlhdm; + + if (!info->mem_enabled) + return ERR_PTR(-ENODEV); + + cxlhdm = devm_kzalloc(dev, sizeof(*cxlhdm), GFP_KERNEL); + if (!cxlhdm) + return ERR_PTR(-ENOMEM); + + cxlhdm->port = port; + cxlhdm->decoder_count = info->ranges; + cxlhdm->target_count = info->ranges; + dev_set_drvdata(&port->dev, cxlhdm); + + return cxlhdm; +} + /** * devm_cxl_setup_hdm - map HDM decoder component registers * @port: cxl_port to map + * @info: cached DVSEC range register info */ -struct cxl_hdm *devm_cxl_setup_hdm(struct cxl_port *port) +struct cxl_hdm *devm_cxl_setup_hdm(struct cxl_port *port, + struct cxl_endpoint_dvsec_info *info) { struct device *dev = &port->dev; struct cxl_hdm *cxlhdm; @@ -119,6 +142,9 @@ struct cxl_hdm *devm_cxl_setup_hdm(struct cxl_port *port) cxlhdm->port = port; crb = ioremap(port->component_reg_phys, CXL_COMPONENT_REG_BLOCK_SIZE); if (!crb) { + if (info->mem_enabled) + return devm_cxl_setup_emulated_hdm(port, info); + dev_err(dev, "No component registers mapped\n"); return ERR_PTR(-ENXIO); } @@ -814,19 +840,15 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, return 0; } -/** - * devm_cxl_enumerate_decoders - add decoder objects per HDM register set - * @cxlhdm: Structure to populate with HDM capabilities - */ -int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, - struct cxl_endpoint_dvsec_info *info) +static void cxl_settle_decoders(struct cxl_hdm *cxlhdm) { void __iomem *hdm = cxlhdm->regs.hdm_decoder; - struct cxl_port *port = cxlhdm->port; - int i, committed; - u64 dpa_base = 0; + int committed, i; u32 ctrl; + if (!hdm) + return; + /* * Since the register resource was recently claimed via request_region() * be careful about trusting the "not-committed" status until the commit @@ -843,6 +865,22 @@ int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, /* ensure that future checks of committed can be trusted */ if (committed != cxlhdm->decoder_count) msleep(20); +} + +/** + * devm_cxl_enumerate_decoders - add decoder objects per HDM register set + * @cxlhdm: Structure to populate with HDM capabilities + * @info: cached DVSEC range register info + */ +int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, + struct cxl_endpoint_dvsec_info *info) +{ + void __iomem *hdm = cxlhdm->regs.hdm_decoder; + struct cxl_port *port = cxlhdm->port; + int i; + u64 dpa_base = 0; + + cxl_settle_decoders(cxlhdm); for (i = 0; i < cxlhdm->decoder_count; i++) { int target_map[CXL_DECODER_MAX_INTERLEAVE] = { 0 }; diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 2e0976446053..cfd869a1fa93 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -379,16 +379,19 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, struct cxl_port *port = cxlhdm->port; struct cxl_port *root; int i, rc, allowed; - u32 global_ctrl; + u32 global_ctrl = 0; - global_ctrl = readl(hdm + CXL_HDM_DECODER_CTRL_OFFSET); + if (hdm) + global_ctrl = readl(hdm + CXL_HDM_DECODER_CTRL_OFFSET); /* * If the HDM Decoder Capability is already enabled then assume * that some other agent like platform firmware set it up. */ - if (global_ctrl & CXL_HDM_DECODER_ENABLE) + if (global_ctrl & CXL_HDM_DECODER_ENABLE || (!hdm && info->mem_enabled)) return devm_cxl_enable_mem(&port->dev, cxlds); + else if (!hdm) + return -ENODEV; root = to_cxl_port(port->dev.parent); while (!is_cxl_root(root) && is_cxl_port(root->dev.parent)) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index ea9548cbc7eb..0ec047cced90 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -643,7 +643,8 @@ struct cxl_endpoint_dvsec_info { }; struct cxl_hdm; -struct cxl_hdm *devm_cxl_setup_hdm(struct cxl_port *port); +struct cxl_hdm *devm_cxl_setup_hdm(struct cxl_port *port, + struct cxl_endpoint_dvsec_info *info); int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, struct cxl_endpoint_dvsec_info *info); int devm_cxl_add_passthrough_decoder(struct cxl_port *port); diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 7f1b71c5cf15..875bf45db4ad 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -55,7 +55,7 @@ static int cxl_port_probe(struct device *dev) return devm_cxl_add_passthrough_decoder(port); } - cxlhdm = devm_cxl_setup_hdm(port); + cxlhdm = devm_cxl_setup_hdm(port, &info); if (IS_ERR(cxlhdm)) return PTR_ERR(cxlhdm); From patchwork Wed Feb 8 19:21:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13133624 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3074C64EC7 for ; Wed, 8 Feb 2023 19:21:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231841AbjBHTVz (ORCPT ); Wed, 8 Feb 2023 14:21:55 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231821AbjBHTVw (ORCPT ); Wed, 8 Feb 2023 14:21:52 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3975353E7A for ; Wed, 8 Feb 2023 11:21:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675884101; x=1707420101; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+VaGE5DKf6ZbLvGvyK94FdZgwMm1sqj4CnNcE4fZ0NE=; b=G3h7epq4SAnrSiIr5Z9F/ncRaR4JN+vRCe2YKxTbJjSZS9BVCqlcvE24 GIcgpT6UOF9JUQerH7fuiQ/VGy8j3DjoqXhjCDfBHlAUtzbCXPMtVB5wx +0nX81Pm0cerEVNWOhDI5UrEDtwh7YfzZWQC4Y9BKahikKlYnnWQACg/9 mdPmsvP4cp9Aj59AYusvpk4DVDr42dROGnnFmSgKQ1kPHwyNde6UKPNB8 Pi8vBcfY+jy/S6J4piziMaZt24K+DlMlgZ/n3FJXpPinfAReoOY5hbcxE jNekFo+/VUKyhFa4CS8x4YjiBBsYQFBJfefNGNpFgO9CFK0G47B1UHVEa g==; X-IronPort-AV: E=McAfee;i="6500,9779,10615"; a="416119852" X-IronPort-AV: E=Sophos;i="5.97,281,1669104000"; d="scan'208";a="416119852" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2023 11:21:41 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10615"; a="912825238" X-IronPort-AV: E=Sophos;i="5.97,281,1669104000"; d="scan'208";a="912825238" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.48.215]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2023 11:21:40 -0800 Subject: [PATCH v4 6/7] cxl: Add emulation when HDM decoders are not committed From: Dave Jiang To: dan.j.williams@intel.com Cc: Jonathan Cameron , linux-cxl@vger.kernel.org Date: Wed, 08 Feb 2023 12:21:38 -0700 Message-ID: <167588409751.1155956.623031449338237326.stgit@djiang5-mobl3.local> In-Reply-To: <167588394236.1155956.8466475582138210344.stgit@djiang5-mobl3.local> References: <167588394236.1155956.8466475582138210344.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org For the case where DVSEC range register(s) are active and HDM decoders are not committed, use RR to provide emulation. A first pass is done to note whether any decoders are committed. If there are no committed endpoint decoders, then DVSEC ranges will be used for emulation. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- v4: - Remove ->emulate_decoders (Dan) - Removed previous patch to just delete in this patch. (Dan) --- drivers/cxl/core/hdm.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index a49543f22dca..39e02f28b6a6 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -730,6 +730,32 @@ static int cxl_setup_hdm_decoder_from_dvsec(struct cxl_port *port, return 0; } +static bool should_emulate_decoders(struct cxl_port *port) +{ + struct cxl_hdm *cxlhdm = dev_get_drvdata(&port->dev); + void __iomem *hdm = cxlhdm->regs.hdm_decoder; + u32 ctrl; + int i; + + if (!is_cxl_endpoint(cxlhdm->port)) + return false; + + if (!hdm) + return true; + + /* + * If any decoders are committed already, there should not be any + * emulated DVSEC decoders. + */ + for (i = 0; i < cxlhdm->decoder_count; i++) { + ctrl = readl(hdm + CXL_HDM_DECODER0_CTRL_OFFSET(i)); + if (FIELD_GET(CXL_HDM_DECODER0_CTRL_COMMITTED, ctrl)) + return false; + } + + return true; +} + static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, int *target_map, void __iomem *hdm, int which, u64 *dpa_base, struct cxl_endpoint_dvsec_info *info) @@ -745,6 +771,9 @@ static int init_hdm_decoder(struct cxl_port *port, struct cxl_decoder *cxld, unsigned char target_id[8]; } target_list; + if (should_emulate_decoders(port)) + return cxl_setup_hdm_decoder_from_dvsec(port, cxld, which, info); + if (is_endpoint_decoder(&cxld->dev)) cxled = to_cxl_endpoint_decoder(&cxld->dev); From patchwork Wed Feb 8 19:21:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13133625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9396DC636D3 for ; Wed, 8 Feb 2023 19:22:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229679AbjBHTWX (ORCPT ); Wed, 8 Feb 2023 14:22:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38178 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231799AbjBHTWP (ORCPT ); Wed, 8 Feb 2023 14:22:15 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 83A791BD1 for ; Wed, 8 Feb 2023 11:22:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675884132; x=1707420132; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZJQuizj0iVj+k2xIEA8xPNDClVKThPUqu/YuCHqnb8A=; b=KDznIjns0eJWgumQEWmF+i3kaHexTEEkIpBRkwC2lqSNRBE4MkVrTlSX ak6CjNpKWGZK0soLfpUuHhKVJRwG4FKMFGcDsRNxaWOVK5Zczh1WCTOgW VH/G7A5iYfwAV+45GXBjTBHa61gtteswrP6X2NzvqmhzV7HiEYmA42/3x H/uM228kj0LLq08RXUK9vRkXRLWxqHZjnugjPLIIAXwIKoPng1USnG1rw IwHk2Y1t5RSPDrfoq89sVKntG9T9YyckV14CK2fhfhvwF6HFdlmG34jkg D7v4ojgbslswH1qJLIXtgweWZDucrwLwBlaw+MpbuOrAmaIlIPMV8XM1w w==; X-IronPort-AV: E=McAfee;i="6500,9779,10615"; a="416119925" X-IronPort-AV: E=Sophos;i="5.97,281,1669104000"; d="scan'208";a="416119925" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2023 11:21:49 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10615"; a="912825275" X-IronPort-AV: E=Sophos;i="5.97,281,1669104000"; d="scan'208";a="912825275" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.48.215]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2023 11:21:49 -0800 Subject: [PATCH v4 7/7] cxl: remove locked check for dvsec_range_allowed() From: Dave Jiang To: dan.j.williams@intel.com Cc: Jonathan Cameron , linux-cxl@vger.kernel.org Date: Wed, 08 Feb 2023 12:21:47 -0700 Message-ID: <167588410599.1155956.845452402773138048.stgit@djiang5-mobl3.local> In-Reply-To: <167588394236.1155956.8466475582138210344.stgit@djiang5-mobl3.local> References: <167588394236.1155956.8466475582138210344.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Remove the CXL_DECODER_F_LOCK check to be permissive of platform BIOSes that allow CXL.mem to be remapped. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang --- v4: - Change commit log for more clarity. (Dan) --- drivers/cxl/core/pci.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index cfd869a1fa93..f18262cbe82a 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -228,8 +228,6 @@ static int dvsec_range_allowed(struct device *dev, void *arg) cxld = to_cxl_decoder(dev); - if (!(cxld->flags & CXL_DECODER_F_LOCK)) - return 0; if (!(cxld->flags & CXL_DECODER_F_RAM)) return 0;