From patchwork Mon Feb 6 20:49:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130574 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9604C636D4 for ; Mon, 6 Feb 2023 20:49:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230214AbjBFUth (ORCPT ); Mon, 6 Feb 2023 15:49:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47632 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230261AbjBFUtf (ORCPT ); Mon, 6 Feb 2023 15:49:35 -0500 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 123CA2BF05; Mon, 6 Feb 2023 12:49:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716574; x=1707252574; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bkGbiCLyd+lG5cBq1nAbZmdVuueIpeBqaWZkt8YeFBs=; b=NHjVfb6wrUFXhNHrO8yoRB+TcLSrNdxAP2eXl9cGu/kfN9tlnIPqNh1G 8gqKB6xBtIGkpF6KLtLEgrL24oOrB/uWSb4HLD6FFERi40K1HkbjrOHB4 DXwoSKXDNxe5TpKRKdly+3hxtSElthVQoD90ovuDU4dw/UUOPUI/qQH2C XaHwYhNMBgU6FAnU9zfbKxPg+h2pIMSeO79no/+fIBje+tUD253sP+Vdv hF7xtMn/d32fuNSavc3l/KoLndvCaXvX39pvNp3BkJ95V4oLn7iNREc16 mMEWih6Y5bO+HvjsHJX1xDmsLqcLTKs7VcIXuWWJ5AsWjbVNwQVLzwMRF w==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="393911803" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="393911803" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:49:33 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="616559345" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="616559345" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:49:32 -0800 Subject: [PATCH 01/18] cxl: Export QTG ids from CFMWS to sysfs From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:49:30 -0700 Message-ID: <167571656940.587790.15913351407119270213.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Export the QoS Throttling Group ID from the CXL Fixed Memory Window Structure (CFMWS) under the root decoder sysfs attributes. CXL rev3.0 9.17.1.3 CXL Fixed Memory Window Structure (CFMWS) Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron --- Documentation/ABI/testing/sysfs-bus-cxl | 7 +++++++ drivers/cxl/acpi.c | 3 +++ drivers/cxl/core/port.c | 14 ++++++++++++++ drivers/cxl/cxl.h | 3 +++ 4 files changed, 27 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 8494ef27e8d2..0932c2f6fbf4 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -294,6 +294,13 @@ Description: (WO) Write a string in the form 'regionZ' to delete that region, provided it is currently idle / not bound to a driver. +What: /sys/bus/cxl/devices/decoderX.Y/qtg_id +Date: Jan, 2023 +KernelVersion: v6.3 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) Shows the QoS Throttling Group ID. The QTG ID for a root + decoder comes from the CFMWS structure of the CEDT. What: /sys/bus/cxl/devices/regionZ/uuid Date: May, 2022 diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index 13cde44c6086..7a71bb5041c7 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -289,6 +289,9 @@ static int cxl_parse_cfmws(union acpi_subtable_headers *header, void *arg, } } } + + cxld->qtg_id = cfmws->qtg_id; + rc = cxl_decoder_add(cxld, target_map); err_xormap: if (rc) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index b631a0520456..fe78daf7e7c8 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -284,6 +284,16 @@ static ssize_t interleave_ways_show(struct device *dev, static DEVICE_ATTR_RO(interleave_ways); +static ssize_t qtg_id_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cxl_decoder *cxld = to_cxl_decoder(dev); + + return sysfs_emit(buf, "%d\n", cxld->qtg_id); +} + +static DEVICE_ATTR_RO(qtg_id); + static struct attribute *cxl_decoder_base_attrs[] = { &dev_attr_start.attr, &dev_attr_size.attr, @@ -303,6 +313,7 @@ static struct attribute *cxl_decoder_root_attrs[] = { &dev_attr_cap_type2.attr, &dev_attr_cap_type3.attr, &dev_attr_target_list.attr, + &dev_attr_qtg_id.attr, SET_CXL_REGION_ATTR(create_pmem_region) SET_CXL_REGION_ATTR(delete_region) NULL, @@ -1606,6 +1617,7 @@ struct cxl_root_decoder *cxl_root_decoder_alloc(struct cxl_port *port, } atomic_set(&cxlrd->region_id, rc); + cxld->qtg_id = CXL_QTG_ID_INVALID; return cxlrd; } EXPORT_SYMBOL_NS_GPL(cxl_root_decoder_alloc, CXL); @@ -1643,6 +1655,7 @@ struct cxl_switch_decoder *cxl_switch_decoder_alloc(struct cxl_port *port, cxld = &cxlsd->cxld; cxld->dev.type = &cxl_decoder_switch_type; + cxld->qtg_id = CXL_QTG_ID_INVALID; return cxlsd; } EXPORT_SYMBOL_NS_GPL(cxl_switch_decoder_alloc, CXL); @@ -1675,6 +1688,7 @@ struct cxl_endpoint_decoder *cxl_endpoint_decoder_alloc(struct cxl_port *port) } cxld->dev.type = &cxl_decoder_endpoint_type; + cxld->qtg_id = CXL_QTG_ID_INVALID; return cxled; } EXPORT_SYMBOL_NS_GPL(cxl_endpoint_decoder_alloc, CXL); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 1b1cf459ac77..f558bbfc0332 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -279,6 +279,7 @@ enum cxl_decoder_type { */ #define CXL_DECODER_MAX_INTERLEAVE 16 +#define CXL_QTG_ID_INVALID -1 /** * struct cxl_decoder - Common CXL HDM Decoder Attributes @@ -290,6 +291,7 @@ enum cxl_decoder_type { * @target_type: accelerator vs expander (type2 vs type3) selector * @region: currently assigned region for this decoder * @flags: memory type capabilities and locking + * @qtg_id: QoS Throttling Group ID * @commit: device/decoder-type specific callback to commit settings to hw * @reset: device/decoder-type specific callback to reset hw settings */ @@ -302,6 +304,7 @@ struct cxl_decoder { enum cxl_decoder_type target_type; struct cxl_region *region; unsigned long flags; + int qtg_id; int (*commit)(struct cxl_decoder *cxld); int (*reset)(struct cxl_decoder *cxld); }; From patchwork Mon Feb 6 20:49:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130575 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DACF2C636D3 for ; Mon, 6 Feb 2023 20:49:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230262AbjBFUts (ORCPT ); Mon, 6 Feb 2023 15:49:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230249AbjBFUtr (ORCPT ); Mon, 6 Feb 2023 15:49:47 -0500 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA2B11F496; Mon, 6 Feb 2023 12:49:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716582; x=1707252582; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=uC8lEuvwHH2deqsIZTrfxKpysQ5Jn97kcwiU6Ljxh8g=; b=S9PReeM6TGhU6Km2U3G7O3gD3h3D1wmL3ubI+oZRJTJO0d8U/smEP1OI 0aU5YK+WZL8s6LQPhX/m4LJRwMcCEcHhAW/UBvVXonwNIYe+npH27jG1S Z/ckkZjTUeEx1u4TIWd1G1gSQHPpa82kVxcswPsTwFN78x/nZK7adUuc+ E+kWvEeoexFaP1pJz9d8ZAaD8C6btkTnjA+uO19msjjnZHDPZJWZ0t9yh paxpIbvH8KCqZKphc07BkkRslgu0hWp8Y+KigH0maT60h5jyQumNHLMVA Cl0QYSG2U7P/TpBmjpuz8BcrYg4L5qnWCH6IzsdSQ6Y5LXrSTKrwQbnpv w==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="393911834" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="393911834" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:49:42 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="616559380" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="616559380" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:49:41 -0800 Subject: [PATCH 02/18] ACPICA: Export acpi_ut_verify_cdat_checksum() From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:49:39 -0700 Message-ID: <167571657859.587790.12435839081602248140.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Export the CDAT checksum verify function so CXL driver can use it to verify CDAT coming from the CXL devices. Given that this function isn't actually being used by ACPI internals, removing the define check of APCI_CHECKSUM_ABORT so the function would return failure on checksum fail since the driver will need to know. Signed-off-by: Dave Jiang --- drivers/acpi/acpica/utcksum.c | 4 +--- include/linux/acpi.h | 7 +++++++ 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/acpi/acpica/utcksum.c b/drivers/acpi/acpica/utcksum.c index c166e4c05ab6..c0f98c8f9a0b 100644 --- a/drivers/acpi/acpica/utcksum.c +++ b/drivers/acpi/acpica/utcksum.c @@ -102,15 +102,13 @@ acpi_ut_verify_cdat_checksum(struct acpi_table_cdat *cdat_table, u32 length) "should be 0x%2.2X", acpi_gbl_CDAT, cdat_table->checksum, checksum)); - -#if (ACPI_CHECKSUM_ABORT) return (AE_BAD_CHECKSUM); -#endif } cdat_table->checksum = checksum; return (AE_OK); } +EXPORT_SYMBOL_GPL(acpi_ut_verify_cdat_checksum); /******************************************************************************* * diff --git a/include/linux/acpi.h b/include/linux/acpi.h index 5e6a876e17ba..09b44afef7df 100644 --- a/include/linux/acpi.h +++ b/include/linux/acpi.h @@ -1504,9 +1504,16 @@ static inline void acpi_init_ffh(void) { } #ifdef CONFIG_ACPI extern void acpi_device_notify(struct device *dev); extern void acpi_device_notify_remove(struct device *dev); +extern acpi_status +acpi_ut_verify_cdat_checksum(struct acpi_table_cdat *cdat_table, u32 length); #else static inline void acpi_device_notify(struct device *dev) { } static inline void acpi_device_notify_remove(struct device *dev) { } +static inline acpi_status +acpi_ut_verify_cdat_checksum(struct acpi_table_cdat *cdat_table, u32 length) +{ + return (AE_NOT_CONFIGURED); +} #endif #endif /*_LINUX_ACPI_H*/ From patchwork Mon Feb 6 20:49:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130576 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11FDFC64EC4 for ; Mon, 6 Feb 2023 20:49:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230249AbjBFUty (ORCPT ); Mon, 6 Feb 2023 15:49:54 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230393AbjBFUtw (ORCPT ); Mon, 6 Feb 2023 15:49:52 -0500 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA9541F496; Mon, 6 Feb 2023 12:49:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716591; x=1707252591; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=xS04BnIp7MCjHPpNH7pRQHA3mknpzH+ZYewjFg6BC9M=; b=iz+FkeKBYOyjWqSIr1C3i2JtdyEovnvF/z9aGCqTZ9GJickOKo3mePZl knsdNDfCTzFZsoyIT1m26FX0rB2Iq7mWh7nK04mAUHhaNGgU6s1qXDxGu 1jMn7+ufY/R+bc/JvtOiKgGA5IcOyj6GRJtFxLhX9azTi7ceaUqMyNpY1 3wyLyfQ6zuAwxUSRAMz00T5LH8CjYDmpUjl01K1uZo02bq+TjJtxrFEb6 r8YHMIMGLT3EPSp/DyFNWFN2F/lttns9JRM6wt1sWK9PIcanP085Vhy66 ZCgvvIIPSBiHiZXUDO+G92EI/3MLdI3YsXsrjQWZXzxVyvwdFnMb4Z2A7 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="393911860" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="393911860" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:49:51 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="616559420" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="616559420" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:49:50 -0800 Subject: [PATCH 03/18] cxl: Add checksum verification to CDAT from CXL From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:49:48 -0700 Message-ID: <167571658747.587790.17252249256706733075.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org A CDAT table is available from a CXL device. The table is read by the driver and cached in software. With the CXL subsystem needing to parse the CDAT table, the checksum should be verified. Add checksum verification after the CDAT table is read from device. Signed-off-by: Dave Jiang --- drivers/cxl/core/pci.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 57764e9cd19d..a24dac36bedd 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -3,6 +3,7 @@ #include #include #include +#include #include #include #include @@ -592,6 +593,7 @@ void read_cdat_data(struct cxl_port *port) struct device *dev = &port->dev; struct device *uport = port->uport; size_t cdat_length; + acpi_status status; int rc; cdat_doe = find_cdat_doe(uport); @@ -620,5 +622,14 @@ void read_cdat_data(struct cxl_port *port) port->cdat.length = 0; dev_err(dev, "CDAT data read error\n"); } + + status = acpi_ut_verify_cdat_checksum(port->cdat.table, port->cdat.length); + if (status != AE_OK) { + /* Don't leave table data allocated on error */ + devm_kfree(dev, port->cdat.table); + port->cdat.table = NULL; + port->cdat.length = 0; + dev_err(dev, "CDAT data checksum error\n"); + } } EXPORT_SYMBOL_NS_GPL(read_cdat_data, CXL); From patchwork Mon Feb 6 20:49:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 548B9C636D3 for ; Mon, 6 Feb 2023 20:50:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230305AbjBFUuI (ORCPT ); Mon, 6 Feb 2023 15:50:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230339AbjBFUuE (ORCPT ); Mon, 6 Feb 2023 15:50:04 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0FD9A2CC51; Mon, 6 Feb 2023 12:50:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716601; x=1707252601; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SVxqrflyskXxJAcWJicsZWPFmU0oE+WsKb5As7c0Ac8=; b=iMF73XUWvLKHqGqkjsMhyiTubl+bYgubRqD8evn/hOVGt2Sfe7DRc4ph u7261ExLWSx5uqkITebPZdUVTIhfvQUjfzLAUIq43M7aMEezHdMAsDN97 fRlyyZtPRBeNDO42BqafymX+JUtowexVN8UnKIXeGJ5D/rXyBAmoAI7mE bT8WEWVR34xoGvfRKrAKuvgdKFYVRSV1iGv1cxqLXjmjIXOYzVQXCFClY 1IToyf0hJ+aP4o7wYbKVIhUxTmEy5EZ7zofrlRoYa1bVbBxgBV8LKX4dR Onq1AKgbUMe33+QdH5IvPIwq4evf3s7dIFfYxSZDzZrHyWGcQksxkJmp/ A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="330604654" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="330604654" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:50:00 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="995471573" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="995471573" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:49:59 -0800 Subject: [PATCH 04/18] cxl: Add common helpers for cdat parsing From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:49:58 -0700 Message-ID: <167571659666.587790.1381783105886436293.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Add helper functions to parse the CDAT table and provide a callback to parse the sub-table. Helpers are provided for DSMAS and DSLBIS sub-table parsing. The code is patterned after the ACPI table parsing helpers. Signed-off-by: Dave Jiang --- drivers/cxl/core/Makefile | 1 drivers/cxl/core/cdat.c | 98 +++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/core/cdat.h | 15 +++++++ drivers/cxl/cxl.h | 9 ++++ 4 files changed, 123 insertions(+) create mode 100644 drivers/cxl/core/cdat.c create mode 100644 drivers/cxl/core/cdat.h diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile index 79c7257f4107..438ce27faf77 100644 --- a/drivers/cxl/core/Makefile +++ b/drivers/cxl/core/Makefile @@ -10,4 +10,5 @@ cxl_core-y += memdev.o cxl_core-y += mbox.o cxl_core-y += pci.o cxl_core-y += hdm.o +cxl_core-y += cdat.o cxl_core-$(CONFIG_CXL_REGION) += region.o diff --git a/drivers/cxl/core/cdat.c b/drivers/cxl/core/cdat.c new file mode 100644 index 000000000000..be09c8a690f5 --- /dev/null +++ b/drivers/cxl/core/cdat.c @@ -0,0 +1,98 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2023 Intel Corporation. All rights reserved. */ +#include "cxl.h" +#include "cdat.h" + +static u8 cdat_get_subtable_entry_type(struct cdat_subtable_entry *entry) +{ + return entry->hdr->type; +} + +static u16 cdat_get_subtable_entry_length(struct cdat_subtable_entry *entry) +{ + return entry->hdr->length; +} + +static bool has_handler(struct cdat_subtable_proc *proc) +{ + return proc->handler; +} + +static int call_handler(struct cdat_subtable_proc *proc, + struct cdat_subtable_entry *ent) +{ + if (proc->handler) + return proc->handler(ent->hdr, proc->arg); + return -EINVAL; +} + +static int cdat_table_parse_entries(enum acpi_cdat_type type, + struct acpi_table_cdat *table_header, + struct cdat_subtable_proc *proc, + unsigned int max_entries) +{ + struct cdat_subtable_entry entry; + unsigned long table_end, entry_len; + int count = 0; + int rc; + + if (!has_handler(proc)) + return -EINVAL; + + table_end = (unsigned long)table_header + table_header->length; + + if (type >= ACPI_CDAT_TYPE_RESERVED) + return -EINVAL; + + entry.type = type; + entry.hdr = (struct acpi_cdat_header *)((unsigned long)table_header + + sizeof(*table_header)); + + while ((unsigned long)entry.hdr < table_end) { + entry_len = cdat_get_subtable_entry_length(&entry); + + if ((unsigned long)entry.hdr + entry_len > table_end) + return -EINVAL; + + if (max_entries && count >= max_entries) + break; + + if (entry_len == 0) + return -EINVAL; + + if (cdat_get_subtable_entry_type(&entry) == type) { + rc = call_handler(proc, &entry); + if (rc) + return rc; + } + + entry.hdr = (struct acpi_cdat_header *)((unsigned long)entry.hdr + entry_len); + count++; + } + + return count; +} + +int cdat_table_parse_dsmas(void *table, cdat_tbl_entry_handler handler, void *arg) +{ + struct acpi_table_cdat *header = (struct acpi_table_cdat *)table; + struct cdat_subtable_proc proc = { + .handler = handler, + .arg = arg, + }; + + return cdat_table_parse_entries(ACPI_CDAT_TYPE_DSMAS, header, &proc, 0); +} +EXPORT_SYMBOL_NS_GPL(cdat_table_parse_dsmas, CXL); + +int cdat_table_parse_dslbis(void *table, cdat_tbl_entry_handler handler, void *arg) +{ + struct acpi_table_cdat *header = (struct acpi_table_cdat *)table; + struct cdat_subtable_proc proc = { + .handler = handler, + .arg = arg, + }; + + return cdat_table_parse_entries(ACPI_CDAT_TYPE_DSLBIS, header, &proc, 0); +} +EXPORT_SYMBOL_NS_GPL(cdat_table_parse_dslbis, CXL); diff --git a/drivers/cxl/core/cdat.h b/drivers/cxl/core/cdat.h new file mode 100644 index 000000000000..f690325e82a6 --- /dev/null +++ b/drivers/cxl/core/cdat.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2023 Intel Corporation. */ +#ifndef __CXL_CDAT_H__ +#define __CXL_CDAT_H__ + +struct cdat_subtable_proc { + cdat_tbl_entry_handler handler; + void *arg; +}; + +struct cdat_subtable_entry { + struct acpi_cdat_header *hdr; + enum acpi_cdat_type type; +}; +#endif diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index f558bbfc0332..839a121c1997 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -9,6 +9,7 @@ #include #include #include +#include /** * DOC: cxl objects @@ -697,6 +698,14 @@ static inline struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev) } #endif +typedef int (*cdat_tbl_entry_handler)(struct acpi_cdat_header *header, void *arg); + +u8 cdat_table_checksum(u8 *buffer, u32 length); +int cdat_table_parse_dsmas(void *table, cdat_tbl_entry_handler handler, + void *arg); +int cdat_table_parse_dslbis(void *table, cdat_tbl_entry_handler handler, + void *arg); + /* * Unit test builds overrides this to __weak, find the 'strong' version * of these symbols in tools/testing/cxl/. From patchwork Mon Feb 6 20:50:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130578 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3281AC05027 for ; Mon, 6 Feb 2023 20:50:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230312AbjBFUuO (ORCPT ); Mon, 6 Feb 2023 15:50:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230317AbjBFUuL (ORCPT ); Mon, 6 Feb 2023 15:50:11 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CAB22BF0A; Mon, 6 Feb 2023 12:50:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716609; x=1707252609; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hh2rH5Q/YssEi0TmAoW2giYHa8vKVnvNuJJXfa9pxWE=; b=XLWN+sOA0v2yGisZtuqQS7JHYu4dfYOX71Oi/WZsoQOx4MKBOIzdIjqK TGNjtPsb8tIXadj2HvbFf1mJR5AyhISAcwQNRu7GS1W3vx/T1ZbVQtjfM imiy4N4BtAEIYgB1l9Dlgn1y38EVHWKWZPwrvNLLGj50UL08PZicGcZxw /+4Zb16ZBP5asz+OJUdEeNcN5JcZVAfXKGiinPKcnbDmr6ZBgcyWienR9 YQuvSOEnW52/bW1H28fWAWBFIylZ8FsBgVOPZFhNDw7sx8HRqstd14mYD C2FxlD51fWvcNLdlUCUCAlGfH6lJs4qEx5DjdQKf3YacPC4p71wK+eSP7 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="330604686" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="330604686" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:50:09 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="995471751" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="995471751" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:50:08 -0800 Subject: [PATCH 05/18] ACPICA: Fix 'struct acpi_cdat_dsmas' spelling mistake From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:50:06 -0700 Message-ID: <167571660543.587790.9945516736671124794.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org 'struct acpi_cadt_dsmas' => 'struct acpi_cdat_dsmas' Fixes: 51aad1a6723b ("ACPICA: Finish support for the CDAT table") Signed-off-by: Dave Jiang --- include/acpi/actbl1.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/acpi/actbl1.h b/include/acpi/actbl1.h index 4175dce3967c..e8297cefde09 100644 --- a/include/acpi/actbl1.h +++ b/include/acpi/actbl1.h @@ -344,7 +344,7 @@ enum acpi_cdat_type { /* Subtable 0: Device Scoped Memory Affinity Structure (DSMAS) */ -struct acpi_cadt_dsmas { +struct acpi_cdat_dsmas { u8 dsmad_handle; u8 flags; u16 reserved; From patchwork Mon Feb 6 20:50:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130579 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E62BC636D4 for ; Mon, 6 Feb 2023 20:50:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230319AbjBFUuU (ORCPT ); Mon, 6 Feb 2023 15:50:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230327AbjBFUuT (ORCPT ); Mon, 6 Feb 2023 15:50:19 -0500 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BED772B097; Mon, 6 Feb 2023 12:50:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716618; x=1707252618; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mfVfcggIP70jrmLKAqmG9n5343VBmt+WOjT7FFp8sF8=; b=Od2sqiBlclKmkibsx3I/2dQF7vBEczn3IQKE6YRbqXJMUkqO8MYq79UE NkZsfWWY7F/Rly16bCvMCtFF7tqKq62aoV1s6pf8jnUxdBAAnF11AMX0E M12o6F6745+g24TuV9qpPuqOy9jk9COPAVjRwXPH7Z71Ey6WVukLt14jL F1L/CoQQep7G/vLaT/2+8AxsvH+C7oT/ekh+hqurCcJ1CmcZ617oRG3cf 5Dgjgqivwc/aWylb7NJpi+kX6kEjzQxwoPLbiIqe+D/O4J9c5L1Ftj3Wl GpVObxYYLRZsk3b45GWgStyga9cfD83Z+yN6rdCp/is58yhccCB1XG0Cy A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="330604730" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="330604730" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:50:18 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="995471902" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="995471902" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:50:16 -0800 Subject: [PATCH 06/18] cxl: Add callback to parse the DSMAS subtables from CDAT From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:50:15 -0700 Message-ID: <167571661375.587790.16681436923769338643.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Provide a callback function to the CDAT parser in order to parse the Device Scoped Memory Affinity Structure (DSMAS). Each DSMAS structure contains the DPA range and its associated attributes in each entry. See the CDAT specification for details. Signed-off-by: Dave Jiang --- drivers/cxl/core/cdat.c | 25 +++++++++++++++++++++++++ drivers/cxl/core/port.c | 2 ++ drivers/cxl/cxl.h | 11 +++++++++++ drivers/cxl/port.c | 8 ++++++++ 4 files changed, 46 insertions(+) diff --git a/drivers/cxl/core/cdat.c b/drivers/cxl/core/cdat.c index be09c8a690f5..f9a64a0f1ee4 100644 --- a/drivers/cxl/core/cdat.c +++ b/drivers/cxl/core/cdat.c @@ -96,3 +96,28 @@ int cdat_table_parse_dslbis(void *table, cdat_tbl_entry_handler handler, void *a return cdat_table_parse_entries(ACPI_CDAT_TYPE_DSLBIS, header, &proc, 0); } EXPORT_SYMBOL_NS_GPL(cdat_table_parse_dslbis, CXL); + +int cxl_dsmas_parse_entry(struct acpi_cdat_header *header, void *arg) +{ + struct cxl_port *port = (struct cxl_port *)arg; + struct dsmas_entry *dent; + struct acpi_cdat_dsmas *dsmas; + + if (header->type != ACPI_CDAT_TYPE_DSMAS) + return -EINVAL; + + dent = devm_kzalloc(&port->dev, sizeof(*dent), GFP_KERNEL); + if (!dent) + return -ENOMEM; + + dsmas = (struct acpi_cdat_dsmas *)((unsigned long)header + sizeof(*header)); + dent->handle = dsmas->dsmad_handle; + dent->dpa_range.start = dsmas->dpa_base_address; + dent->dpa_range.end = dsmas->dpa_base_address + dsmas->dpa_length - 1; + + mutex_lock(&port->cdat.dsmas_lock); + list_add_tail(&dent->list, &port->cdat.dsmas_list); + mutex_unlock(&port->cdat.dsmas_lock); + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_dsmas_parse_entry, CXL); diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index fe78daf7e7c8..2b27319cfd42 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -660,6 +660,8 @@ static struct cxl_port *cxl_port_alloc(struct device *uport, device_set_pm_not_required(dev); dev->bus = &cxl_bus_type; dev->type = &cxl_port_type; + INIT_LIST_HEAD(&port->cdat.dsmas_list); + mutex_init(&port->cdat.dsmas_lock); return port; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 839a121c1997..1e5e69f08480 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -520,6 +521,8 @@ struct cxl_port { struct cxl_cdat { void *table; size_t length; + struct list_head dsmas_list; + struct mutex dsmas_lock; /* lock for dsmas_list */ } cdat; bool cdat_available; }; @@ -698,6 +701,12 @@ static inline struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev) } #endif +struct dsmas_entry { + struct list_head list; + struct range dpa_range; + u16 handle; +}; + typedef int (*cdat_tbl_entry_handler)(struct acpi_cdat_header *header, void *arg); u8 cdat_table_checksum(u8 *buffer, u32 length); @@ -706,6 +715,8 @@ int cdat_table_parse_dsmas(void *table, cdat_tbl_entry_handler handler, int cdat_table_parse_dslbis(void *table, cdat_tbl_entry_handler handler, void *arg); +int cxl_dsmas_parse_entry(struct acpi_cdat_header *header, void *arg); + /* * Unit test builds overrides this to __weak, find the 'strong' version * of these symbols in tools/testing/cxl/. diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 5453771bf330..b1da73e99bab 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -61,6 +61,14 @@ static int cxl_port_probe(struct device *dev) if (rc) return rc; + if (port->cdat.table) { + rc = cdat_table_parse_dsmas(port->cdat.table, + cxl_dsmas_parse_entry, + (void *)port); + if (rc < 0) + dev_dbg(dev, "Failed to parse DSMAS: %d\n", rc); + } + rc = cxl_hdm_decode_init(cxlds, cxlhdm); if (rc) return rc; From patchwork Mon Feb 6 20:50:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130580 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CE52C05027 for ; Mon, 6 Feb 2023 20:50:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230317AbjBFUue (ORCPT ); Mon, 6 Feb 2023 15:50:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230340AbjBFUud (ORCPT ); Mon, 6 Feb 2023 15:50:33 -0500 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49A1F20050; Mon, 6 Feb 2023 12:50:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716632; x=1707252632; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kIXOGWIlRzYzArn4iLknajFZdj1TLid0ppmpb6d89J4=; b=PNvr+yo7RFbI0rT2jVpuhKRBR+JTIi0X3mMCy+dKv56WhS4vELf/Dpqk r9RDS4SVGipTZuoxnkRG4Sxkeqn/WjBpTiGZdVK0M/d1xK6386wAlmtUz yZm/nacjWHwY0o474Xu9xohMy+8j6T6Mp5/5z/+khSYcNgqvros2o0ZnA cU342kmevUzg87iOjIQvs5dofDujh5aoAFF1U8BtQzFP2tVV2nC7jto/+ TnDUXiemSa8xfijuJngWKuE6GgP9DQgFHaARXjh+ZKEm1O6m0KVARhZh8 JV9eJJ0WuTDRYPbhcJYAzdfpDo6CxCYEnuAcsAEft/2MdYOcOQ1g0UAla Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="393911985" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="393911985" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:50:27 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="616559542" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="616559542" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:50:26 -0800 Subject: [PATCH 07/18] cxl: Add callback to parse the DSLBIS subtable from CDAT From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:50:23 -0700 Message-ID: <167571662248.587790.4362747686454305108.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Provide a callback to parse the Device Scoped Latency and Bandwidth Information Structure (DSLBIS) in the CDAT structures. The DSLBIS contains the bandwidth and latency information that's tied to a DSMAS handle. The driver will retrieve the read and write latency and bandwidth associated with the DSMAS which is tied to a DPA range. Signed-off-by: Dave Jiang --- drivers/cxl/core/cdat.c | 34 ++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 2 ++ drivers/cxl/port.c | 9 ++++++++- include/acpi/actbl1.h | 5 +++++ 4 files changed, 49 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/core/cdat.c b/drivers/cxl/core/cdat.c index f9a64a0f1ee4..3c8f3956487e 100644 --- a/drivers/cxl/core/cdat.c +++ b/drivers/cxl/core/cdat.c @@ -121,3 +121,37 @@ int cxl_dsmas_parse_entry(struct acpi_cdat_header *header, void *arg) return 0; } EXPORT_SYMBOL_NS_GPL(cxl_dsmas_parse_entry, CXL); + +int cxl_dslbis_parse_entry(struct acpi_cdat_header *header, void *arg) +{ + struct cxl_port *port = (struct cxl_port *)arg; + struct dsmas_entry *dent; + struct acpi_cdat_dslbis *dslbis; + u64 val; + + if (header->type != ACPI_CDAT_TYPE_DSLBIS) + return -EINVAL; + + dslbis = (struct acpi_cdat_dslbis *)((unsigned long)header + sizeof(*header)); + if ((dslbis->flags & ACPI_CEDT_DSLBIS_MEM_MASK) != + ACPI_CEDT_DSLBIS_MEM_MEMORY) + return 0; + + if (dslbis->data_type > ACPI_HMAT_WRITE_BANDWIDTH) + return -ENXIO; + + /* Value calculation with base_unit, see ACPI Spec 6.5 5.2.28.4 */ + val = dslbis->entry[0] * dslbis->entry_base_unit; + + mutex_lock(&port->cdat.dsmas_lock); + list_for_each_entry(dent, &port->cdat.dsmas_list, list) { + if (dslbis->handle == dent->handle) { + dent->qos[dslbis->data_type] = val; + break; + } + } + mutex_unlock(&port->cdat.dsmas_lock); + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_dslbis_parse_entry, CXL); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 1e5e69f08480..849b22236f1d 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -705,6 +705,7 @@ struct dsmas_entry { struct list_head list; struct range dpa_range; u16 handle; + u64 qos[ACPI_HMAT_WRITE_BANDWIDTH + 1]; }; typedef int (*cdat_tbl_entry_handler)(struct acpi_cdat_header *header, void *arg); @@ -716,6 +717,7 @@ int cdat_table_parse_dslbis(void *table, cdat_tbl_entry_handler handler, void *arg); int cxl_dsmas_parse_entry(struct acpi_cdat_header *header, void *arg); +int cxl_dslbis_parse_entry(struct acpi_cdat_header *header, void *arg); /* * Unit test builds overrides this to __weak, find the 'strong' version diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index b1da73e99bab..8de311208b37 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -65,8 +65,15 @@ static int cxl_port_probe(struct device *dev) rc = cdat_table_parse_dsmas(port->cdat.table, cxl_dsmas_parse_entry, (void *)port); - if (rc < 0) + if (rc > 0) { + rc = cdat_table_parse_dslbis(port->cdat.table, + cxl_dslbis_parse_entry, + (void *)port); + if (rc <= 0) + dev_dbg(dev, "Failed to parse DSLBIS: %d\n", rc); + } else { dev_dbg(dev, "Failed to parse DSMAS: %d\n", rc); + } } rc = cxl_hdm_decode_init(cxlds, cxlhdm); diff --git a/include/acpi/actbl1.h b/include/acpi/actbl1.h index e8297cefde09..ff6092e45196 100644 --- a/include/acpi/actbl1.h +++ b/include/acpi/actbl1.h @@ -369,6 +369,11 @@ struct acpi_cdat_dslbis { u16 reserved2; }; +/* Flags for subtable above */ + +#define ACPI_CEDT_DSLBIS_MEM_MASK GENMASK(3, 0) +#define ACPI_CEDT_DSLBIS_MEM_MEMORY 0 + /* Subtable 2: Device Scoped Memory Side Cache Information Structure (DSMSCIS) */ struct acpi_cdat_dsmscis { From patchwork Mon Feb 6 20:50:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130581 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17551C64EC4 for ; Mon, 6 Feb 2023 20:50:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230359AbjBFUui (ORCPT ); Mon, 6 Feb 2023 15:50:38 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48394 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230354AbjBFUui (ORCPT ); Mon, 6 Feb 2023 15:50:38 -0500 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E473B2CC48; Mon, 6 Feb 2023 12:50:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716637; x=1707252637; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kG3UAx4/7B+P3TM5sA0oWhO+QJjyaL/AvJzU6IihwKo=; b=nXwVBxMRlMyozmJ5R77NelmxAznLgmSCx5Y170Yb+0x+3qYDHMj3oAcz WsnYxkDGhF2spyaamSnHzSyfMvevVLvfPjd7er2V+tya2TNynM4ECI9bo v/haxor6R4Wn/OL96Aer9uHO2EzxJmLDiULbY7DMiXRRITlY/SOTJBbI5 wp2dwdhkdEbShlIptbNgdoipNb/06YQYYqyTqKanLNs4f3SOxTrM4/m1I TUqGcaN9ihsaQGBZigRpLJN1bXfnN91TDB/3EIVFQw2t9dnAnQVGrz65H DVr2RIRhkBEZC1byILCDDJyV0Yzh6GVxCCNQcoWPtwwTaN+zBhXxMFJf6 w==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="393912014" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="393912014" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:50:36 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="616559659" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="616559659" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:50:35 -0800 Subject: [PATCH 08/18] cxl: Add support for _DSM Function for retrieving QTG ID From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:50:33 -0700 Message-ID: <167571663199.587790.13615282047168132392.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org CXL spec v3.0 9.17.3 CXL Root Device Specific Methods (_DSM) Add support to retrieve QTG ID via ACPI _DSM call. The _DSM call requires an input of an ACPI package with 4 dwords (read latency, write latency, read bandwidth, write bandwidth). The call returns a package with 1 WORD that provides the max supported QTG ID and a package that may contain 0 or more WORDs as the recommended QTG IDs in the recommended order. Signed-off-by: Dave Jiang --- drivers/cxl/core/Makefile | 1 drivers/cxl/core/acpi.c | 99 +++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 15 +++++++ 3 files changed, 115 insertions(+) create mode 100644 drivers/cxl/core/acpi.c diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile index 438ce27faf77..11ccc2016ab7 100644 --- a/drivers/cxl/core/Makefile +++ b/drivers/cxl/core/Makefile @@ -11,4 +11,5 @@ cxl_core-y += mbox.o cxl_core-y += pci.o cxl_core-y += hdm.o cxl_core-y += cdat.o +cxl_core-y += acpi.o cxl_core-$(CONFIG_CXL_REGION) += region.o diff --git a/drivers/cxl/core/acpi.c b/drivers/cxl/core/acpi.c new file mode 100644 index 000000000000..86dc6c9c1f24 --- /dev/null +++ b/drivers/cxl/core/acpi.c @@ -0,0 +1,99 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2022 Intel Corporation. All rights reserved. */ +#include +#include +#include +#include +#include +#include +#include "cxlpci.h" +#include "cxl.h" + +const guid_t acpi_cxl_qtg_id_guid = + GUID_INIT(0xF365F9A6, 0xA7DE, 0x4071, + 0xA6, 0x6A, 0xB4, 0x0C, 0x0B, 0x4F, 0x8E, 0x52); + +/** + * cxl_acpi_evaluate_qtg_dsm - Retrieve QTG ids via ACPI _DSM + * @handle: ACPI handle + * @input: bandwidth and latency data + * + * Issue QTG _DSM with accompanied bandwidth and latency data in order to get + * the QTG IDs that falls within the performance data. + */ +struct qtg_dsm_output *cxl_acpi_evaluate_qtg_dsm(acpi_handle handle, + struct qtg_dsm_input *input) +{ + struct qtg_dsm_output *output; + union acpi_object *out_obj, *out_buf, *pkg, in_buf, in_obj; + int len; + int rc; + + in_obj.type = ACPI_TYPE_PACKAGE; + in_obj.package.count = 1; + in_obj.package.elements = &in_buf; + in_buf.type = ACPI_TYPE_BUFFER; + in_buf.buffer.pointer = (u8 *)input; + in_buf.buffer.length = sizeof(u32) * 4; + + out_obj = acpi_evaluate_dsm(handle, &acpi_cxl_qtg_id_guid, 1, 1, &in_obj); + if (!out_obj) + return ERR_PTR(-ENXIO); + + if (out_obj->type != ACPI_TYPE_PACKAGE) { + rc = -ENXIO; + goto err; + } + + /* + * CXL spec v3.0 9.17.3.1 + * There should be 2 elements in the package. 1 WORD for max QTG ID supported + * by the platform, and the other a package of recommended QTGs + */ + if (out_obj->package.count != 2) { + rc = -ENXIO; + goto err; + } + + pkg = &out_obj->package.elements[1]; + if (pkg->type != ACPI_TYPE_PACKAGE) { + rc = -ENXIO; + goto err; + } + + out_buf = &pkg->package.elements[0]; + if (out_buf->type != ACPI_TYPE_BUFFER) { + rc = -ENXIO; + goto err; + } + + len = out_buf->buffer.length; + output = kmalloc(len + sizeof(*output), GFP_KERNEL); + if (!output) { + rc = -ENOMEM; + goto err; + } + + /* It's legal to have 0 QTG entries */ + if (len == 0) { + output->nr = 0; + goto out; + } + + /* Malformed package, not multiple of WORD size */ + if (len % sizeof(u16)) { + rc = -ENXIO; + goto out; + } + + output->nr = len / sizeof(u16); + memcpy(output->qtg_ids, out_buf->buffer.pointer, len); +out: + ACPI_FREE(out_obj); + return output; + +err: + ACPI_FREE(out_obj); + return ERR_PTR(rc); +} +EXPORT_SYMBOL_NS_GPL(cxl_acpi_evaluate_qtg_dsm, CXL); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 849b22236f1d..e70df07f9b4b 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -719,6 +719,21 @@ int cdat_table_parse_dslbis(void *table, cdat_tbl_entry_handler handler, int cxl_dsmas_parse_entry(struct acpi_cdat_header *header, void *arg); int cxl_dslbis_parse_entry(struct acpi_cdat_header *header, void *arg); +struct qtg_dsm_input { + u32 rd_lat; + u32 wr_lat; + u32 rd_bw; + u32 wr_bw; +}; + +struct qtg_dsm_output { + int nr; + u16 qtg_ids[]; +}; + +struct qtg_dsm_output *cxl_acpi_evaluate_qtg_dsm(acpi_handle handle, + struct qtg_dsm_input *input); + /* * Unit test builds overrides this to __weak, find the 'strong' version * of these symbols in tools/testing/cxl/. From patchwork Mon Feb 6 20:50:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130582 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05D9EC05027 for ; Mon, 6 Feb 2023 20:50:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229884AbjBFUur (ORCPT ); Mon, 6 Feb 2023 15:50:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48514 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230354AbjBFUur (ORCPT ); Mon, 6 Feb 2023 15:50:47 -0500 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 517D22B097; Mon, 6 Feb 2023 12:50:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716646; x=1707252646; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8IOW1adM9oC2jAAA+J+4B+32FWUmU6UHY5MmsRIO4bg=; b=dEfTOyVJ/0Gw2ooe/ex5AwWYrBd7xc6f5eMuTR+9sftlfWEjfjYkH7cy g8G7wEH49kkUUB7IUbfW0lLXjX8OsQzJJJd05UdLY2LaQ+uCU31R189Dk 7ZA7kbSkcnVc6hB29h9OTrMP/duuwqf/1VMaeQiKJX4M4lkbHXRHxDauH Ve5GupE0qghcO32TlVo/E+LlzlCjr0hcaxm1v0vU5XN4XhPsGB6zoqgMd fPjTs6fk5XUtwqdJybS2GyAAcpsYT0EWVBbHGTLIAya975IXkJCP7jCX7 JQdpK0ZIAnREYhQhcWlntDlPVN5ooJIpzWRLyEKx5ssiq9joi0AO23Erw g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="393912042" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="393912042" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:50:45 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="616559702" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="616559702" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:50:44 -0800 Subject: [PATCH 09/18] cxl: Add helper function to retrieve ACPI handle of CXL root device From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:50:42 -0700 Message-ID: <167571664152.587790.608196538260467034.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Provide a helper to find the ACPI0017 device in order to issue the _DSM. The helper will take the 'struct device' from a cxl_port and iterate until the root device is reached. The ACPI handle will be returned from the root device. Signed-off-by: Dave Jiang --- drivers/cxl/core/acpi.c | 30 ++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 1 + 2 files changed, 31 insertions(+) diff --git a/drivers/cxl/core/acpi.c b/drivers/cxl/core/acpi.c index 86dc6c9c1f24..05fcd4751619 100644 --- a/drivers/cxl/core/acpi.c +++ b/drivers/cxl/core/acpi.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include "cxlpci.h" #include "cxl.h" @@ -13,6 +14,35 @@ const guid_t acpi_cxl_qtg_id_guid = GUID_INIT(0xF365F9A6, 0xA7DE, 0x4071, 0xA6, 0x6A, 0xB4, 0x0C, 0x0B, 0x4F, 0x8E, 0x52); +/** + * cxl_acpi_get_root_acpi_handle - get the ACPI handle of the CXL root device + * @dev: 'struct device' to start searching from. Should be from cxl_port->dev. + * Looks for the ACPI0017 device and return the ACPI handle + **/ +acpi_handle cxl_acpi_get_rootdev_handle(struct device *dev) +{ + struct device *itr = dev, *root_dev; + acpi_handle handle; + + if (!dev) + return ERR_PTR(-EINVAL); + + while (itr->parent) { + root_dev = itr; + itr = itr->parent; + } + + if (!dev_is_platform(root_dev)) + return ERR_PTR(-ENODEV); + + handle = ACPI_HANDLE(root_dev); + if (!handle) + return ERR_PTR(-ENODEV); + + return handle; +} +EXPORT_SYMBOL_NS_GPL(cxl_acpi_get_rootdev_handle, CXL); + /** * cxl_acpi_evaluate_qtg_dsm - Retrieve QTG ids via ACPI _DSM * @handle: ACPI handle diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index e70df07f9b4b..ac6ea550ab0a 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -733,6 +733,7 @@ struct qtg_dsm_output { struct qtg_dsm_output *cxl_acpi_evaluate_qtg_dsm(acpi_handle handle, struct qtg_dsm_input *input); +acpi_handle cxl_acpi_get_rootdev_handle(struct device *dev); /* * Unit test builds overrides this to __weak, find the 'strong' version From patchwork Mon Feb 6 20:50:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130583 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64D52C05027 for ; Mon, 6 Feb 2023 20:51:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230362AbjBFUu7 (ORCPT ); Mon, 6 Feb 2023 15:50:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48654 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230354AbjBFUu6 (ORCPT ); Mon, 6 Feb 2023 15:50:58 -0500 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 426261F917; Mon, 6 Feb 2023 12:50:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716655; x=1707252655; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dHwcCEIBHwYLkC6sY/0WBKRXyhrXJ9a0pJB/q9IXXU0=; b=jEPKvdnoule+NB+I61CvRB2gbtmpTFkpBFNNeY/88YvYjPfNYdsOSWBl fYo8d0opUiiz23P9138Y7MQPNlaVLMDT3MJas19z0vl5Nqd5f4RkfCCvq 0mZeV5f5GZ6zg8AF96lScN7dTnMngRzbRs/3AU0cDgNeIsur3Yo1VFc7C KDiTrQWkPoQRtAStbkV1gq/2M+13XqjmHdPaGgeQLXAF8uNonHz9FjL6w jzOL8K7MCeJcebzoiPUSX+u6AaX+99easWA/TYJ0MZyz6ZF2ZCljF05Uz VB+vWXxXvIRrG6pIBGEhn1nf+3QV7wrYaFgfkUwqyHEyyk3+JWdpvJx4E g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="391708411" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="391708411" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:50:55 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="840492964" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="840492964" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:50:54 -0800 Subject: [PATCH 10/18] PCI: Export pcie_get_speed() using the code from sysfs PCI link speed show function From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:50:52 -0700 Message-ID: <167571665075.587790.11513782507200128278.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Move the logic in current_link_speed_show() to a common function and export that functiuon as pcie_get_speed() to allow other drivers to to retrieve the current negotiated link speed. Signed-off-by: Dave Jiang --- drivers/pci/pci-sysfs.c | 12 ++---------- drivers/pci/pci.c | 20 ++++++++++++++++++++ include/linux/pci.h | 1 + 3 files changed, 23 insertions(+), 10 deletions(-) diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c index dd0d9d9bc509..0217bb5ca8fa 100644 --- a/drivers/pci/pci-sysfs.c +++ b/drivers/pci/pci-sysfs.c @@ -205,17 +205,9 @@ static ssize_t current_link_speed_show(struct device *dev, struct device_attribute *attr, char *buf) { struct pci_dev *pci_dev = to_pci_dev(dev); - u16 linkstat; - int err; - enum pci_bus_speed speed; - - err = pcie_capability_read_word(pci_dev, PCI_EXP_LNKSTA, &linkstat); - if (err) - return -EINVAL; - speed = pcie_link_speed[linkstat & PCI_EXP_LNKSTA_CLS]; - - return sysfs_emit(buf, "%s\n", pci_speed_string(speed)); + return sysfs_emit(buf, "%s\n", + pci_speed_string(pcie_get_speed(pci_dev))); } static DEVICE_ATTR_RO(current_link_speed); diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index fba95486caaf..d0131b5623b1 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -6215,6 +6215,26 @@ enum pcie_link_width pcie_get_width_cap(struct pci_dev *dev) } EXPORT_SYMBOL(pcie_get_width_cap); +/** + * pcie_get_speed - query for the PCI device's current link speed + * @dev: PCI device to query + * + * Query the PCI device current link speed. + */ +enum pci_bus_speed pcie_get_speed(struct pci_dev *dev) +{ + u16 linkstat, cls; + int err; + + err = pcie_capability_read_word(dev, PCI_EXP_LNKSTA, &linkstat); + if (err) + return PCI_SPEED_UNKNOWN; + + cls = FIELD_GET(PCI_EXP_LNKSTA_CLS, linkstat); + return pcie_link_speed[cls]; +} +EXPORT_SYMBOL(pcie_get_speed); + /** * pcie_bandwidth_capable - calculate a PCI device's link bandwidth capability * @dev: PCI device diff --git a/include/linux/pci.h b/include/linux/pci.h index adffd65e84b4..6a065986ff8f 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -303,6 +303,7 @@ enum pci_bus_speed { PCI_SPEED_UNKNOWN = 0xff, }; +enum pci_bus_speed pcie_get_speed(struct pci_dev *dev); enum pci_bus_speed pcie_get_speed_cap(struct pci_dev *dev); enum pcie_link_width pcie_get_width_cap(struct pci_dev *dev); From patchwork Mon Feb 6 20:51:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130584 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0012C05027 for ; Mon, 6 Feb 2023 20:51:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230370AbjBFUvG (ORCPT ); Mon, 6 Feb 2023 15:51:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229984AbjBFUvF (ORCPT ); Mon, 6 Feb 2023 15:51:05 -0500 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 611F12A9B9; Mon, 6 Feb 2023 12:51:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716664; x=1707252664; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=k6tkI3jQT3b1QSCpxAKZuKl9uTp2US9fbB/Zkgo7434=; b=aUwa4iHSI5n1Jlio8abtSdLseBuPKvNd+zoz59gmmPJbOarewhtC5fnA M/myhP90wh27628J31ZbzFLQmuZ8QmTk5Cr4ir8cWKKmR/mWQ6DokOzbe bnl3uYfKbJARgQA3LnXRPN75MFbhXRyo6RuHco4/tIkw6kFp1I+arPG+A dm9XN4o/AivFlMYLxHVgJTezZeRQUQOjEn5CfQUzSNNkNBoKRiSRU1dpA u8JnLY7o3nU7tQOb5bguHGcDtOqqAi3OVk+2VXYTGzoGXZ63ITr9Bna11 Jx6GXwmSHbUu3eZbnnMaAbPwhLI4+BWuEthwMb8pwbxDM5KtzxVKd/Mhv g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="391708461" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="391708461" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:51:03 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="840493008" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="840493008" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:51:03 -0800 Subject: [PATCH 11/18] PCI: Export pcie_get_width() using the code from sysfs PCI link width show function From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:51:01 -0700 Message-ID: <167571666013.587790.16270669112177554916.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Move the logic in current_link_width_show() to a common function and export that functiuon as pcie_get_width() to allow other drivers to to retrieve the current negotiated link width. Signed-off-by: Dave Jiang --- drivers/pci/pci-sysfs.c | 9 +-------- drivers/pci/pci.c | 20 ++++++++++++++++++++ include/linux/pci.h | 1 + 3 files changed, 22 insertions(+), 8 deletions(-) diff --git a/drivers/pci/pci-sysfs.c b/drivers/pci/pci-sysfs.c index 0217bb5ca8fa..139096c39380 100644 --- a/drivers/pci/pci-sysfs.c +++ b/drivers/pci/pci-sysfs.c @@ -215,15 +215,8 @@ static ssize_t current_link_width_show(struct device *dev, struct device_attribute *attr, char *buf) { struct pci_dev *pci_dev = to_pci_dev(dev); - u16 linkstat; - int err; - err = pcie_capability_read_word(pci_dev, PCI_EXP_LNKSTA, &linkstat); - if (err) - return -EINVAL; - - return sysfs_emit(buf, "%u\n", - (linkstat & PCI_EXP_LNKSTA_NLW) >> PCI_EXP_LNKSTA_NLW_SHIFT); + return sysfs_emit(buf, "%u\n", pcie_get_width(pci_dev)); } static DEVICE_ATTR_RO(current_link_width); diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index d0131b5623b1..0858fa2f1c2d 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -6235,6 +6235,26 @@ enum pci_bus_speed pcie_get_speed(struct pci_dev *dev) } EXPORT_SYMBOL(pcie_get_speed); +/** + * pcie_get_width - query for the PCI device's current link width + * @dev: PCI device to query + * + * Query the PCI device current negoiated width. + */ + +enum pcie_link_width pcie_get_width(struct pci_dev *dev) +{ + u16 linkstat; + int err; + + err = pcie_capability_read_word(dev, PCI_EXP_LNKSTA, &linkstat); + if (err) + return PCIE_LNK_WIDTH_UNKNOWN; + + return FIELD_GET(PCI_EXP_LNKSTA_NLW, linkstat); +} +EXPORT_SYMBOL(pcie_get_width); + /** * pcie_bandwidth_capable - calculate a PCI device's link bandwidth capability * @dev: PCI device diff --git a/include/linux/pci.h b/include/linux/pci.h index 6a065986ff8f..21eca09a98e2 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -305,6 +305,7 @@ enum pci_bus_speed { enum pci_bus_speed pcie_get_speed(struct pci_dev *dev); enum pci_bus_speed pcie_get_speed_cap(struct pci_dev *dev); +enum pcie_link_width pcie_get_width(struct pci_dev *dev); enum pcie_link_width pcie_get_width_cap(struct pci_dev *dev); struct pci_vpd { From patchwork Mon Feb 6 20:51:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130585 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98B38C636D4 for ; Mon, 6 Feb 2023 20:51:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230057AbjBFUvQ (ORCPT ); Mon, 6 Feb 2023 15:51:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230416AbjBFUvO (ORCPT ); Mon, 6 Feb 2023 15:51:14 -0500 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D8541F917; Mon, 6 Feb 2023 12:51:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716673; x=1707252673; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2upboONhPsEQRYVRQaehuwu0mriRbjkDwyzG7PvshPA=; b=JRkSMXG5/oUfK+twbkWccGOH9j40oGSLjgMIMABc8eRF//RsdYCMFjXV iT98cP4JLzPZM6LCaPJ8IFBwMrZwqEznmVmPqh4VwG6yR4/lHTwBur0Ab ILJZX9nZdv3McVMUQ+glWKnX1/xah6jtPqwBPYNd0Y2mY5BgyfREg1zLe HW03pEJmCFE+rJnKAqYrqjfiucR88Qgfa+/L0LGI6LIfLy8AyoGxP6zhk 3JEydzf9v3AcDtsKMrvDk/Ih9kJaQaUVGTHGedqyrCXS6PA2jgYY1zBOq UCXdKJTi0cXZ+YEdkadauR5TUFO2Yrz/nbc7hJHbDiDTVHPgLHM6C7zID Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="391708501" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="391708501" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:51:13 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="840493029" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="840493029" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:51:12 -0800 Subject: [PATCH 12/18] cxl: Add helpers to calculate pci latency for the CXL device From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:51:10 -0700 Message-ID: <167571666898.587790.4824622451425607591.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org The latency is calculated by dividing the FLIT size over the bandwidth. Add support to retrieve the FLIT size for the CXL device and calculate the latency of the downstream link. Signed-off-by: Dave Jiang --- drivers/cxl/core/pci.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxlpci.h | 14 ++++++++++ 2 files changed, 81 insertions(+) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index a24dac36bedd..54ac6f8825ff 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -633,3 +633,70 @@ void read_cdat_data(struct cxl_port *port) } } EXPORT_SYMBOL_NS_GPL(read_cdat_data, CXL); + +static int pcie_speed_to_mbps(enum pci_bus_speed speed) +{ + switch (speed) { + case PCIE_SPEED_2_5GT: + return 2500; + case PCIE_SPEED_5_0GT: + return 5000; + case PCIE_SPEED_8_0GT: + return 8000; + case PCIE_SPEED_16_0GT: + return 16000; + case PCIE_SPEED_32_0GT: + return 32000; + case PCIE_SPEED_64_0GT: + return 64000; + default: + break; + } + + return -EINVAL; +} + +static int cxl_pci_mbits_to_mbytes(struct pci_dev *pdev) +{ + int mbits; + + mbits = pcie_speed_to_mbps(pcie_get_speed(pdev)); + if (mbits < 0) + return mbits; + + return mbits >> 3; +} + +static int cxl_get_flit_size(struct pci_dev *pdev) +{ + if (cxl_pci_flit_256(pdev)) + return 256; + + return 66; +} + +/** + * cxl_pci_get_latency - calculate the link latency for the PCIe link + * @pdev - PCI device + * + * CXL Memory Device SW Guide v1.0 2.11.4 Link latency calculation + * Link latency = LinkPropagationLatency + FlitLatency + RetimerLatency + * LinkProgationLatency is negligible, so 0 will be used + * RetimerLatency is assumed to be neglibible and 0 will be used + * FlitLatency = FlitSize / LinkBandwidth + * FlitSize is defined by spec. CXL v3.0 4.2.1. + * 68B flit is used up to 32GT/s. >32GT/s, 256B flit size is used. + * The FlitLatency is converted to pico-seconds. + */ +long cxl_pci_get_latency(struct pci_dev *pdev) +{ + long bw, flit_size; + + bw = cxl_pci_mbits_to_mbytes(pdev); + if (bw < 0) + return bw; + + flit_size = cxl_get_flit_size(pdev); + return flit_size * 1000000L / bw; +} +EXPORT_SYMBOL_NS_GPL(cxl_pci_get_latency, CXL); diff --git a/drivers/cxl/cxlpci.h b/drivers/cxl/cxlpci.h index 920909791bb9..d64a3e0458ab 100644 --- a/drivers/cxl/cxlpci.h +++ b/drivers/cxl/cxlpci.h @@ -62,8 +62,22 @@ enum cxl_regloc_type { CXL_REGLOC_RBI_TYPES }; +/* + * CXL v3.0 6.2.3 Table 6-4 + * The table indicates that if PCIe Flit Mode is set, then CXL is in 256B flits + * mode, otherwise it's 68B flits mode. + */ +static inline bool cxl_pci_flit_256(struct pci_dev *pdev) +{ + u32 lnksta2; + + pcie_capability_read_dword(pdev, PCI_EXP_LNKSTA2, &lnksta2); + return lnksta2 & BIT(10); +} + int devm_cxl_port_enumerate_dports(struct cxl_port *port); struct cxl_dev_state; int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm); void read_cdat_data(struct cxl_port *port); +long cxl_pci_get_latency(struct pci_dev *pdev); #endif /* __CXL_PCI_H__ */ From patchwork Mon Feb 6 20:51:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130586 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0366DC636D3 for ; Mon, 6 Feb 2023 20:51:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230392AbjBFUvZ (ORCPT ); Mon, 6 Feb 2023 15:51:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49002 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230354AbjBFUvY (ORCPT ); Mon, 6 Feb 2023 15:51:24 -0500 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DDF032CC75; Mon, 6 Feb 2023 12:51:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716682; x=1707252682; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7O4X6qGSX/zqCT/RteJSiBi67krWrHZ5oj0nquyYo4E=; b=A34CYjWWBaewdOMX/sDjQPomVG7y3O7v75KR77Zud+tWMW6SLXq8czBb o1+P4HZnza1nWCZvXhUD5qZ2UCj2HWmGMpB57gQBO8RKAB+iI090/zS0/ b7qm4FKLqHMsYgWt+P8DiJZUDEjvL4BC/kzGuhWayczoY8SvZ63WOeqDO 603jyvatNXCi7Sqxkkv3VPZ32+WnSJ1TL+Xw1xObBHjA85Q33gqQoIwZQ 8450okwD1+d6CONJmx4ATdgs2EozgremCCxxI1u9Jt4lIS+PN3Hh36ZTx U3Sy72eNcKvMFK/L8nOaGKle1KuVYlI1vtMUCFvlMCTqxGmlS6fQzLdbG A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="391708550" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="391708550" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:51:22 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="840493062" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="840493062" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:51:21 -0800 Subject: [PATCH 13/18] cxl: Add latency and bandwidth calculations for the CXL path From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:51:19 -0700 Message-ID: <167571667794.587790.14172786993094257614.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org CXL Memory Device SW Guide rev1.0 2.11.2 provides instruction on how to caluclate latency and bandwidth for CXL memory device. Calculate minimum bandwidth and total latency for the path from the CXL device to the root port. The calculates values are stored in the cached DSMAS entries attached to the cxl_port of the CXL device. For example for a device that is directly attached to a host bus: Total Latency = Device Latency (from CDAT) + Dev to Host Bus (HB) Link Latency Min Bandwidth = Link Bandwidth between Host Bus and CXL device For a device that has a switch in between host bus and CXL device: Total Latency = Device (CDAT) Latency + Dev to Switch Link Latency + Switch (CDAT) Latency + Switch to HB Link Latency Min Bandwidth = min(dev to switch bandwidth, switch to HB bandwidth) Signed-off-by: Dave Jiang The internal latency for a switch can be retrieved from the CDAT of the switch PCI device. However, since there's no easy way to retrieve that right now on Linux, a guesstimated constant is used per switch to simplify the driver code. Signed-off-by: Dave Jiang --- drivers/cxl/core/port.c | 60 +++++++++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 9 +++++++ drivers/cxl/port.c | 42 +++++++++++++++++++++++++++++++++ 3 files changed, 111 insertions(+) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 2b27319cfd42..aa260361ba7d 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1899,6 +1899,66 @@ bool schedule_cxl_memdev_detach(struct cxl_memdev *cxlmd) } EXPORT_SYMBOL_NS_GPL(schedule_cxl_memdev_detach, CXL); +int cxl_port_get_downstream_qos(struct cxl_port *port, long *bw, long *lat) +{ + long total_lat = 0, latency; + long min_bw = INT_MAX; + struct pci_dev *pdev; + struct cxl_port *p; + struct device *dev; + int devices = 0; + + /* Grab the device that is the PCI device for CXL memdev */ + dev = port->uport->parent; + /* Skip if it's not PCI, most likely a cxl_test device */ + if (!dev_is_pci(dev)) + return 0; + + pdev = to_pci_dev(dev); + min_bw = pcie_bandwidth_available(pdev, NULL, NULL, NULL); + if (min_bw == 0) + return -ENXIO; + + /* convert to MB/s from Mb/s */ + min_bw >>= 3; + + p = port; + do { + struct cxl_dport *dport; + + latency = cxl_pci_get_latency(pdev); + if (latency < 0) + return latency; + + total_lat += latency; + devices++; + + dport = p->parent_dport; + if (!dport) + break; + + p = dport->port; + dev = p->uport; + if (!dev_is_pci(dev)) + break; + pdev = to_pci_dev(dev); + } while (1); + + /* + * Add an approximate latency to the switch. Currently there + * is no easy mechanism to read the CDAT for switches. 'devices' + * should account for all the PCI devices encountered minus the + * root device. So the number of switches would be 'devices - 1' + * to account for the CXL device. + */ + total_lat += CXL_SWITCH_APPROX_LAT * (devices - 1); + + *bw = min_bw; + *lat = total_lat; + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_port_get_downstream_qos, CXL); + /* for user tooling to ensure port disable work has completed */ static ssize_t flush_store(struct bus_type *bus, const char *buf, size_t count) { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index ac6ea550ab0a..86668fab6e91 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -480,6 +480,13 @@ struct cxl_pmem_region { struct cxl_pmem_region_mapping mapping[]; }; +/* + * Set in picoseconds per ACPI spec 6.5 Table 5.148 Entry Base Unit. + * This is an approximate constant to use for switch latency calculation + * until there's a way to access switch CDAT. + */ +#define CXL_SWITCH_APPROX_LAT 5000 + /** * struct cxl_port - logical collection of upstream port devices and * downstream port devices to construct a CXL memory @@ -706,6 +713,7 @@ struct dsmas_entry { struct range dpa_range; u16 handle; u64 qos[ACPI_HMAT_WRITE_BANDWIDTH + 1]; + int qtg_id; }; typedef int (*cdat_tbl_entry_handler)(struct acpi_cdat_header *header, void *arg); @@ -734,6 +742,7 @@ struct qtg_dsm_output { struct qtg_dsm_output *cxl_acpi_evaluate_qtg_dsm(acpi_handle handle, struct qtg_dsm_input *input); acpi_handle cxl_acpi_get_rootdev_handle(struct device *dev); +int cxl_port_get_downstream_qos(struct cxl_port *port, long *bw, long *lat); /* * Unit test builds overrides this to __weak, find the 'strong' version diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 8de311208b37..d72e38f9ae44 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -30,6 +30,44 @@ static void schedule_detach(void *cxlmd) schedule_cxl_memdev_detach(cxlmd); } +static int cxl_port_qos_calculate(struct cxl_port *port) +{ + struct qtg_dsm_output *output; + struct qtg_dsm_input input; + struct dsmas_entry *dent; + long min_bw, total_lat; + acpi_handle handle; + int rc; + + rc = cxl_port_get_downstream_qos(port, &min_bw, &total_lat); + if (rc) + return rc; + + handle = cxl_acpi_get_rootdev_handle(&port->dev); + if (IS_ERR(handle)) + return PTR_ERR(handle); + + mutex_lock(&port->cdat.dsmas_lock); + list_for_each_entry(dent, &port->cdat.dsmas_list, list) { + input.rd_lat = dent->qos[ACPI_HMAT_READ_LATENCY] + total_lat; + input.wr_lat = dent->qos[ACPI_HMAT_WRITE_LATENCY] + total_lat; + input.rd_bw = min_t(int, min_bw, + dent->qos[ACPI_HMAT_READ_BANDWIDTH]); + input.wr_bw = min_t(int, min_bw, + dent->qos[ACPI_HMAT_WRITE_BANDWIDTH]); + + output = cxl_acpi_evaluate_qtg_dsm(handle, &input); + if (IS_ERR(output)) + continue; + + dent->qtg_id = output->qtg_ids[0]; + kfree(output); + } + mutex_unlock(&port->cdat.dsmas_lock); + + return 0; +} + static int cxl_port_probe(struct device *dev) { struct cxl_port *port = to_cxl_port(dev); @@ -74,6 +112,10 @@ static int cxl_port_probe(struct device *dev) } else { dev_dbg(dev, "Failed to parse DSMAS: %d\n", rc); } + + rc = cxl_port_qos_calculate(port); + if (rc) + dev_dbg(dev, "Failed to do QoS calculations\n"); } rc = cxl_hdm_decode_init(cxlds, cxlhdm); From patchwork Mon Feb 6 20:51:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130587 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CC43C636D4 for ; Mon, 6 Feb 2023 20:51:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230397AbjBFUvf (ORCPT ); Mon, 6 Feb 2023 15:51:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230263AbjBFUve (ORCPT ); Mon, 6 Feb 2023 15:51:34 -0500 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 25D8E1DBB8; Mon, 6 Feb 2023 12:51:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716691; x=1707252691; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=uQRUC8QEU+C+pD79WhvBKGvAFobEMV5CpOt5lEMC6XE=; b=TWv2Ml5pc/B1KK6PJ9q/w0wT+BP/o/v1oYxmM/HxwTexFlQcFzkrTR9S Qk4vNihB7ZQq+vOKprUokQDMhIcCcEmNTK8wtahIDMUX74NdPDMO+VfRZ 1xHzd8w/Oe+m+t/RJ1cqbi5skslKXM0QN/1wl1O87Bs4fdbv37uvM2sLq dVTNXYRfikoue7fc8pW09wkdEF9lLhrHY57ec1WrCJufNiTl6teY3GXoZ gPgpyDzyAh6lKMDL1Zm/gGUeqzb/74L+X7lPAAqJ5WDzvnTQsvjMhk0bk gABLIlx8a6W3fWDmDdp/7KQFczMkpogkD5rvn1LiztuE1nk+jPZ/dONaJ g==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="308958787" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="308958787" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:51:30 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="735280218" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="735280218" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:51:30 -0800 Subject: [PATCH 14/18] cxl: Wait Memory_Info_Valid before access memory related info From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:51:28 -0700 Message-ID: <167571668726.587790.16814881883553586342.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org CXL rev3.0 8.1.3.8.2 Memory_Info_valid field The Memory_Info_Valid bit indicates that the CXL Range Size High and Size Low registers are valid. The bit must be set within 1 second of reset deassertion to the device. Check valid bit before we check the Memory_Active bit when waiting for cxl_await_media_ready() to ensure that the memory info is valid for consumption. Signed-off-by: Dave Jiang --- drivers/cxl/core/pci.c | 25 +++++++++++++++++++++++-- drivers/cxl/port.c | 20 ++++++++++---------- 2 files changed, 33 insertions(+), 12 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 54ac6f8825ff..79a1348e7b98 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -111,11 +111,32 @@ int cxl_await_media_ready(struct cxl_dev_state *cxlds) int d = cxlds->cxl_dvsec; bool active = false; u64 md_status; + u32 temp; int rc, i; - for (i = media_ready_timeout; i; i--) { - u32 temp; + /* Check MEM INFO VALID bit first, give up after 1s */ + i = 1; + do { + rc = pci_read_config_dword(pdev, + d + CXL_DVSEC_RANGE_SIZE_LOW(0), + &temp); + if (rc) + return rc; + active = FIELD_GET(CXL_DVSEC_MEM_INFO_VALID, temp); + if (active) + break; + msleep(1000); + } while (i--); + + if (!active) { + dev_err(&pdev->dev, + "timeout awaiting memory valid after 1 second.\n"); + return -ETIMEDOUT; + } + + /* Check MEM ACTIVE bit, up to 60s timeout by default */ + for (i = media_ready_timeout; i; i--) { rc = pci_read_config_dword( pdev, d + CXL_DVSEC_RANGE_SIZE_LOW(0), &temp); if (rc) diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index d72e38f9ae44..03380c18fc52 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -99,6 +99,16 @@ static int cxl_port_probe(struct device *dev) if (rc) return rc; + rc = cxl_hdm_decode_init(cxlds, cxlhdm); + if (rc) + return rc; + + rc = cxl_await_media_ready(cxlds); + if (rc) { + dev_err(dev, "Media not active (%d)\n", rc); + return rc; + } + if (port->cdat.table) { rc = cdat_table_parse_dsmas(port->cdat.table, cxl_dsmas_parse_entry, @@ -117,16 +127,6 @@ static int cxl_port_probe(struct device *dev) if (rc) dev_dbg(dev, "Failed to do QoS calculations\n"); } - - rc = cxl_hdm_decode_init(cxlds, cxlhdm); - if (rc) - return rc; - - rc = cxl_await_media_ready(cxlds); - if (rc) { - dev_err(dev, "Media not active (%d)\n", rc); - return rc; - } } rc = devm_cxl_enumerate_decoders(cxlhdm); From patchwork Mon Feb 6 20:51:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130588 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D0B6C05027 for ; Mon, 6 Feb 2023 20:51:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230253AbjBFUvs (ORCPT ); Mon, 6 Feb 2023 15:51:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49212 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230354AbjBFUvo (ORCPT ); Mon, 6 Feb 2023 15:51:44 -0500 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A59012DE73; Mon, 6 Feb 2023 12:51:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716700; x=1707252700; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Hv/W896d+eaEclcVrbd282vgqnn+BAffzWLknUdDUJk=; b=YRhREw5gro5NYQfRTwAN0jPlEe87bZooZdPMUWaMNFQDj6miAnWQDCq/ vTK/ZZJEtLYg2PNdKpCNGey2xdEMBHovlHKhcSVPubiahu9B/4SXbWXne UPPoGejBlG5Vo1wQ1I5OR1fQMGoRua2cIBbIGQRUTt2af/BkOmKNOmPBb TV/UUhlbPR3oH5GbQWCCrgK3IkatSULPaV/8TyPWbAoFXQwhROpLeEqHi GDG6sbhcoeDpwuGx+8zMu3F6YidF0I+u7WetpUTbfT98p6+3tH5dBzJrk Rpq+sFOmQLe3k/UH9+5jAst6JedyUvRokMWVkWU7lnCpu2oQdPhLT+qny Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="327946615" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="327946615" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:51:40 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="790552345" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="790552345" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:51:39 -0800 Subject: [PATCH 15/18] cxl: Move identify and partition query from pci probe to port probe From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:51:37 -0700 Message-ID: <167571669593.587790.12939497495344674151.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Move the enumeration of device capacity to cxl_port_probe() from cxl_pci_probe(). The size and capacity information should be read after cxl_await_media_ready() so the data is valid. Signed-off-by: Dave Jiang --- drivers/cxl/pci.c | 8 -------- drivers/cxl/port.c | 8 ++++++++ 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 258004f34281..e35ed250214e 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -484,14 +484,6 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (rc) return rc; - rc = cxl_dev_state_identify(cxlds); - if (rc) - return rc; - - rc = cxl_mem_create_range_info(cxlds); - if (rc) - return rc; - cxlmd = devm_cxl_add_memdev(cxlds); if (IS_ERR(cxlmd)) return PTR_ERR(cxlmd); diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 03380c18fc52..b7a4a1be2945 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -127,6 +127,14 @@ static int cxl_port_probe(struct device *dev) if (rc) dev_dbg(dev, "Failed to do QoS calculations\n"); } + + rc = cxl_dev_state_identify(cxlds); + if (rc) + return rc; + + rc = cxl_mem_create_range_info(cxlds); + if (rc) + return rc; } rc = devm_cxl_enumerate_decoders(cxlhdm); From patchwork Mon Feb 6 20:51:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 412BCC636D3 for ; Mon, 6 Feb 2023 20:51:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230403AbjBFUvv (ORCPT ); Mon, 6 Feb 2023 15:51:51 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49252 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230354AbjBFUvu (ORCPT ); Mon, 6 Feb 2023 15:51:50 -0500 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F1E492B097; Mon, 6 Feb 2023 12:51:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716709; x=1707252709; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ANJK4WgXvZBb1RvzfDrGlJT/1Xf1pViUoSVpIMURtjI=; b=UFFYySA1LWF+9VBYxRHlZFKTA3MveMNcoufbCJvzRZcCekQUtFeIESsb S1yo7yBWVADFLTCFsTx749F9A8aCZSNYa0EFzJ8WVMNwHBW3rXnHvb3d6 55scEKV4a4/VzxHsHIyium4w+Gcbf080f2zi/xBYWI9Q7826jBXeoOEKq uRqykxNH2mtPi9PERlIi7TkMy/6uU3fGh3CbDkB56n5UBBIpmOuQpJ7gY UpfUd6oQBSPRoeZfFfaTv/i0NrotkjnzSCHqTiK8u69/KyMchMmI9Q6OP 198pnKk1dL4mQ4txGjntmF1ehuWH7iOpQY1qKH0pOQIvmHN90bY3rEoTH w==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="327946637" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="327946637" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:51:49 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="790552372" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="790552372" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:51:48 -0800 Subject: [PATCH 16/18] cxl: Move reading of CDAT data from device to after media is ready From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:51:46 -0700 Message-ID: <167571670516.587790.14112456054041985666.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org The CDAT data is only valid after the hardware signals the media is ready. Move the reading to after cxl_await_media_ready() has succeeded. Signed-off-by: Dave Jiang --- drivers/cxl/port.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index b7a4a1be2945..6b2ad22487f5 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -91,9 +91,6 @@ static int cxl_port_probe(struct device *dev) struct cxl_memdev *cxlmd = to_cxl_memdev(port->uport); struct cxl_dev_state *cxlds = cxlmd->cxlds; - /* Cache the data early to ensure is_visible() works */ - read_cdat_data(port); - get_device(&cxlmd->dev); rc = devm_add_action_or_reset(dev, schedule_detach, cxlmd); if (rc) @@ -109,6 +106,8 @@ static int cxl_port_probe(struct device *dev) return rc; } + /* Cache the data early to ensure is_visible() works */ + read_cdat_data(port); if (port->cdat.table) { rc = cdat_table_parse_dsmas(port->cdat.table, cxl_dsmas_parse_entry, From patchwork Mon Feb 6 20:51:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130590 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD909C05027 for ; Mon, 6 Feb 2023 20:52:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230154AbjBFUwA (ORCPT ); Mon, 6 Feb 2023 15:52:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49390 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230354AbjBFUv7 (ORCPT ); Mon, 6 Feb 2023 15:51:59 -0500 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38A1D2A9B9; Mon, 6 Feb 2023 12:51:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716719; x=1707252719; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=JpDWvJWFRhu/T3wx6PiMHyV4AusHz3nMjcnJti5tzgQ=; b=C4ac6kTbhvRe4YFdoKZ0+CdGsPD5tbHltLifxfRzanL4u2ICN/Q77yQT 4CBGBnDJI4E0I3Py3ur3X/0k2O72qg1+0WHJHnNZngSPmsW0DhG53oJiY E6buD3syBJM4F+2C/Dwfm3+Dz4tPbzPcF/xTgWOyaQgpzv+cV/FIvicCL JwTv+Uf+aSrMLvXOkFJsB/401MRaYWr1wnbyh23uRkJ3P5cBUxzw1Nz0Q WjAqbogEHKkBANw3SZDelxW0FBqjrwdp3GTu49IbGUvIS1Ou/608SwZ9L G74AyC5ReSLSY2bj6mrswsnOk405IiFYXBS6ev0hFUW7FGCf+XfZmE7Iw A==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="327946663" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="327946663" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:51:58 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="790552392" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="790552392" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:51:57 -0800 Subject: [PATCH 17/18] cxl: Attach QTG IDs to the DPA ranges for the device From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:51:55 -0700 Message-ID: <167571671461.587790.3035047445704886954.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Match the DPA ranges of the mem device and the calcuated DPA range attached to the DSMAS. If a match is found, then assign the QTG ID to the relevant DPA range of the memory device. Signed-off-by: Dave Jiang --- drivers/cxl/core/mbox.c | 2 ++ drivers/cxl/cxlmem.h | 2 ++ drivers/cxl/port.c | 35 +++++++++++++++++++++++++++++++++++ 3 files changed, 39 insertions(+) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index b03fba212799..2a7b07d65010 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -869,6 +869,8 @@ struct cxl_dev_state *cxl_dev_state_create(struct device *dev) mutex_init(&cxlds->mbox_mutex); cxlds->dev = dev; + cxlds->pmem_qtg_id = -1; + cxlds->ram_qtg_id = -1; return cxlds; } diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index ab138004f644..d88b88ecc807 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -251,6 +251,8 @@ struct cxl_dev_state { struct resource dpa_res; struct resource pmem_res; struct resource ram_res; + int pmem_qtg_id; + int ram_qtg_id; u64 total_bytes; u64 volatile_only_bytes; u64 persistent_only_bytes; diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 6b2ad22487f5..c4cee69d6625 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -68,6 +68,39 @@ static int cxl_port_qos_calculate(struct cxl_port *port) return 0; } +static bool dpa_match_qtg_range(struct range *dpa, struct range *qtg) +{ + if (dpa->start >= qtg->start && dpa->end <= qtg->end) + return true; + return false; +} + +static void cxl_dev_set_qtg(struct cxl_port *port, struct cxl_dev_state *cxlds) +{ + struct dsmas_entry *dent; + struct range ram_range = { + .start = cxlds->ram_res.start, + .end = cxlds->ram_res.end, + }; + struct range pmem_range = { + .start = cxlds->pmem_res.start, + .end = cxlds->pmem_res.end, + }; + + mutex_lock(&port->cdat.dsmas_lock); + list_for_each_entry(dent, &port->cdat.dsmas_list, list) { + if (dpa_match_qtg_range(&ram_range, &dent->dpa_range)) { + cxlds->ram_qtg_id = dent->qtg_id; + break; + } + if (dpa_match_qtg_range(&pmem_range, &dent->dpa_range)) { + cxlds->pmem_qtg_id = dent->qtg_id; + break; + } + } + mutex_unlock(&port->cdat.dsmas_lock); +} + static int cxl_port_probe(struct device *dev) { struct cxl_port *port = to_cxl_port(dev); @@ -134,6 +167,8 @@ static int cxl_port_probe(struct device *dev) rc = cxl_mem_create_range_info(cxlds); if (rc) return rc; + + cxl_dev_set_qtg(port, cxlds); } rc = devm_cxl_enumerate_decoders(cxlhdm); From patchwork Mon Feb 6 20:52:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13130591 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD03DC636D3 for ; Mon, 6 Feb 2023 20:52:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230415AbjBFUwL (ORCPT ); Mon, 6 Feb 2023 15:52:11 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49564 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230354AbjBFUwL (ORCPT ); Mon, 6 Feb 2023 15:52:11 -0500 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B363A2E825; Mon, 6 Feb 2023 12:52:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675716727; x=1707252727; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fbw0gfA3CV6zl+8Xhzyb5C6vwjJRzvTIToKyZi8GRJ8=; b=kPz8dQLcYlw1nLTMALh8ePtTx/1k9pYQ3GZSNh4zU9fd/95cuSRjMNYO F4mgIOOTc4BoA2v8AMcFZjZzBWkh+3onjKYSy28Jdn6ftFmarJ+z53h+0 fgqMX3sixbhRlmn6MawqwADRBboF2vC/dBOuhwb6oMjqF+1USKGNq0jZA yG6GzLCEAQ06qqPyAAK6q9EurOtHt4mpGfmxkv4atsUzfnFTKNjyf9Ph9 lvLhmsih2Q+hyhmZz6JbrpG+jMwLvOrMoXtvmyn++L1Zn0ryiZjKGpbZE Te4AQszL5Y4RKqvc7M8xCOPcrouQtGgJrFAURGgBb5Ez+QFpHshhS1HmB Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="329328291" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="329328291" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:52:07 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10613"; a="809264138" X-IronPort-AV: E=Sophos;i="5.97,276,1669104000"; d="scan'208";a="809264138" Received: from djiang5-mobl3.amr.corp.intel.com (HELO djiang5-mobl3.local) ([10.212.111.195]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Feb 2023 12:52:07 -0800 Subject: [PATCH 18/18] cxl: Export sysfs attributes for device QTG IDs From: Dave Jiang To: linux-cxl@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, rafael@kernel.org, bhelgaas@google.com, robert.moore@intel.com Date: Mon, 06 Feb 2023 13:52:05 -0700 Message-ID: <167571672370.587790.13206197631776290440.stgit@djiang5-mobl3.local> In-Reply-To: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> References: <167571650007.587790.10040913293130712882.stgit@djiang5-mobl3.local> User-Agent: StGit/1.5 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Export qtg_id sysfs attributes for the respective ram and pmem DPA range of a CXL device. The QTG ID should show up as /sys/bus/cxl/devices/memX/pmem/qtg_id for pmem or as /sys/bus/cxl/devices/memX/ram/qtg_id for ram. Signed-off-by: Dave Jiang --- Documentation/ABI/testing/sysfs-bus-cxl | 15 +++++++++++++++ drivers/cxl/core/memdev.c | 26 ++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 0932c2f6fbf4..8133a13e118d 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -27,6 +27,14 @@ Description: identically named field in the Identify Memory Device Output Payload in the CXL-2.0 specification. +What: /sys/bus/cxl/devices/memX/ram/qtg_id +Date: January, 2023 +KernelVersion: v6.3 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) Shows calculated QoS Throttling Group ID for the + "Volatile Only Capacity" DPA range. + What: /sys/bus/cxl/devices/memX/pmem/size Date: December, 2020 @@ -37,6 +45,13 @@ Description: identically named field in the Identify Memory Device Output Payload in the CXL-2.0 specification. +What: /sys/bus/cxl/devices/memX/pmem/qtg_id +Date: January, 2023 +KernelVersion: v6.3 +Contact: linux-cxl@vger.kernel.org +Description: + (RO) Shows calculated QoS Throttling Group ID for the + "Persistent Only Capacity" DPA range. What: /sys/bus/cxl/devices/memX/serial Date: January, 2022 diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index a74a93310d26..06f9ac929ef4 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -76,6 +76,18 @@ static ssize_t ram_size_show(struct device *dev, struct device_attribute *attr, static struct device_attribute dev_attr_ram_size = __ATTR(size, 0444, ram_size_show, NULL); +static ssize_t ram_qtg_id_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_dev_state *cxlds = cxlmd->cxlds; + + return sysfs_emit(buf, "%d\n", cxlds->ram_qtg_id); +} + +static struct device_attribute dev_attr_ram_qtg_id = + __ATTR(qtg_id, 0444, ram_qtg_id_show, NULL); + static ssize_t pmem_size_show(struct device *dev, struct device_attribute *attr, char *buf) { @@ -89,6 +101,18 @@ static ssize_t pmem_size_show(struct device *dev, struct device_attribute *attr, static struct device_attribute dev_attr_pmem_size = __ATTR(size, 0444, pmem_size_show, NULL); +static ssize_t pmem_qtg_id_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_dev_state *cxlds = cxlmd->cxlds; + + return sysfs_emit(buf, "%d\n", cxlds->pmem_qtg_id); +} + +static struct device_attribute dev_attr_pmem_qtg_id = + __ATTR(qtg_id, 0444, pmem_qtg_id_show, NULL); + static ssize_t serial_show(struct device *dev, struct device_attribute *attr, char *buf) { @@ -117,11 +141,13 @@ static struct attribute *cxl_memdev_attributes[] = { static struct attribute *cxl_memdev_pmem_attributes[] = { &dev_attr_pmem_size.attr, + &dev_attr_pmem_qtg_id.attr, NULL, }; static struct attribute *cxl_memdev_ram_attributes[] = { &dev_attr_ram_size.attr, + &dev_attr_ram_qtg_id.attr, NULL, };