From patchwork Mon Jul 15 17:28:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733700 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2075.outbound.protection.outlook.com [40.107.92.75]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F9B8481CD; Mon, 15 Jul 2024 17:28:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.75 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064536; cv=fail; b=CAQmnwglEJNhoJ++7cybPnYKs7YEZ/uTsxIt2xzG7DYokgTEvWHPPYJ5T4Q5JBfQdBLBIQV3evAO5qFcyJvjsxGWIjlim0BsqXJ37pEr/O8Tbj3BWxTrB0wLgoy0ruJNqJvcu/yCH5SrzxR7zstCODn4kznTppNZn7zWIUp7DS8= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064536; c=relaxed/simple; bh=Oa/e9wb5h3jNEP3DHIepxNpYhzzLgaOQrgbxp5b0rxo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=g11mBdE6A8ticJ9AV40QbrkS6RYglWsAef9ImWm0fNDRNo6kbawkLCzZHI9REct1x4i8a7g8EKxUbTG45KvDeSv9L4y3XIAXXHQxTJsapt+EU2o+ekkZ3EYmMGrSa4fHJ7B+zqO3LVlss+TYTF3x62BS8GpW+xrPv/9lD8G3EEI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=S4YVMii4; arc=fail smtp.client-ip=40.107.92.75 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="S4YVMii4" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BvrOLBh+0KT8qqIKFDwkNkiJeO7W/2/hxc7DwZrEhRhHouBNr/3IwcyzIN6xkQ492EaU7SW+JNp25F8lxEYaslQFqKBVbusLL+iLbacBwreL1T9tYghF+mAABTS+7eivYO9qFdt0E58RD7DGGyDhvd+NEMf5NHS9o2BX7ySA607VXugqVotHW4frxJBJQyVMlaPvTMOt831tyHIhhTOhFJn9utUJcMXF+wTovd2Dudaxc2xxGsLqV9yXof3Q/lwL3sEVSQ9gKJ4+Z1Fy2oMb1oHN6SHzCG3T8DSSkf5ohDuqw6VEk9l0SR1lJNiEg8dNzN2q1962Lf3PCaKf3ma67A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YMNBtYKx5gHyDrtdQ/82TEODfgFpLHEKyB2LfAdsiOE=; b=fAmwZECaxz/Ftqe4iXUvJnXvCjTfW/0G0r6m9MqKafcKRKHnU8Rj9RbRXXmuAHF8G1xrNDzarcPCKBIaHHOYABjFCpZPIxqwt1L3QSYvaAR83cXHv+Yej2fM6XFEFaKCyNVkeRGsmv8ugFTF/ypgC/BKxC8sJ7Ak8GdHa/zBtOfLDo5gPxY6+xn+Fn2eOMWvkTCaRuNYHaMHy7FlXm0tGheddlF2HWQa4sdGp7nT/Bqynt/NW3vSEuaKjzWvDujMQoKyxyLStwPGPYYnEFC8wuDPm/dErzppcPfYB8Ms857BlsQECF60qWAFcvdxG1bPgF+teSliW1IPMSa4lgkZmQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YMNBtYKx5gHyDrtdQ/82TEODfgFpLHEKyB2LfAdsiOE=; b=S4YVMii4fnlAWMBEumBC3i4o6m38+UdNtLxuYMgllrPgSe2ha1RkmUbfc6EXu+cfQq1VsUpDzc3mnyak6+IZK7TN++sHORAj4ArzFffTVKQligvkC+YsitHiZS771qOHspnu5kEGOTUgkCkGUTqxpu+ZPg6k5NeL0k5lPWMt7y0= Received: from BYAPR07CA0064.namprd07.prod.outlook.com (2603:10b6:a03:60::41) by SJ1PR12MB6289.namprd12.prod.outlook.com (2603:10b6:a03:458::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.23; Mon, 15 Jul 2024 17:28:45 +0000 Received: from SJ1PEPF000023D4.namprd21.prod.outlook.com (2603:10b6:a03:60:cafe::7d) by BYAPR07CA0064.outlook.office365.com (2603:10b6:a03:60::41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28 via Frontend Transport; Mon, 15 Jul 2024 17:28:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by SJ1PEPF000023D4.mail.protection.outlook.com (10.167.244.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.5 via Frontend Transport; Mon, 15 Jul 2024 17:28:45 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:28:44 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:28:43 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero Subject: [PATCH v2 01/15] cxl: add type2 device basic support Date: Mon, 15 Jul 2024 18:28:21 +0100 Message-ID: <20240715172835.24757-2-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received-SPF: None (SATLEXMB03.amd.com: alejandro.lucero-palau@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF000023D4:EE_|SJ1PR12MB6289:EE_ X-MS-Office365-Filtering-Correlation-Id: fd750184-5d36-402a-fecc-08dca4f3947f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|82310400026|36860700013|921020; X-Microsoft-Antispam-Message-Info: +nrcrbMOW95CQ8TL1MmdUFHHaKvqEVmnB7MH7swcVA22p2gbvAss2L0C8NokDXPKxSEhhFGnvCpUYWudKNHuxxu7GIpCoMrLhuaoK+lmza1hpxlS+B8u02ECiLfCqDDnk8WsRjYjQCI/mop4Aisdb35nFlEHJ5pzl5M9mixzx/80MGBYoC/K/tdtT6AAd6KVHVRy8Brq5Ve6886x0E5mJkNH7s97/NJr/xCgnI8rOGpB3guF04IFChz4WlQ5vlVfkLfL8P0qUxRkJSPo8HhSO6cmfzA0QE/DItgyIb6XbZI58V8n4rNLais42Or+z0afseYYQkXdLjzWcjPlHMFI1+GKFmSjuB9HbMsAmSfXki9VS2vnPhmtweeU+QNvvtKYETCAWF/1FYXjqkgeUBwgeeayN+sOT88kyIQzcqQygpHzELn+1dXQE/ju3lT+wm3bzLqvfMrgwA9EYUbCym1YQoTav+c7qUOyKXg30nTTvucAj8iUjh5l5QD1At/MNJctgaoxGtf4Hb0fGSV7mbLoEwWUfvYZhygXC/fhv36iIEZGMauYLF607WLVN4bDL+BLZl0TfzypdhCIt1+hHkOCdlpzdLs1djtnpr13fcgq8FYM+UUAkB9IYnZAnzNGWF1X6XHXankipk94Kf1TsCfDn+oRBzESX3h1WeuPe3u/68asj+WIm3mgTHHDuHbIkHfZDiOIo4X9xvV0SaX1qwG8XcMNlGaUGIjxLK/kxhZoONE0lsjgKlFWUYcaqWSybAgjzW0LQySoXyzd1gfgJzxCZtX4/HLDgBV86EDyrUNi5V3ljOOx2smU+DeXP2TKgl578jl54qvuKoQa9JbP7ti9NK03ywlj8NmBzgIpl3AtMtEJb+IY6QJJUA46luCTLekw2P75rBipcLpW04HXL1+SCeEHrGCYsbTB1pLoHxJhmtHIF9+GyykC9evJqYG79ea9WC9P90Mn3T3Dg3fXfkdGsYSoeQCZVt+AK2YNkvDbUrd7h8pS01M4brItde/SDdE1keaMzYfF96/6JPv8k9LAG7ORQdWoayXe5STf2m05ilzHCSyw54PuBOZIkw0buh2sAq5vY8KSV6flm6rMo9JjI5K4yfZx7CR8S7RJBi9umv/ETlY8tbayg9Z2lw/vAQTo+5vUa37j6kCNpKEAC6qfT0+/ViOmm4AJLOJbETarDpquo3phFS7yp3gIt+1oX5lOT9svdS2CbWdC/HNhcLdefks3dNRNVYP9dwrhfvssIglVTTUSPfJHg0ZzX6pb6uG/MqAE0lVcsEAp4+JzJTGg/apN5c+3NBaBi/sC8/ePIaPb+ISXux1NfGARK6jPzQzfZK7jnrhauXpmaPgC+kf+SLN3/x+KwnRbKoMVjunCh8MsCXhBqHqhGEll7iuwVY7EHCeG025Hg+GSwao89o59wg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(1800799024)(82310400026)(36860700013)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:28:45.3786 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fd750184-5d36-402a-fecc-08dca4f3947f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF000023D4.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6289 From: Alejandro Lucero Differientiate Type3, aka memory expanders, from Type2, aka device accelerators, with a new function for initializing cxl_dev_state. Create opaque struct to be used by accelerators relying on new access functions in following patches. Add SFC ethernet network driver as the client. Based on https://lore.kernel.org/linux-cxl/168592149709.1948938.8663425987110396027.stgit@dwillia2-xfh.jf.intel.com/T/#m52543f85d0e41ff7b3063fdb9caa7e845b446d0e Signed-off-by: Alejandro Lucero Co-developed-by: Dan Williams --- drivers/cxl/core/memdev.c | 52 ++++++++++++++++++++++++++ drivers/net/ethernet/sfc/Makefile | 2 +- drivers/net/ethernet/sfc/efx.c | 4 ++ drivers/net/ethernet/sfc/efx_cxl.c | 53 +++++++++++++++++++++++++++ drivers/net/ethernet/sfc/efx_cxl.h | 29 +++++++++++++++ drivers/net/ethernet/sfc/net_driver.h | 4 ++ include/linux/cxl_accel_mem.h | 22 +++++++++++ include/linux/cxl_accel_pci.h | 23 ++++++++++++ 8 files changed, 188 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/sfc/efx_cxl.c create mode 100644 drivers/net/ethernet/sfc/efx_cxl.h create mode 100644 include/linux/cxl_accel_mem.h create mode 100644 include/linux/cxl_accel_pci.h diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 0277726afd04..61b5d35b49e7 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -8,6 +8,7 @@ #include #include #include +#include #include "trace.h" #include "core.h" @@ -615,6 +616,25 @@ static void detach_memdev(struct work_struct *work) static struct lock_class_key cxl_memdev_key; +struct cxl_dev_state *cxl_accel_state_create(struct device *dev) +{ + struct cxl_dev_state *cxlds; + + cxlds = devm_kzalloc(dev, sizeof(*cxlds), GFP_KERNEL); + if (!cxlds) + return ERR_PTR(-ENOMEM); + + cxlds->dev = dev; + cxlds->type = CXL_DEVTYPE_DEVMEM; + + cxlds->dpa_res = DEFINE_RES_MEM_NAMED(0, 0, "dpa"); + cxlds->ram_res = DEFINE_RES_MEM_NAMED(0, 0, "ram"); + cxlds->pmem_res = DEFINE_RES_MEM_NAMED(0, 0, "pmem"); + + return cxlds; +} +EXPORT_SYMBOL_NS_GPL(cxl_accel_state_create, CXL); + static struct cxl_memdev *cxl_memdev_alloc(struct cxl_dev_state *cxlds, const struct file_operations *fops) { @@ -692,6 +712,38 @@ static int cxl_memdev_open(struct inode *inode, struct file *file) return 0; } + +void cxl_accel_set_dvsec(struct cxl_dev_state *cxlds, u16 dvsec) +{ + cxlds->cxl_dvsec = dvsec; +} +EXPORT_SYMBOL_NS_GPL(cxl_accel_set_dvsec, CXL); + +void cxl_accel_set_serial(struct cxl_dev_state *cxlds, u64 serial) +{ + cxlds->serial= serial; +} +EXPORT_SYMBOL_NS_GPL(cxl_accel_set_serial, CXL); + +void cxl_accel_set_resource(struct cxl_dev_state *cxlds, struct resource res, + enum accel_resource type) +{ + switch (type) { + case CXL_ACCEL_RES_DPA: + cxlds->dpa_res = res; + return; + case CXL_ACCEL_RES_RAM: + cxlds->ram_res = res; + return; + case CXL_ACCEL_RES_PMEM: + cxlds->pmem_res = res; + return; + default: + dev_err(cxlds->dev, "unkown resource type (%u)\n", type); + } +} +EXPORT_SYMBOL_NS_GPL(cxl_accel_set_resource, CXL); + static int cxl_memdev_release_file(struct inode *inode, struct file *file) { struct cxl_memdev *cxlmd = diff --git a/drivers/net/ethernet/sfc/Makefile b/drivers/net/ethernet/sfc/Makefile index 8f446b9bd5ee..e80c713c3b0c 100644 --- a/drivers/net/ethernet/sfc/Makefile +++ b/drivers/net/ethernet/sfc/Makefile @@ -7,7 +7,7 @@ sfc-y += efx.o efx_common.o efx_channels.o nic.o \ mcdi_functions.o mcdi_filters.o mcdi_mon.o \ ef100.o ef100_nic.o ef100_netdev.o \ ef100_ethtool.o ef100_rx.o ef100_tx.o \ - efx_devlink.o + efx_devlink.o efx_cxl.o sfc-$(CONFIG_SFC_MTD) += mtd.o sfc-$(CONFIG_SFC_SRIOV) += sriov.o ef10_sriov.o ef100_sriov.o ef100_rep.o \ mae.o tc.o tc_bindings.o tc_counters.o \ diff --git a/drivers/net/ethernet/sfc/efx.c b/drivers/net/ethernet/sfc/efx.c index e9d9de8e648a..cb3f74d30852 100644 --- a/drivers/net/ethernet/sfc/efx.c +++ b/drivers/net/ethernet/sfc/efx.c @@ -33,6 +33,7 @@ #include "selftest.h" #include "sriov.h" #include "efx_devlink.h" +#include "efx_cxl.h" #include "mcdi_port_common.h" #include "mcdi_pcol.h" @@ -899,6 +900,7 @@ static void efx_pci_remove(struct pci_dev *pci_dev) efx_pci_remove_main(efx); efx_fini_io(efx); + pci_dbg(efx->pci_dev, "shutdown successful\n"); efx_fini_devlink_and_unlock(efx); @@ -1109,6 +1111,8 @@ static int efx_pci_probe(struct pci_dev *pci_dev, if (rc) goto fail2; + efx_cxl_init(efx); + rc = efx_pci_probe_post_io(efx); if (rc) { /* On failure, retry once immediately. diff --git a/drivers/net/ethernet/sfc/efx_cxl.c b/drivers/net/ethernet/sfc/efx_cxl.c new file mode 100644 index 000000000000..4554dd7cca76 --- /dev/null +++ b/drivers/net/ethernet/sfc/efx_cxl.c @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0-only +/**************************************************************************** + * Driver for AMD network controllers and boards + * Copyright (C) 2024, Advanced Micro Devices, Inc. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published + * by the Free Software Foundation, incorporated herein by reference. + */ + + +#include +#include +#include + +#include "net_driver.h" +#include "efx_cxl.h" + +#define EFX_CTPIO_BUFFER_SIZE (1024*1024*256) + +void efx_cxl_init(struct efx_nic *efx) +{ + struct pci_dev *pci_dev = efx->pci_dev; + struct efx_cxl *cxl = efx->cxl; + struct resource res; + u16 dvsec; + + dvsec = pci_find_dvsec_capability(pci_dev, PCI_VENDOR_ID_CXL, + CXL_DVSEC_PCIE_DEVICE); + + if (!dvsec) + return; + + pci_info(pci_dev, "CXL CXL_DVSEC_PCIE_DEVICE capability found"); + + cxl->cxlds = cxl_accel_state_create(&pci_dev->dev); + if (IS_ERR(cxl->cxlds)) { + pci_info(pci_dev, "CXL accel device state failed"); + return; + } + + cxl_accel_set_dvsec(cxl->cxlds, dvsec); + cxl_accel_set_serial(cxl->cxlds, pci_dev->dev.id); + + res = DEFINE_RES_MEM(0, EFX_CTPIO_BUFFER_SIZE); + cxl_accel_set_resource(cxl->cxlds, res, CXL_ACCEL_RES_DPA); + + res = DEFINE_RES_MEM_NAMED(0, EFX_CTPIO_BUFFER_SIZE, "ram"); + cxl_accel_set_resource(cxl->cxlds, res, CXL_ACCEL_RES_RAM); +} + + +MODULE_IMPORT_NS(CXL); diff --git a/drivers/net/ethernet/sfc/efx_cxl.h b/drivers/net/ethernet/sfc/efx_cxl.h new file mode 100644 index 000000000000..76c6794c20d8 --- /dev/null +++ b/drivers/net/ethernet/sfc/efx_cxl.h @@ -0,0 +1,29 @@ +// SPDX-License-Identifier: GPL-2.0-only +/**************************************************************************** + * Driver for AMD network controllers and boards + * Copyright (C) 2024, Advanced Micro Devices, Inc. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 as published + * by the Free Software Foundation, incorporated herein by reference. + */ + +#ifndef EFX_CXL_H +#define EFX_CLX_H + +#include + +struct efx_nic; + +struct efx_cxl { + cxl_accel_state *cxlds; + struct cxl_memdev *cxlmd; + struct cxl_root_decoder *cxlrd; + struct cxl_port *endpoint; + struct cxl_endpoint_decoder *cxled; + struct cxl_region *efx_region; + void __iomem *ctpio_cxl; +}; + +void efx_cxl_init(struct efx_nic *efx); +#endif diff --git a/drivers/net/ethernet/sfc/net_driver.h b/drivers/net/ethernet/sfc/net_driver.h index f2dd7feb0e0c..58b7517afea4 100644 --- a/drivers/net/ethernet/sfc/net_driver.h +++ b/drivers/net/ethernet/sfc/net_driver.h @@ -814,6 +814,8 @@ enum efx_xdp_tx_queues_mode { struct efx_mae; +struct efx_cxl; + /** * struct efx_nic - an Efx NIC * @name: Device name (net device name or bus id before net device registered) @@ -962,6 +964,7 @@ struct efx_mae; * @tc: state for TC offload (EF100). * @devlink: reference to devlink structure owned by this device * @dl_port: devlink port associated with the PF + * @cxl: details of related cxl objects * @mem_bar: The BAR that is mapped into membase. * @reg_base: Offset from the start of the bar to the function control window. * @monitor_work: Hardware monitor workitem @@ -1148,6 +1151,7 @@ struct efx_nic { struct devlink *devlink; struct devlink_port *dl_port; + struct efx_cxl *cxl; unsigned int mem_bar; u32 reg_base; diff --git a/include/linux/cxl_accel_mem.h b/include/linux/cxl_accel_mem.h new file mode 100644 index 000000000000..daf46d41f59c --- /dev/null +++ b/include/linux/cxl_accel_mem.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2024 Advanced Micro Devices, Inc. */ + +#include + +#ifndef __CXL_ACCEL_MEM_H +#define __CXL_ACCEL_MEM_H + +enum accel_resource{ + CXL_ACCEL_RES_DPA, + CXL_ACCEL_RES_RAM, + CXL_ACCEL_RES_PMEM, +}; + +typedef struct cxl_dev_state cxl_accel_state; +cxl_accel_state *cxl_accel_state_create(struct device *dev); + +void cxl_accel_set_dvsec(cxl_accel_state *cxlds, u16 dvsec); +void cxl_accel_set_serial(cxl_accel_state *cxlds, u64 serial); +void cxl_accel_set_resource(struct cxl_dev_state *cxlds, struct resource res, + enum accel_resource); +#endif diff --git a/include/linux/cxl_accel_pci.h b/include/linux/cxl_accel_pci.h new file mode 100644 index 000000000000..c337ae8797e6 --- /dev/null +++ b/include/linux/cxl_accel_pci.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2024 Advanced Micro Devices, Inc. */ + +#ifndef __CXL_ACCEL_PCI_H +#define __CXL_ACCEL_PCI_H + +/* CXL 2.0 8.1.3: PCIe DVSEC for CXL Device */ +#define CXL_DVSEC_PCIE_DEVICE 0 +#define CXL_DVSEC_CAP_OFFSET 0xA +#define CXL_DVSEC_MEM_CAPABLE BIT(2) +#define CXL_DVSEC_HDM_COUNT_MASK GENMASK(5, 4) +#define CXL_DVSEC_CTRL_OFFSET 0xC +#define CXL_DVSEC_MEM_ENABLE BIT(2) +#define CXL_DVSEC_RANGE_SIZE_HIGH(i) (0x18 + (i * 0x10)) +#define CXL_DVSEC_RANGE_SIZE_LOW(i) (0x1C + (i * 0x10)) +#define CXL_DVSEC_MEM_INFO_VALID BIT(0) +#define CXL_DVSEC_MEM_ACTIVE BIT(1) +#define CXL_DVSEC_MEM_SIZE_LOW_MASK GENMASK(31, 28) +#define CXL_DVSEC_RANGE_BASE_HIGH(i) (0x20 + (i * 0x10)) +#define CXL_DVSEC_RANGE_BASE_LOW(i) (0x24 + (i * 0x10)) +#define CXL_DVSEC_MEM_BASE_LOW_MASK GENMASK(31, 28) + +#endif From patchwork Mon Jul 15 17:28:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733697 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2057.outbound.protection.outlook.com [40.107.223.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A5403482FF; Mon, 15 Jul 2024 17:28:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.57 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064532; cv=fail; b=Rzi816VMm2rn9NqoTpg02+vP+OdpIA7ET7awVmyvvNEuUH8sgP26tWLZIiRKCFZQDPEw9t9GymM6hFp8Pm6fVCNM9AZUKjMqmVJHq+eLJB8tMzKql/T9C0pYSXb63oKHlYO+yIEHizLC66AOWyDWL04Yh+eQzCVO8QxjVMqex1w= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064532; c=relaxed/simple; bh=JiZD/sVW0oMAP22Ldy3xtseUJoplCp0JQCJVzAEJrpQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=J+CG4QyIkucoeloeB0tS3afEbffofql5bsiQzYrmCZyAhbmWf942nrFbOMMl/Mhe8AJOBkonkRtiTil4KSDmiZeS6SRLK6Pn41R1Q7kqrFBuQuLo9zYGk3RCOHYUdocjjovwNPLbR/33vekFYkjrsksK0tQ2hQMtUb4e64KHttc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=C9badfe4; arc=fail smtp.client-ip=40.107.223.57 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="C9badfe4" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=D1s9u9NXJS+5Ju7309QqLCO6AprZlPdUPhCljezSV0drUbWoGfuEgwY2BQp4hKC5Btq+PTuTUXvLlRurjtFb9gVb+nJQfWZN+dweuQUpf96rGs4ssprWHovgN3dndIK0rTn4N0qwle+2vsiUgR2hEO1uJYQ3Sup8b4iCZgLUnkZbG1tFfktRmG2HiVUPfbn91jH4DX6OYOIbRNuo7fgSfJrqeDAggXLKmwzkxdG3fDmb76s5YNl2TazMBSO4zaHeAITCloAfY6FFucvLqRJo9plucQuP82OOpLUFq1TPDCtj4cRUs+GDONb3sPcJ1kpu8MT5KqW7n/s+wxUkpiAT1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KX2PANqmejnodiJ8QBVFxKSGeEKF7jqaqv/zITOH6i4=; b=NbQon9xZQki8yN1VehObGxbO//r7NPTpkcpPOXiccO9PRk64HQw/CckeBSgXGPKN7ih7Nx+8pKxcskPDi2mAPCSXYP1ctQf0wiv+vDkR5YTzlmpVY33gwyQlKeMluZXjZuhSk6b9wZWhrCgtwF1KuguQ9N3HoNpq/zP10ig5XlQDwJeSapDQNV0iOwmF5K/HWHmByWvWxj58wWu7yta9024QKR/aC29+oMtwgMXd0tjw6HHaFK2Z2vihnKxTFFts8UNSwpjPZe+t7Ai+6M8Jg314NfxwMJYJPAz8IpLX3XBnLxYGeqAmpIKzXrWIrekipfavJMQOWK+Yrsn8gTWcEA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KX2PANqmejnodiJ8QBVFxKSGeEKF7jqaqv/zITOH6i4=; b=C9badfe4fpoCD7En26d2icJhCjPwZopuP43Zf/W+LQ7t5any65XMbI7gxcS5/vU8lYB2JzLudp7TqlmpfCbGocIFLPXu4u9ueBkbwrhAePBLW6KTGrAVOIY6D6ele5G6LoxwyBFZu/Gff0VOtNHv7XbH8B8pq88xiShJp486A+8= Received: from DM5PR07CA0085.namprd07.prod.outlook.com (2603:10b6:4:ae::14) by SA0PR12MB4349.namprd12.prod.outlook.com (2603:10b6:806:98::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28; Mon, 15 Jul 2024 17:28:48 +0000 Received: from CY4PEPF0000EDD4.namprd03.prod.outlook.com (2603:10b6:4:ae:cafe::ac) by DM5PR07CA0085.outlook.office365.com (2603:10b6:4:ae::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.29 via Frontend Transport; Mon, 15 Jul 2024 17:28:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EDD4.mail.protection.outlook.com (10.167.241.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.11 via Frontend Transport; Mon, 15 Jul 2024 17:28:46 +0000 Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:28:46 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:28:45 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:28:44 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero Subject: [PATCH v2 02/15] cxl: add function for type2 cxl regs setup Date: Mon, 15 Jul 2024 18:28:22 +0100 Message-ID: <20240715172835.24757-3-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received-SPF: None (SATLEXMB05.amd.com: alejandro.lucero-palau@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD4:EE_|SA0PR12MB4349:EE_ X-MS-Office365-Filtering-Correlation-Id: 13e8ae9f-2cb1-43b2-d1f6-08dca4f395bb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|82310400026|376014|921020; X-Microsoft-Antispam-Message-Info: 83hoWr5eG3agCsHUGtRgGFXzC2v58wvk8H/DTNdECfDWkdrn2Ic4bSCyWO5BaCF3jKGSE2Ui7ouFhCdyAnk2zgAZW6UA1uMsl46mVM9Q12xgwKMvEHtx6SpODAsoctQ4pHUAv+Q1iLFZiR1RBD46ZmkIBHBF2gObjUmm2Kj55G76oTw238x6VNaoNF8t5SRaDxmEJ3AOojD6whnaEIYYocg4P8V8krm8C4AYuUkgyIDAY1JmzIOTZmHvF8Y3AiqGH6OoyQHXTnKFOPKl5BYx9Q2FZfiLct7KL6Qy0bkm+juhBZBQmqAesfk88L2w1xT98PA82TleiviQiD2gs4i49pByCIoYHBisvRgxAcxZ/I0m3Pur9e1p7Mu842vb3xXsC8wnK46fphhpyxyBKUNhx+2T8TAE9YOZvxxBgoZ5hAc9/424xYRHZ6rAr4CBYkxV+f+4Etv+Es3so7036r5sfcXuWeFU/NB/rGwU+g0lG2aspkbjS53hyxNh7vIFCH4yJllSJZsySkZME6GYxnhpyvQVqc8t9rEjFKHhm9VEP/TpK6eWN/NA75uKTA61MJaVlX8TFReiInrAY57ih2SqQ/YHLtIEf/8b1UkVXati0Go4MzuXGoR7407sWj53bYVBh1fba8IKr0dvSxDB/C2tDo97mggaOjhZ96764FTvDrlfG5IzPylETFK60BPNBJSswvW5xQH6H6xVPTNC6My3+tAwXPMj+1lYwKrQih39+MkPu26ftWYlDA7t85GYcX+K1vYq8ZHDzamR6F1txQc6yIIcjiih4AXb7L0unX3kT8d1J7ZKcthNhW1ARwXJ7b0EtXQaIF4lM7mf8sRiqNJBsmJrZU3g6HeYomN1IdCAJElM0Q1cFNP+k63UP9JrT+5RvOWDYCX5PeWiIJbsc9dWmTmBY07aA+Hvp0ONyjO5cinqbjkiiPBFLP5cYgxCnMkHGS0tQQ1qpZotQZvzURa8Kg1yj3oQqGxgKYyfZcx791nMLQDJhaYeFlmqy0Oo86V34qTlw/KJvKJXAiQ8VjOxky9Tz7nobTGhEegdeWLuJZuMp1HUq3JIAMfbGipyogJ7HrmxLb9Wj6roof7Nai/2htqV9qI1euQw44OmcmV/sWG/nOD0P58WWFSQiVpwWzlE4TJiPYFCchtUph+YsZ5v9Yky9IH5LuFTuYvKQOiEiSDFdl0ixmYgQ0+AV/+qoYcM/GuWbf6JOjkSjAbp/M0UW7wZqBTYZXxmLpaPAUW+aKWL/n9Zb3jC375PD31mVpTipUzaEKKrjBylhjGFzIhtal+E0qto12qlBOz5G5nd2Pzktr/ZLu/ObQ1Chvog6UkrPiz46kyQS553qmCMgXIEanl38eMjXZQgd+ydYPzoaG4Sv/L+UK0Jfbd9kW98vrVOQURpXIsMtH3WRdadSwhlEioM8seFBYB+5lsUUBXHDC1vg52+UhmOLKCbbQvzg6pxxQN4e5RQLmn34L/cRA2zhA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(82310400026)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:28:46.9483 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 13e8ae9f-2cb1-43b2-d1f6-08dca4f395bb X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD4.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4349 From: Alejandro Lucero Create a new function for a type2 device initialising the opaque cxl_dev_state struct regarding cxl regs setup and mapping. Signed-off-by: Alejandro Lucero --- drivers/cxl/pci.c | 28 ++++++++++++++++++++++++++++ drivers/net/ethernet/sfc/efx_cxl.c | 3 +++ include/linux/cxl_accel_mem.h | 1 + 3 files changed, 32 insertions(+) diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index e53646e9f2fb..b34d6259faf4 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -11,6 +11,7 @@ #include #include #include +#include #include "cxlmem.h" #include "cxlpci.h" #include "cxl.h" @@ -521,6 +522,33 @@ static int cxl_pci_setup_regs(struct pci_dev *pdev, enum cxl_regloc_type type, return cxl_setup_regs(map); } +int cxl_pci_accel_setup_regs(struct pci_dev *pdev, struct cxl_dev_state *cxlds) +{ + struct cxl_register_map map; + int rc; + + rc = cxl_pci_setup_regs(pdev, CXL_REGLOC_RBI_MEMDEV, &map); + if (rc) + return rc; + + rc = cxl_map_device_regs(&map, &cxlds->regs.device_regs); + if (rc) + return rc; + + rc = cxl_pci_setup_regs(pdev, CXL_REGLOC_RBI_COMPONENT, + &cxlds->reg_map); + if (rc) + dev_warn(&pdev->dev, "No component registers (%d)\n", rc); + + rc = cxl_map_component_regs(&cxlds->reg_map, &cxlds->regs.component, + BIT(CXL_CM_CAP_CAP_ID_RAS)); + if (rc) + dev_dbg(&pdev->dev, "Failed to map RAS capability.\n"); + + return rc; +} +EXPORT_SYMBOL_NS_GPL(cxl_pci_accel_setup_regs, CXL); + static int cxl_pci_ras_unmask(struct pci_dev *pdev) { struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); diff --git a/drivers/net/ethernet/sfc/efx_cxl.c b/drivers/net/ethernet/sfc/efx_cxl.c index 4554dd7cca76..10c4fb915278 100644 --- a/drivers/net/ethernet/sfc/efx_cxl.c +++ b/drivers/net/ethernet/sfc/efx_cxl.c @@ -47,6 +47,9 @@ void efx_cxl_init(struct efx_nic *efx) res = DEFINE_RES_MEM_NAMED(0, EFX_CTPIO_BUFFER_SIZE, "ram"); cxl_accel_set_resource(cxl->cxlds, res, CXL_ACCEL_RES_RAM); + + if (cxl_pci_accel_setup_regs(pci_dev, cxl->cxlds)) + pci_info(pci_dev, "CXL accel setup regs failed"); } diff --git a/include/linux/cxl_accel_mem.h b/include/linux/cxl_accel_mem.h index daf46d41f59c..ca7af4a9cefc 100644 --- a/include/linux/cxl_accel_mem.h +++ b/include/linux/cxl_accel_mem.h @@ -19,4 +19,5 @@ void cxl_accel_set_dvsec(cxl_accel_state *cxlds, u16 dvsec); void cxl_accel_set_serial(cxl_accel_state *cxlds, u64 serial); void cxl_accel_set_resource(struct cxl_dev_state *cxlds, struct resource res, enum accel_resource); +int cxl_pci_accel_setup_regs(struct pci_dev *pdev, struct cxl_dev_state *cxlds); #endif From patchwork Mon Jul 15 17:28:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733698 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2050.outbound.protection.outlook.com [40.107.237.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33BA54C62A; Mon, 15 Jul 2024 17:28:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.50 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064535; cv=fail; b=BvEcXP8gh2+eSDZBAVnWpot3m9uouDyHHs/K3KWejbo6uwkwEPT/o7YVAxhFXFMbI4E8SSPyVsTSccpclF3e3fc+Q7BvINWpJ1MYTcmmZDD+NguiZ9CfhB/2ROXBOICcwrQWlGU6L34+4HVV2iiAQ1V5wlopGTocoAY3+A6OQP0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064535; c=relaxed/simple; bh=F8GS8p34b0XV0Mca70ix3TkEkexLAGl85pp7ZlgDGf4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=sc6PTPv6hf/pu2twIH4cCILwonaRQ1FW5zBKHwR3sSZjXNLfnWTwt4eQML1y7FljUC2xHZECcDxJMK8Qkd8IX0zpzfpafyQGHvp9Z8QYy73uWdAX7zSF6SWPrdp7T1hVs4JhnQ37imGvfdntBkhrksg0UZOthN/tAqh3aNt9KzU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=b8949fBq; arc=fail smtp.client-ip=40.107.237.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="b8949fBq" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ZokcBjthEsfakMuIdGH4kBMPTd5+5PV/yky0d5l0iTenoxjUYZq5WSyv2G/BTZkE2fmeJORpkzIZq0iqF/KFqSh5BoD+QQRkzi5DHR/DME+IJS9Uk+3DdynwVgQRKgZ2mt7Mn0kgEJM6/Ces8AaGgkbrWIWPBsAIxbjb721LA8P+DSnOBTnDK+1cF8ewaUoVbZbKBdF6frIhH9Hy8qcruM0WgFhfmvecuBqYQIDBP+dYkX+3wPMKBTK7ze5lvcGnSszVHNksJSI/NOTqtBm1dbhhrvMVRikzkG00R+jrgyb77WSfyx5SP7GDwKZWJq+Badr3Y/9V8JfEiE7LB5MHOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OPUzow2Cq0ECdT/dOJDe38VC/lapsSkFdWjr+H1KO3Y=; b=kv3pMzOTOxE3bb3QbsXU1ZpqCGMC6DiWQ0diheOXhLCvi5tLBBUB+WxozL6NtTPWkS1TJEzVL5YIQql5S4Eenu498tGYZIbnPXcKEdIL+kDCSdu6wHTQW9VAQM3hCys/g7CnUXMYCflWBoA2OJbO/ap6Bo9kMzJBzhG6Zn1my6zy+ZVGOxDWUD6gxkvwViVpxP9AODvD+Rkqcko/78tdJeQ92o/gYcPVltA5JI+koaET6BvJpD8lggMgkeUjFrIGLE/9xxP5zz6QGS9n1lPEMdVrfZXaq4usdMURnel0CzX4aRqw/D9PJ5/T3OB47TGAooNxE2MedpMgrWm067zzKQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OPUzow2Cq0ECdT/dOJDe38VC/lapsSkFdWjr+H1KO3Y=; b=b8949fBqZygVmlp9m1FDgdOa9VbDkIP+2ijvYAkvXljVEyV0SVXfYi4YOWji/fTTTn1ahkoRzt/bs7XGrg2CyR2jsQv2Tf5FxH8zYlSIla9+VRcWGQHye1oQ302CqFCHaQ7zQL5eJFX1q5pqz8ugSVtF058d55F7pO+0r75a4js= Received: from MW4PR04CA0155.namprd04.prod.outlook.com (2603:10b6:303:85::10) by IA1PR12MB6649.namprd12.prod.outlook.com (2603:10b6:208:3a2::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.25; Mon, 15 Jul 2024 17:28:49 +0000 Received: from SJ1PEPF000023D8.namprd21.prod.outlook.com (2603:10b6:303:85:cafe::d5) by MW4PR04CA0155.outlook.office365.com (2603:10b6:303:85::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.29 via Frontend Transport; Mon, 15 Jul 2024 17:28:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by SJ1PEPF000023D8.mail.protection.outlook.com (10.167.244.73) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.5 via Frontend Transport; Mon, 15 Jul 2024 17:28:48 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:28:47 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:28:46 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero Subject: [PATCH v2 03/15] cxl: add function for type2 resource request Date: Mon, 15 Jul 2024 18:28:23 +0100 Message-ID: <20240715172835.24757-4-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received-SPF: None (SATLEXMB03.amd.com: alejandro.lucero-palau@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF000023D8:EE_|IA1PR12MB6649:EE_ X-MS-Office365-Filtering-Correlation-Id: c4c7fbf3-99a1-4a1b-72cb-08dca4f39633 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700013|376014|921020; X-Microsoft-Antispam-Message-Info: Z7BkGHQSmfSWlp2uUygq1exiXG0qpIrCEFKKUcjCwOEgZdQKdo77hYwam7SHv0FuIiTqEj01w1DAYAQUNHx8wkOb1paTktc0ZgVXbr0kgkoKRzf+9HkzB55ERamUYo105KDNfAift8vfZE8+nYs0zHW1OkyNW5hhdbHKEa8yJN2qRlpqTYZNVmGBpL8mRDu6g4ezqtDZs4KurOCck7s407N2CCVXgeTQrMu0pqp6w1wA2VPfqC2jrw+Z2MMSvfYxzZkUQhcEcvUcgMDMfG94J5vC3dEKtuV7LZHrhD+SYiCLj8DX8WUBrGALQLF09+sE+Mj4ycBSCwBrqUcPWwa5N2clNJ13EVk2W1Cshwuaiaq4I5hf+qXJc/w+gcm2Llypbn4b5txxetMyylIxptbqmRqNwaDlDX67txBJOWooIWTAu/3qGVQz7B+LXBCIjUDCpDmTB7zO4kJ8rD22arMaOLgzP7br2wrRzP6q2al1JeNQUE3QfCv6KbmzM6F7KjwoNA7IIzkWRfrTojyIfnWFPlUhL4L7q4gCqxmHpEJNwlJZKBkyUhDdMMWCL+lqD8gpXiuRgVTZOonJxeTgmncrJ2nc5K7OY216jJBgXeKU7cmSDuBhBfyVRwm6ADVEmKfyG2auvtDs+icIEwsEV8GBkmUJMDuvV0fo7OXHKkFes0QiC8+jPPo44AgR3Cv4uPvrtx9n9KVAxL/seTQEf6oOM3KNpUK4qKkXjnDfQDwg02+gR26x1qzFE74qlicL2AqdKVmEFKdEH8rjFudAhKJtXwi1gTLTzld8ezJzzIbwqs5g9/wHzW+hTPWuYJNNLGINZv+DR+FXMkGhqmJQpjNnJj5kYG3998Xf9UYjGp7wQLMLWdjxKx9jQOaYHGnn7JipLHZPZa3Qm2MwiQmYhxu/zj1pLqn5DSATxgcHdNjshymOlNo27UtoWhaLL/G8TwfdJC+N84KVBMTZirz+PHp+MiprTBonGyEHrEK4PnhsSwwuxaZyyWJNEzqgZmE8WOuo5TUD0s+0h9Q0hxTBOB9QrrypzrbCZgA9qj/qgZEaF0KeQIbFhImv+1U+ypLcUlYUiZKD4VpkgSJ4l85iRoxgyLsLt2YT0LLWmWEC2E2IHgHVw48HBmmBF6lTv6nOseH976y7+tPMWFZjcFiULwTkkNEDyhyKsTn+lcltk1Utcupp2U3IJaKHIaApjaF31lF+gGtswRJQN3Q+3VvqyvoChcixVV4/ef0RBnBh7dCgciXnc9nQ+hcWW4/YUBzWtFoEkxgLBnWl0LsliznG+jTzmi+4Q7xnRdnQ4tzVmhFMQBKw08xwn41SixOX6FGJixM5XXLkcqu/1OEVkfROHcYByXvZVbUypNYE1YAX2Cp75UOo5ORtiw+Vu9p4NlN2CCaWAgHcMLgNzALjJ8u44/MXEjOSC18qUctBIGU6iItrZaE= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700013)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:28:48.2167 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c4c7fbf3-99a1-4a1b-72cb-08dca4f39633 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF000023D8.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6649 From: Alejandro Lucero Create a new function for a type2 device requesting a resource passing the opaque struct to work with. Signed-off-by: Alejandro Lucero --- drivers/cxl/core/memdev.c | 13 +++++++++++++ drivers/net/ethernet/sfc/efx_cxl.c | 7 ++++++- include/linux/cxl_accel_mem.h | 1 + 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 61b5d35b49e7..04c3a0f8bc2e 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -744,6 +744,19 @@ void cxl_accel_set_resource(struct cxl_dev_state *cxlds, struct resource res, } EXPORT_SYMBOL_NS_GPL(cxl_accel_set_resource, CXL); +int cxl_accel_request_resource(struct cxl_dev_state *cxlds, bool is_ram) +{ + int rc; + + if (is_ram) + rc = request_resource(&cxlds->dpa_res, &cxlds->ram_res); + else + rc = request_resource(&cxlds->dpa_res, &cxlds->pmem_res); + + return rc; +} +EXPORT_SYMBOL_NS_GPL(cxl_accel_request_resource, CXL); + static int cxl_memdev_release_file(struct inode *inode, struct file *file) { struct cxl_memdev *cxlmd = diff --git a/drivers/net/ethernet/sfc/efx_cxl.c b/drivers/net/ethernet/sfc/efx_cxl.c index 10c4fb915278..9cefcaf3caca 100644 --- a/drivers/net/ethernet/sfc/efx_cxl.c +++ b/drivers/net/ethernet/sfc/efx_cxl.c @@ -48,8 +48,13 @@ void efx_cxl_init(struct efx_nic *efx) res = DEFINE_RES_MEM_NAMED(0, EFX_CTPIO_BUFFER_SIZE, "ram"); cxl_accel_set_resource(cxl->cxlds, res, CXL_ACCEL_RES_RAM); - if (cxl_pci_accel_setup_regs(pci_dev, cxl->cxlds)) + if (cxl_pci_accel_setup_regs(pci_dev, cxl->cxlds)) { pci_info(pci_dev, "CXL accel setup regs failed"); + return; + } + + if (cxl_accel_request_resource(cxl->cxlds, true)) + pci_info(pci_dev, "CXL accel resource request failed"); } diff --git a/include/linux/cxl_accel_mem.h b/include/linux/cxl_accel_mem.h index ca7af4a9cefc..c7b254edc096 100644 --- a/include/linux/cxl_accel_mem.h +++ b/include/linux/cxl_accel_mem.h @@ -20,4 +20,5 @@ void cxl_accel_set_serial(cxl_accel_state *cxlds, u64 serial); void cxl_accel_set_resource(struct cxl_dev_state *cxlds, struct resource res, enum accel_resource); int cxl_pci_accel_setup_regs(struct pci_dev *pdev, struct cxl_dev_state *cxlds); +int cxl_accel_request_resource(struct cxl_dev_state *cxlds, bool is_ram); #endif From patchwork Mon Jul 15 17:28:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733699 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2066.outbound.protection.outlook.com [40.107.237.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 825E44D599; Mon, 15 Jul 2024 17:28:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.66 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064535; cv=fail; b=CAJe0Ucleo5ail8q/HYl7m4VA91NzhphDw2FNfiD17Cs8pdW6Mv6vEeN74zQFE3uqT/GXp9kCmR5Iv3SHJYMJmnImjrxg/mz6pXYvEOyUJYCfcoad2pBO5EU8GXWP/F3teqXO73QIUN2xjyV+MIP+4jOdKLMTzq9zWSJ7A6n8aw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064535; c=relaxed/simple; bh=KbvS0L2V6WUOenSh3BY8rwT1gN0n3MDbBGja9lJobfE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QMKQ/po7woJwAbLMS9Xw78EIhQ+UaBzJa4i7luJC9le/LdimW6ijr4iapGhfmWKjLPzdeWfZdjhS7KvjOUbR9+EmJ08NQcMHmwmDMCvyAw5go6JcTKb36KoKbMxDK8I17dcTmQNL8qeXq0tET6fAe7GpUF9MhezedUumCrtDRpw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=anyp9Gz6; arc=fail smtp.client-ip=40.107.237.66 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="anyp9Gz6" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=blLSIidqKWOxUODssdzmXP4dpwcyA3oyU1nJ1y1GaftowIKp28sQlFD9xWdc5hZ1/TLnOk3r8OxreoTjEeL9bcCbeitElu2rOC3hhBTF1quUVLHVLiXN0mifHTqYNmiL4u9v7mZEmCvBRoe8V+LjCCZ7kFFlwLfFb3yMiPO/mHzOSXiBCr7Dm8jTvtvHZ0MfPmiXTihV2t6w00lal6/zfyuArHdPqV6sp4sA8cehjp+QdHNUj+Q8WwHKzxj5UWtz5GSpK+oPnEImMDT0Xbq4AWDRhMUxwlIwBYkQaz4iHyKizhZWdZyr2KnIEIFOh2h4XgAXHRdvUjUroAYEiPHgWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VvurDS4JY07gUZudNXkHJINl7mq9EFEWTO8lAzDZTLA=; b=N9/q5RXbm5p0sAwyRBtRF+/zBZ+Q/VsErbYtAdK110xFARKYjj2rf+hxupwdmkkJStyBoni/5JJQx/VghyBJZLLQNm4KHlX0RAcH0YAaWiAmRliuXNrKFBkbnN09stx62zIcDxuq8aM0AyFfq8wjbreIQneV+YraQeJszlsyRRGnQd3rmm/KFQDkVE6keOUNsg+PhBC2nKivWNlP0LL1ICMf4QNqQ+AKBpLDDVF2GfvtZ2/Xg2QJ7pNb/Q3koyPbZ1TYHEEOWrDD8kCfSx0DbEkdIHMYGUHm0YUaHRbv46uvxc0c5LZkTz2zvonLwnfN/d2mLk/Ng/xV5PTbq2rbug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VvurDS4JY07gUZudNXkHJINl7mq9EFEWTO8lAzDZTLA=; b=anyp9Gz67ps92iiqhIa3MPQRs96I/MQWFUQtEiUjGwy67CJGLHP+cJX9fAXN0cRnR5hVYZzMeRFsDZC7/+OlILyKUpCpgEjAdI/IjXkaDGVqSYKS9T7/VVo5gSpj/dVHrropnzBlCFHftLL78qGsaTWs+J9DSHYY4X1NK+8CJxo= Received: from CY8PR12CA0028.namprd12.prod.outlook.com (2603:10b6:930:49::28) by SN7PR12MB7833.namprd12.prod.outlook.com (2603:10b6:806:344::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.31; Mon, 15 Jul 2024 17:28:49 +0000 Received: from CY4PEPF0000EDD7.namprd03.prod.outlook.com (2603:10b6:930:49:cafe::77) by CY8PR12CA0028.outlook.office365.com (2603:10b6:930:49::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28 via Frontend Transport; Mon, 15 Jul 2024 17:28:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EDD7.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.11 via Frontend Transport; Mon, 15 Jul 2024 17:28:49 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:28:48 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:28:47 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero Subject: [PATCH v2 04/15] cxl: add capabilities field to cxl_dev_state Date: Mon, 15 Jul 2024 18:28:24 +0100 Message-ID: <20240715172835.24757-5-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received-SPF: None (SATLEXMB04.amd.com: alejandro.lucero-palau@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD7:EE_|SN7PR12MB7833:EE_ X-MS-Office365-Filtering-Correlation-Id: 9fec45c5-ffbe-49df-6726-08dca4f39707 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|82310400026|36860700013|1800799024|921020; X-Microsoft-Antispam-Message-Info: mfgIHKXP7DJtK04sGYoIvrCQ3tEa2lo+p/fEIcpMughz+Cx68fTCbDNvV85RPlflBH3Tk3Yjoqyr4mcYA1ScAaV4bHbaupORuDv1hNyLQghYwBibz0TvhLK3Q/W5WLuYYrBTcszOYeYxzXqsYxO8Kkg/OGWgJee0DoKJg7pvbngn1hsX53h/VW6BNeUHOWH4y8FUTN4CD75lYssN+DagIzYGTLq2/0doOuLq74PfUYdip3ZAiT16l/RXlf4trB8gkVujOJPJI1/KbjkjhY8l4+3cuhsYAWQa6YE+QFuTHK5sdbnv9ngKMq9pZdYaKGnejUIPx7AUnEGOGyaxVeI2GCKsXVjlML8cg7/wkOJzHiVDKz5nRxXh5xgbzuQ4Xnjfe/LaNHUgz9Wth3zgbMlgaJB2YwXAiXMXnweX9gSz2P9nmnEydTbkFDdSU19REa+HnIDsEHmOnfW4w9bEi1Czup2RIZFnndD5EMFhVvcuZ+rSS7gTDbjOlnlc02fF3rAKnu8zgUsovBc2NZGLCjYgutbr67XYvhpi/slYO6a/ozGqvQ+q13P64pTOb6JxkVgY+pkzfnOITMftj7LuLdvGzcfMXSeDdmCyyHg8f7yeczMlwQ7ONdYOEbU83+bAViSISPZ1Is5H6VoK9ipE2jl0LAZac++Q2R+GYSXNMmSGiFVYZymaXRz5NW/f9gHXv7WNt4YZ+Mh4m0pLRvRZLR5kGGvQiicRorDw5KBMbEIzeTBZOf6W3KwKvziASeL/SqkP3USEorPZ6v3RTmxb1rapdjWZY+Mz2v7L7t5mvblZYRNI8xCFsH+EAYzUPnhuI1HzCKFmLcjZ5UOG3ShRq1tqzHCbOgWVi9b5bEiLcxevZ4J3cj2CdoocHZ8vhjUXXzdHIh2YcAVohTVxJ+/r4VSwBSiG8DXAqXjm332MD03Z8b2NOtxUULn5x1Nq+76bDFx6tUIZpfYdBAFRbxfdgGxNErNgybFOkhmWTRpXykaQ3z+Ky1zGKbGGb9AD8AEUbzS/88f5skkwNHdtyoFxxHKhVHW07u6QK2ex7yDWJ0Y2jheeLs09F7ophvkBpU6P2c1JTcNWtHomqhQsnrRPYrFnnz21gd5EWg5YRK60b+2K4pw7iMVkDWxyycxaIBBCYulv0vLUPctbnXp64upE7R6VkavzSoAh6O0TIAW3sbw+OzQZFqzhESiGb+nBY/Pws38m7tYW9s9GC/8YkRb4U1u9ij0nrij2tIfjU9c3p1b+SNJbd4vQguoqDOCPTL19Hm+bYDQjM9pZZtwu4OSoHKX8h8GOixSH9M6h2M6gA0Mtrxzg4Vj/EbzW1p/rPlmBPKAM8fUbz/QFEB6mydsil7jqoIc18jGV7TtJDMIxIguDXV10KnfeAxE6g0GSwhkEgPAuznGb5YxjdaFzvRk7dZFWt6V+bXqoSnjPp1jPIBzDhHdnimEks8wJbCnRH2AAdtaY X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(82310400026)(36860700013)(1800799024)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:28:49.6370 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9fec45c5-ffbe-49df-6726-08dca4f39707 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD7.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7833 From: Alejandro Lucero Type2 devices have some Type3 functionalities as optional like an mbox or an hdm decoder, and CXL core needs a way to know what a CXL accelerator implements. Add a new field for keeping device capabilities to be initialised by Type2 drivers. Advertise all those capabilities for Type3. Signed-off-by: Alejandro Lucero --- drivers/cxl/core/mbox.c | 1 + drivers/cxl/core/memdev.c | 4 +++- drivers/cxl/core/port.c | 2 +- drivers/cxl/core/regs.c | 11 ++++++----- drivers/cxl/cxl.h | 2 +- drivers/cxl/cxlmem.h | 4 ++++ drivers/cxl/pci.c | 15 +++++++++------ drivers/net/ethernet/sfc/efx_cxl.c | 3 ++- include/linux/cxl_accel_mem.h | 5 ++++- 9 files changed, 31 insertions(+), 16 deletions(-) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 2626f3fff201..2ba7d36e3f38 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -1424,6 +1424,7 @@ struct cxl_memdev_state *cxl_memdev_state_create(struct device *dev) mds->cxlds.reg_map.host = dev; mds->cxlds.reg_map.resource = CXL_RESOURCE_NONE; mds->cxlds.type = CXL_DEVTYPE_CLASSMEM; + mds->cxlds.capabilities = CXL_DRIVER_CAP_HDM | CXL_DRIVER_CAP_MBOX; mds->ram_perf.qos_class = CXL_QOS_CLASS_INVALID; mds->pmem_perf.qos_class = CXL_QOS_CLASS_INVALID; diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 04c3a0f8bc2e..b4205ecca365 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -616,7 +616,7 @@ static void detach_memdev(struct work_struct *work) static struct lock_class_key cxl_memdev_key; -struct cxl_dev_state *cxl_accel_state_create(struct device *dev) +struct cxl_dev_state *cxl_accel_state_create(struct device *dev, uint8_t caps) { struct cxl_dev_state *cxlds; @@ -631,6 +631,8 @@ struct cxl_dev_state *cxl_accel_state_create(struct device *dev) cxlds->ram_res = DEFINE_RES_MEM_NAMED(0, 0, "ram"); cxlds->pmem_res = DEFINE_RES_MEM_NAMED(0, 0, "pmem"); + cxlds->capabilities = caps; + return cxlds; } EXPORT_SYMBOL_NS_GPL(cxl_accel_state_create, CXL); diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 887ed6e358fb..d66c6349ed2d 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -763,7 +763,7 @@ static int cxl_setup_comp_regs(struct device *host, struct cxl_register_map *map map->reg_type = CXL_REGLOC_RBI_COMPONENT; map->max_size = CXL_COMPONENT_REG_BLOCK_SIZE; - return cxl_setup_regs(map); + return cxl_setup_regs(map, 0); } static int cxl_port_setup_regs(struct cxl_port *port, diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c index e1082e749c69..9d218ebe180d 100644 --- a/drivers/cxl/core/regs.c +++ b/drivers/cxl/core/regs.c @@ -421,7 +421,7 @@ static void cxl_unmap_regblock(struct cxl_register_map *map) map->base = NULL; } -static int cxl_probe_regs(struct cxl_register_map *map) +static int cxl_probe_regs(struct cxl_register_map *map, uint8_t caps) { struct cxl_component_reg_map *comp_map; struct cxl_device_reg_map *dev_map; @@ -437,11 +437,12 @@ static int cxl_probe_regs(struct cxl_register_map *map) case CXL_REGLOC_RBI_MEMDEV: dev_map = &map->device_map; cxl_probe_device_regs(host, base, dev_map); - if (!dev_map->status.valid || !dev_map->mbox.valid || + if (!dev_map->status.valid || + ((caps & CXL_DRIVER_CAP_MBOX) && !dev_map->mbox.valid) || !dev_map->memdev.valid) { dev_err(host, "registers not found: %s%s%s\n", !dev_map->status.valid ? "status " : "", - !dev_map->mbox.valid ? "mbox " : "", + ((caps & CXL_DRIVER_CAP_MBOX) && !dev_map->mbox.valid) ? "mbox " : "", !dev_map->memdev.valid ? "memdev " : ""); return -ENXIO; } @@ -455,7 +456,7 @@ static int cxl_probe_regs(struct cxl_register_map *map) return 0; } -int cxl_setup_regs(struct cxl_register_map *map) +int cxl_setup_regs(struct cxl_register_map *map, uint8_t caps) { int rc; @@ -463,7 +464,7 @@ int cxl_setup_regs(struct cxl_register_map *map) if (rc) return rc; - rc = cxl_probe_regs(map); + rc = cxl_probe_regs(map, caps); cxl_unmap_regblock(map); return rc; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index a6613a6f8923..9973430d975f 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -300,7 +300,7 @@ int cxl_find_regblock_instance(struct pci_dev *pdev, enum cxl_regloc_type type, struct cxl_register_map *map, int index); int cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type, struct cxl_register_map *map); -int cxl_setup_regs(struct cxl_register_map *map); +int cxl_setup_regs(struct cxl_register_map *map, uint8_t caps); struct cxl_dport; resource_size_t cxl_rcd_component_reg_phys(struct device *dev, struct cxl_dport *dport); diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index af8169ccdbc0..8f2a820bd92d 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -405,6 +405,9 @@ struct cxl_dpa_perf { int qos_class; }; +#define CXL_DRIVER_CAP_HDM 0x1 +#define CXL_DRIVER_CAP_MBOX 0x2 + /** * struct cxl_dev_state - The driver device state * @@ -438,6 +441,7 @@ struct cxl_dev_state { struct resource ram_res; u64 serial; enum cxl_devtype type; + uint8_t capabilities; }; /** diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index b34d6259faf4..e2a978312281 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -502,7 +502,8 @@ static int cxl_rcrb_get_comp_regs(struct pci_dev *pdev, } static int cxl_pci_setup_regs(struct pci_dev *pdev, enum cxl_regloc_type type, - struct cxl_register_map *map) + struct cxl_register_map *map, + uint8_t cxl_dev_caps) { int rc; @@ -519,7 +520,7 @@ static int cxl_pci_setup_regs(struct pci_dev *pdev, enum cxl_regloc_type type, if (rc) return rc; - return cxl_setup_regs(map); + return cxl_setup_regs(map, cxl_dev_caps); } int cxl_pci_accel_setup_regs(struct pci_dev *pdev, struct cxl_dev_state *cxlds) @@ -527,7 +528,8 @@ int cxl_pci_accel_setup_regs(struct pci_dev *pdev, struct cxl_dev_state *cxlds) struct cxl_register_map map; int rc; - rc = cxl_pci_setup_regs(pdev, CXL_REGLOC_RBI_MEMDEV, &map); + rc = cxl_pci_setup_regs(pdev, CXL_REGLOC_RBI_MEMDEV, &map, + cxlds->capabilities); if (rc) return rc; @@ -536,7 +538,7 @@ int cxl_pci_accel_setup_regs(struct pci_dev *pdev, struct cxl_dev_state *cxlds) return rc; rc = cxl_pci_setup_regs(pdev, CXL_REGLOC_RBI_COMPONENT, - &cxlds->reg_map); + &cxlds->reg_map, cxlds->capabilities); if (rc) dev_warn(&pdev->dev, "No component registers (%d)\n", rc); @@ -850,7 +852,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) dev_warn(&pdev->dev, "Device DVSEC not present, skip CXL.mem init\n"); - rc = cxl_pci_setup_regs(pdev, CXL_REGLOC_RBI_MEMDEV, &map); + rc = cxl_pci_setup_regs(pdev, CXL_REGLOC_RBI_MEMDEV, &map, + cxlds->capabilities); if (rc) return rc; @@ -863,7 +866,7 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) * still be useful for management functions so don't return an error. */ rc = cxl_pci_setup_regs(pdev, CXL_REGLOC_RBI_COMPONENT, - &cxlds->reg_map); + &cxlds->reg_map, cxlds->capabilities); if (rc) dev_warn(&pdev->dev, "No component registers (%d)\n", rc); else if (!cxlds->reg_map.component_map.ras.valid) diff --git a/drivers/net/ethernet/sfc/efx_cxl.c b/drivers/net/ethernet/sfc/efx_cxl.c index 9cefcaf3caca..37d8bfdef517 100644 --- a/drivers/net/ethernet/sfc/efx_cxl.c +++ b/drivers/net/ethernet/sfc/efx_cxl.c @@ -33,7 +33,8 @@ void efx_cxl_init(struct efx_nic *efx) pci_info(pci_dev, "CXL CXL_DVSEC_PCIE_DEVICE capability found"); - cxl->cxlds = cxl_accel_state_create(&pci_dev->dev); + cxl->cxlds = cxl_accel_state_create(&pci_dev->dev, + CXL_ACCEL_DRIVER_CAP_HDM); if (IS_ERR(cxl->cxlds)) { pci_info(pci_dev, "CXL accel device state failed"); return; diff --git a/include/linux/cxl_accel_mem.h b/include/linux/cxl_accel_mem.h index c7b254edc096..0ba2195b919b 100644 --- a/include/linux/cxl_accel_mem.h +++ b/include/linux/cxl_accel_mem.h @@ -12,8 +12,11 @@ enum accel_resource{ CXL_ACCEL_RES_PMEM, }; +#define CXL_ACCEL_DRIVER_CAP_HDM 0x1 +#define CXL_ACCEL_DRIVER_CAP_MBOX 0x2 + typedef struct cxl_dev_state cxl_accel_state; -cxl_accel_state *cxl_accel_state_create(struct device *dev); +cxl_accel_state *cxl_accel_state_create(struct device *dev, uint8_t caps); void cxl_accel_set_dvsec(cxl_accel_state *cxlds, u16 dvsec); void cxl_accel_set_serial(cxl_accel_state *cxlds, u64 serial); From patchwork Mon Jul 15 17:28:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733701 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2078.outbound.protection.outlook.com [40.107.93.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5528750285; Mon, 15 Jul 2024 17:28:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.78 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064537; cv=fail; b=ZcbqYCpJddVcAQb0B7KD2iZ+01BFWNnQ5jlS9jMN+Y0NncDmXO0oH8lpgC9RYekC7XgSGZqERcydAQNZy/0u5Fac5iQZNzi01/VC5ybPmlAKTQY/tNvwKolvYTVOsW50lfIVH5xqXvmJMJlEJJaChpF0qB+Soq5OqolmYqyoHfE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064537; c=relaxed/simple; bh=jWl2LN9rAvMP12J+Jb5x05YIg2LLk0ytcompyghuznY=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=klJFXg/Yr0PCF3qFaQsuozSqdGahEnOErWjFU52f3JW1ssrt4wjeMoHJlmszxMU6IcBhMXZ+paELGzAGSIhAiYJXQnZ2OuW63G83Tem0LJgKtGb1z4ktU1kT6SlwK4LfyTDHkgBPr9/IFoPlZi2IwwJ/QxXpC09weQ3Iz8KVWSw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=4FlbBdEx; arc=fail smtp.client-ip=40.107.93.78 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="4FlbBdEx" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=u7M+pomQEIJTyZsALy5kk3BQHfB/fi/A7i0Qf5Lr3sbME0er5JkxotRyuNtjSvw4MiVA/aQgb/Hso9lStUORuh2wso58AjJh3EZok9ojL4HuRAs4ex4EQXSX9XrXJkj4IzzRbDvXi6AFtkzIck2tpnLCPSIGCFQ7VSfm4Smkv5bwMOq0RFNRKGr1+Zr0M/ygE0z1jNvVzHEuKNsR/wBCoyNMDu3Jsok5SPVL5rTe7Ax2aA6QsvvTgRc/QLm76ODo0NQ308ry/w+ek4DLa9l+8tdDuA2otHE8ovV7HDGmJ+Wk3VuPUmjhq83Y+Cbu1vaREXYAzu/jANcjgKGMdIPXCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=L3vC5F3A/E95qWGqMXa9QOWGkI3xa+Vj8uXJAL6p01k=; b=sDDKOepsZeTybT+tU0FA9S16Vh9b3i0X1lnHCQkrhQXptWxZSDBQUg9QeUA3mm8RtGcdAOtPTbbtoQbUFVBtg3PHNEkgiUMT4MZVKaRYVuYAsZiNUFAwHEuOojM5fWSVWZ1LXomjRUGTVRdVC1PRalv1zTO8RiOcc6BIG4D6X/IAqVQ/sk/5M0eHzptkDhwShEVGh0nXc0RdQ0h5M33iKQ5AgDbmdihLa/tkT6RYamIcTsH+T0lMYgwA0vKMKyDlVTRRFWwwJMH93P9/ou3dXwfgAGAMLBaoLFhJ8YhnIppbHYK7znTi3TimcLQu7t/O/8IJNdH9WBIH9G/6fkc7XQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L3vC5F3A/E95qWGqMXa9QOWGkI3xa+Vj8uXJAL6p01k=; b=4FlbBdEx1aiwEarsnwOU2OrCUcQYy+JTss0L7aPxPBm0rRjuGVknFE08DEmTOpxy/8Q0g5HASoUjBXdMqt9/Kxtqg3d6xm+7+IhDuQlTQF012DphX3nl+EiyS1I682/QZ8rLbLZ2vTjfklLCmXheC6KxYEeSfL9CwAOhpo0HP00= Received: from CY8PR12CA0026.namprd12.prod.outlook.com (2603:10b6:930:49::27) by BY1PR12MB8448.namprd12.prod.outlook.com (2603:10b6:a03:534::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.22; Mon, 15 Jul 2024 17:28:53 +0000 Received: from CY4PEPF0000EDD7.namprd03.prod.outlook.com (2603:10b6:930:49:cafe::63) by CY8PR12CA0026.outlook.office365.com (2603:10b6:930:49::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.29 via Frontend Transport; Mon, 15 Jul 2024 17:28:53 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EDD7.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.11 via Frontend Transport; Mon, 15 Jul 2024 17:28:53 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:28:50 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:28:49 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero Subject: [PATCH v2 05/15] cxl: fix use of resource_contains Date: Mon, 15 Jul 2024 18:28:25 +0100 Message-ID: <20240715172835.24757-6-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received-SPF: None (SATLEXMB04.amd.com: alejandro.lucero-palau@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD7:EE_|BY1PR12MB8448:EE_ X-MS-Office365-Filtering-Correlation-Id: dd8b144b-87bd-463c-2494-08dca4f3994d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|82310400026|36860700013|1800799024|921020; X-Microsoft-Antispam-Message-Info: O3b2mkGo20isTVj7WbO7mv2Oew3AnM8D/1JB58qFtWGPb9J3bkHjxQl+Dc4QjH2Bmb7HRzTD3+AdgYzlc2Brp0kv7Bqd9+mO0wbxV7WA7SqPV9D4S/yyy6uPi074w5+dkOGz23IZKKXbUG804qEBTIRLa6IIWclc+c2ctScbqXhoE2t26KGME/4a+RO09whrxaLmtBDJj+TiTMq2hLo/1r5IoLq9eOZ9o7jUeq+b1H8Lo2BNzaTBsNuSxxpi8fUrK6I2/PQxVzD+n+OFDCcyYgCincGnC9pxLHDhoYF0buuqTYylBamiOyR+c7uf1m1E1eJErGd2BNlstk0vC0mmkB2g+5Gj49vx4UqXm3Mp8eyKjJGutz4qzRlGsFH5lA8YKdaiqAn5tAaVtREPQNl0bFjUJzDXHBvJ4ftl3rU0+DjbVtckyQErnPueXYRiWV9xZxi35LX+wo3rpbV62r7a7AH3WK4uONYtitXuVvTX+KPIl9Ljo1hemymPl4lCvU3eMJ/a/VK+WKr3BSCLleNczGfo/Yxz4Ptm6GQrqyGgXDSIhpidXPCvWRALoaS8Y52RcizPgktydFdvBYBnd5xrxzdmmW6b8bOvblUJI5ODrmrB76EE0pcYK2STVBVKWJlV5sZoLttdwQwBzF4asW1MbamUiOnQHVwPJYkFUJP1+vx5dvXHi8ikQxZ6vbk+AHfBXa3ncxilwdFlmpziEu215wgVMcb6QaFXij4HopQupgMhB+kIOiiKqYNHPAtIGUAQNN/ZjKAZUP5T9ZOg2jncTRrcQBLjUnH2PvSq9e4O8rQMKqY2EFtInQi4oKv6JZm3NOHYsf+7G+9fqbgZAtsVLxZyJFdOQke6CUvCMtq5Ar9SjETVmyP84urDb8os8+BCPs5CQ6uubxMHLRr5aBASEfhPOtsgy2t1lYvcy8RMNEPdOjWDc6HZQqKfnQpwdeMgYjQsvAP6gbf7he6YbS4NnqWrqKABe4sXuajwVAXkjhyw1Ygiiq8+7y5laLbX90gl4hsi1UJRLkmx/QGfBQI9dD+sazHVvsZvJHGyHJci4dcltgfrWyUrXvl6B8nen1mkfEudJXyErsjgOvc9pBN3GqDN+Zwv0mN9fedDK53FVO8c6RpUr7W/3SyHJxE/N+HrUc40kMVwEyUkU5fXsL47s9bfwOlQQLi32pxEAVZ+aVmfFIrV+RJpk4xlWntTws/JLi7rSo30v0hG5Ag7SfWGRU+BCNjmPFFjOxLQ9LyMm/x/dtUNot4vq8//l9EaA23LCO6o1kd8/4vEeQ7VEpt1QQIzI6sC9zoNPvMnINgj0cnXFjpJ8gakGWtkJCDmzmCw0iM0/1UjGGUG+YO9wE8vtp+oYaQC2ZMvLlo4Ek9EFlZQP09fz77kCFGOVPMlIxyTZ41BEb/QfjJTXc5L4zbjfpl/LdYOiUA15UEQScCk5a+bQi2jk0xHxvO7zzNHKHtG0LxT1dVeuT2TKgFabjIPlw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(82310400026)(36860700013)(1800799024)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:28:53.4339 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dd8b144b-87bd-463c-2494-08dca4f3994d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD7.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR12MB8448 From: Alejandro Lucero For a resource defined with size zero, resource contains will also return true. Add resource size check before using it. Signed-off-by: Alejandro Lucero --- drivers/cxl/core/hdm.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 3df10517a327..4af9225d4b59 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -327,10 +327,13 @@ static int __cxl_dpa_reserve(struct cxl_endpoint_decoder *cxled, cxled->dpa_res = res; cxled->skip = skipped; - if (resource_contains(&cxlds->pmem_res, res)) + if ((resource_size(&cxlds->pmem_res)) && (resource_contains(&cxlds->pmem_res, res))) { + printk("%s: resource_contains CXL_DECODER_PMEM\n", __func__); cxled->mode = CXL_DECODER_PMEM; - else if (resource_contains(&cxlds->ram_res, res)) + } else if ((resource_size(&cxlds->ram_res)) && (resource_contains(&cxlds->ram_res, res))) { + printk("%s: resource_contains CXL_DECODER_RAM\n", __func__); cxled->mode = CXL_DECODER_RAM; + } else { dev_warn(dev, "decoder%d.%d: %pr mixed mode not supported\n", port->id, cxled->cxld.id, cxled->dpa_res); From patchwork Mon Jul 15 17:28:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733702 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2043.outbound.protection.outlook.com [40.107.96.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6AF1B535B7; Mon, 15 Jul 2024 17:28:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.96.43 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064539; cv=fail; b=DDoXpqQo0YftW/CkmBH6WhinqLlxXXusOsMed7xfeBvT9jSZKb+835XVUY7LfWUCZ6Xiw5CKG6nPXRrs8KkXIu7dzcgM6o3M09YiSTmWNN+2eAXq2Q/Rv2oXSutBygz/tykm9kC9GpjQUKhaVRCa2wEc5Qlp1B+tP6Ble8Pgdzg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064539; c=relaxed/simple; bh=3TifW9WFW7DjDvpsXYzB3X5fPqgcKoS+0+St87XYMNM=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=JTu2Hiwc+nQ79TTCr7c/GUw4QkVsfAZUI2NtWd3GApgdts67W1GmJ5D3sxpkESus8ek1tuHMVHmx1ov/67Ge/P6B+zsP+waur7oQCSTt5iL/Xyzf1bu2JousblYgStR4T8J6mW55o9GZHeXmTKELXOLnVlteagCFNQ+ZgNIEKMY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=xgiQj5v2; arc=fail smtp.client-ip=40.107.96.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="xgiQj5v2" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=rJ/d585n89kFudGFDgcI0iaeVcGzga1KZGGIn0yTA9rjzpPtkzLApfru0iWvKvyFYecrnB0xoyY3WKPvLB3dXvTOE77PXQ2mdez5zTyyxhclNekRilNuwdXhtYDr6qySjms26IhtEntnj/UM8PY1TlytZOG3WwAMfyLTvLJ2nksvidiUojR28GFXtbJSKoXb+tOkbyILZZ46GVmMpf9mRC+uVm84Phfa6kc9BDOQeciuwh/PJECZdU4gDkeggjUGFLsTcU0ucXtivcuOmmm6rL6ZcaX1Om1CWU+4Nm3Hmv55Y1dSYz543qn4Hnxe92VYJ0QW13LpfFkvRkh7+ZBa5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=k2QcRmqYhrwOkpEDbHI3M4kvRFcZfeVsePYkcK6LbgM=; b=FreNl4bL74rLusYJTk3OJKCHqds+Oyc7CHXBqGujmAFQhuEjfOqbct8nY6Wt8Kk3wLQDDEFdTmM3jf3S6zVhuauDX620rrT86RKo9UwV+++mXDNGj7LLyPGkx/0SLTce9FXMeFPM/44Cvizd6kdA1YrsLSKGpASVGbGa7tolxlJoxS3t/CghZyoSxLVYnEcCWNcbNg0a02b9p+oOObCc1dWL5uKJGbf0BUMqdgc15efhKyPRQSxvPRP8UrP/O7QpEdAEkLhh4MnSM3xUoXBElaLlTCdIf0bACLCZlm9F3ejv+xQG3CEaDNeQpDHUgt2fGG7y/OZErEbuckdqKSURKg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=k2QcRmqYhrwOkpEDbHI3M4kvRFcZfeVsePYkcK6LbgM=; b=xgiQj5v20ygqoIIfWFWyISZtsKp7VGMzj95AKO/D02Y6ffWnYafcP3PIYXWBOQ3QmXE+UDmgg/s/hv5aLQ2aSLFfZEPY9Emur3TmFtJ07QScX4RNEqbiN5dySUjXHR29sPUmqJwCEgaVd+9HXnnZ403RsoLZdy9Pl8L5Vbm+roE= Received: from CY8PR12CA0032.namprd12.prod.outlook.com (2603:10b6:930:49::23) by CY8PR12MB8194.namprd12.prod.outlook.com (2603:10b6:930:76::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28; Mon, 15 Jul 2024 17:28:54 +0000 Received: from CY4PEPF0000EDD7.namprd03.prod.outlook.com (2603:10b6:930:49:cafe::57) by CY8PR12CA0032.outlook.office365.com (2603:10b6:930:49::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28 via Frontend Transport; Mon, 15 Jul 2024 17:28:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EDD7.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.11 via Frontend Transport; Mon, 15 Jul 2024 17:28:54 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:28:51 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:28:50 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero Subject: [PATCH v2 06/15] cxl: add function for setting media ready by an accelerator Date: Mon, 15 Jul 2024 18:28:26 +0100 Message-ID: <20240715172835.24757-7-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received-SPF: None (SATLEXMB04.amd.com: alejandro.lucero-palau@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD7:EE_|CY8PR12MB8194:EE_ X-MS-Office365-Filtering-Correlation-Id: 12b2a20f-734c-4a10-bdbd-08dca4f399a7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|36860700013|82310400026|921020; X-Microsoft-Antispam-Message-Info: Eb2alPltNkuP0t8NC9hc6nT0Q01LrUSZbqlxn9PhrUupClqNI701jzl+kDcRNSxHwBHLvHSwnHrLJdVKQ+i84rYf3fbMez+bSzlMrtsdoV7WhZMRs7zAr4d8Dh3243Sc5AAC6YGRxRNpW0UH5t4Bf9JYSzxySZdZ+/UPBh8bqRR7LdkXCnl5kjYDT8LSdBOKcQt8MN/XuU1JeHFDp76/eH43oHtEMFtxRQ5LWluyMdDXwi5mdwp9qNVqanhIh4sVgWiM2Nl5jeVTOzyavlCAUZH08+DCra+LtdxPJqwf4BZpobgtL7ADe4OSIKzPucfqh6pcB9DQfspi8jxNoAmeF2/6ANyGLeGA0QrGQHmjqL2GxC8BClB5Rec5PM5BKtffrbYN38k31aWo4gbtUrcBpzr+WH8CkifPejVKWjas2QY5I0lzunrJRe7OZjV/42ZbZm02Lf0NjRPZWaCggJcYsi1SWn9l5qeaN9zdWmZOjNWnNqedcFINJYwkRqKLNr17b3YVXmN07E9UxssafJWxrjKNBIdIkvDdkMT8TSJ/K94j3OC62Y/rLB4BLk/DC2LGZ+In0Coxpbkt63OxlM7Xf7DkduMXm7paovaLzAd4riBb4Dq4Fc3h7MUWT42Vh5pEqNgFwdwlwNDcsk46W3MYsgZ7OL5zsKg8jFddkExl/EonZjVQfMDAjWFycftmAiyZv/XB3n8xV9s3JBxVnt8fXWF6agD/FYIp8GI4oMY5liuRppsaP7teUHL4q6goefWdEAdq8y7BR+bT9K0qlmAH+slF46wKuNjiJ0aE1JGFh9i/Z8U+o2UQi/N8zyy7H9Fs+nR/raDa+Rr6RpcBqku5TsNSrSpc2WsXLvpLUMcDilv5twxmsJQueIQ8MG/ydhftrJLzjyq2JZm4UyFT0X1dcpgWZ+O/ZX7TF8M+C3b0t3XJ+cDlSLJyNMlIOH5rbScvVnbKI++flcH4Bedw7Ch+SEei/xFEJ/p6Rnv2Sl3tFjT5lIMaLP8ShhStdcBqj3u1pSukQzO5TBtOFzYGRC7+L7+hZ5FkYQCMGxHG0t4qpj93y1vWXIlamfGMo2q5iJoOTB1nVyGWc4iTnuhS89wYXk3YtykYmLnQrS4Gm6qky+8Ep0rXjU+SjDkhi1Zai6G5nharFECBdVvosj0X0/4/EIDiH3zPA2ButeaZ4tuARBygf0YpiEfupQBYDNrUnQ5hKbgs1q3LfsF9c8cetvUkwNk96HVTdsA4lev9GrFQdph61K58v66VVJfKPsvAJLBL1Hv3wV0UtaECuOSYCwSAe7otfl4eDBsaWNSrkXsu8A+HT3Dv+NwK9DhiQOmwpS7MqBL64bi+mFd5bCg1AHzJz0gmhOnLprvLMYOBINbpRiGpm69QMDCcJzOYMn0PsOfgR4wnETsiUMx4VQK0IO060fq6B5NyGNtK0ZQePeMtaO5UDKtPjMYiu2UgnjZ33vcLdy+UKF/1q+ZiUrDKgBMWFQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:28:54.0276 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 12b2a20f-734c-4a10-bdbd-08dca4f399a7 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD7.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB8194 From: Alejandro Lucero A Type-2 driver can require to set the memory availability explicitly. Add a function to the exported CXL API for accelerator drivers. Signed-off-by: Alejandro Lucero --- drivers/cxl/core/memdev.c | 7 ++++++- drivers/net/ethernet/sfc/efx_cxl.c | 5 +++++ include/linux/cxl_accel_mem.h | 2 ++ 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index b4205ecca365..58a51e7fd37f 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -714,7 +714,6 @@ static int cxl_memdev_open(struct inode *inode, struct file *file) return 0; } - void cxl_accel_set_dvsec(struct cxl_dev_state *cxlds, u16 dvsec) { cxlds->cxl_dvsec = dvsec; @@ -759,6 +758,12 @@ int cxl_accel_request_resource(struct cxl_dev_state *cxlds, bool is_ram) } EXPORT_SYMBOL_NS_GPL(cxl_accel_request_resource, CXL); +void cxl_accel_set_media_ready(struct cxl_dev_state *cxlds) +{ + cxlds->media_ready = true; +} +EXPORT_SYMBOL_NS_GPL(cxl_accel_set_media_ready, CXL); + static int cxl_memdev_release_file(struct inode *inode, struct file *file) { struct cxl_memdev *cxlmd = diff --git a/drivers/net/ethernet/sfc/efx_cxl.c b/drivers/net/ethernet/sfc/efx_cxl.c index 37d8bfdef517..a84fe7992c53 100644 --- a/drivers/net/ethernet/sfc/efx_cxl.c +++ b/drivers/net/ethernet/sfc/efx_cxl.c @@ -56,6 +56,11 @@ void efx_cxl_init(struct efx_nic *efx) if (cxl_accel_request_resource(cxl->cxlds, true)) pci_info(pci_dev, "CXL accel resource request failed"); + + if (!cxl_await_media_ready(cxl->cxlds)) + cxl_accel_set_media_ready(cxl->cxlds); + else + pci_info(pci_dev, "CXL accel media not active"); } diff --git a/include/linux/cxl_accel_mem.h b/include/linux/cxl_accel_mem.h index 0ba2195b919b..b883c438a132 100644 --- a/include/linux/cxl_accel_mem.h +++ b/include/linux/cxl_accel_mem.h @@ -24,4 +24,6 @@ void cxl_accel_set_resource(struct cxl_dev_state *cxlds, struct resource res, enum accel_resource); int cxl_pci_accel_setup_regs(struct pci_dev *pdev, struct cxl_dev_state *cxlds); int cxl_accel_request_resource(struct cxl_dev_state *cxlds, bool is_ram); +void cxl_accel_set_media_ready(struct cxl_dev_state *cxlds); +int cxl_await_media_ready(struct cxl_dev_state *cxlds); #endif From patchwork Mon Jul 15 17:28:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733703 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2087.outbound.protection.outlook.com [40.107.94.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D2DD54279; Mon, 15 Jul 2024 17:28:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.87 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064539; cv=fail; b=uBXnLdvuuyvwTP1j9z+yOjH7w8BVxmO/IfjHt82jUVUNWvhrvJUfK9q0Gox30J4zbWJzIwHC8ylisJm1T+C7I3Xv+YvU0aDID/sdQ3fA5Bmd/5Z8ZrYX3fQGjpBvKJMFa44pgSOdp3vVnYVWma93OylG94ZgU7tXKMb+Ba+Mhfk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064539; c=relaxed/simple; bh=vYBOFI2bT0XfO6WSIeRGLJWYCmAdV4Ge1NDI7L/Dj+4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=P183NFcj9I9jo9+EgSkM55aFxackrnvq60vC7alfbUVp2INBZjZCkWpMOSMobKt9e2eEUmlXnIepXVLxKRFqn4B7Pwmpyd5rJQ20/YtCCJhnEIPrR/JoCXNBLMVN7QH+t2FRFSMD9PHBQTDkcz39Nw3V2Rqnf9t5ZkyPKAKy3lI= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=aZRtPhyG; arc=fail smtp.client-ip=40.107.94.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="aZRtPhyG" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=WBSjLcUFFLumIRlsFaI9RRixHVByweFgrOd08KIK8187hJHt7K3kUTdaNrYMcwvU3c1UL5/43O365LQP1CPdO2k+xwXs5pVoJQpMcoiZTl7FelxrXL5qcohDe0vVOOexc3+ktWzBIwLRZKjkzO3RRqFIbMJihUPg8t7O/yDbnU8HjRTZQWdKXG4RL+ZoynOFbfZC2Q6BLq74KizoQ1ZP3TW+SJ51F/d0F99Q6TaGQhF+9RwiDmdD39LBJuVWoilkbyJlCgVSHUTrVqAVo0mTj/38BPPjPHR3vgKeCZ3RjHgmZ5hUyV6QcoxbYn3I3HahIHui78uuoSRqXTCYmo3Pbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Wp5zQNnwdNVAo/zrWt6sVtKewFcNn61WN4rfGbpfDIQ=; b=uKDlvhk/lL8ehVK5cYND8zCkhC5GVRzlBEow7FVOM1uU2Y+iu+rL5QfaSD1YIK7SVLyz1LToI0YdY2qIy/ytZthRl0ZwCee5UWWAf30kIpWtZhHEF6V/MJruzRuW90Q1trZAnUyiRu3ySnPAWZJqOgKT5Fh28FJpZdppw7qlk1qTmqZJuPQNawsrFHsDK3tyXr7zLsUlCZjXscp6YSojYuNSIOVwb7v6J52gN7Wogx5WQfGhAiY635K1tq7JxT3K2t+SbfJX8QCqvVwWXlYGy8bbzTkPzPTOKV+d6kqlaFbh77c6sLyz7MK6XAVcpoxMzytY8DRmM3BdNw5rJNJpSw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Wp5zQNnwdNVAo/zrWt6sVtKewFcNn61WN4rfGbpfDIQ=; b=aZRtPhyG7JYCZeuhMDl5+qPQeAxnDzrfp3H9PNOQwy4YvT5tnDf1EhNQ4pGutmmGq9c2fl0/W2fnE8Wd0idESPVOUIM5nu0PnjH9ABMSM2oXaMwByB8yrLhQmdRAlAzs0SrSTtXuncF/IRzmN+uLmuPXh7S9xXO9D+aHd1Fz6Tc= Received: from CY8PR12CA0046.namprd12.prod.outlook.com (2603:10b6:930:49::24) by DS7PR12MB6310.namprd12.prod.outlook.com (2603:10b6:8:95::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28; Mon, 15 Jul 2024 17:28:54 +0000 Received: from CY4PEPF0000EDD7.namprd03.prod.outlook.com (2603:10b6:930:49:cafe::4d) by CY8PR12CA0046.outlook.office365.com (2603:10b6:930:49::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28 via Frontend Transport; Mon, 15 Jul 2024 17:28:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EDD7.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.11 via Frontend Transport; Mon, 15 Jul 2024 17:28:54 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:28:53 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:28:52 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero Subject: [PATCH v2 07/15] cxl: support type2 memdev creation Date: Mon, 15 Jul 2024 18:28:27 +0100 Message-ID: <20240715172835.24757-8-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received-SPF: None (SATLEXMB04.amd.com: alejandro.lucero-palau@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD7:EE_|DS7PR12MB6310:EE_ X-MS-Office365-Filtering-Correlation-Id: 49257c5c-dd99-4194-1d12-08dca4f399e8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|82310400026|376014|921020; X-Microsoft-Antispam-Message-Info: BQQ8IvjF7W81b734u3uEyUBDHwhEu4vwLCsTgelebOtd4hlnpGOjLfvt/3I2hJOiI0G7mx8QU7NlfUWnLOYX6WI4kS4TH2oQaVEhueHvMlJyG1FY2di6rbb2Vr6jCiZ6PGM3pDJO5hA7H2XBnhCAXkrKwCxUjdS30pSb9NYe5PBq2o3JglLVK9hon6hv3eCQYzs0JACaT1u94d/nxwEuaYSFfXrAeg6k8pJTx6wpUyweLGVTIS5RKRw3WCOvIuIc/b+pvYs3Q0ZA2cFl4/7dZ4OKAGDv5V7Oi9ESma0Y6K0Q/8nOApt04RFm8/8iMGcAzRaApCf0iDazR8/xKUqNxW0bqHXAeSmr7h8pOcEwoiZxD/XndGTlWT94WzIf9WkJ9Ql46ONUNy9q9XQ9KYNC3IWRp0JXQATwCd1j5GHr9GKIm4Zg01qebdymDKeThmUmHODBMQrOcDemkYtbEjQBB1L3kJnT1flWzDJ1z7PiCQtkmOthZ9LPz/IH6D2dLl2QBxQLA0e+jZ6LHoO7SCr/eSVo5Uugy+TWPgf8axHuzobM7cwLhkr/L+9DIYyJUCjz+NqIP8M42XU949Zf65rPnee2ZBhmAGUnskEzSzMWFLvGNKz/wLTi8W8VJft98It0t72IgbtyI4yB6EgG+nPoh+ijfrmS12nlUz7CTevUVaUgveA1/11WE3xd0YkqFCydsLJNSm7O3Fe5wGfVxMIEUI8GeXZXVseK9t/cOrO4i/H5ev5NdfHHq5gw+EIN1Nn7no7ksPQ5jivGPJblYjxx4uEJEEwdoZyoBAnprbhZGfqRbwTWsDBYQ1UIS6UHyPZpeDED/WAd1PVkzm/WeQJnKlwzUS7eA0TsMttBhYT34AOcbp/P3lHuo/s/jn7yAAvVLIKdpOEeAUJnp7xFCTHJtuzF/6Kd7oeVTlSMl4JRKqbAAMEFGLN9T6skhUgUk6Y3UODxuZuUkElx8CLRgL/PJqzFXpHTBgS7D1DlNH3DAF1gtuMcgDTvEfwn+wFJCjgYhrJU7aGfiqqu3+ZuYEftscdoxxcJTZGkv0FQ/vtrpRXAKVDAMPHPdLwiDd9VnKzLf0HyCRiO6QG3MfsIBqmt1cDfEl/c6Y0Gp2/6F1QM1VOKzJobbOfLOaIx1m/lTwIFMI4DHMXvKFKxiCEF44MQwxkQF1PASeSJYiK1P/bWu9rRZD4duj+PKJmPs2Uxti6Xxp+2BIGNxGON92enxZYAyMqU43yPOEQYwBQLW31ki7PIxUkhxsTPa8st1T5weJhNrWTfMd69+oZ11lE+6SCmvf2VVst+Vr9O28aThHPRqJmzlDwQBYzvRGeydMIokbIG02Egc84PHxCM1YY06B//J+E0qiyxKVTljain1OyRX9r4BkTz0ybXay2MDovIkm9riUKfR6OaD1NFLKv0VIpnkBp1g/qsxPFEV+mL1ouOhLXRcBaOvjgWnMNwyKpVN2ttECpXzBi6X7DHHFOVPSTjwg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(82310400026)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:28:54.4651 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 49257c5c-dd99-4194-1d12-08dca4f399e8 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD7.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6310 From: Alejandro Lucero Add memdev creation from sfc driver. Current cxl core is relying on a CXL_DEVTYPE_CLASSMEM type device when creating a memdev leading to problems when obtaining cxl_memdev_state references from a CXL_DEVTYPE_DEVMEM type. This last device type is managed by a specific vendor driver and does not need same sysfs files since not userspace intervention is expected. This patch checks for the right device type in those functions using cxl_memdev_state. Signed-off-by: Alejandro Lucero --- drivers/cxl/core/cdat.c | 3 +++ drivers/cxl/core/memdev.c | 9 +++++++++ drivers/cxl/mem.c | 17 +++++++++++------ drivers/net/ethernet/sfc/efx_cxl.c | 10 ++++++++-- include/linux/cxl_accel_mem.h | 3 +++ 5 files changed, 34 insertions(+), 8 deletions(-) diff --git a/drivers/cxl/core/cdat.c b/drivers/cxl/core/cdat.c index bb83867d9fec..0d4679c137d4 100644 --- a/drivers/cxl/core/cdat.c +++ b/drivers/cxl/core/cdat.c @@ -558,6 +558,9 @@ void cxl_region_perf_data_calculate(struct cxl_region *cxlr, }; struct cxl_dpa_perf *perf; + if (!mds) + return; + switch (cxlr->mode) { case CXL_DECODER_RAM: perf = &mds->ram_perf; diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 58a51e7fd37f..b902948b121f 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -468,6 +468,9 @@ static umode_t cxl_ram_visible(struct kobject *kobj, struct attribute *a, int n) struct cxl_memdev *cxlmd = to_cxl_memdev(dev); struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds); + if (!mds) + return 0; + if (a == &dev_attr_ram_qos_class.attr) if (mds->ram_perf.qos_class == CXL_QOS_CLASS_INVALID) return 0; @@ -487,6 +490,9 @@ static umode_t cxl_pmem_visible(struct kobject *kobj, struct attribute *a, int n struct cxl_memdev *cxlmd = to_cxl_memdev(dev); struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds); + if (!mds) + return 0; + if (a == &dev_attr_pmem_qos_class.attr) if (mds->pmem_perf.qos_class == CXL_QOS_CLASS_INVALID) return 0; @@ -507,6 +513,9 @@ static umode_t cxl_memdev_security_visible(struct kobject *kobj, struct cxl_memdev *cxlmd = to_cxl_memdev(dev); struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds); + if (!mds) + return 0; + if (a == &dev_attr_security_sanitize.attr && !test_bit(CXL_SEC_ENABLED_SANITIZE, mds->security.enabled_cmds)) return 0; diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 2f1b49bfe162..f76af75a87b7 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -131,12 +131,14 @@ static int cxl_mem_probe(struct device *dev) dentry = cxl_debugfs_create_dir(dev_name(dev)); debugfs_create_devm_seqfile(dev, "dpamem", dentry, cxl_mem_dpa_show); - if (test_bit(CXL_POISON_ENABLED_INJECT, mds->poison.enabled_cmds)) - debugfs_create_file("inject_poison", 0200, dentry, cxlmd, - &cxl_poison_inject_fops); - if (test_bit(CXL_POISON_ENABLED_CLEAR, mds->poison.enabled_cmds)) - debugfs_create_file("clear_poison", 0200, dentry, cxlmd, - &cxl_poison_clear_fops); + if (mds) { + if (test_bit(CXL_POISON_ENABLED_INJECT, mds->poison.enabled_cmds)) + debugfs_create_file("inject_poison", 0200, dentry, cxlmd, + &cxl_poison_inject_fops); + if (test_bit(CXL_POISON_ENABLED_CLEAR, mds->poison.enabled_cmds)) + debugfs_create_file("clear_poison", 0200, dentry, cxlmd, + &cxl_poison_clear_fops); + } rc = devm_add_action_or_reset(dev, remove_debugfs, dentry); if (rc) @@ -222,6 +224,9 @@ static umode_t cxl_mem_visible(struct kobject *kobj, struct attribute *a, int n) struct cxl_memdev *cxlmd = to_cxl_memdev(dev); struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds); + if (!mds) + return 0; + if (a == &dev_attr_trigger_poison_list.attr) if (!test_bit(CXL_POISON_ENABLED_LIST, mds->poison.enabled_cmds)) diff --git a/drivers/net/ethernet/sfc/efx_cxl.c b/drivers/net/ethernet/sfc/efx_cxl.c index a84fe7992c53..0abe66490ef5 100644 --- a/drivers/net/ethernet/sfc/efx_cxl.c +++ b/drivers/net/ethernet/sfc/efx_cxl.c @@ -57,10 +57,16 @@ void efx_cxl_init(struct efx_nic *efx) if (cxl_accel_request_resource(cxl->cxlds, true)) pci_info(pci_dev, "CXL accel resource request failed"); - if (!cxl_await_media_ready(cxl->cxlds)) + if (!cxl_await_media_ready(cxl->cxlds)) { cxl_accel_set_media_ready(cxl->cxlds); - else + } else { pci_info(pci_dev, "CXL accel media not active"); + return; + } + + cxl->cxlmd = devm_cxl_add_memdev(&pci_dev->dev, cxl->cxlds); + if (IS_ERR(cxl->cxlmd)) + pci_info(pci_dev, "CXL accel memdev creation failed"); } diff --git a/include/linux/cxl_accel_mem.h b/include/linux/cxl_accel_mem.h index b883c438a132..442ed9862292 100644 --- a/include/linux/cxl_accel_mem.h +++ b/include/linux/cxl_accel_mem.h @@ -26,4 +26,7 @@ int cxl_pci_accel_setup_regs(struct pci_dev *pdev, struct cxl_dev_state *cxlds); int cxl_accel_request_resource(struct cxl_dev_state *cxlds, bool is_ram); void cxl_accel_set_media_ready(struct cxl_dev_state *cxlds); int cxl_await_media_ready(struct cxl_dev_state *cxlds); + +struct cxl_memdev *devm_cxl_add_memdev(struct device *host, + struct cxl_dev_state *cxlds); #endif From patchwork Mon Jul 15 17:28:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733704 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2089.outbound.protection.outlook.com [40.107.96.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 035F76CDC0; Mon, 15 Jul 2024 17:28:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.96.89 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064541; cv=fail; b=Xak9eXewzwP6uyK2BAx42Dk2K/ebBlYUIzUjAFj7Q6PQXPIccrQFVNg1qopisvXmL1WSjPTrNLmOMtdjlzXiPoCQIKbdhpJWWxdQCO/TSZK9xQVI9XmMFrKRO/ohARmswUwuUeOx2bgFq3k2GTsgeCWHPR7FPamnAZNe1dVdKjU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064541; c=relaxed/simple; bh=gv8IqeUd/PvmEVS7FXkAfTqWkvW5R7riZvLEbUSb8ss=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=uookPFpdVcZbg2x5AVtMf8N2esx69BYLVmqOzsq/eL096yuTRSGdOPNPSoLx9gwPljlT5GS48g8eoBVTdhOgA4yGpHCfPR3KZRyWmV7WoW/iHTPzf2nG3AVgpR1cEz29jTEUJZ4VxbiazHkFilz1PhxjgeaeKSaHjebQReJ+YC4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=V25qE8Mp; arc=fail smtp.client-ip=40.107.96.89 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="V25qE8Mp" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=SwPSO7g8n3UhU4QmLivk0bumNs02cn5idemFwxPg6A/z11ydXtIiIduPSBphGarlXKQmyU/KekwDV0kj5CtSD8/KLZWVQR62L7SzEfYczUXKa8Iy+SjvV+CpyzYwiejxuaQ2MvmKLrZJ8Y26fvGKIQBw+ag5nsMXq7bCTXUk8W9OpQw66p3Qs0vMaZ0OxwdWq00j5n9Ww2cgLfQinPKJ4cbP5Ycslc2PIf2HqJlmSJJkinlHGJ9jbfR1f0mC+hpTmCAN7HC83fPjOAMT2Ri9vTYCUzLhhNtf53X7f3SpeILJyo3yH5dya+lB+IoouBgSXiGwJepriIh0MCty/TMzuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=I1h2bTIlDVA5eUc+B8YwO9nUxz2ompCDE5qER87lyeg=; b=PwKrWeOOdgg1AeMRUdDjR0ehC4zi4Mx6OrQeNM2WRKst9byB3Ewc1Gn/kiC7u/HDbDlU/abGBxJ+i6A2lD/pZuScnfCR0Bc0wqZFRc2tCIsbLk+nbcZcjeGCIrj0e1sCwaR+sTaKZ7+9JuIrKjP5TuNTsLBYZJo+BLykFi9SySKl2IggnpiXtfX/dw7WOX3svsH/nmYJUD1/NdPQ4alZVuqefEw6nye79zvjWplw+PcDIaAkl+RQuOeVVmAyYgMyVGnXO8BMuMph4O41advq++efRKElQm8LNQy90ADaFbDsQfKrAzuBigM7+683T0AD2D90sZEYlLUU6qBuahgNJA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=I1h2bTIlDVA5eUc+B8YwO9nUxz2ompCDE5qER87lyeg=; b=V25qE8MpfYLTnfcQpNf3rrBE7Ll/Rt6MgkkGYp1yqcI9CrPMJilEfjeyxsge9xwN0rgRQvZBb58vrpKQb8tiVggo9eLWuInXw3M2Ite6P3AQpCZ5z/wjnBcRpreqUBFsq2PMC/fbkWYUXmSOF5T08sHZ+sylEf29ai4/GvlDlU4= Received: from CY8PR12CA0034.namprd12.prod.outlook.com (2603:10b6:930:49::22) by MN6PR12MB8471.namprd12.prod.outlook.com (2603:10b6:208:473::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28; Mon, 15 Jul 2024 17:28:57 +0000 Received: from CY4PEPF0000EDD7.namprd03.prod.outlook.com (2603:10b6:930:49:cafe::a) by CY8PR12CA0034.outlook.office365.com (2603:10b6:930:49::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.29 via Frontend Transport; Mon, 15 Jul 2024 17:28:56 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EDD7.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.11 via Frontend Transport; Mon, 15 Jul 2024 17:28:56 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:28:54 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:28:53 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero Subject: [PATCH v2 08/15] cxl: indicate probe deferral Date: Mon, 15 Jul 2024 18:28:28 +0100 Message-ID: <20240715172835.24757-9-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received-SPF: None (SATLEXMB04.amd.com: alejandro.lucero-palau@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD7:EE_|MN6PR12MB8471:EE_ X-MS-Office365-Filtering-Correlation-Id: f5a4338f-ea7e-4089-d32d-08dca4f39b22 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|36860700013|82310400026|921020; X-Microsoft-Antispam-Message-Info: DDKE2x7QzmGz6gCFCyWzZjPishiAfOe/wkoV6Vi4hjanhZGgnDgwx5x0tMeW93JcnvgXgnujVieGic7mNz+zZpACeu4xzN5mKSNmC7LrF0QyUDq1pPQOHs5w43cXABYTz9+fLwINKbhU0cHxXWenwx+kbnQGhTrX4SoYdyE+VP3YlzuD+iZOZV2IbzM6939pRBSoufa0q6kHICXaMmjHVq1VoTZxd0+ewYvxHf/bawaSWnwOD473SQ4Z65OgH6phIBNGQyaT94ocqcwc1Mm7Rw9lomD/CSj6ltLauEWLv7qqYeyY3A5F9wn3pZ7Mhl9TzGKNNPiBXgx4iQHbUGp6Z/annrCUW4i6tVc2MiKQWcUOliY2+mO76IJQ1sBcJGUqkkQRXS39ndJU8ZzZhTb3HeT11ZGy5IVbUzzBGs26b64VBSeWse1+rHMpGjTuV+dVJTbWM82aui6DfXILyB8fz6I+4HvBNjfkK7bFeI/6ZF0t939OsbbZk3q84YiDF9d2poB6dNExGvyhAEnEYJajBYtEpJPKofUAtvUyvGU4gsd8LV0pl6OX5YWqNNlNZ5g6ecBkD3clUKYkES+uyoKCg3N4NU4A6jUE6z2LsIsF34QIya7RK2kScg/3uDR25xmLlcdXuVAYhXT3PW1uTDdqfOmaIZFfrGdHYNpbPiT1Lt3kL/PTqhVMOyXX8zWZqczXdYJXBcua9m/Y68K6gNbaZVObCPfEoxjQSyfuswEJZmnFh5MvKf/rn+mesNWKikNgjfkZdgudMbXCMhJIvCx3q3tx5iZIfgZw8AVlKUKJ6plQeslRVymHXIr2s9sliuA2TDrjXcWuHmmtFABhEAdDsLszMvWYQK3T7ovQHQGdCgORAugO2MMtJf++eG47j7S4A9xxKUUPLDyhu8/DGEDT6ajSPKHMeS3srW5tS5yiPL+FMOD/8bS/0ZRDU33rnNLsRVzc/GcW6z+PsIzOYvKvGRNa2aRH4QoPpNxM6nFf7Yo/8SgO1L5fegWbIV2AszAcIW7eFFppwM7pjTz9nGpeTb/qn149MC9rbLXPXPJXGyN1aZQy00mQlc+HyCBt+I4MIXeFLSq+a5lZJ+N1d5R1aQibLMZ5a5Vn9KRRWD2WLxFcMaO3WWFRWAtLkPciQ8SjwCdH7/Ti7RU1RNM3ZSSlipMqm76pbUTvVYFQ+uGuJek/E+YWi+19jwjs03EqDNKJSFJ1sRT6TTXEOCkazgZxGRsfcro2xKT5TkhAQSJLCFlqfuT18xKSGXOykhpAe1nleaXWA0TUcPRuiEnYMbbIaxXKRamLUtLbffBhlRjeKQtVX60ckyRJfye7TIAQ7IZ+FZm6hpI0rbfU7cSVVltkPrlttGc5hTAQnRyVHrd8LngWX5LZW/91wB6NRDpW7Xj7Ez8jOqZkM5ELdh7t++VdZg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:28:56.5276 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f5a4338f-ea7e-4089-d32d-08dca4f39b22 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD7.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN6PR12MB8471 From: Alejandro Lucero The first stop for a CXL accelerator driver that wants to establish new CXL.mem regions is to register a 'struct cxl_memdev. That kicks off cxl_mem_probe() to enumerate all 'struct cxl_port' instances in the topology up to the root. If the root driver has not attached yet the expectation is that the driver waits until that link is established. The common cxl_pci_driver has reason to keep the 'struct cxl_memdev' device attached to the bus until the root driver attaches. An accelerator may want to instead defer probing until CXL resources can be acquired. Use the @endpoint attribute of a 'struct cxl_memdev' to convey when accelerator driver probing should be defferred vs failed. Provide that indication via a new cxl_acquire_endpoint() API that can retrieve the probe status of the memdev. The first consumer of this API is a test driver that excercises the CXL Type-2 flow. Based on https://lore.kernel.org/linux-cxl/168592149709.1948938.8663425987110396027.stgit@dwillia2-xfh.jf.intel.com/T/#m18497367d2ae38f88e94c06369eaa83fa23e92b2 Signed-off-by: Alejandro Lucero Co-developed-by: Dan Williams --- drivers/cxl/core/memdev.c | 41 ++++++++++++++++++++++++++++++ drivers/cxl/core/port.c | 2 +- drivers/cxl/mem.c | 7 +++-- drivers/net/ethernet/sfc/efx_cxl.c | 10 +++++++- include/linux/cxl_accel_mem.h | 3 +++ 5 files changed, 59 insertions(+), 4 deletions(-) diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index b902948b121f..d51c8bfb32e3 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -1137,6 +1137,47 @@ struct cxl_memdev *devm_cxl_add_memdev(struct device *host, } EXPORT_SYMBOL_NS_GPL(devm_cxl_add_memdev, CXL); +/* + * Try to get a locked reference on a memdev's CXL port topology + * connection. Be careful to observe when cxl_mem_probe() has deposited + * a probe deferral awaiting the arrival of the CXL root driver +*/ +struct cxl_port *cxl_acquire_endpoint(struct cxl_memdev *cxlmd) +{ + struct cxl_port *endpoint; + int rc = -ENXIO; + + device_lock(&cxlmd->dev); + endpoint = cxlmd->endpoint; + if (!endpoint) + goto err; + + if (IS_ERR(endpoint)) { + rc = PTR_ERR(endpoint); + goto err; + } + + device_lock(&endpoint->dev); + if (!endpoint->dev.driver) + goto err_endpoint; + + return endpoint; + +err_endpoint: + device_unlock(&endpoint->dev); +err: + device_unlock(&cxlmd->dev); + return ERR_PTR(rc); +} +EXPORT_SYMBOL_NS(cxl_acquire_endpoint, CXL); + +void cxl_release_endpoint(struct cxl_memdev *cxlmd, struct cxl_port *endpoint) +{ + device_unlock(&endpoint->dev); + device_unlock(&cxlmd->dev); +} +EXPORT_SYMBOL_NS(cxl_release_endpoint, CXL); + static void sanitize_teardown_notifier(void *data) { struct cxl_memdev_state *mds = data; diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index d66c6349ed2d..3c6b896c5f65 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1553,7 +1553,7 @@ static int add_port_attach_ep(struct cxl_memdev *cxlmd, */ dev_dbg(&cxlmd->dev, "%s is a root dport\n", dev_name(dport_dev)); - return -ENXIO; + return -EPROBE_DEFER; } parent_port = find_cxl_port(dparent, &parent_dport); diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index f76af75a87b7..383a6f4829d3 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -145,13 +145,16 @@ static int cxl_mem_probe(struct device *dev) return rc; rc = devm_cxl_enumerate_ports(cxlmd); - if (rc) + if (rc) { + cxlmd->endpoint = ERR_PTR(rc); return rc; + } parent_port = cxl_mem_find_port(cxlmd, &dport); if (!parent_port) { dev_err(dev, "CXL port topology not found\n"); - return -ENXIO; + cxlmd->endpoint = ERR_PTR(-EPROBE_DEFER); + return -EPROBE_DEFER; } if (resource_size(&cxlds->pmem_res) && IS_ENABLED(CONFIG_CXL_PMEM)) { diff --git a/drivers/net/ethernet/sfc/efx_cxl.c b/drivers/net/ethernet/sfc/efx_cxl.c index 0abe66490ef5..2cf4837ddfc1 100644 --- a/drivers/net/ethernet/sfc/efx_cxl.c +++ b/drivers/net/ethernet/sfc/efx_cxl.c @@ -65,8 +65,16 @@ void efx_cxl_init(struct efx_nic *efx) } cxl->cxlmd = devm_cxl_add_memdev(&pci_dev->dev, cxl->cxlds); - if (IS_ERR(cxl->cxlmd)) + if (IS_ERR(cxl->cxlmd)) { pci_info(pci_dev, "CXL accel memdev creation failed"); + return; + } + + cxl->endpoint = cxl_acquire_endpoint(cxl->cxlmd); + if (IS_ERR(cxl->endpoint)) + pci_info(pci_dev, "CXL accel acquire endpoint failed"); + + cxl_release_endpoint(cxl->cxlmd, cxl->endpoint); } diff --git a/include/linux/cxl_accel_mem.h b/include/linux/cxl_accel_mem.h index 442ed9862292..701910021df8 100644 --- a/include/linux/cxl_accel_mem.h +++ b/include/linux/cxl_accel_mem.h @@ -29,4 +29,7 @@ int cxl_await_media_ready(struct cxl_dev_state *cxlds); struct cxl_memdev *devm_cxl_add_memdev(struct device *host, struct cxl_dev_state *cxlds); + +struct cxl_port *cxl_acquire_endpoint(struct cxl_memdev *cxlmd); +void cxl_release_endpoint(struct cxl_memdev *cxlmd, struct cxl_port *endpoint); #endif From patchwork Mon Jul 15 17:28:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733707 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2074.outbound.protection.outlook.com [40.107.223.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 20A5171B3A; Mon, 15 Jul 2024 17:29:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.74 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064545; cv=fail; b=XYL+FU5toyWZ//C+dawXbRt8eSLbO/jLOjMucf5mZSr5yZ4NFlozLlruhseVTFci7hHdIeFcZR7g1gsJbXE+kpEWUnIzitPpwTC4cAOrWd9uFtoMKB2jLsE9rqNJMXddUaLM5opgJyMCSxWXCAfdPXaeMX/161tTAk03B9py910= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064545; c=relaxed/simple; bh=qgWS3BfT14ZBavTF8bZ0VMsweurB3s2FqgLcm22BZr8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=M248xshtNQTr0CGoQT8UoekE7nRcRm6Fl7mc/wTusyrnG4E08L5YjJxuIWrgET52ujiLOVEPtWF3SfFhbfYNCqQqg3n9rgEd/3P5ImLWbBto+u8Y0oyxTcFq34BwwQBZ9IT6gFRGj0Dvhr3nYCog9XK2UOFRwv81jpcU4qKqtHo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=lQkuLVOQ; arc=fail smtp.client-ip=40.107.223.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="lQkuLVOQ" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=RXQf5tCwLd6waYAu799YkA1IbM4FAS4YTs+X71jkNyIAfi6+ZfwtIhHNpug8Csv4gQtIIc8nEq2j/3w1OtE1Xk7Sps4/BePayzTFi2INozBYPVCx8yEQWRzE79n0VAAdQSBdRthxwvUU+iRtj1qkbaFd5C+HsiuLz8Qu+w1Z3iBPynYXv84sfUFmHBLyLB7Xc3tdmNwJzu8WsO0iRTZkXG3ASyVlDY4Cc98UKOJh7bndcavQA2kqzZnMT/sAOlJIvuvyjiZXc4mPfy4XpZaCWJOvRmDrWVIIwm7Zvce4EA5/FDAl2bKvE5NAPaUYj7yZis7Fv3ClZPxwyYvgFrXvCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Dizz2TTkzDjjd1y2eDQGqIAEiQuyoYn0IDgCcbZi8F4=; b=MqOe0Lz43X7WO+Ho4u/eJBRdL3tl8Uz/q7GWs4Ef5AF3/NTI4kCSX6aEw9jkgrbAp4nQqzhYCMBqWFlHLqxmZxqpb1jtjQ4qh2nfTa2R82XegnRC7msDv7fZDR/rV4xd4QGxTZosSjx1c7hOcCIszSbN75c20mf+ysFk4i5WLePqKL4dAeepT7Us2HkGI/j7e25GnlSEbxFxPtEOIMeH6/Drjiuz/neNONJfmKii414lVyBuRDPI73MpmFTLXu3dulwqMu5hSTpdEN5BUtcpxy7H/kNNAKbv14gnh/dLhW0lDYkjwb52R4O3DzaUisCa+E1lvnzOb2gukaq9hMzjGA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Dizz2TTkzDjjd1y2eDQGqIAEiQuyoYn0IDgCcbZi8F4=; b=lQkuLVOQq+RDmO2KuRFHBlqw7rVODxLROkzEO0cQS9vq2Ta+ddqtSzBBfheTs8WmO+B5sE9jUqULB5zgE12LEtRXXJl9gXh61DW7KIweftCUWWyTNpzmdrmJrZJsV7SIrzWMgzpKtwYJ1wryR124aYCkWvXDneXNAf0lBz4ahyc= Received: from CY8PR12CA0047.namprd12.prod.outlook.com (2603:10b6:930:49::20) by DM4PR12MB5939.namprd12.prod.outlook.com (2603:10b6:8:6a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.29; Mon, 15 Jul 2024 17:28:58 +0000 Received: from CY4PEPF0000EDD7.namprd03.prod.outlook.com (2603:10b6:930:49:cafe::e) by CY8PR12CA0047.outlook.office365.com (2603:10b6:930:49::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.29 via Frontend Transport; Mon, 15 Jul 2024 17:28:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EDD7.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.11 via Frontend Transport; Mon, 15 Jul 2024 17:28:58 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:28:56 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:28:56 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:28:55 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero Subject: [PATCH v2 09/15] cxl: define a driver interface for HPA free space enumaration Date: Mon, 15 Jul 2024 18:28:29 +0100 Message-ID: <20240715172835.24757-10-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD7:EE_|DM4PR12MB5939:EE_ X-MS-Office365-Filtering-Correlation-Id: 1e34b7c1-e34f-4764-a086-08dca4f39c13 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|1800799024|36860700013|921020; X-Microsoft-Antispam-Message-Info: =?utf-8?q?MegpPyX+oMzQudzPtMoGExXXj0KQVSa?= =?utf-8?q?FnOeFJ2JBmqUr8YERDQXsfiW+6varH5ml/1m/qn4nSzSiu1Lj4W/sqNTMIFoePvga?= =?utf-8?q?cPcYJvJJWM013t9qm4YhWNYiwGzGpMwd0c7RqWIaBlL8TkBQiRDD0un2d3eHMJ/+D?= =?utf-8?q?ivQMqEln1qCQxLmw8h9eYvBUzsVljewmx0w5zCU+rvhbfsuUaI0YGZowT7hYuTSxv?= =?utf-8?q?c7nAewQ/HEolB3PoaDV/vHGKaTEk14O6DejMyY4wUSh4r0KTz4ghapYvg/fV8YMMx?= =?utf-8?q?kRhqED1i/jm0dePjGcuR84px3Z/9e8gpIcAFYPg8UWz1Gb5VOLXuUYddaJkwUwA/p?= =?utf-8?q?Aw76FZs8EvY/QF/eh00y7Mh2OxSZiIxNKd4xddX6xnZYMZ+jOZYdVhHyLUzCsAq3V?= =?utf-8?q?MqwWrrfUjjWXm15+1t4Raw4cVb/YGeKDgrnnwRbSXdvqx3IM851fpHDtaZURrFS0X?= =?utf-8?q?AeS9MoGE4u3UQenPN0HfFSskHKwZTc+SfGJBaeKusxXJqHPwdjRjPviNMQu1HzLNr?= =?utf-8?q?FL4lgVHKasXJUdSmu3gdT8vEM4K3PgJkHWMwB0jkEsiCzz6qElLzAF9WO1aNDjatq?= =?utf-8?q?NfdormJ+8rdN4bmYngN5nDGd10bQ1F+OBNVpWsp6/BNC/yzwFKP51UTkPA+KH8e2p?= =?utf-8?q?knzgdIMd9mGA5NiDJdZ3fImz2A4YoN49QMdyHW74YMk1sE5JKB9y+TGd9atGHDWbT?= =?utf-8?q?0GKLCrXn6pY6RLF+GQqVVgCDuhB+wIszdYlvFOxIdj7y9ni0QXQgfnPE8tSNfscLu?= =?utf-8?q?D3WsDQrNQ0lURkMhbDj86lPR2et//HV7Rdq9EFI9wLUzIERbteKQ6ghnfVdjgcQFN?= =?utf-8?q?Sy5RvZDQu+osIJS6OCV4siy0D5AzlrHlx/tzYFnwWdXH0TEprVD4XotyQNJlvDRXu?= =?utf-8?q?Kw1wt2LkPePfiJwQ+XY3tBGjF8rmEBFS2JhaTGUB1H+LWpS9k7eYpaHYXCu6abODB?= =?utf-8?q?LtJZdeWIpgVECabg3fA1D2PFQciphQG1qzz2nFDorqSCBTY6HdEW4jwZ3BVKWqIhz?= =?utf-8?q?7cjYOobAC6wMlEN9FIKEySNWKNmIcgGJFUoM1Gq1cFs3sh2sMBxngGqM3rnU+5Chh?= =?utf-8?q?kJWjM4XKv7gBvAgaKYZ0/th3drTJf7aa9GxhO/nMGfLtnTpzCwOb8/AmTga7VlgHO?= =?utf-8?q?BCrlSyxTLm3mJpQu+jo1DwkCukTnctO9qzNS1mJcMwosDQ57IBfi6bOCuS5IQQm7B?= =?utf-8?q?QjsmyCxT4bc64EU9LTcdNmaqGdl55t7qMmIaLExl587suVNW577/cZNAhFELOyFB8?= =?utf-8?q?Nu2+wQxhjmTkka5Ak5jr2mrXx/ZvD9Hq4BNAl7w2zllZEyL33Jv0xmCbGb4o6qnTl?= =?utf-8?q?EcI2mtHsWCFe4QAoiNXxyY8hqcVGYrEVHw=3D=3D?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(376014)(1800799024)(36860700013)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:28:58.1057 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1e34b7c1-e34f-4764-a086-08dca4f39c13 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD7.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5939 From: Alejandro Lucero CXL region creation involves allocating capacity from device DPA (device-physical-address space) and assigning it to decode a given HPA (host-physical-address space). Before determining how much DPA to allocate the amount of available HPA must be determined. Also, not all HPA is create equal, some specifically targets RAM, some target PMEM, some is prepared for device-memory flows like HDM-D and HDM-DB, and some is host-only (HDM-H). Wrap all of those concerns into an API that retrieves a root decoder (platform CXL window) that fits the specified constraints and the capacity available for a new region. Based on https://lore.kernel.org/linux-cxl/168592149709.1948938.8663425987110396027.stgit@dwillia2-xfh.jf.intel.com/T/#m6fbe775541da3cd477d65fa95c8acdc347345b4f Signed-off-by: Alejandro Lucero Co-developed-by: Dan Williams --- drivers/cxl/core/region.c | 161 +++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 3 + drivers/cxl/cxlmem.h | 5 + drivers/net/ethernet/sfc/efx_cxl.c | 14 +++ include/linux/cxl_accel_mem.h | 9 ++ 5 files changed, 192 insertions(+) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 538ebd5a64fd..ca464bfef77b 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -702,6 +702,167 @@ static int free_hpa(struct cxl_region *cxlr) return 0; } + +struct cxlrd_max_context { + struct device * const *host_bridges; + int interleave_ways; + unsigned long flags; + resource_size_t max_hpa; + struct cxl_root_decoder *cxlrd; +}; + +static int find_max_hpa(struct device *dev, void *data) +{ + struct cxlrd_max_context *ctx = data; + struct cxl_switch_decoder *cxlsd; + struct cxl_root_decoder *cxlrd; + struct resource *res, *prev; + struct cxl_decoder *cxld; + resource_size_t max; + int found; + + if (!is_root_decoder(dev)) + return 0; + + cxlrd = to_cxl_root_decoder(dev); + cxld = &cxlrd->cxlsd.cxld; + if ((cxld->flags & ctx->flags) != ctx->flags) { + dev_dbg(dev, "find_max_hpa, flags not matching: %08lx vs %08lx\n", + cxld->flags, ctx->flags); + return 0; + } + + /* A Host bridge could have more interleave ways than an + * endpoint, couldn´t it? + * + * What does interleave ways mean here in terms of the requestor? + * Why the FFMWS has 0 interleave ways but root port has 1? + */ + if (cxld->interleave_ways != ctx->interleave_ways) { + dev_dbg(dev, "find_max_hpa, interleave_ways not matching\n"); + return 0; + } + + cxlsd = &cxlrd->cxlsd; + + guard(rwsem_read)(&cxl_region_rwsem); + found = 0; + for (int i = 0; i < ctx->interleave_ways; i++) + for (int j = 0; j < ctx->interleave_ways; j++) + if (ctx->host_bridges[i] == + cxlsd->target[j]->dport_dev) { + found++; + break; + } + + if (found != ctx->interleave_ways) { + dev_dbg(dev, "find_max_hpa, no interleave_ways found\n"); + return 0; + } + + /* + * Walk the root decoder resource range relying on cxl_region_rwsem to + * preclude sibling arrival/departure and find the largest free space + * gap. + */ + lockdep_assert_held_read(&cxl_region_rwsem); + max = 0; + res = cxlrd->res->child; + if (!res) + max = resource_size(cxlrd->res); + else + max = 0; + + for (prev = NULL; res; prev = res, res = res->sibling) { + struct resource *next = res->sibling; + resource_size_t free = 0; + + if (!prev && res->start > cxlrd->res->start) { + free = res->start - cxlrd->res->start; + max = max(free, max); + } + if (prev && res->start > prev->end + 1) { + free = res->start - prev->end + 1; + max = max(free, max); + } + if (next && res->end + 1 < next->start) { + free = next->start - res->end + 1; + max = max(free, max); + } + if (!next && res->end + 1 < cxlrd->res->end + 1) { + free = cxlrd->res->end + 1 - res->end + 1; + max = max(free, max); + } + } + + if (max > ctx->max_hpa) { + if (ctx->cxlrd) + put_device(CXLRD_DEV(ctx->cxlrd)); + get_device(CXLRD_DEV(cxlrd)); + ctx->cxlrd = cxlrd; + ctx->max_hpa = max; + dev_info(CXLRD_DEV(cxlrd), "found %pa bytes of free space\n", &max); + } + return 0; +} + +/** + * cxl_get_hpa_freespace - find a root decoder with free capacity per constraints + * @endpoint: an endpoint that is mapped by the returned decoder + * @interleave_ways: number of entries in @host_bridges + * @flags: CXL_DECODER_F flags for selecting RAM vs PMEM, and HDM-H vs HDM-D[B] + * @max: output parameter of bytes available in the returned decoder + * + * The return tuple of a 'struct cxl_root_decoder' and 'bytes available (@max)' + * is a point in time snapshot. If by the time the caller goes to use this root + * decoder's capacity the capacity is reduced then caller needs to loop and + * retry. + * + * The returned root decoder has an elevated reference count that needs to be + * put with put_device(cxlrd_dev(cxlrd)). Locking context is with + * cxl_{acquire,release}_endpoint(), that ensures removal of the root decoder + * does not race. + */ +struct cxl_root_decoder *cxl_get_hpa_freespace(struct cxl_port *endpoint, + int interleave_ways, + unsigned long flags, + resource_size_t *max) +{ + + struct cxlrd_max_context ctx = { + .host_bridges = &endpoint->host_bridge, + .interleave_ways = interleave_ways, + .flags = flags, + }; + struct cxl_port *root_port; + struct cxl_root *root; + + if (!is_cxl_endpoint(endpoint)) { + dev_dbg(&endpoint->dev, "hpa requestor is not an endpoint\n"); + return ERR_PTR(-EINVAL); + } + + root = find_cxl_root(endpoint); + if (!root) { + dev_dbg(&endpoint->dev, "endpoint can not be related to a root port\n"); + return ERR_PTR(-ENXIO); + } + + root_port = &root->port; + down_read(&cxl_region_rwsem); + device_for_each_child(&root_port->dev, &ctx, find_max_hpa); + up_read(&cxl_region_rwsem); + put_device(&root_port->dev); + + if (!ctx.cxlrd) + return ERR_PTR(-ENOMEM); + + *max = ctx.max_hpa; + return ctx.cxlrd; +} +EXPORT_SYMBOL_NS_GPL(cxl_get_hpa_freespace, CXL); + + static ssize_t size_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t len) { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 9973430d975f..d3fdd2c1e066 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -770,6 +770,9 @@ struct cxl_decoder *to_cxl_decoder(struct device *dev); struct cxl_root_decoder *to_cxl_root_decoder(struct device *dev); struct cxl_switch_decoder *to_cxl_switch_decoder(struct device *dev); struct cxl_endpoint_decoder *to_cxl_endpoint_decoder(struct device *dev); + +#define CXLRD_DEV(cxlrd) &cxlrd->cxlsd.cxld.dev + bool is_root_decoder(struct device *dev); bool is_switch_decoder(struct device *dev); bool is_endpoint_decoder(struct device *dev); diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 8f2a820bd92d..a0e0795ec064 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -877,4 +877,9 @@ struct cxl_hdm { struct seq_file; struct dentry *cxl_debugfs_create_dir(const char *dir); void cxl_dpa_debug(struct seq_file *file, struct cxl_dev_state *cxlds); +struct cxl_root_decoder *cxl_get_hpa_freespace(struct cxl_port *endpoint, + int interleave_ways, + unsigned long flags, + resource_size_t *max); + #endif /* __CXL_MEM_H__ */ diff --git a/drivers/net/ethernet/sfc/efx_cxl.c b/drivers/net/ethernet/sfc/efx_cxl.c index 2cf4837ddfc1..6d49571ccff7 100644 --- a/drivers/net/ethernet/sfc/efx_cxl.c +++ b/drivers/net/ethernet/sfc/efx_cxl.c @@ -22,6 +22,7 @@ void efx_cxl_init(struct efx_nic *efx) { struct pci_dev *pci_dev = efx->pci_dev; struct efx_cxl *cxl = efx->cxl; + resource_size_t max = 0; struct resource res; u16 dvsec; @@ -74,6 +75,19 @@ void efx_cxl_init(struct efx_nic *efx) if (IS_ERR(cxl->endpoint)) pci_info(pci_dev, "CXL accel acquire endpoint failed"); + cxl->cxlrd = cxl_get_hpa_freespace(cxl->endpoint, 1, + CXL_DECODER_F_RAM | CXL_DECODER_F_TYPE2, + &max); + + if (IS_ERR(cxl->cxlrd)) { + pci_info(pci_dev, "CXL accel get HPA failed"); + goto out; + } + + if (max < EFX_CTPIO_BUFFER_SIZE) + pci_info(pci_dev, "CXL accel not enough free HPA space %llu < %u\n", + max, EFX_CTPIO_BUFFER_SIZE); +out: cxl_release_endpoint(cxl->cxlmd, cxl->endpoint); } diff --git a/include/linux/cxl_accel_mem.h b/include/linux/cxl_accel_mem.h index 701910021df8..f3e77688ffe0 100644 --- a/include/linux/cxl_accel_mem.h +++ b/include/linux/cxl_accel_mem.h @@ -6,6 +6,10 @@ #ifndef __CXL_ACCEL_MEM_H #define __CXL_ACCEL_MEM_H +#define CXL_DECODER_F_RAM BIT(0) +#define CXL_DECODER_F_PMEM BIT(1) +#define CXL_DECODER_F_TYPE2 BIT(2) + enum accel_resource{ CXL_ACCEL_RES_DPA, CXL_ACCEL_RES_RAM, @@ -32,4 +36,9 @@ struct cxl_memdev *devm_cxl_add_memdev(struct device *host, struct cxl_port *cxl_acquire_endpoint(struct cxl_memdev *cxlmd); void cxl_release_endpoint(struct cxl_memdev *cxlmd, struct cxl_port *endpoint); + +struct cxl_root_decoder *cxl_get_hpa_freespace(struct cxl_port *endpoint, + int interleave_ways, + unsigned long flags, + resource_size_t *max); #endif From patchwork Mon Jul 15 17:28:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733705 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2078.outbound.protection.outlook.com [40.107.223.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 37EED7173C; Mon, 15 Jul 2024 17:29:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.78 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064544; cv=fail; b=KwDaTPwe1DvuGZYJgN/2qG5g4HqHG7g0qfNfKOtehQvF7ElXWv3PRoOudX2lIGzW2r0bDbq0zA6W0UxLsvvdacZovLypzYXqLCt7veFvOB8WMHvitzdOPl6IuEk6e4k9OmBA/3YaXGD23FE6jV+t3EIG3BMYfi/FwRECgLNuF2Y= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064544; c=relaxed/simple; bh=pU6ylkH8U/vWKVgxV1dXn2pBW2C/ATOuIpeV7kiKAPI=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=GhFsNf0xR2mRGkNJt+B4JdoNsJzcWXcVDZlSB/bJER2U1QpX6x/RWLQ5MUYIFHNII4JhrIiOid6lubYVtrEDTd4YwU6ch0eem7uY87hSNONxIjBlw+YyJ0XWb8/H+20Op+iaMnR/cBb7QocWwWikJgTcZ55jHhcXSQLMsq8P1EY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=EqasE+L5; arc=fail smtp.client-ip=40.107.223.78 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="EqasE+L5" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=sIiod1/egHUNNqR5cEM0JBjuNEB8sYXNtX0XNRkzKNp3Aqu1S29CIMOjLcBux6cGtQiR4QbAijog9N3Dl8bJgKnoan20FIUYVWQsDebguzmWvWkVoZpflyYAvaZyOAfE7GtjBjBxNEmoP6xxlL66iAnJb/xNP3Qnbnh2obbqfycfNSM50+FdpCMHMoF0BAZNP2hL0wto5mqAPEJyv/H/MQKEOoTmfxx8ed/gIBAjotImcipuWtI8YWAfrkkYLe9i6aqwEt3y1ey75QjaAyUmcFdOwVi1swijb86Nhcq2w9Vv8l8ZOCAce2N7nqZZkeuPzBwHj2sWEZQrKW+8Pl5hvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZjXnjXwhOwLS5GKsMstS8ETSnnG0c+V+5JINt7DPHbQ=; b=wCl4tLwdFWk5uN8FtD83HmMN+LyXBhgqjIpHeKQCFhNIvXs1FFDTedQ0nV7sF8n8fHH92hNARlFdDBE/eeTpsko6q/x6UXVstfuiA8HxbXp46hWQT32jxuuBEX0L5fP0UMeqXhgvo+D/pXSPA+42C4D+DoOZKFXtjA3m2+bIn/yZS/exnyp5L5dm7lJEjTxA9O9uZgUpQ8F665tJ45poVPxR5J0rl0jURlduuKSKCv3hGoMzlrWL4KxWWIZOOHLBhPbpulAx8nnvGAELrJDVt4DsP1virKsTGRBS0bvRyCd1mkaPpyJjp+e7bOE/agedYQvd9U8FvOg303X4hpor7w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZjXnjXwhOwLS5GKsMstS8ETSnnG0c+V+5JINt7DPHbQ=; b=EqasE+L5XdDISp7204Hp0AsN5CEx+7h+S6QwbMMBuqY8HInDjebBS5d3ecQmEIA3dxrcoK1pwaRMQaSPxABmEk9jMkFEG0Ok0mGxnGzKauhmpOIt/oWx4bHiSoq1pPH6TG6LXo8QRGrN4lF+zbGF2x857CdQkudyzD5YaS+1k64= Received: from BYAPR02CA0024.namprd02.prod.outlook.com (2603:10b6:a02:ee::37) by PH7PR12MB7793.namprd12.prod.outlook.com (2603:10b6:510:270::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28; Mon, 15 Jul 2024 17:28:59 +0000 Received: from SJ1PEPF000023DA.namprd21.prod.outlook.com (2603:10b6:a02:ee:cafe::81) by BYAPR02CA0024.outlook.office365.com (2603:10b6:a02:ee::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28 via Frontend Transport; Mon, 15 Jul 2024 17:28:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by SJ1PEPF000023DA.mail.protection.outlook.com (10.167.244.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.5 via Frontend Transport; Mon, 15 Jul 2024 17:28:58 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:28:58 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:28:57 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:28:56 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero Subject: [PATCH v2 10/15] cxl: define a driver interface for DPA allocation Date: Mon, 15 Jul 2024 18:28:30 +0100 Message-ID: <20240715172835.24757-11-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF000023DA:EE_|PH7PR12MB7793:EE_ X-MS-Office365-Filtering-Correlation-Id: 43d8f4c3-424a-481f-0742-08dca4f39c85 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|82310400026|921020; X-Microsoft-Antispam-Message-Info: CfMT5XW+m0bI0jWoz7ZHNx3qsgRK8pSvjdyB0nZCGBBHdu7W+90djFTaA9isQ2rZ8DUB9Y+gCHvXPZl2DiAghsgW7uiZXYJaQrN8bdNUgy0j43RHYpJL9Ieit1uQ0nfHOfNZYqdPZwhgU9xFfntn5bhey22y5g7SyOTYTz/9i2B8M8W55n2RZJ8DnDJemI6JfoT8I9YTXiomWxNccojgrGtYPvQU8gaQX4gjppoH3BHp9bku7n9hk/L5z6cckDyXzsxr2OFgfK2s0RRtvDkDz/qaoB5z2QMJhNKlvoKgnj+H6MofnplfAqyZYOHHXYJtubGIMQa6J/eb5R8Xp2AptJhtnw5DW5uUNiDODjzGP2ZYtaEPaUvbzkuF4vjd9a5DwrHVuZH+HCJhJh3SjGsjJb6Qp5ZLN4vBX0a3Uoj3hgAT+moDs0FL+mp6+OF0v7xSWakv2aYsFCv18SF7aaH0WRdQnx5VHAAse3zh3AnC6BD9djZWPO9iU1LEUq0qpCT2gHbfiE0H8Kjd+S/JhDuu5vSMzeKp+KpM+93tjANyRf+Dn4I6ePhhRM8BaV2uLnnXWbSCBA9OZ+X9eG2JTNXZU1HKlYOtaBZwkppa3xEcir6clVGR7zhzlXHSBEda7jrZWA2MGiVVkaV/PiFPYvyK8ISCcBYUCcJIaKmOwldFFHVmIg7LDPpNRI+/9sdSwNO/wkwcHV8zTptkHVU8nZ//2BRGlgSJ0ZokLB3R1BjIFJeITDKcdDFXJCe+xdDVP0GrZ8tGgkhRCYGSOXNDERN4QFeTRpRyN88aPpZRpMdiGwL6bC5DAjw21J8cUjA1vavcGQrZbJWzJUtWbcIsWUxSygONo9JslQrzFclJmFcn0j3WXCDr7+dwzepv60B55uu1vzpXfZ/sLDzUn+akxppVYIJfNCK2wyL+jMFhk/PPUU/sRAeiSrcziHoWqe4P1NJQ1f3CWaulCCuajUqe3cOY0jziDunvYNbbMgakcU3SidG5kjCHJZa5EJBZuyPI/vzNJRoxo+IvqW/THAp0vBGE/O+kuoYygx42SLy/BBglejEoksmGXsvFEn/x1S3ylSTUUJR/eJ+aaJPh46mTzWHC9u0TXdXHjI/KUoyf2lUPZnyPzbSh/mOnGZrsvhhOfHKon3+HtC71sgfYWx/32tPos1BliGmqz5o8pA/5psCle9f0LgA3Z4b73JwXsrzYCiTIWbuwUFAYqieFz0/9RUl41VG0GxIFv3yaEp1PbcOww7ne7T3IHYaz09P8iKGtvKh2Ibg5wbrH5l/CRDwNk0WDXaZ93Vmg1gN9KnpYDzXA3GLiH9pvEVbx5aYFH5jlRawZyiUWRSlaZp6B7iTElVITfx09aLnHEOoIqgFbLbg6e3G9gjLMUCcZnxWL+ZmDDvXi X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:28:58.8333 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 43d8f4c3-424a-481f-0742-08dca4f39c85 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF000023DA.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7793 From: Alejandro Lucero Region creation involves finding available DPA (device-physical-address) capacity to map into HPA (host-physical-address) space. Given the HPA capacity constraint, define an API, cxl_request_dpa(), that has the flexibility to map the minimum amount of memory the driver needs to operate vs the total possible that can be mapped given HPA availability. Factor out the core of cxl_dpa_alloc, that does free space scanning, into a cxl_dpa_freespace() helper, and use that to balance the capacity available to map vs the @min and @max arguments to cxl_request_dpa. Based on https://lore.kernel.org/linux-cxl/168592149709.1948938.8663425987110396027.stgit@dwillia2-xfh.jf.intel.com/T/#m4271ee49a91615c8af54e3ab20679f8be3099393 Signed-off-by: Alejandro Lucero Co-developed-by: Dan Williams --- drivers/cxl/core/core.h | 1 + drivers/cxl/core/hdm.c | 153 +++++++++++++++++++++++++---- drivers/net/ethernet/sfc/efx.c | 2 + drivers/net/ethernet/sfc/efx_cxl.c | 18 +++- drivers/net/ethernet/sfc/efx_cxl.h | 1 + include/linux/cxl_accel_mem.h | 7 ++ 6 files changed, 161 insertions(+), 21 deletions(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 625394486459..a243ff12c0f4 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -76,6 +76,7 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled, enum cxl_decoder_mode mode); int cxl_dpa_alloc(struct cxl_endpoint_decoder *cxled, unsigned long long size); int cxl_dpa_free(struct cxl_endpoint_decoder *cxled); +int cxl_dpa_free(struct cxl_endpoint_decoder *cxled); resource_size_t cxl_dpa_size(struct cxl_endpoint_decoder *cxled); resource_size_t cxl_dpa_resource_start(struct cxl_endpoint_decoder *cxled); diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 4af9225d4b59..3e53ae222d40 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -3,6 +3,7 @@ #include #include #include +#include #include "cxlmem.h" #include "core.h" @@ -420,6 +421,7 @@ int cxl_dpa_free(struct cxl_endpoint_decoder *cxled) up_write(&cxl_dpa_rwsem); return rc; } +EXPORT_SYMBOL_NS_GPL(cxl_dpa_free, CXL); int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled, enum cxl_decoder_mode mode) @@ -467,30 +469,17 @@ int cxl_dpa_set_mode(struct cxl_endpoint_decoder *cxled, return rc; } -int cxl_dpa_alloc(struct cxl_endpoint_decoder *cxled, unsigned long long size) +static resource_size_t cxl_dpa_freespace(struct cxl_endpoint_decoder *cxled, + resource_size_t *start_out, + resource_size_t *skip_out) { struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); resource_size_t free_ram_start, free_pmem_start; - struct cxl_port *port = cxled_to_port(cxled); struct cxl_dev_state *cxlds = cxlmd->cxlds; - struct device *dev = &cxled->cxld.dev; resource_size_t start, avail, skip; struct resource *p, *last; - int rc; - - down_write(&cxl_dpa_rwsem); - if (cxled->cxld.region) { - dev_dbg(dev, "decoder attached to %s\n", - dev_name(&cxled->cxld.region->dev)); - rc = -EBUSY; - goto out; - } - if (cxled->cxld.flags & CXL_DECODER_F_ENABLE) { - dev_dbg(dev, "decoder enabled\n"); - rc = -EBUSY; - goto out; - } + lockdep_assert_held(&cxl_dpa_rwsem); for (p = cxlds->ram_res.child, last = NULL; p; p = p->sibling) last = p; @@ -528,14 +517,45 @@ int cxl_dpa_alloc(struct cxl_endpoint_decoder *cxled, unsigned long long size) skip_end = start - 1; skip = skip_end - skip_start + 1; } else { - dev_dbg(dev, "mode not set\n"); - rc = -EINVAL; + avail = 0; + } + + if (!avail) + return 0; + if (start_out) + *start_out = start; + if (skip_out) + *skip_out = skip; + return avail; +} + +int cxl_dpa_alloc(struct cxl_endpoint_decoder *cxled, unsigned long long size) +{ + struct cxl_port *port = cxled_to_port(cxled); + struct device *dev = &cxled->cxld.dev; + resource_size_t start, avail, skip; + int rc; + + down_write(&cxl_dpa_rwsem); + if (cxled->cxld.region) { + dev_dbg(dev, "EBUSY, decoder attached to %s\n", + dev_name(&cxled->cxld.region->dev)); + rc = -EBUSY; goto out; } + if (cxled->cxld.flags & CXL_DECODER_F_ENABLE) { + dev_dbg(dev, "EBUSY, decoder enabled\n"); + rc = -EBUSY; + goto out; + } + + avail = cxl_dpa_freespace(cxled, &start, &skip); + if (size > avail) { dev_dbg(dev, "%pa exceeds available %s capacity: %pa\n", &size, - cxl_decoder_mode_name(cxled->mode), &avail); + cxled->mode == CXL_DECODER_RAM ? "ram" : "pmem", + &avail); rc = -ENOSPC; goto out; } @@ -550,6 +570,99 @@ int cxl_dpa_alloc(struct cxl_endpoint_decoder *cxled, unsigned long long size) return devm_add_action_or_reset(&port->dev, cxl_dpa_release, cxled); } +static int find_free_decoder(struct device *dev, void *data) +{ + struct cxl_endpoint_decoder *cxled; + struct cxl_port *port; + + if (!is_endpoint_decoder(dev)) + return 0; + + cxled = to_cxl_endpoint_decoder(dev); + port = cxled_to_port(cxled); + + if (cxled->cxld.id != port->hdm_end + 1) { + return 0; + } + return 1; +} + +/** + * cxl_request_dpa - search and reserve DPA given input constraints + * @endpoint: an endpoint port with available decoders + * @mode: DPA operation mode (ram vs pmem) + * @min: the minimum amount of capacity the call needs + * @max: extra capacity to allocate after min is satisfied + * + * Given that a region needs to allocate from limited HPA capacity it + * may be the case that a device has more mappable DPA capacity than + * available HPA. So, the expectation is that @min is a driver known + * value for how much capacity is needed, and @max is based the limit of + * how much HPA space is available for a new region. + * + * Returns a pinned cxl_decoder with at least @min bytes of capacity + * reserved, or an error pointer. The caller is also expected to own the + * lifetime of the memdev registration associated with the endpoint to + * pin the decoder registered as well. + */ +struct cxl_endpoint_decoder *cxl_request_dpa(struct cxl_port *endpoint, + bool is_ram, + resource_size_t min, + resource_size_t max) +{ + struct cxl_endpoint_decoder *cxled; + enum cxl_decoder_mode mode; + struct device *cxled_dev; + resource_size_t alloc; + int rc; + + if (!IS_ALIGNED(min | max, SZ_256M)) + return ERR_PTR(-EINVAL); + + down_read(&cxl_dpa_rwsem); + + cxled_dev = device_find_child(&endpoint->dev, NULL, find_free_decoder); + if (!cxled_dev) + cxled = ERR_PTR(-ENXIO); + else + cxled = to_cxl_endpoint_decoder(cxled_dev); + + up_read(&cxl_dpa_rwsem); + + if (IS_ERR(cxled)) + return cxled; + + if (is_ram) + mode = CXL_DECODER_RAM; + else + mode = CXL_DECODER_PMEM; + + rc = cxl_dpa_set_mode(cxled, mode); + if (rc) + goto err; + + down_read(&cxl_dpa_rwsem); + alloc = cxl_dpa_freespace(cxled, NULL, NULL); + up_read(&cxl_dpa_rwsem); + + if (max) + alloc = min(max, alloc); + if (alloc < min) { + rc = -ENOMEM; + goto err; + } + + rc = cxl_dpa_alloc(cxled, alloc); + if (rc) + goto err; + + return cxled; +err: + put_device(cxled_dev); + return ERR_PTR(rc); +} +EXPORT_SYMBOL_NS_GPL(cxl_request_dpa, CXL); + static void cxld_set_interleave(struct cxl_decoder *cxld, u32 *ctrl) { u16 eig; diff --git a/drivers/net/ethernet/sfc/efx.c b/drivers/net/ethernet/sfc/efx.c index cb3f74d30852..9cfe29002d98 100644 --- a/drivers/net/ethernet/sfc/efx.c +++ b/drivers/net/ethernet/sfc/efx.c @@ -901,6 +901,8 @@ static void efx_pci_remove(struct pci_dev *pci_dev) efx_fini_io(efx); + efx_cxl_exit(efx); + pci_dbg(efx->pci_dev, "shutdown successful\n"); efx_fini_devlink_and_unlock(efx); diff --git a/drivers/net/ethernet/sfc/efx_cxl.c b/drivers/net/ethernet/sfc/efx_cxl.c index 6d49571ccff7..b5626d724b52 100644 --- a/drivers/net/ethernet/sfc/efx_cxl.c +++ b/drivers/net/ethernet/sfc/efx_cxl.c @@ -84,12 +84,28 @@ void efx_cxl_init(struct efx_nic *efx) goto out; } - if (max < EFX_CTPIO_BUFFER_SIZE) + if (max < EFX_CTPIO_BUFFER_SIZE) { pci_info(pci_dev, "CXL accel not enough free HPA space %llu < %u\n", max, EFX_CTPIO_BUFFER_SIZE); + goto out; + } + + cxl->cxled = cxl_request_dpa(cxl->endpoint, true, EFX_CTPIO_BUFFER_SIZE, + EFX_CTPIO_BUFFER_SIZE); + if (IS_ERR(cxl->cxled)) + pci_info(pci_dev, "CXL accel request DPA failed"); out: cxl_release_endpoint(cxl->cxlmd, cxl->endpoint); } +void efx_cxl_exit(struct efx_nic *efx) +{ + struct efx_cxl *cxl = efx->cxl; + + if (cxl->cxled) + cxl_dpa_free(cxl->cxled); + + return; + } MODULE_IMPORT_NS(CXL); diff --git a/drivers/net/ethernet/sfc/efx_cxl.h b/drivers/net/ethernet/sfc/efx_cxl.h index 76c6794c20d8..59d5217a684c 100644 --- a/drivers/net/ethernet/sfc/efx_cxl.h +++ b/drivers/net/ethernet/sfc/efx_cxl.h @@ -26,4 +26,5 @@ struct efx_cxl { }; void efx_cxl_init(struct efx_nic *efx); +void efx_cxl_exit(struct efx_nic *efx); #endif diff --git a/include/linux/cxl_accel_mem.h b/include/linux/cxl_accel_mem.h index f3e77688ffe0..d4ecb5bb4fc8 100644 --- a/include/linux/cxl_accel_mem.h +++ b/include/linux/cxl_accel_mem.h @@ -2,6 +2,7 @@ /* Copyright(c) 2024 Advanced Micro Devices, Inc. */ #include +#include #ifndef __CXL_ACCEL_MEM_H #define __CXL_ACCEL_MEM_H @@ -41,4 +42,10 @@ struct cxl_root_decoder *cxl_get_hpa_freespace(struct cxl_port *endpoint, int interleave_ways, unsigned long flags, resource_size_t *max); + +struct cxl_endpoint_decoder *cxl_request_dpa(struct cxl_port *endpoint, + bool is_ram, + resource_size_t min, + resource_size_t max); +int cxl_dpa_free(struct cxl_endpoint_decoder *cxled); #endif From patchwork Mon Jul 15 17:28:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733706 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2089.outbound.protection.outlook.com [40.107.236.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B38CA446D1; Mon, 15 Jul 2024 17:29:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.236.89 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064545; cv=fail; b=eIp0Yoc/GuxTqTx5zxYhSDfaYnJfA+Z4ktcY+S2brfaxPg4bP/fK0bv0G1KxIGXsJtbXhaO9g7SEETD8oXzHPQt7w6brnd+tnnvM0DwbzcXkpJ2XWzFXNim/0fcZXwXejphq1CuZza8lv97isUx9BpUVBtB481O5VXcGt/SLbbk= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064545; c=relaxed/simple; bh=FOW6J+oQinweZYW2Y3V1Roo+XlJheeLT/W5x4VsppN8=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=j09qdy4v7MFM4TxhPDMQv6L1FMLRxTVN1adV4CVv/wU4l/8VD+7jZy8Ofd8TJp4N4X+M71RwB2Hd1nPHZ5lCHMQd/K+nyRx4x2sayl96zBz3jIGAfL2mlsDblBG6Q/hXzgWdXAFkEoWlav7d6ZchAMFaNEP1Ca6+BMQO2WybMYc= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=IRsVo9Cr; arc=fail smtp.client-ip=40.107.236.89 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="IRsVo9Cr" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gdBWpCKSDQ2PzVkhlp8SEEtgtUL6cuOkk1ajcGMQLLC+QUql7vHgCQ6er4WpJTqwKNMiTsPrRQZ1OrQ4Wz7fg8FgeerfelV2nD3sLbSwwfLSWwt3HJ8ncU9Kjqzuw4qI7GozBff7PW8W5M4ik+BsLRf9cEBVPKT8vfeFNI72JjKPsH8EZiiYvXg3nRPrvmm8h4yGe5EK4Hv7qpZHpLnz6T0wOLbyzuylmGLlYwYXrrELpMfMJmSYfEuYCSV61QIDXkvqO5/mZnDlIV90KvTl7z6NRclRZWMLP+L2v18vNj9IinVIDy90kk+PDggIXgAIfoX4ju/bw/U39mt1s43W4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6htK/OoKboLNfKt5yS2v8d8VTCpFP6cnDFWCkr4RKAo=; b=PUTCogWtz8pnaOUNGdRJ/Sm9/KedngR/neWGC6d33qW6to6fMlvtqzfwjyiRr1KRdkVTWkfd3yrUDohY1akP4N17nZgIUMBgk4wkkasYosVSgvhNMKi/+gCglIblH9JmMmuKJ6Sd+J/5gGlZDLowZls04F/0YrLuQgh1Y4GlrY8zqu/hnVg/CVhSCH/TsCm9BO89fblc0EoUCE/MDuzoxQmcdpl8rUmLeaR/wDeaZnNSbxuc+pO+FQdNKmexmJ6hUdbBsS26mQGq3gDvZDSBqXd/d3cEcjN3mpFr9SSp5b6dYY+jo/th5Ya1b7Mp377cWzyWO0kQ1vjXBw0eMyLIpw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6htK/OoKboLNfKt5yS2v8d8VTCpFP6cnDFWCkr4RKAo=; b=IRsVo9CrVNsZh6tvBHLUWI8eugZpVm2PGVuPaYY44u45DLJLTy4cZGvFcci/Co6bhSwNjyOCyMXeh+ITB02PlGgIU645thiPTxwu5RX2Y9acyRI+fkZRfdzhQJqXsFIK+OHjuAp4OLPa07l1bCyXvXiYGPIEbFIjkhGpXwWWqt0= Received: from CY8PR12CA0044.namprd12.prod.outlook.com (2603:10b6:930:49::10) by DS0PR12MB8342.namprd12.prod.outlook.com (2603:10b6:8:f9::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.29; Mon, 15 Jul 2024 17:29:00 +0000 Received: from CY4PEPF0000EDD7.namprd03.prod.outlook.com (2603:10b6:930:49:cafe::7b) by CY8PR12CA0044.outlook.office365.com (2603:10b6:930:49::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.29 via Frontend Transport; Mon, 15 Jul 2024 17:29:00 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EDD7.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.11 via Frontend Transport; Mon, 15 Jul 2024 17:28:59 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:28:59 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:28:57 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero Subject: [PATCH v2 11/15] cxl: make region type based on endpoint type Date: Mon, 15 Jul 2024 18:28:31 +0100 Message-ID: <20240715172835.24757-12-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received-SPF: None (SATLEXMB04.amd.com: alejandro.lucero-palau@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD7:EE_|DS0PR12MB8342:EE_ X-MS-Office365-Filtering-Correlation-Id: 224f4914-48d6-4966-09ee-08dca4f39d21 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|36860700013|82310400026|921020; X-Microsoft-Antispam-Message-Info: q4z2NQyT/YXkftEo6LAGc/zB7cQfQr+WU57LOrar8E4iLPXd980nAiobm2fm4Wmn/6H2xM0wCpfkhtDUHgDRrpXMjt260Q11sSa06dZWdheqE/9e0lA02YTyAQIAnomUeY9IENz61ITl5udFyMgy3mUR2nN4yGoHN6ht9JIpq6HSsACcOTthEmIS3fvHRCDlitgHu+cOM9UgKn4G8xBoM4khpEh69GYxYz00CtZa3REU7yO1dYB7xGbGTuQdnK604CbO4Oz0SX8mFUCi/VlUCQHWprgPpqXUX63NE4xKibamnpRbg8NcvqrYJ0CdE0Fk8jOgS62oDmQBpgYhbvH6Q9XNPsW8CfpMqyOaMTPk9VfStoei0ok53SSD6mch/AD+V9rFAZFGQjIAton/5euCoJYXOBLyFYOc4xhmTvpyh3oc77U28mA330UL1lDa3SmZHo6Ng/oTochaPrnfwry2DyOdjsUCzhCxnJYnk0pmQW7EXIWwUeDaToKGr8fiD5dKUrjZJDetno6XehdMQZ0HQzSm9Dr9N2bnJSoAVyJWVePVrCk+0g+YarAj+G9TpAe/pdmyGVm42QQwLODB67+rA9g/GBMprtpb5zG20suWHUqcfYKFr59Y2bRDLoXXGbDlhewS8uRqYmcuRADmleyUhdqvDGBlNkPYlMlj9kDuuMVuqmlxZFp0E9+xgM1ZAKCxzy6lMmt3Ml3VwJk4UdlmVzkqiLz/cm14KRFC/LuHPZI97K2+xRxQi1FF0br9IRPU+9ezQMbXXBD3fBWrMP/qddk+2oyyTVm6vJvCuY9d3kXvcyn59ZnldU6ekoD21fm+IWTVX64J1LG6JM8GbFcXs0lOM3z5THAWBgZBcJ0ugxwjpDdboEG0HtJH8Z2i8gGRzzYBZ8INoGGhIcWpqwb5P4vJUK1iPfNzW4SThw5hc3TsZZ24RfKYsK3WgpgZuV8Hs1GX9z6G8w/GFk8cNPjK1R04k58M+pcJKtdZJaGFQO1Zoy2hWSGwZsjVgVOEwjKh1kUcZhmj+HrpbVdguKADj//UuJt+e59EtmDXGTCIzy8ADRmEX6i/VGaFM9Uh92KBVaVoy4TDGjqCDT8GOHP+RSQc/AefV2Lsert82QO93vz5nh+/8iD8bz/7Zbx41bbaBDRPIn42W4YW8gM+7Pi9OZWjTb5PMD9cAl/OMazwuEHP9ZMO2jebwk7MtJBhEl/+m1FtjAt1wdG1kjoH9u8suOsK2J9dIkQon8x6Hbs/dMo/sefaGvO7+ov4x1Myvjf2ND8wKfB9SNkD5YwYfcOZVQ5ssZl32GXn43wIW9/tCvAgemtISaQuPj3RavReI2MIy2q/kDTnU5xR48VHPtPbdDJG+O21SoOYP1Bup4hrryhQ4LbQwBcHgYki4VXegduESLZ7UXxCCdqD/TqU+OOaWWfHBGL0nKCGfk3P5PodDoiaGVbqogq4XN5BvPvLIdSepfFT085wM27UFoorYA1cAA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:28:59.8714 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 224f4914-48d6-4966-09ee-08dca4f39d21 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD7.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8342 From: Alejandro Lucero Current code is expecting Type3 or CXL_DECODER_HOSTONLYMEM devices only. Suport for Type2 implies region type needs to be based on the endpoint type instead. Signed-off-by: Alejandro Lucero --- drivers/cxl/core/region.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index ca464bfef77b..5cc71b8868bc 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -2645,7 +2645,8 @@ static ssize_t create_ram_region_show(struct device *dev, } static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd, - enum cxl_decoder_mode mode, int id) + enum cxl_decoder_mode mode, int id, + enum cxl_decoder_type target_type) { int rc; @@ -2667,7 +2668,7 @@ static struct cxl_region *__create_region(struct cxl_root_decoder *cxlrd, return ERR_PTR(-EBUSY); } - return devm_cxl_add_region(cxlrd, id, mode, CXL_DECODER_HOSTONLYMEM); + return devm_cxl_add_region(cxlrd, id, mode, target_type); } static ssize_t create_pmem_region_store(struct device *dev, @@ -2682,7 +2683,8 @@ static ssize_t create_pmem_region_store(struct device *dev, if (rc != 1) return -EINVAL; - cxlr = __create_region(cxlrd, CXL_DECODER_PMEM, id); + cxlr = __create_region(cxlrd, CXL_DECODER_PMEM, id, + CXL_DECODER_HOSTONLYMEM); if (IS_ERR(cxlr)) return PTR_ERR(cxlr); @@ -2702,7 +2704,8 @@ static ssize_t create_ram_region_store(struct device *dev, if (rc != 1) return -EINVAL; - cxlr = __create_region(cxlrd, CXL_DECODER_RAM, id); + cxlr = __create_region(cxlrd, CXL_DECODER_RAM, id, + CXL_DECODER_HOSTONLYMEM); if (IS_ERR(cxlr)) return PTR_ERR(cxlr); @@ -3364,7 +3367,8 @@ static struct cxl_region *construct_region(struct cxl_root_decoder *cxlrd, do { cxlr = __create_region(cxlrd, cxled->mode, - atomic_read(&cxlrd->region_id)); + atomic_read(&cxlrd->region_id), + cxled->cxld.target_type); } while (IS_ERR(cxlr) && PTR_ERR(cxlr) == -EBUSY); if (IS_ERR(cxlr)) { From patchwork Mon Jul 15 17:28:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733709 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2054.outbound.protection.outlook.com [40.107.223.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BDE3E4D8AE; Mon, 15 Jul 2024 17:29:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.54 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064552; cv=fail; b=YFTFakeBsLfcXJ8BPoaLa5VJsj42FjLwmVTxLJ3toC/VzYTDvbyoNlRtYbWlCIqgaujdlNllOdSU+I51L/fnKbyKmqRXCGoQL0njS1wDBCBfv4WHlrMycLw39O+cfFsBtA2l9+sviW2I8oQXFAYoZX0FX5YORzB9M93PUFO/rZY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064552; c=relaxed/simple; bh=Pcx01+JoQy6MNS1OD9L/ItqN/D6WPwR+kVqAFX56bI0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ewBU6gUwCEjhO5mzIl2XByN2VxCLuC8YnmNy40lVHuaub9TFbX6oQ0qRMH3Epn9PRjhedNYUmLJb+J7et7b/6xjrMdqcn9f3dOKMJQFbsCqZfzpboBQ8ypQljL+XoUsOujl82y1mwc6FqC8sGzuICtSxJ8oDhbJsnXe8w1Lgd0A= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=463E++Es; arc=fail smtp.client-ip=40.107.223.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="463E++Es" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dtnkmjIUY5nM4yjrt+JqYVNesP6la7hDx/Pv0DvCcQ0hQKKwCzN5VDnWzIdJqrn+PjGJEUbUEqW7blwfIlJN8OIdKWvcY361HsvCmHC+rc5KrJdaGVN4Uy1bWcpFtR4YdvyYl/bb7f5xYOIXvs9e8aEJHQ4NMPR9EmWY+7bBDFmEgg5j05WUjQIdtVdGGmPWY6IBp+Obbtm/eBk+KDqJ+Bw3slCTmYdbQgk/PH+X0SmE0mWh5KwPzAvGQoXh7CYAssC30qm2BUbaYIF3sAIFYFL0NrGcoNpV3JcadNPfdowekMqvqkcMj+2ktHzxl5N0Gsvht23CJNziqhn2mQ9qNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cgbzVYdE6aZLPnVX/enqasFlmXjMj4PxplaAW0wb364=; b=HJEXDJThtBYdTU2Mbwp/xGv/1s8DzC1dOze78UjPsQZC7VlH7zwLq7NG1BqIsFwz81vtKIxHb9yD0RdG+RvEPJ2Iam7+rY0xezKcA0xk/Jvgk/foV02LbrH26mDTPGr0tm6vltUprPfsE3o5OuxW+DagrYLrDCEhX7QcxeAb2mZAu0VBi6edRmoCHYc4n+Vc7EoCQz/m/v6k142BsHCdMx/oFxwVokGcqMoO6jhyqWsWkIM1wgXVCWptCTlo94jCDQXSvzEwYxwC3zPxIGbxXvIF0xxdV9YC72xVgEHZim5IxQv5+9dZc5FguPu2lsmUlcw3vsOTtlZNkpgOms+6Ug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cgbzVYdE6aZLPnVX/enqasFlmXjMj4PxplaAW0wb364=; b=463E++EsLX8O5d+JbBLSwhAhDu+L1GGHJtBmw+50uKDSk344uNuOeO6Sg8xyYcBRFri51w/bn0OqslwXo335fwTbJc72jSachQZ3lMAQ0dQCcrpyA1yWJMI/mtnq53B6O+jrY5yv3XeI+3XoX1zwL5Nm46cEdklW6/Y7yo4+3kI= Received: from CY8PR12CA0042.namprd12.prod.outlook.com (2603:10b6:930:49::21) by SA1PR12MB6775.namprd12.prod.outlook.com (2603:10b6:806:25a::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.36; Mon, 15 Jul 2024 17:29:05 +0000 Received: from CY4PEPF0000EDD7.namprd03.prod.outlook.com (2603:10b6:930:49:cafe::b) by CY8PR12CA0042.outlook.office365.com (2603:10b6:930:49::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.29 via Frontend Transport; Mon, 15 Jul 2024 17:29:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EDD7.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.11 via Frontend Transport; Mon, 15 Jul 2024 17:29:02 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:29:00 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:28:59 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero Subject: [PATCH v2 12/15] cxl: allow region creation by type2 drivers Date: Mon, 15 Jul 2024 18:28:32 +0100 Message-ID: <20240715172835.24757-13-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received-SPF: None (SATLEXMB04.amd.com: alejandro.lucero-palau@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD7:EE_|SA1PR12MB6775:EE_ X-MS-Office365-Filtering-Correlation-Id: 61351246-6a4a-4e1e-d30d-08dca4f39e6e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|1800799024|82310400026|376014|921020; X-Microsoft-Antispam-Message-Info: 6GBtQwBt2irAOf8Jey778/OFrw7qQRA7wAuvlzYckqD1Cu6XnG6l8uOkkvI1ns8kz00ro9f31kuonr1KhkAKd2RIg1swpKm5x5ZTRrhKDXRBQSo6phBKBee8Inm61OiFAwwqo8jQR0yewGYc46asMtAcobA8nBTHy2ajaWrhknAIJegUBdF2ZVbH1T3JkfyUKiT5ZUlZ8liSEkXYTp0Jx8tyrqP6ivXPyYMwvhF1w9019cOc39IR+cugrrL50KfT0MFXpuinqH1M4q3N5FAv9Yc2FQneVU2jih+7/u+1yrp0M6Ep+aBggm801TqoNi0R/D73FPmjIUl/2tStSnfV5iDTHR+ym1EBLukl1qfgyes+88CE6ZQB5fmCLchUDsDPxu5fLRSxorU7ZSherVo3JNtioEn8662wDXz+IsLP4d8W6FROmxciiFM3BRg7EMt/ZTwIb3gWKUHaciBChT43c0943TlqUb6DahZ0h8Kvw2tJhq+hG6j01hSNjp6zdOMhrKWWyV4godmE7tyCfDhuIpI52kTUvJDBXd80LBjpsmTeVyp43xrXccvhSlqP20MKCl5mptp8p4PNqXDDJME6uuDXTeN4qestG1sUJln/kqYQw2onhdR2DhHOZWkErcoyQ+YVtSLEL4Tq4CVyxcw02B2QefOQiJM/lCkTNGOntB2aG1YWReNA3n4qMIPy9ftiib0fYwd+hbEefGhi+J+bbg//cKSSkR4XlMSGyu4cBu2uTI06gYKTLkLNETPHrdXuBHdkEca9ou75Q8X7RVJU6ZJal138mixsutXBU6dtLMeNwlYw3vJeFgLIVeRgV1C+SuKLY1K7tL+FiW6Ge24eL7Eq96xZes6cLkauDAVsgJMTK6Gpt17ZT9T4JEan1VkgCUUcbIfh597sz/v3XetafanpVamsgJbfj5TvF0Cn9mBdfEU6ImQJzvADYEYegq8HcMoWGuG6MNQbBVkUe69Ucvp3LfScZH5+IcAFgt1LJf+04PpKwJ63GYZZfil8EmTsrs876p76AtDpc7kFKRyydtpETcz+TUqQneUpSyuyIgGSh+KZ5GsGlYh7V3pUp3HNM8r1EtyxTWzw73Ep0iFBrcEa3wFpQo5TIPlzn+oP++dcB1gax4Jk+5WadVC6ikdHxXpE8IrXlknWw4B+RBRHjnl9e3861UX1KMSPM9LBhAQTec2W8tnpfqYLUophNX2fKUcuhBty8g5sbe0rzDr1FzQXG9mJx6ZrzRGe1oaVwKa74bRSFK1qiBK/YY5lUS0hZDTuPn/oZi8m5oHTB+vLgdKGXUAv1xJMdftp9FocVqjM8fIviPmOy6GsSEcY6Xr2053qOiWVTUBE+CTvz7vFfk2DoKEe9P9Z5XCqlnKjVjsvWDAiuoZorYzE89YnUrEPF9sE4jrgUaTyHIqKJiV27Q== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(1800799024)(82310400026)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:29:02.0589 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 61351246-6a4a-4e1e-d30d-08dca4f39e6e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD7.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6775 From: Alejandro Lucero Creating a CXL region requires userspace intervention through the cxl sysfs files. Type2 support should allow accelerator drivers to create such cxl region from kernel code. Adding that functionality and integrating it with current support for memory expanders. Based on https://lore.kernel.org/linux-cxl/168592149709.1948938.8663425987110396027.stgit@dwillia2-xfh.jf.intel.com/T/#m84598b534cc5664f5bb31521ba6e41c7bc213758 Signed-off-by: Alejandro Lucero Signed-off-by: Dan Williams --- drivers/cxl/core/region.c | 265 ++++++++++++++++++++++------- drivers/cxl/cxl.h | 1 + drivers/cxl/cxlmem.h | 4 +- drivers/net/ethernet/sfc/efx_cxl.c | 15 +- include/linux/cxl_accel_mem.h | 5 + 5 files changed, 231 insertions(+), 59 deletions(-) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 5cc71b8868bc..697c8df83a4b 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -479,22 +479,14 @@ static ssize_t interleave_ways_show(struct device *dev, static const struct attribute_group *get_cxl_region_target_group(void); -static ssize_t interleave_ways_store(struct device *dev, - struct device_attribute *attr, - const char *buf, size_t len) +static int set_interleave_ways(struct cxl_region *cxlr, int val) { - struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev->parent); + struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(cxlr->dev.parent); struct cxl_decoder *cxld = &cxlrd->cxlsd.cxld; - struct cxl_region *cxlr = to_cxl_region(dev); struct cxl_region_params *p = &cxlr->params; - unsigned int val, save; - int rc; + int save, rc; u8 iw; - rc = kstrtouint(buf, 0, &val); - if (rc) - return rc; - rc = ways_to_eiw(val, &iw); if (rc) return rc; @@ -509,25 +501,42 @@ static ssize_t interleave_ways_store(struct device *dev, return -EINVAL; } - rc = down_write_killable(&cxl_region_rwsem); - if (rc) - return rc; - if (p->state >= CXL_CONFIG_INTERLEAVE_ACTIVE) { - rc = -EBUSY; - goto out; - } + lockdep_assert_held_write(&cxl_region_rwsem); + if (p->state >= CXL_CONFIG_INTERLEAVE_ACTIVE) + return -EBUSY; save = p->interleave_ways; p->interleave_ways = val; rc = sysfs_update_group(&cxlr->dev.kobj, get_cxl_region_target_group()); if (rc) p->interleave_ways = save; -out: + + return rc; +} + +static ssize_t interleave_ways_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_region *cxlr = to_cxl_region(dev); + unsigned int val; + int rc; + + rc = kstrtouint(buf, 0, &val); + if (rc) + return rc; + + rc = down_write_killable(&cxl_region_rwsem); + if (rc) + return rc; + + rc = set_interleave_ways(cxlr, val); up_write(&cxl_region_rwsem); if (rc) return rc; return len; } + static DEVICE_ATTR_RW(interleave_ways); static ssize_t interleave_granularity_show(struct device *dev, @@ -547,21 +556,14 @@ static ssize_t interleave_granularity_show(struct device *dev, return rc; } -static ssize_t interleave_granularity_store(struct device *dev, - struct device_attribute *attr, - const char *buf, size_t len) +static int set_interleave_granularity(struct cxl_region *cxlr, int val) { - struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(dev->parent); + struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(cxlr->dev.parent); struct cxl_decoder *cxld = &cxlrd->cxlsd.cxld; - struct cxl_region *cxlr = to_cxl_region(dev); struct cxl_region_params *p = &cxlr->params; - int rc, val; + int rc; u16 ig; - rc = kstrtoint(buf, 0, &val); - if (rc) - return rc; - rc = granularity_to_eig(val, &ig); if (rc) return rc; @@ -577,21 +579,36 @@ static ssize_t interleave_granularity_store(struct device *dev, if (cxld->interleave_ways > 1 && val != cxld->interleave_granularity) return -EINVAL; + lockdep_assert_held_write(&cxl_region_rwsem); + if (p->state >= CXL_CONFIG_INTERLEAVE_ACTIVE) + return -EBUSY; + + p->interleave_granularity = val; + return 0; +} + +static ssize_t interleave_granularity_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_region *cxlr = to_cxl_region(dev); + int rc, val; + + rc = kstrtoint(buf, 0, &val); + if (rc) + return rc; + rc = down_write_killable(&cxl_region_rwsem); if (rc) return rc; - if (p->state >= CXL_CONFIG_INTERLEAVE_ACTIVE) { - rc = -EBUSY; - goto out; - } - p->interleave_granularity = val; -out: + rc = set_interleave_granularity(cxlr, val); up_write(&cxl_region_rwsem); if (rc) return rc; return len; } + static DEVICE_ATTR_RW(interleave_granularity); static ssize_t resource_show(struct device *dev, struct device_attribute *attr, @@ -2193,7 +2210,7 @@ static int cxl_region_attach(struct cxl_region *cxlr, return 0; } -static int cxl_region_detach(struct cxl_endpoint_decoder *cxled) +int cxl_region_detach(struct cxl_endpoint_decoder *cxled) { struct cxl_port *iter, *ep_port = cxled_to_port(cxled); struct cxl_region *cxlr = cxled->cxld.region; @@ -2252,6 +2269,7 @@ static int cxl_region_detach(struct cxl_endpoint_decoder *cxled) put_device(&cxlr->dev); return rc; } +EXPORT_SYMBOL_NS_GPL(cxl_region_detach, CXL); void cxl_decoder_kill_region(struct cxl_endpoint_decoder *cxled) { @@ -2746,6 +2764,14 @@ cxl_find_region_by_name(struct cxl_root_decoder *cxlrd, const char *name) return to_cxl_region(region_dev); } +static void drop_region(struct cxl_region *cxlr) +{ + struct cxl_root_decoder *cxlrd = to_cxl_root_decoder(cxlr->dev.parent); + struct cxl_port *port = cxlrd_to_port(cxlrd); + + devm_release_action(port->uport_dev, unregister_region, cxlr); +} + static ssize_t delete_region_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t len) @@ -3353,17 +3379,18 @@ static int match_region_by_range(struct device *dev, void *data) return rc; } -/* Establish an empty region covering the given HPA range */ -static struct cxl_region *construct_region(struct cxl_root_decoder *cxlrd, - struct cxl_endpoint_decoder *cxled) +static void construct_region_end(void) +{ + up_write(&cxl_region_rwsem); +} + +static struct cxl_region *construct_region_begin(struct cxl_root_decoder *cxlrd, + struct cxl_endpoint_decoder *cxled) { struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); - struct cxl_port *port = cxlrd_to_port(cxlrd); - struct range *hpa = &cxled->cxld.hpa_range; struct cxl_region_params *p; struct cxl_region *cxlr; - struct resource *res; - int rc; + int err = 0; do { cxlr = __create_region(cxlrd, cxled->mode, @@ -3372,8 +3399,7 @@ static struct cxl_region *construct_region(struct cxl_root_decoder *cxlrd, } while (IS_ERR(cxlr) && PTR_ERR(cxlr) == -EBUSY); if (IS_ERR(cxlr)) { - dev_err(cxlmd->dev.parent, - "%s:%s: %s failed assign region: %ld\n", + dev_err(cxlmd->dev.parent,"%s:%s: %s failed assign region: %ld\n", dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), __func__, PTR_ERR(cxlr)); return cxlr; @@ -3383,23 +3409,47 @@ static struct cxl_region *construct_region(struct cxl_root_decoder *cxlrd, p = &cxlr->params; if (p->state >= CXL_CONFIG_INTERLEAVE_ACTIVE) { dev_err(cxlmd->dev.parent, - "%s:%s: %s autodiscovery interrupted\n", + "%s:%s: %s region setup interrupted\n", dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), __func__); - rc = -EBUSY; - goto err; + err = -EBUSY; + } + + if (err) { + construct_region_end(); + drop_region(cxlr); + return ERR_PTR(err); } + return cxlr; +} + + +/* Establish an empty region covering the given HPA range */ +static struct cxl_region *construct_region(struct cxl_root_decoder *cxlrd, + struct cxl_endpoint_decoder *cxled) +{ + struct cxl_memdev *cxlmd = cxled_to_memdev(cxled); + struct range *hpa = &cxled->cxld.hpa_range; + struct cxl_region_params *p; + struct cxl_region *cxlr; + struct resource *res; + int rc; + + cxlr = construct_region_begin(cxlrd, cxled); + if (IS_ERR(cxlr)) + return cxlr; set_bit(CXL_REGION_F_AUTO, &cxlr->flags); res = kmalloc(sizeof(*res), GFP_KERNEL); if (!res) { rc = -ENOMEM; - goto err; + goto out; } *res = DEFINE_RES_MEM_NAMED(hpa->start, range_len(hpa), dev_name(&cxlr->dev)); + rc = insert_resource(cxlrd->res, res); if (rc) { /* @@ -3412,6 +3462,7 @@ static struct cxl_region *construct_region(struct cxl_root_decoder *cxlrd, __func__, dev_name(&cxlr->dev)); } + p = &cxlr->params; p->res = res; p->interleave_ways = cxled->cxld.interleave_ways; p->interleave_granularity = cxled->cxld.interleave_granularity; @@ -3419,24 +3470,124 @@ static struct cxl_region *construct_region(struct cxl_root_decoder *cxlrd, rc = sysfs_update_group(&cxlr->dev.kobj, get_cxl_region_target_group()); if (rc) - goto err; + goto out; dev_dbg(cxlmd->dev.parent, "%s:%s: %s %s res: %pr iw: %d ig: %d\n", - dev_name(&cxlmd->dev), dev_name(&cxled->cxld.dev), __func__, - dev_name(&cxlr->dev), p->res, p->interleave_ways, - p->interleave_granularity); + dev_name(&cxlmd->dev), + dev_name(&cxled->cxld.dev), __func__, + dev_name(&cxlr->dev), p->res, + p->interleave_ways, + p->interleave_granularity); /* ...to match put_device() in cxl_add_to_region() */ get_device(&cxlr->dev); up_write(&cxl_region_rwsem); +out: + construct_region_end(); + if (rc) { + drop_region(cxlr); + return ERR_PTR(rc); + } + return cxlr; +} + +static struct cxl_region * +__construct_new_region(struct cxl_root_decoder *cxlrd, + struct cxl_endpoint_decoder **cxled, int ways) +{ + struct cxl_decoder *cxld = &cxlrd->cxlsd.cxld; + struct cxl_region_params *p; + resource_size_t size = 0; + struct cxl_region *cxlr; + int rc, i; + + /* If interleaving is not supported, why does ways need to be at least 1? */ + if (ways < 1) + return ERR_PTR(-EINVAL); + + cxlr = construct_region_begin(cxlrd, cxled[0]); + if (IS_ERR(cxlr)) + return cxlr; + + rc = set_interleave_ways(cxlr, ways); + if (rc) + goto out; + + rc = set_interleave_granularity(cxlr, cxld->interleave_granularity); + if (rc) + goto out; + + down_read(&cxl_dpa_rwsem); + for (i = 0; i < ways; i++) { + if (!cxled[i]->dpa_res) + break; + size += resource_size(cxled[i]->dpa_res); + } + up_read(&cxl_dpa_rwsem); + + if (i < ways) + goto out; + + rc = alloc_hpa(cxlr, size); + if (rc) + goto out; + + down_read(&cxl_dpa_rwsem); + for (i = 0; i < ways; i++) { + rc = cxl_region_attach(cxlr, cxled[i], i); + if (rc) + break; + } + up_read(&cxl_dpa_rwsem); + + if (rc) + goto out; + + rc = cxl_region_decode_commit(cxlr); + if (rc) + goto out; + p = &cxlr->params; + p->state = CXL_CONFIG_COMMIT; +out: + construct_region_end(); + if (rc) { + drop_region(cxlr); + return ERR_PTR(rc); + } return cxlr; +} -err: - up_write(&cxl_region_rwsem); - devm_release_action(port->uport_dev, unregister_region, cxlr); - return ERR_PTR(rc); +/** + * cxl_create_region - Establish a region given an array of endpoint decoders + * @cxlrd: root decoder to allocate HPA + * @cxled: array of endpoint decoders with reserved DPA capacity + * @ways: size of @cxled array + * + * Returns a fully formed region in the commit state and attached to the + * cxl_region driver. + */ +struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd, + struct cxl_endpoint_decoder **cxled, + int ways) +{ + struct cxl_region *cxlr; + + mutex_lock(&cxlrd->range_lock); + cxlr = __construct_new_region(cxlrd, cxled, ways); + mutex_unlock(&cxlrd->range_lock); + + if (IS_ERR(cxlr)) + return cxlr; + + if (device_attach(&cxlr->dev) <= 0) { + dev_err(&cxlr->dev, "failed to create region\n"); + drop_region(cxlr); + return ERR_PTR(-ENODEV); + } + return cxlr; } +EXPORT_SYMBOL_NS_GPL(cxl_create_region, CXL); int cxl_add_to_region(struct cxl_port *root, struct cxl_endpoint_decoder *cxled) { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index d3fdd2c1e066..1bf3b74ff959 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -905,6 +905,7 @@ void cxl_coordinates_combine(struct access_coordinate *out, bool cxl_endpoint_decoder_reset_detected(struct cxl_port *port); +int cxl_region_detach(struct cxl_endpoint_decoder *cxled); /* * Unit test builds overrides this to __weak, find the 'strong' version * of these symbols in tools/testing/cxl/. diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index a0e0795ec064..377bb3cd2d47 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -881,5 +881,7 @@ struct cxl_root_decoder *cxl_get_hpa_freespace(struct cxl_port *endpoint, int interleave_ways, unsigned long flags, resource_size_t *max); - +struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd, + struct cxl_endpoint_decoder **cxled, + int ways); #endif /* __CXL_MEM_H__ */ diff --git a/drivers/net/ethernet/sfc/efx_cxl.c b/drivers/net/ethernet/sfc/efx_cxl.c index b5626d724b52..4012e3faa298 100644 --- a/drivers/net/ethernet/sfc/efx_cxl.c +++ b/drivers/net/ethernet/sfc/efx_cxl.c @@ -92,8 +92,18 @@ void efx_cxl_init(struct efx_nic *efx) cxl->cxled = cxl_request_dpa(cxl->endpoint, true, EFX_CTPIO_BUFFER_SIZE, EFX_CTPIO_BUFFER_SIZE); - if (IS_ERR(cxl->cxled)) + if (IS_ERR(cxl->cxled)) { pci_info(pci_dev, "CXL accel request DPA failed"); + return; + } + + cxl->efx_region = cxl_create_region(cxl->cxlrd, &cxl->cxled, 1); + if (!cxl->efx_region) { + pci_info(pci_dev, "CXL accel create region failed"); + cxl_dpa_free(cxl->cxled); + return; + } + out: cxl_release_endpoint(cxl->cxlmd, cxl->endpoint); } @@ -102,6 +112,9 @@ void efx_cxl_exit(struct efx_nic *efx) { struct efx_cxl *cxl = efx->cxl; + if (cxl->efx_region) + cxl_region_detach(cxl->cxled); + if (cxl->cxled) cxl_dpa_free(cxl->cxled); diff --git a/include/linux/cxl_accel_mem.h b/include/linux/cxl_accel_mem.h index d4ecb5bb4fc8..a5f9ffc24509 100644 --- a/include/linux/cxl_accel_mem.h +++ b/include/linux/cxl_accel_mem.h @@ -48,4 +48,9 @@ struct cxl_endpoint_decoder *cxl_request_dpa(struct cxl_port *endpoint, resource_size_t min, resource_size_t max); int cxl_dpa_free(struct cxl_endpoint_decoder *cxled); +struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd, + struct cxl_endpoint_decoder **cxled, + int ways); + +int cxl_region_detach(struct cxl_endpoint_decoder *cxled); #endif From patchwork Mon Jul 15 17:28:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733708 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2089.outbound.protection.outlook.com [40.107.101.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50C764D5BD; Mon, 15 Jul 2024 17:29:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.101.89 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064551; cv=fail; b=k5BLQbSXqvjh7vCkwH59EiazdOU/qlAnfTyzYB/flGHgUa4JwF98Q2AZrKdv8BsBlqT4y8mhdFB5WIiSl6ZpDv2c0bx3fnVhUQsloWwlv4xo8SGdLxp49k0Kw5D623HcPSGESOsVLxwL3ZkGcCcIFKeuuv16XLSdIpCR1m4o4CA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064551; c=relaxed/simple; bh=BakCz81xtmBSGDdcYNKcYSvgjSlGriobi6063kHj8mo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=trh2OTDQ9PrryDhBynzcYBrFOjfBomWj3F0PlkNlTu5t79TyFSr0yZtLhn2JYUzMc7sDZ8Sgyf0MOu3A/pGPyqPBxr+ZN4fZi8dGqQclkLJBz+dI8jqyhTbEBmL6YobKR92+WPf9joit8nORDIz8eTKHSRCSU9AlQXKH0omRhSw= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=4NOj26nT; arc=fail smtp.client-ip=40.107.101.89 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="4NOj26nT" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=LcOl9HVIjC/fsWVQva8HQAbA6DcmUqGNMj37oszPwFTbWTie3VMF369g3A0vA09QBQBkYNTdILqCZAXXCzltK3cMd2uw2ilN9wMuszH99t9KmtReBdh8Fb2TOjhVLQqnTd9PW2WKvBCU2jIV6IllbLFZQgeep3wArKSZXIl9TvEw2H713CjkqDfUCbuaJBKOPLwCEmxi08wKAFVjNVT7Y8fNMZ1iwq4Kehji/iJhcc5DypIaLh+WbYuWwC5eL2gJXyueLgrjVmtxOThordqSufQ7/cR2TRURiXFtqVLqtbmlJrArhs5sqzToLPUxr30wcN5zJ6YidyIQEp0uxrfMiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=N/Fw9yqq9m7TDHJciSAzr376wo3U1OJY+js4w4DOIRs=; b=VYjRjFtawXFuJ/YGOBqHw/Wh0f32IwHxdUtYs/nzuurmmglVRAO4qcgzjh+kPkDle/dthbVF/FuUxw6OdBrKwYUB/jOd0UksewQ5kIHW/JOEf18sMIjQMcuKOAWtImAakl7lkVFgHQMJpXDjXpPXysQodOunuvb28IjJoZ05rB8CESvsN4dvYbagB3XcA7lO42AdzBHIjSWlieX5nHRRY68eT0j+fhV8gg+jncDQyZ2Ub8YeZw6+zPYPHEYLqN6B13ER1PJBz5hhKPX0PDNKW0UtwilFuBHriKMrul3CuK66lmbKOaI5pASPyP9vA9m6kiseqFdph3wr/4dzQv+XCw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N/Fw9yqq9m7TDHJciSAzr376wo3U1OJY+js4w4DOIRs=; b=4NOj26nT+TqObBe8JSJuZXoeFeSARESEyGXTDI4ZaDxqZHRvmYLhL8rZMV7gBoYdOLgt0x9gOAzrVrQWWks31JmwKhIwPxfAfJvs4KvSOgtkgGaKRmTeIkDHi05Ap/c4rLqPqzknPXmKBgy5FzxbRi3FCQ+2az8xj08RL0auR3w= Received: from CY8PR12CA0042.namprd12.prod.outlook.com (2603:10b6:930:49::21) by IA0PR12MB9047.namprd12.prod.outlook.com (2603:10b6:208:402::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.29; Mon, 15 Jul 2024 17:29:06 +0000 Received: from CY4PEPF0000EDD7.namprd03.prod.outlook.com (2603:10b6:930:49:cafe::67) by CY8PR12CA0042.outlook.office365.com (2603:10b6:930:49::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.29 via Frontend Transport; Mon, 15 Jul 2024 17:29:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EDD7.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.11 via Frontend Transport; Mon, 15 Jul 2024 17:29:06 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:29:02 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:29:00 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero , Alejandro Lucero Subject: [PATCH v2 13/15] cxl: preclude device memory to be used for dax Date: Mon, 15 Jul 2024 18:28:33 +0100 Message-ID: <20240715172835.24757-14-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received-SPF: None (SATLEXMB04.amd.com: alejandro.lucero-palau@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD7:EE_|IA0PR12MB9047:EE_ X-MS-Office365-Filtering-Correlation-Id: 3420371f-7b18-4c8f-1820-08dca4f3a0dd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700013|376014|921020; X-Microsoft-Antispam-Message-Info: Uv8GkJkq5mQhqOCKJu6QRGRq2dDWukNvo7C6//Kniraw1mXt/5vHGbmjcz56g1qfHIhhrzELVMDviQWNNKr5kvnLc0YRm9Smr9TalaypElrytVHQy3Me9yISoEUUDtdhvA3EUGMpfsZ8A/vPG1KvCujc8UKtelpMhvnYx1/TBQgwgB8M+f33pqK6DabuZXeqNsQ+QNfAxFtFuEF6l5ar45bHojzJtqG3pu6nSbcWNZZG3UXr86rIdIx1Dn6mPH4vJA3HaKUFsbsBYOZiys0SEjIUxD5dDCykgsVG6D9IZV5aAGnJ3sIg9/tmlFEuEt7i9kVxU9sciKgpgzqEfXiHyVOj5fFuXJ8UtDGOaswn5SQZ73id5V/m5h4QIwOFG43dJbXRhUbJZnj1EdHUx7dS+kHT7lpwmX84Jr80MUKwwzCzbj8WzjXrLLRyXG4ud6MY8wB626hcMESq9oEHgrv1fi9AlTLqw/3ayOYZ6a/R+zmpgwVomQAPYFVUqYURoEeFDjlaIxelPT8VBzX25WpGNw889IHJWH7P7OqZVVbQshFH8VV2/JfZffCvZMuZBuPhsIQMsgBTYnPeSSaQ8LMv0Ruws+OPMeSk7yfqsE43dmS8I6KwVd8iT+xZ55LKjgeAPb/HDqLZdT2oyC8eoFurME8Mcofh4EAXn3UtTPWFkh1dYMrhiyP2x+Snt9oQzAmf+3jaiJfIozGNd5UFe/NzS5IFpRUe0hON4FllllQMRK6UNKlPbi3+Tm3eZe26RM3v5EhbUpN9W7wj/Er2yzWKwgkGhuvXr6EN2oCbW7mc8pRDfCQRcNYz65xTJe9gzlVVoOUl3fiU52bq1I+W7ca1mt/FofTjSYPGZkEaPoOtLDCFvP9+E23XTChPE+LTwn962ShOrIldOzR+r9wLLiVca3lRxycqNNpsTKLPtVF8aPmlRM5FUiLaiWacX1QLWrUZWZ1smYpsfpSRDWe3iNWq9WtMI6/6Y9AJXX0xWmgJaajt+PXyqaWbcIm/FybiSbe3xvTthrA1CwaUngT9Suq1FGkTp+KY90AJryuBQnd4WPLfYstS113zEjiwnTKWgP27dCgtXYToYiyWNABFMlXiKRkTuBTQbfTl/dSvJAk2VXDjsek/m1fZ2aIM9ud7oSyxX2gI8C0xrbnHuCpWizmnOpPtzMM9JNVFQXR7HIj8FUmqU7mnZiw88aNseLi0wDrsonQ+Mo4YaOaL8GrQJWjdRsThkGwwFLp3NXrlZxqOsJliKTlDi5mnOnJSKQqQSrcJUHEZtvohlJkjCCVNuZu3XOsIxRWAgqJeGSlZWdtzNcwUH/TmBYCORlVjToAaZw0RmQX6n3V3obUGJKAtDa9Tfmu4FTPJxcmT/hMGrYqz6at/77OopeZB3t0Vh5nGSwtPmHEXHQdJ5t2gWyz+Cvpz8MUHNN5c/ig1ko0A1isSZr7AjZge/zAzU8FeP7kARH55+jxHZHIjElbmSeQZ2K8d3Q== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700013)(376014)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:29:06.1370 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3420371f-7b18-4c8f-1820-08dca4f3a0dd X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD7.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB9047 From: Alejandro Lucero By definition a type2 cxl device will use the host managed memory for specific functionality, therefore it should not be available to other uses. Signed-off-by: Alejandro Lucero --- drivers/cxl/core/region.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 697c8df83a4b..c8fc14ac437e 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -3704,6 +3704,9 @@ static int cxl_region_probe(struct device *dev) case CXL_DECODER_PMEM: return devm_cxl_add_pmem_region(cxlr); case CXL_DECODER_RAM: + if (cxlr->type != CXL_DECODER_HOSTONLYMEM) + return 0; + /* * The region can not be manged by CXL if any portion of * it is already online as 'System RAM' From patchwork Mon Jul 15 17:28:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733711 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2053.outbound.protection.outlook.com [40.107.243.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8001082486; Mon, 15 Jul 2024 17:29:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.243.53 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064555; cv=fail; b=HDb4SN2FkQLx6/e2cobBkrRhIStdg93IUNAWMVLV/8nfbqcCbA1IBgGQmbYvBnzsUjYfgDmij0EFvVvEw7yoVdNPLN5ofWnXTwch85lpbiQ9pwBaFAw7GEBlGYaA8SHcXkdGpDb66FL6xeJApb3bQdYLA8hHdFhzyT/cnr8BIOI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064555; c=relaxed/simple; bh=l/+FmP7W4y9wt+tPE2M28dGJ00elP29yEJE2jwO9fiQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=QTH1XuTUzLoUH+hJYgUfn0hkq8+X6LhiB5hYXv0IZx3otwhDLElOtFR3CBLv9qm7RGpEzyte5Nz1GyNBHVAavfs0CX8xwW0Lb0hxOaGqu8tw/XTthjF6DB1oj3h0wvQHQddZke6MN/oIwPVR7x6u2QZyMMy9pV+B6C5ckR4BAGo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=nzVqBfxL; arc=fail smtp.client-ip=40.107.243.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="nzVqBfxL" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CGXO476LWRzpLfWpfyQvm28yANeWbEdtwXHp6g+SQ9wxfcT85zH3HUdUKHjaDtz7Nwv98KJ4FblKnz3vcfGTCmmt42Ynmtz41UjoweJq2PB2Xka+ObHyHLQHtVIuDltOn0hWHJsdRJ464vDIQz03qz4QA104V0/cuGX7HWuLYHgEvWwntM0kd8VqBYu1nZJEqWcfDf+b+0+Lg504NvHlpydBllDngOgi2ZIgltI77d7Pec0OuUyOcI4trwoX3ZLt3XsDPGKNZq+q3BDydbx6+Nyyt+51sYBZM1Nm0eI3Gjj7amEGkM75dpfuz8Oi3FjVo0w+5QIMaQ3dPImAeFt02w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SdujPSOjy/j4GFMNo1rdfTLRyXkFQ36MUwtV/GPAJUU=; b=P4p6SEUQmuDoh0RkGUrnyoTQ7S+ydfOuUOkmznBL/bLW5yaHMdzeSiEgHTwnjwERsUIH5tzbkR5VohhaCjWdLteqFmAASQxnQVaPZxoBL0E7uVtUUtf6kZJ21OpFN3YkiNdK2aSzRtWU1/eiOFKKRmvpzh8nPu02tBpmmUx5yNSnMspp6QEztjitADTGbAYbO7wdbrc5Glx6cSVUj+u9nqcZWHGEzGA+EDngkInIMb5a72JLG3Bi0ZsojFQcsj8pk0mhmqUzdqYYNXlso8HL2zqPQWJC4dKUEyb1pLJCNL6I3MqyBT+bfcf4faxjRRcWUVImEz+3SLkTz1gBjZIV1A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SdujPSOjy/j4GFMNo1rdfTLRyXkFQ36MUwtV/GPAJUU=; b=nzVqBfxLYQUFF7kMIKM1HMjmxEmAbijydqeeVS6dxPhaHayWRKZRmk0EdlAvczE1rioKoXPiE/pVTPHEIU2eRrVbYHfBV6v33ENUOYZHlCAmLomF9C6s6qcBbfK5TJlia+dUpjSBg2z6jtAoTEY4yL8/4/3qjQNHghQmaG33bsc= Received: from BYAPR02CA0021.namprd02.prod.outlook.com (2603:10b6:a02:ee::34) by IA0PR12MB8424.namprd12.prod.outlook.com (2603:10b6:208:40c::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.29; Mon, 15 Jul 2024 17:29:06 +0000 Received: from SJ1PEPF000023DA.namprd21.prod.outlook.com (2603:10b6:a02:ee:cafe::f3) by BYAPR02CA0021.outlook.office365.com (2603:10b6:a02:ee::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28 via Frontend Transport; Mon, 15 Jul 2024 17:29:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by SJ1PEPF000023DA.mail.protection.outlook.com (10.167.244.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.5 via Frontend Transport; Mon, 15 Jul 2024 17:29:05 +0000 Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:29:04 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:29:03 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:29:02 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero Subject: [PATCH v2 14/15] cxl: add function for obtaining params from a region Date: Mon, 15 Jul 2024 18:28:34 +0100 Message-ID: <20240715172835.24757-15-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received-SPF: None (SATLEXMB05.amd.com: alejandro.lucero-palau@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF000023DA:EE_|IA0PR12MB8424:EE_ X-MS-Office365-Filtering-Correlation-Id: de1c62ff-e12e-4af6-6625-08dca4f3a08d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|1800799024|36860700013|921020; X-Microsoft-Antispam-Message-Info: c85xg5TBlfyDHjEsvF0J8UTozPGpoYQyk+Crwva4Fq5UMXJGh3zgaYyhSqy7/ax+uYN+iqXKJcaGafA7ynAygVpGzmDtw8tk4pLXw+tOSUMxdMp9lH/XGBuoNi9Qvs/BQUNneiwWRfA6PrLqYJx5x4SsLantV7D2D9+cdvPWisTkcuBkD1/4BiRP5R8y8YLZtmnxOJUlZgB3obV61k1SacTBpW2WnE1+AfwYneXZLwkn/xLvFksG0aAWMk1EbkDscSTawd47iB/L0RPb+MrQlIzh6vELDIbi/OHLoRtC6riPuXePleRn5cflXSDVwqstKm5kIRs1G31KnAEw8dR720XVd41g0fV5Xws9Lx34TGHQ0mpFLCVK1tNYH75zDm3TudgKspraHaVHXzQeu71rW9seQADEmuDqQzOIrMw6NdQteDQueYMK5+fqukY3YxD2DzG2flY38FGfqti3vsffVOVw5SHqkPTQHwO93HWX6xxEnyhrFIOIpfraGNmExwQ3g2HUt64GtrNj5wEdQL6NX8MltMEwU/hhzrU3xHQj4zVfB3EBWd1YGqAWr2xqIGp5G4IO2muR4uFjh7nv+9HYTOXp1Q+63Cpj31n9iWveAlpC0gkqkkUjqmAKOJEA/UiI7rjM5YtKF8m+mhAA+Z6dIdMX/xCPIKFvrECPl4PAA1YBvMAl70NaKjvk8BTwTOVlX47ySlwiTRHMUrvH4shjAYpe8FmsL6jMkQtjjYp1uqIz2lJxLPH7sUhzEsHy0s296SeKns+UpCD+ttIHjrLRw9Z9kkw7KWsZgtQR6aOHhc7dQxytrBAxH+GAV2L6LeSdYg5zjxbisEgwl891tY8V26Mw8YCA45ZRmzsrZQ2NUh4uHkb6x6rnqcdUAUnuYTr1RhigxAiPQXM6PvkDRsquyyplNSRWpKIJMj1srmSusTJzr1rXugQrm9Nryq/qNfO15cE9crcHXMJAxDatyq5R0fNoI9xeP/yN8kN99huD6XAiUm+Y52Y5DaUcv6RgVtNY1NV2Ooeu6xPXS+cCcRnQYODvrUQ3cdudM6Ig3W/eJrVqs702lmGz+oUY88t3mEqW5FBGVyGfAtnxT7JdWH8A6G0SHUBD0tNHEaAmAhiiTooKpuQ6KjCHGZ0sBqIfrcxevr14/Duv63cf0ZMPDp3P5zhi3QKaliX3Y9oSA5F70p0NFee5ct5R8sJH2WhmeN9lhw9zfGJDAG+KdCQCopZvxl2XOc3yZ91kx2BOEkhU9ihe/3r6QXwzwy8+e3dTKkndvMekFUpi3R/Rjlej2xvka/eKPe5NSynk5VQHrQaFRvCgx/uPRYBses4D4AHq/EKCONKhv7r8rkrNzEIXUQkt6gGaV/iGzF3ZblJ4n5oWztllzNHGf7Cyfr1OgmLeS8/esyFUUkzMvx+btR0mPOzGZxUZx7joXz1AOvpezc/Xo9o= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(376014)(1800799024)(36860700013)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:29:05.5991 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: de1c62ff-e12e-4af6-6625-08dca4f3a08d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF000023DA.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8424 From: Alejandro Lucero A CXL region struct contains the physical address to work with. Add a function for given a opaque cxl region struct returns the params to be used for mapping such memory range. Signed-off-by: Alejandro Lucero --- drivers/cxl/core/region.c | 16 ++++++++++++++++ drivers/cxl/cxl.h | 3 +++ include/linux/cxl_accel_mem.h | 2 ++ 3 files changed, 21 insertions(+) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index c8fc14ac437e..9ff10923e9fc 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -3345,6 +3345,22 @@ static int devm_cxl_add_dax_region(struct cxl_region *cxlr) return rc; } +int cxl_accel_get_region_params(struct cxl_region *region, + resource_size_t *start, resource_size_t *end) +{ + if (!region) + return -ENODEV; + + if (!region->params.res) { + return -ENODEV; + } + *start = region->params.res->start; + *end = region->params.res->end; + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_accel_get_region_params, CXL); + static int match_root_decoder_by_range(struct device *dev, void *data) { struct range *r1, *r2 = data; diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 1bf3b74ff959..b4c4c4455ef1 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -906,6 +906,9 @@ void cxl_coordinates_combine(struct access_coordinate *out, bool cxl_endpoint_decoder_reset_detected(struct cxl_port *port); int cxl_region_detach(struct cxl_endpoint_decoder *cxled); + +int cxl_accel_get_region_params(struct cxl_region *region, + resource_size_t *start, resource_size_t *end); /* * Unit test builds overrides this to __weak, find the 'strong' version * of these symbols in tools/testing/cxl/. diff --git a/include/linux/cxl_accel_mem.h b/include/linux/cxl_accel_mem.h index a5f9ffc24509..5d715eea6e91 100644 --- a/include/linux/cxl_accel_mem.h +++ b/include/linux/cxl_accel_mem.h @@ -53,4 +53,6 @@ struct cxl_region *cxl_create_region(struct cxl_root_decoder *cxlrd, int ways); int cxl_region_detach(struct cxl_endpoint_decoder *cxled); +int cxl_accel_get_region_params(struct cxl_region *region, + resource_size_t *start, resource_size_t *end); #endif From patchwork Mon Jul 15 17:28:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Lucero Palau, Alejandro" X-Patchwork-Id: 13733710 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2065.outbound.protection.outlook.com [40.107.93.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 572F44D8B1; Mon, 15 Jul 2024 17:29:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.65 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064553; cv=fail; b=rV64B//3DS35k06378TaJvmuucTzLF0RHVZhdhbVo8MQZBmMJYd/josNYQU5Go6CJQ0oXIBYssKRFJ9p/ewTLZ82++Z/OZg72X2yVY2Bh38uTgpYsf91gL43uM8NzjYwRy5XWspiWCAa1JI/wsAFnpt2n59yGbQX2DZ4jMu/5Sg= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1721064553; c=relaxed/simple; bh=pPvdLyUcZvGHXVqIIg6J0VgU0IqwiNdrrcApl/+iJiQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=PRQsMeBmJai0q6Q7C5GiStijcIDFjRnZnn4TGE8l9rbrbryZuunRAxqqc4hgZyA55nzmeUGd/hbm0PWULKzb4bzEjtEgbK0VzST2JqXDLUxbTGeHTLSYZZOiQHN4OFB1iJEAWvHPDLVKiVdt0LIaF25H04fZyHDvPyktSXB4KX4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=HFqTMk9Y; arc=fail smtp.client-ip=40.107.93.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="HFqTMk9Y" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XkF6PcS4ZkRsU46lcTB3g23YhpiMsomhZ2jr191TfaTNFklQagI/N4vUXNKxe8d8Du3ixybeIM5JAbHVkqAxgQmulw5vENttJ2iLw0xPA2VYir7r/rTsmkBTHitsFvr4qKurlbLkXQVlc8Wv3KWBEcZR92FFni5iU5ymIM7D4qE/vMvF213IpibUNyMO7AOWa7ViOinGlhp6veZVauLeE5xdxnJfnSfyq0Rw/Q8xU+1JizMSoO64NZH5tpaawrWBDK1iTzXLYDJr7PfWHjBHEgx5S+MhevqqLJpUHAOOaO6+Z0ehD1ioIMiY7CdESvCb1SEX+jwUZTmihzv+5ctTrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qGyrIBQHmZoiBbIwHpDxe4I8NUNz2zk037VCCPCvC+I=; b=MfmK1tE7IyKSylHBFl51Tc/vJUY+Xv4SZHo4Db4Wb3cJETwDorE/fUeoJ0mXqhhtYzv+UN7wM3sPb/2jp7CCMBcYYjk8ruKb9bkYk9Sajo2UMVJpEwlbeGer2DdtVlOgsJbRDlyUEZA4MZvu0fZInxUj/nce+70vkIC7JhBcRRE8jKWE8GVIJqyatW9Z2m44+mSwPdyj2Z2zqXZ7qZGJ1XlfaerK9Pm3EenxvGtlW/MrgGdtWhCfiFi4gPlbruEs+yAeOidBOcvUDxsoFtgqf1Q1gadur1rr8rCSjmuakIzGwr03e+NeKB2JmTpkj78QOMBUA6IOZXh53nV/KO6fmQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qGyrIBQHmZoiBbIwHpDxe4I8NUNz2zk037VCCPCvC+I=; b=HFqTMk9Y8CvElFh9FtKGnlvUiEOK5No3fcHUoM9DciaWLxWe2KxxnPuYntQGjErm4l73M39VHFMMvlH5wejpnMSBwldXm/nINwXvHQumRnI4YzwPdnv0eKUis7to49PP3joxCl5m+xe+NYrXMspgWFDlOYU5gOdRLVBHcf6BmQk= Received: from CY8PR12CA0045.namprd12.prod.outlook.com (2603:10b6:930:49::16) by MN0PR12MB6341.namprd12.prod.outlook.com (2603:10b6:208:3c2::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.28; Mon, 15 Jul 2024 17:29:08 +0000 Received: from CY4PEPF0000EDD7.namprd03.prod.outlook.com (2603:10b6:930:49:cafe::ce) by CY8PR12CA0045.outlook.office365.com (2603:10b6:930:49::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.29 via Frontend Transport; Mon, 15 Jul 2024 17:29:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EDD7.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7784.11 via Frontend Transport; Mon, 15 Jul 2024 17:29:08 +0000 Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:29:05 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 15 Jul 2024 12:29:05 -0500 Received: from xcbalucerop41x.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 15 Jul 2024 12:29:04 -0500 From: To: , , , , , , , , , CC: Alejandro Lucero Subject: [PATCH v2 15/15] efx: support pio mapping based on cxl Date: Mon, 15 Jul 2024 18:28:35 +0100 Message-ID: <20240715172835.24757-16-alejandro.lucero-palau@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> References: <20240715172835.24757-1-alejandro.lucero-palau@amd.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Received-SPF: None (SATLEXMB05.amd.com: alejandro.lucero-palau@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EDD7:EE_|MN0PR12MB6341:EE_ X-MS-Office365-Filtering-Correlation-Id: 562b48ef-12ae-491d-450b-08dca4f3a20e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|36860700013|376014|82310400026|921020; X-Microsoft-Antispam-Message-Info: mCDPWET5iNheMoZRwCq0M7jcwO/xrnGScXe1OkUxn+AOBdv1+ZbF9KG9lpCjvxzIBjXyEjxGEge1iitHP1IMbUBU6J6c+bchulrk5CK+Aut3aPegkD23b0gKeBek3CU6buZ+iQZrwOZEox8DLVQB7eiDyf7rT4MY/+OI1OV1PlMXIVXwtPXPparztCCRJk/aTFBBiRmnKU+aOLTTUxDN0KMMv+w5HURD/RGmXHSQmee+XmQpJP6M6RzpYgB1hcoqyCkTnMniQ+p54cq10ZSx4TIeDf2SxwAxAkPMOdrDSPYyR8CusD4v9cXlauWqwntPdLpuqTbbzQlD5QkLp7lbo2zsqCddFYLr4lY0/CvxmSdhwdw19z0lpdtaKReBjFleJj8T1anO2yJZ8r0scDLvqwHMXom5GFrGjMnsb491j/yqxnAmYmrGFs59Y/kuhmwWjK+sv1wEShH1oteInJNLkUQVdieZMP3aSokyNcLpmUT9HIXxEUBG2wKrkmNvu21op/byJdmNi24OA+zHP/CoqtWgxlTwhaEiTAAxYsdKTfRxE56D0N8XbODInOe6PNMlQWVSpUjuuezvDFfyWSmT3s+Hjg8BeSve+ySTYOBVnoBr1azM41+jf1V6awfMgIbORAVMbUvQaQc1upx/RqeIaBgD/rUv7s2kQuGfHzs5nxpNJsxr2wBq3rjoMX0TrVCzkxT3mYHUWA5JMmP7hmr3M3soktaAeWT0nJlWuIAJ37Clz9W34T0J2PIm2uuoH5tP711PAEfPKvr+eiozDdbx10aNmwM4G9MgEbhoTCip1ppCoJzCzzrCgNhhqEwXWie8ysKaHpqCH4H0lnkwVAhgBeo6I+oMH4s8gpvNl74fklZQFwil98lleCmCk1whyU2ROlkSBMlJUgaI3UV73yQBkiqStI58O4qq154HTVv5fg4xRPw/OhBRtxCf44BlELYfKrCtVCPmgvfwLjQi95KujQb5P2JEIWBbzNsI6NY28c2hNFBt87cmd+dXFEN1lArCf6JKYz2KpolaR46X1v3JH/I8vKu9eyS/LT5cCHsJeu6hAcljsPMo7yqmknk9CBkQrqvn30BxfN+jf632WsTPhlcA2Ev8SavvAVBx3cRVqLAR8gvmbmx6fIEu9YzSxummx6C5bJzadZTaNOAqoJ4zIVllZ3/iTx5tUtcIfF1v0uOYzhe4j/Jqt8/xr7mLEbadxi40IyMY+8EpkotrEpQDTKDa1/8jeklsgQy6A+tADkNyQAZsfS4GmPW4YfHurUvvhoGGnLxwIfhb4PcsHy3YPhXSEXP2qyeB289RNXWSVLTgU2ydL8cbuqNxjpbSaWBkpMg4F3sPQDhi9RqWxVdRtucj8mWjVsw1eWzBXgdIMXu3bJo37Kfugoo0MBZEmBFbL/rx1xLvQvh5jhgxLLfUPLchqanNjcz/5GVGoWgP6BE/nYE8WI7C9zAzhmG2FbBfozD1mVcqzcB1ec4tGy8aqQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(36860700013)(376014)(82310400026)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jul 2024 17:29:08.1370 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 562b48ef-12ae-491d-450b-08dca4f3a20e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EDD7.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6341 From: Alejandro Lucero With a device supporting CXL and successfully initialised, use the cxl region to map the memory range and use this mapping for PIO buffers. Signed-off-by: Alejandro Lucero --- drivers/net/ethernet/sfc/ef10.c | 25 +++++++++++++++++++++---- drivers/net/ethernet/sfc/efx_cxl.c | 12 +++++++++++- drivers/net/ethernet/sfc/mcdi_pcol.h | 3 +++ drivers/net/ethernet/sfc/nic.h | 1 + 4 files changed, 36 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/sfc/ef10.c b/drivers/net/ethernet/sfc/ef10.c index 8fa6c0e9195b..3924076d2628 100644 --- a/drivers/net/ethernet/sfc/ef10.c +++ b/drivers/net/ethernet/sfc/ef10.c @@ -24,6 +24,7 @@ #include #include #include +#include "efx_cxl.h" /* Hardware control for EF10 architecture including 'Huntington'. */ @@ -177,6 +178,12 @@ static int efx_ef10_init_datapath_caps(struct efx_nic *efx) efx->num_mac_stats); } + if (outlen < MC_CMD_GET_CAPABILITIES_V7_OUT_LEN) + nic_data->datapath_caps3 = 0; + else + nic_data->datapath_caps3 = MCDI_DWORD(outbuf, + GET_CAPABILITIES_V7_OUT_FLAGS3); + return 0; } @@ -1275,10 +1282,20 @@ static int efx_ef10_dimension_resources(struct efx_nic *efx) return -ENOMEM; } nic_data->pio_write_vi_base = pio_write_vi_base; - nic_data->pio_write_base = - nic_data->wc_membase + - (pio_write_vi_base * efx->vi_stride + ER_DZ_TX_PIOBUF - - uc_mem_map_size); + + if ((nic_data->datapath_caps3 & + (1 << MC_CMD_GET_CAPABILITIES_V10_OUT_CXL_CONFIG_ENABLE_LBN)) && + efx->cxl->ctpio_cxl) + { + nic_data->pio_write_base = + efx->cxl->ctpio_cxl + + (pio_write_vi_base * efx->vi_stride + ER_DZ_TX_PIOBUF - + uc_mem_map_size); + } else { + nic_data->pio_write_base =nic_data->wc_membase + + (pio_write_vi_base * efx->vi_stride + ER_DZ_TX_PIOBUF - + uc_mem_map_size); + } rc = efx_ef10_link_piobufs(efx); if (rc) diff --git a/drivers/net/ethernet/sfc/efx_cxl.c b/drivers/net/ethernet/sfc/efx_cxl.c index 4012e3faa298..8e65ef42a572 100644 --- a/drivers/net/ethernet/sfc/efx_cxl.c +++ b/drivers/net/ethernet/sfc/efx_cxl.c @@ -21,8 +21,8 @@ void efx_cxl_init(struct efx_nic *efx) { struct pci_dev *pci_dev = efx->pci_dev; + resource_size_t start, end, max = 0; struct efx_cxl *cxl = efx->cxl; - resource_size_t max = 0; struct resource res; u16 dvsec; @@ -104,6 +104,13 @@ void efx_cxl_init(struct efx_nic *efx) return; } + cxl_accel_get_region_params(cxl->efx_region, &start, &end); + + cxl->ctpio_cxl = ioremap(start, end - start); + if (!cxl->ctpio_cxl) { + pci_info(pci_dev, "CXL accel create region failed"); + cxl_dpa_free(cxl->cxled); + } out: cxl_release_endpoint(cxl->cxlmd, cxl->endpoint); } @@ -112,6 +119,9 @@ void efx_cxl_exit(struct efx_nic *efx) { struct efx_cxl *cxl = efx->cxl; + if (cxl->ctpio_cxl) + iounmap(cxl->ctpio_cxl); + if (cxl->efx_region) cxl_region_detach(cxl->cxled); diff --git a/drivers/net/ethernet/sfc/mcdi_pcol.h b/drivers/net/ethernet/sfc/mcdi_pcol.h index cd297e19cddc..05fd5e021142 100644 --- a/drivers/net/ethernet/sfc/mcdi_pcol.h +++ b/drivers/net/ethernet/sfc/mcdi_pcol.h @@ -18374,6 +18374,9 @@ #define MC_CMD_GET_CAPABILITIES_V10_OUT_DYNAMIC_MPORT_JOURNAL_OFST 148 #define MC_CMD_GET_CAPABILITIES_V10_OUT_DYNAMIC_MPORT_JOURNAL_LBN 14 #define MC_CMD_GET_CAPABILITIES_V10_OUT_DYNAMIC_MPORT_JOURNAL_WIDTH 1 +#define MC_CMD_GET_CAPABILITIES_V10_OUT_CXL_CONFIG_ENABLE_OFST 148 +#define MC_CMD_GET_CAPABILITIES_V10_OUT_CXL_CONFIG_ENABLE_LBN 16 +#define MC_CMD_GET_CAPABILITIES_V10_OUT_CXL_CONFIG_ENABLE_WIDTH 1 /* These bits are reserved for communicating test-specific capabilities to * host-side test software. All production drivers should treat this field as * opaque. diff --git a/drivers/net/ethernet/sfc/nic.h b/drivers/net/ethernet/sfc/nic.h index 1db64fc6e909..cd635f4f7f94 100644 --- a/drivers/net/ethernet/sfc/nic.h +++ b/drivers/net/ethernet/sfc/nic.h @@ -186,6 +186,7 @@ struct efx_ef10_nic_data { bool must_check_datapath_caps; u32 datapath_caps; u32 datapath_caps2; + u32 datapath_caps3; unsigned int rx_dpcpu_fw_id; unsigned int tx_dpcpu_fw_id; bool must_probe_vswitching;