From patchwork Tue Apr 11 18:02:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13208011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FB57C76196 for ; Tue, 11 Apr 2023 18:04:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229947AbjDKSEm (ORCPT ); Tue, 11 Apr 2023 14:04:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46498 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230215AbjDKSE3 (ORCPT ); Tue, 11 Apr 2023 14:04:29 -0400 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on20616.outbound.protection.outlook.com [IPv6:2a01:111:f400:7e83::616]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 929F06A48; Tue, 11 Apr 2023 11:03:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=B1alQLzKEg2H/tXGvd4w6CaqljvaM3RW1R0gsDpaQqBpYrJJW73pWy2wn4hIfEoS0pCVPeBIi4oyjsqGl/hesjdMH6XF/6B4pxaoxFSW6vQ1kp3xST6N/FrlAO55h+0urufwHHnm49/pnZcgUtOW9+ryp8wwvH1idsvUj9yauUnmvifpQu0UjOSyZByhO20xykL4K6EDoSlvrPipOd5h7AqFkGwrRF/M+sX8K6dZoNcfPkmjt163nDcTAyZhUwXcEjWr0YRshy7qDXPanen0+QvbRz87AvPLXNobxzAHp1RRB4AbN01sF3ssEQ6QSYneSNFXtlqbhzTjVXbZp2uZwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Y3P6ZpZ4Byd7FgMB+f1+rDbZwUVfEc3EjiGHpk2Yobk=; b=nBPcDMMzE2Chel5vRMPZLfrtfTWZ0o/DfSCUzSdNgcVf1kUZrV3YzPHH59KtstHh0V4Qn1ltfbnuMIBLUolIj+M12EIoXITCQbM7KPppou6CQVptQT9FvCHvzEMimjgSk6801jCcu95RyhvZCkGZ7OXxCpS/4ZSpDrWjCeoFS60kJ9SnHuhoBrmCra36RMKT3Rje9NepbFQr49Piwc3NUI0QmDVWlnV+5yslCy4NMzRe1INb8dHRSwXRJ9C2dX4nFlXqzJ90zI24x8ckpBYkLDBywSwX5ZZYqVqFqvAEbnGZdw0T8SeiYnhGH0sPhAmNSptdofE4Pw5DmVxho49xOQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y3P6ZpZ4Byd7FgMB+f1+rDbZwUVfEc3EjiGHpk2Yobk=; b=EwvyqoQyovibIZb2OcSZh+UrXCr3IaScsyoxvdFcEqMjf9lrvgvZBPPgQyl47smSqt+B5Ch/3ekI8XOl11g05KRu2DsT/NQovjeJjKHkaSttAmOW4cC0DMPnhevXxR2pjVGxxZShczOwXL35xnsbIAnxpEugH/wOkPvU0jAFRX4= Received: from BN8PR04CA0049.namprd04.prod.outlook.com (2603:10b6:408:d4::23) by DS0PR12MB8413.namprd12.prod.outlook.com (2603:10b6:8:f9::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.35; Tue, 11 Apr 2023 18:03:24 +0000 Received: from BN8NAM11FT045.eop-nam11.prod.protection.outlook.com (2603:10b6:408:d4:cafe::86) by BN8PR04CA0049.outlook.office365.com (2603:10b6:408:d4::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.40 via Frontend Transport; Tue, 11 Apr 2023 18:03:24 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT045.mail.protection.outlook.com (10.13.177.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6298.29 via Frontend Transport; Tue, 11 Apr 2023 18:03:24 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 11 Apr 2023 13:03:23 -0500 From: Terry Bowman To: , , , , , , , CC: , , , Subject: [PATCH v3 1/6] cxl/pci: Add RCH downstream port AER and RAS register discovery Date: Tue, 11 Apr 2023 13:02:57 -0500 Message-ID: <20230411180302.2678736-2-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230411180302.2678736-1-terry.bowman@amd.com> References: <20230411180302.2678736-1-terry.bowman@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT045:EE_|DS0PR12MB8413:EE_ X-MS-Office365-Filtering-Correlation-Id: ba7dd313-8306-41cc-86a5-08db3ab70b36 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5sUFGJyL8V4y5zuln0KFmPAmG01vMrqxu1c2IL0FcwFMHxMWwzpM0r4IyCmswA5tGvxg2xp7fmpIMfZpGa2qOO4avtUBjEJrpmDhRhy49m4sUL/cCJAHoV7bPbm5fvY+POj2ipvsq87CadlpsU2kFblkiaus2HUCZqlJibfQopqGYhlC5OobcD2PBAj5Nxr5eD0FXb7FGe+a9bmVBYHx4MSN3rF+J0FoNF15+ZhxOAP0v8Hr9wP+vwSJjvK9yEsZ6ut88pSKMCcs8GkECRWih9vDS/gh/i4PX4hZSRxD6gLhhIcJRZpXzsZRDzpWjN2m8eU/oMB8tuOI1IaSDNIumG4oQybgBKolybqwoXEVFmQsdeW9JRE/p25pcM7gH3Zc92rKzFrETrnDAuaOdBE9hDNfl7ZK7aXVPgANvkj27q9WP3nHcS07x7S961nl2gmy8oT28e6WPlQcey+xhhAUXxpgSmhMcT7quZvO5V0RV2dPhcR3Ftosl8/NlhaDNMmg2hFresfj2t+4SIipUVOk3xwCZ4yntjMkGRi4/I9pj80ziyACB5Bh706VzdUqx8mLJFBItp+yVF8XtF1P6iAyg2Jik1Go51EsC7iBhenzGc85OxrdNdU7hTLQ0UL+oiCrI+aBqdYOixU8V8Wb3L7hypMj5wog58n/+tuaiOR2laiJbT/+NXJnagCNvmSU3xxt4SP145tUfdhGmcEichs7mR6M3gDEuP9yK1ZfaQgo6eI= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(136003)(376002)(346002)(396003)(39860400002)(451199021)(40470700004)(36840700001)(46966006)(44832011)(36756003)(7416002)(40460700003)(82310400005)(2906002)(5660300002)(8936002)(8676002)(40480700001)(86362001)(7696005)(6666004)(26005)(1076003)(83380400001)(36860700001)(110136005)(54906003)(478600001)(47076005)(2616005)(426003)(336012)(16526019)(186003)(70586007)(70206006)(82740400003)(356005)(81166007)(41300700001)(4326008)(316002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Apr 2023 18:03:24.4137 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ba7dd313-8306-41cc-86a5-08db3ab70b36 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT045.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8413 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org Restricted CXL host (RCH) downstream port AER information is not currently logged while in the error state. One problem preventing existing PCIe AER functions from logging errors is the AER registers are not accessible. The CXL driver requires changes to find RCH downstream port AER registers for purpose of error logging. RCH downstream ports are not enumerated during a PCI bus scan and are instead discovered using system firmware, ACPI in this case.[1] The downstream port is implemented as a Root Complex Register Block (RCRB). The RCRB is a 4k memory block containing PCIe registers based on the PCIe root port.[2] The RCRB includes AER extended capability registers used for reporting errors. Note, the RCH's AER Capability is located in the RCRB memory space instead of PCI configuration space, thus its register access is different. Existing kernel PCIe AER functions can not be used to manage the downstream port AER capabilities because the port was not enumerated during PCI scan and the registers are not PCI config accessible. Discover RCH downstream port AER extended capability registers. This requires using MMIO accesses to search for extended AER capability in RCRB register space. [1] CXL 3.0 Spec, 9.11.2 - System Firmware View of CXL 1.1 Hierarchy [2] CXL 3.0 Spec, 8.2.1.1 - RCH Downstream Port RCRB Co-developed-by: Robert Richter Signed-off-by: Robert Richter Signed-off-by: Terry Bowman Signed-off-by: Dan Williams --- drivers/cxl/core/regs.c | 93 +++++++++++++++++++++++++++++++++++------ drivers/cxl/cxl.h | 5 +++ drivers/cxl/mem.c | 39 +++++++++++------ 3 files changed, 113 insertions(+), 24 deletions(-) diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c index 1476a0299c9b..bde1fffab09e 100644 --- a/drivers/cxl/core/regs.c +++ b/drivers/cxl/core/regs.c @@ -332,10 +332,36 @@ int cxl_find_regblock(struct pci_dev *pdev, enum cxl_regloc_type type, } EXPORT_SYMBOL_NS_GPL(cxl_find_regblock, CXL); +static void __iomem *cxl_map_reg(struct device *dev, struct cxl_register_map *map, + char *name) +{ + + if (!request_mem_region(map->resource, map->max_size, name)) + return NULL; + + map->base = ioremap(map->resource, map->max_size); + if (!map->base) { + release_mem_region(map->resource, map->max_size); + return NULL; + } + + return map->base; +} + +static void cxl_unmap_reg(struct device *dev, struct cxl_register_map *map) +{ + iounmap(map->base); + release_mem_region(map->resource, map->max_size); +} + resource_size_t cxl_rcrb_to_component(struct device *dev, resource_size_t rcrb, enum cxl_rcrb which) { + struct cxl_register_map map = { + .resource = rcrb, + .max_size = SZ_4K + }; resource_size_t component_reg_phys; void __iomem *addr; u32 bar0, bar1; @@ -343,7 +369,10 @@ resource_size_t cxl_rcrb_to_component(struct device *dev, u32 id; if (which == CXL_RCRB_UPSTREAM) - rcrb += SZ_4K; + map.resource += SZ_4K; + + if (!cxl_map_reg(dev, &map, "CXL RCRB")) + return CXL_RESOURCE_NONE; /* * RCRB's BAR[0..1] point to component block containing CXL @@ -351,21 +380,12 @@ resource_size_t cxl_rcrb_to_component(struct device *dev, * the PCI Base spec here, esp. 64 bit extraction and memory * ranges alignment (6.0, 7.5.1.2.1). */ - if (!request_mem_region(rcrb, SZ_4K, "CXL RCRB")) - return CXL_RESOURCE_NONE; - addr = ioremap(rcrb, SZ_4K); - if (!addr) { - dev_err(dev, "Failed to map region %pr\n", addr); - release_mem_region(rcrb, SZ_4K); - return CXL_RESOURCE_NONE; - } - + addr = map.base; id = readl(addr + PCI_VENDOR_ID); cmd = readw(addr + PCI_COMMAND); bar0 = readl(addr + PCI_BASE_ADDRESS_0); bar1 = readl(addr + PCI_BASE_ADDRESS_1); - iounmap(addr); - release_mem_region(rcrb, SZ_4K); + cxl_unmap_reg(dev, &map); /* * Sanity check, see CXL 3.0 Figure 9-8 CXL Device that Does Not @@ -396,3 +416,52 @@ resource_size_t cxl_rcrb_to_component(struct device *dev, return component_reg_phys; } EXPORT_SYMBOL_NS_GPL(cxl_rcrb_to_component, CXL); + +u16 cxl_rcrb_to_aer(struct device *dev, resource_size_t rcrb) +{ + struct cxl_register_map map = { + .resource = rcrb, + .max_size = SZ_4K, + }; + u32 cap_hdr; + u16 offset = 0; + + if (!cxl_map_reg(dev, &map, "CXL RCRB")) + return 0; + + cap_hdr = readl(map.base + offset); + while (PCI_EXT_CAP_ID(cap_hdr) != PCI_EXT_CAP_ID_ERR) { + + offset = PCI_EXT_CAP_NEXT(cap_hdr); + if (!offset) { + cxl_unmap_reg(dev, &map); + return 0; + } + cap_hdr = readl(map.base + offset); + } + + dev_dbg(dev, "found AER extended capability (0x%x)\n", offset); + cxl_unmap_reg(dev, &map); + + return offset; +} +EXPORT_SYMBOL_NS_GPL(cxl_rcrb_to_aer, CXL); + +u16 cxl_component_to_ras(struct device *dev, resource_size_t component_reg_phys) +{ + struct cxl_register_map map = { + .resource = component_reg_phys, + .max_size = CXL_COMPONENT_REG_BLOCK_SIZE, + }; + + if (!cxl_map_reg(dev, &map, "component")) + return 0; + + cxl_probe_component_regs(dev, map.base, &map.component_map); + cxl_unmap_reg(dev, &map); + if (!map.component_map.ras.valid) + return 0; + + return map.component_map.ras.offset; +} +EXPORT_SYMBOL_NS_GPL(cxl_component_to_ras, CXL); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 044a92d9813e..df64c402e6e6 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -270,6 +270,9 @@ enum cxl_rcrb { resource_size_t cxl_rcrb_to_component(struct device *dev, resource_size_t rcrb, enum cxl_rcrb which); +u16 cxl_rcrb_to_aer(struct device *dev, resource_size_t rcrb); +u16 cxl_component_to_ras(struct device *dev, + resource_size_t component_reg_phys); #define CXL_RESOURCE_NONE ((resource_size_t) -1) #define CXL_TARGET_STRLEN 20 @@ -601,6 +604,8 @@ struct cxl_dport { int port_id; resource_size_t component_reg_phys; resource_size_t rcrb; + u16 aer_cap; + u16 ras_cap; bool rch; struct cxl_port *port; }; diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 39c4b54f0715..014295ab6bc6 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -45,13 +45,36 @@ static int cxl_mem_dpa_show(struct seq_file *file, void *data) return 0; } +static void cxl_setup_rcrb(struct cxl_dev_state *cxlds, + struct cxl_dport *parent_dport) +{ + struct cxl_memdev *cxlmd = cxlds->cxlmd; + + if (!parent_dport->rch) + return; + + /* + * The component registers for an RCD might come from the + * host-bridge RCRB if they are not already mapped via the + * typical register locator mechanism. + */ + if (cxlds->component_reg_phys == CXL_RESOURCE_NONE) + cxlds->component_reg_phys = cxl_rcrb_to_component( + &cxlmd->dev, parent_dport->rcrb, CXL_RCRB_UPSTREAM); + + parent_dport->aer_cap = cxl_rcrb_to_aer(parent_dport->dport, + parent_dport->rcrb); + + parent_dport->ras_cap = cxl_component_to_ras(parent_dport->dport, + parent_dport->component_reg_phys); +} + static int devm_cxl_add_endpoint(struct device *host, struct cxl_memdev *cxlmd, struct cxl_dport *parent_dport) { struct cxl_port *parent_port = parent_dport->port; struct cxl_dev_state *cxlds = cxlmd->cxlds; struct cxl_port *endpoint, *iter, *down; - resource_size_t component_reg_phys; int rc; /* @@ -66,17 +89,9 @@ static int devm_cxl_add_endpoint(struct device *host, struct cxl_memdev *cxlmd, ep->next = down; } - /* - * The component registers for an RCD might come from the - * host-bridge RCRB if they are not already mapped via the - * typical register locator mechanism. - */ - if (parent_dport->rch && cxlds->component_reg_phys == CXL_RESOURCE_NONE) - component_reg_phys = cxl_rcrb_to_component( - &cxlmd->dev, parent_dport->rcrb, CXL_RCRB_UPSTREAM); - else - component_reg_phys = cxlds->component_reg_phys; - endpoint = devm_cxl_add_port(host, &cxlmd->dev, component_reg_phys, + cxl_setup_rcrb(cxlds, parent_dport); + + endpoint = devm_cxl_add_port(host, &cxlmd->dev, cxlds->component_reg_phys, parent_dport); if (IS_ERR(endpoint)) return PTR_ERR(endpoint); From patchwork Tue Apr 11 18:02:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13208012 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01F5AC77B6F for ; Tue, 11 Apr 2023 18:04:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229753AbjDKSE4 (ORCPT ); Tue, 11 Apr 2023 14:04:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46374 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230002AbjDKSEj (ORCPT ); Tue, 11 Apr 2023 14:04:39 -0400 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2061c.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe59::61c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 442B072B5; Tue, 11 Apr 2023 11:04:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Xcz1WYnwGD9AeMxuDRrhfkMxIO4Rh4TStJ3t52kY50+eeGXylxartGaDGzYAfdxYLu89yf1XhWOTR78/aXZdMCrVZSDYFUZIFcTsjBFr1m5D5GJR0RybomxPvPIQtoUOxOnmUMTei2V1K8xq7Af4JukOmVCu590dLGB5JCxfiKT5SDV3GtEttBsxazW8o7905bM1uu+K95uu9qtKLxPtGjQK5GNRb40GDysUQitv12HPxtDXWHiaqd8PKnfX9gsILJri9ixSeYhInAnOHV+yMQgcEKKMHYWFa4+Vl6M2pZUJre1nVnmYe1xtACaTbb1ssKxWQqd+xYVZXDVskrORuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=suNq20ozk6lHdvyNJXs15e+AcBTMOET/Yr6d+SImQSI=; b=FvEAgQjwJgnFr11+bEbDMp5J326PlIfEDk+IiZ18OZWW7QbjnEY7lEWE6k8LOr1eiGw3e2QwqRdJ1+Ro9hZ8c0GuBOxLZRrx5juxMmjGsPF4xgHZ72s0KjZDOPcuVrZv8fzACyFYYMHu1DAgkdIVnZGzePU41/CYIhBGmczERkqTtLLd2t+kHCR6zqCwkIuniC1qHWnTiZeH7vFst2cqkUhsuldh4TTldqPFgybNvmAw7tEgo2z0KoukAO+pQ8wyTWxFjbBrRJSHeJcs9OHzSfCNFBFMcMlLYn3CM/3rEvbkJ0y9+LfpRGz5zQbk1lXT/5TlNmi9/TaO1pGl7moTvA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=suNq20ozk6lHdvyNJXs15e+AcBTMOET/Yr6d+SImQSI=; b=avSXcZiZ2KIniYbTChZA0HehMiIc9BPFrAq+ClnVk7DG8+FBh0FVQGGQUOBf6YAuPyEd+U8sefwDzMItFb3amOsYw+mlRsyybtLqVFlQT2j4qS87kGkRdcznpBVlw9IVSx5cz2E1rCdIuy58d2rJ6vZ26VqVW5Uzk3H2y1jI+mw= Received: from BN9PR03CA0328.namprd03.prod.outlook.com (2603:10b6:408:112::33) by DM6PR12MB4155.namprd12.prod.outlook.com (2603:10b6:5:221::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.36; Tue, 11 Apr 2023 18:03:40 +0000 Received: from BN8NAM11FT061.eop-nam11.prod.protection.outlook.com (2603:10b6:408:112:cafe::f4) by BN9PR03CA0328.outlook.office365.com (2603:10b6:408:112::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.38 via Frontend Transport; Tue, 11 Apr 2023 18:03:40 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT061.mail.protection.outlook.com (10.13.177.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6298.29 via Frontend Transport; Tue, 11 Apr 2023 18:03:39 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 11 Apr 2023 13:03:37 -0500 From: Terry Bowman To: , , , , , , , CC: , , , , Ard Biesheuvel , Subject: [PATCH v3 2/6] efi/cper: Export cper_mem_err_unpack() for use by modules Date: Tue, 11 Apr 2023 13:02:58 -0500 Message-ID: <20230411180302.2678736-3-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230411180302.2678736-1-terry.bowman@amd.com> References: <20230411180302.2678736-1-terry.bowman@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT061:EE_|DM6PR12MB4155:EE_ X-MS-Office365-Filtering-Correlation-Id: eb7447cf-7eaa-4a5c-1e86-08db3ab7147f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TcOhJD+8G8sQipuvVxbAecE2ctR+zR8aaiMVVf+4gkW2A74mRgc9IW5c5Z1rF+F7f3NZAlzg8MQoYumz3qK8X0VjnZEG/H7rOsOPpVgLW22MJytrZ6FaqD8/ECgn5qz7KRXV/7DDW398h8BxJX9aStKjlgOIfMtZd37QsXkNM+DxubA74wmrIIZ2bUjxEQb+f1ba4EkFxDWgV3qEBJTvoMdDBn3t/LPUkCeXZ0TrGqaMwUwe2N1NxGXhbWeiS3rB6pGOpNbaAG21ygTZ0P+UwpXLlhnG/uz1Bo16TgdAkFkaj+g44IWXg0A+bp728uKHdDJTEKNb4SmpEauC1sGB1y4CxmGL7/z1rrPh4qLSXh6cXPwCJf3oPyUzwBlN2V/MPS2S22U+fWso4Rp+5PPOAna3ug/Tor93GtKMWTnfmO40DE5h1tdPxoUzg/WABJp+EG2kNSpEb+tgjKcb+tgb5sNMZL3r2OMrS5K8VvLT/oy5ec+zVHYufpUFw0Z1umBd4kEmOwCbH5ps22uSzA5sJ1OnkdX0JzojzQnenavJcJULXYEDUCf/tkGAfOhBcY1O1xXr/k8n7x3XXcpFzwPitkaAD1njvR8y7ik0bfhDfaSt23YekJ9G6UtF6WwF+bGJLr3MuOgFo0sMs7tyDz8C7A+V9QrB+dh+qckD5MSvGle4+Oz4DdvEPHjWSWfoBPbhqDCWazMwzGgGYi+U75pVzzfd41lGEhw8RgRU82oeUQ0= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(136003)(346002)(396003)(39860400002)(376002)(451199021)(40470700004)(36840700001)(46966006)(36756003)(86362001)(41300700001)(110136005)(70586007)(316002)(4326008)(478600001)(7696005)(70206006)(8676002)(54906003)(40480700001)(36860700001)(5660300002)(82310400005)(7416002)(44832011)(2906002)(8936002)(4744005)(81166007)(47076005)(82740400003)(356005)(16526019)(186003)(6666004)(26005)(1076003)(336012)(2616005)(426003)(40460700003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Apr 2023 18:03:39.9889 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: eb7447cf-7eaa-4a5c-1e86-08db3ab7147f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT061.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4155 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org The CXL driver plans to use cper_print_aer() for restricted CXL host (RCH) logging. This is not currently possible if CXL is built as a loadable module because cper_print_aer() depends on cper_mem_err_unpack() which is not exported. Export cper_mem_err_unpack() to enable cper_print_aer() usage in CXL and other loadable modules. Signed-off-by: Terry Bowman Cc: Ard Biesheuvel Cc: linux-efi@vger.kernel.org Acked-by: Ard Biesheuvel --- drivers/firmware/efi/cper.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/firmware/efi/cper.c b/drivers/firmware/efi/cper.c index 35c37f667781..ff15e12160ae 100644 --- a/drivers/firmware/efi/cper.c +++ b/drivers/firmware/efi/cper.c @@ -350,6 +350,7 @@ const char *cper_mem_err_unpack(struct trace_seq *p, return ret; } +EXPORT_SYMBOL_GPL(cper_mem_err_unpack); static void cper_print_mem(const char *pfx, const struct cper_sec_mem_err *mem, int len) From patchwork Tue Apr 11 18:02:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13208013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D1A1C76196 for ; Tue, 11 Apr 2023 18:05:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229822AbjDKSFI (ORCPT ); Tue, 11 Apr 2023 14:05:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46374 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230101AbjDKSE6 (ORCPT ); Tue, 11 Apr 2023 14:04:58 -0400 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on20600.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe5a::600]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7155C5240; Tue, 11 Apr 2023 11:04:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=j5hpMkQCt3O+uUmXA/0ti3l+vGqP6/RrVr0faU9ATBrsXNcIVb8s/OeOvgKgNl7MJCoeqvYLNQjWX8HNpyASirgq+rOZpfkk4dOULxd3occqdPPryxQ8CE2hcmqJlMmIro0xZgflVPm1V+4qjDeMVd27UScrl+OiV4sMB0+Yg0yWcws/a+v+iXJ2WixFa49FNkeFeC1FL+mxEDW4Sk89/SDozbtgcKTj6YUZSwuSFAhgiQwusMzknI7fJB/in64W7ZsGRXAIsSoRNkThCujZNEOAYnsCbtgeLTESZorsuVtVZXNbJfV9uTuMb/DW7t+EJtUCjkUEJFCz+Pw/R6a6OQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9jWizZyOkncM0zUzSAlj45CvBH14g/iZi511HWH1pLE=; b=jpD6YtWPsdIrTRAyTcQQmc2XJ6uCaGFOrWcyyWjKDG3xmmiMhmeUPoUwT9b7nwiKH2Z2UQ7IsCvWtK7rcqLH5sNF5JkU4hO8bJfjGKhrgta8uBCwusfUsRgUBldCvFtrHuFcWwQUXROy8CGo17kyOFCHySFWOEyTajr0ndZ72uGqFmqiKi3RDlWP7v/KjE5gkwRhjaXghY1cmhngbwBUuyFv/oaXTdi5s1WYWioeOGFhCZvY4B+ExjuoHXGyUNGstLzfwMOGsdFM0qxp5G/JwoHOmn6Gqhd62oinAjlj6FGISYJeZOyAOLgcQ03dET6kFUpBTlVheNiITDFLTV9EkQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9jWizZyOkncM0zUzSAlj45CvBH14g/iZi511HWH1pLE=; b=JYXIXHFQvkgc0kNDxxOiyk9wQ4ASCRQFASIns+OfJ4X1WBsePAVXIGz6rjbkb59AvH0IKyPb7/QYwLVtSG6XeZYcMSmlMWIVzywFSyvvpTAT3vfdcdG6a8XDG3rOu1geawEkDvwdf9STavtusV/Ga0i4CTV7M9/qxx74tLmVPXs= Received: from BN8PR03CA0022.namprd03.prod.outlook.com (2603:10b6:408:94::35) by PH8PR12MB6988.namprd12.prod.outlook.com (2603:10b6:510:1bf::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.38; Tue, 11 Apr 2023 18:03:55 +0000 Received: from BN8NAM11FT053.eop-nam11.prod.protection.outlook.com (2603:10b6:408:94:cafe::70) by BN8PR03CA0022.outlook.office365.com (2603:10b6:408:94::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.38 via Frontend Transport; Tue, 11 Apr 2023 18:03:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT053.mail.protection.outlook.com (10.13.177.209) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6298.29 via Frontend Transport; Tue, 11 Apr 2023 18:03:54 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 11 Apr 2023 13:03:54 -0500 From: Terry Bowman To: , , , , , , , CC: , , , , Mahesh J Salgaonkar , Oliver O'Halloran , Subject: [PATCH v3 3/6] PCI/AER: Export cper_print_aer() for use by modules Date: Tue, 11 Apr 2023 13:02:59 -0500 Message-ID: <20230411180302.2678736-4-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230411180302.2678736-1-terry.bowman@amd.com> References: <20230411180302.2678736-1-terry.bowman@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT053:EE_|PH8PR12MB6988:EE_ X-MS-Office365-Filtering-Correlation-Id: 2910ac8b-68a9-4e8f-cbd2-08db3ab71d62 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CGxjmd2xup7L9WjX75UTvjxyStC6eQj/aOm4kZYgwr1OTOmWcE18MjVl5AHTA12Rta87jq1BfySovZl6SOsxBx957uvEGaiUvqkw6W8gZZIdzD2NBKx+TBog7v2Yl/W5DVw37cYCYkRrdZf4q3icFbVUsJXIb2fXHwwwycUFW/AvQMiD+2oWt8wUidfZisuOzuSbyPFt1FUyDu17Wy2x40X7Dz+goBhgsQz8poHlI64RBAPeLk9W+9gbh5sYxOUQzWSs90eOmoJDoJ8Cci9uJdxEXCopyjkeZ1yZ8RIla8t/HChgPEGKjNqjfwo7GNSFHv2P3XYi/e67RPVMZXnYcTJkMffXcTawcUeqyRiqzfWfHPLnHZFinNzonNWXN9R5jGxnHxU+kEWL+0t4j43ZtH1VF7aT8VXj2HzUHInF5UUPJ7jxIp0jo8LQ8YVrgWNtET6E0QN1iOVfAnbpQNys4Eruq77K6ojktoOedn/XlxdNkqVOD7MnxaufA2h3GIqcLxyrhh9wXylahlwvzjbG9sgLgkMGWKCIQbvZV8LEPVOZVp4K7tpw4DNjPjyxo47Vwy1hy4uDwNT96qynbSU4FDmzSO7Rid8IrtfLUD14aoYm7ufEKFkDjHrBv9ROBViafM9Wij5NH8fXdYtF10P08GGs+vZ5K9IK7Q04GcqpDUIJwPu/n1t4MD06BY9+dBPMArSAzUpUisx84//8/BijSlPGOYBNKD/JWCIHpt8MmW4= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(376002)(39860400002)(396003)(136003)(451199021)(40470700004)(46966006)(36840700001)(2906002)(40460700003)(36756003)(54906003)(478600001)(26005)(8936002)(2616005)(4744005)(81166007)(47076005)(356005)(82740400003)(44832011)(16526019)(1076003)(7416002)(86362001)(5660300002)(186003)(40480700001)(336012)(426003)(36860700001)(70586007)(8676002)(70206006)(4326008)(82310400005)(110136005)(7696005)(316002)(41300700001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Apr 2023 18:03:54.9028 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2910ac8b-68a9-4e8f-cbd2-08db3ab71d62 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT053.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB6988 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org The CXL driver plans to use cper_print_aer() for restricted CXL host (RCH) logging. cper_print_aer() is not exported and as a result is not available to the CXL driver or other loadable modules. Export cper_print_aer() making it available to CXL and other loadable modules. Signed-off-by: Terry Bowman Cc: Mahesh J Salgaonkar Cc: "Oliver O'Halloran" Cc: Bjorn Helgaas Cc: linux-pci@vger.kernel.org Reviewed-by: Jonathan Cameron --- drivers/pci/pcie/aer.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index f6c24ded134c..7a25b62d9e01 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -812,6 +812,7 @@ void cper_print_aer(struct pci_dev *dev, int aer_severity, trace_aer_event(dev_name(&dev->dev), (status & ~mask), aer_severity, tlp_header_valid, &aer->header_log); } +EXPORT_SYMBOL_GPL(cper_print_aer); #endif /** From patchwork Tue Apr 11 18:03:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13208014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E5E4C7619A for ; Tue, 11 Apr 2023 18:05:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230215AbjDKSFi (ORCPT ); Tue, 11 Apr 2023 14:05:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49182 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229626AbjDKSFh (ORCPT ); Tue, 11 Apr 2023 14:05:37 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on20601.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eae::601]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85D3D7684; Tue, 11 Apr 2023 11:04:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=d0N5+CehIIaHIi42c26eKqgJsQXXriAZRJukJ3pb/hexa5xonFKlgMK4thOHwB/LFAVss0PJB+Z0fYZPAMa98vRZlz5ZODxo9PMIVKolij1qHR6gpDRvoSI9zboWHjPQnEe9bcic7gc52c8Ui71h65GLDbetSh3WpUDzlCJuhDjg/Doovfab2aCrvr88YDJkxzaH+aR/JlPNkG4GOvWaD5dKGsjlhc1Lgv1hGNod0IcsVMsw7y3e48XLU7BoGGxrP8sHLS2BBCR7wxwtMnM1FjO0DTo6lm+Eyaw0cYdCVfFQZxUWJwnxiTSYeoMrOK0QDRW3bNHp6Ww9hCnNq4Jfow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+HNEpXsg4Vtf2TLjMwcMF50f0xfm/vpo555pR2k8lXk=; b=mwtG1xQH+sniPDj5YBAUC8+jl+fxwueqYlUsAkJFAQDCCn+xneTjq2ktSwHKIy+QOEG9UYcQh3CfmuZXcCwAhOAZZB63EQM6H2CyIONhLuwgoGGRyHLDjO2zoycEVVvark04j81+qvTFYNPlc/9EUaxXnslDH3u0mdeqWq8mUcQiFKjACQfXXZp18Ey2mFM8XuQiUH5OlkNkBQIoB5kqI/gl4z8k/NfGDrk3naYWUMYQXrE8KYmrrSP+nGu6x+6wGQpcAPWJszqLwqGv55/Fqpa4iA8WeiEfFVPr10QUnlZES0aiglCh5y8ZgegT68+9ZxHkJFRBcVGh3G9hFdLC2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+HNEpXsg4Vtf2TLjMwcMF50f0xfm/vpo555pR2k8lXk=; b=LqNBbxNoRPoPxzIn9m9yL2a8TsUSjoiEPET5Q7Qa4BHNsye3HuCc0pYArOpiNvQ8bWOb9BU/hlaqSeSgc8Og8OTTRux6TIbEllJkEfeu9lQ5l4L8/KYi9cDKxgD7SFThgjZeB/bmeIGhneMEEfCctcO+E1136bgV4sZjuFvMmRo= Received: from BN9PR03CA0158.namprd03.prod.outlook.com (2603:10b6:408:f4::13) by SA3PR12MB7858.namprd12.prod.outlook.com (2603:10b6:806:306::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.38; Tue, 11 Apr 2023 18:04:16 +0000 Received: from BN8NAM11FT037.eop-nam11.prod.protection.outlook.com (2603:10b6:408:f4:cafe::8) by BN9PR03CA0158.outlook.office365.com (2603:10b6:408:f4::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.40 via Frontend Transport; Tue, 11 Apr 2023 18:04:16 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT037.mail.protection.outlook.com (10.13.177.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6298.28 via Frontend Transport; Tue, 11 Apr 2023 18:04:16 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 11 Apr 2023 13:04:15 -0500 From: Terry Bowman To: , , , , , , , CC: , , , Subject: [PATCH v3 4/6] cxl/pci: Add RCH downstream port error logging Date: Tue, 11 Apr 2023 13:03:00 -0500 Message-ID: <20230411180302.2678736-5-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230411180302.2678736-1-terry.bowman@amd.com> References: <20230411180302.2678736-1-terry.bowman@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT037:EE_|SA3PR12MB7858:EE_ X-MS-Office365-Filtering-Correlation-Id: b0f7c8db-ed4e-4a2b-3a56-08db3ab72a13 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: h3ORJ36bD2zMhLj9KyNfnogWptBCoN7eT6i/f7qkoehkgPlSNyedEsjQuvPot1CaxeMB9CTJzOHgo+ACxQUSF1fg57bL0vfax1c0S7Ga69XwfBOcdcQtaPOt+lOscSOQ4X5+xAqikB/Wb9Cd7B4fXjYfQJscu6x8cFzGAyFzCKwBuTQ5EVV4B8PntqW6ND7+m4NZ5Htcp7DbYeuQ+mx411rGpU3jH49Cd/kkEJy8OoGnnG8bTIMdXOL+OEV7UZ1Z1Z7IV1apJsDlA9+fwAWokJ9SijdOAhgOTItQag+NFe+ZDpe6X6G14MxyCguAoOxeD4FHIbu09PaHTeTjiI3sn32Kq5v6XFEdujXxq4fsRMLX5NHczFb4fhsC+In9Hruu+E2Kux3o/q8hSOZmOrS3g21ZhtCQWuO8jAyDFb79hvKf9Ovni8k//ODzaoz9PYYAlu7RU+z8gnDHkDO4iIGDvzuoUDTPC2jIMpZRjLpn/QjXXMuUglt+esIxgyuAwYXZ156K8rP9FySpkuYyTFk4wkGJbaUsmJCyEcZ+CIUH9qCkMu9C+pv9dKizh/BTu7TeUl2qcvTLH193wEM6nQ+By7jkqLEbIWX6R+Is6XenYQp3/f48EzMn0DKzCIXRwr2Djs2OzxZSiOC+T6Q8PQKJ95cOYkiGrFQ4BBqiVJ/JGmgxMaHYl5I0AOlrw8xaPjVtiF+rV73C2XLcRT6WQAVwfwJ739+MeOODrIkjR+BCXgw= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(396003)(376002)(346002)(136003)(39860400002)(451199021)(40470700004)(36840700001)(46966006)(478600001)(7696005)(316002)(1076003)(110136005)(26005)(54906003)(186003)(16526019)(5660300002)(6666004)(30864003)(2906002)(44832011)(70206006)(70586007)(4326008)(7416002)(41300700001)(8676002)(8936002)(82310400005)(356005)(81166007)(82740400003)(40460700003)(40480700001)(47076005)(83380400001)(36756003)(2616005)(336012)(36860700001)(426003)(86362001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Apr 2023 18:04:16.1917 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b0f7c8db-ed4e-4a2b-3a56-08db3ab72a13 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT037.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB7858 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org RCH downstream port error logging is missing in the current CXL driver. The missing AER and RAS error logging is needed for communicating driver error details to userspace. Update the driver to include PCIe AER and CXL RAS error logging. Add RCH downstream port error handling into the existing RCiEP handler. The downstream port error handler is added to the RCiEP error handler because the downstream port is implemented in a RCRB, is not PCI enumerable, and as a result is not directly accessible to the PCI AER root port driver. The AER root port driver calls the RCiEP handler for handling RCD errors and RCH downstream port protocol errors. Update mem.c to include RAS and AER setup. This includes AER and RAS capability discovery and mapping for later use in the error handler. Disable RCH downstream port's root port cmd interrupts.[1] Update existing RCiEP correctable and uncorrectable handlers to also call the RCH handler. The RCH handler will read the RCH AER registers, check for error severity, and if an error exists will log using an existing kernel AER trace routine. The RCH handler will also log downstream port RAS errors if they exist. [1] CXL 3.0 Spec, 12.2.1.1 - RCH Downstream Port Detected Errors Co-developed-by: Robert Richter Signed-off-by: Robert Richter Signed-off-by: Terry Bowman --- drivers/cxl/core/pci.c | 126 ++++++++++++++++++++++++++++++++++++---- drivers/cxl/core/regs.c | 1 + drivers/cxl/cxl.h | 13 +++++ drivers/cxl/mem.c | 73 +++++++++++++++++++++++ 4 files changed, 201 insertions(+), 12 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 523d5b9fd7fc..d435ed2ff8b6 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -613,32 +614,88 @@ void read_cdat_data(struct cxl_port *port) } EXPORT_SYMBOL_NS_GPL(read_cdat_data, CXL); -void cxl_cor_error_detected(struct pci_dev *pdev) +/* Get AER severity. Return false if there is no error. */ +static bool cxl_rch_get_aer_severity(struct aer_capability_regs *aer_regs, + int *severity) +{ + if (aer_regs->uncor_status & ~aer_regs->uncor_mask) { + if (aer_regs->uncor_status & PCI_ERR_ROOT_FATAL_RCV) + *severity = AER_FATAL; + else + *severity = AER_NONFATAL; + return true; + } + + if (aer_regs->cor_status & ~aer_regs->cor_mask) { + *severity = AER_CORRECTABLE; + return true; + } + + return false; +} + +/* + * Copy the AER capability registers to a buffer. This is necessary + * because RCRB AER capability is MMIO mapped. Clear the status + * after copying. + * + * @aer_base: base address of AER capability block in RCRB + * @aer_regs: destination for copying AER capability + */ +static bool cxl_rch_get_aer_info(void __iomem *aer_base, + struct aer_capability_regs *aer_regs) +{ + int read_cnt = PCI_AER_CAPABILITY_LENGTH / sizeof(u32); + u32 *aer_regs_buf = (u32 *)aer_regs; + int n; + + if (!aer_base) + return false; + + for (n = 0; n < read_cnt; n++) + aer_regs_buf[n] = readl(aer_base + n * sizeof(u32)); + + writel(aer_regs->uncor_status, aer_base + PCI_ERR_UNCOR_STATUS); + writel(aer_regs->cor_status, aer_base + PCI_ERR_COR_STATUS); + + return true; +} + +static void __cxl_log_correctable_ras(struct cxl_dev_state *cxlds, + void __iomem *ras_base) { - struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); void __iomem *addr; u32 status; - if (!cxlds->regs.ras) + if (!ras_base) return; - addr = cxlds->regs.ras + CXL_RAS_CORRECTABLE_STATUS_OFFSET; + addr = ras_base + CXL_RAS_CORRECTABLE_STATUS_OFFSET; status = readl(addr); if (status & CXL_RAS_CORRECTABLE_STATUS_MASK) { writel(status & CXL_RAS_CORRECTABLE_STATUS_MASK, addr); trace_cxl_aer_correctable_error(cxlds->cxlmd, status); } } -EXPORT_SYMBOL_NS_GPL(cxl_cor_error_detected, CXL); + +static void cxl_log_correctable_ras_endpoint(struct cxl_dev_state *cxlds) +{ + return __cxl_log_correctable_ras(cxlds, cxlds->regs.ras); +} + +static void cxl_log_correctable_ras_dport(struct cxl_dev_state *cxlds) +{ + return __cxl_log_correctable_ras(cxlds, cxlds->regs.dport_ras); +} /* CXL spec rev3.0 8.2.4.16.1 */ -static void header_log_copy(struct cxl_dev_state *cxlds, u32 *log) +static void header_log_copy(void __iomem *ras_base, u32 *log) { void __iomem *addr; u32 *log_addr; int i, log_u32_size = CXL_HEADERLOG_SIZE / sizeof(u32); - addr = cxlds->regs.ras + CXL_RAS_HEADER_LOG_OFFSET; + addr = ras_base + CXL_RAS_HEADER_LOG_OFFSET; log_addr = log; for (i = 0; i < log_u32_size; i++) { @@ -652,17 +709,18 @@ static void header_log_copy(struct cxl_dev_state *cxlds, u32 *log) * Log the state of the RAS status registers and prepare them to log the * next error status. Return 1 if reset needed. */ -static bool cxl_report_and_clear(struct cxl_dev_state *cxlds) +static bool __cxl_report_and_clear(struct cxl_dev_state *cxlds, + void __iomem *ras_base) { u32 hl[CXL_HEADERLOG_SIZE_U32]; void __iomem *addr; u32 status; u32 fe; - if (!cxlds->regs.ras) + if (!ras_base) return false; - addr = cxlds->regs.ras + CXL_RAS_UNCORRECTABLE_STATUS_OFFSET; + addr = ras_base + CXL_RAS_UNCORRECTABLE_STATUS_OFFSET; status = readl(addr); if (!(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK)) return false; @@ -670,7 +728,7 @@ static bool cxl_report_and_clear(struct cxl_dev_state *cxlds) /* If multiple errors, log header points to first error from ctrl reg */ if (hweight32(status) > 1) { void __iomem *rcc_addr = - cxlds->regs.ras + CXL_RAS_CAP_CONTROL_OFFSET; + ras_base + CXL_RAS_CAP_CONTROL_OFFSET; fe = BIT(FIELD_GET(CXL_RAS_CAP_CONTROL_FE_MASK, readl(rcc_addr))); @@ -678,13 +736,54 @@ static bool cxl_report_and_clear(struct cxl_dev_state *cxlds) fe = status; } - header_log_copy(cxlds, hl); + header_log_copy(ras_base, hl); trace_cxl_aer_uncorrectable_error(cxlds->cxlmd, status, fe, hl); writel(status & CXL_RAS_UNCORRECTABLE_STATUS_MASK, addr); return true; } +static bool cxl_report_and_clear(struct cxl_dev_state *cxlds) +{ + return __cxl_report_and_clear(cxlds, cxlds->regs.ras); +} + +static bool cxl_report_and_clear_dport(struct cxl_dev_state *cxlds) +{ + return __cxl_report_and_clear(cxlds, cxlds->regs.dport_ras); +} + +static void cxl_rch_log_error(struct cxl_dev_state *cxlds) +{ + struct pci_dev *pdev = to_pci_dev(cxlds->dev); + struct aer_capability_regs aer_regs; + int severity; + + if (!cxl_rch_get_aer_info(cxlds->regs.aer, &aer_regs)) + return; + + if (!cxl_rch_get_aer_severity(&aer_regs, &severity)) + return; + + cper_print_aer(pdev, severity, &aer_regs); + + if (severity == AER_CORRECTABLE) + cxl_log_correctable_ras_dport(cxlds); + else + cxl_report_and_clear_dport(cxlds); +} + +void cxl_cor_error_detected(struct pci_dev *pdev) +{ + struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); + + if (cxlds->rcd) + cxl_rch_log_error(cxlds); + + cxl_log_correctable_ras_endpoint(cxlds); +} +EXPORT_SYMBOL_NS_GPL(cxl_cor_error_detected, CXL); + pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, pci_channel_state_t state) { @@ -693,6 +792,9 @@ pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, struct device *dev = &cxlmd->dev; bool ue; + if (cxlds->rcd) + cxl_rch_log_error(cxlds); + /* * A frozen channel indicates an impending reset which is fatal to * CXL.mem operation, and will likely crash the system. On the off diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c index bde1fffab09e..dfa6fcfc428a 100644 --- a/drivers/cxl/core/regs.c +++ b/drivers/cxl/core/regs.c @@ -198,6 +198,7 @@ void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr, return ret_val; } +EXPORT_SYMBOL_NS_GPL(devm_cxl_iomap_block, CXL); int cxl_map_component_regs(struct device *dev, struct cxl_component_regs *regs, struct cxl_register_map *map, unsigned long map_mask) diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index df64c402e6e6..dae3f141ffcb 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -66,6 +66,8 @@ #define CXL_DECODER_MIN_GRANULARITY 256 #define CXL_DECODER_MAX_ENCODED_IG 6 +#define PCI_AER_CAPABILITY_LENGTH 56 + static inline int cxl_hdm_decoder_count(u32 cap_hdr) { int val = FIELD_GET(CXL_HDM_DECODER_COUNT_MASK, cap_hdr); @@ -209,6 +211,15 @@ struct cxl_regs { struct_group_tagged(cxl_device_regs, device_regs, void __iomem *status, *mbox, *memdev; ); + + /* + * Pointer to RCH cxl_dport AER. (only for RCH/RCD mode) + * @dport_aer: CXL 2.0 12.2.11 RCH Downstream Port-detected Errors + */ + struct_group_tagged(cxl_rch_regs, rch_regs, + void __iomem *aer; + void __iomem *dport_ras; + ); }; struct cxl_reg_map { @@ -249,6 +260,8 @@ struct cxl_register_map { }; }; +void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr, + resource_size_t length); void cxl_probe_component_regs(struct device *dev, void __iomem *base, struct cxl_component_reg_map *map); void cxl_probe_device_regs(struct device *dev, void __iomem *base, diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index 014295ab6bc6..dd5ae0a4560c 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -4,6 +4,7 @@ #include #include #include +#include #include "cxlmem.h" #include "cxlpci.h" @@ -45,6 +46,71 @@ static int cxl_mem_dpa_show(struct seq_file *file, void *data) return 0; } +static void rch_disable_root_ints(void __iomem *aer_base) +{ + u32 aer_cmd_mask, aer_cmd; + + /* + * Disable RCH root port command interrupts. + * CXL3.0 12.2.1.1 - RCH Downstream Port-detected Errors + */ + aer_cmd_mask = (PCI_ERR_ROOT_CMD_COR_EN | + PCI_ERR_ROOT_CMD_NONFATAL_EN | + PCI_ERR_ROOT_CMD_FATAL_EN); + aer_cmd = readl(aer_base + PCI_ERR_ROOT_COMMAND); + aer_cmd &= ~aer_cmd_mask; + writel(aer_cmd, aer_base + PCI_ERR_ROOT_COMMAND); +} + +static int cxl_rch_map_ras(struct cxl_dev_state *cxlds, + struct cxl_dport *parent_dport) +{ + struct device *dev = parent_dport->dport; + resource_size_t aer_phys, ras_phys; + void __iomem *aer, *dport_ras; + + if (!parent_dport->rch) + return 0; + + if (!parent_dport->aer_cap || !parent_dport->ras_cap || + parent_dport->component_reg_phys == CXL_RESOURCE_NONE) + return -ENODEV; + + aer_phys = parent_dport->aer_cap + parent_dport->rcrb; + aer = devm_cxl_iomap_block(dev, aer_phys, + PCI_AER_CAPABILITY_LENGTH); + + if (!aer) + return -ENOMEM; + + ras_phys = parent_dport->ras_cap + parent_dport->component_reg_phys; + dport_ras = devm_cxl_iomap_block(dev, ras_phys, + CXL_RAS_CAPABILITY_LENGTH); + + if (!dport_ras) + return -ENOMEM; + + cxlds->regs.aer = aer; + cxlds->regs.dport_ras = dport_ras; + + return 0; +} + +static int cxl_setup_ras(struct cxl_dev_state *cxlds, + struct cxl_dport *parent_dport) +{ + int rc; + + rc = cxl_rch_map_ras(cxlds, parent_dport); + if (rc) + return rc; + + if (cxlds->rcd) + rch_disable_root_ints(cxlds->regs.aer); + + return rc; +} + static void cxl_setup_rcrb(struct cxl_dev_state *cxlds, struct cxl_dport *parent_dport) { @@ -91,6 +157,13 @@ static int devm_cxl_add_endpoint(struct device *host, struct cxl_memdev *cxlmd, cxl_setup_rcrb(cxlds, parent_dport); + rc = cxl_setup_ras(cxlds, parent_dport); + /* Continue with RAS setup errors */ + if (rc) + dev_warn(&cxlmd->dev, "CXL RAS setup failed: %d\n", rc); + else + dev_info(&cxlmd->dev, "CXL error handling enabled\n"); + endpoint = devm_cxl_add_port(host, &cxlmd->dev, cxlds->component_reg_phys, parent_dport); if (IS_ERR(endpoint)) From patchwork Tue Apr 11 18:03:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13208015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 605AEC7619A for ; Tue, 11 Apr 2023 18:06:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230281AbjDKSG4 (ORCPT ); Tue, 11 Apr 2023 14:06:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51312 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230300AbjDKSGz (ORCPT ); Tue, 11 Apr 2023 14:06:55 -0400 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2060c.outbound.protection.outlook.com [IPv6:2a01:111:f400:7e8a::60c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0CFF36E86; Tue, 11 Apr 2023 11:06:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=M46oy+YV3e7UEe1zBeJwESept1jLMuX0htCKf7Z2bxJ6/s3cP2PIOGcxb4JFQiKrkxtSHHJrBmmz6mfSdo6TjoWDH/H1LC/eNGrn+L9mO/Z9n5MwBDXBbNcz5GLjHcChYxG/ONj0jjWiWjrAO9PSSC1nTLnzGBTPV28KRBX2TrzrpVyBMdoOluO8DxIq4djheEn06X4AjqJgjV2j2OmMvyNRjKE7vtlqQ+sKWlYV39rAFbMVmC/pau7/H9WueGNVP94ZXL6qOFPjo+XnDN4lRWRXsDZbYuaQrKrSRf4yhGE6f1Ch0fWD1BDFJ0htApfrMN9iRVYN7qqBW1xZqbo+2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=L4ZCRrCWSpGRon5DPUy2i9yAnHrELFU55+UIaeoiKNk=; b=n2tpFdonLsYRsMPKB9iKtOPHh8TMrMw8EPDgenFmYFZ4FBdMi8DQqV+145mhf1TkddVqkpif6EASi9rMCH3jljTC9tEXS3GCAjojXT4FE8xQEewLyfcRq9b0mnO3XqW6zkuDTxL1RaVnwPKxzad2jnkVPNRsKrgs/5TwxW3v+HVwO4PntDoTd4r8+9O1UebAXljpvVQcwp782P4knHET1PbqS/DjZsU30OjktMP0OUnlzB3/FtQtwu0lhKgxSswguyJy09yyj0nSuqu/TNN8v2sa2boGSg8Aso2syh8ZnuAwGdDLTO7hU1OTch/6XZELn+KwxlmqAg2NQH5jSAaT/g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L4ZCRrCWSpGRon5DPUy2i9yAnHrELFU55+UIaeoiKNk=; b=GuKX/iEP5UMHAlZgtoehfc5/yfO+DpYWb2dAuN3VF5yI3KRas25NVkTVxqLK82YczL+hzrnhmalaqPM5nvHcnMAJeZ8m+u35V43rdbMBkHFLHqCtA7PymcsUFbTPcf2RluibNGBO8zX35SrAGwzFsUgXCxxzLggYAgJFHRXwRvs= Received: from BN9P223CA0030.NAMP223.PROD.OUTLOOK.COM (2603:10b6:408:10b::35) by BY5PR12MB4131.namprd12.prod.outlook.com (2603:10b6:a03:212::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.34; Tue, 11 Apr 2023 18:04:37 +0000 Received: from BN8NAM11FT105.eop-nam11.prod.protection.outlook.com (2603:10b6:408:10b:cafe::4b) by BN9P223CA0030.outlook.office365.com (2603:10b6:408:10b::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.40 via Frontend Transport; Tue, 11 Apr 2023 18:04:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT105.mail.protection.outlook.com (10.13.176.183) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6298.26 via Frontend Transport; Tue, 11 Apr 2023 18:04:36 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 11 Apr 2023 13:04:35 -0500 From: Terry Bowman To: , , , , , , , CC: , , , , Oliver O'Halloran , Mahesh J Salgaonkar , , Subject: [PATCH v3 5/6] PCI/AER: Forward RCH downstream port-detected errors to the CXL.mem dev handler Date: Tue, 11 Apr 2023 13:03:01 -0500 Message-ID: <20230411180302.2678736-6-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230411180302.2678736-1-terry.bowman@amd.com> References: <20230411180302.2678736-1-terry.bowman@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT105:EE_|BY5PR12MB4131:EE_ X-MS-Office365-Filtering-Correlation-Id: cfb8d8ab-f159-4f87-e6e8-08db3ab73643 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0SGQNGoS+upoqTqw25eWvqc01KDWnJIhSbtrpBrny5SAnUnv0MsVG0IJBWafPX7dF1H5M887rhrOLFC2qMrVxyYGUWSet+Od+9IRQbrn2DPs9mP+g7JmvMoDd5CAPHHYdrhn/0/PlZDlLatMD9V5WeXZeLNuJ3DRvW9Qx1KrH0hQFuyB6uMT/MQAX2kn162lyrsw/o6hA9qtiRLUeSgpfV5eGfO3UbO4YlyCsJgiHkH8U3Nu9fNpkBEWNjSNtdJHL9v0pQab9X8pnxaKoSUbVIUa8ueh8GeV0j9B8qLpkPasXst4evuSswNSYeyYu8EWzcSmT8ctourzLRpNcW7eo3wupFY9YzwztnOf0Isqp2+NFdYqdQtM9JOXg9gkkQFVkqH9yXZ+Ewkq1/UUsWaKAfbIrdXkcAtKaJK5sIOawoelIRc0xozrDeB2P3RXmkUj9RApJxOPlZ8oEYQ61u0N2NZOfBusBcAORGPaTecs5DH3zHJI8FnDH3jEVhoq1amMtMYVO9A9fkKQjZIltrG/gYwNlbDhza8ZMqJysGI2/26NytRnLDMf5a2Abg/+RVa1a+Fd+EdZhwX2itgVIQH0/eBF3HZrKLqY7SrYJWJstBIs4DoGRgmRH6VKE3IiaE4G+PA8auXQM6QMpy3tcxmRTXJrVRn+ILH+vhUOb5pbB/gs9w+JJI4REJ2WAdK/DDjetARroK5aB32tmQF4mk2UOC/sJW9lwmKoYh0kOsnhRoE= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(376002)(136003)(396003)(39860400002)(451199021)(46966006)(40470700004)(36840700001)(8936002)(966005)(40480700001)(41300700001)(40460700003)(7696005)(426003)(2616005)(336012)(83380400001)(47076005)(316002)(82310400005)(110136005)(82740400003)(54906003)(36860700001)(81166007)(356005)(36756003)(44832011)(6666004)(5660300002)(7416002)(16526019)(8676002)(4326008)(478600001)(186003)(86362001)(70206006)(1076003)(70586007)(26005)(2906002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Apr 2023 18:04:36.6434 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cfb8d8ab-f159-4f87-e6e8-08db3ab73643 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT105.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4131 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Robert Richter In Restricted CXL Device (RCD) mode a CXL device is exposed as an RCiEP, but CXL downstream and upstream ports are not enumerated and not visible in the PCIe hierarchy. Protocol and link errors are sent to an RCEC. Restricted CXL host (RCH) downstream port-detected errors are signaled as internal AER errors, either Uncorrectable Internal Error (UIE) or Corrected Internal Errors (CIE). The error source is the id of the RCEC. A CXL handler must then inspect the error status in various CXL registers residing in the dport's component register space (CXL RAS cap) or the dport's RCRB (AER ext cap). [1] Errors showing up in the RCEC's error handler must be handled and connected to the CXL subsystem. Implement this by forwarding the error to all CXL devices below the RCEC. Since the entire CXL device is controlled only using PCIe Configuration Space of device 0, Function 0, only pass it there [2]. These devices have the Memory Device class code set (PCI_CLASS_MEMORY_CXL, 502h) and the existing cxl_pci driver can implement the handler. In addition to errors directed to the CXL endpoint device, the handler must also inspect the CXL downstream port's CXL RAS and PCIe AER external capabilities that is connected to the device. Since CXL downstream port errors are signaled using internal errors, the handler requires those errors to be unmasked. This is subject of a follow-on patch. The reason for choosing this implementation is that a CXL RCEC device is bound to the AER port driver, but the driver does not allow it to register a custom specific handler to support CXL. Connecting the RCEC hard-wired with a CXL handler does not work, as the CXL subsystem might not be present all the time. The alternative to add an implementation to the portdrv to allow the registration of a custom RCEC error handler isn't worth doing it as CXL would be its only user. Instead, just check for an CXL RCEC and pass it down to the connected CXL device's error handler. With this approach the code can entirely be implemented in the PCIe AER driver and is independent of the CXL subsystem. The CXL driver only provides the handler. [1] CXL 3.0 spec, 12.2.1.1 RCH Downstream Port-detected Errors [2] CXL 3.0 spec, 8.1.3 PCIe DVSEC for CXL Devices Co-developed-by: Terry Bowman Signed-off-by: Robert Richter Signed-off-by: Terry Bowman Cc: "Oliver O'Halloran" Cc: Bjorn Helgaas Cc: Mahesh J Salgaonkar Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-pci@vger.kernel.org --- drivers/pci/pcie/Kconfig | 8 ++++++ drivers/pci/pcie/aer.c | 61 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) diff --git a/drivers/pci/pcie/Kconfig b/drivers/pci/pcie/Kconfig index 228652a59f27..b0dbd864d3a3 100644 --- a/drivers/pci/pcie/Kconfig +++ b/drivers/pci/pcie/Kconfig @@ -49,6 +49,14 @@ config PCIEAER_INJECT gotten from: https://git.kernel.org/cgit/linux/kernel/git/gong.chen/aer-inject.git/ +config PCIEAER_CXL + bool "PCI Express CXL RAS support" + default y + depends on PCIEAER && CXL_PCI + help + This enables CXL error handling for Restricted CXL Hosts + (RCHs). + # # PCI Express ECRC # diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 7a25b62d9e01..171a08fd8ebd 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -946,6 +946,65 @@ static bool find_source_device(struct pci_dev *parent, return true; } +#ifdef CONFIG_PCIEAER_CXL + +static bool is_cxl_mem_dev(struct pci_dev *dev) +{ + /* + * A CXL device is controlled only using PCIe Configuration + * Space of device 0, Function 0. + */ + if (dev->devfn != PCI_DEVFN(0, 0)) + return false; + + /* Right now there is only a CXL.mem driver */ + if ((dev->class >> 8) != PCI_CLASS_MEMORY_CXL) + return false; + + return true; +} + +static bool is_internal_error(struct aer_err_info *info) +{ + if (info->severity == AER_CORRECTABLE) + return info->status & PCI_ERR_COR_INTERNAL; + + return info->status & PCI_ERR_UNC_INTN; +} + +static void handle_error_source(struct pci_dev *dev, struct aer_err_info *info); + +static int cxl_handle_error_iter(struct pci_dev *dev, void *data) +{ + struct aer_err_info *e_info = (struct aer_err_info *)data; + + if (!is_cxl_mem_dev(dev)) + return 0; + + /* pci_dev_put() in handle_error_source() */ + dev = pci_dev_get(dev); + if (dev) + handle_error_source(dev, e_info); + + return 0; +} + +static void cxl_handle_error(struct pci_dev *dev, struct aer_err_info *info) +{ + /* + * CXL downstream port errors are signaled as RCEC internal + * errors. Forward them to all CXL devices below the RCEC. + */ + if (pci_pcie_type(dev) == PCI_EXP_TYPE_RC_EC && + is_internal_error(info)) + pcie_walk_rcec(dev, cxl_handle_error_iter, info); +} + +#else +static inline void cxl_handle_error(struct pci_dev *dev, + struct aer_err_info *info) { } +#endif + /** * handle_error_source - handle logging error into an event log * @dev: pointer to pci_dev data structure of error source device @@ -957,6 +1016,8 @@ static void handle_error_source(struct pci_dev *dev, struct aer_err_info *info) { int aer = dev->aer_cap; + cxl_handle_error(dev, info); + if (info->severity == AER_CORRECTABLE) { /* * Correctable error does not need software intervention. From patchwork Tue Apr 11 18:03:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Bowman, Terry" X-Patchwork-Id: 13208016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A590C7619A for ; Tue, 11 Apr 2023 18:08:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230358AbjDKSIW (ORCPT ); Tue, 11 Apr 2023 14:08:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230359AbjDKSIO (ORCPT ); Tue, 11 Apr 2023 14:08:14 -0400 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2074.outbound.protection.outlook.com [40.107.212.74]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D7586E90; Tue, 11 Apr 2023 11:08:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VIjwjJSQA1vN8iO0862himxM4p6zRzygQnvCL2KVDgqbgF6sMCdPY6ChV4+G6PcSH7TXNcSKMj9BqquJwSl8r4u0ajL3dKEQ3M3OJfSPhohT5XeXNky1BlDQobbiBd4Wzc6oi5z8OvpCg+6Fe9HsYdH6cznmrbl1ZL8p0+rmB8TeaqqZrA5pBetiAjopCo5RVQ/OUFOs53DIyt7r9cJvinxSBvaPtdeB8vPtw0IedTltFcz8blPSXbQLH8FEYXn1ZLOkKAde9AmWreaGKd2Lb6X57LBlkbNfp2iSKwNNBv3vdQX3fCZjHkNXmLFUlvD2tmraXVCG8UFwtGsnCRVWIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=o2zFiMs2oIqp2Cdsmd91Xau6Ybwk/eHR8N/sswJ3q5w=; b=QPeEBp1J76sxjxDvvebDyUE6dDKNVS3ZyMM57FqLAzADMOx/jbvk21wgGY6/trehTcfkEcDPQmkJl84WpXt5VheVvdyqBlqXIO+WCDI4nXYTYSpqLWc7nOVBWjYwGRf+UUTMAqhJ5jfacw4f/pndTVQ3dg4G88VmVuxkujgBQZrySCf9UxFM/tkVw5y0E3ffL+HVHgprOO/7bR6AtwotOM1NM2LU53QMT/tFLnRj2sre56640ztjtPUFqrn7VvGBimGXbkv1oQUtnKqhqvxrqVyf8bpngE6mVqbsHwbjYaEhMstV59yWExbiv+fNRvrjVGq/VcxGVQtoIho0LdT9ZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=o2zFiMs2oIqp2Cdsmd91Xau6Ybwk/eHR8N/sswJ3q5w=; b=e+LB4dOWTtUba/lCdS/fki7n7bXhzeYN8XSppVLF7hI/MjlCJNrY+q4AGLYKh2xjMv/6aG04mY1RomA+79QXqXKLVP3aNdREgAAN0K03wvVDiKbu8JJNzO/xJIjUEt4TxD1Bb+Zl7hxcOLz0ZaN891BuWgueytzfrfjyAd0eiSY= Received: from BN9PR03CA0696.namprd03.prod.outlook.com (2603:10b6:408:ef::11) by MN0PR12MB6032.namprd12.prod.outlook.com (2603:10b6:208:3cc::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.34; Tue, 11 Apr 2023 18:04:49 +0000 Received: from BN8NAM11FT114.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ef:cafe::76) by BN9PR03CA0696.outlook.office365.com (2603:10b6:408:ef::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.40 via Frontend Transport; Tue, 11 Apr 2023 18:04:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT114.mail.protection.outlook.com (10.13.177.46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6298.28 via Frontend Transport; Tue, 11 Apr 2023 18:04:49 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Tue, 11 Apr 2023 13:04:48 -0500 From: Terry Bowman To: , , , , , , , CC: , , , , Oliver O'Halloran , Mahesh J Salgaonkar , , Subject: [PATCH v3 6/6] PCI/AER: Unmask RCEC internal errors to enable RCH downstream port error handling Date: Tue, 11 Apr 2023 13:03:02 -0500 Message-ID: <20230411180302.2678736-7-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230411180302.2678736-1-terry.bowman@amd.com> References: <20230411180302.2678736-1-terry.bowman@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT114:EE_|MN0PR12MB6032:EE_ X-MS-Office365-Filtering-Correlation-Id: ad78fed2-a404-40a9-905f-08db3ab73e0c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4/c+xi/etY6MLRknICeekGbnE6bJxjNwp6pnt699UyTwdTs6N4zTSLLZav23gOPlFY0OJFZlksGJ4WeL04X3kAaE/mW3DFu2uVDWeftbFegM8swQ75SbvjitlrHq7czWB0IcgC5VmfqkBWr6KU8nPvsiwcEH+D9JyszdZlItcmdtYszyFHINa28INo+4vI0RxGzRLQPTEBUcRBfoAco5X8hIihmlVw8D6nl5C5W82B0xdKjDJdaBdluY5txM+5G9tUHdN67/kaSeYKoaJxV5fmVRfkaOpLmwd5zC4gw28+O/AmNWGbXFXI09TNlkbMW6go+LEwE/fmk9oGdnknHu7UwZtlc1am6VKYqfBzKR9QAtYG9qx3wh36E+63ysNiEFVcsrLdYIYVS26ciFYSRo+UOSvckLYrNvBXthggOomfCgsovrgwGBd3Aa2EEblWkuplsJalj4uQlox7lTYfIwx3QnDCszseAnFeEB8kY+rC30HhmGOicRc1KapYG6iGh5qr1g/rfgWM1vFSIOQThYJ5LQSEd6LCyBYYjrOPLK/NFUUX35TxRlqVORcYPlTpjHEX+C45xvXZqusK2D7rsoW2SgTU12ExvAwUbm+CP+2mp6P6H46Ja3RLbX1W5Sh1nEZWhSqgM2u91DE1dN4zQJ3+2eTe/jUnfoi2padZbxa1orT96pFRiG6ghZ1UC3ZTom90ucwBbmVDmget8FWesMX6ZI+zOvHGD/WjYem3kOHVn0vPrTA04JBdTz5ndraZ0kEilaiCF84ierqoYqB05T6A== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(136003)(39860400002)(396003)(376002)(346002)(451199021)(46966006)(40470700004)(36840700001)(40460700003)(426003)(2906002)(26005)(336012)(16526019)(186003)(1076003)(2616005)(70206006)(47076005)(83380400001)(41300700001)(70586007)(8676002)(54906003)(7416002)(478600001)(110136005)(5660300002)(8936002)(40480700001)(7696005)(44832011)(6666004)(316002)(4326008)(81166007)(356005)(36860700001)(82740400003)(36756003)(82310400005)(86362001)(36900700001)(309714004);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Apr 2023 18:04:49.6869 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ad78fed2-a404-40a9-905f-08db3ab73e0c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT114.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6032 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Robert Richter RCEC AER corrected and uncorrectable internal errors (CIE/UIE) are disabled by default. [1][2] Enable them to receive CXL downstream port errors of a Restricted CXL Host (RCH). [1] CXL 3.0 Spec, 12.2.1.1 - RCH Downstream Port Detected Errors [2] PCIe Base Spec 6.0, 7.8.4.3 Uncorrectable Error Mask Register, 7.8.4.6 Correctable Error Mask Register Co-developed-by: Terry Bowman Signed-off-by: Robert Richter Signed-off-by: Terry Bowman Cc: "Oliver O'Halloran" Cc: Bjorn Helgaas Cc: Mahesh J Salgaonkar Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-pci@vger.kernel.org --- drivers/pci/pcie/aer.c | 73 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index 171a08fd8ebd..3973c731e11d 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -1000,7 +1000,79 @@ static void cxl_handle_error(struct pci_dev *dev, struct aer_err_info *info) pcie_walk_rcec(dev, cxl_handle_error_iter, info); } +static bool cxl_error_is_native(struct pci_dev *dev) +{ + struct pci_host_bridge *host = pci_find_host_bridge(dev->bus); + + if (pcie_ports_native) + return true; + + return host->native_aer && host->native_cxl_error; +} + +static int handles_cxl_error_iter(struct pci_dev *dev, void *data) +{ + int *handles_cxl = data; + + *handles_cxl = is_cxl_mem_dev(dev) && cxl_error_is_native(dev); + + return *handles_cxl; +} + +static bool handles_cxl_errors(struct pci_dev *rcec) +{ + int handles_cxl = 0; + + if (!rcec->aer_cap) + return false; + + if (pci_pcie_type(rcec) == PCI_EXP_TYPE_RC_EC) + pcie_walk_rcec(rcec, handles_cxl_error_iter, &handles_cxl); + + return !!handles_cxl; +} + +static int __cxl_unmask_internal_errors(struct pci_dev *rcec) +{ + int aer, rc; + u32 mask; + + /* + * Internal errors are masked by default, unmask RCEC's here + * PCI6.0 7.8.4.3 Uncorrectable Error Mask Register (Offset 08h) + * PCI6.0 7.8.4.6 Correctable Error Mask Register (Offset 14h) + */ + aer = rcec->aer_cap; + rc = pci_read_config_dword(rcec, aer + PCI_ERR_UNCOR_MASK, &mask); + if (rc) + return rc; + mask &= ~PCI_ERR_UNC_INTN; + rc = pci_write_config_dword(rcec, aer + PCI_ERR_UNCOR_MASK, mask); + if (rc) + return rc; + + rc = pci_read_config_dword(rcec, aer + PCI_ERR_COR_MASK, &mask); + if (rc) + return rc; + mask &= ~PCI_ERR_COR_INTERNAL; + rc = pci_write_config_dword(rcec, aer + PCI_ERR_COR_MASK, mask); + + return rc; +} + +static void cxl_unmask_internal_errors(struct pci_dev *rcec) +{ + if (!handles_cxl_errors(rcec)) + return; + + if (__cxl_unmask_internal_errors(rcec)) + dev_err(&rcec->dev, "cxl: Failed to unmask internal errors"); + else + dev_dbg(&rcec->dev, "cxl: Internal errors unmasked"); +} + #else +static inline void cxl_unmask_internal_errors(struct pci_dev *dev) { } static inline void cxl_handle_error(struct pci_dev *dev, struct aer_err_info *info) { } #endif @@ -1397,6 +1469,7 @@ static int aer_probe(struct pcie_device *dev) return status; } + cxl_unmask_internal_errors(port); aer_enable_rootport(rpc); pci_info(port, "enabled with IRQ %d\n", dev->irq); return 0;