From patchwork Tue Sep 6 13:47:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nipun Gupta X-Patchwork-Id: 12967593 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB4F8C6FA8C for ; Tue, 6 Sep 2022 14:20:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241391AbiIFOU1 (ORCPT ); Tue, 6 Sep 2022 10:20:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35334 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241665AbiIFOR2 (ORCPT ); Tue, 6 Sep 2022 10:17:28 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2042.outbound.protection.outlook.com [40.107.220.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02CA3DEF7; Tue, 6 Sep 2022 06:49:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XemBOO3nC6WNAP+aI+9SSmsvwLFQuoEHbTqjUQN+JiHjEoO2k7aLdUwYo9zQ8rQaAAy76QTdB+OpASnLZ83qVgAc7TAEtW3pcJJgmZEWxmVPhuQQIu7YvD5vI4UVntHiS2WI3GD4Fm0lu825S0i1Q/+doGjQVb8LrWa4Mcc+EMGxzr637jaVeFeGCF6zaLuDLizWYW+0qG7A0ZmW4hr5BnNJlVyIbCxTYEHh/IpeQaCNCMo0ie1kR7HNgoNABy5QfreUoDDntMONvJvAWS51dFotaXxa45d8+5jIJX0lwl8iF2psmgdX12O3odV0QiePW4bsMDD5XRc954PcqUHpRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uCJIG/W6utr64/ZhPwmV6hiUCVl91LQmkAFBIzF02+s=; b=n6qUfxojKz/a+43tjPHftgay9mJJBXUu2lissSOQ7B9r2YSszs7AbnBy2h5FHixp+iH6p7PE+HpdmHI4r2+gUu6UwNfqSxfWr6zCF2U0jFWlvuskEc9pjbrseOALBgjtwMTbi5GuWTIXeszEeji4uGZaX36YEmZn4R9DVxAUYF3JSwT8c+sFkm+/gQuFdChMypIqXjhGE6oRr6Nf1iLwUPtFEk5nq4nKbfnccAm3ON09AzzwUfk9JJmpBe6O+T3x8Ycite1m0M1xQq2H+xRokOvfq5M06yML+wddKxg2eHPxVdV86mgsgkHLWBu3keTW7i4dCAp9gBsGKOhwXVvOJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uCJIG/W6utr64/ZhPwmV6hiUCVl91LQmkAFBIzF02+s=; b=otFUDPNyUSc4xTmscwM7i2gMEHFRi2Y0g8bXOWGmziKYGmoXdCIn+txcZe/fFw6lyltUKdibSZbcCQqLHhuk/1oVOSS1qgiHmj9E/GQXOnXZvmCTIvqXrnLgPKKiWQqTcK3Q4r5wCBfAeZ7LK6ujuy3gxmH1+0xyZRYi1zS/d60= Received: from CY5PR22CA0004.namprd22.prod.outlook.com (2603:10b6:930:16::29) by DM4PR12MB7573.namprd12.prod.outlook.com (2603:10b6:8:10f::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.12; Tue, 6 Sep 2022 13:48:29 +0000 Received: from CY4PEPF0000B8ED.namprd05.prod.outlook.com (2603:10b6:930:16:cafe::d7) by CY5PR22CA0004.outlook.office365.com (2603:10b6:930:16::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.12 via Frontend Transport; Tue, 6 Sep 2022 13:48:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by CY4PEPF0000B8ED.mail.protection.outlook.com (10.167.241.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5612.10 via Frontend Transport; Tue, 6 Sep 2022 13:48:29 +0000 Received: from SATLEXMB07.amd.com (10.181.41.45) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 6 Sep 2022 08:48:23 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB07.amd.com (10.181.41.45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 6 Sep 2022 06:48:22 -0700 Received: from xhdipdslab49.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.28 via Frontend Transport; Tue, 6 Sep 2022 08:48:12 -0500 From: Nipun Gupta To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , CC: , , , , , , Nipun Gupta Subject: [RFC PATCH v3 1/7] dt-bindings: bus: add CDX bus device tree bindings Date: Tue, 6 Sep 2022 19:17:55 +0530 Message-ID: <20220906134801.4079497-2-nipun.gupta@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906134801.4079497-1-nipun.gupta@amd.com> References: <20220803122655.100254-1-nipun.gupta@amd.com> <20220906134801.4079497-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000B8ED:EE_|DM4PR12MB7573:EE_ X-MS-Office365-Filtering-Correlation-Id: c70549d4-1204-48dc-1f19-08da900e7b09 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0C4YTxBp6+Y0di6IdLA3qW++l/vhqi7kx5E5zz2EoLsqUPAcKHDC3Zlrc3NwxOhq2kO5ZIoWj0EMnDE45WyD3R17se5u6eehQnOxtEBNmkkTY5Qv9DD7RddGrcl13lZ+C5mXT5ubOOWzPfAYIUG0cX9T2eLGmz2jFA4Cm87ntsABLvBcFM7z/SIH6JAnPAmzGg3HMxSZzv8UdssAMSHkF0+RXzjcHAXEAJTRaG1jCgek5sk3G+Ey2cTUq2p1pDOM9KwD8AE0mFKdjSkVOkyjdsvqcaB7Rt8edZDZqfhi8S/FS3Dwyl14pTLnACf0k5fIX9SGjZduB6aVao4lycprnL/AGTMXl4QyF7SiW4qzAxV6fqPJV+lVvKVuDiGeyILAFwKSD6e3XCuk6HigA9esPezeSGFGq7eqJDTWJ/FP0v3R9aEai8L2Rr7Ae615a4CDM7SCsuEmKGK9TUf0ORsEBb6HV65ipdfi5mRqnlIrPswPQ2qjJDG03Is9tTKl0eMv0H8xqZvsiJNei/8f3PTi4u6vtq7OZaRVqmwhiKQOyKX2zpOFeW9+OD0YGwtIiKOfBpT/r33hkjo+H1mmIrImyAymCZmmnPGfF/4AtO35NdRprbERYCFtXtYrB1+QArtH9OsLdMfVU9fdp1iEEH+J9GnoBx1mUfeKS5w2ayf+v/QPaqfY3jfTtRKtpfAjjXfeX5obLiiZN8bdQvm7Ubb6oFAWRmdq/ZW0LCpRGbzFm6VRhjKH7QwRynVqBap8Jp5gt8gSQmaropna5psX1lWLvdXcnB2YF/sW4LiEUc22HHkW5zJ8VrQ42WpUka174FUvfL/LX6URnm4xSydSkOlKMgAL1LHgE4TKfDhXasq3eCaTGfx69hQZqeWIvGYjzi5NJAk4RVvRNTBXgFmFIHKr73jJlI3XDF4XJXg2IX9y9OOVq6U8bsBhu8U0B5flWDOm X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(376002)(136003)(346002)(396003)(39860400002)(40470700004)(36840700001)(46966006)(8936002)(7416002)(5660300002)(36860700001)(44832011)(2906002)(83380400001)(36756003)(8676002)(40460700003)(82740400003)(40480700001)(70586007)(4326008)(70206006)(110136005)(316002)(54906003)(6666004)(86362001)(82310400005)(47076005)(426003)(336012)(81166007)(2616005)(26005)(1076003)(186003)(356005)(921005)(41300700001)(478600001)(966005)(36900700001)(2101003)(83996005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2022 13:48:29.1983 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c70549d4-1204-48dc-1f19-08da900e7b09 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000B8ED.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7573 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This patch adds a devicetree binding documentation for CDX bus. CDX bus controller dynamically detects CDX bus and the devices on these bus using CDX firmware. Signed-off-by: Nipun Gupta --- .../devicetree/bindings/bus/xlnx,cdx.yaml | 75 +++++++++++++++++++ MAINTAINERS | 6 ++ 2 files changed, 81 insertions(+) create mode 100644 Documentation/devicetree/bindings/bus/xlnx,cdx.yaml diff --git a/Documentation/devicetree/bindings/bus/xlnx,cdx.yaml b/Documentation/devicetree/bindings/bus/xlnx,cdx.yaml new file mode 100644 index 000000000000..0aa5599ada8e --- /dev/null +++ b/Documentation/devicetree/bindings/bus/xlnx,cdx.yaml @@ -0,0 +1,75 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/bus/xlnx,cdx.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: AMD CDX bus controller + +description: | + CDX bus controller for AMD devices is implemented to dynamically + detect CDX bus and devices on these bus using the firmware. + The CDX bus manages multiple FPGA based hardware devices, which + can support network, crypto or any other specialized type of + devices. These FPGA based devices can be added/modified dynamically + on run-time. + + All devices on the CDX bus will have a unique streamid (for IOMMU) + and a unique device ID (for MSI) corresponding to a requestor ID + (one to one associated with the device). The streamid and deviceid + are used to configure SMMU and GIC-ITS respectively. + + iommu-map property is used to define the set of stream ids + corresponding to each device and the associated IOMMU. + + The MSI writes are accompanied by sideband data (Device ID). + The msi-map property is used to associate the devices with the + device ID as well as the associated ITS controller. + +maintainers: + - Nipun Gupta + - Nikhil Agarwal + +properties: + compatible: + const: xlnx,cdxbus-controller-1.0 + + reg: + maxItems: 1 + + iommu-map: true + + msi-map: true + +required: + - compatible + - reg + - iommu-map + - msi-map + +additionalProperties: false + +examples: + - | + smmu@ec000000 { + compatible = "arm,smmu-v3"; + #iommu-cells = <1>; + }; + + gic@e2000000 { + compatible = "arm,gic-v3"; + interrupt-controller; + its: gic-its@e2040000 { + compatible = "arm,gic-v3-its"; + msi-controller; + }; + }; + + cdx: cdx@4000000 { + compatible = "xlnx,cdxbus-controller-1.0"; + reg = <0x00000000 0x04000000 0 0x1000>; + /* define map for RIDs 250-259 */ + iommu-map = <250 &smmu 250 10>; + /* define msi map for RIDs 250-259 */ + msi-map = <250 &its 250 10>; + }; diff --git a/MAINTAINERS b/MAINTAINERS index 9d7f64dc0efe..f0598b3d731c 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -934,6 +934,12 @@ S: Supported F: drivers/crypto/ccp/ F: include/linux/ccp.h +AMD CDX BUS DRIVER +M: Nipun Gupta +M: Nikhil Agarwal +S: Maintained +F: Documentation/devicetree/bindings/bus/xlnx,cdx.yaml + AMD CRYPTOGRAPHIC COPROCESSOR (CCP) DRIVER - SEV SUPPORT M: Brijesh Singh M: Tom Lendacky From patchwork Tue Sep 6 13:47:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nipun Gupta X-Patchwork-Id: 12967594 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13340C6FA83 for ; Tue, 6 Sep 2022 14:20:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241490AbiIFOU3 (ORCPT ); Tue, 6 Sep 2022 10:20:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39604 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241477AbiIFOST (ORCPT ); Tue, 6 Sep 2022 10:18:19 -0400 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam07on2087.outbound.protection.outlook.com [40.107.212.87]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80D9813F44; Tue, 6 Sep 2022 06:49:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ip1A1B6FDOM6rlP75tR7hdjI1Zw5m35rOB1WJOpTXLjjMHxVgcUmg7njnDFxxE5T0Z4vsebUtJRciao0I5IU4vsPjF9OnwGRFrP2OJCQGZ3sURG2CrFXep5ej4TKYqd17mz85XYqQWjXOGu8WlUGJVK/PMr8Ghc0067ouscJyspFFWKQSFaQaK4tZ7VvaHAekK9eoSnSTftPTSVPZ/bjVEokf3/7uZKSXDWO1AXlPaa1UdkilkzpqmWijiba0ZB49+vV296POBUYT0hhN/jwhiOELAwXdr/y7OP9pj6y8CRmM23yXLcD5GRDs5dmdzRxAGfOEs0fa+A3jIflQUMm6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=19Or6RyAdh73nl9HFYxk5QBVg/k5Qxra85q9l63LoP8=; b=X04dggzMiigRkhFQSh3e+bhtbLNclDZZ4UMzDg9lt7jXLgfPtymTbDW/DV83LlaO+9fqOjuJwlffWy12nZTbNZjlS9cm2zb84U+BP77mtZxzQnFqrBqekq8rPLutPtLFj5sQO0eMsUmn5GBuR1ADX+lcoJWqD46BVmKhuuPjgxbY8ChWhta5HHua8+zC7Xm84O3jHFToKo5RMJ3ObNYN/UUpYpyp+0WQHS1G+YIBWt052azxoNkmdzPasImj8JnN8mktqngSTpZ4Y9TpSFdBT1Ir2jIOVgtvWFkHduxxOt09V+tFrzOFQIUUUcslU7jtbwJmYOEbGHrdLHRJiD+d7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=19Or6RyAdh73nl9HFYxk5QBVg/k5Qxra85q9l63LoP8=; b=xUcxF1tCZtqRFlI0poi98gSIybzal6mmTErfsYIkzjb841Ij2UAOjqXmKZYTmvc+0NfwvCaEir92EtMuVblwMRQzeKU104ZT60FpFx+wGYqg8Kec78az7MRJ1nbiGYZsR6RKoa0DFgjACD6amPs6YlGRqKDgPiXLHJ5Rll+UPSE= Received: from BN1PR14CA0010.namprd14.prod.outlook.com (2603:10b6:408:e3::15) by BL1PR12MB5141.namprd12.prod.outlook.com (2603:10b6:208:309::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.12; Tue, 6 Sep 2022 13:48:34 +0000 Received: from BN8NAM11FT023.eop-nam11.prod.protection.outlook.com (2603:10b6:408:e3:cafe::5f) by BN1PR14CA0010.outlook.office365.com (2603:10b6:408:e3::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.12 via Frontend Transport; Tue, 6 Sep 2022 13:48:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT023.mail.protection.outlook.com (10.13.177.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5612.13 via Frontend Transport; Tue, 6 Sep 2022 13:48:33 +0000 Received: from SATLEXMB07.amd.com (10.181.41.45) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 6 Sep 2022 08:48:33 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB07.amd.com (10.181.41.45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 6 Sep 2022 06:48:32 -0700 Received: from xhdipdslab49.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.28 via Frontend Transport; Tue, 6 Sep 2022 08:48:22 -0500 From: Nipun Gupta To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , CC: , , , , , , Nipun Gupta Subject: [RFC PATCH v3 2/7] bus/cdx: add the cdx bus driver Date: Tue, 6 Sep 2022 19:17:56 +0530 Message-ID: <20220906134801.4079497-3-nipun.gupta@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906134801.4079497-1-nipun.gupta@amd.com> References: <20220803122655.100254-1-nipun.gupta@amd.com> <20220906134801.4079497-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT023:EE_|BL1PR12MB5141:EE_ X-MS-Office365-Filtering-Correlation-Id: a0c9127b-b601-49f2-0181-08da900e7dc0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qK9Jof4FCZafs7uTKXSL7rdMpkcmjveHidReNyoI/yUbCgb58RgKdQ4hEIoGrLkEUZ0oHWqLo105zP03EBTZKJ5a7CfKvvhm2hRsmCxYLGwVNIRDcMnPHaiNmeiyLH4YurGF4EEWtYx6824Wcn9JEAmHGlxKGjMEaTkXXtFl+7mbYMhD3YZCwVudUSQlgAH3Opblsxjf19PaDV4v+RINqkiJfuOkmfm3m2LJH0hUfnnY/Exf+XMdkzPcboaS4uER8CGWdy/AVJE1LDFCgzSXUr0RkyM8YVYD0cTtNRRpEzD3ZxACVoV+SWZv+FaNJd0t8WiqUR9rna7C2i72Lnn08c+AKPmgcEbS1vR6l8Jp4MdgwNPtweewAY4v0kaTtiTXrJeKwdjrPercW2kGi5PnhSQNse+Y6k9AL9qi406dHgOvUXOJAohiTa6Oeu6i6xZS3ERBuXEwkLFheiE2HBmOT9LyOtypGYL58KEjGEOft+zwF+rqrgkEOCmwo+xS/aOzzvMRw93DliYt4sqX6sfO9oi3vBpAtUKRrNfITaQ6E9/637Rl0w+7TdtK0pv+MQOZQf8LK6uGUnS298oXpBxlsg0i3V3iy8OicSTbnJWJH0Dc1MK3ihNvMeVagAVX8Zr35FeAFYa/r5XP/RoirQC9lMcTydl6UpgeyluvBlidfpzc30m72oNHJvr63JhqQZFdpx3DOfcXomDbmgtxtT7AEfZklRFTtzqCl8Z4z6GaziSRhJ0UvTq2FlNSOfX9vbfUmbgTitmTpEalrDzizIFc3aKq/VB5oQ+6E0QqoIoolgVgqzLO4wDyWzIz2/XPDoxaWuNROP1fdvg4lTulQbUOo+AeKBKHEvpP2WwXSjEjocSkXmagaiNOG0Iu4sCvUTOf X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(346002)(39860400002)(136003)(396003)(376002)(36840700001)(46966006)(40470700004)(41300700001)(70206006)(7416002)(86362001)(70586007)(5660300002)(82310400005)(110136005)(44832011)(8936002)(4326008)(316002)(36756003)(6666004)(8676002)(30864003)(478600001)(186003)(83380400001)(2616005)(336012)(2906002)(426003)(26005)(40460700003)(54906003)(40480700001)(47076005)(356005)(36860700001)(81166007)(1076003)(921005)(82740400003)(36900700001)(2101003)(83996005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2022 13:48:33.9242 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a0c9127b-b601-49f2-0181-08da900e7dc0 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT023.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5141 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org CDX bus supports the scanning and probing of FPGA based devices. These devices are registers as CDX devices. The bus driver sets up the basic infrastructure and fetches the device related information from the firmware. CDX bus is capable of scanning devices dynamically, supporting rescanning of dynamically added, removed or updated devices. Signed-off-by: Nipun Gupta --- Basic overview of CDX bus architecture is provided in [patch 0/7]. Please NOTE: This RFC change does not support the CDX bus firmware interface as it is under development, and this series aims to get an early feedback from the community. Firmware interaction are stubbed as MCDI APIs which is a protocol used by AMD to interact with Firmware. MAINTAINERS | 2 + drivers/bus/Kconfig | 1 + drivers/bus/Makefile | 3 + drivers/bus/cdx/Kconfig | 7 + drivers/bus/cdx/Makefile | 3 + drivers/bus/cdx/cdx.c | 437 ++++++++++++++++++++++++++++++ drivers/bus/cdx/cdx.h | 34 +++ drivers/bus/cdx/mcdi_stubs.c | 54 ++++ drivers/bus/cdx/mcdi_stubs.h | 76 ++++++ include/linux/cdx/cdx_bus.h | 93 +++++++ include/linux/mod_devicetable.h | 13 + scripts/mod/devicetable-offsets.c | 4 + scripts/mod/file2alias.c | 12 + 13 files changed, 739 insertions(+) create mode 100644 drivers/bus/cdx/Kconfig create mode 100644 drivers/bus/cdx/Makefile create mode 100644 drivers/bus/cdx/cdx.c create mode 100644 drivers/bus/cdx/cdx.h create mode 100644 drivers/bus/cdx/mcdi_stubs.c create mode 100644 drivers/bus/cdx/mcdi_stubs.h create mode 100644 include/linux/cdx/cdx_bus.h diff --git a/MAINTAINERS b/MAINTAINERS index f0598b3d731c..61af11c9fe06 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -939,6 +939,8 @@ M: Nipun Gupta M: Nikhil Agarwal S: Maintained F: Documentation/devicetree/bindings/bus/xlnx,cdx.yaml +F: drivers/bus/cdx/* +F: include/linux/cdx/* AMD CRYPTOGRAPHIC COPROCESSOR (CCP) DRIVER - SEV SUPPORT M: Brijesh Singh diff --git a/drivers/bus/Kconfig b/drivers/bus/Kconfig index 7bfe998f3514..b0324efb9a6a 100644 --- a/drivers/bus/Kconfig +++ b/drivers/bus/Kconfig @@ -251,5 +251,6 @@ config DA8XX_MSTPRI source "drivers/bus/fsl-mc/Kconfig" source "drivers/bus/mhi/Kconfig" +source "drivers/bus/cdx/Kconfig" endmenu diff --git a/drivers/bus/Makefile b/drivers/bus/Makefile index d90eed189a65..88649111c395 100644 --- a/drivers/bus/Makefile +++ b/drivers/bus/Makefile @@ -20,6 +20,9 @@ obj-$(CONFIG_INTEL_IXP4XX_EB) += intel-ixp4xx-eb.o obj-$(CONFIG_MIPS_CDMM) += mips_cdmm.o obj-$(CONFIG_MVEBU_MBUS) += mvebu-mbus.o +#CDX bus +obj-$(CONFIG_CDX_BUS) += cdx/ + # Interconnect bus driver for OMAP SoCs. obj-$(CONFIG_OMAP_INTERCONNECT) += omap_l3_smx.o omap_l3_noc.o diff --git a/drivers/bus/cdx/Kconfig b/drivers/bus/cdx/Kconfig new file mode 100644 index 000000000000..c1eed5225328 --- /dev/null +++ b/drivers/bus/cdx/Kconfig @@ -0,0 +1,7 @@ +config CDX_BUS + bool "CDX Bus driver" + help + Driver to enable CDX Bus infrastructure. CDX bus is + capable of scanning devices dynamically, supporting + rescanning of dynamically added, removed or updated + devices. diff --git a/drivers/bus/cdx/Makefile b/drivers/bus/cdx/Makefile new file mode 100644 index 000000000000..e91bfe706294 --- /dev/null +++ b/drivers/bus/cdx/Makefile @@ -0,0 +1,3 @@ +obj-$(CONFIG_CDX_BUS) += cdx-bus-driver.o + +cdx-bus-driver-objs := cdx.o cdx_msi.o mcdi_stubs.o diff --git a/drivers/bus/cdx/cdx.c b/drivers/bus/cdx/cdx.c new file mode 100644 index 000000000000..fc417c32c59b --- /dev/null +++ b/drivers/bus/cdx/cdx.c @@ -0,0 +1,437 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Platform driver for CDX bus. + * + * Copyright (C) 2022, Advanced Micro Devices, Inc. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "cdx.h" +#include "mcdi_stubs.h" + +/* + * Default DMA mask for devices on a CDX bus + */ +#define CDX_DEFAULT_DMA_MASK (~0ULL) + +struct platform_device *cdx_controller_pdev; + +static int cdx_bus_device_discovery(struct platform_device *pdev); + +static int cdx_unregister_device(struct device *dev, + void * __always_unused data) +{ + struct cdx_device *cdx_dev = to_cdx_device(dev); + + kfree(cdx_dev->driver_override); + cdx_dev->driver_override = NULL; + + /* + * Do not free cdx_dev here as it would be freed in + * cdx_device_release() called from within put_device(). + */ + device_del(&cdx_dev->dev); + put_device(&cdx_dev->dev); + + return 0; +} + +void cdx_unregister_devices(struct bus_type *bus) +{ + int ret; + + /* Reset all the devices attached to cdx bus */ + ret = bus_for_each_dev(bus, NULL, NULL, cdx_unregister_device); + if (ret) + pr_err("error in CDX unregister for all devices\n"); +} + +/** + * cdx_match_one_device - Tell if a CDX device structure has a matching + * CDX device id structure + * @id: single CDX device id structure to match + * @dev: the CDX device structure to match against + * + * Returns the matching cdx_device_id structure or %NULL if there is no match. + */ +static inline const struct cdx_device_id * +cdx_match_one_device(const struct cdx_device_id *id, + const struct cdx_device *dev) +{ + if ((id->vendor == dev->vendor) && (id->device == dev->device)) + return id; + return NULL; +} + +/** + * cdx_match_id - See if a CDX device matches a given cdx_id table + * @ids: array of CDX device ID structures to search in + * @dev: the CDX device structure to match against. + * + * Used by a driver to check whether a CDX device is in its list of + * supported devices. Returns the matching cdx_device_id structure or + * %NULL if there is no match. + */ +static inline const struct cdx_device_id * +cdx_match_id(const struct cdx_device_id *ids, struct cdx_device *dev) +{ + if (ids) { + while (ids->vendor || ids->device) { + if (cdx_match_one_device(ids, dev)) + return ids; + ids++; + } + } + return NULL; +} + +/** + * cdx_bus_match - device to driver matching callback + * @dev: the cdx device to match against + * @drv: the device driver to search for matching cdx device + * structures + * + * Returns 1 on success, 0 otherwise. + */ +static int cdx_bus_match(struct device *dev, struct device_driver *drv) +{ + struct cdx_device *cdx_dev = to_cdx_device(dev); + struct cdx_driver *cdx_drv = to_cdx_driver(drv); + const struct cdx_device_id *found_id; + + /* When driver_override is set, only bind to the matching driver */ + if (cdx_dev->driver_override) + return !strcmp(cdx_dev->driver_override, cdx_drv->driver.name); + + found_id = cdx_match_id(cdx_drv->match_id_table, cdx_dev); + if (found_id) + return true; + + return false; +} + +static int cdx_dma_configure(struct device *dev) +{ + struct cdx_device *cdx_dev = to_cdx_device(dev); + u32 input_id = cdx_dev->req_id; + int ret; + + ret = of_dma_configure_id(dev, dev->parent->of_node, 0, &input_id); + if (ret) { + dev_err(dev, "of_dma_configure_id() failed\n"); + return ret; + } + + return 0; +} + +struct bus_type cdx_bus_type = { + .name = "cdx", + .match = cdx_bus_match, + .dma_configure = cdx_dma_configure, +}; +EXPORT_SYMBOL_GPL(cdx_bus_type); + +static int cdx_driver_probe(struct device *dev) +{ + struct cdx_driver *cdx_drv = to_cdx_driver(dev->driver); + struct cdx_device *cdx_dev = to_cdx_device(dev); + int error; + + error = cdx_drv->probe(cdx_dev); + if (error < 0) { + if (error != -EPROBE_DEFER) + dev_err(dev, "%s failed: %d\n", __func__, error); + return error; + } + + return 0; +} + +static int cdx_driver_remove(struct device *dev) +{ + struct cdx_driver *cdx_drv = to_cdx_driver(dev->driver); + struct cdx_device *cdx_dev = to_cdx_device(dev); + int error; + + error = cdx_drv->remove(cdx_dev); + if (error < 0) { + dev_err(dev, "%s failed: %d\n", __func__, error); + return error; + } + + return 0; +} + +static void cdx_driver_shutdown(struct device *dev) +{ + struct cdx_driver *cdx_drv = to_cdx_driver(dev->driver); + struct cdx_device *cdx_dev = to_cdx_device(dev); + + cdx_drv->shutdown(cdx_dev); +} + +/* + * __cdx_driver_register - registers a CDX device driver + */ +int __cdx_driver_register(struct cdx_driver *cdx_driver, + struct module *owner) +{ + int error; + + cdx_driver->driver.owner = owner; + cdx_driver->driver.bus = &cdx_bus_type; + + if (cdx_driver->probe) + cdx_driver->driver.probe = cdx_driver_probe; + + if (cdx_driver->remove) + cdx_driver->driver.remove = cdx_driver_remove; + + if (cdx_driver->shutdown) + cdx_driver->driver.shutdown = cdx_driver_shutdown; + + error = driver_register(&cdx_driver->driver); + if (error < 0) { + pr_err("driver_register() failed for %s: %d\n", + cdx_driver->driver.name, error); + return error; + } + + return 0; +} +EXPORT_SYMBOL_GPL(__cdx_driver_register); + +/* + * cdx_driver_unregister - unregisters a device driver from the + * CDX bus + */ +void cdx_driver_unregister(struct cdx_driver *cdx_driver) +{ + driver_unregister(&cdx_driver->driver); +} +EXPORT_SYMBOL_GPL(cdx_driver_unregister); + +static void cdx_device_release(struct device *dev) +{ + struct cdx_device *cdx_dev = to_cdx_device(dev); + + kfree(cdx_dev); +} + +static int cdx_device_add(struct device *parent, + struct cdx_dev_params_t *dev_params) +{ + struct cdx_device *cdx_dev; + int ret; + + cdx_dev = kzalloc(sizeof(*cdx_dev), GFP_KERNEL); + if (!cdx_dev) { + dev_err(parent, + "memory allocation for cdx dev failed\n"); + return -ENOMEM; + } + + /* Populate resource */ + memcpy(cdx_dev->res, dev_params->res, sizeof(struct resource) * + dev_params->res_count); + cdx_dev->res_count = dev_params->res_count; + + /* Populate CDX dev params */ + cdx_dev->req_id = dev_params->req_id; + cdx_dev->vendor = dev_params->vendor; + cdx_dev->device = dev_params->device; + cdx_dev->bus_id = dev_params->bus_id; + cdx_dev->func_id = dev_params->func_id; + cdx_dev->dma_mask = CDX_DEFAULT_DMA_MASK; + + /* Initiaize generic device */ + device_initialize(&cdx_dev->dev); + cdx_dev->dev.parent = parent; + cdx_dev->dev.bus = &cdx_bus_type; + cdx_dev->dev.dma_mask = &cdx_dev->dma_mask; + cdx_dev->dev.release = cdx_device_release; + + /* Set Name */ + dev_set_name(&cdx_dev->dev, "cdx-%02x:%02x", cdx_dev->bus_id, + cdx_dev->func_id); + + ret = device_add(&cdx_dev->dev); + if (ret != 0) { + dev_err(&cdx_dev->dev, + "cdx device add failed: %d", ret); + goto fail; + } + + return 0; + +fail: + /* + * Do not free cdx_dev here as it would be freed in + * cdx_device_release() called from within put_device(). + */ + put_device(&cdx_dev->dev); + + return ret; +} + +static int cdx_bus_device_discovery(struct platform_device *pdev) +{ + struct cdx_mcdi_t *cdx_mcdi = platform_get_drvdata(pdev); + int num_cdx_bus, num_cdx_func; + uint8_t bus_id, func_id; + int ret; + + cdx_controller_pdev = pdev; + + /* MCDI FW Read: Fetch the number of CDX buses present*/ + num_cdx_bus = cdx_mcdi_get_num_buses(cdx_mcdi); + + for (bus_id = 0; bus_id < num_cdx_bus; bus_id++) { + /* MCDI FW Read: Fetch the number of devices present */ + num_cdx_func = cdx_mcdi_get_num_funcs(cdx_mcdi, bus_id); + + for (func_id = 0; func_id < num_cdx_func; func_id++) { + struct cdx_dev_params_t dev_params; + + /* MCDI FW: Get the device config */ + ret = cdx_mcdi_get_func_config(cdx_mcdi, bus_id, + func_id, &dev_params); + if (ret) { + dev_err(&pdev->dev, + "CDX device config get failed for bus: %d\n", + ret); + goto fail; + } + + /* Add the device to the cdx bus */ + ret = cdx_device_add(&pdev->dev, &dev_params); + if (ret == -EPROBE_DEFER) { + goto fail; + } else if (ret) { + dev_err(&pdev->dev, + "registering cdx dev: %d failed: %d\n", + func_id, ret); + goto fail; + } else { + dev_dbg(&pdev->dev, + "CDX dev: %d on cdx bus: %d created\n", + func_id, bus_id); + } + } + } + + return 0; +fail: + cdx_unregister_devices(&cdx_bus_type); + return ret; +} + +static int cdx_bus_probe(struct platform_device *pdev) +{ + struct cdx_mcdi_t *cdx_mcdi; + int ret; + + cdx_mcdi = kzalloc(sizeof(*cdx_mcdi), GFP_KERNEL); + if (!cdx_mcdi) { + dev_err(&pdev->dev, "Failed to allocate memory for cdx_mcdi\n"); + return -ENOMEM; + } + + /* MCDI FW: Initialize the FW path */ + ret = cdx_mcdi_init(cdx_mcdi); + if (ret) { + dev_err(&pdev->dev, "MCDI Initialization failed: %d\n", ret); + goto mcdi_init_fail; + } + platform_set_drvdata(pdev, cdx_mcdi); + + /* Discover all the devices on the bus */ + ret = cdx_bus_device_discovery(pdev); + if (ret) { + dev_err(&pdev->dev, + "CDX bus device discovery failed: %d\n", ret); + goto device_discovery_fail; + } + + return 0; + +mcdi_init_fail: + kfree(cdx_mcdi); +device_discovery_fail: + cdx_mcdi_finish(cdx_mcdi); + + return ret; +} + +static int cdx_bus_remove(struct platform_device *pdev) +{ + struct cdx_mcdi_t *cdx_mcdi = platform_get_drvdata(pdev); + + cdx_unregister_devices(&cdx_bus_type); + + cdx_mcdi_finish(cdx_mcdi); + kfree(cdx_mcdi); + + return 0; +} + +static void cdx_bus_shutdown(struct platform_device *pdev) +{ + cdx_bus_remove(pdev); +} + +static const struct of_device_id cdx_match_table[] = { + {.compatible = "xlnx,cdxbus-controller-1.0",}, + { }, +}; + +MODULE_DEVICE_TABLE(of, cdx_match_table); + +static struct platform_driver cdx_bus_driver = { + .driver = { + .name = "cdx-bus", + .pm = NULL, + .of_match_table = cdx_match_table, + }, + .probe = cdx_bus_probe, + .remove = cdx_bus_remove, + .shutdown = cdx_bus_shutdown, +}; + +static int __init cdx_bus_driver_init(void) +{ + int ret; + + ret = bus_register(&cdx_bus_type); + if (ret < 0) { + pr_err("bus type registration failed for CDX: %d\n", ret); + return ret; + } + + ret = platform_driver_register(&cdx_bus_driver); + if (ret < 0) { + pr_err("platform_driver_register() failed: %d\n", ret); + goto fail; + } + + return 0; + +fail: + bus_unregister(&cdx_bus_type); + return ret; +} +postcore_initcall(cdx_bus_driver_init); diff --git a/drivers/bus/cdx/cdx.h b/drivers/bus/cdx/cdx.h new file mode 100644 index 000000000000..db0569431c10 --- /dev/null +++ b/drivers/bus/cdx/cdx.h @@ -0,0 +1,34 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Header file for the CDX Bus + * + * Copyright (C) 2022, Advanced Micro Devices, Inc. + */ + +#ifndef _CDX_H_ +#define _CDX_H_ + +#include +#include + +/** + * struct cdx_dev_params_t - CDX device parameters + * @vendor: Vendor ID for CDX device + * @device: Device ID for CDX device + * @bus_id: Bus ID for this CDX device + * @func_id: Function ID for this device + * @res: array of MMIO region entries + * @res_count: number of valid MMIO regions + * @req_id: Requestor ID associated with CDX device + */ +struct cdx_dev_params_t { + u16 vendor; + u16 device; + u8 bus_id; + u8 func_id; + struct resource res[MAX_CDX_DEV_RESOURCES]; + u8 res_count; + u32 req_id; +}; + +#endif /* _CDX_H_ */ diff --git a/drivers/bus/cdx/mcdi_stubs.c b/drivers/bus/cdx/mcdi_stubs.c new file mode 100644 index 000000000000..cc9d30fa02f8 --- /dev/null +++ b/drivers/bus/cdx/mcdi_stubs.c @@ -0,0 +1,54 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * MCDI Firmware interaction for CDX bus. + * + * Copyright (C) 2022, Advanced Micro Devices, Inc. + */ + +#include + +#include "cdx.h" +#include "mcdi_stubs.h" + +int cdx_mcdi_init(struct cdx_mcdi_t *cdx_mcdi) +{ + cdx_mcdi->id = 0; + cdx_mcdi->flags = 0; + + return 0; +} + +void cdx_mcdi_finish(struct cdx_mcdi_t *cdx_mcdi) +{ +} + +int cdx_mcdi_get_num_buses(struct cdx_mcdi_t *cdx_mcdi) +{ + return 1; +} + +int cdx_mcdi_get_num_funcs(struct cdx_mcdi_t *cdx_mcdi, int bus_num) +{ + return 1; +} + +int cdx_mcdi_get_func_config(struct cdx_mcdi_t *cdx_mcdi, + uint8_t bus_id, uint8_t func_id, + struct cdx_dev_params_t *dev_params) +{ + dev_params->res[0].start = 0xe4020000; + dev_params->res[0].end = 0xe4020FFF; + dev_params->res[0].flags = IORESOURCE_MEM; + dev_params->res[1].start = 0xe4100000; + dev_params->res[1].end = 0xE411FFFF; + dev_params->res[1].flags = IORESOURCE_MEM; + dev_params->res_count = 2; + + dev_params->req_id = 0x250; + dev_params->vendor = 0x10ee; + dev_params->device = 0x8084; + dev_params->bus_id = bus_id; + dev_params->func_id = func_id; + + return 0; +} diff --git a/drivers/bus/cdx/mcdi_stubs.h b/drivers/bus/cdx/mcdi_stubs.h new file mode 100644 index 000000000000..7b6f79d48ce9 --- /dev/null +++ b/drivers/bus/cdx/mcdi_stubs.h @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Header file for MCDI FW interaction for CDX bus. + * + * Copyright (C) 2022, Advanced Micro Devices, Inc. + */ + +#ifndef _MCDI_STUBS_H_ +#define _MCDI_STUBS_H_ + +#include "cdx.h" + +/** + * struct cdx_mcdi_t - CDX MCDI Firmware interface, to interact + * with CDX controller. + * @id: ID for MCDI Firmware interface + * @flags: Associated flags + */ +struct cdx_mcdi_t { + u32 id; + u32 flags; + /* Have more MCDI interface related data */ +}; + +/** + * cdx_mcdi_init - Initialize the MCDI Firmware interface + * for the CDX controller. + * @cdx_mcdi: pointer to MCDI interface + * + * Return 0 on success, <0 on failure + */ +int cdx_mcdi_init(struct cdx_mcdi_t *cdx_mcdi); + +/** + * cdx_mcdi_finish - Close the MCDI Firmware interface. + * @cdx_mcdi: pointer to MCDI interface + */ +void cdx_mcdi_finish(struct cdx_mcdi_t *cdx_mcdi); + +/** + * cdx_mcdi_get_num_buses - Get the total number of busses on + * the controller. + * @cdx_mcdi: pointer to MCDI interface. + * + * Return total number of busses available on the controller, + * <0 on failure + */ +int cdx_mcdi_get_num_buses(struct cdx_mcdi_t *cdx_mcdi); + +/** + * cdx_mcdi_get_num_funcs - Get the total number of functions on + * a particular bus of the controller. + * @cdx_mcdi: pointer to MCDI interface. + * @bus_num: Bus number. + * + * Return total number of functions available on the bus, <0 on failure + */ +int cdx_mcdi_get_num_funcs(struct cdx_mcdi_t *cdx_mcdi, int bus_num); + +/** + * cdx_mcdi_get_func_config - Get configuration for a particular + * bus_id:func_id + * @cdx_mcdi: pointer to MCDI interface. + * @bus_num: Bus number. + * @func_id: Function number. + * @dev_params: Pointer to cdx_dev_params_t, this is populated by this + * function with the configuration corresponding to the provided + * bus_id:func_id. + * + * Return 0 total number of functions available on the bus, <0 on failure + */ +int cdx_mcdi_get_func_config(struct cdx_mcdi_t *cdx_mcdi, + uint8_t bus_id, uint8_t func_id, + struct cdx_dev_params_t *dev_params); + +#endif /* _MCDI_STUBS_H_ */ diff --git a/include/linux/cdx/cdx_bus.h b/include/linux/cdx/cdx_bus.h new file mode 100644 index 000000000000..6e870b2c87d9 --- /dev/null +++ b/include/linux/cdx/cdx_bus.h @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * CDX bus public interface + * + * Copyright (C) 2022, Advanced Micro Devices, Inc. + * + */ +#ifndef _CDX_BUS_H_ +#define _CDX_BUS_H_ + +#include +#include + +#define MAX_CDX_DEV_RESOURCES 6 + +/** + * struct cdx_device - CDX device object + * @dev: Linux driver model device object + * @vendor: Vendor ID for CDX device + * @device: Device ID for CDX device + * @bus_id: Bus ID for this CDX device + * @func_id: Function ID for this device + * @res: array of MMIO region entries + * @res_count: number of valid MMIO regions + * @dma_mask: Default DMA mask + * @flags: CDX device flags + * @req_id: Requestor ID associated with CDX device + * @driver_override: driver name to force a match; do not set directly, + * because core frees it; use driver_set_override() to + * set or clear it. + */ +struct cdx_device { + struct device dev; + u16 vendor; + u16 device; + u8 bus_id; + u8 func_id; + struct resource res[MAX_CDX_DEV_RESOURCES]; + u8 res_count; + u64 dma_mask; + u16 flags; + u32 req_id; + const char *driver_override; +}; + +#define to_cdx_device(_dev) \ + container_of(_dev, struct cdx_device, dev) + +/** + * struct cdx_driver - CDX device driver + * @driver: Generic device driver + * @match_id_table: table of supported device matching Ids + * @probe: Function called when a device is added + * @remove: Function called when a device is removed + * @shutdown: Function called at shutdown time to quiesce the device + * @suspend: Function called when a device is stopped + * @resume: Function called when a device is resumed + * @driver_managed_dma: Device driver doesn't use kernel DMA API for DMA. + * For most device drivers, no need to care about this flag + * as long as all DMAs are handled through the kernel DMA API. + * For some special ones, for example VFIO drivers, they know + * how to manage the DMA themselves and set this flag so that + * the IOMMU layer will allow them to setup and manage their + * own I/O address space. + */ +struct cdx_driver { + struct device_driver driver; + const struct cdx_device_id *match_id_table; + int (*probe)(struct cdx_device *dev); + int (*remove)(struct cdx_device *dev); + void (*shutdown)(struct cdx_device *dev); + int (*suspend)(struct cdx_device *dev, pm_message_t state); + int (*resume)(struct cdx_device *dev); + bool driver_managed_dma; +}; + +#define to_cdx_driver(_drv) \ + container_of(_drv, struct cdx_driver, driver) + +/* + * Macro to avoid include chaining to get THIS_MODULE + */ +#define cdx_driver_register(drv) \ + __cdx_driver_register(drv, THIS_MODULE) + +int __must_check __cdx_driver_register(struct cdx_driver *cdx_driver, + struct module *owner); + +void cdx_driver_unregister(struct cdx_driver *driver); + +extern struct bus_type cdx_bus_type; + +#endif /* _CDX_BUS_H_ */ diff --git a/include/linux/mod_devicetable.h b/include/linux/mod_devicetable.h index 549590e9c644..9b94be83d53e 100644 --- a/include/linux/mod_devicetable.h +++ b/include/linux/mod_devicetable.h @@ -911,4 +911,17 @@ struct ishtp_device_id { kernel_ulong_t driver_data; }; +/** + * struct cdx_device_id - CDX device identifier + * @vendor: vendor ID + * @device: Device ID + * + * Type of entries in the "device Id" table for CDX devices supported by + * a CDX device driver. + */ +struct cdx_device_id { + __u16 vendor; + __u16 device; +}; + #endif /* LINUX_MOD_DEVICETABLE_H */ diff --git a/scripts/mod/devicetable-offsets.c b/scripts/mod/devicetable-offsets.c index c0d3bcb99138..62dc988df84d 100644 --- a/scripts/mod/devicetable-offsets.c +++ b/scripts/mod/devicetable-offsets.c @@ -262,5 +262,9 @@ int main(void) DEVID(ishtp_device_id); DEVID_FIELD(ishtp_device_id, guid); + DEVID(cdx_device_id); + DEVID_FIELD(cdx_device_id, vendor); + DEVID_FIELD(cdx_device_id, device); + return 0; } diff --git a/scripts/mod/file2alias.c b/scripts/mod/file2alias.c index 80d973144fde..c36e1f624e39 100644 --- a/scripts/mod/file2alias.c +++ b/scripts/mod/file2alias.c @@ -1452,6 +1452,17 @@ static int do_dfl_entry(const char *filename, void *symval, char *alias) return 1; } +/* Looks like: cdx:vNdN */ +static int do_cdx_entry(const char *filename, void *symval, + char *alias) +{ + DEF_FIELD(symval, cdx_device_id, vendor); + DEF_FIELD(symval, cdx_device_id, device); + + sprintf(alias, "cdx:v%08Xd%08Xd", vendor, device); + return 1; +} + /* Does namelen bytes of name exactly match the symbol? */ static bool sym_is(const char *name, unsigned namelen, const char *symbol) { @@ -1531,6 +1542,7 @@ static const struct devtable devtable[] = { {"ssam", SIZE_ssam_device_id, do_ssam_entry}, {"dfl", SIZE_dfl_device_id, do_dfl_entry}, {"ishtp", SIZE_ishtp_device_id, do_ishtp_entry}, + {"cdx", SIZE_cdx_device_id, do_cdx_entry}, }; /* Create MODULE_ALIAS() statements. From patchwork Tue Sep 6 13:47:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nipun Gupta X-Patchwork-Id: 12967592 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09251C6FA95 for ; Tue, 6 Sep 2022 14:20:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241267AbiIFOUY (ORCPT ); Tue, 6 Sep 2022 10:20:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37568 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241277AbiIFOSA (ORCPT ); Tue, 6 Sep 2022 10:18:00 -0400 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2042.outbound.protection.outlook.com [40.107.244.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 33E181A3A9; Tue, 6 Sep 2022 06:49:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TfMevd7NqJZQr54fAiJsrhAbnVEFr+9BNYbhrXledM7+IB4Z6UcNDiiJRbGoBrID7qsT5eXIUdD3dyetOjHO9wPqE0XU60NReVn1ceV7kljHVGcyC00r8TDgkRw1q+wPBscPo/pkEJgNVZKJOMvNzVTjEUjt1VZFMo0YyXrFJKgiO1tTyjXs/tjQfXtcdmc2BIYCdgqzmvFdXKHCq8IKXDtHD2/CoNLzXza5g/66i6T20a5jpn195YNkC1GJcbIjpPN1w4OgrWMdD0x+Z/QtlG8ZhOoUwwH2Jny+eQYD9D3C7aZvDn1UpJNppGO4dJXbup1RaluOr4ec3Mc1ykVKsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HFLqRFQwdCB/AVMKNaSF9HMOVyLB+Gzq9vpFzIaP/vg=; b=c3EQR/c+4X5JUspMN3VqQaRTQ6bqb22uJMoVgA43dGvjoG0vSAjyK/uFE9hLxUt8uIsw60dLOa0pweJ2/g/swUqG66qtrXGtoHg2tdgBicAKFgjKBqdZB2SdZdqyJVX3XAkx2reObwpIWcbKqxgWTnljKmamXR4F29shed+oJ7z8LUrXq5nlP3Fps7/BN2N6WwqE9GvevT/KxWFgAi6ztkVUD7l7sbQAfpiVLu3lIeTz1hS9dgw7pxO89O1oDb7inrRYZOLM6M+xfa2wBWYKFhOpuVE2dXqDdxXRLKHlxguTWS3oFNONaDGwYPbv2Rg/elpoa2BJu8lstMw8LzbAIw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HFLqRFQwdCB/AVMKNaSF9HMOVyLB+Gzq9vpFzIaP/vg=; b=JghcdcGjFdNP2ZTwRrEgGmP/xlfEWHmUNOZGQlwZRUEWelrMHh1WIDuP2p+kVXA7dfAhjNMs3oVWbSUhy47p1Dh8TXN5zTKbYQ5hUEe8AV1KAOOCG1MU1yWI1Ded2vUaSQuxjHOby2tr0qah+fRiBWJH4SFJU/nH+tIa8Padmk4= Received: from BN9P223CA0018.NAMP223.PROD.OUTLOOK.COM (2603:10b6:408:10b::23) by DM4PR12MB5279.namprd12.prod.outlook.com (2603:10b6:5:39f::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.12; Tue, 6 Sep 2022 13:48:45 +0000 Received: from BN8NAM11FT028.eop-nam11.prod.protection.outlook.com (2603:10b6:408:10b:cafe::30) by BN9P223CA0018.outlook.office365.com (2603:10b6:408:10b::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.12 via Frontend Transport; Tue, 6 Sep 2022 13:48:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT028.mail.protection.outlook.com (10.13.176.225) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5588.10 via Frontend Transport; Tue, 6 Sep 2022 13:48:44 +0000 Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 6 Sep 2022 08:48:43 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 6 Sep 2022 08:48:42 -0500 Received: from xhdipdslab49.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.28 via Frontend Transport; Tue, 6 Sep 2022 08:48:33 -0500 From: Nipun Gupta To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , CC: , , , , , , Nipun Gupta Subject: [RFC PATCH v3 3/7] iommu/arm-smmu-v3: support ops registration for CDX bus Date: Tue, 6 Sep 2022 19:17:57 +0530 Message-ID: <20220906134801.4079497-4-nipun.gupta@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906134801.4079497-1-nipun.gupta@amd.com> References: <20220803122655.100254-1-nipun.gupta@amd.com> <20220906134801.4079497-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d6010912-b4b0-452e-5e44-08da900e843d X-MS-TrafficTypeDiagnostic: DM4PR12MB5279:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fKhFPQ2rEF7/crTGWm+MaKqkCDVo8LCQKiW4QfOG/HCHMOrT5dU+l+bPHL2/fNa6Mp+yhr6kiWeffwo4qNXl3K+TuNDsT38/EERDq3fqeE/DxBcWXCnlIkmlrHX8+Y2QGozG3hmAbWY8Uxz9vjTaWuhRKx2CJNY8UcXounR3DlWj320sGh0Qn3GJAeq2nTWm98uB7EK8pTeC3lHUEKMN2aY6yvoh8Qn+rMQt5nELGG/mbaTSnCk6rYTt2aYbFHGhuftrOTt4ZgHRqHKOqj6hl3Vtyn8Ppj05knrb1s6ZtBbDqkRcf1UpMRggJJgBuhfDUAxWLhoGYmRxUUK9Of+0ul0XeDpcu1x04jMlttdNDidtrvNP1q8+N/Z/3H0lldGpQfrHOrLEiXw+TADsfqb/RAMhyc1R33gikxxipDi79FxkbNKxpSBp+5bGTcgEPw8lQKu7xbPSkcfhxmiNdu0M+gJjIrk9+bdYkOxjiWhvr39a7w1jJ9Z+9oJZEWI/5Ik67/Of5szInixUjiPD0wUSoe2V1qW+RvmIbAFyfiIzWaxuOJmsKQKvVSlo7VvrmkMo6JWmZPJvO0KcsTcfaNk3oOH1WDxXBENZYazENgTXDqznr1PL1pirYDDQt1kAhuJrNfK02GO1v1zwyRnfR/ATsGAP4mcojn77TauNHDqGBR7CgSkjorrcXHu+e9duEZ0BOHB+Qqm48tccaQWQsY06PzvWmvSYur5B1eFQIMA8KPpjgNEyLO6/2a4B8CmgjW7zyodLvK6nlvcwSaUwOhw4N/R9u8xY8VkruSJEchGCXCBRwYkOjyyXna0xI8jxBdj1jKSGzAvzyb5j/eVj3RcD3DyU3BT4ut6fjQ1Ws6bXPd5zbQhoSC8Yu16pY6pBZxej X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(136003)(396003)(376002)(346002)(39860400002)(46966006)(36840700001)(40470700004)(36860700001)(5660300002)(316002)(83380400001)(110136005)(4326008)(8676002)(7416002)(54906003)(36756003)(44832011)(70206006)(8936002)(70586007)(2906002)(40480700001)(26005)(41300700001)(478600001)(86362001)(426003)(2616005)(336012)(82310400005)(186003)(1076003)(6666004)(40460700003)(356005)(921005)(81166007)(82740400003)(47076005)(36900700001)(83996005)(2101003);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2022 13:48:44.7965 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d6010912-b4b0-452e-5e44-08da900e843d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT028.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5279 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org With new CDX bus supported for AMD FPGA devices on ARM platform, the bus requires registration for the SMMU v3 driver. Signed-off-by: Nipun Gupta --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index d32b02336411..8ec9f2baf12d 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -29,6 +29,7 @@ #include #include +#include #include "arm-smmu-v3.h" #include "../../iommu-sva-lib.h" @@ -3690,16 +3691,27 @@ static int arm_smmu_set_bus_ops(struct iommu_ops *ops) if (err) goto err_reset_pci_ops; } +#endif +#ifdef CONFIG_CDX_BUS + if (cdx_bus_type.iommu_ops != ops) { + err = bus_set_iommu(&cdx_bus_type, ops); + if (err) + goto err_reset_amba_ops; + } #endif if (platform_bus_type.iommu_ops != ops) { err = bus_set_iommu(&platform_bus_type, ops); if (err) - goto err_reset_amba_ops; + goto err_reset_cdx_ops; } return 0; -err_reset_amba_ops: +err_reset_cdx_ops: +#ifdef CONFIG_CDX_BUS + bus_set_iommu(&cdx_bus_type, NULL); +#endif +err_reset_amba_ops: __maybe_unused; #ifdef CONFIG_ARM_AMBA bus_set_iommu(&amba_bustype, NULL); #endif From patchwork Tue Sep 6 13:47:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nipun Gupta X-Patchwork-Id: 12967591 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8486C6FA83 for ; Tue, 6 Sep 2022 14:20:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241186AbiIFOUV (ORCPT ); Tue, 6 Sep 2022 10:20:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37566 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241952AbiIFOTG (ORCPT ); Tue, 6 Sep 2022 10:19:06 -0400 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on20608.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe5a::608]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 747C42DA99; Tue, 6 Sep 2022 06:50:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DICauO5hqixwXFKSQTKdNMFcXmzgwFB5DxXonw0y26XV5KzWOi6vnNSYos00Xb/GGYeoA5GVk4mXa5DFRRCYoUmE/YdCBcSrbHX/PraTdZpvo5GCV3scmhVfLeGTDA8P/Ajmcqmp1c18tA47qRECGTPfqTA9nAGQRDeuO692vDG3FoAewSEHwAzX0CZxTd5fjYcDwoG6ldkxKHdbGpoQLqhBkj+93sBqI3hTcoBC5mXhsziHRXDj7d0gDJ4WNmoYpQ+f2qqzg3X+kSJJBpGNUAS1ZYtKHPtM9Uji9P38uZCqDs2mhlROCcTmIrmV7VVgEnCyRVLchJHq7d4X4oS1fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=aCp0J6mFf0usW5jRRvzPBYB3jYF/nC/WPt5Rmt/qE4I=; b=OmPyzFTyrWdQxrBSe0Q69PXO2zHg22i8Rt1L7trE+6kdQ/IFQbIDwmK21CgpDFsDfj4xV1P9jL30SEdmcGwzPchgBaghoWD/rKsX6+B63y2CWvOkIBtyKzLJ98uVNUhewL8u+ZvY47hg4auBZAerSKvgOY+8pRHO4JiO0pbvo343AnVgXRlVeYsbiR0vIpU3Txnu70wE+XzZIyK5R/O8iwq+tuROUqmavFuDrcoEAX1os6RPRgcXRKcTI4TTBgkJNngkCT5U3QWqbG2iU/kx7jXo9IrmJ54wtvNX+MxerjFH9kx0MZcOvr8Y9GLQhvKBxSU9FP0j1aRYCOnhi4yoMA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aCp0J6mFf0usW5jRRvzPBYB3jYF/nC/WPt5Rmt/qE4I=; b=Sit96zWuBZiqy0vhS7QcBH68wYK4yGh1ddBagnYffPNyZUTD1bColb0M/IZ3+Z2SkKtVY7VsUvbBfNBqP0jIqxmgco4lEj+Yi1TB0BM+FHuvVnoQC7uQTQZEu30SIrSzZc8l5lgy8l7Rgcm/79L24jqovw/M4hNAfXjFOW66X2Y= Received: from CY8PR22CA0014.namprd22.prod.outlook.com (2603:10b6:930:45::23) by SJ0PR12MB7082.namprd12.prod.outlook.com (2603:10b6:a03:4ae::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.11; Tue, 6 Sep 2022 13:48:54 +0000 Received: from CY4PEPF0000B8EF.namprd05.prod.outlook.com (2603:10b6:930:45::4) by CY8PR22CA0014.outlook.office365.com (2603:10b6:930:45::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.12 via Frontend Transport; Tue, 6 Sep 2022 13:48:54 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by CY4PEPF0000B8EF.mail.protection.outlook.com (10.167.241.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5612.9 via Frontend Transport; Tue, 6 Sep 2022 13:48:54 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 6 Sep 2022 08:48:53 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 6 Sep 2022 08:48:52 -0500 Received: from xhdipdslab49.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.28 via Frontend Transport; Tue, 6 Sep 2022 08:48:43 -0500 From: Nipun Gupta To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , CC: , , , , , , Nipun Gupta Subject: [RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its domain as parent Date: Tue, 6 Sep 2022 19:17:58 +0530 Message-ID: <20220906134801.4079497-5-nipun.gupta@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906134801.4079497-1-nipun.gupta@amd.com> References: <20220803122655.100254-1-nipun.gupta@amd.com> <20220906134801.4079497-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000B8EF:EE_|SJ0PR12MB7082:EE_ X-MS-Office365-Filtering-Correlation-Id: 224e707c-e629-46e6-c8a8-08da900e89c2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6Bqco0WgZCmwCPkWZlZZSKX/xxpdS0VtHkrEye1n0O9PlKi+42ECHyv0kK4iAYDmGleQgaPZVuugBzfjevyhKZGdaiK/SlY2TcZlHWoA026BGrBqHtX6wkZtziTMXtomR/NpCummpTQ/yYrawtcY1IhlBgB1Ny3/7O05hmpUodH/WJpRUhFVMWptYZB3DQcUF1kqFet5VRsK5AuOdq7i+0vZzhS/TNT3wz9snNOeX6Ob9NSyDOoTDQYJv1dGTm2Xd2SuOYZulf4NOx9+cdHWJEw2+gTVyjgB5+k4YchRnksZJQdwdbYd2hk/Yy1/74WkkyLWDjS2b2sSnSiRWU0q628X6CCXPQM5jI81WZqryov8z29BhbpDfk/7rdIcHEDwMCIhKFq3PbSmCYl5b13vH7D87HYyKkuy4kVS9mWLcNBcNI+bfKcRgkj5meJBhEMRwnqPf+/hDYvUo6R4XL1zvHK/b7wnu0PEIO8Fbbjm/rtmecj4vFAFWD6QPAcm7jXe72v1m7LUP3ynyb+N6fxcNgGeiG0UfHzxiME8ImStA4emtjb4QD4siBTRb/UDlWA4uXMbkvwQYw7EsCye8gQ2ET4sr9MTWVWAhADCBdCfwqeUsLg7yphcRCHYoJ7yCKHOlVbFlnTp6XZItjYyFwYMlyXz5V3RmtvtO45fAVHVvaoKL+7mc8SVnzD8n6eViSAd2GDMMq1fumAAgh6+IuENj8+KmYhfQjjtV9R93v/v5Dw9vCBcJPRL1pPJpdYx0j9lr46qL7kMfvWxmv1+KRCEC3F5SXbN7RUGMPz7C5aIby3tJoNvL3vOc+5upnqiv0fdyzfSZeQEG1HdMpxEbn39wmeKnyTF7uXtaThYGbMdU4JK/4cYE/jpbMHX03X3xVLt X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(346002)(396003)(136003)(39860400002)(376002)(40470700004)(46966006)(36840700001)(921005)(83380400001)(356005)(81166007)(36756003)(1076003)(426003)(86362001)(47076005)(6666004)(41300700001)(336012)(26005)(186003)(2616005)(40460700003)(40480700001)(54906003)(82310400005)(4326008)(110136005)(316002)(70206006)(8676002)(8936002)(478600001)(2906002)(70586007)(36860700001)(30864003)(82740400003)(5660300002)(7416002)(44832011)(2101003)(83996005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2022 13:48:54.0282 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 224e707c-e629-46e6-c8a8-08da900e89c2 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000B8EF.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB7082 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Devices on cdx bus are dynamically detected and registered using platform_device_register API. As these devices are not linked to of node they need a separate MSI domain for handling device ID to be provided to the GIC ITS domain. This also introduces APIs to alloc and free IRQs for CDX domain. Signed-off-by: Nipun Gupta Signed-off-by: Nikhil Agarwal --- drivers/bus/cdx/cdx.c | 18 +++ drivers/bus/cdx/cdx.h | 19 +++ drivers/bus/cdx/cdx_msi.c | 236 +++++++++++++++++++++++++++++++++++ drivers/bus/cdx/mcdi_stubs.c | 1 + include/linux/cdx/cdx_bus.h | 19 +++ 5 files changed, 293 insertions(+) create mode 100644 drivers/bus/cdx/cdx_msi.c diff --git a/drivers/bus/cdx/cdx.c b/drivers/bus/cdx/cdx.c index fc417c32c59b..02ececce1c84 100644 --- a/drivers/bus/cdx/cdx.c +++ b/drivers/bus/cdx/cdx.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include "cdx.h" @@ -236,6 +237,7 @@ static int cdx_device_add(struct device *parent, struct cdx_dev_params_t *dev_params) { struct cdx_device *cdx_dev; + struct irq_domain *cdx_msi_domain; int ret; cdx_dev = kzalloc(sizeof(*cdx_dev), GFP_KERNEL); @@ -252,6 +254,7 @@ static int cdx_device_add(struct device *parent, /* Populate CDX dev params */ cdx_dev->req_id = dev_params->req_id; + cdx_dev->num_msi = dev_params->num_msi; cdx_dev->vendor = dev_params->vendor; cdx_dev->device = dev_params->device; cdx_dev->bus_id = dev_params->bus_id; @@ -269,6 +272,21 @@ static int cdx_device_add(struct device *parent, dev_set_name(&cdx_dev->dev, "cdx-%02x:%02x", cdx_dev->bus_id, cdx_dev->func_id); + /* If CDX MSI domain is not created, create one. */ + cdx_msi_domain = cdx_find_msi_domain(parent); + if (!cdx_msi_domain) { + cdx_msi_domain = cdx_msi_domain_init(parent); + if (!cdx_msi_domain) { + dev_err(&cdx_dev->dev, + "cdx_msi_domain_init() failed: %d", ret); + kfree(cdx_dev); + return -1; + } + } + + /* Set the MSI domain */ + dev_set_msi_domain(&cdx_dev->dev, cdx_msi_domain); + ret = device_add(&cdx_dev->dev); if (ret != 0) { dev_err(&cdx_dev->dev, diff --git a/drivers/bus/cdx/cdx.h b/drivers/bus/cdx/cdx.h index db0569431c10..95df440ebd73 100644 --- a/drivers/bus/cdx/cdx.h +++ b/drivers/bus/cdx/cdx.h @@ -20,6 +20,7 @@ * @res: array of MMIO region entries * @res_count: number of valid MMIO regions * @req_id: Requestor ID associated with CDX device + * @num_msi: Number of MSI's supported by the device */ struct cdx_dev_params_t { u16 vendor; @@ -29,6 +30,24 @@ struct cdx_dev_params_t { struct resource res[MAX_CDX_DEV_RESOURCES]; u8 res_count; u32 req_id; + u32 num_msi; }; +/** + * cdx_msi_domain_init - Init the CDX bus MSI domain. + * @dev: Device of the CDX bus controller + * + * Return CDX MSI domain, NULL on failure + */ +struct irq_domain *cdx_msi_domain_init(struct device *dev); + +/** + * cdx_find_msi_domain - Get the CDX-MSI domain. + * @dev: CDX controller generic device + * + * Return CDX MSI domain, NULL on error or if CDX-MSI domain is + * not yet created. + */ +struct irq_domain *cdx_find_msi_domain(struct device *parent); + #endif /* _CDX_H_ */ diff --git a/drivers/bus/cdx/cdx_msi.c b/drivers/bus/cdx/cdx_msi.c new file mode 100644 index 000000000000..2fb7bac18393 --- /dev/null +++ b/drivers/bus/cdx/cdx_msi.c @@ -0,0 +1,236 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * AMD CDX bus driver MSI support + * + * Copyright (C) 2022, Advanced Micro Devices, Inc. + * + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "cdx.h" + +#ifdef GENERIC_MSI_DOMAIN_OPS +/* + * Generate a unique ID identifying the interrupt (only used within the MSI + * irqdomain. Combine the req_id with the interrupt index. + */ +static irq_hw_number_t cdx_domain_calc_hwirq(struct cdx_device *dev, + struct msi_desc *desc) +{ + /* + * Make the base hwirq value for req_id*10000 so it is readable + * as a decimal value in /proc/interrupts. + */ + return (irq_hw_number_t)(desc->msi_index + (dev->req_id * 10000)); +} + +static void cdx_msi_set_desc(msi_alloc_info_t *arg, + struct msi_desc *desc) +{ + arg->desc = desc; + arg->hwirq = cdx_domain_calc_hwirq(to_cdx_device(desc->dev), desc); +} +#else +#define cdx_msi_set_desc NULL +#endif + +static void cdx_msi_update_dom_ops(struct msi_domain_info *info) +{ + struct msi_domain_ops *ops = info->ops; + + if (!ops) + return; + + /* set_desc should not be set by the caller */ + if (!ops->set_desc) + ops->set_desc = cdx_msi_set_desc; +} + +static void cdx_msi_write_msg(struct irq_data *irq_data, + struct msi_msg *msg) +{ + /* + * Do nothing as CDX devices have these pre-populated + * in the hardware itself. + */ +} + +static void cdx_msi_update_chip_ops(struct msi_domain_info *info) +{ + struct irq_chip *chip = info->chip; + + if (!chip) + return; + + /* + * irq_write_msi_msg should not be set by the caller + */ + if (!chip->irq_write_msi_msg) + chip->irq_write_msi_msg = cdx_msi_write_msg; +} +/** + * cdx_msi_create_irq_domain - Create a CDX MSI interrupt domain + * @fwnode: Optional firmware node of the interrupt controller + * @info: MSI domain info + * @parent: Parent irq domain + * + * Updates the domain and chip ops and creates a CDX MSI + * interrupt domain. + * + * Returns: + * A domain pointer or NULL in case of failure. + */ +static struct irq_domain *cdx_msi_create_irq_domain(struct fwnode_handle *fwnode, + struct msi_domain_info *info, + struct irq_domain *parent) +{ + if (WARN_ON((info->flags & MSI_FLAG_LEVEL_CAPABLE))) + info->flags &= ~MSI_FLAG_LEVEL_CAPABLE; + if (info->flags & MSI_FLAG_USE_DEF_DOM_OPS) + cdx_msi_update_dom_ops(info); + if (info->flags & MSI_FLAG_USE_DEF_CHIP_OPS) + cdx_msi_update_chip_ops(info); + info->flags |= MSI_FLAG_ALLOC_SIMPLE_MSI_DESCS | MSI_FLAG_FREE_MSI_DESCS; + + return msi_create_irq_domain(fwnode, info, parent); +} + +int cdx_msi_domain_alloc_irqs(struct device *dev, unsigned int irq_count) +{ + struct irq_domain *msi_domain; + int ret; + + msi_domain = dev_get_msi_domain(dev); + if (!msi_domain) { + dev_err(dev, "msi domain get failed\n"); + return -EINVAL; + } + + ret = msi_setup_device_data(dev); + if (ret) { + dev_err(dev, "msi setup device failed: %d\n", ret); + return ret; + } + + msi_lock_descs(dev); + if (msi_first_desc(dev, MSI_DESC_ALL)) + ret = -EINVAL; + msi_unlock_descs(dev); + if (ret) { + dev_err(dev, "msi setup device failed: %d\n", ret); + return ret; + } + + ret = msi_domain_alloc_irqs(msi_domain, dev, irq_count); + if (ret) + dev_err(dev, "Failed to allocate IRQs\n"); + + return ret; +} +EXPORT_SYMBOL(cdx_msi_domain_alloc_irqs); + +void cdx_msi_domain_free_irqs(struct device *dev) +{ + struct irq_domain *msi_domain; + + msi_domain = dev_get_msi_domain(dev); + if (!msi_domain) + return; + + msi_domain_free_irqs(msi_domain, dev); +} +EXPORT_SYMBOL(cdx_msi_domain_free_irqs); + +static struct irq_chip cdx_msi_irq_chip = { + .name = "CDX-MSI", + .irq_mask = irq_chip_mask_parent, + .irq_unmask = irq_chip_unmask_parent, + .irq_eoi = irq_chip_eoi_parent, + .irq_set_affinity = msi_domain_set_affinity +}; + +static int cdx_msi_prepare(struct irq_domain *msi_domain, + struct device *dev, + int nvec, msi_alloc_info_t *info) +{ + struct cdx_device *cdx_dev = to_cdx_device(dev); + struct msi_domain_info *msi_info; + struct device *parent = dev->parent; + u32 dev_id; + int ret; + + /* Retrieve device ID from requestor ID using parent device */ + ret = of_map_id(parent->of_node, cdx_dev->req_id, "msi-map", + "msi-map-mask", NULL, &dev_id); + if (ret) { + dev_err(dev, "of_map_id failed for MSI: %d\n", ret); + return ret; + } + + /* Set the device Id to be passed to the GIC-ITS */ + info->scratchpad[0].ul = dev_id; + + msi_info = msi_get_domain_info(msi_domain->parent); + + /* Allocate at least 32 MSIs, and always as a power of 2 */ + nvec = max_t(int, 32, roundup_pow_of_two(nvec)); + return msi_info->ops->msi_prepare(msi_domain->parent, dev, nvec, info); +} + +static struct msi_domain_ops cdx_msi_ops __ro_after_init = { + .msi_prepare = cdx_msi_prepare, +}; + +static struct msi_domain_info cdx_msi_domain_info = { + .flags = (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS), + .ops = &cdx_msi_ops, + .chip = &cdx_msi_irq_chip, +}; + +struct irq_domain *cdx_msi_domain_init(struct device *dev) +{ + struct irq_domain *parent; + struct irq_domain *cdx_msi_domain; + struct fwnode_handle *fwnode_handle; + struct device_node *parent_node; + struct device_node *np = dev->of_node; + + fwnode_handle = of_node_to_fwnode(np); + + parent_node = of_parse_phandle(np, "msi-map", 1); + if (!parent_node) { + dev_err(dev, "msi-map not present on cdx controller\n"); + return NULL; + } + + parent = irq_find_matching_fwnode(of_node_to_fwnode(parent_node), + DOMAIN_BUS_NEXUS); + if (!parent || !msi_get_domain_info(parent)) { + dev_err(dev, "unable to locate ITS domain\n"); + return NULL; + } + + cdx_msi_domain = cdx_msi_create_irq_domain(fwnode_handle, + &cdx_msi_domain_info, parent); + if (!cdx_msi_domain) { + dev_err(dev, "unable to create CDX-MSI domain\n"); + return NULL; + } + + dev_dbg(dev, "CDX-MSI domain created\n"); + + return cdx_msi_domain; +} + +struct irq_domain *cdx_find_msi_domain(struct device *parent) +{ + return irq_find_host(parent->of_node); +} diff --git a/drivers/bus/cdx/mcdi_stubs.c b/drivers/bus/cdx/mcdi_stubs.c index cc9d30fa02f8..2c8db1f5a057 100644 --- a/drivers/bus/cdx/mcdi_stubs.c +++ b/drivers/bus/cdx/mcdi_stubs.c @@ -45,6 +45,7 @@ int cdx_mcdi_get_func_config(struct cdx_mcdi_t *cdx_mcdi, dev_params->res_count = 2; dev_params->req_id = 0x250; + dev_params->num_msi = 4; dev_params->vendor = 0x10ee; dev_params->device = 0x8084; dev_params->bus_id = bus_id; diff --git a/include/linux/cdx/cdx_bus.h b/include/linux/cdx/cdx_bus.h index 6e870b2c87d9..bf86024783d2 100644 --- a/include/linux/cdx/cdx_bus.h +++ b/include/linux/cdx/cdx_bus.h @@ -25,6 +25,7 @@ * @dma_mask: Default DMA mask * @flags: CDX device flags * @req_id: Requestor ID associated with CDX device + * @num_msi: Number of MSI's supported by the device * @driver_override: driver name to force a match; do not set directly, * because core frees it; use driver_set_override() to * set or clear it. @@ -40,6 +41,7 @@ struct cdx_device { u64 dma_mask; u16 flags; u32 req_id; + u32 num_msi; const char *driver_override; }; @@ -90,4 +92,21 @@ void cdx_driver_unregister(struct cdx_driver *driver); extern struct bus_type cdx_bus_type; +/** + * cdx_msi_domain_alloc_irqs - Allocate MSI's for the CDX device + * @dev: device pointer + * @irq_count: Number of MSI's to be allocated + * + * Return 0 for success, -errno on failure + */ +int cdx_msi_domain_alloc_irqs(struct device *dev, unsigned int irq_count); + +/** + * cdx_msi_domain_free_irqs - Free MSI's for CDX device + * @dev: device pointer + * + * Return 0 for success, -errno on failure + */ +void cdx_msi_domain_free_irqs(struct device *dev); + #endif /* _CDX_BUS_H_ */ From patchwork Tue Sep 6 13:47:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nipun Gupta X-Patchwork-Id: 12967590 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 908CAECAAA1 for ; Tue, 6 Sep 2022 14:20:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236854AbiIFOUS (ORCPT ); Tue, 6 Sep 2022 10:20:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38058 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241972AbiIFOTI (ORCPT ); Tue, 6 Sep 2022 10:19:08 -0400 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2062.outbound.protection.outlook.com [40.107.244.62]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AEEE42F65E; Tue, 6 Sep 2022 06:50:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PDosOneLBOD/vZLb77ApRCuBcAmUYng7y0C7AUVrzsA6IjYp7kG74kTkVHPFmsjIyUf0rUpbXRivX2eVPDn8zlMox5yu3w63WJ/WIF28bFzNhfr+JS3Mmya7lgtuj5sLi1koOROKEJ4Qwm7tWsWaxzpvqtuMchtAgmkgy1MMcivgQUlyE7EcnMlxePBxhg7wplg6hSSHrMdhgeu+PbVTymumuu/x1U1524ELLE4D9HloMs1IUtFgt7R4Cg3Nm2DJ97gHqttCEyAq/CeavDBpv+Og+87iyHMELgkp8n18vG64OYqzMuQ7w2Ivgz8JunxFdaCakr2W8y/bpVE9m/Fxow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Y57p5ouHP4x8F9d9gwp37pMkcc6TTtTd0Z/zpH6bcTs=; b=eyQt9UqpoyVwCPncYVvTxL/D35Jv7haxOv7y2XVHyrJP21SCaK0NlEGIx7KwmpKSmaKZFOcO1P833C3XpSdKPS95gd6gFomsTiThtu8P48cCH9P/9ezMuwj4DqY+gDKgT7yJguVGaU7+Yooe5RH1Vsr3cc1IzOi6RuqJOcg9Xyz7dRIVG2yiyJYgn5IdRGmrLj3bTlRfj4gY7gHuV2YpqVnxOBH1N976ByDOtj8kXrYpz6MO3ZIGOEP8QPzQF6mJ4fWhaHvCNT9g8zmnsCOH6fHNYPT+k8E4x2Tap2JykmX77Zy4VPerAmVfORyKqeUDzNp5iU039/EiAVxRHc8XYA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y57p5ouHP4x8F9d9gwp37pMkcc6TTtTd0Z/zpH6bcTs=; b=GJ/nlLLWmexxn39Q97Ag/1fQR7UH0S1e6mudd9LF0xYwYkNjeSV6PD6mucw3Xn3PUMy3K1Etr9ZZHjnA/+LeN6g37zdjH7ef5WWxmccnNuI1qBclZOWlFABitNcx1msqy6KE2nOp/7GXNPcACClMMIRjrMO98OaBlixTFhR66iY= Received: from BN0PR04CA0133.namprd04.prod.outlook.com (2603:10b6:408:ed::18) by DM6PR12MB4202.namprd12.prod.outlook.com (2603:10b6:5:219::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Tue, 6 Sep 2022 13:49:04 +0000 Received: from BN8NAM11FT039.eop-nam11.prod.protection.outlook.com (2603:10b6:408:ed:cafe::25) by BN0PR04CA0133.outlook.office365.com (2603:10b6:408:ed::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10 via Frontend Transport; Tue, 6 Sep 2022 13:49:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT039.mail.protection.outlook.com (10.13.177.169) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5588.10 via Frontend Transport; Tue, 6 Sep 2022 13:49:03 +0000 Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 6 Sep 2022 08:49:03 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 6 Sep 2022 06:49:02 -0700 Received: from xhdipdslab49.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.28 via Frontend Transport; Tue, 6 Sep 2022 08:48:53 -0500 From: Nipun Gupta To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , CC: , , , , , , Nipun Gupta Subject: [RFC PATCH v3 5/7] bus/cdx: add bus and device attributes Date: Tue, 6 Sep 2022 19:17:59 +0530 Message-ID: <20220906134801.4079497-6-nipun.gupta@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906134801.4079497-1-nipun.gupta@amd.com> References: <20220803122655.100254-1-nipun.gupta@amd.com> <20220906134801.4079497-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: eb72c8db-bfc4-4e71-64f6-08da900e8fa4 X-MS-TrafficTypeDiagnostic: DM6PR12MB4202:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: gneOVv2jGrgvkTEE79Utgcm8kX2tgoleuRqJG14wR//ocqqT4qCRucHUqScaEktwc5hB1TxvPFEncLN6fTQHiORa7t8RqRMg+LUU6wfA+cFOPZiQqmXs6HGeAqY5Tw6mevwByQ8V6uLWXOCX9Rfq6rlUw6mEJnlkO6DwIccSSMCtr+yR/rp5npdQl5nL3R2coTFgKsadr+6XB7QivyGPb2TpRu0bdODEi3s7TJLCAtL+6c+RgShYlFLdhgL/4QMkbMGH+WNPsnACBQVdFim4brkFD74flfH3fAbfSbvt/5X8l7YGRvjxeEQdCixeJ+wpTTAoB5gL9YCYYO+7NZV+hXdUxhw+gcuhoNtZv7JYIYw0XuzRthIO4cu4+e75Zivd7LYUtyriyYst8RcHivWccMbzJYbHQBm2ZC1xDPEPT1HmbZC9COiscMb+IcaBp9tM4d3JXBr0Y75r5jR7FVj3/ELaHqImRYxwlmCzEfnAwUBQH88xutXAUo+qEZIAFxhZWFTApG7XEq6UFJUAy939LRHjDf05tV7T57sp+0UjUbttD4JPY6WUr3Wsy4oZSzTgSLa7Aa2O6FQ8lbyD2tSjDOR0omwK+5klGG/2S37LI67TMif1gjI49oVflcwReluqqf9CFLGGaNksf70bxjejOuEvr2evP80MqZX5QJqCtsA9zdcIfWnt0i3FA4pVcZdD8JlQ1pJAptOYgPV4qfUNNLx7Prd5UaE92ohQABpUVc6Zpao3p3ilqRZkVZ3Lnl2nMe1xR6AyJQRpqVnm4syzxwjfTuzkkDJlYSZeeSS0xq+4qq+T+M/CAGZcJfsaLFyXNGKeEZy/J6O1WV8wokXi66souE/Z26AzV5gM75zUCucXMERPRcniAV/oEQtA+AZM X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(39860400002)(376002)(346002)(396003)(136003)(46966006)(40470700004)(36840700001)(4326008)(44832011)(82310400005)(40480700001)(110136005)(316002)(86362001)(54906003)(36756003)(82740400003)(2906002)(356005)(921005)(81166007)(40460700003)(8936002)(5660300002)(7416002)(36860700001)(83380400001)(70586007)(8676002)(47076005)(70206006)(426003)(336012)(41300700001)(2616005)(186003)(1076003)(478600001)(26005)(83996005)(36900700001)(2101003);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2022 13:49:03.9579 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: eb72c8db-bfc4-4e71-64f6-08da900e8fa4 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT039.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4202 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This change adds te support for rescanning and reset of the CDX buses, as well as option to optionally reset any devices on the bus. It also enables sysfs entry for vendor and device id. Sysfs entries are provided in CDX controller: - rescan of the CDX controller. - reset all the devices present on CDX buses. Sysfs entry is provided in each of the platform device detected by the CDX controller - vendor id - device id - modalias - reset of the device. Signed-off-by: Puneet Gupta Signed-off-by: Nipun Gupta --- Documentation/ABI/testing/sysfs-bus-cdx | 54 +++++++++ drivers/bus/cdx/cdx.c | 148 ++++++++++++++++++++++++ drivers/bus/cdx/mcdi_stubs.c | 6 + drivers/bus/cdx/mcdi_stubs.h | 11 ++ include/linux/cdx/cdx_bus.h | 8 ++ 5 files changed, 227 insertions(+) create mode 100644 Documentation/ABI/testing/sysfs-bus-cdx diff --git a/Documentation/ABI/testing/sysfs-bus-cdx b/Documentation/ABI/testing/sysfs-bus-cdx new file mode 100644 index 000000000000..5f84f0a93a66 --- /dev/null +++ b/Documentation/ABI/testing/sysfs-bus-cdx @@ -0,0 +1,54 @@ +What: /sys/bus/cdx/rescan +Date: August 2022 +Contact: puneet.gupta@amd.com +Description: + Writing 1 to this would cause rescan of the bus + and devices on the CDX bus. Any new devices would + be scanned and added to the list of linux devices + and any devices removed are also deleted from linux. + + For example:: + + # echo 1 > /sys/bus/cdx/rescan + +What: /sys/bus/cdx/reset_all +Date: August 2022 +Contact: puneet.gupta@amd.com +Description: + Writing 1 to this would reset all the devices present + on the CDX bus + + For example:: + + # echo 1 > /sys/bus/cdx/reset_all + +What: /sys/bus/cdx/devices/.../vendor +Date: August 2022 +Contact: nipun.gupta@amd.com +Description: + Vendor ID for this CDX device + +What: /sys/bus/cdx/devices/.../device +Date: August 2022 +Contact: nipun.gupta@amd.com +Description: + Device ID for this CDX device + +What: /sys/bus/cdx/devices/.../reset +What: /sys/bus/cdx/devices/.../reset +Date: August 2022 +Contact: nipun.gupta@amd.com +Description: + Writing 1 to this would reset the CDX device + + For example:: + + # echo 1 > /sys/bus/cdx/.../reset + +What: /sys/bus/cdx/devices/.../modalias +Date: August 2022 +Contact: nipun.gupta@amd.com +Description: + A CDX device has modalias: + + - v`vendor`d`device` diff --git a/drivers/bus/cdx/cdx.c b/drivers/bus/cdx/cdx.c index 02ececce1c84..9b7a69de6a8f 100644 --- a/drivers/bus/cdx/cdx.c +++ b/drivers/bus/cdx/cdx.c @@ -32,6 +32,23 @@ struct platform_device *cdx_controller_pdev; static int cdx_bus_device_discovery(struct platform_device *pdev); +static int reset_cdx_device(struct device *dev, void * __always_unused data) +{ + struct platform_device *cdx_bus_pdev = to_platform_device(dev->parent); + struct cdx_device *cdx_dev = to_cdx_device(dev); + struct cdx_mcdi_t *cdx_mcdi = platform_get_drvdata(cdx_bus_pdev); + + /* MCDI FW: reset particular device represented by bus:func */ + cdx_mcdi_func_reset(cdx_mcdi, cdx_dev->bus_id, cdx_dev->func_id); + + return 0; +} + +int cdx_dev_reset(struct device *dev) +{ + return reset_cdx_device(dev, NULL); +} + static int cdx_unregister_device(struct device *dev, void * __always_unused data) { @@ -139,10 +156,141 @@ static int cdx_dma_configure(struct device *dev) return 0; } +static ssize_t vendor_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cdx_device *cdx_dev = to_cdx_device(dev); + + return sprintf(buf, "0x%x\n", cdx_dev->vendor); +} +static DEVICE_ATTR_RO(vendor); + +static ssize_t device_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cdx_device *cdx_dev = to_cdx_device(dev); + + return sprintf(buf, "0x%x\n", cdx_dev->device); +} +static DEVICE_ATTR_RO(device); + +static ssize_t reset_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + int ret = 0; + bool reset = count > 0 && *buf != '0'; + + if (!reset) + return count; + + ret = reset_cdx_device(dev, NULL); + if (ret) + return ret; + + return count; +} +static DEVICE_ATTR_WO(reset); + +static ssize_t modalias_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cdx_device *cdx_dev = to_cdx_device(dev); + + return sprintf(buf, "cdx:v%08Xd%d\n", cdx_dev->vendor, + cdx_dev->device); +} +static DEVICE_ATTR_RO(modalias); + +static ssize_t driver_override_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct cdx_device *cdx_dev = to_cdx_device(dev); + int ret; + + if (WARN_ON(dev->bus != &cdx_bus_type)) + return -EINVAL; + + ret = driver_set_override(dev, &cdx_dev->driver_override, buf, count); + if (ret) + return ret; + + return count; +} + +static ssize_t driver_override_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + struct cdx_device *cdx_dev = to_cdx_device(dev); + + return snprintf(buf, PAGE_SIZE, "%s\n", cdx_dev->driver_override); +} +static DEVICE_ATTR_RW(driver_override); + +static struct attribute *cdx_dev_attrs[] = { + &dev_attr_reset.attr, + &dev_attr_vendor.attr, + &dev_attr_device.attr, + &dev_attr_modalias.attr, + &dev_attr_driver_override.attr, + NULL, +}; +ATTRIBUTE_GROUPS(cdx_dev); + +static ssize_t rescan_store(struct bus_type *bus, + const char *buf, size_t count) +{ + bool rescan = count > 0 && *buf != '0'; + int ret = 0; + + if (!rescan) + return count; + + /* Unregister all the devices on the bus */ + cdx_unregister_devices(&cdx_bus_type); + + /* Rescan all the devices */ + ret = cdx_bus_device_discovery(cdx_controller_pdev); + if (ret) + return ret; + + return count; +} +static BUS_ATTR_WO(rescan); + +static ssize_t reset_all_store(struct bus_type *bus, + const char *buf, size_t count) +{ + bool reset = count > 0 && *buf != '0'; + int ret = 0; + + if (!reset) + return count; + + /* Reset all the devices attached to cdx bus */ + ret = bus_for_each_dev(bus, NULL, NULL, reset_cdx_device); + if (ret) { + pr_err("error in CDX bus reset\n"); + return 0; + } + + return count; +} +static BUS_ATTR_WO(reset_all); + +static struct attribute *cdx_bus_attrs[] = { + &bus_attr_rescan.attr, + &bus_attr_reset_all.attr, + NULL, +}; +ATTRIBUTE_GROUPS(cdx_bus); + struct bus_type cdx_bus_type = { .name = "cdx", .match = cdx_bus_match, .dma_configure = cdx_dma_configure, + .dev_groups = cdx_dev_groups, + .bus_groups = cdx_bus_groups, }; EXPORT_SYMBOL_GPL(cdx_bus_type); diff --git a/drivers/bus/cdx/mcdi_stubs.c b/drivers/bus/cdx/mcdi_stubs.c index 2c8db1f5a057..2bc042d2b061 100644 --- a/drivers/bus/cdx/mcdi_stubs.c +++ b/drivers/bus/cdx/mcdi_stubs.c @@ -53,3 +53,9 @@ int cdx_mcdi_get_func_config(struct cdx_mcdi_t *cdx_mcdi, return 0; } + +int cdx_mcdi_func_reset(struct cdx_mcdi_t *cdx_mcdi, + uint8_t bus_id, uint8_t func_id) +{ + return 0; +} diff --git a/drivers/bus/cdx/mcdi_stubs.h b/drivers/bus/cdx/mcdi_stubs.h index 7b6f79d48ce9..535218bcdce0 100644 --- a/drivers/bus/cdx/mcdi_stubs.h +++ b/drivers/bus/cdx/mcdi_stubs.h @@ -73,4 +73,15 @@ int cdx_mcdi_get_func_config(struct cdx_mcdi_t *cdx_mcdi, uint8_t bus_id, uint8_t func_id, struct cdx_dev_params_t *dev_params); +/** + * cdx_mcdi_func_reset - Reset cdx device represented by bus_id:func_id + * @cdx_mcdi: pointer to MCDI interface. + * @bus_num: Bus number. + * @func_id: Function number. + * + * Return 0 on success, <0 on failure + */ +int cdx_mcdi_func_reset(struct cdx_mcdi_t *cdx_mcdi, + uint8_t bus_id, uint8_t func_id); + #endif /* _MCDI_STUBS_H_ */ diff --git a/include/linux/cdx/cdx_bus.h b/include/linux/cdx/cdx_bus.h index bf86024783d2..8306b4d3fd82 100644 --- a/include/linux/cdx/cdx_bus.h +++ b/include/linux/cdx/cdx_bus.h @@ -109,4 +109,12 @@ int cdx_msi_domain_alloc_irqs(struct device *dev, unsigned int irq_count); */ void cdx_msi_domain_free_irqs(struct device *dev); +/** + * cdx_dev_reset - Reset CDX device + * @dev: device pointer + * + * Return 0 for success, -errno on failure + */ +int cdx_dev_reset(struct device *dev); + #endif /* _CDX_BUS_H_ */ From patchwork Tue Sep 6 13:48:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nipun Gupta X-Patchwork-Id: 12967618 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50469C6FA83 for ; Tue, 6 Sep 2022 14:30:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241850AbiIFOaC (ORCPT ); Tue, 6 Sep 2022 10:30:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242284AbiIFO25 (ORCPT ); Tue, 6 Sep 2022 10:28:57 -0400 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2068.outbound.protection.outlook.com [40.107.223.68]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 715F290C4E; Tue, 6 Sep 2022 06:55:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WhSJ/hEQHkrHv78eeyaNXfV3TDRPKt+3VJHh9PAKdbpH6K9JXMXrWPsCTnqy3n2YEYmuCazKNxQ6Yz3BIUHW3n1i9RxTymwkOyq/172pAnDbwhYYHb8e4Nd6SzOsKA8/ltNVwh35JwOmiGPt/RShzB2ToXly982P28czhm4bDyGmcOqCGecGOSbeyeL/BVZLYsRpEzBRecLFfEAZPjj0dc5EpmVn4W2kS4c5ucVsHKQXDrhzLOju4nmjwfKdnp5XjFQOCrIoZxnwphaWTrPHQ/f5CakTJkyPfRv8/dYk1XNd3EkRjyiAkUwnOdfj3XUohOYZjXdHfmyai/8JKHQUcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wpweB7oA0mMZDfBd2KNin3ucSqF3KYPf7GJcDMrWzRY=; b=W6P2RK6x/4NTLsGyx8JyOLAmon+79dSjZhMEQpFp7z0jKDTXDACuj3oC1QYRTJcjequTpY6vrl3RWAi3vSh2uJp+BPMwLK/1b/mJsOU3le+/DDeQKQNmhULAdekgTWjSHiHb5Jph+IF3W+57G5rOUxy5OYHTjo/LrJpzb3WfgF192LqxifbYqtsjVwMOu2LkIfVoqtcd7OgrgHekNqMeg+GBa6dsty2dSGkv788OZwSvO2sZkpzySi9GqyOeTYyYn2tCKKTE3ZL/jH8LChnbqf3bzY9TpITSZndzTgwj+bEF+k2elLIPiRW6a+LPMJLhjix6M32CLSEl2lrcigF20Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wpweB7oA0mMZDfBd2KNin3ucSqF3KYPf7GJcDMrWzRY=; b=rs1lpTM6v3rUr8ECTKlIQs+1+KevgfwUxKXmVQl+YkJIhPbpgfSzlo75PcimXhYkRu74VzgNe0d5hlc3fPfq5P/YdkDsOkEk2vAJqgVGG7Ondf8N5l0P6c6j8pGqAccFYcFnH0HnKJCzdA98Y2hLg3kNoMH9g2DP9GT2tKtSJ6A= Received: from CY5PR14CA0016.namprd14.prod.outlook.com (2603:10b6:930:2::22) by CH0PR12MB5043.namprd12.prod.outlook.com (2603:10b6:610:e2::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Tue, 6 Sep 2022 13:49:14 +0000 Received: from CY4PEPF0000B8EE.namprd05.prod.outlook.com (2603:10b6:930:2:cafe::6a) by CY5PR14CA0016.outlook.office365.com (2603:10b6:930:2::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.12 via Frontend Transport; Tue, 6 Sep 2022 13:49:14 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C Received: from SATLEXMB03.amd.com (165.204.84.17) by CY4PEPF0000B8EE.mail.protection.outlook.com (10.167.241.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5612.9 via Frontend Transport; Tue, 6 Sep 2022 13:49:14 +0000 Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 6 Sep 2022 08:49:13 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 6 Sep 2022 06:49:12 -0700 Received: from xhdipdslab49.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.28 via Frontend Transport; Tue, 6 Sep 2022 08:49:03 -0500 From: Nipun Gupta To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , CC: , , , , , , Nipun Gupta Subject: [RFC PATCH v3 6/7] vfio/cdx: add support for CDX bus Date: Tue, 6 Sep 2022 19:18:00 +0530 Message-ID: <20220906134801.4079497-7-nipun.gupta@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906134801.4079497-1-nipun.gupta@amd.com> References: <20220803122655.100254-1-nipun.gupta@amd.com> <20220906134801.4079497-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000B8EE:EE_|CH0PR12MB5043:EE_ X-MS-Office365-Filtering-Correlation-Id: 95dc4e33-e60f-426e-a555-08da900e95c9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uelHK8Yhw5qrAwUOJWicBVCZcZhvxFIlnFTk0T5P4PrSmB2tw3xCzUtFQAObs7Qp0DGrlL+9P+qFa9f/pjwx90vf7kMiF/L/LNjyXaIN+UzmR/+tJxlOpdteRH32IaRjUY5MVxN2QXxqZMzeDamv/AYJg8mZTNJLODzKXNC6Cf/fbAiIVSoEVD4bR0lxlTS/j/Bx1lHmHCMmB8kA0FfViJlHpznBjC2PlI2PR5usVLMBD1BQuWeNvnm7yta5GtL7tSaARebZDmFgD0DDmQt/jwpm8QBAHic16QQPsIUnUHGPFFzX08sGQNd9PmOPSPIFxQIQVZ6vKuMXxgAMfRPpNjj55uVSDBiMjHs2/w8r5+/sgZIF7KUqGA7/HksoPun/RsH7lRYFIQ05YpaKGf7Y/Dok2KJ42nmAx0U6o27VxnJSkZx8mDLwOqoGGPvaJEMImQAkNNP1M3D5UcgORNeg9Ldhst6Izq9P1ih4vpjBjAhpPh94fiMiHIc+K0Tv5e6Xwv3GasuGBtAZIB/+/8BnJWMtqvt0nSCEHGK81oi//bFMhe7FP0gBkTQVSDf5alP9VfFw5Z6JecC7NJa1jk7t6DOUoOjr+ioxF770PnZKpmjeycwumP3W1nEVRpYpBf6w80RrREV2lJd+DhwQ9mEmOqYi2VZ8JWGt0BfX2SDFeCjPdpBRQpqFeO77M+OJRJTQ8IzPj0JGdJ3uIIIF0HO3aJKam9Iv3Jkw8iakhXFxFCl3IXoyuX3KppvnWVlCrM05HJaSSHLN/kh4l8KGHc3lZwd4iBvPIbYbkiZuU3ayc/BWzJ89Uqh78QnlTmTk9lfyzMIXAcX7mAt6bhuIOL5UzSZ7bsfPsGwY55Yt6ekfYj2KZ9X52eDB2x4nD862KCo+ X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(376002)(39860400002)(396003)(346002)(136003)(36840700001)(46966006)(40470700004)(82740400003)(478600001)(2616005)(426003)(41300700001)(47076005)(336012)(83380400001)(26005)(1076003)(8936002)(82310400005)(186003)(44832011)(7416002)(30864003)(5660300002)(316002)(110136005)(40460700003)(40480700001)(54906003)(2906002)(4326008)(70586007)(70206006)(36756003)(8676002)(86362001)(81166007)(921005)(356005)(36860700001)(83996005)(36900700001)(2101003);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2022 13:49:14.1589 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 95dc4e33-e60f-426e-a555-08da900e95c9 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000B8EE.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB5043 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org vfio-cdx driver enables IOCTLs for user space to query MMIO regions for CDX devices and mmap them. This change also adds support for reset of CDX devices. This change adds the VFIO CDX driver and enables the following ioctls for CDX devices: - VFIO_DEVICE_GET_INFO: - VFIO_DEVICE_GET_REGION_INFO - VFIO_DEVICE_RESET Signed-off-by: Nipun Gupta --- MAINTAINERS | 1 + drivers/vfio/Makefile | 1 + drivers/vfio/cdx/Kconfig | 10 + drivers/vfio/cdx/Makefile | 4 + drivers/vfio/cdx/vfio_cdx.c | 284 ++++++++++++++++++++++++++++ drivers/vfio/cdx/vfio_cdx_private.h | 32 ++++ 6 files changed, 332 insertions(+) create mode 100644 drivers/vfio/cdx/Kconfig create mode 100644 drivers/vfio/cdx/Makefile create mode 100644 drivers/vfio/cdx/vfio_cdx.c create mode 100644 drivers/vfio/cdx/vfio_cdx_private.h diff --git a/MAINTAINERS b/MAINTAINERS index 61af11c9fe06..e66a93ca2218 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -940,6 +940,7 @@ M: Nikhil Agarwal S: Maintained F: Documentation/devicetree/bindings/bus/xlnx,cdx.yaml F: drivers/bus/cdx/* +F: drivers/vfio/cdx/* F: include/linux/cdx/* AMD CRYPTOGRAPHIC COPROCESSOR (CCP) DRIVER - SEV SUPPORT diff --git a/drivers/vfio/Makefile b/drivers/vfio/Makefile index 1a32357592e3..7f4f5c891c28 100644 --- a/drivers/vfio/Makefile +++ b/drivers/vfio/Makefile @@ -12,3 +12,4 @@ obj-$(CONFIG_VFIO_PCI) += pci/ obj-$(CONFIG_VFIO_PLATFORM) += platform/ obj-$(CONFIG_VFIO_MDEV) += mdev/ obj-$(CONFIG_VFIO_FSL_MC) += fsl-mc/ +obj-$(CONFIG_CDX_BUS) += cdx/ diff --git a/drivers/vfio/cdx/Kconfig b/drivers/vfio/cdx/Kconfig new file mode 100644 index 000000000000..728d7294a2c7 --- /dev/null +++ b/drivers/vfio/cdx/Kconfig @@ -0,0 +1,10 @@ +config VFIO_CDX + tristate "VFIO support for CDX bus devices" + depends on CDX_BUS + select EVENTFD + help + Driver to enable VFIO support for the devices on CDX bus. + This is required to make use of CDX devices present in + the system using the VFIO framework. + + If you don't know what to do here, say N. diff --git a/drivers/vfio/cdx/Makefile b/drivers/vfio/cdx/Makefile new file mode 100644 index 000000000000..7a4183d76802 --- /dev/null +++ b/drivers/vfio/cdx/Makefile @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: (GPL-2.0+ OR BSD-3-Clause) + +#vfio-fsl-mc-y := vfio_cdx.o vfio_cdx_intr.o +obj-y += vfio_cdx.o vfio_cdx_intr.o diff --git a/drivers/vfio/cdx/vfio_cdx.c b/drivers/vfio/cdx/vfio_cdx.c new file mode 100644 index 000000000000..2e5bd494057a --- /dev/null +++ b/drivers/vfio/cdx/vfio_cdx.c @@ -0,0 +1,284 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2022, Advanced Micro Devices, Inc. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "vfio_cdx_private.h" + +static struct cdx_driver vfio_cdx_driver; + +static int vfio_cdx_open_device(struct vfio_device *core_vdev) +{ + struct vfio_cdx_device *vdev = + container_of(core_vdev, struct vfio_cdx_device, vdev); + struct cdx_device *cdx_dev = vdev->cdx_dev; + int count = cdx_dev->res_count; + int i; + + vdev->regions = kcalloc(count, sizeof(struct vfio_cdx_region), + GFP_KERNEL); + if (!vdev->regions) + return -ENOMEM; + + for (i = 0; i < count; i++) { + struct resource *res = &cdx_dev->res[i]; + + vdev->regions[i].addr = res->start; + vdev->regions[i].size = resource_size(res); + vdev->regions[i].type = res->flags; + /* + * Only regions addressed with PAGE granularity may be + * MMAPed securely. + */ + if (!(vdev->regions[i].addr & ~PAGE_MASK) && + !(vdev->regions[i].size & ~PAGE_MASK)) + vdev->regions[i].flags |= + VFIO_REGION_INFO_FLAG_MMAP; + vdev->regions[i].flags |= VFIO_REGION_INFO_FLAG_READ; + if (!(cdx_dev->res[i].flags & IORESOURCE_READONLY)) + vdev->regions[i].flags |= VFIO_REGION_INFO_FLAG_WRITE; + } + + return 0; +} + +static void vfio_cdx_regions_cleanup(struct vfio_cdx_device *vdev) +{ + kfree(vdev->regions); +} + +static int vfio_cdx_reset_device(struct vfio_cdx_device *vdev) +{ + struct vfio_device *core_vdev = &vdev->vdev; + + return cdx_dev_reset(core_vdev->dev); +} + +static void vfio_cdx_close_device(struct vfio_device *core_vdev) +{ + struct vfio_cdx_device *vdev = + container_of(core_vdev, struct vfio_cdx_device, vdev); + int ret; + + vfio_cdx_regions_cleanup(vdev); + + /* reset the device before cleaning up the interrupts */ + ret = vfio_cdx_reset_device(vdev); + if (WARN_ON(ret)) + dev_warn(core_vdev->dev, + "VFIO_CDX: reset device has failed (%d)\n", ret); +} + +static long vfio_cdx_ioctl(struct vfio_device *core_vdev, + unsigned int cmd, unsigned long arg) +{ + unsigned long minsz; + struct vfio_cdx_device *vdev = + container_of(core_vdev, struct vfio_cdx_device, vdev); + struct cdx_device *cdx_dev = vdev->cdx_dev; + + switch (cmd) { + case VFIO_DEVICE_GET_INFO: + { + struct vfio_device_info info; + + minsz = offsetofend(struct vfio_device_info, num_irqs); + + if (copy_from_user(&info, (void __user *)arg, minsz)) + return -EFAULT; + + if (info.argsz < minsz) + return -EINVAL; + + info.flags = VFIO_DEVICE_FLAGS_RESET; + + info.num_regions = cdx_dev->res_count; + info.num_irqs = 1; + + return copy_to_user((void __user *)arg, &info, minsz) ? + -EFAULT : 0; + } + case VFIO_DEVICE_GET_REGION_INFO: + { + struct vfio_region_info info; + + minsz = offsetofend(struct vfio_region_info, offset); + + if (copy_from_user(&info, (void __user *)arg, minsz)) + return -EFAULT; + + if (info.argsz < minsz) + return -EINVAL; + + if (info.index >= cdx_dev->res_count) + return -EINVAL; + + /* map offset to the physical address */ + info.offset = VFIO_CDX_INDEX_TO_OFFSET(info.index); + info.size = vdev->regions[info.index].size; + info.flags = vdev->regions[info.index].flags; + + if (copy_to_user((void __user *)arg, &info, minsz)) + return -EFAULT; + return 0; + } + case VFIO_DEVICE_RESET: + { + return vfio_cdx_reset_device(vdev); + } + default: + return -ENOTTY; + } +} + +static int vfio_cdx_mmap_mmio(struct vfio_cdx_region region, + struct vm_area_struct *vma) +{ + u64 size = vma->vm_end - vma->vm_start; + u64 pgoff, base; + u8 region_cacheable; + + pgoff = vma->vm_pgoff & + ((1U << (VFIO_CDX_OFFSET_SHIFT - PAGE_SHIFT)) - 1); + base = pgoff << PAGE_SHIFT; + + if (region.size < PAGE_SIZE || base + size > region.size) + return -EINVAL; + + region_cacheable = (region.type & IORESOURCE_MEM); + if (!region_cacheable) + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); + + vma->vm_pgoff = (region.addr >> PAGE_SHIFT) + pgoff; + + return remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff, + size, vma->vm_page_prot); +} + +static int vfio_cdx_mmap(struct vfio_device *core_vdev, + struct vm_area_struct *vma) +{ + struct vfio_cdx_device *vdev = + container_of(core_vdev, struct vfio_cdx_device, vdev); + struct cdx_device *cdx_dev = vdev->cdx_dev; + unsigned int index; + + index = vma->vm_pgoff >> (VFIO_CDX_OFFSET_SHIFT - PAGE_SHIFT); + + if (vma->vm_end < vma->vm_start) + return -EINVAL; + if (vma->vm_start & ~PAGE_MASK) + return -EINVAL; + if (vma->vm_end & ~PAGE_MASK) + return -EINVAL; + if (!(vma->vm_flags & VM_SHARED)) + return -EINVAL; + if (index >= cdx_dev->res_count) + return -EINVAL; + + if (!(vdev->regions[index].flags & VFIO_REGION_INFO_FLAG_MMAP)) + return -EINVAL; + + if (!(vdev->regions[index].flags & VFIO_REGION_INFO_FLAG_READ) + && (vma->vm_flags & VM_READ)) + return -EINVAL; + + if (!(vdev->regions[index].flags & VFIO_REGION_INFO_FLAG_WRITE) + && (vma->vm_flags & VM_WRITE)) + return -EINVAL; + + vma->vm_private_data = cdx_dev; + + return vfio_cdx_mmap_mmio(vdev->regions[index], vma); +} + +static const struct vfio_device_ops vfio_cdx_ops = { + .name = "vfio-cdx", + .open_device = vfio_cdx_open_device, + .close_device = vfio_cdx_close_device, + .ioctl = vfio_cdx_ioctl, + .mmap = vfio_cdx_mmap, +}; + +static int vfio_cdx_probe(struct cdx_device *cdx_dev) +{ + struct vfio_cdx_device *vdev; + struct device *dev = &cdx_dev->dev; + int ret; + + vdev = kzalloc(sizeof(*vdev), GFP_KERNEL); + if (!vdev) + return -ENOMEM; + + vfio_init_group_dev(&vdev->vdev, dev, &vfio_cdx_ops); + vdev->cdx_dev = cdx_dev; + vdev->dev = dev; + + ret = vfio_assign_device_set(&vdev->vdev, dev); + if (ret) { + dev_err(dev, "VFIO_CDX: vfio_assign_device_set failed\n"); + goto out_uninit; + } + + ret = vfio_register_group_dev(&vdev->vdev); + if (ret) { + dev_err(dev, "VFIO_CDX: Failed to add to vfio group\n"); + goto out_uninit; + } + + dev_set_drvdata(dev, vdev); + return 0; + +out_uninit: + vfio_uninit_group_dev(&vdev->vdev); + kfree(vdev); + return ret; +} + +static int vfio_cdx_remove(struct cdx_device *cdx_dev) +{ + struct device *dev = &cdx_dev->dev; + struct vfio_cdx_device *vdev = dev_get_drvdata(dev); + + vfio_unregister_group_dev(&vdev->vdev); + + vfio_uninit_group_dev(&vdev->vdev); + kfree(vdev); + return 0; +} + +static struct cdx_driver vfio_cdx_driver = { + .probe = vfio_cdx_probe, + .remove = vfio_cdx_remove, + .driver = { + .name = "vfio-cdx", + .owner = THIS_MODULE, + }, + .driver_managed_dma = true, +}; + +static int __init vfio_cdx_driver_init(void) +{ + return cdx_driver_register(&vfio_cdx_driver); +} + +static void __exit vfio_cdx_driver_exit(void) +{ + cdx_driver_unregister(&vfio_cdx_driver); +} + +module_init(vfio_cdx_driver_init); +module_exit(vfio_cdx_driver_exit); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("VFIO for CDX devices - User Level meta-driver"); diff --git a/drivers/vfio/cdx/vfio_cdx_private.h b/drivers/vfio/cdx/vfio_cdx_private.h new file mode 100644 index 000000000000..d87b55663462 --- /dev/null +++ b/drivers/vfio/cdx/vfio_cdx_private.h @@ -0,0 +1,32 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2022, Advanced Micro Devices, Inc. + */ + +#ifndef VFIO_CDX_PRIVATE_H +#define VFIO_CDX_PRIVATE_H + +#define VFIO_CDX_OFFSET_SHIFT 40 +#define VFIO_CDX_OFFSET_MASK (((u64)(1) << VFIO_CDX_OFFSET_SHIFT) - 1) + +#define VFIO_CDX_OFFSET_TO_INDEX(off) ((off) >> VFIO_CDX_OFFSET_SHIFT) + +#define VFIO_CDX_INDEX_TO_OFFSET(index) \ + ((u64)(index) << VFIO_CDX_OFFSET_SHIFT) + +struct vfio_cdx_region { + u32 flags; + u32 type; + u64 addr; + resource_size_t size; + void __iomem *ioaddr; +}; + +struct vfio_cdx_device { + struct vfio_device vdev; + struct cdx_device *cdx_dev; + struct device *dev; + struct vfio_cdx_region *regions; +}; + +#endif /* VFIO_CDX_PRIVATE_H */ From patchwork Tue Sep 6 13:48:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nipun Gupta X-Patchwork-Id: 12967595 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1AF9C6FA83 for ; Tue, 6 Sep 2022 14:20:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241535AbiIFOUg (ORCPT ); Tue, 6 Sep 2022 10:20:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37574 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242065AbiIFOT3 (ORCPT ); Tue, 6 Sep 2022 10:19:29 -0400 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on20626.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe5a::626]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 489DD7C747; Tue, 6 Sep 2022 06:50:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DTiqYh0R9IQLyNNSTuMVrymfIIyWYTZWt+9q6+69OlUCcpfTO+IKfFysCZgrQFbTiQvEBzyttkvfn3BCYcHLcNl58Ldm8dX/8OQl5YBE2I1aAehg6dFWbEBNLztVxoNj6AaZ1df598XXQAhAYe/fNfKU3ouwu2FPOTEe6kY2otnQ98g2WdN16W7g39j11eWvsmT05gDAqiLMKBCAnuutaB5LmlabhDN4RmwgA/thFyNU/cqqL1cgWZsIQougCGMJVF3qbWLrlCGnrB7JEeUl5O5n9zUlSXGymXTeg0IH2PnfXH2YjYU5LWSCTCHf/owjWdHRTKiEXLRDU5hZqLv/4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Jxrq6UNgnw9n2PFbKZm7D+rKr3QmAMqj2BlP6aDUCpw=; b=QTfMfPL/Ap0A4KjGRhXj8Bs9VLZceO5bpgnlFMYXz/8HJNNPw+yIdm2T5xEXYsV/8Of/6hYWOg1uYK4iz8W3WTq00vPaidptwBLt74FR8D62OCpOg1xGIb98RZ2veiZbld70D3lLiK+w+SSR/wvl7kMW4jS6gZnY/+InB82zjmMjS4k1An6hdVXNQgs7/3ftHRr++OxRq3qew2FmCvQd5xeWe9b8GfQLSVsARampb6V7oIl2cfk/TRLMi8A/t8+KX59x1GWmwJEM3MOzxajxkDYdTQqoq9M+3alN7beHq+AJ17MQj9EYpwDIFZGbaCF6k6grapBt58poZjKy8GHJ8A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Jxrq6UNgnw9n2PFbKZm7D+rKr3QmAMqj2BlP6aDUCpw=; b=Ymg6nWRPA9xIWZpCwDD7oNuTez2e9YK84/olVSEJyQcwieQSZm+3QHheag1GXffLwuCGp92809ei7bCpQljyTlhTvZXuSxShoAAjcPAfomxxliI/bLgp9SLKWp3ggaJ9Pk3eYe5AL0Jdb7KF5iF4G2uCeUXvmWcOFqcwi9fPg/M= Received: from DS7PR07CA0021.namprd07.prod.outlook.com (2603:10b6:5:3af::23) by DM4PR12MB5988.namprd12.prod.outlook.com (2603:10b6:8:6b::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.17; Tue, 6 Sep 2022 13:49:24 +0000 Received: from DM6NAM11FT048.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3af:cafe::cc) by DS7PR07CA0021.outlook.office365.com (2603:10b6:5:3af::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.12 via Frontend Transport; Tue, 6 Sep 2022 13:49:24 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT048.mail.protection.outlook.com (10.13.173.114) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5588.10 via Frontend Transport; Tue, 6 Sep 2022 13:49:23 +0000 Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 6 Sep 2022 08:49:23 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Tue, 6 Sep 2022 06:49:22 -0700 Received: from xhdipdslab49.xilinx.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2375.28 via Frontend Transport; Tue, 6 Sep 2022 08:49:13 -0500 From: Nipun Gupta To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , CC: , , , , , , Nipun Gupta Subject: [RFC PATCH v3 7/7] vfio/cdx: add interrupt support Date: Tue, 6 Sep 2022 19:18:01 +0530 Message-ID: <20220906134801.4079497-8-nipun.gupta@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220906134801.4079497-1-nipun.gupta@amd.com> References: <20220803122655.100254-1-nipun.gupta@amd.com> <20220906134801.4079497-1-nipun.gupta@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 458526e5-3474-4c86-6482-08da900e9b95 X-MS-TrafficTypeDiagnostic: DM4PR12MB5988:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: F2mSicpu8HDq8g0JXDKxhSzoQTioj5+TO8p7RIfF18MjYsqFATw7U1PwCYS564tpQdS37NTeypjwbgJwGwM+RBT0DwbS6w3OOckBOjKAgpOOFETWkb6ke+h7oKyF42m6Nn9hvEugpL6vK8JzcfdOVIDUi6bRexaIcfRHEPLBnpDCn6L3yT6R7x1mthpyzPY+B0ieMJLm0v0D9G1vL2/2iLIr84s6MHVyS0JrFlAu0/vdyp8TlWdVQwcgqlZvPXiF+sKnXB4SikY2twCfie6MtyA2uERG/5PfllRBw8Xm2DxibqcJCcnHRxete/47HEP3yWd2afSqH/CcO1kXVgt2dSuHZyrEghMoAEMKn2KjlmMAJAVv4iIC4P/e5uihc1CC1AyeZT4vAfN/FFWmhXRDTmEOEjcg42RAIK1QS4xM7p06I8U7kdyon3nnyBaeUSuNTqbBQNykUEGJsHTTqHGNHvvDgmf3X+xzaSR3CGieMseKzwgXPFa83R0RNCfECWjFUn1VXMpY8/oN4iW+HwtIYQituXoaLWRtxK294IP+3qKXw6GYdy269FeczU5aMCcTK2SYbnqtHD8bvPNGcdk2xGGt55/YjBCAkUuQ8Fi0pw/qBEwQ0mKlJ4DoPZte0sF1KWl7xl7BOFxpbqrUEs4wVoD9iDHsY5XH5TeDxJF4PUrlXsV/jSbzZ9h18vxGCy/QODJNuT/kVn7p//qgR2id2TIHraTFQ0q8xKFGat1WcgYyBuH9DMbJlu5FrpwUPLMljl2TRUZdk1QR5oIcDJUj8Lz1397am8rbPM+LhUUfccXjuhipqss0WeR/mAsDEkiMX4etuAmBF2IjISYif+04SkLhMvoEvsrBWMTebFds9RMaUpQzG3s1P2DthGZiXR+h X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230016)(4636009)(39860400002)(376002)(136003)(346002)(396003)(46966006)(40470700004)(36840700001)(5660300002)(186003)(1076003)(336012)(47076005)(426003)(44832011)(36860700001)(4326008)(70586007)(8676002)(7416002)(8936002)(2906002)(83380400001)(70206006)(2616005)(110136005)(41300700001)(478600001)(316002)(81166007)(40480700001)(921005)(82310400005)(40460700003)(6666004)(82740400003)(36756003)(54906003)(86362001)(356005)(26005)(36900700001)(83996005)(2101003);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Sep 2022 13:49:23.9609 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 458526e5-3474-4c86-6482-08da900e9b95 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT048.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5988 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This patch allows to set an eventfd for cdx device interrupts and also to trigger the interrupt eventfd from userspace. All CDX device interrupts are MSIs. The MSIs are allocated from the CDX-MSI domain. Signed-off-by: Nipun Gupta --- drivers/vfio/cdx/vfio_cdx.c | 53 +++++++ drivers/vfio/cdx/vfio_cdx_intr.c | 212 ++++++++++++++++++++++++++++ drivers/vfio/cdx/vfio_cdx_private.h | 18 +++ 3 files changed, 283 insertions(+) create mode 100644 drivers/vfio/cdx/vfio_cdx_intr.c diff --git a/drivers/vfio/cdx/vfio_cdx.c b/drivers/vfio/cdx/vfio_cdx.c index 2e5bd494057a..4591b8057b2f 100644 --- a/drivers/vfio/cdx/vfio_cdx.c +++ b/drivers/vfio/cdx/vfio_cdx.c @@ -77,6 +77,8 @@ static void vfio_cdx_close_device(struct vfio_device *core_vdev) if (WARN_ON(ret)) dev_warn(core_vdev->dev, "VFIO_CDX: reset device has failed (%d)\n", ret); + + vfio_cdx_irqs_cleanup(vdev); } static long vfio_cdx_ioctl(struct vfio_device *core_vdev, @@ -132,6 +134,57 @@ static long vfio_cdx_ioctl(struct vfio_device *core_vdev, return -EFAULT; return 0; } + case VFIO_DEVICE_GET_IRQ_INFO: + { + struct vfio_irq_info info; + + minsz = offsetofend(struct vfio_irq_info, count); + if (copy_from_user(&info, (void __user *)arg, minsz)) + return -EFAULT; + + if (info.argsz < minsz) + return -EINVAL; + + if (info.index >= 1) + return -EINVAL; + + info.flags = VFIO_IRQ_INFO_EVENTFD; + info.count = cdx_dev->num_msi; + + if (copy_to_user((void __user *)arg, &info, minsz)) + return -EFAULT; + return 0; + } + case VFIO_DEVICE_SET_IRQS: + { + struct vfio_irq_set hdr; + u8 *data = NULL; + int ret = 0; + size_t data_size = 0; + + minsz = offsetofend(struct vfio_irq_set, count); + + if (copy_from_user(&hdr, (void __user *)arg, minsz)) + return -EFAULT; + + ret = vfio_set_irqs_validate_and_prepare(&hdr, + cdx_dev->num_msi, 1, &data_size); + if (ret) + return ret; + + if (data_size) { + data = memdup_user((void __user *)(arg + minsz), + data_size); + if (IS_ERR(data)) + return PTR_ERR(data); + } + + ret = vfio_cdx_set_irqs_ioctl(vdev, hdr.flags, + hdr.index, hdr.start, hdr.count, data); + kfree(data); + + return ret; + } case VFIO_DEVICE_RESET: { return vfio_cdx_reset_device(vdev); diff --git a/drivers/vfio/cdx/vfio_cdx_intr.c b/drivers/vfio/cdx/vfio_cdx_intr.c new file mode 100644 index 000000000000..20fe87bce464 --- /dev/null +++ b/drivers/vfio/cdx/vfio_cdx_intr.c @@ -0,0 +1,212 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2022, Advanced Micro Devices, Inc. + */ + +#include +#include +#include +#include +#include +#include + +#include "linux/cdx/cdx_bus.h" +#include "vfio_cdx_private.h" + +static irqreturn_t vfio_cdx_msihandler(int irq_no, void *arg) +{ + struct eventfd_ctx *trigger = arg; + + eventfd_signal(trigger, 1); + return IRQ_HANDLED; +} + +static int vfio_cdx_msi_enable(struct vfio_cdx_device *vdev, int nvec) +{ + struct device *dev = vdev->dev; + int msi_idx, ret; + + vdev->cdx_irqs = kcalloc(nvec, sizeof(struct vfio_cdx_irq), GFP_KERNEL); + if (!vdev->cdx_irqs) + return -ENOMEM; + + /* Allocate cdx MSIs */ + ret = cdx_msi_domain_alloc_irqs(dev, nvec); + if (ret < 0) { + kfree(vdev->cdx_irqs); + return ret; + } + + for (msi_idx = 0; msi_idx < nvec; msi_idx++) + vdev->cdx_irqs[msi_idx].irq_no = msi_get_virq(dev, msi_idx); + + vdev->irq_count = nvec; + vdev->config_msi = 1; + + return 0; +} + +static int vfio_cdx_msi_set_vector_signal(struct vfio_cdx_device *vdev, + int vector, int fd) +{ + struct eventfd_ctx *trigger; + int irq_no, ret; + + if (vector < 0 || vector >= vdev->irq_count) + return -EINVAL; + + irq_no = vdev->cdx_irqs[vector].irq_no; + + if (vdev->cdx_irqs[vector].trigger) { + free_irq(irq_no, vdev->cdx_irqs[vector].trigger); + kfree(vdev->cdx_irqs[vector].name); + eventfd_ctx_put(vdev->cdx_irqs[vector].trigger); + vdev->cdx_irqs[vector].trigger = NULL; + } + + if (fd < 0) + return 0; + + vdev->cdx_irqs[vector].name = kasprintf(GFP_KERNEL, "vfio-msi[%d](%s)", + vector, dev_name(vdev->dev)); + if (!vdev->cdx_irqs[vector].name) + return -ENOMEM; + + trigger = eventfd_ctx_fdget(fd); + if (IS_ERR(trigger)) { + kfree(vdev->cdx_irqs[vector].name); + return PTR_ERR(trigger); + } + + ret = request_irq(irq_no, vfio_cdx_msihandler, 0, + vdev->cdx_irqs[vector].name, trigger); + if (ret) { + kfree(vdev->cdx_irqs[vector].name); + eventfd_ctx_put(trigger); + return ret; + } + + vdev->cdx_irqs[vector].trigger = trigger; + + return 0; +} + +static int vfio_cdx_msi_set_block(struct vfio_cdx_device *vdev, + unsigned int start, unsigned int count, + int32_t *fds) +{ + int i, j, ret = 0; + + if (start >= vdev->irq_count || start + count > vdev->irq_count) + return -EINVAL; + + for (i = 0, j = start; i < count && !ret; i++, j++) { + int fd = fds ? fds[i] : -1; + + ret = vfio_cdx_msi_set_vector_signal(vdev, j, fd); + } + + if (ret) { + for (--j; j >= (int)start; j--) + vfio_cdx_msi_set_vector_signal(vdev, j, -1); + } + + return ret; +} + +static void vfio_cdx_msi_disable(struct vfio_cdx_device *vdev) +{ + struct device *dev = vdev->dev; + + vfio_cdx_msi_set_block(vdev, 0, vdev->irq_count, NULL); + + if (vdev->config_msi == 1) + cdx_msi_domain_free_irqs(dev); + + vdev->config_msi = 0; + vdev->irq_count = 0; + + kfree(vdev->cdx_irqs); +} + +static int vfio_cdx_set_msi_trigger(struct vfio_cdx_device *vdev, + unsigned int index, unsigned int start, + unsigned int count, uint32_t flags, + void *data) +{ + struct cdx_device *cdx_dev = vdev->cdx_dev; + int i; + + if (start + count > cdx_dev->num_msi) + return -EINVAL; + + if (!count && (flags & VFIO_IRQ_SET_DATA_NONE)) { + vfio_cdx_msi_disable(vdev); + return 0; + } + + if (flags & VFIO_IRQ_SET_DATA_EVENTFD) { + s32 *fds = data; + int ret; + + if (vdev->config_msi) + return vfio_cdx_msi_set_block(vdev, start, count, + fds); + ret = vfio_cdx_msi_enable(vdev, start + count); + if (ret) + return ret; + + ret = vfio_cdx_msi_set_block(vdev, start, count, fds); + if (ret) + vfio_cdx_msi_disable(vdev); + + return ret; + } + + for (i = start; i < start + count; i++) { + if (!vdev->cdx_irqs[i].trigger) + continue; + if (flags & VFIO_IRQ_SET_DATA_NONE) { + eventfd_signal(vdev->cdx_irqs[i].trigger, 1); + } else if (flags & VFIO_IRQ_SET_DATA_BOOL) { + u8 *bools = data; + + if (bools[i - start]) + eventfd_signal(vdev->cdx_irqs[i].trigger, 1); + } + } + + return 0; +} + +int vfio_cdx_set_irqs_ioctl(struct vfio_cdx_device *vdev, + u32 flags, unsigned int index, + unsigned int start, unsigned int count, + void *data) +{ + if (flags & VFIO_IRQ_SET_ACTION_TRIGGER) + return vfio_cdx_set_msi_trigger(vdev, index, start, + count, flags, data); + else + return -EINVAL; +} + +/* Free All IRQs for the given device */ +void vfio_cdx_irqs_cleanup(struct vfio_cdx_device *vdev) +{ + /* + * Device does not support any interrupt or the interrupts + * were not configured + */ + if (!vdev->cdx_irqs) + return; + + vfio_cdx_set_msi_trigger(vdev, 1, 0, 0, + VFIO_IRQ_SET_DATA_NONE, NULL); + + if (vdev->config_msi) { + cdx_msi_domain_free_irqs(vdev->dev); + kfree(vdev->cdx_irqs); + } + vdev->cdx_irqs = NULL; +} diff --git a/drivers/vfio/cdx/vfio_cdx_private.h b/drivers/vfio/cdx/vfio_cdx_private.h index d87b55663462..9f93fc8cabfd 100644 --- a/drivers/vfio/cdx/vfio_cdx_private.h +++ b/drivers/vfio/cdx/vfio_cdx_private.h @@ -14,6 +14,14 @@ #define VFIO_CDX_INDEX_TO_OFFSET(index) \ ((u64)(index) << VFIO_CDX_OFFSET_SHIFT) +struct vfio_cdx_irq { + u32 flags; + u32 count; + int irq_no; + struct eventfd_ctx *trigger; + char *name; +}; + struct vfio_cdx_region { u32 flags; u32 type; @@ -27,6 +35,16 @@ struct vfio_cdx_device { struct cdx_device *cdx_dev; struct device *dev; struct vfio_cdx_region *regions; + struct vfio_cdx_irq *cdx_irqs; + uint32_t irq_count; + uint32_t config_msi; }; +int vfio_cdx_set_irqs_ioctl(struct vfio_cdx_device *vdev, + u32 flags, unsigned int index, + unsigned int start, unsigned int count, + void *data); + +void vfio_cdx_irqs_cleanup(struct vfio_cdx_device *vdev); + #endif /* VFIO_CDX_PRIVATE_H */