From patchwork Mon Feb 24 09:18:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mykyta Poturai X-Patchwork-Id: 13987650 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D3945C021B3 for ; Mon, 24 Feb 2025 09:19:05 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.894978.1303598 (Exim 4.92) (envelope-from ) id 1tmUbf-0004I7-2a; Mon, 24 Feb 2025 09:18:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 894978.1303598; Mon, 24 Feb 2025 09:18:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tmUbe-0004Hw-W0; Mon, 24 Feb 2025 09:18:30 +0000 Received: by outflank-mailman (input) for mailman id 894978; Mon, 24 Feb 2025 09:18:29 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1tmUbd-0003pb-7K for xen-devel@lists.xenproject.org; Mon, 24 Feb 2025 09:18:29 +0000 Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on20603.outbound.protection.outlook.com [2a01:111:f403:260d::603]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 4ec72298-f290-11ef-9aae-95dc52dad729; Mon, 24 Feb 2025 10:18:28 +0100 (CET) Received: from PAVPR03MB10102.eurprd03.prod.outlook.com (2603:10a6:102:30d::12) by AS8PR03MB8859.eurprd03.prod.outlook.com (2603:10a6:20b:56f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.18; Mon, 24 Feb 2025 09:18:25 +0000 Received: from PAVPR03MB10102.eurprd03.prod.outlook.com ([fe80::35ac:8893:c31c:b971]) by PAVPR03MB10102.eurprd03.prod.outlook.com ([fe80::35ac:8893:c31c:b971%3]) with mapi id 15.20.8466.016; Mon, 24 Feb 2025 09:18:25 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 4ec72298-f290-11ef-9aae-95dc52dad729 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CRyHYSypBSLR9j1ELa59f0k5ebczB4IDkRPe4t26s4SdwLxwl8LWhIoeNxtWU9P3OTC8pAx3Mp0wfD8VnpNzv2GVpUhNizrhWQz2TpaHb5EwnPMqOA1jV1dAmYI37w8tg8oXqVFQ1mYZHaE0gbx8TZH6K6YLkbD8PSXKCw4Z72q43b4k9bZv4JfOha4vDu1YrtrDoGSP1MwV5Vjrlu58Cy40BwjKbLwSs10F2ZtUCLtJM0NpTbB6bBGE2HEfIb+VpNX3rCmTLUePbw3AZ57KB61OkXeeAH27fLUaCpshfmAjFZYLWrBimMbyCHJ50XGkyX3cnjOFzJcb2kHXWEBMCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fC2pKV6Knn2B6+xxOok9PcE5unrt7qlO9zl/Phe/vsk=; b=qHU+q2ZJwbwyiJ3NDMCiyBohHJG6ap8USGDuxrf2al2qs3g34xpf0GEaWKE7HMPCK1A/ToRM/aZzZq3wpBHHBp1cO9LJ7oLx5hteKeq+7k3HMWKBvH7acrPXXCzN8tB7eSl1+bwp8mTYfvOFOkDMsvEKQLWv64wdWMyITkgV1X91PBb/uZXomxanu2f8q2Tglti/w8BJacfiv0HQHcB424af1ULBpH9k8A7+IDVw6TVw+9AykkIzdd1zDhOimVhxOUhYrekxh4epPekgB6/Q7hqm7q8PxZeP2Ke3ynK9Df62jc+sl45bFm20unkk8Y3yq8HiOsMVtkMiXLFR9wMGUA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=epam.com; dmarc=pass action=none header.from=epam.com; dkim=pass header.d=epam.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=epam.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fC2pKV6Knn2B6+xxOok9PcE5unrt7qlO9zl/Phe/vsk=; b=iUEqzOOBFiGYsozQK1nW3oOGHPXjZzdwUabeds5Wwot4AhKHUyMk/ExuiyENXCsEf9M0EbFRty1t37hqjGFWKjAuml2Sh6Lt54Gq0dFvdBMz0nuKAIjl40ylSPtIxVYUvz+i3NrF23pMowXCSr56ZlAlw0Um+zIFx9xgbgiXGLw4rnxmVzrTF4jbUzblaP17X2Mfz5LIjXypp1UIaI7hh7pccfTv9lWCWe6qNN0menq9lijkJb+7w0ZyaO6/s7BCsm6hQocrqX0HkLIN84WoTMeBV+KSlZw0VMfMkRHskTH2m4QZrDYiVvFY2j+wSobBk8eVx4JOLwb1YrtznrO4bw== From: Mykyta Poturai To: "xen-devel@lists.xenproject.org" CC: Oleksandr Andrushchenko , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Mykyta Poturai Subject: [PATCH 3/7] xen/arm: add support for PCI child bus Thread-Topic: [PATCH 3/7] xen/arm: add support for PCI child bus Thread-Index: AQHbhp0OxLzto7VX00+joODTbNcGbw== Date: Mon, 24 Feb 2025 09:18:25 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=epam.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PAVPR03MB10102:EE_|AS8PR03MB8859:EE_ x-ms-office365-filtering-correlation-id: eb8219a2-3ecf-47a4-c2e0-08dd54b43112 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0;ARA:13230040|376014|366016|1800799024|38070700018; x-microsoft-antispam-message-info: =?iso-8859-1?q?Kfji/g30rB6MQ7+4rMf+a44BLb?= =?iso-8859-1?q?xM8hDq+3mINog32do68sT/ho4hwFmqrEnt1aWY0r0PqfV/gfN97dYexm28Dr?= =?iso-8859-1?q?Cs+ZpIgafVPZZhmwReONWL788eD8PNX9kziGNtp9/9+RvIhI9BaYctDIMRAp?= =?iso-8859-1?q?53HoN3Ug7sjkj/oj9BKHf5XGovUqlcVbyVK07H0Hc2IqZOL7lGyrEYcCKehx?= =?iso-8859-1?q?mS+hGYGOS4rjyPaRgDhVgwQ3MHY5p/KpiAVN7dP9QDmWso4dAWqzCjoN91C1?= =?iso-8859-1?q?JK4UKzJUMo9F0I3rQQJQAMKEnnw75lASFcDimOHYtwukwJLu76UWz4F3zu2X?= =?iso-8859-1?q?pyMCRXcyvRxsx1xYs9Oq1jCYmXQez5etv7ojzWN0DUDBXDE+7CoLtM9JQqs1?= =?iso-8859-1?q?7xQ6m+keO3UPvTsovkkXS+1YjM5RqGneiZshpqd0kdVdJ/DRDone1wIK/EWD?= =?iso-8859-1?q?+U8DjhUsqJGUTLQMCXPj5/g/24WqG2lyf6mfib/S2rOWgxn5fA6+f0k2o7d6?= =?iso-8859-1?q?nJkblqLKxx79YzSQyUhsPi4MfGBbSLIXq9dyOsLmqTmwSFYCFQ9be5fQMvWq?= =?iso-8859-1?q?cBZjwxgtEcFOkJXlT5yF1vnyuh1jCVSvsLa3nknA96hnLJnGtdtE6HoepZij?= =?iso-8859-1?q?eov7LGEVxIiFL+cW8rv91n38xwf1z7+8j6oDd/2DQi55w96SgZF2rPyVXd3R?= =?iso-8859-1?q?4TqdnQwbRXnBjT+PHbRlqCHfmikNPXIN3G+XJkwPOO7IbG/8ma68LL4X15BE?= =?iso-8859-1?q?sapHyrXOsB0f/TCHMqgfv6q6AMzhrYxigvj4/Ieqvk83e5IJEMmHMZQPeIfQ?= =?iso-8859-1?q?WPzUmb0ibDE8hJs9oCJzhSFBe8BjaJ1v3uTHWBMOWVOrskZYKG1351+jCqXr?= =?iso-8859-1?q?d3srMv4V6AMgys9qTwFepstnkT9n2Su7RgaZEHrqFWbZ58mzJwpO6Q3gEk3t?= =?iso-8859-1?q?2tph4Y46RQ/SKf/olzUpEPTbQZVl308U1uqpLcZd7my5upFVT/aVNx37KKp4?= =?iso-8859-1?q?6mWiOh2d5zyNA2zDZFKvCijdZlZiytDjLKxmoD8Ib2/xxhKcRsxVQX05x30J?= =?iso-8859-1?q?BT79Aa9sGxAMy478/Pk/7lJ0DNYQl1zyVronC+ywuDORP+QLoL3UXcNBO2mm?= =?iso-8859-1?q?iVmLxP7grFTrrYLKfyFxnBfA21s8JgB7bvdDOvQJsdAkx/ile16al0cR+I0/?= =?iso-8859-1?q?PZQBeM3eJDH1uZS7kWAG72lJuelrf4ZYnwm12rPG8Y01F8/gsMPJ3W37hpoa?= =?iso-8859-1?q?SzRkJtfrH3jYB6vVhp26BihyS/A9YtPtO83vqUP2VioiJ1vi1F5CMxKVUyd7?= =?iso-8859-1?q?9ucXtiplap7gpoi4egn6lzzH+QGM7GnSuoetOqU4OEpV35yWsUv3GMG5a6lD?= =?iso-8859-1?q?60ImWPWbkwKMkzPkMZLMvyWXts3wd30tqCFOliisGA0mzUNFMgnRw4cLDzgI?= =?iso-8859-1?q?CA6vq0Jh7LatMoUPFPI5Z/fg=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PAVPR03MB10102.eurprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024)(38070700018);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?q?Q4fGGAqvqEomoJ8O4Hvti5P?= =?iso-8859-1?q?lZP9HrR4++KZjCzve+29nRd/itM9UVef5BZbh6acr1md63K6ywUA3pxtbLiL?= =?iso-8859-1?q?yjQoUeVyl5CPRD2L//TNyaxtQJXsP+RXbyjcW+k7yk4WHkcVoMmcXCEb8Ocu?= =?iso-8859-1?q?2YQX/wINdeHfDt/C53l5/ftziaZD1UgvSnaKVfucKkFfxtodlgZA/0CogbCH?= =?iso-8859-1?q?vlhAP05c9R8wa433tkVLQstEBTdchpHkEJ56BBS/iC/EAHzqLpKsCaCTtQXH?= =?iso-8859-1?q?vabmlzMC/1kOAolNe+tLM2QCMUD921axQVX/GvMBvEjqS2pCiA/rDt9PSD9O?= =?iso-8859-1?q?anxSlADoZmudq4VnHv0RaBm5pMDZFrV23ZfPo4veompBenAd+WX1NAtgHU2+?= =?iso-8859-1?q?68sQkv45kgKO3EEHpfQJTUgjW6nPqdr2dJzKPV7zIlAk93baMRIP0wqniKCk?= =?iso-8859-1?q?jcMesuXQKobXB9/MjAyTBlDC6kseoXW5r6rTIS95V9ypyJYh8cfGzr8rMHM4?= =?iso-8859-1?q?uu4aNKMUp7k7xzskjP4/q12inyo3kg7St+vwN/DWT8j78tB6dUXIUciNdtBg?= =?iso-8859-1?q?ZDVkxJrSCWlgkOpjgxps9GYk23KRJ2+mmMmR49l5QtO5LlLBng5ToGjEytCr?= =?iso-8859-1?q?yz+jfHoeL2tHig7diY+06XKWNbU0a4OkW/wwFmIq6RI0ZivUV8zi1IImJzkM?= =?iso-8859-1?q?UM7Agz8dmv+k5HAeJfw93hdKCdmJ/dHbq0HeeFjXclj4jJcWEUnF7rzs9TlA?= =?iso-8859-1?q?QCbWBY77NeQMdj7dmTF9JvrjQGJUU930ghftrNwmDuzAg/9Bg0+b1EpPAns8?= =?iso-8859-1?q?yuh9y45GsI+FTvGprr5XIU/lpqhRQpikOx8i11DNchc8P5XJcrJiBB71vbea?= =?iso-8859-1?q?5RAzsXjow/bM9WfMKFGEvNkrRHeETb2XI17q68QTtWcKCOuzfs9n/SDRXFKs?= =?iso-8859-1?q?vgDJeJKFOgy6BIIOWqjAsb5WaQhq7/bx3C/b3ENxXhi0ukCjeIJ9LhDAreYV?= =?iso-8859-1?q?9bRMiEQfaPQEH6Sa9qNvbxf8XwCs9LGn8QrkdLZLOlrqwWsb2+Pnk4a/V3PV?= =?iso-8859-1?q?CSWlWB76qDuUide+zR0QXn27vh1cYQg5oEQsIQV/d/xfWx3bTnfaFYkgtZet?= =?iso-8859-1?q?qgTW3ByfymeBjNS2Zi5JR8yK4WZcZGtNedPww5UV6E8G01BdwdQVyS/tlbUc?= =?iso-8859-1?q?ifKHOjFdZm6RmgLdGUpNDIHmGUjnohV1wJe2CtOi/f1vPEeZ65FnVkT7foOD?= =?iso-8859-1?q?F1to1ggzCr4E26heX48X1V8YuUavLRA2R6A/MWSoWtSoVnND3AhHF06ifUZw?= =?iso-8859-1?q?G2qaaFlrKE2Y82aOEJlsaJQeLkFRBy/W+fhoEP1Nyxtrzq1jSMvmEVI4J9Rd?= =?iso-8859-1?q?U54kwGUZXeHGO3ZfzZyQM3vwAAtyo42CBn5KRKcKU3eDWQv03r+S79HOVV3U?= =?iso-8859-1?q?Cz3UseTW+3RQOG+rjMukX1yuZmEmGZTxrWsocuzyUn7uyvU5xm4wValcu/mT?= =?iso-8859-1?q?kJuuw61Jo5dyzbClWG7pck2LXeI6rdsX+uxrVYZTQQMruuARF/A2WwVRnNGG?= =?iso-8859-1?q?NBfnCkyHqG2rQXRj/frN66T7U4FlwRaOEyMRsDaOKrp8VkE0w/J+Q3j1GGvR?= =?iso-8859-1?q?/SduzelBUkXpQEaJMSwKv/PEtdrCbJuEcfhaIRA=3D=3D?= MIME-Version: 1.0 X-OriginatorOrg: epam.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PAVPR03MB10102.eurprd03.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: eb8219a2-3ecf-47a4-c2e0-08dd54b43112 X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Feb 2025 09:18:25.0273 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: e2VJMz00H2MbPRQ53I4TNRoOnRL9ITS6rHGQ13t3dmwLmjXS+UFrPAi524qspd8aprBcthaVPit/5u3PVp6RCg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR03MB8859 From: Oleksandr Andrushchenko PCI host bridges often have different ways to access the root and child bus configuration spaces. One of the examples is Designware's host bridge and its multiple clones [1]. Linux kernel implements this by instantiating a child bus when device drivers provide not only the usual pci_ops to access ECAM space (this is the case for the generic host bridge), but also means to access the child bus which has a dedicated configuration space and own implementation for accessing the bus, e.g. child_ops. For Xen it is not feasible to fully implement PCI bus infrastructure as Linux kernel does, but still child bus can be supported. Add support for the PCI child bus which includes the following changes: - introduce bus mapping functions depending on SBDF - assign bus start and end for the child bus and re-configure the same for the parent (root) bus - make pci_find_host_bridge be aware of multiple busses behind the same bridge - update pci_host_bridge_mappings, so it also doesn't map to guest the memory spaces belonging to the child bus - make pci_host_common_probe accept one more pci_ops structure for the child bus - install MMIO handlers for the child bus - re-work vpci_mmio_{write|read} with parent and child approach in mind [1] https://elixir.bootlin.com/linux/v5.15/source/drivers/pci/controller/dwc Signed-off-by: Oleksandr Andrushchenko Signed-off-by: Mykyta Poturai --- xen/arch/arm/include/asm/pci.h | 14 ++++- xen/arch/arm/pci/ecam.c | 17 ++++-- xen/arch/arm/pci/pci-access.c | 37 ++++++++++-- xen/arch/arm/pci/pci-host-common.c | 86 +++++++++++++++++++++------ xen/arch/arm/pci/pci-host-generic.c | 2 +- xen/arch/arm/pci/pci-host-zynqmp.c | 2 +- xen/arch/arm/vpci.c | 91 ++++++++++++++++++++++++----- 7 files changed, 200 insertions(+), 49 deletions(-) diff --git a/xen/arch/arm/include/asm/pci.h b/xen/arch/arm/include/asm/pci.h index e1f63d75e3..0012c2ae9e 100644 --- a/xen/arch/arm/include/asm/pci.h +++ b/xen/arch/arm/include/asm/pci.h @@ -69,6 +69,9 @@ struct pci_host_bridge { struct pci_config_window* cfg; /* Pointer to the bridge config window */ const struct pci_ops *ops; void *priv; /* Private data of the bridge. */ + /* Child bus */ + struct pci_config_window* child_cfg; + const struct pci_ops *child_ops; }; struct pci_ops { @@ -97,15 +100,20 @@ struct pci_ecam_ops { /* Default ECAM ops */ extern const struct pci_ecam_ops pci_generic_ecam_ops; -struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev, - const struct pci_ecam_ops *ops, - size_t priv_sz); +struct pci_host_bridge * +pci_host_common_probe(struct dt_device_node *dev, + const struct pci_ecam_ops *ops, + const struct pci_ecam_ops *child_ops, + size_t priv_sz); int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf, uint32_t reg, uint32_t len, uint32_t *value); int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf, uint32_t reg, uint32_t len, uint32_t value); void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge, pci_sbdf_t sbdf, uint32_t where); +void __iomem *pci_ecam_map_bus_generic(const struct pci_config_window *cfg, + const struct pci_ecam_ops *ops, + pci_sbdf_t sbdf, uint32_t where); bool pci_ecam_need_p2m_hwdom_mapping(struct domain *d, struct pci_host_bridge *bridge, uint64_t addr); diff --git a/xen/arch/arm/pci/ecam.c b/xen/arch/arm/pci/ecam.c index 3987f96b01..5486881c5c 100644 --- a/xen/arch/arm/pci/ecam.c +++ b/xen/arch/arm/pci/ecam.c @@ -20,12 +20,10 @@ /* * Function to implement the pci_ops->map_bus method. */ -void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge, - pci_sbdf_t sbdf, uint32_t where) +void __iomem *pci_ecam_map_bus_generic(const struct pci_config_window *cfg, + const struct pci_ecam_ops *ops, + pci_sbdf_t sbdf, uint32_t where) { - const struct pci_config_window *cfg = bridge->cfg; - const struct pci_ecam_ops *ops = - container_of(bridge->ops, const struct pci_ecam_ops, pci_ops); unsigned int devfn_shift = ops->bus_shift - 8; void __iomem *base; unsigned int busn = sbdf.bus; @@ -39,6 +37,15 @@ void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge, return base + (sbdf.devfn << devfn_shift) + where; } +void __iomem *pci_ecam_map_bus(struct pci_host_bridge *bridge, + pci_sbdf_t sbdf, uint32_t where) +{ + const struct pci_ecam_ops *ops = + container_of(bridge->ops, const struct pci_ecam_ops, pci_ops); + + return pci_ecam_map_bus_generic(bridge->cfg, ops, sbdf, where); +} + bool __init pci_ecam_need_p2m_hwdom_mapping(struct domain *d, struct pci_host_bridge *bridge, uint64_t addr) diff --git a/xen/arch/arm/pci/pci-access.c b/xen/arch/arm/pci/pci-access.c index 9f9aac43d7..4da607ac3f 100644 --- a/xen/arch/arm/pci/pci-access.c +++ b/xen/arch/arm/pci/pci-access.c @@ -18,10 +18,31 @@ #define INVALID_VALUE (~0U) #define PCI_ERR_VALUE(len) GENMASK(0, len * 8) +static const struct pci_ops *get_ops(struct pci_host_bridge *bridge, + pci_sbdf_t sbdf) +{ + if ( bridge->child_ops ) + { + struct pci_config_window* cfg = bridge->child_cfg; + + if ( (sbdf.bus >= cfg->busn_start) && (sbdf.bus <= cfg->busn_end) ) + return bridge->child_ops; + } + return bridge->ops; +} + +static void __iomem *map_bus(struct pci_host_bridge *bridge, pci_sbdf_t sbdf, + uint32_t reg) +{ + const struct pci_ops *ops = get_ops(bridge, sbdf); + + return ops->map_bus(bridge, sbdf, reg); +} + int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf, uint32_t reg, uint32_t len, uint32_t *value) { - void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg); + void __iomem *addr = map_bus(bridge, sbdf, reg); if ( !addr ) { @@ -50,7 +71,7 @@ int pci_generic_config_read(struct pci_host_bridge *bridge, pci_sbdf_t sbdf, int pci_generic_config_write(struct pci_host_bridge *bridge, pci_sbdf_t sbdf, uint32_t reg, uint32_t len, uint32_t value) { - void __iomem *addr = bridge->ops->map_bus(bridge, sbdf, reg); + void __iomem *addr = map_bus(bridge, sbdf, reg); if ( !addr ) return -ENODEV; @@ -78,14 +99,16 @@ static uint32_t pci_config_read(pci_sbdf_t sbdf, unsigned int reg, { uint32_t val = PCI_ERR_VALUE(len); struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus); + const struct pci_ops *ops; if ( unlikely(!bridge) ) return val; - if ( unlikely(!bridge->ops->read) ) + ops = get_ops(bridge, sbdf); + if ( unlikely(!ops->read) ) return val; - bridge->ops->read(bridge, sbdf, reg, len, &val); + ops->read(bridge, sbdf, reg, len, &val); return val; } @@ -94,14 +117,16 @@ static void pci_config_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int len, uint32_t val) { struct pci_host_bridge *bridge = pci_find_host_bridge(sbdf.seg, sbdf.bus); + const struct pci_ops *ops; if ( unlikely(!bridge) ) return; - if ( unlikely(!bridge->ops->write) ) + ops = get_ops(bridge, sbdf); + if ( unlikely(!ops->write) ) return; - bridge->ops->write(bridge, sbdf, reg, len, val); + ops->write(bridge, sbdf, reg, len, val); } /* diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c index e4e05d1176..3824146561 100644 --- a/xen/arch/arm/pci/pci-host-common.c +++ b/xen/arch/arm/pci/pci-host-common.c @@ -57,17 +57,12 @@ static void pci_ecam_free(struct pci_config_window *cfg) xfree(cfg); } -static struct pci_config_window * __init -gen_pci_init(struct dt_device_node *dev, const struct pci_ecam_ops *ops) +static void __init gen_pci_init_bus_range(struct dt_device_node *dev, + struct pci_host_bridge *bridge, + struct pci_config_window *cfg) { - int err, cfg_reg_idx; u32 bus_range[2]; - paddr_t addr, size; - struct pci_config_window *cfg; - - cfg = xzalloc(struct pci_config_window); - if ( !cfg ) - return NULL; + int err; err = dt_property_read_u32_array(dev, "bus-range", bus_range, ARRAY_SIZE(bus_range)); @@ -82,6 +77,36 @@ gen_pci_init(struct dt_device_node *dev, const struct pci_ecam_ops *ops) if ( cfg->busn_end > cfg->busn_start + 0xff ) cfg->busn_end = cfg->busn_start + 0xff; } +} + +static void __init gen_pci_init_bus_range_child(struct dt_device_node *dev, + struct pci_host_bridge *bridge, + struct pci_config_window *cfg) +{ + cfg->busn_start = bridge->cfg->busn_start + 1; + cfg->busn_end = bridge->cfg->busn_end; + bridge->cfg->busn_end = bridge->cfg->busn_start; + + printk(XENLOG_INFO "Root bus end updated: [bus %x-%x]\n", + bridge->cfg->busn_start, bridge->cfg->busn_end); +} + +static struct pci_config_window * __init +gen_pci_init(struct dt_device_node *dev, struct pci_host_bridge *bridge, + const struct pci_ecam_ops *ops, + void (*init_bus_range)(struct dt_device_node *dev, + struct pci_host_bridge *bridge, + struct pci_config_window *cfg)) +{ + int err, cfg_reg_idx; + paddr_t addr, size; + struct pci_config_window *cfg; + + cfg = xzalloc(struct pci_config_window); + if ( !cfg ) + return NULL; + + init_bus_range(dev, bridge, cfg); if ( ops->cfg_reg_index ) { @@ -208,9 +233,11 @@ static int pci_bus_find_domain_nr(struct dt_device_node *dev) return domain; } -struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev, - const struct pci_ecam_ops *ops, - size_t priv_sz) +struct pci_host_bridge * +pci_host_common_probe(struct dt_device_node *dev, + const struct pci_ecam_ops *ops, + const struct pci_ecam_ops *child_ops, + size_t priv_sz) { struct pci_host_bridge *bridge; struct pci_config_window *cfg; @@ -225,7 +252,7 @@ struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev, return ERR_PTR(-ENOMEM); /* Parse and map our Configuration Space windows */ - cfg = gen_pci_init(dev, ops); + cfg = gen_pci_init(dev, bridge, ops, gen_pci_init_bus_range); if ( !cfg ) { err = -ENOMEM; @@ -245,6 +272,21 @@ struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev, bridge->segment = domain; + if ( child_ops ) + { + /* Parse and map child's Configuration Space windows */ + cfg = gen_pci_init(dev, bridge, child_ops, + gen_pci_init_bus_range_child); + if ( !cfg ) + { + err = -ENOMEM; + goto err_child; + } + + bridge->child_cfg = cfg; + bridge->child_ops = &child_ops->pci_ops; + } + if ( priv_sz ) { bridge->priv = xzalloc_bytes(priv_sz); @@ -262,6 +304,9 @@ struct pci_host_bridge *pci_host_common_probe(struct dt_device_node *dev, err_priv: xfree(bridge->priv); +err_child: + xfree(bridge->cfg); + err_exit: xfree(bridge); @@ -296,9 +341,11 @@ struct pci_host_bridge *pci_find_host_bridge(uint16_t segment, uint8_t bus) { if ( bridge->segment != segment ) continue; - if ( (bus < bridge->cfg->busn_start) || (bus > bridge->cfg->busn_end) ) - continue; - return bridge; + if ( bridge->child_cfg && (bus >= bridge->child_cfg->busn_start) && + (bus <= bridge->child_cfg->busn_end) ) + return bridge; + if ( (bus >= bridge->cfg->busn_start) && (bus <= bridge->cfg->busn_end) ) + return bridge; } return NULL; @@ -364,6 +411,7 @@ int __init pci_host_bridge_mappings(struct domain *d) { const struct dt_device_node *dev = bridge->dt_node; unsigned int i; + bool need_mapping; for ( i = 0; i < dt_number_of_address(dev); i++ ) { @@ -379,7 +427,11 @@ int __init pci_host_bridge_mappings(struct domain *d) return err; } - if ( bridge->ops->need_p2m_hwdom_mapping(d, bridge, addr) ) + need_mapping = bridge->ops->need_p2m_hwdom_mapping(d, bridge, addr); + if ( need_mapping && bridge->child_ops ) + need_mapping = bridge->child_ops-> + need_p2m_hwdom_mapping(d, bridge, addr); + if ( need_mapping ) { err = map_range_to_domain(dev, addr, size, &mr_data); if ( err ) diff --git a/xen/arch/arm/pci/pci-host-generic.c b/xen/arch/arm/pci/pci-host-generic.c index dde6a79a8e..08d94879b7 100644 --- a/xen/arch/arm/pci/pci-host-generic.c +++ b/xen/arch/arm/pci/pci-host-generic.c @@ -29,7 +29,7 @@ static const struct dt_device_match __initconstrel gen_pci_dt_match[] = static int __init pci_host_generic_probe(struct dt_device_node *dev, const void *data) { - return PTR_RET(pci_host_common_probe(dev, &pci_generic_ecam_ops, 0)); + return PTR_RET(pci_host_common_probe(dev, &pci_generic_ecam_ops, NULL, 0)); } DT_DEVICE_START(pci_gen, "PCI HOST GENERIC", DEVICE_PCI_HOSTBRIDGE) diff --git a/xen/arch/arm/pci/pci-host-zynqmp.c b/xen/arch/arm/pci/pci-host-zynqmp.c index c796447ac4..0b51ff6bd9 100644 --- a/xen/arch/arm/pci/pci-host-zynqmp.c +++ b/xen/arch/arm/pci/pci-host-zynqmp.c @@ -47,7 +47,7 @@ static const struct dt_device_match __initconstrel nwl_pcie_dt_match[] = static int __init pci_host_generic_probe(struct dt_device_node *dev, const void *data) { - return PTR_RET(pci_host_common_probe(dev, &nwl_pcie_ops, 0)); + return PTR_RET(pci_host_common_probe(dev, &nwl_pcie_ops, NULL, 0)); } DT_DEVICE_START(pci_gen, "PCI HOST ZYNQMP", DEVICE_PCI_HOSTBRIDGE) diff --git a/xen/arch/arm/vpci.c b/xen/arch/arm/vpci.c index b63a356bb4..05af4ef390 100644 --- a/xen/arch/arm/vpci.c +++ b/xen/arch/arm/vpci.c @@ -8,15 +8,17 @@ #include static pci_sbdf_t vpci_sbdf_from_gpa(const struct pci_host_bridge *bridge, - paddr_t gpa) + paddr_t gpa, bool use_root) { pci_sbdf_t sbdf; if ( bridge ) { - sbdf.sbdf = VPCI_ECAM_BDF(gpa - bridge->cfg->phys_addr); + const struct pci_config_window *cfg = use_root ? bridge->cfg : + bridge->child_cfg; + sbdf.sbdf = VPCI_ECAM_BDF(gpa - cfg->phys_addr); sbdf.seg = bridge->segment; - sbdf.bus += bridge->cfg->busn_start; + sbdf.bus += cfg->busn_start; } else sbdf.sbdf = VPCI_ECAM_BDF(gpa - GUEST_VPCI_ECAM_BASE); @@ -24,11 +26,9 @@ static pci_sbdf_t vpci_sbdf_from_gpa(const struct pci_host_bridge *bridge, return sbdf; } -static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info, - register_t *r, void *p) +static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info, register_t *r, + pci_sbdf_t sbdf) { - struct pci_host_bridge *bridge = p; - pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa); const unsigned int access_size = (1U << info->dabt.size) * 8; const register_t invalid = GENMASK_ULL(access_size - 1, 0); /* data is needed to prevent a pointer cast on 32bit */ @@ -46,31 +46,86 @@ static int vpci_mmio_read(struct vcpu *v, mmio_info_t *info, return 0; } -static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info, - register_t r, void *p) +static int vpci_mmio_read_root(struct vcpu *v, mmio_info_t *info, + register_t *r, void *p) +{ + struct pci_host_bridge *bridge = p; + pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa, true); + + return vpci_mmio_read(v, info, r, sbdf); +} + +static int vpci_mmio_read_child(struct vcpu *v, mmio_info_t *info, + register_t *r, void *p) { struct pci_host_bridge *bridge = p; - pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa); + pci_sbdf_t sbdf = vpci_sbdf_from_gpa(bridge, info->gpa, false); + return vpci_mmio_read(v, info, r, sbdf); +} + +static int vpci_mmio_write(struct vcpu *v, mmio_info_t *info, + register_t r, pci_sbdf_t sbdf) +{ return vpci_ecam_write(sbdf, ECAM_REG_OFFSET(info->gpa), 1U << info->dabt.size, r); } +static int vpci_mmio_write_root(struct vcpu *v, mmio_info_t *info, + register_t r, void *p) +{ + struct pci_host_bridge *bridge = p; + pci_sbdf_t sbdf; + + if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, + true, &sbdf) ) + return 0; + + return vpci_mmio_write(v, info, r, sbdf); +} + +static int vpci_mmio_write_child(struct vcpu *v, mmio_info_t *info, + register_t r, void *p) +{ + struct pci_host_bridge *bridge = p; + pci_sbdf_t sbdf; + + if ( !vpci_sbdf_from_gpa(v->domain, bridge, info->gpa, + false, &sbdf) ) + return 0; + + return vpci_mmio_write(v, info, r, sbdf); +} + static const struct mmio_handler_ops vpci_mmio_handler = { - .read = vpci_mmio_read, - .write = vpci_mmio_write, + .read = vpci_mmio_read_root, + .write = vpci_mmio_write_root, +}; + +static const struct mmio_handler_ops vpci_mmio_handler_child = { + .read = vpci_mmio_read_child, + .write = vpci_mmio_write_child, }; static int vpci_setup_mmio_handler_cb(struct domain *d, struct pci_host_bridge *bridge) { struct pci_config_window *cfg = bridge->cfg; + int count = 1; register_mmio_handler(d, &vpci_mmio_handler, cfg->phys_addr, cfg->size, bridge); - /* We have registered a single MMIO handler. */ - return 1; + if ( bridge->child_ops ) + { + struct pci_config_window *cfg = bridge->child_cfg; + + register_mmio_handler(d, &vpci_mmio_handler_child, + cfg->phys_addr, cfg->size, bridge); + count++; + } + + return count; } int domain_vpci_init(struct domain *d) @@ -101,8 +156,12 @@ int domain_vpci_init(struct domain *d) static int vpci_get_num_handlers_cb(struct domain *d, struct pci_host_bridge *bridge) { - /* Each bridge has a single MMIO handler for the configuration space. */ - return 1; + int count = 1; + + if ( bridge->child_cfg ) + count++; + + return count; } unsigned int domain_vpci_get_num_mmio_handlers(struct domain *d)