From patchwork Mon Dec 4 21:09:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13479111 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0360CC10DC3 for ; Mon, 4 Dec 2023 21:10:00 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.647310.1010310 (Exim 4.92) (envelope-from ) id 1rAGCG-0005od-D8; Mon, 04 Dec 2023 21:09:44 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 647310.1010310; Mon, 04 Dec 2023 21:09:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rAGCG-0005oW-A4; Mon, 04 Dec 2023 21:09:44 +0000 Received: by outflank-mailman (input) for mailman id 647310; Mon, 04 Dec 2023 21:09:43 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rAGCF-0005XB-7r for xen-devel@lists.xenproject.org; Mon, 04 Dec 2023 21:09:43 +0000 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on20605.outbound.protection.outlook.com [2a01:111:f400:7eab::605]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 6fa195a7-92e9-11ee-9b0f-b553b5be7939; Mon, 04 Dec 2023 22:09:40 +0100 (CET) Received: from MW4PR04CA0330.namprd04.prod.outlook.com (2603:10b6:303:82::35) by SJ2PR12MB8848.namprd12.prod.outlook.com (2603:10b6:a03:537::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.33; Mon, 4 Dec 2023 21:09:34 +0000 Received: from CO1PEPF000044F4.namprd05.prod.outlook.com (2603:10b6:303:82:cafe::a7) by MW4PR04CA0330.outlook.office365.com (2603:10b6:303:82::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.33 via Frontend Transport; Mon, 4 Dec 2023 21:09:34 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000044F4.mail.protection.outlook.com (10.167.241.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7068.20 via Frontend Transport; Mon, 4 Dec 2023 21:09:33 +0000 Received: from SATLEXMB08.amd.com (10.181.40.132) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 4 Dec 2023 15:09:32 -0600 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB08.amd.com (10.181.40.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.32; Mon, 4 Dec 2023 13:09:32 -0800 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend Transport; Mon, 4 Dec 2023 15:09:31 -0600 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 6fa195a7-92e9-11ee-9b0f-b553b5be7939 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kj2Y1ESJVqX/l7ZysLMAcHn2IH/EqtzxL8Miq3DKJ/i1xNLkx/BaLcYsP5y6JPOcplXAR5kXrC3WOaCpWYhZTV5kHFWcz63znLsRmFNmLdxgIyVkcUQazU9FN8hoQpwHlpdZb/5CyTKtZCubPoPU4EiawGPxkwRuBljfaSMh3+krftI8iAMh47CmgBmrFkXsHRVp5ffPM2b3K6A5HMpRczIo93ZvZWnFWr0/oKni7o3iacrARNJcLeF9D0nxxuDA9Mb1avL+s3Jd7HQBtxaW3LwBupUw8DjLZDDM0vlOJ+ImHHLycgggDL3NoJxEbURIKL1YThZmn1erXe8B3gyUnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Y+quwPLtW2g+qaggyRdl6Ui8Hi5RH/K25fGJJ240XZ0=; b=OBTCU1ViBAvE5r8yfWDNkQv9SmoqT/o3oVzd9vZJyD5+EMZDPiyRBvM9E97eEEBQTPsfS49Hspz+MDIYE5jzx+p4OdcIQk80uVl6RleG1wKCZQJyB/YFlrFuXF745VM6fD264IfEWl4R7uvZWHs6o29pGu/iEK6h5ASJbBCxlRypa/AGcEc968eEUhu8CyQSgJZIPw3fMYK6KROiU7Yujuq7heQNYetMV+fXYHi/TAqU/fGQVubK1Rl5xrOFEosWJqH0wOyLaAxNmd7/vOmk5R9m0oMCL4Jl56oLC72kbq5UPat3ucYsnJOLc9uKN1T2mRQE+Jp3vaCqaFkeynST1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y+quwPLtW2g+qaggyRdl6Ui8Hi5RH/K25fGJJ240XZ0=; b=AFiCAd0hbSlBoV78LjC3Y5WzBAwFeDFgxa6ZbxFYaGMb888D37+UWQFf3pBhQJvIA6qd9Dr6YEdzWw7P5cq6Mp7/PSKfG5gaeFfj62U2EdKRwbdkiRAOfS/bgzyypRJrZ5aIPEcfYNgfkr9EjggQ0ijTkWGy5SSmoC6Yc3+HVtU= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Stewart Hildebrand , =?utf-8?q?Roger_Pau_Mon?= =?utf-8?q?n=C3=A9?= , Wei Liu , Anthony PERARD , Andrew Cooper , George Dunlap , "Jan Beulich" , Julien Grall , "Stefano Stabellini" Subject: [PATCH v10 1/2] xen/vpci: header: status register handler Date: Mon, 4 Dec 2023 16:09:12 -0500 Message-ID: <20231204210916.3651999-2-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20231204210916.3651999-1-stewart.hildebrand@amd.com> References: <20231204210916.3651999-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044F4:EE_|SJ2PR12MB8848:EE_ X-MS-Office365-Filtering-Correlation-Id: 7934089c-0147-454b-59a1-08dbf50d5081 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wCAynt1HobAl+DYfbdl2gH1IjTVSlQdw2egznzNpuJOyjMbWawsZwAcqCA6nahyaZU9utrp0K1+ihnzpGZsKME7z4JoGKkGUrKrTHyXQ9h/qK53k9yGnKx4e3Y8opIGT3Lgklk6iRSDLDM+WYw6SfZRzU7GAILRwFvmSWze0Ka8+FotqCv93S3NWsS6+x0DxKZm5xfno1bT0R3BdYv60pqfPxYmyTVicEKXib+oJ6oKo6d2frdcslkIGV1il5bjpWyM4F5Fbl+7M6o/YIjN5PfjBiBfFzC+IL1poy50C9BPoZ+61BBuX9PGUxBVcw8q4cI2dj9QmxUN3sr0eSJkf+LFlCu6kL9a+KPOe+BkxsaH5tX5aSXO7lUYg/WT70ztI/h28amXlF4Temghl20oGoKLj3fDQeCi/A/RW4x4ZZYRr5Fy2t8v7WLeNYaoCL1a33szK8X+Um5yhiaw6LYz/4VQUTpFuiL52D2Pf3fXlgA1iXWCB+YFcye08ODhgGzDR0xDEgdnRu77hVX9yJvDH/f/hmLfAuxior9uO9wKCDxSNI+rlTGhVEMmVaMAaGAx7QfAMTqPs75wptJaQEQPxdu3zbPgM//61VZLp8U8MY3geYHz8nC8NPP2WMm4aZhmDTHBwG2NhSzQAN1zNH1rrzW41cQu7Y6IwjEOTpvu8kQ6EdPKWEd79TgaxAZqNGtsm/Nd66zdPnUTzBqNhAYjYDi47Dw/f61hI8JyXrCFe5l2UQaTeSobG0gQZk/F8b4kRWsFFeq1ZWBYeLP8LLyR/9A== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(136003)(396003)(376002)(346002)(39860400002)(230922051799003)(64100799003)(451199024)(82310400011)(1800799012)(186009)(40470700004)(36840700001)(46966006)(40460700003)(66899024)(6916009)(54906003)(316002)(44832011)(8936002)(86362001)(8676002)(4326008)(70206006)(478600001)(70586007)(966005)(19627235002)(41300700001)(36756003)(2906002)(30864003)(5660300002)(36860700001)(356005)(81166007)(47076005)(26005)(2616005)(426003)(1076003)(82740400003)(83380400001)(336012)(6666004)(40480700001)(36900700001)(357404004);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Dec 2023 21:09:33.5529 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7934089c-0147-454b-59a1-08dbf50d5081 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044F4.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8848 Introduce a handler for the PCI status register, with ability to mask the capabilities bit. The status register contains RsvdZ bits, read-only bits, and write-1-to-clear bits. Additionally, we use RsvdP to mask the capabilities bit. Introduce bitmasks to handle these in vPCI. If a bit in the bitmask is set, then the special meaning applies: ro_mask: read normal, guest write ignore (preserve on write to hardware) rw1c_mask: read normal, write 1 to clear rsvdp_mask: read as zero, guest write ignore (preserve on write to hardware) rsvdz_mask: read as zero, guest write ignore (write zero to hardware) The RO/RW1C/RsvdP/RsvdZ naming and definitions were borrowed from the PCI Express Base 6.1 specification. RsvdP/RsvdZ bits help Xen enforce our view of the world. Xen preserves the value of read-only bits on write to hardware, discarding the guests write value. This is done in case hardware wrongly implements R/O bits as R/W. The mask_cap_list flag will be set in a follow-on change. Signed-off-by: Stewart Hildebrand Reviewed-by: Roger Pau Monné --- v9->v10: * change a word in the commit message * add Roger's R-b tag v8->v9: * check that masks don't have bits set above register size * rename variable in vpci_write_helper() * only export one vpci_add_register* function, make the other one static inline * style fixups v7->v8: * move PCI_STATUS_UDF to rsvdz_mask (per PCI Express Base 6 spec) * add support for rsvdp bits * add tests for ro/rw1c/rsvdp/rsvdz bits in tools/tests/vpci/main.c * dropped R-b tag [1] since the patch has changed moderately since the last rev [1] https://lists.xenproject.org/archives/html/xen-devel/2023-09/msg00909.html v6->v7: * re-work args passed to vpci_add_register_mask() (called in init_bars()) * also check for overlap of (rsvdz_mask & ro_mask) in add_register() * slightly adjust masking operation in vpci_write_helper() v5->v6: * remove duplicate PCI_STATUS_CAP_LIST in constant definition * style fixup in constant definitions * s/res_mask/rsvdz_mask/ * combine a new masking operation into single line * preserve r/o bits on write * get rid of status_read. Instead, use rsvdz_mask for conditionally masking PCI_STATUS_CAP_LIST bit * add comment about PCI_STATUS_CAP_LIST and rsvdp behavior * add sanity checks in add_register * move mask_cap_list from struct vpci_header to local variable v4->v5: * add support for res_mask * add support for ro_mask (squash ro_mask patch) * add constants for reserved, read-only, and rw1c masks v3->v4: * move mask_cap_list setting to the capabilities patch * single pci_conf_read16 in status_read * align mask_cap_list bitfield in struct vpci_header * change to rw1c bit mask instead of treating whole register as rw1c * drop subsystem prefix on renamed add_register function v2->v3: * new patch --- tools/tests/vpci/main.c | 111 +++++++++++++++++++++++++++++++++++++ xen/drivers/vpci/header.c | 12 ++++ xen/drivers/vpci/vpci.c | 52 ++++++++++++----- xen/include/xen/pci_regs.h | 9 +++ xen/include/xen/vpci.h | 24 ++++++-- 5 files changed, 189 insertions(+), 19 deletions(-) diff --git a/tools/tests/vpci/main.c b/tools/tests/vpci/main.c index b9a0a6006bb9..64d4552936c7 100644 --- a/tools/tests/vpci/main.c +++ b/tools/tests/vpci/main.c @@ -70,6 +70,28 @@ static void vpci_write32(const struct pci_dev *pdev, unsigned int reg, *(uint32_t *)data = val; } +struct mask_data { + uint32_t val; + uint32_t rw1c_mask; +}; + +static uint32_t vpci_read32_mask(const struct pci_dev *pdev, unsigned int reg, + void *data) +{ + const struct mask_data *md = data; + + return md->val; +} + +static void vpci_write32_mask(const struct pci_dev *pdev, unsigned int reg, + uint32_t val, void *data) +{ + struct mask_data *md = data; + + md->val = val | (md->val & md->rw1c_mask); + md->val &= ~(val & md->rw1c_mask); +} + #define VPCI_READ(reg, size, data) ({ \ data = vpci_read((pci_sbdf_t){ .sbdf = 0 }, reg, size); \ }) @@ -94,9 +116,21 @@ static void vpci_write32(const struct pci_dev *pdev, unsigned int reg, assert(!vpci_add_register(test_pdev.vpci, fread, fwrite, off, size, \ &store)) +#define VPCI_ADD_REG_MASK(fread, fwrite, off, size, store, \ + ro_mask, rw1c_mask, rsvdp_mask, rsvdz_mask) \ + assert(!vpci_add_register_mask(test_pdev.vpci, fread, fwrite, off, size, \ + &store, \ + ro_mask, rw1c_mask, rsvdp_mask, rsvdz_mask)) + #define VPCI_ADD_INVALID_REG(fread, fwrite, off, size) \ assert(vpci_add_register(test_pdev.vpci, fread, fwrite, off, size, NULL)) +#define VPCI_ADD_INVALID_REG_MASK(fread, fwrite, off, size, \ + ro_mask, rw1c_mask, rsvdp_mask, rsvdz_mask) \ + assert(vpci_add_register_mask(test_pdev.vpci, fread, fwrite, off, size, \ + NULL, ro_mask, rw1c_mask, rsvdp_mask, \ + rsvdz_mask)) + #define VPCI_REMOVE_REG(off, size) \ assert(!vpci_remove_register(test_pdev.vpci, off, size)) @@ -154,6 +188,7 @@ main(int argc, char **argv) uint16_t r20[2] = { }; uint32_t r24 = 0; uint8_t r28, r30; + struct mask_data r32; unsigned int i; int rc; @@ -213,6 +248,24 @@ main(int argc, char **argv) /* Try to add a register with missing handlers. */ VPCI_ADD_INVALID_REG(NULL, NULL, 8, 2); + /* Try to add registers with the same bits set in multiple masks. */ + VPCI_ADD_INVALID_REG_MASK(vpci_read32, vpci_write32, 8, 4, 1, 1, 0, 0); + VPCI_ADD_INVALID_REG_MASK(vpci_read32, vpci_write32, 8, 4, 1, 0, 1, 0); + VPCI_ADD_INVALID_REG_MASK(vpci_read32, vpci_write32, 8, 4, 1, 0, 0, 1); + VPCI_ADD_INVALID_REG_MASK(vpci_read32, vpci_write32, 8, 4, 0, 1, 1, 0); + VPCI_ADD_INVALID_REG_MASK(vpci_read32, vpci_write32, 8, 4, 0, 1, 0, 1); + VPCI_ADD_INVALID_REG_MASK(vpci_read32, vpci_write32, 8, 4, 0, 0, 1, 1); + + /* Try to add registers with mask bits set beyond the register size */ + VPCI_ADD_INVALID_REG_MASK(vpci_read8, vpci_write8, 8, 1, 0x100U, 0, 0, 0); + VPCI_ADD_INVALID_REG_MASK(vpci_read8, vpci_write8, 8, 1, 0, 0x100U, 0, 0); + VPCI_ADD_INVALID_REG_MASK(vpci_read8, vpci_write8, 8, 1, 0, 0, 0x100U, 0); + VPCI_ADD_INVALID_REG_MASK(vpci_read8, vpci_write8, 8, 1, 0, 0, 0, 0x100U); + VPCI_ADD_INVALID_REG_MASK(vpci_read16, vpci_write16, 8, 2, 0x10000U,0,0,0); + VPCI_ADD_INVALID_REG_MASK(vpci_read16, vpci_write16, 8, 2, 0,0x10000U,0,0); + VPCI_ADD_INVALID_REG_MASK(vpci_read16, vpci_write16, 8, 2, 0,0,0x10000U,0); + VPCI_ADD_INVALID_REG_MASK(vpci_read16, vpci_write16, 8, 2, 0,0,0,0x10000U); + /* Read/write of unset register. */ VPCI_READ_CHECK(8, 4, 0xffffffff); VPCI_READ_CHECK(8, 2, 0xffff); @@ -287,6 +340,64 @@ main(int argc, char **argv) VPCI_ADD_REG(vpci_read8, vpci_write8, 30, 1, r30); VPCI_WRITE_CHECK(28, 4, 0xffacffdc); + /* + * Test ro/rw1c/rsvdp/rsvdz masks. + * + * 32 24 16 8 0 + * +------+------+------+------+ + * |rsvdz |rsvdp | rw1c | ro | 32 + * +------+------+------+------+ + * + */ + r32.rw1c_mask = 0x0000ff00U; + VPCI_ADD_REG_MASK(vpci_read32_mask, vpci_write32_mask, 32, 4, r32, + 0x000000ffU /* RO */, + r32.rw1c_mask /* RW1C */, + 0x00ff0000U /* RsvdP */, + 0xff000000U /* RsvdZ */); + + /* ro */ + r32.val = 0x0f0f0f0fU; + VPCI_READ_CHECK(32, 1, 0x0f); + VPCI_WRITE(32, 1, 0x5a); + VPCI_READ_CHECK(32, 1, 0x0f); + assert(r32.val == 0x000f0f0fU); + + /* rw1c */ + r32.val = 0x0f0f0f0fU; + VPCI_READ_CHECK(33, 1, 0x0f); + VPCI_WRITE(33, 1, 0x5a); + VPCI_READ_CHECK(33, 1, 0x05); + assert(r32.val == 0x000f050fU); + + /* rsvdp */ + r32.val = 0x0f0f0f0fU; + VPCI_READ_CHECK(34, 1, 0); + VPCI_WRITE(34, 1, 0x5a); + VPCI_READ_CHECK(34, 1, 0); + assert(r32.val == 0x000f0f0fU); + + /* rsvdz */ + r32.val = 0x0f0f0f0fU; + VPCI_READ_CHECK(35, 1, 0); + VPCI_WRITE(35, 1, 0x5a); + VPCI_READ_CHECK(35, 1, 0); + assert(r32.val == 0x000f0f0fU); + + /* write all 0's */ + r32.val = 0x0f0f0f0fU; + VPCI_READ_CHECK(32, 4, 0x00000f0fU); + VPCI_WRITE(32, 4, 0); + VPCI_READ_CHECK(32, 4, 0x00000f0fU); + assert(r32.val == 0x000f0f0fU); + + /* write all 1's */ + r32.val = 0x0f0f0f0fU; + VPCI_READ_CHECK(32, 4, 0x00000f0fU); + VPCI_WRITE(32, 4, 0xffffffffU); + VPCI_READ_CHECK(32, 4, 0x0000000fU); + assert(r32.val == 0x000f000fU); + /* Finally try to remove a couple of registers. */ VPCI_REMOVE_REG(28, 1); VPCI_REMOVE_REG(24, 4); diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index 767c1ba718d7..351318121e48 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -521,6 +521,7 @@ static int cf_check init_bars(struct pci_dev *pdev) struct vpci_header *header = &pdev->vpci->header; struct vpci_bar *bars = header->bars; int rc; + bool mask_cap_list = false; switch ( pci_conf_read8(pdev->sbdf, PCI_HEADER_TYPE) & 0x7f ) { @@ -544,6 +545,17 @@ static int cf_check init_bars(struct pci_dev *pdev) if ( rc ) return rc; + /* Utilize rsvdp_mask to hide PCI_STATUS_CAP_LIST from the guest. */ + rc = vpci_add_register_mask(pdev->vpci, vpci_hw_read16, vpci_hw_write16, + PCI_STATUS, 2, NULL, + PCI_STATUS_RO_MASK & + ~(mask_cap_list ? PCI_STATUS_CAP_LIST : 0), + PCI_STATUS_RW1C_MASK, + mask_cap_list ? PCI_STATUS_CAP_LIST : 0, + PCI_STATUS_RSVDZ_MASK); + if ( rc ) + return rc; + if ( pdev->ignore_bars ) return 0; diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index 3bec9a4153da..d569f596a4c3 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -29,6 +29,10 @@ struct vpci_register { unsigned int offset; void *private; struct list_head node; + uint32_t ro_mask; + uint32_t rw1c_mask; + uint32_t rsvdp_mask; + uint32_t rsvdz_mask; }; #ifdef __XEN__ @@ -145,9 +149,17 @@ uint32_t cf_check vpci_hw_read32( return pci_conf_read32(pdev->sbdf, reg); } -int vpci_add_register(struct vpci *vpci, vpci_read_t *read_handler, - vpci_write_t *write_handler, unsigned int offset, - unsigned int size, void *data) +void cf_check vpci_hw_write16( + const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data) +{ + pci_conf_write16(pdev->sbdf, reg, val); +} + +int vpci_add_register_mask(struct vpci *vpci, vpci_read_t *read_handler, + vpci_write_t *write_handler, unsigned int offset, + unsigned int size, void *data, uint32_t ro_mask, + uint32_t rw1c_mask, uint32_t rsvdp_mask, + uint32_t rsvdz_mask) { struct list_head *prev; struct vpci_register *r; @@ -155,7 +167,14 @@ int vpci_add_register(struct vpci *vpci, vpci_read_t *read_handler, /* Some sanity checks. */ if ( (size != 1 && size != 2 && size != 4) || offset >= PCI_CFG_SPACE_EXP_SIZE || (offset & (size - 1)) || - (!read_handler && !write_handler) ) + (!read_handler && !write_handler) || (ro_mask & rw1c_mask) || + (ro_mask & rsvdp_mask) || (ro_mask & rsvdz_mask) || + (rw1c_mask & rsvdp_mask) || (rw1c_mask & rsvdz_mask) || + (rsvdp_mask & rsvdz_mask) ) + return -EINVAL; + + if ( size != 4 && + ((ro_mask | rw1c_mask | rsvdp_mask | rsvdz_mask) >> (8 * size)) ) return -EINVAL; r = xmalloc(struct vpci_register); @@ -167,6 +186,10 @@ int vpci_add_register(struct vpci *vpci, vpci_read_t *read_handler, r->size = size; r->offset = offset; r->private = data; + r->ro_mask = ro_mask; + r->rw1c_mask = rw1c_mask; + r->rsvdp_mask = rsvdp_mask; + r->rsvdz_mask = rsvdz_mask; spin_lock(&vpci->lock); @@ -376,6 +399,7 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size) } val = r->read(pdev, r->offset, r->private); + val &= ~(r->rsvdp_mask | r->rsvdz_mask); /* Check if the read is in the middle of a register. */ if ( r->offset < emu.offset ) @@ -407,26 +431,26 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size) /* * Perform a maybe partial write to a register. - * - * Note that this will only work for simple registers, if Xen needs to - * trap accesses to rw1c registers (like the status PCI header register) - * the logic in vpci_write will have to be expanded in order to correctly - * deal with them. */ static void vpci_write_helper(const struct pci_dev *pdev, const struct vpci_register *r, unsigned int size, unsigned int offset, uint32_t data) { + uint32_t curval = 0; + uint32_t preserved_mask = r->ro_mask | r->rsvdp_mask; + ASSERT(size <= r->size); - if ( size != r->size ) + if ( (size != r->size) || preserved_mask ) { - uint32_t val; - - val = r->read(pdev, r->offset, r->private); - data = merge_result(val, data, size, offset); + curval = r->read(pdev, r->offset, r->private); + curval &= ~r->rw1c_mask; + data = merge_result(curval, data, size, offset); } + data &= ~(preserved_mask | r->rsvdz_mask); + data |= curval & preserved_mask; + r->write(pdev, r->offset, data & (0xffffffffU >> (32 - 8 * r->size)), r->private); } diff --git a/xen/include/xen/pci_regs.h b/xen/include/xen/pci_regs.h index 84b18736a85d..9909b27425a5 100644 --- a/xen/include/xen/pci_regs.h +++ b/xen/include/xen/pci_regs.h @@ -66,6 +66,15 @@ #define PCI_STATUS_REC_MASTER_ABORT 0x2000 /* Set on master abort */ #define PCI_STATUS_SIG_SYSTEM_ERROR 0x4000 /* Set when we drive SERR */ #define PCI_STATUS_DETECTED_PARITY 0x8000 /* Set on parity error */ +#define PCI_STATUS_RSVDZ_MASK 0x0046 /* Includes PCI_STATUS_UDF */ + +#define PCI_STATUS_RO_MASK (PCI_STATUS_IMM_READY | PCI_STATUS_INTERRUPT | \ + PCI_STATUS_CAP_LIST | PCI_STATUS_66MHZ | PCI_STATUS_FAST_BACK | \ + PCI_STATUS_DEVSEL_MASK) +#define PCI_STATUS_RW1C_MASK (PCI_STATUS_PARITY | \ + PCI_STATUS_SIG_TARGET_ABORT | PCI_STATUS_REC_TARGET_ABORT | \ + PCI_STATUS_REC_MASTER_ABORT | PCI_STATUS_SIG_SYSTEM_ERROR | \ + PCI_STATUS_DETECTED_PARITY) #define PCI_CLASS_REVISION 0x08 /* High 24 bits are class, low 8 revision */ #define PCI_REVISION_ID 0x08 /* Revision ID */ diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index d743d96a10b8..85c52a1ebac7 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -32,11 +32,23 @@ int __must_check vpci_add_handlers(struct pci_dev *pdev); void vpci_remove_device(struct pci_dev *pdev); /* Add/remove a register handler. */ -int __must_check vpci_add_register(struct vpci *vpci, - vpci_read_t *read_handler, - vpci_write_t *write_handler, - unsigned int offset, unsigned int size, - void *data); +int __must_check vpci_add_register_mask(struct vpci *vpci, + vpci_read_t *read_handler, + vpci_write_t *write_handler, + unsigned int offset, unsigned int size, + void *data, uint32_t ro_mask, + uint32_t rw1c_mask, uint32_t rsvdp_mask, + uint32_t rsvdz_mask); +static inline int __must_check vpci_add_register(struct vpci *vpci, + vpci_read_t *read_handler, + vpci_write_t *write_handler, + unsigned int offset, + unsigned int size, void *data) +{ + return vpci_add_register_mask(vpci, read_handler, write_handler, offset, + size, data, 0, 0, 0, 0); +} + int __must_check vpci_remove_register(struct vpci *vpci, unsigned int offset, unsigned int size); @@ -50,6 +62,8 @@ uint32_t cf_check vpci_hw_read16( const struct pci_dev *pdev, unsigned int reg, void *data); uint32_t cf_check vpci_hw_read32( const struct pci_dev *pdev, unsigned int reg, void *data); +void cf_check vpci_hw_write16( + const struct pci_dev *pdev, unsigned int reg, uint32_t val, void *data); /* * Check for pending vPCI operations on this vcpu. Returns true if the vcpu From patchwork Mon Dec 4 21:09:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stewart Hildebrand X-Patchwork-Id: 13479112 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D1B1AC4167B for ; Mon, 4 Dec 2023 21:10:08 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.647311.1010320 (Exim 4.92) (envelope-from ) id 1rAGCR-0006Bo-QR; Mon, 04 Dec 2023 21:09:55 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 647311.1010320; Mon, 04 Dec 2023 21:09:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rAGCR-0006Bh-MY; Mon, 04 Dec 2023 21:09:55 +0000 Received: by outflank-mailman (input) for mailman id 647311; Mon, 04 Dec 2023 21:09:54 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rAGCQ-000698-6M for xen-devel@lists.xenproject.org; Mon, 04 Dec 2023 21:09:54 +0000 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on20601.outbound.protection.outlook.com [2a01:111:f400:7eae::601]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 773c3930-92e9-11ee-98e5-6d05b1d4d9a1; Mon, 04 Dec 2023 22:09:53 +0100 (CET) Received: from MW4PR03CA0024.namprd03.prod.outlook.com (2603:10b6:303:8f::29) by CH2PR12MB4087.namprd12.prod.outlook.com (2603:10b6:610:7f::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.34; Mon, 4 Dec 2023 21:09:46 +0000 Received: from CO1PEPF000044F5.namprd05.prod.outlook.com (2603:10b6:303:8f:cafe::24) by MW4PR03CA0024.outlook.office365.com (2603:10b6:303:8f::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.34 via Frontend Transport; Mon, 4 Dec 2023 21:09:46 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000044F5.mail.protection.outlook.com (10.167.241.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7068.20 via Frontend Transport; Mon, 4 Dec 2023 21:09:45 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Mon, 4 Dec 2023 15:09:45 -0600 Received: from ubuntu.mshome.net (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.34 via Frontend Transport; Mon, 4 Dec 2023 15:09:43 -0600 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 773c3930-92e9-11ee-98e5-6d05b1d4d9a1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mV9Pq8L4nzuROBsNtI82U7W7V+rhRLK19D12bK8f8AYvYYUNf+SpJ4nIwEd3YYRPs+vlGYJCfB0JG724Jf1gnfpmawn3CjPZgpf/PcegPNYC069eeTZCqmCGrgFe07AxeYKjoqifIeYccvgmg6cqR+9dcEF2b1RpPvS2Sq9nmZTgEe+LiVClkalr8o36/D4Dtc8hYES+C5IiiVblmUj9XxQkY6vsCdzpad63Q01F7GBaqcw9ZtP822CVqZPHSI+dHxYaqhRcxvtU6KuY8K7drupfm3ibipSIomChX2hmjoXJutBk+oOsXO0YexLgpql2VNfqWe4TsQ+fodmL8aXCKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pRL0UvNdHD3s4CJd428rmdW97NYNmAI0AVnnctl7IJU=; b=Z1HfNLdeiGKu1soBCl5MFZvWX0UcXxTrNyUlHnUv51XW0GThB8SMh5C8/5DxBh6gWoZ4XPCwEbPC1RpDJZaGUbQQMmTXe/TLoCUfqMQPpqKpgD6X7KejOPohLDvJv+bf57DauHquNn09QadlYd/eQ9rRkX59Ce878RhiMKNcbjzPV8qOm1Qka4zBz8ZKc23H29JGCUwC7bXPcbFqAtIUxYfBEpVP6lA5+J2r+htLaDi04+LLQf/vU76rQ/SvFcdnd3Y9/wti+UmxvHKAGFqKH2xyr1qncI8Rdo9rezhGcuqKS8i+3UUG8DGqcQeshwGRvxRkvFg/jh+/adZBS71W0w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pRL0UvNdHD3s4CJd428rmdW97NYNmAI0AVnnctl7IJU=; b=1+PcMvFAzrGbT/lN/U9caiCr+MipdVfLb0KqXBW+V5C4NE/LiVlWog/AYZlUP3GE7kphuecEKMqMmaWtYJlO7wHQFSHVugYpYpt1tx3PYmG4EaHnNpw9A8j9Wb9jOAzD+rLkONWozEcI7FCpDyDWUMfDL0qNJTGMop5jW7ptzJQ= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Stewart Hildebrand To: CC: Stewart Hildebrand , Andrew Cooper , George Dunlap , "Jan Beulich" , Julien Grall , "Stefano Stabellini" , Wei Liu , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v10 2/2] xen/vpci: header: filter PCI capabilities Date: Mon, 4 Dec 2023 16:09:13 -0500 Message-ID: <20231204210916.3651999-3-stewart.hildebrand@amd.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20231204210916.3651999-1-stewart.hildebrand@amd.com> References: <20231204210916.3651999-1-stewart.hildebrand@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044F5:EE_|CH2PR12MB4087:EE_ X-MS-Office365-Filtering-Correlation-Id: 50ea0472-51c2-45db-a118-08dbf50d57e5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 8JiYhka1OAEpNF0t93kym/VsJ18OBG/x4E5ac6kj6rKXYF5VR1weSjOfsG6Zc3rScWc3HfTSuTD+LkRXZ4xg2eeozYgzz1GSFVNa0t2LCBj3D0bnPvP5tp+aNLhiTnzjq9fHC2jLwMSt+7jdmrpi6ih8q+8qGmt2j44PAO0RcYi2ya7oSf4+Rj+51sk4MwcJEhTcIKTrlKtFv9QCJ3F679vr0hePKZlfCK9yeh4MpjZqhXhBjrIIhuPAfDHotEWQm7UQDiIjOwN2aWT+9Kfm2MULKBUrkTlDbsDdIotZ87d7kFs1mQKJwXGzt/f6eZhhhDFdqDUuDyV22dMnSfzxWS3dKwcPt5N1+i/tiLtXA+Fk4fpCZNsA+JS2KTuKizUjzz4xdtT/RrPqXqYaRYR4YCCdnDclebEzVL63wcdR+yAF3uLNMoAiU2OgyzvXnYW4DU2gdDZVYe//a6Lvs2WRTwFaQagS/OsUb1RBg6CZNaHICSsXPkDXOdfmDhXBJ8bDWCfzYq848yXvqN49Fhojo7M9YYdzoFN6wZaCk5yBj7nAI902f3/PdI09+Ytm8JpwnyDVz3b8FnCW5RUisRM2GGGJuo3VoxuGidViVXs0wuqSIPCwz7kznl8b22uFByvcHXEEsvmcIgFVdoxPBJr9YJBn4Ax3w17Bi8rS28bZDxeB4PsmyDX7AJ0iXRvlhknbuH7TKi6FkVNKHWIMDbuK3EIfCfsi3GTFY5o4FeWnDKV+NPVZMsYqoCtMH6dDKC0xd3A/71vNSQUOhBV0p/1gFg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(4636009)(346002)(376002)(39860400002)(396003)(136003)(230922051799003)(82310400011)(1800799012)(64100799003)(451199024)(186009)(40470700004)(46966006)(36840700001)(336012)(26005)(478600001)(83380400001)(47076005)(6666004)(356005)(81166007)(426003)(1076003)(40480700001)(82740400003)(36756003)(2616005)(316002)(6916009)(54906003)(70586007)(70206006)(36860700001)(5660300002)(4326008)(86362001)(2906002)(8936002)(8676002)(40460700003)(44832011)(41300700001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Dec 2023 21:09:45.8916 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 50ea0472-51c2-45db-a118-08dbf50d57e5 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044F5.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4087 Currently, Xen vPCI only supports virtualizing the MSI and MSI-X capabilities. Hide all other PCI capabilities (including extended capabilities) from domUs for now, even though there may be certain devices/drivers that depend on being able to discover certain capabilities. We parse the physical PCI capabilities linked list and add vPCI register handlers for the next elements, inserting our own next value, thus presenting a modified linked list to the domU. Introduce helper functions vpci_hw_read8 and vpci_read_val. The vpci_read_val helper function returns a fixed value, which may be used for read as zero registers, or registers whose value doesn't change. Introduce pci_find_next_cap_ttl() helper while adapting the logic from pci_find_next_cap() to suit our needs, and implement the existing pci_find_next_cap() in terms of the new helper. Rename init_bars() to init_header() since it is now doing more than initializing BARs. Signed-off-by: Stewart Hildebrand Reviewed-by: Roger Pau Monné Reviewed-by: Jan Beulich --- v9->v10: * rename s/cap/caps/ parameter in pci_find_next_cap_ttl() * make caps parameter in pci_find_next_cap_ttl() an actual array * make supported_caps static * add Jan's R-b tag * mention renaming s/init_bars/init_header/ in commit message v8->v9: * move local variable definitions inside loop in pci_find_next_cap_ttl() * constify supported_caps array and cap parameter of pci_find_next_cap_ttl() * add comment by vpci_read_val() helper * rename s/init_bars/init_header/ * add Roger's R-b tag v7->v8: * use to array instead of match function * include lib.h for ARRAY_SIZE * don't emulate PCI_CAPABILITY_LIST register if PCI_STATUS_CAP_LIST bit is not set in hardware * spell out RAZ/WI acronym * dropped R-b tag since the patch has changed moderately since the last rev v6->v7: * no change v5->v6: * add register handlers before status register handler in init_bars() * s/header->mask_cap_list/mask_cap_list/ v4->v5: * use more appropriate types, continued * get rid of unnecessary hook function * add Jan's R-b v3->v4: * move mask_cap_list setting to this patch * leave pci_find_next_cap signature alone * use more appropriate types v2->v3: * get rid of > 0 in loop condition * implement pci_find_next_cap in terms of new pci_find_next_cap_ttl function so that hypothetical future callers wouldn't be required to pass &ttl. * change NULL to (void *)0 for RAZ value passed to vpci_read_val * change type of ttl to unsigned int * remember to mask off the low 2 bits of next in the initial loop iteration * change return type of pci_find_next_cap and pci_find_next_cap_ttl * avoid wrapping the PCI_STATUS_CAP_LIST condition by using ! instead of == 0 v1->v2: * change type of ttl to int * use switch statement instead of if/else * adapt existing pci_find_next_cap helper instead of rolling our own * pass ttl as in/out * "pass through" the lower 2 bits of the next pointer * squash helper functions into this patch to avoid transient dead code situation * extended capabilities RAZ/WI --- xen/drivers/pci/pci.c | 33 ++++++++++++------- xen/drivers/vpci/header.c | 67 +++++++++++++++++++++++++++++++++++++-- xen/drivers/vpci/vpci.c | 12 +++++++ xen/include/xen/pci.h | 3 ++ xen/include/xen/vpci.h | 6 ++++ 5 files changed, 108 insertions(+), 13 deletions(-) diff --git a/xen/drivers/pci/pci.c b/xen/drivers/pci/pci.c index 3569ccb24e9e..edf5b9f7ae9f 100644 --- a/xen/drivers/pci/pci.c +++ b/xen/drivers/pci/pci.c @@ -39,31 +39,42 @@ unsigned int pci_find_cap_offset(pci_sbdf_t sbdf, unsigned int cap) return 0; } -unsigned int pci_find_next_cap(pci_sbdf_t sbdf, unsigned int pos, - unsigned int cap) +unsigned int pci_find_next_cap_ttl(pci_sbdf_t sbdf, unsigned int pos, + const unsigned int caps[], unsigned int n, + unsigned int *ttl) { - u8 id; - int ttl = 48; - - while ( ttl-- ) + while ( (*ttl)-- ) { + unsigned int id, i; + pos = pci_conf_read8(sbdf, pos); if ( pos < 0x40 ) break; - pos &= ~3; - id = pci_conf_read8(sbdf, pos + PCI_CAP_LIST_ID); + id = pci_conf_read8(sbdf, (pos & ~3) + PCI_CAP_LIST_ID); if ( id == 0xff ) break; - if ( id == cap ) - return pos; + for ( i = 0; i < n; i++ ) + { + if ( id == caps[i] ) + return pos; + } - pos += PCI_CAP_LIST_NEXT; + pos = (pos & ~3) + PCI_CAP_LIST_NEXT; } + return 0; } +unsigned int pci_find_next_cap(pci_sbdf_t sbdf, unsigned int pos, + unsigned int cap) +{ + unsigned int ttl = 48; + + return pci_find_next_cap_ttl(sbdf, pos, &cap, 1, &ttl) & ~3; +} + /** * pci_find_ext_capability - Find an extended capability * @sbdf: PCI device to query diff --git a/xen/drivers/vpci/header.c b/xen/drivers/vpci/header.c index 351318121e48..58195549d50a 100644 --- a/xen/drivers/vpci/header.c +++ b/xen/drivers/vpci/header.c @@ -18,6 +18,7 @@ */ #include +#include #include #include #include @@ -513,7 +514,7 @@ static void cf_check rom_write( rom->addr = val & PCI_ROM_ADDRESS_MASK; } -static int cf_check init_bars(struct pci_dev *pdev) +static int cf_check init_header(struct pci_dev *pdev) { uint16_t cmd; uint64_t addr, size; @@ -545,6 +546,68 @@ static int cf_check init_bars(struct pci_dev *pdev) if ( rc ) return rc; + if ( !is_hardware_domain(pdev->domain) ) + { + if ( pci_conf_read16(pdev->sbdf, PCI_STATUS) & PCI_STATUS_CAP_LIST ) + { + /* Only expose capabilities to the guest that vPCI can handle. */ + unsigned int next, ttl = 48; + static const unsigned int supported_caps[] = { + PCI_CAP_ID_MSI, + PCI_CAP_ID_MSIX, + }; + + next = pci_find_next_cap_ttl(pdev->sbdf, PCI_CAPABILITY_LIST, + supported_caps, + ARRAY_SIZE(supported_caps), &ttl); + + rc = vpci_add_register(pdev->vpci, vpci_read_val, NULL, + PCI_CAPABILITY_LIST, 1, + (void *)(uintptr_t)next); + if ( rc ) + return rc; + + next &= ~3; + + if ( !next ) + /* + * If we don't have any supported capabilities to expose to the + * guest, mask the PCI_STATUS_CAP_LIST bit in the status + * register. + */ + mask_cap_list = true; + + while ( next && ttl ) + { + unsigned int pos = next; + + next = pci_find_next_cap_ttl(pdev->sbdf, + pos + PCI_CAP_LIST_NEXT, + supported_caps, + ARRAY_SIZE(supported_caps), &ttl); + + rc = vpci_add_register(pdev->vpci, vpci_hw_read8, NULL, + pos + PCI_CAP_LIST_ID, 1, NULL); + if ( rc ) + return rc; + + rc = vpci_add_register(pdev->vpci, vpci_read_val, NULL, + pos + PCI_CAP_LIST_NEXT, 1, + (void *)(uintptr_t)next); + if ( rc ) + return rc; + + next &= ~3; + } + } + + /* Extended capabilities read as zero, write ignore */ + rc = vpci_add_register(pdev->vpci, vpci_read_val, NULL, 0x100, 4, + (void *)0); + if ( rc ) + return rc; + } + /* Utilize rsvdp_mask to hide PCI_STATUS_CAP_LIST from the guest. */ rc = vpci_add_register_mask(pdev->vpci, vpci_hw_read16, vpci_hw_write16, PCI_STATUS, 2, NULL, @@ -642,7 +705,7 @@ static int cf_check init_bars(struct pci_dev *pdev) return (cmd & PCI_COMMAND_MEMORY) ? modify_bars(pdev, cmd, false) : 0; } -REGISTER_VPCI_INIT(init_bars, VPCI_PRIORITY_MIDDLE); +REGISTER_VPCI_INIT(init_header, VPCI_PRIORITY_MIDDLE); /* * Local variables: diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index d569f596a4c3..72ef277c4f8e 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -137,6 +137,18 @@ static void cf_check vpci_ignored_write( { } +uint32_t cf_check vpci_read_val( + const struct pci_dev *pdev, unsigned int reg, void *data) +{ + return (uintptr_t)data; +} + +uint32_t cf_check vpci_hw_read8( + const struct pci_dev *pdev, unsigned int reg, void *data) +{ + return pci_conf_read8(pdev->sbdf, reg); +} + uint32_t cf_check vpci_hw_read16( const struct pci_dev *pdev, unsigned int reg, void *data) { diff --git a/xen/include/xen/pci.h b/xen/include/xen/pci.h index 50d7dfb2a2fd..2a0a01bf92e5 100644 --- a/xen/include/xen/pci.h +++ b/xen/include/xen/pci.h @@ -205,6 +205,9 @@ int pci_mmcfg_read(unsigned int seg, unsigned int bus, int pci_mmcfg_write(unsigned int seg, unsigned int bus, unsigned int devfn, int reg, int len, u32 value); unsigned int pci_find_cap_offset(pci_sbdf_t sbdf, unsigned int cap); +unsigned int pci_find_next_cap_ttl(pci_sbdf_t sbdf, unsigned int pos, + const unsigned int caps[], unsigned int n, + unsigned int *ttl); unsigned int pci_find_next_cap(pci_sbdf_t sbdf, unsigned int pos, unsigned int cap); unsigned int pci_find_ext_capability(pci_sbdf_t sbdf, unsigned int cap); diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index 85c52a1ebac7..d20c301a3db3 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -57,7 +57,13 @@ uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, unsigned int size); void vpci_write(pci_sbdf_t sbdf, unsigned int reg, unsigned int size, uint32_t data); +/* Helper to return the value passed in data. */ +uint32_t cf_check vpci_read_val( + const struct pci_dev *pdev, unsigned int reg, void *data); + /* Passthrough handlers. */ +uint32_t cf_check vpci_hw_read8( + const struct pci_dev *pdev, unsigned int reg, void *data); uint32_t cf_check vpci_hw_read16( const struct pci_dev *pdev, unsigned int reg, void *data); uint32_t cf_check vpci_hw_read32(