From patchwork Mon Jul 8 11:41:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13726455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 39B8FC3DA44 for ; Mon, 8 Jul 2024 11:42:16 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.755302.1163621 (Exim 4.92) (envelope-from ) id 1sQmkl-0004HA-2F; Mon, 08 Jul 2024 11:41:55 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 755302.1163621; Mon, 08 Jul 2024 11:41:55 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sQmkk-0004H3-U2; Mon, 08 Jul 2024 11:41:54 +0000 Received: by outflank-mailman (input) for mailman id 755302; Mon, 08 Jul 2024 11:41:53 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sQmkj-00042n-NK for xen-devel@lists.xenproject.org; Mon, 08 Jul 2024 11:41:53 +0000 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on20600.outbound.protection.outlook.com [2a01:111:f403:2417::600]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 1096dd9e-3d1f-11ef-8776-851b0ebba9a2; Mon, 08 Jul 2024 13:41:51 +0200 (CEST) Received: from BN1PR14CA0030.namprd14.prod.outlook.com (2603:10b6:408:e3::35) by CY5PR12MB6372.namprd12.prod.outlook.com (2603:10b6:930:e::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.35; Mon, 8 Jul 2024 11:41:48 +0000 Received: from BN1PEPF00005FFE.namprd05.prod.outlook.com (2603:10b6:408:e3:cafe::ca) by BN1PR14CA0030.outlook.office365.com (2603:10b6:408:e3::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.36 via Frontend Transport; Mon, 8 Jul 2024 11:41:47 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by BN1PEPF00005FFE.mail.protection.outlook.com (10.167.243.230) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7762.17 via Frontend Transport; Mon, 8 Jul 2024 11:41:47 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 8 Jul 2024 06:41:43 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1096dd9e-3d1f-11ef-8776-851b0ebba9a2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KrJaAoR+wYEksI7ApLm4aHwpRoJxOJNhTZxW6zezpzv2gEj2yS1Vb1PbueX0lR7iHjpHBm/aEyF5hcPm+5ylXkmEaS/aELAUG95LBleE3gXLpCUHRJ91/I1ImqBhzPcfBfxop6lh/MmuD5fjKD7tg9LlsCySwSTjrRZOABR1VAidyVKna4CYtM7aVCo3FL1nsvuRTdTPOsLEUUY9u5O5wmTRv5uCSS4KxjPjC6qc4DVtyIN9IMwbXtVPjS9bSSqCmXc9dTADjsm3IFBnOxm3+UwIU7tOkiNDIZcrexgTYrPNjdPmC3CO+yP0dJx3coX8Ow411c21LI8B81IK8tadlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kSBan8AMKi3heWU3rnAw95no+PojTUn8JnZuEPQvIPs=; b=Cz2cUwRtE6at8RPXZp9Qfut3J/cpxiBJpYigaz5FfNSmlBODmr+1aKBcE8cDxleDNDXABQrxqoLilAeNhlCUdhKkWq9henLO9LVF7tCN+CHA7gBOj9iIYmkQy73piiuPaIeDZc+BVioW8X0fSUFBMEzzJwUqQ1pPcIEvdHRJIoL9nL0ZLkDE1cEy3TE+3KrBjBOzRLgm7I0Kj7Qsrqu5DF+nC92b9wWSSs8S2Q8bkx9kmoj7YlvZAx5MRr6flLW8fsyX9H3d08+UC/VTBFRF6yqbKivvWzF1/0QegEAH3Xa/FGUuJpF9eMGkVSYO25KP6xS5Mu97F9PGi4Pno/PsTA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kSBan8AMKi3heWU3rnAw95no+PojTUn8JnZuEPQvIPs=; b=MP/Zv7manSkjDoob1VR4CcIQzX7r0IIldnBb+vYimmFiTi5WcvVEg89GKVxmOjOL6NTxRGrxzhY3bUl+XO0G7s9/EmxH5uGbKsrRHdxPy1uylUnhWn+9+3jt2opcX+07GlVkYhJ/CFGC5F0t0OV8NvHEFFuv9YRQP3fzOaklleM= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: CC: Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Anthony PERARD , "Juergen Gross" , "Daniel P . Smith" , Stewart Hildebrand , Jiqian Chen , Huang Rui , Stewart Hildebrand Subject: [XEN PATCH v12 1/7] xen/pci: Add hypercall to support reset of pcidev Date: Mon, 8 Jul 2024 19:41:18 +0800 Message-ID: <20240708114124.407797-2-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708114124.407797-1-Jiqian.Chen@amd.com> References: <20240708114124.407797-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00005FFE:EE_|CY5PR12MB6372:EE_ X-MS-Office365-Filtering-Correlation-Id: a42e1c98-7f0d-44b0-395e-08dc9f42f32f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|7416014|376014|36860700013; X-Microsoft-Antispam-Message-Info: 4LRigjHWRbgfomhnzwcKB+OcGBIcPNOEEqyJsMtDrQs+p1N9lIw+2HOsG7AoFbu6I5kmeNH5LEzi2dm2GakdPXzJIMx/FtY5buaTMra4lpVj8LvbhmgqsxpGnhBqPHnk9x5uYdTs1dFczMbpgt6AqDOlgo/JvolgVTUIPwwOEcG1XPp8NRbTvCykIyRACTfzPFthFNRIUHlrEm7U4wcXUfkLDEbaR6J3yZlov9E6tewtMASaf1haz8V49WvQfg2sLJsccnlJZpBbpO3CnCtaex8I/rofc4ajs+Vl4rRaYk0ObMHfc5LPzV5V6yos/CxpstrCnXCMgMJ49v/EyIZUWebDLTa7WEikR5cyBIpTapXRVhHB6zjV068eFj4aUN3p3YAcOp7dWsc1FwGg1tI+DM4iW/joxsKqOfAdptTxApKn8FI+BD36ADqfEbX4+YIPzqJDMZhqZyOzqW1ZX6S7/XKgYkJQKMTYVbARcf+KIR75tw73nOsYZ797KyoHb7i+UYJmoXPZe/IotnP9MMZ4UiUo1upCTUdjDp4NUwlC4KkGYh/okOHceulKfSQR2oMOk7raB2K8JDnScelXxDPS+AyFVK6WNB6kId9kicfbeHXKHNEV8lhj7MMmVJzYiFXgRE8LL7WKoUYzI/PXwybLRvSjACAyEbYUQsHS/0aGLhRho/+Azs5dykF3aWNQe2dibzfclf/2hbAeEemWu/l0Bh1mot/s8KM2sjzNU6DmaaLWjQ+P+X82aBf4K0hnAsgid1N01d4qe9ZFm1jUSsvqbSxwMGJ8WqVC36Jn/Bjeq1RnPQeLEVzOkdxhB5ZGyI+Bj+KGwGCSwMYdlGGoj3aq4yDt5BRmT59BNAKi6byTM2VtrgTZoWvLAzOR3gwtovHmqUhn2/o0hElA9EqKLWGrjTFUr4LDPNMxeyKF1ZpugJUkKvehm/J9yIInKM493X+Z4OdNHiLAM+5/0CYPdvt+UZ5wUfi56PSn3prkXtx9XL5F0eEoE4OWakle5CaXEP1VYQwvDxZNA/9nAIDgEWzh4lZ73f8f52KYQfdvLlPq25bjyHuGmswI9QUkohEu7Orq0Edmftjsx6nv6mz0oN5uFozcqkffA0hJWgYMoNu/M7gHQQsCZcueRXc/yAyVxVMi+4GROXBdpf9wM5fVCIfyt3p7uioNsL+hQSlzhMyXUgSKfjDrpZ4pFGh+JXyzLWwN9TXeEkooOSnXIEBzcfYHMwhFQYxrQTBrp6eEARJZ2KkmC1SOY4tkLcZ+WtlE1WLraop5dE02EH+rWDFmyJ38O6HNyKYMOZSB9buNtBeruhvFZ9tFGhnfv0n503x/CTgbzWNmWqGjOtx29nxkfQHJv1FxzifyTvTB0Ll3xr+SbPZ63Wjntb1BXE0dzYAPAhOm+2U0+PDEnQVlLo6/2MMCVFdb4t1I07FHpfPDGcwW+7CjthtP73MX0PLMijPpNLez X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(7416014)(376014)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2024 11:41:47.5759 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a42e1c98-7f0d-44b0-395e-08dc9f42f32f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00005FFE.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6372 When a device has been reset on dom0 side, the Xen hypervisor doesn't get notification, so the cached state in vpci is all out of date compare with the real device state. To solve that problem, add a new hypercall to support the reset of pcidev and clear the vpci state of device. So that once the state of device is reset on dom0 side, dom0 can call this hypercall to notify hypervisor. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui Signed-off-by: Jiqian Chen Reviewed-by: Stewart Hildebrand Reviewed-by: Stefano Stabellini Reviewed-by: Jan Beulich --- xen/arch/x86/hvm/hypercall.c | 1 + xen/drivers/pci/physdev.c | 52 ++++++++++++++++++++++++++++++++++++ xen/drivers/vpci/vpci.c | 10 +++++++ xen/include/public/physdev.h | 16 +++++++++++ xen/include/xen/vpci.h | 8 ++++++ 5 files changed, 87 insertions(+) diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c index 7fb3136f0c7c..0fab670a4871 100644 --- a/xen/arch/x86/hvm/hypercall.c +++ b/xen/arch/x86/hvm/hypercall.c @@ -83,6 +83,7 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) case PHYSDEVOP_pci_mmcfg_reserved: case PHYSDEVOP_pci_device_add: case PHYSDEVOP_pci_device_remove: + case PHYSDEVOP_pci_device_state_reset: case PHYSDEVOP_dbgp_op: if ( !is_hardware_domain(currd) ) return -ENOSYS; diff --git a/xen/drivers/pci/physdev.c b/xen/drivers/pci/physdev.c index 42db3e6d133c..c0f47945d955 100644 --- a/xen/drivers/pci/physdev.c +++ b/xen/drivers/pci/physdev.c @@ -2,6 +2,7 @@ #include #include #include +#include #ifndef COMPAT typedef long ret_t; @@ -67,6 +68,57 @@ ret_t pci_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) break; } + case PHYSDEVOP_pci_device_state_reset: + { + struct pci_device_state_reset dev_reset; + struct pci_dev *pdev; + pci_sbdf_t sbdf; + + ret = -EOPNOTSUPP; + if ( !is_pci_passthrough_enabled() ) + break; + + ret = -EFAULT; + if ( copy_from_guest(&dev_reset, arg, 1) != 0 ) + break; + + sbdf = PCI_SBDF(dev_reset.dev.seg, + dev_reset.dev.bus, + dev_reset.dev.devfn); + + ret = xsm_resource_setup_pci(XSM_PRIV, sbdf.sbdf); + if ( ret ) + break; + + pcidevs_lock(); + pdev = pci_get_pdev(NULL, sbdf); + if ( !pdev ) + { + pcidevs_unlock(); + ret = -ENODEV; + break; + } + + write_lock(&pdev->domain->pci_lock); + pcidevs_unlock(); + switch ( dev_reset.reset_type ) + { + case PCI_DEVICE_STATE_RESET_COLD: + case PCI_DEVICE_STATE_RESET_WARM: + case PCI_DEVICE_STATE_RESET_HOT: + case PCI_DEVICE_STATE_RESET_FLR: + ret = vpci_reset_device_state(pdev, dev_reset.reset_type); + break; + + default: + ret = -EOPNOTSUPP; + break; + } + write_unlock(&pdev->domain->pci_lock); + + break; + } + default: ret = -ENOSYS; break; diff --git a/xen/drivers/vpci/vpci.c b/xen/drivers/vpci/vpci.c index 1e6aa5d799b9..7e914d1eff9f 100644 --- a/xen/drivers/vpci/vpci.c +++ b/xen/drivers/vpci/vpci.c @@ -172,6 +172,16 @@ int vpci_assign_device(struct pci_dev *pdev) return rc; } + +int vpci_reset_device_state(struct pci_dev *pdev, + uint32_t reset_type) +{ + ASSERT(rw_is_write_locked(&pdev->domain->pci_lock)); + + vpci_deassign_device(pdev); + return vpci_assign_device(pdev); +} + #endif /* __XEN__ */ static int vpci_register_cmp(const struct vpci_register *r1, diff --git a/xen/include/public/physdev.h b/xen/include/public/physdev.h index f0c0d4727c0b..3cfde3fd2389 100644 --- a/xen/include/public/physdev.h +++ b/xen/include/public/physdev.h @@ -296,6 +296,13 @@ DEFINE_XEN_GUEST_HANDLE(physdev_pci_device_add_t); */ #define PHYSDEVOP_prepare_msix 30 #define PHYSDEVOP_release_msix 31 +/* + * Notify the hypervisor that a PCI device has been reset, so that any + * internally cached state is regenerated. Should be called after any + * device reset performed by the hardware domain. + */ +#define PHYSDEVOP_pci_device_state_reset 32 + struct physdev_pci_device { /* IN */ uint16_t seg; @@ -305,6 +312,15 @@ struct physdev_pci_device { typedef struct physdev_pci_device physdev_pci_device_t; DEFINE_XEN_GUEST_HANDLE(physdev_pci_device_t); +struct pci_device_state_reset { + physdev_pci_device_t dev; +#define PCI_DEVICE_STATE_RESET_COLD 0 +#define PCI_DEVICE_STATE_RESET_WARM 1 +#define PCI_DEVICE_STATE_RESET_HOT 2 +#define PCI_DEVICE_STATE_RESET_FLR 3 + uint32_t reset_type; +}; + #define PHYSDEVOP_DBGP_RESET_PREPARE 1 #define PHYSDEVOP_DBGP_RESET_DONE 2 diff --git a/xen/include/xen/vpci.h b/xen/include/xen/vpci.h index da8d0f41e6f4..6be812dbc04a 100644 --- a/xen/include/xen/vpci.h +++ b/xen/include/xen/vpci.h @@ -38,6 +38,8 @@ int __must_check vpci_assign_device(struct pci_dev *pdev); /* Remove all handlers and free vpci related structures. */ void vpci_deassign_device(struct pci_dev *pdev); +int __must_check vpci_reset_device_state(struct pci_dev *pdev, + uint32_t reset_type); /* Add/remove a register handler. */ int __must_check vpci_add_register_mask(struct vpci *vpci, @@ -282,6 +284,12 @@ static inline int vpci_assign_device(struct pci_dev *pdev) static inline void vpci_deassign_device(struct pci_dev *pdev) { } +static inline int __must_check vpci_reset_device_state(struct pci_dev *pdev, + uint32_t reset_type) +{ + return 0; +} + static inline void vpci_dump_msi(void) { } static inline uint32_t vpci_read(pci_sbdf_t sbdf, unsigned int reg, From patchwork Mon Jul 8 11:41:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13726456 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96DB5C3271E for ; Mon, 8 Jul 2024 11:42:16 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.755304.1163640 (Exim 4.92) (envelope-from ) id 1sQmkr-0004n9-Gu; Mon, 08 Jul 2024 11:42:01 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 755304.1163640; Mon, 08 Jul 2024 11:42:01 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sQmkr-0004n0-EB; Mon, 08 Jul 2024 11:42:01 +0000 Received: by outflank-mailman (input) for mailman id 755304; Mon, 08 Jul 2024 11:42:00 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sQmkq-00042h-9a for xen-devel@lists.xenproject.org; Mon, 08 Jul 2024 11:42:00 +0000 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on20604.outbound.protection.outlook.com [2a01:111:f403:200a::604]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 14c06e82-3d1f-11ef-bbfb-fd08da9f4363; Mon, 08 Jul 2024 13:41:58 +0200 (CEST) Received: from BN9PR03CA0784.namprd03.prod.outlook.com (2603:10b6:408:13f::9) by PH0PR12MB7888.namprd12.prod.outlook.com (2603:10b6:510:28b::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.35; Mon, 8 Jul 2024 11:41:52 +0000 Received: from BN1PEPF00006001.namprd05.prod.outlook.com (2603:10b6:408:13f:cafe::5f) by BN9PR03CA0784.outlook.office365.com (2603:10b6:408:13f::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.36 via Frontend Transport; Mon, 8 Jul 2024 11:41:51 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by BN1PEPF00006001.mail.protection.outlook.com (10.167.243.233) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7762.17 via Frontend Transport; Mon, 8 Jul 2024 11:41:51 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 8 Jul 2024 06:41:47 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 14c06e82-3d1f-11ef-bbfb-fd08da9f4363 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Kf6GrPl2eL5LWJxBR+ApYMf7KS4Sv39mS17uxD5ldjtr6UwVThX62/FEewnX/po35cTmfQbppc5+SL2xxFHTdz+7YhY/mk+p5ogRy41Kga+yD5M0B+4R2ild4kgByhJIBmHVjImxmy8GtkValMUwGmSUaxLeyId3a6KOWtIeadJoW+cgEymqjKIpCt/9HFt7a+g97Grliuve3AXm9Y5zecnB9XWWzCyt5iJ1WP8HJr5HR7N5EvNOlBELvhVX8DXeNL623LwFhkO3VZlF51OdG41OCfcetoGL6yncJ68bEsDJ12zYMGUBxZMt3HVgLEGjm4eoMd4n7yG+JEnS9VgK8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mHHPfEU7Tj3wFWabDhvJJiTUzdQGCtYEiTn+now2Hu8=; b=egX97bG05FPNnK+goiF0xgLwN9ZUOjP0UKNwF2ZXCAOphpRJQ8J5GM0e+oSS5g4j/zYO/SFqMc5dGHiXPb4O7EZVg41x+Hk2wmszL+SpFbckkzoHWOs613Xkgr1LHVp84WVL/i1jRx7a9+eMbOIbdr1Fc0PxP1HDbV+tPUJdpx2+JS5B8LjecOWoI1EU92ZwdmJW0ZIxekz+uxf0eWU562EMUgGLgLq2az2467Vge+APXuVvpiiH2413ulGC5YP3An+5kIVFFmj48gdjI2XcRc5Sl1FwPHPw4jasdnVDJZaywcqB5KC28jWETFeIT3r9CFrbLYxS267ALCyQPc1Y1Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mHHPfEU7Tj3wFWabDhvJJiTUzdQGCtYEiTn+now2Hu8=; b=bcB0yuKzUDYrtIxhNXSHk4d31fHy04dg19o1oSGTEVevFUYWX4wPhCvIsmv25H92bf5IeYEbOaboaiwGF1ecrQSsC20jsJLgu5w7G4uI57NMRAZ4kEWrHSa7kNcbmB6YapazqjJMJWT7JNCM6WGAS03geBs+2wSOTLQshOKyA/0= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: CC: Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Anthony PERARD , "Juergen Gross" , "Daniel P . Smith" , Stewart Hildebrand , Jiqian Chen , Huang Rui Subject: [XEN PATCH v12 2/7] x86/pvh: Allow (un)map_pirq when dom0 is PVH Date: Mon, 8 Jul 2024 19:41:19 +0800 Message-ID: <20240708114124.407797-3-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708114124.407797-1-Jiqian.Chen@amd.com> References: <20240708114124.407797-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00006001:EE_|PH0PR12MB7888:EE_ X-MS-Office365-Filtering-Correlation-Id: b3592a91-5eb1-4bcd-97f5-08dc9f42f545 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|1800799024|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: aNFVr3j6gTwSBsLAPTQl2+bKvo9vjxJKgNEFuxlsBO2Z2OCNUnzdO4Th+ROWXRQwvZ7SiPMAGXvBAOYUsGCFEPC3o89Od4hUlLi5h8EE0uCIb5aXAA0dAJpK5C4AKWKY3KGP5PfMJU2Ewdg9A0uFrTUwOsCmEyOH5iKcDCoV1NT+g7/o6s9rpM1oDaBbaYhefkGBP5vYndtLhPXIFzAZsDHEMro4B8v4YthAmPL7j1H30Hqg9+caVWTziind6Tgnp5++U/zgtPIGeFC9qZDSvVQ3j8rR/oAFpX5o3ckpQIMGarclB0aCgcM8MnZ/RnJgqNlERCosTfhJMBjIhBID4JGMnWglbBuu2Y+9+5QXA68lPnZH432UzQQYcjAALLtA+JWK7KYaHcDMEUbKFTug0MAHuMuk5iRMx/D3tlAQgQJJ0JXlZDCRtgIItY9sskHtDy+Y84NdsVLCULuH4ZsQqZjIAhUV116U6W6szlD2CNvbazeC8C4XmqdvdhYpIqzBeuNOcDxObSWsVuYjGXLaAz8Tvsmf/h/xKRmzg8mxVMhSCk0/1Gvoll9lcT256dHpPkdjyqkwK+5ACVwBnwvs/ugCIwAJPV2+/TLYcUJVzCMJcCNJLDiYWqf+DdMH4Aj2cv+UbA6uFIzeGyJ157tCBB9rrUgdNoA4p/PJXyOM9WS6qNkZawFV2kUFZMwXJOWor06nG6R8RvlPCgQsrijsmR7j9G0a5JeQsdNB1gb2I1YcB3kGR27ZdKPBHEVfmrkx5dmLg84IxQkmPfE1HFw/yA5yE9pGRFU+zqyukb4hD5Xy7i1saoQLsAO45CJc3dO2yciXHjMioBbxZu1cq1FbPnDEQaP6hXFQp8m41TMrIkaXMT9fkdX+Qw99Alwse/a6rKUicCPQctrm6j53u+yxiW8ETT0eRhXp+4VxQtgtiS7pPg4tky2/La8OQrtIw/u2gI+LCSRTjE1EjouGwXH447i/dIM0nFhIv+crt8PNTFZqoyv7bLTA7wFwd4AFHT2OYwZZwZbatJebL4HGYHVEmL9SIygWeOqwtRL5aLSTJur0kYBdQ+mieCLp02dzkVHrVeP0NYS6B0kt3L+FXNOGdxprsw7c8hesyT2SD9WphuLj+zD3tU6QTDnLaN1lQPVP5Tpd8BDsrTHonDJ6U+dLEvgYU3Rs3+ipCkyt8GutLU82XQsLf3mjleqkR8ffWcB93O/b7bbg2OkxtQWq9KEaSBic45sLsvp5mQoyE9QlW9KgUh+CvoFz38GXODpR21IIPBi5sZ0rvaOHMDp79lVE+3FQwH3pgtMWjLgbz1jwISpWrFNGJ9BHxugEm3MBld7WVo9QcpB89lb7kVOkTqOh1dvcKCc2Cv3FHKAYol/9M3obukw2FVsq2Fc+fY2oVc84rRFoVO2dFxhUh8NMPNWNBfe5JBRr9481M5L9nA0njblpqR/JSNDX2BQu/gXKULyV X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(7416014)(1800799024)(36860700013)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2024 11:41:51.0779 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b3592a91-5eb1-4bcd-97f5-08dc9f42f545 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00006001.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7888 If run Xen with PVH dom0 and hvm domU, hvm will map a pirq for a passthrough device by using gsi, see qemu code xen_pt_realize->xc_physdev_map_pirq and libxl code pci_add_dm_done->xc_physdev_map_pirq. Then xc_physdev_map_pirq will call into Xen, but in hvm_physdev_op, PHYSDEVOP_map_pirq is not allowed because currd is PVH dom0 and PVH has no X86_EMU_USE_PIRQ flag, it will fail at has_pirq check. So, allow PHYSDEVOP_map_pirq when dom0 is PVH and also allow PHYSDEVOP_unmap_pirq for the removal device path to unmap pirq. And add a new check to prevent (un)map when the subject domain doesn't have a notion of PIRQ. So that the interrupt of a passthrough device can be successfully mapped to pirq for domU with a notion of PIRQ when dom0 is PVH Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui Signed-off-by: Jiqian Chen Reviewed-by: Jan Beulich Reviewed-by: Stefano Stabellini --- xen/arch/x86/hvm/hypercall.c | 6 ++++++ xen/arch/x86/physdev.c | 12 ++++++++++-- 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c index 0fab670a4871..03ada3c880bd 100644 --- a/xen/arch/x86/hvm/hypercall.c +++ b/xen/arch/x86/hvm/hypercall.c @@ -71,8 +71,14 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) switch ( cmd ) { + /* + * Only being permitted for management of other domains. + * Further restrictions are enforced in do_physdev_op. + */ case PHYSDEVOP_map_pirq: case PHYSDEVOP_unmap_pirq: + break; + case PHYSDEVOP_eoi: case PHYSDEVOP_irq_status_query: case PHYSDEVOP_get_free_pirq: diff --git a/xen/arch/x86/physdev.c b/xen/arch/x86/physdev.c index d6dd622952a9..9f30a8c63a06 100644 --- a/xen/arch/x86/physdev.c +++ b/xen/arch/x86/physdev.c @@ -323,7 +323,11 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) if ( !d ) break; - ret = physdev_map_pirq(d, map.type, &map.index, &map.pirq, &msi); + /* Only mapping when the subject domain has a notion of PIRQ */ + if ( !is_hvm_domain(d) || has_pirq(d) ) + ret = physdev_map_pirq(d, map.type, &map.index, &map.pirq, &msi); + else + ret = -EOPNOTSUPP; rcu_unlock_domain(d); @@ -346,7 +350,11 @@ ret_t do_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) if ( !d ) break; - ret = physdev_unmap_pirq(d, unmap.pirq); + /* Only unmapping when the subject domain has a notion of PIRQ */ + if ( !is_hvm_domain(d) || has_pirq(d) ) + ret = physdev_unmap_pirq(d, unmap.pirq); + else + ret = -EOPNOTSUPP; rcu_unlock_domain(d); From patchwork Mon Jul 8 11:41:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13726453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 34D60C3271E for ; Mon, 8 Jul 2024 11:42:14 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.755303.1163630 (Exim 4.92) (envelope-from ) id 1sQmkq-0004Y0-Am; Mon, 08 Jul 2024 11:42:00 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 755303.1163630; Mon, 08 Jul 2024 11:42:00 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sQmkq-0004Xp-7W; Mon, 08 Jul 2024 11:42:00 +0000 Received: by outflank-mailman (input) for mailman id 755303; Mon, 08 Jul 2024 11:41:59 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sQmkp-00042h-9Q for xen-devel@lists.xenproject.org; Mon, 08 Jul 2024 11:41:59 +0000 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on20628.outbound.protection.outlook.com [2a01:111:f403:200a::628]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 14d4aa0d-3d1f-11ef-bbfb-fd08da9f4363; Mon, 08 Jul 2024 13:41:58 +0200 (CEST) Received: from BN7PR02CA0030.namprd02.prod.outlook.com (2603:10b6:408:20::43) by MW4PR12MB7432.namprd12.prod.outlook.com (2603:10b6:303:22d::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.35; Mon, 8 Jul 2024 11:41:55 +0000 Received: from BN1PEPF00005FFC.namprd05.prod.outlook.com (2603:10b6:408:20:cafe::4b) by BN7PR02CA0030.outlook.office365.com (2603:10b6:408:20::43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.36 via Frontend Transport; Mon, 8 Jul 2024 11:41:54 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by BN1PEPF00005FFC.mail.protection.outlook.com (10.167.243.228) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7762.17 via Frontend Transport; Mon, 8 Jul 2024 11:41:54 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 8 Jul 2024 06:41:50 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 14d4aa0d-3d1f-11ef-bbfb-fd08da9f4363 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q9j5/RNdK47rKocWPaGOn8sv/FDf72RssvHOc3ml7s6K7NdQAAmheVx/1TGJJ94yXGL2EknWPxxYiCBFZsbRg/DB05mCHxcxU2QWdrD5nh8NzMtHnzi+8DX5eiR+lqCkSae84oMzMLoYQb1lglZUgaKvYMCHN+0i00mUcZXgemYjr3tVvDh915iEYFLF6UnZmoFC/4nRC7aS0R5m7hPZOmkF2IVhfxkHi62dx/ZsWP/uQCgtJdiyxk+9aWl56EYQLgRB9RgHMoRNkvJAuOOW1YS9E411x4tEPqDQfjeN081JrcOAMpxH41So1RoWF7Ij1VNahW6T/ypeW1sUXi3udw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=szi3iqQfbMQzivoN8ExWAziYbJZhc5SH7BnahLl6nJA=; b=cs5odoNzjBzWYiIaIN8at8IYZY2ts+rc5rim7fB4D6wzmnie6WfnxyAbYDOgSCtE8/kcNJGq3jxatPX41Y7nzCX2qlVCZv+ZcITYgmmt3W9klyuJaf7rfsNzTnZYdZ0AcBGVWnReiCjkL2LwnSAUI9gAVNT+o4Te+rNvxdWzDvL9hJDc6eWaDcLI5CuKJdZtlb8NiWYcWvikJ1gBSHCgz8cCw/c6nP0nsh9JqdXckcY1qyDzRSboFepHOZKGJpQqznItyyqY60Wc7wNlfd5Jd+p+VyHq/lNoT/gXBxyilyqqtYvaNJiiZ/F7TH1Cv/xrENGiTBlyrHdtvzhUHi6KIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=szi3iqQfbMQzivoN8ExWAziYbJZhc5SH7BnahLl6nJA=; b=0jK59+CuSyL0UyagxDFSafhqK/7yLnL/mo5KU8z8ogKLI83URIimMUhx9Vpo8qqjgem15Sg5KaQ0mAiKMs5cV1B0bCaPG6HlperOQdXkT2O5OkUOQnl5rKFGuvbUPOk9AY7C2kmoAowuHdsfSi5FagnEw2+nMrI5zkC01TTx3Xg= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: CC: Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Anthony PERARD , "Juergen Gross" , "Daniel P . Smith" , Stewart Hildebrand , Jiqian Chen , Huang Rui Subject: [XEN PATCH v12 3/7] x86/pvh: Add PHYSDEVOP_setup_gsi for PVH dom0 Date: Mon, 8 Jul 2024 19:41:20 +0800 Message-ID: <20240708114124.407797-4-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708114124.407797-1-Jiqian.Chen@amd.com> References: <20240708114124.407797-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00005FFC:EE_|MW4PR12MB7432:EE_ X-MS-Office365-Filtering-Correlation-Id: 7091c7f4-b1fa-430b-79d2-08dc9f42f75d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|82310400026|1800799024|36860700013|7416014; X-Microsoft-Antispam-Message-Info: SPucu6IMkVwJuyqLy+kkNrIm8wEv+5Q2BvhGUlMZxTH5Ndq7AnPbBI3UWVfprAD/Iq4yJplyn9CNL5kg8haVFdl7WitxItdsiOXAxgWLx+Em8dYXLr18haYfTf4fi8MM7/Y5ZkU4CoMwe1bQyqhmHl47cD9wkDsx8tTixh8c0cB622scGoaB4fCgvoJwseNXTAoKRGkX6LPJ0pYlZ1TmE60NQxhwfAY5CAxXtt9oF0NY2VZrdpTtqbuTjvodz8cQa2Xp2TUrMjM69z8AMusALKoi8hqbsEQHwj7R1WVeWPxQnebrwlz1uvZsRfin254Z8zfZ0nvOERoRvNN0ecS9LLic0c3jvQTLdLEhbWs0TWP+UWK1aMv6x9Vh2Uyfg7877GMI+48Yqt7CrJO6YNpwjQ9C+W9vRMhumxeJ5mn3J9Ox2QKDVaN3De0eJx5Qex6064/w1blCo2m0tvsbr+aDkMeImKli4baJRgu2I1qzihwER+/86WT3u54meFIklSqmaehkQQT3EgzpEc6IIT5V0ksw5Q5BvnNO7MN1UL/ntafEK52KnGtbDj09yNPfADgBJwkh0jmbUqs5hQCbtkp6jKRQ3ZAAPdKMIOLTW90oYLLDVqFffBZL2ZDGFEeGxpYVsvHz9LUtXyrVrEXrsKijUv9PQL/43suhsSrY15Ogvu3T2bMaXWOMSlhpiYzczGCBJkbRHmUaqwEPr3Q74Nu2hiErg0NvkXpZ24nxvidls4kf4IoPH8v1u+BA4KwLnPpx7xQ0QaET2XE7BOSwFzeE/FAPe3poPs6lmqoFWkeceQbDi77wzazxCHuUYaPr9GOqCEpuQPLlJtYuorOPBL50N4C9N/OsX+Ryoe1PQkw3bIX/NYGi9oS18TniK2jAi+jmy9rLufTDyd6U7l5H6ZG+1YoqkGNmGOSlmYI/FVl5NThEA/0/EZlryh0kzWx5cFR3wlYAA+3DBcZcutt5F3++mSSD1uIXkosmlxteUW3lm8JUw8fUnE+kIRazIsG3EsEChfGf4pSFz/2fAj87VIciUN7B1L1AQV7Nz34j9gbv8IOBmVeyN4HYgPsvSIMF5fGtwPUNK2A+u4Yumutoa7PxQNKTkRGhi/3avf1oG/8qYfpZLj/pCPdumIL4MDCJSK2yKJk+Mqhz2if+E3Pv7MJUh7UpnWPP1IWt0+XMaoVPFQZBxJekwc/wsJL0M06bsy2YmluFs/wbePAYslRMqWmNo0oaurlFzvxNDR0x0dWXmbAMiVaWEOltGr2dNmc102ZtvRXAWCBHddQZElVzbIdI9Eu+3F78yJji9A7F8+6RskfIKb2qqnznaIc+ZsJNOj4v67z70Il30h5IaeGp0aVpT0+MZyI4xNx6RhqIf9qYX8x04o0K/tUNfhDD+9bXppCas0BbOXK8Pt3/hhJ4gju5rTUeSXioh9jZnc/Lg3THufc+tixPT8FnQaGvzw0BqdP+ X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(82310400026)(1800799024)(36860700013)(7416014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2024 11:41:54.5742 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7091c7f4-b1fa-430b-79d2-08dc9f42f75d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00005FFC.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7432 The gsi of a passthrough device must be configured for it to be able to be mapped into a hvm domU. But When dom0 is PVH, the gsis may not get registered(see below clarification), it causes the info of apic, pin and irq not be added into irq_2_pin list, and the handler of irq_desc is not set, then when passthrough a device, setting ioapic affinity and vector will fail. To fix above problem, on Linux kernel side, a new code will need to call PHYSDEVOP_setup_gsi for passthrough devices to register gsi when dom0 is PVH. So, add PHYSDEVOP_setup_gsi into hvm_physdev_op for above purpose. Clarify two questions: First, why the gsi of devices belong to PVH dom0 can work? Because when probe a driver to a normal device, it uses the normal probe function of pci device, in its callstack, it requests irq and unmask corresponding ioapic of gsi, then trap into xen and register gsi finally. Callstack is(on linux kernel side) pci_device_probe-> request_threaded_irq-> irq_startup-> __unmask_ioapic-> io_apic_write, then trap into xen hvmemul_do_io-> hvm_io_intercept-> hvm_process_io_intercept-> vioapic_write_indirect-> vioapic_hwdom_map_gsi-> mp_register_gsi. So that the gsi can be registered. Second, why the gsi of passthrough device can't work when dom0 is PVH? Because when assign a device to passthrough, it uses the specific probe function of pciback, in its callstack, it doesn't install a fake irq handler due to the ISR is not running. So that mp_register_gsi on Xen side is never called, then the gsi is not registered. Callstack is(on linux kernel side) pcistub_probe->pcistub_seize-> pcistub_init_device-> xen_pcibk_reset_device-> xen_pcibk_control_isr->isr_on==0. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui Signed-off-by: Jiqian Chen Reviewed-by: Stefano Stabellini --- xen/arch/x86/hvm/hypercall.c | 1 + 1 file changed, 1 insertion(+) diff --git a/xen/arch/x86/hvm/hypercall.c b/xen/arch/x86/hvm/hypercall.c index 03ada3c880bd..cfe82d0f96ed 100644 --- a/xen/arch/x86/hvm/hypercall.c +++ b/xen/arch/x86/hvm/hypercall.c @@ -86,6 +86,7 @@ long hvm_physdev_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg) return -ENOSYS; break; + case PHYSDEVOP_setup_gsi: case PHYSDEVOP_pci_mmcfg_reserved: case PHYSDEVOP_pci_device_add: case PHYSDEVOP_pci_device_remove: From patchwork Mon Jul 8 11:41:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13726454 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 81E6CC3DA42 for ; Mon, 8 Jul 2024 11:42:15 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.755305.1163650 (Exim 4.92) (envelope-from ) id 1sQmku-000551-Oy; Mon, 08 Jul 2024 11:42:04 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 755305.1163650; Mon, 08 Jul 2024 11:42:04 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sQmku-00054q-LU; Mon, 08 Jul 2024 11:42:04 +0000 Received: by outflank-mailman (input) for mailman id 755305; Mon, 08 Jul 2024 11:42:02 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sQmks-00042h-Mj for xen-devel@lists.xenproject.org; Mon, 08 Jul 2024 11:42:02 +0000 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2060e.outbound.protection.outlook.com [2a01:111:f400:7e88::60e]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 16c23c5b-3d1f-11ef-bbfb-fd08da9f4363; Mon, 08 Jul 2024 13:42:01 +0200 (CEST) Received: from BN9PR03CA0789.namprd03.prod.outlook.com (2603:10b6:408:13f::14) by SA0PR12MB7461.namprd12.prod.outlook.com (2603:10b6:806:24b::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.35; Mon, 8 Jul 2024 11:41:58 +0000 Received: from BN1PEPF00006001.namprd05.prod.outlook.com (2603:10b6:408:13f:cafe::2a) by BN9PR03CA0789.outlook.office365.com (2603:10b6:408:13f::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.36 via Frontend Transport; Mon, 8 Jul 2024 11:41:58 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by BN1PEPF00006001.mail.protection.outlook.com (10.167.243.233) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7762.17 via Frontend Transport; Mon, 8 Jul 2024 11:41:58 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 8 Jul 2024 06:41:54 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 16c23c5b-3d1f-11ef-bbfb-fd08da9f4363 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SGE7zOpqxHmjwGlys5GDhICzhIr2DA2J4DJc2hci/AOm4gJxDVsYFMGJx6yrxvsXO6El+duMyMHVkrQPvP14PwRIxZ45bPgyyxYwH+arToz/ENzatgjfLncZALohKqxsuvtrHwEoLeTvU7Ju4OOjJKaxMh3mHjtDRcvZQ3/v85ujJyqZAXeh/munrcdNITgbg7i9BphKv3Mbzuormy/1d/+A9Jz+m0YvpEhTb8vp2oHq3UcJB8JX8BxPXxFlmsHKyv6QKb7XDxF7K2IyRJ0EZL/eIuqBo29Y7M9CodAWSrZZhTcHwE+zheIykIxOaWlu4QRRMNDB5LqXIcmFuQspkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pSlnFw5PGp5VrI5znVnt18TuUunwL5FC7Z7l+VWzRbc=; b=J5yDZdxpczM3TVGpbMiloCh/M2YjFS2a7iTSpCTHMyhpjuf5mTjrltl9PgLTJVN75h0xugX+bT8IER48CruXDbvWj4Ttl6ApLU0pQBeyu4YfDKwCFGeZeWKFWDWxGaVz/BoG6gghk6V2NLQlD4FWDyqsMkYrXTGIVryVRqMZihqQlUne/FSahGkNzgdfvMeVjKlQxn+C4/5tj1ltmuEzUr9sdKg7VUhQBEY2nMrKp6ExOUuw8kwPWFt6oVMyD/K0CnUN0UM88s6u42z2+mAtO5ZBOkaGK/gfbwGYNiBrGXcQNRFp8Fuy+lTpFB1T836kLgsIp3u91LRqtAtiMLVAZg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pSlnFw5PGp5VrI5znVnt18TuUunwL5FC7Z7l+VWzRbc=; b=JXbAODFk9PBfo3mGmJIA6B2WMww6qLzFVbyX870mfAUiFDRnawcZqNvhF6301KsUWdtVHJ/UUVwGi3wTpn2Q/pmwMNSiLR0HjmlL/hsUdpLMK7l2Ovm3uq5KVqxf5On5+5/m+wE5up+0nXHo0TC+NxZbAMGahAI//uDqXWd9+HI= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: CC: Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Anthony PERARD , "Juergen Gross" , "Daniel P . Smith" , Stewart Hildebrand , Jiqian Chen , Huang Rui Subject: [XEN PATCH v12 4/7] x86/domctl: Add hypercall to set the access of x86 gsi Date: Mon, 8 Jul 2024 19:41:21 +0800 Message-ID: <20240708114124.407797-5-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708114124.407797-1-Jiqian.Chen@amd.com> References: <20240708114124.407797-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00006001:EE_|SA0PR12MB7461:EE_ X-MS-Office365-Filtering-Correlation-Id: 793dcf44-a83e-4a90-5516-08dc9f42f96f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|82310400026|7416014|1800799024|36860700013; X-Microsoft-Antispam-Message-Info: pJ/cj7MV7QGrSXCQdXTvUI2+1k4rZ9HyUma+k7lNs1yeRQP/7Bjy6bsLxZgyqlAtj7QFkmOdT/KOXhvzEFygzG5vIWxb+Qhq0rC5BxouRoLl5l0RAnh38DfZQJhej/LUQxwm3q54KDD1tlzfp3wRE0EKwijbj7InP6HPlsEy1eC2mW1epZ5vD8NIduX9R5D4//qWrtTvCLkcZKPVur5VUB7Gio9+GfJ+0dGw4dUAAYPk1CF/u0++bg57OfCTR1O8a5+iXZttthn7y9Uk4oix1LWKDvzOisl9wvNDzskcz+xXSST/8vH8S6t+c8u4Ttprhjf72tXj+rftVo1LG6OYCprHp/GMKBLWk/pG6SC25rG68pGGAiKMNBUSfkwYaAYb5RPjTsiAEoKQ7JACsG45hoE0GqeIZDjfjFOcU3Ns46PnhF17fy0sp8i1nOnL2OMw6Ece69X56OK3f9vvFt/XTb67inkyQ0L0AgqpKNxviKMQe7+lvOGwy8faiShz1yMMIsX6lPIZuPIPDfeyYhm3FJUHsjMEkBTn2fPctLqBq+tLqU3bkh4+3Xo5Y9XkR80Lm0mYJHvY257/3096esM89ZR4vRF1Tl6pRmVVuuUT2yiAlk2hE5Gcs9gvfSFukrATqHYrkybmoXNhj5VsCWpU1K4iktBl7Ba/aCGDk7mlcpRoaxgrB48Hh06lyK3/CikMpE31LJ+eowP7GfJWSx3Kjkx1qHqupjiIP1RrHPIsJLr5G8ezGxY0p5oL8Q7wcCP88zudHKhU7nZ7DrjUCxCtiHiPjQ+AI8tfzLIY56P0jYehBiUOphkUFX4eq989H0oDnJ1A4SB60E/KKHtlxxoB4ewGi/FTkIShvENRCwrAuHlZ7XuAFn03SL0NUrCvUkvx1zHz0u29R6phDIjxwqmmUKWN22VYHS9AFcB2UsN670wKij/i40dcoGznyFRQoUGVnwIwsWKvgyemZ9JJv5ZVmDW96mGbS86y7SDEPT+9hLPdLH1CaqdT+pkARfN6TEgQKfSYEUQfH9wyULCz0Iz1b7O/IAOxd6AQ7HPcFxYE8ByT74HwGChveMlEj1dH/OB0sCG1O2nLhuKtuNGdu3EnEQUJfxeLKB7pWNS4CT54xK+I8l70S31mSIWFi5i7OPADOCMX9OtujASX93MFM/lWNG4RTjHVglZdlV9We7lhmREhCIxKhtEIFaj8cI6WjNFu/ILHgvvksMYQp/oOIdZppHlfd8tcj9XafSZsZgTph9lA4xItpQCgz800Gq4v5H/7luO4OJdjferE4JdXUgLIB449XBOfP51hSscvsH1uC5OZBFN1WgPA3i4xGaMO+SLl58/+PKK0xgTvvZ9nTKWYZTQsQRUZVBfo0kWIYr8XeghjUGkQ62p7bmH6g4Gsvvhe2Rsx/WJhp0rsbxqM3j2IpGPqYR6J6JZdzXbGfqwOisG+++8kpl9Vfx62roOiGpd+ X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(82310400026)(7416014)(1800799024)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2024 11:41:58.0623 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 793dcf44-a83e-4a90-5516-08dc9f42f96f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00006001.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB7461 Some type of domains don't have PIRQs, like PVH, it doesn't do PHYSDEVOP_map_pirq for each gsi. When passthrough a device to guest base on PVH dom0, callstack pci_add_dm_done->XEN_DOMCTL_irq_permission will fail at function domain_pirq_to_irq, because PVH has no mapping of gsi, pirq and irq on Xen side. What's more, current hypercall XEN_DOMCTL_irq_permission requires passing in pirq to set the access of irq, it is not suitable for dom0 that doesn't have PIRQs. So, add a new hypercall XEN_DOMCTL_gsi_permission to grant/deny the permission of irq(translate from x86 gsi) to dumU when dom0 has no PIRQs. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui Signed-off-by: Jiqian Chen Reviewed-by: Jan Beulich Reviewed-by: Stefano Stabellini --- CC: Daniel P . Smith Remaining comment @Daniel P . Smith: + ret = -EPERM; + if ( !irq_access_permitted(currd, irq) || + xsm_irq_permission(XSM_HOOK, d, irq, access_flag) ) + goto gsi_permission_out; Is it okay to issue the XSM check using the translated value, not the one that was originally passed into the hypercall? --- xen/arch/x86/domctl.c | 32 ++++++++++++++++++++++++++++++ xen/arch/x86/include/asm/io_apic.h | 2 ++ xen/arch/x86/io_apic.c | 17 ++++++++++++++++ xen/arch/x86/mpparse.c | 5 ++--- xen/include/public/domctl.h | 9 +++++++++ xen/xsm/flask/hooks.c | 1 + 6 files changed, 63 insertions(+), 3 deletions(-) diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index 9190e11faaa3..4e9e4c4cfed3 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -36,6 +36,7 @@ #include #include #include +#include static int update_domain_cpu_policy(struct domain *d, xen_domctl_cpu_policy_t *xdpc) @@ -237,6 +238,37 @@ long arch_do_domctl( break; } + case XEN_DOMCTL_gsi_permission: + { + int irq; + unsigned int gsi = domctl->u.gsi_permission.gsi; + uint8_t access_flag = domctl->u.gsi_permission.access_flag; + + /* Check all bits and pads are zero except lowest bit */ + ret = -EINVAL; + if ( access_flag & ( ~XEN_DOMCTL_GSI_PERMISSION_MASK ) ) + goto gsi_permission_out; + for ( i = 0; i < ARRAY_SIZE(domctl->u.gsi_permission.pad); ++i ) + if ( domctl->u.gsi_permission.pad[i] ) + goto gsi_permission_out; + + if ( gsi > highest_gsi() || (irq = gsi_2_irq(gsi)) <= 0 ) + goto gsi_permission_out; + + ret = -EPERM; + if ( !irq_access_permitted(currd, irq) || + xsm_irq_permission(XSM_HOOK, d, irq, access_flag) ) + goto gsi_permission_out; + + if ( access_flag ) + ret = irq_permit_access(d, irq); + else + ret = irq_deny_access(d, irq); + + gsi_permission_out: + break; + } + case XEN_DOMCTL_getpageframeinfo3: { unsigned int num = domctl->u.getpageframeinfo3.num; diff --git a/xen/arch/x86/include/asm/io_apic.h b/xen/arch/x86/include/asm/io_apic.h index 78268ea8f666..7e86d8337758 100644 --- a/xen/arch/x86/include/asm/io_apic.h +++ b/xen/arch/x86/include/asm/io_apic.h @@ -213,5 +213,7 @@ unsigned highest_gsi(void); int ioapic_guest_read( unsigned long physbase, unsigned int reg, u32 *pval); int ioapic_guest_write(unsigned long physbase, unsigned int reg, u32 val); +int mp_find_ioapic(int gsi); +int gsi_2_irq(int gsi); #endif diff --git a/xen/arch/x86/io_apic.c b/xen/arch/x86/io_apic.c index d2a313c4ac72..5968c8055671 100644 --- a/xen/arch/x86/io_apic.c +++ b/xen/arch/x86/io_apic.c @@ -955,6 +955,23 @@ static int pin_2_irq(int idx, int apic, int pin) return irq; } +int gsi_2_irq(int gsi) +{ + int ioapic, pin, irq; + + ioapic = mp_find_ioapic(gsi); + if ( ioapic < 0 ) + return -EINVAL; + + pin = gsi - io_apic_gsi_base(ioapic); + + irq = apic_pin_2_gsi_irq(ioapic, pin); + if ( irq <= 0 ) + return -EINVAL; + + return irq; +} + static inline int IO_APIC_irq_trigger(int irq) { int apic, idx, pin; diff --git a/xen/arch/x86/mpparse.c b/xen/arch/x86/mpparse.c index d8ccab2449c6..7786a3337760 100644 --- a/xen/arch/x86/mpparse.c +++ b/xen/arch/x86/mpparse.c @@ -841,8 +841,7 @@ static struct mp_ioapic_routing { } mp_ioapic_routing[MAX_IO_APICS]; -static int mp_find_ioapic ( - int gsi) +int mp_find_ioapic(int gsi) { unsigned int i; @@ -914,7 +913,7 @@ void __init mp_register_ioapic ( return; } -unsigned __init highest_gsi(void) +unsigned highest_gsi(void) { unsigned x, res = 0; for (x = 0; x < nr_ioapics; x++) diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index 2a49fe46ce25..877e35ab1376 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -464,6 +464,13 @@ struct xen_domctl_irq_permission { uint8_t pad[3]; }; +/* XEN_DOMCTL_gsi_permission */ +struct xen_domctl_gsi_permission { + uint32_t gsi; +#define XEN_DOMCTL_GSI_PERMISSION_MASK 1 + uint8_t access_flag; /* flag to specify enable/disable of x86 gsi access */ + uint8_t pad[3]; +}; /* XEN_DOMCTL_iomem_permission */ struct xen_domctl_iomem_permission { @@ -1306,6 +1313,7 @@ struct xen_domctl { #define XEN_DOMCTL_get_paging_mempool_size 85 #define XEN_DOMCTL_set_paging_mempool_size 86 #define XEN_DOMCTL_dt_overlay 87 +#define XEN_DOMCTL_gsi_permission 88 #define XEN_DOMCTL_gdbsx_guestmemio 1000 #define XEN_DOMCTL_gdbsx_pausevcpu 1001 #define XEN_DOMCTL_gdbsx_unpausevcpu 1002 @@ -1328,6 +1336,7 @@ struct xen_domctl { struct xen_domctl_setdomainhandle setdomainhandle; struct xen_domctl_setdebugging setdebugging; struct xen_domctl_irq_permission irq_permission; + struct xen_domctl_gsi_permission gsi_permission; struct xen_domctl_iomem_permission iomem_permission; struct xen_domctl_ioport_permission ioport_permission; struct xen_domctl_hypercall_init hypercall_init; diff --git a/xen/xsm/flask/hooks.c b/xen/xsm/flask/hooks.c index 5e88c71b8e22..a5b134c91101 100644 --- a/xen/xsm/flask/hooks.c +++ b/xen/xsm/flask/hooks.c @@ -685,6 +685,7 @@ static int cf_check flask_domctl(struct domain *d, int cmd) case XEN_DOMCTL_shadow_op: case XEN_DOMCTL_ioport_permission: case XEN_DOMCTL_ioport_mapping: + case XEN_DOMCTL_gsi_permission: #endif #ifdef CONFIG_HAS_PASSTHROUGH /* From patchwork Mon Jul 8 11:41:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13726457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E964FC3DA47 for ; Mon, 8 Jul 2024 11:42:17 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.755306.1163660 (Exim 4.92) (envelope-from ) id 1sQmkz-0005Qk-3f; Mon, 08 Jul 2024 11:42:09 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 755306.1163660; Mon, 08 Jul 2024 11:42:09 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sQmkz-0005Qb-0J; Mon, 08 Jul 2024 11:42:09 +0000 Received: by outflank-mailman (input) for mailman id 755306; Mon, 08 Jul 2024 11:42:07 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sQmkx-00042n-R1 for xen-devel@lists.xenproject.org; Mon, 08 Jul 2024 11:42:07 +0000 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2060f.outbound.protection.outlook.com [2a01:111:f403:2009::60f]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 199b86e5-3d1f-11ef-8776-851b0ebba9a2; Mon, 08 Jul 2024 13:42:06 +0200 (CEST) Received: from BN0PR04CA0012.namprd04.prod.outlook.com (2603:10b6:408:ee::17) by IA1PR12MB6625.namprd12.prod.outlook.com (2603:10b6:208:3a3::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.35; Mon, 8 Jul 2024 11:42:01 +0000 Received: from BN1PEPF00006002.namprd05.prod.outlook.com (2603:10b6:408:ee:cafe::24) by BN0PR04CA0012.outlook.office365.com (2603:10b6:408:ee::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.36 via Frontend Transport; Mon, 8 Jul 2024 11:42:01 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by BN1PEPF00006002.mail.protection.outlook.com (10.167.243.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7762.17 via Frontend Transport; Mon, 8 Jul 2024 11:42:01 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 8 Jul 2024 06:41:57 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 199b86e5-3d1f-11ef-8776-851b0ebba9a2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=c4B1XBl+lTO8+doz6ICL12nryJCgEppYPLFLiMCdUcYg+i6rcH4gPLjs0bJZp83GcIywOz4QhaPS6kEtVEQ4QkNJ89Tr//6JfcaOv4ixJZekD+5YUfZo8g8qJERDgJxXQt1yTIr3wr6Jy0ipXH/WqTOkF476WBN6Rs2WGN7SAsp6k94QBWg+KQ286CXZOJCmocodO+2dbc9mv4lZ2aGvkUvtUMEN3UVv4cbuzroYo05Ga1PjWc2EzcE9VXkOkO/KOslEwDmxtq/9nVGJwgT/NSjZbPhMN29XkistEsoGnj3zxOQfSQKztwMb9JqlCzG+m37T+Z83PTF0yxcoVdsg+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=iSUMECa+KJwIDIvEDAv8tiYigyciwYLZAvCuoQwKiN4=; b=e3eWwaYEPAGOwwSvTXPPDca7MrMSEmAC9Qh7nsxHGH8Rgdq2EoBNRTeSGdn944OBrI5wwC6NIpmoO5OfI9VqFTTQGtDzzJaWHKYfHSqr7pCGSY7SmWAx9FjwM4LGZ8JlH5/n29XTA/C8k5JcZMryonTJtyI5XYKEBTAVYzJBWRU1pAjvYacLtdQl79KuZ+U7BPF6eB3XffkU+mzvQQvtsepMWSO4+omIRB81aFCrFy+TRxQS/NrbKlOzC1v/c1zsdqfxQQ/YwGxTi4uy23GJhkdLYkDm/r3Fo9+eyoVxpoYsImKba9j/cG30JNuJTZe0ndNM6MOgxNbqsjW6AthNvw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iSUMECa+KJwIDIvEDAv8tiYigyciwYLZAvCuoQwKiN4=; b=3jAkEQvqG3tkxsEC210/MedWfZMJCadvdalzPK8zjNk5CdJFZJNWbU+24ajIfOnSso3fCQmVOE1sUBiKBqdUnq8rrl2rplYeMr9BresgH1LSATpdUn/mo5EzQFmpqfJyNarfzddEB+ZQcWcPud4ZzRMXgQ2WV5efmQkWiXCAHUk= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: CC: Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Anthony PERARD , "Juergen Gross" , "Daniel P . Smith" , Stewart Hildebrand , Jiqian Chen , Huang Rui Subject: [XEN PATCH v12 5/7] tools/libxc: Allow gsi be mapped into a free pirq Date: Mon, 8 Jul 2024 19:41:22 +0800 Message-ID: <20240708114124.407797-6-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708114124.407797-1-Jiqian.Chen@amd.com> References: <20240708114124.407797-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00006002:EE_|IA1PR12MB6625:EE_ X-MS-Office365-Filtering-Correlation-Id: 6f27321d-6750-4afe-4137-08dc9f42fb8d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|7416014|82310400026|1800799024|36860700013; X-Microsoft-Antispam-Message-Info: NTuSCTQpmqvM2TR0b9BM1YozPKeQPGIBcJfs1iaTVytLksCTo35odu5b1Q2/tM2Nn1hRjhWy9mUJdyr5VVAzfjmDEyAwYPeL4dcHDsdRszVN5d3CJyilOOaWLMrTMtps9yNl4dGS7EIxb8me2h3LePVucyJP5ja8Wn8YRJnvnJT/VcVJ6qMKZo/P3YqWTzfAcTh4w6kj86PU5ikFI7aeUjes6DyUbo+VW52CoBoQL1CbatDL5HjJxDG1fseiVjnVpQyFqkiIqbez1EztoVTYkJ/A+N9T8IVHOQ9Uz5w8Dj0x1W3S97TqPXDorjEbhdbzSFBmQD9f9VglimTnweZLiEyeqtI1fcxQm1kDje2dVmMhWHxYZ0qy0WYBrNh/VFSuKhUcJknxgd1iytZTxXPdHOFE1zwQVcIHzVuetWUr9QW7JfErTA4xOU+85BwPtX1B/AnU3YvUljCL8/Ns9t2mEyDh0fkrQNRNqGthoCaXvMcHkP6218Xa2YqTRw8g0Reo6d0fi9rNsFESeKwtwfyUPl4p2aqcoZ9SdD0ru+xf47BTl4lKcYbkijuOZ7SmIoTm3ojtv7G8aMmbxr2jMrS4r72TzsraY01iTYY2Dm67Hybx58S8WIegW0Bk7pVJZ+S236O3eCdadXdqk2ISuNmWDffR1TtWlmPNUxueNtbhmWN2FkAD0HuG/k6edI8+t26Jw00IJuLnyPC5Aluk8DjrdZ+7NrRk7/JyZbxz0YWcfwQ0A1BRFz2QDx45cxUopzZLAwO+BMtKf7GuAaNIG30qcLZ9bB0Ic4PooN7JnMbIQmKUVz4ROnmJLHVYkwWP25dKcZlpLZtlQVv99vs+P30hzYcA5VfTcEciKFOC+NnbqxevlpH5Jrf6V3PLQm1H1sUXUL5Wk/vQgsisw09zoM7Y5edRj9dfqgWvVkRmKqoL6T6Gpitd9GVjxT/v1uUb5LFGLjsEPJjOvGO0e9m2NQ/vc2oVsCnNFbIXfcy6LnLXbU4260HOoKiAsyMPVwA3I8Mxrj8TvAqtillDYUIefCx0c0E7tcQ7GnbJn5FK2DuBSCCILnmSjYbMe+1tTiIGYLwo8j6QmHFyaSb/hVSqInVVXrpNRSv5SZkLDLolLBQBszqKJykISyl5XCIGqnQ+P1jPW50S9A42GHFStgYVFV4hblg+u9Hz/L7HCzUN7o5ouYmygZ6iGNPxYLCvNUEWrx4kr6RgDmC5Y+lccMRb3xipN8tFgysQJAlImD0ZoPNEnL8g/2e0pziL7Grc8XD13LF/RzokgjTYwMydAbPamAbOliKpeqAZB8UXOrUcZHQJI+lZqhWYcPpNpvj6UBWYUNn1nnLAEUxe4qnO7DiZ1vs2g6gN1YQoT76Q3/twBcZgMD5Ikz13SihDBu/0SI+hJXTqkGD3N4qpAsYwTPBK4nnyUcvpAH+XaUfv4atB7r3uAu0rzjwcaLYqqF94RSxo5bcV X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(376014)(7416014)(82310400026)(1800799024)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2024 11:42:01.6139 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6f27321d-6750-4afe-4137-08dc9f42fb8d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00006002.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6625 Hypercall PHYSDEVOP_map_pirq support to map a gsi into a specific pirq or a free pirq, it depends on the parameter pirq(>0 or <0). But in current xc_physdev_map_pirq, it set *pirq=index when parameter pirq is <0, it causes to force all cases to be mapped to a specific pirq. That has some problems, one is caller can't get a free pirq value, another is that once the pecific pirq was already mapped to other gsi, then it will fail. So, change xc_physdev_map_pirq to allow to pass negative parameter in and then get a free pirq. There are four caller of xc_physdev_map_pirq in original codes, so clarify the affect below(just need to clarify the pirq<0 case): First, pci_add_dm_done->xc_physdev_map_pirq, it pass irq to pirq parameter, if pirq<0 means irq<0, then it will fail at check "index < 0" in allocate_and_map_gsi_pirq and get EINVAL, logic is the same as original code. Second, domcreate_launch_dm->libxl__arch_domain_map_irq-> xc_physdev_map_pirq, the passed pirq is always >=0, so no affect. Third, pyxc_physdev_map_pirq->xc_physdev_map_pirq, not sure, so add the check logic into pyxc_physdev_map_pirq to keep the same behavior. Fourth, xen_pt_realize->xc_physdev_map_pirq, it wants to allocate a pirq for gsi, but it isn't necessary to get pirq whose value is equal with the value of gsi. After this patch, it will get a free pirq, and it also can work. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui Signed-off-by: Jiqian Chen --- tools/libs/ctrl/xc_physdev.c | 2 +- tools/python/xen/lowlevel/xc/xc.c | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/libs/ctrl/xc_physdev.c b/tools/libs/ctrl/xc_physdev.c index 460a8e779ce8..e9fcd755fa62 100644 --- a/tools/libs/ctrl/xc_physdev.c +++ b/tools/libs/ctrl/xc_physdev.c @@ -50,7 +50,7 @@ int xc_physdev_map_pirq(xc_interface *xch, map.domid = domid; map.type = MAP_PIRQ_TYPE_GSI; map.index = index; - map.pirq = *pirq < 0 ? index : *pirq; + map.pirq = *pirq; rc = do_physdev_op(xch, PHYSDEVOP_map_pirq, &map, sizeof(map)); diff --git a/tools/python/xen/lowlevel/xc/xc.c b/tools/python/xen/lowlevel/xc/xc.c index 9feb12ae2b16..f8c9db7115ee 100644 --- a/tools/python/xen/lowlevel/xc/xc.c +++ b/tools/python/xen/lowlevel/xc/xc.c @@ -774,6 +774,8 @@ static PyObject *pyxc_physdev_map_pirq(PyObject *self, if ( !PyArg_ParseTupleAndKeywords(args, kwds, "iii", kwd_list, &dom, &index, &pirq) ) return NULL; + if ( pirq < 0 ) + pirq = index; ret = xc_physdev_map_pirq(xc->xc_handle, dom, index, &pirq); if ( ret != 0 ) return pyxc_error_to_exception(xc->xc_handle); From patchwork Mon Jul 8 11:41:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13726459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5804EC3271E for ; Mon, 8 Jul 2024 11:42:21 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.755308.1163669 (Exim 4.92) (envelope-from ) id 1sQml3-0005pD-C6; Mon, 08 Jul 2024 11:42:13 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 755308.1163669; Mon, 08 Jul 2024 11:42:13 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sQml3-0005p1-94; Mon, 08 Jul 2024 11:42:13 +0000 Received: by outflank-mailman (input) for mailman id 755308; Mon, 08 Jul 2024 11:42:12 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sQml1-00042n-V6 for xen-devel@lists.xenproject.org; Mon, 08 Jul 2024 11:42:11 +0000 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2062d.outbound.protection.outlook.com [2a01:111:f400:7ea9::62d]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 1bb61ed9-3d1f-11ef-8776-851b0ebba9a2; Mon, 08 Jul 2024 13:42:10 +0200 (CEST) Received: from BN0PR04CA0011.namprd04.prod.outlook.com (2603:10b6:408:ee::16) by LV8PR12MB9407.namprd12.prod.outlook.com (2603:10b6:408:1f9::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.35; Mon, 8 Jul 2024 11:42:05 +0000 Received: from BN1PEPF00006002.namprd05.prod.outlook.com (2603:10b6:408:ee:cafe::f0) by BN0PR04CA0011.outlook.office365.com (2603:10b6:408:ee::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.36 via Frontend Transport; Mon, 8 Jul 2024 11:42:05 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by BN1PEPF00006002.mail.protection.outlook.com (10.167.243.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7762.17 via Frontend Transport; Mon, 8 Jul 2024 11:42:05 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 8 Jul 2024 06:42:01 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1bb61ed9-3d1f-11ef-8776-851b0ebba9a2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DclRDhOpTFS7Fdf8VKHRCvLd+3s5L4NbP9FI3rypdDdxc/Dx3GNXuh6kifA/sJC8m5nZsJ4zkl7UKVuMP7j/f5hs21OyX7iIkDPjPFiaUhjmKI5dspMKTDXFS6zu20aCW1iBFIw7EwcNEGS6X8SfL2aqN/mwGad1rNo/kbVeHzpOLxlj/9BXa7PhSeMM21czC2tU+PB/Xp21ShJ1WUTc61Ke0TLqLkkEPgKK7atsfxIVr94xdsVPSDlvhRoWnr1d84PR1jaS3pohgGpo6DHJqZ73QWfn6sYSeAJWUKZNa9A9j90aQ9iKGqSskgysHTE+BxHKqwFDXnnrkkpX8x90ng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yo1KApXtMVEKsU/lZ7Ft3m3DQ4K3qbtV00FVwm1Ugf8=; b=EzX5az+ptmd3tsFByfzROAPaMztZYIKdj3HARl/p0dqfN7abINo0PjxQLzoruLrkrpxoKUSh4GPmjAPINja7rRjiXsZuQt/d24feKKCRRvF78wzGjsAuDdqLQJte/HpZ+caQKUtnVloTi2tjW4GzoNNwFoNA7zCCRoX1zFN5I9lME09+hyubu4GOk79skXdN3YOX+37UvboaZ/ZYLGr03sgBUAS2VEr/jSokTD+6KKj1Wxn/nuVz4iTqHy2/f9/s63Xar8OBhlHe/f/FpVPIwW06IZSTTfJLFGTsvTDcQWT0gm1aKBBx7FQjaCk+xV5Q0Fv8cJycwwj3AejdvFUATQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yo1KApXtMVEKsU/lZ7Ft3m3DQ4K3qbtV00FVwm1Ugf8=; b=gSq6+Yo7xO/kLTVb6S8jaYTQQ9cpsJ/6/HtjiLH8LUZ3gYumrtJgXoqizZvA9OaNoAPPCDfPh5kqvRoAbngn14mGVzpUNcNUu0cCGRBmfOu/NG1J6LIoWRcLUsXKADexfA957ZZSnQc55KFsb/zeJ291lcI0vE4B1ZaHyYeHwC0= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: CC: Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Anthony PERARD , "Juergen Gross" , "Daniel P . Smith" , Stewart Hildebrand , Jiqian Chen , Huang Rui Subject: [RFC XEN PATCH v12 6/7] tools: Add new function to get gsi from dev Date: Mon, 8 Jul 2024 19:41:23 +0800 Message-ID: <20240708114124.407797-7-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708114124.407797-1-Jiqian.Chen@amd.com> References: <20240708114124.407797-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00006002:EE_|LV8PR12MB9407:EE_ X-MS-Office365-Filtering-Correlation-Id: 2b4bf36d-18ca-480a-02ad-08dc9f42fd95 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|1800799024|36860700013|7416014|376014; X-Microsoft-Antispam-Message-Info: yMBnDU4A0ylO69dXzDF/3RlmlfgprKt1LyDFY5MoWOuxP28uezYC0m0k5en8wMyEXVTYT3PLjnixmRLp/1otf9zaAQZdEpByNr2T68uMBQN00X8XBsIrRHXjWDWWOXaTPJ6WT7O8JLuKTOgVgIkmnXvYnwVfnNqANxQtAua8qRrw78o3NTsMLpm9okBOBsddJHL+tHeuFVsBWOsn6pQllTc1iMvuZqg4ObYeSVC1sAXQzlkKZnGaxyUwaiJwR6z4dbDqKOu5Bcdu/GemqCz5t/G9pOYZbzqRORQa0MBbWd4wittgECKUzLK1mOzJ8zHWI/4KsftLK9Hs+IR4kcdLKXCHsNx9c9hdSJDouPYv6zsBzYKxKwNsOeIyPr1lNUITPfusjrg6J1aZrjwmsucGZCXfZBq7amPFtdWj9bNuu7WXbKK9F/SthprtBy78KmayxqClr+toJ84FOigeixeUolKmfX+dI2jFZDTW4eIbx5yVF7zlMoOTrFewO1LdVodOXVHS4WObwW0p8+KtKnkMWTlioCwrWNiUVP74l1VMgQjON3pbdcCLfZiKv2HNhkRNBIdu3M6BldAEVQOs/IGFlA2zCCwubvcv+iTlSqY44iSUXEb8PYWSAY0KnZU84nkktdvUgkG0XUZw3ozoQoVKA0mr+QfOPjynfXM6Hqecl6s4YQB9V12mehmRFtr5daQF+IY6L2qPvXoLZ5wyBZqRmC6Sq8Istz2JKQVsl/W7QIgzY/Y1llEWRxXoTOEUIWJJL47FmJYo/dcZrGAhTmuP56JRNX85thLHoMiZTKT26QM5M72oV2OMLZ7+3+Xz09+r36lTHhmULUn6jynU0Ki+8p7HAuSknAPWsI8Yo+aUbdDYUzrk5AyOe839sS4jkMTYBAHaiD6suVlZsifoNT8moofWv6ebMFkbIubn+ot+SzAYFFACSTKwtF3NMM2PBuVhSnfuOQTCpxqOawJXUWh8iHb012dthRo8E2L08t1zcGS+hQE9PuQVXzpctN5kCBmoYTyLhfe3RkjUIE1bMdq748l1AuINqZ8TtSetxgndkgtIguUPeArKBjlhpMyz/lmG0uOPrF7GRa5Og6718EPLquGghZgtwUfIfkz0BF0tFz8MsgmWAASqnI7twFfNa8igiTkk9KTsCkT84GvydLz5cU6JXzQqeF9x3s/fMyKlyLw7W5esVnJTbj4Yvvupljtjq9IPy/6UoOQn3YQmIq5e+plb3weQwU03mVm1Q1vOs6uKgpmlKKWyifzlN+umbPZgRKNkwyQRtzwuDn+qmmHI+Mfb14g+rsmN+Z0KPeQ/2g/uszBjpVmuOKFgWuyO54HE/Fm19OcY5HP00j1/iDTIdvK5b2eLVUdOTxyMWMPvdrFGfj3MMO26imRahY3l/Jm+A97U19z1B6qRW/EVLq3knA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(1800799024)(36860700013)(7416014)(376014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2024 11:42:05.0201 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2b4bf36d-18ca-480a-02ad-08dc9f42fd95 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00006002.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR12MB9407 When passthrough a device to domU, QEMU and xl tools use its gsi number to do pirq mapping, see QEMU code xen_pt_realize->xc_physdev_map_pirq, and xl code pci_add_dm_done->xc_physdev_map_pirq, but the gsi number is got from file /sys/bus/pci/devices//irq, that is wrong, because irq is not equal with gsi, they are in different spaces, so pirq mapping fails. And in current codes, there is no method to get gsi for userspace. For above purpose, add new function to get gsi, and the corresponding ioctl is implemented on linux kernel side. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui Signed-off-by: Chen Jiqian --- RFC: it needs to wait for the corresponding third patch on linux kernel side to be merged. https://lore.kernel.org/xen-devel/20240607075109.126277-4-Jiqian.Chen@amd.com/ This patch must be merged after the patch on linux kernel side CC: Anthony PERARD Remaining comment @Anthony PERARD: Do I need to make " opening of /dev/xen/privcmd " as a single function, then use it in this patch and other libraries? --- tools/include/xen-sys/Linux/privcmd.h | 7 ++++++ tools/include/xenctrl.h | 2 ++ tools/libs/ctrl/xc_physdev.c | 35 +++++++++++++++++++++++++++ 3 files changed, 44 insertions(+) diff --git a/tools/include/xen-sys/Linux/privcmd.h b/tools/include/xen-sys/Linux/privcmd.h index bc60e8fd55eb..4cf719102116 100644 --- a/tools/include/xen-sys/Linux/privcmd.h +++ b/tools/include/xen-sys/Linux/privcmd.h @@ -95,6 +95,11 @@ typedef struct privcmd_mmap_resource { __u64 addr; } privcmd_mmap_resource_t; +typedef struct privcmd_gsi_from_pcidev { + __u32 sbdf; + __u32 gsi; +} privcmd_gsi_from_pcidev_t; + /* * @cmd: IOCTL_PRIVCMD_HYPERCALL * @arg: &privcmd_hypercall_t @@ -114,6 +119,8 @@ typedef struct privcmd_mmap_resource { _IOC(_IOC_NONE, 'P', 6, sizeof(domid_t)) #define IOCTL_PRIVCMD_MMAP_RESOURCE \ _IOC(_IOC_NONE, 'P', 7, sizeof(privcmd_mmap_resource_t)) +#define IOCTL_PRIVCMD_GSI_FROM_PCIDEV \ + _IOC(_IOC_NONE, 'P', 10, sizeof(privcmd_gsi_from_pcidev_t)) #define IOCTL_PRIVCMD_UNIMPLEMENTED \ _IOC(_IOC_NONE, 'P', 0xFF, 0) diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h index 9ceca0cffc2f..3720e22b399a 100644 --- a/tools/include/xenctrl.h +++ b/tools/include/xenctrl.h @@ -1641,6 +1641,8 @@ int xc_physdev_unmap_pirq(xc_interface *xch, uint32_t domid, int pirq); +int xc_physdev_gsi_from_pcidev(xc_interface *xch, uint32_t sbdf); + /* * LOGGING AND ERROR REPORTING */ diff --git a/tools/libs/ctrl/xc_physdev.c b/tools/libs/ctrl/xc_physdev.c index e9fcd755fa62..54edb0f3c0dc 100644 --- a/tools/libs/ctrl/xc_physdev.c +++ b/tools/libs/ctrl/xc_physdev.c @@ -111,3 +111,38 @@ int xc_physdev_unmap_pirq(xc_interface *xch, return rc; } +int xc_physdev_gsi_from_pcidev(xc_interface *xch, uint32_t sbdf) +{ + int rc = -1; + +#if defined(__linux__) + int fd; + privcmd_gsi_from_pcidev_t dev_gsi = { + .sbdf = sbdf, + .gsi = 0, + }; + + fd = open("/dev/xen/privcmd", O_RDWR); + + if (fd < 0 && (errno == ENOENT || errno == ENXIO || errno == ENODEV)) { + /* Fallback to /proc/xen/privcmd */ + fd = open("/proc/xen/privcmd", O_RDWR); + } + + if (fd < 0) { + PERROR("Could not obtain handle on privileged command interface"); + return rc; + } + + rc = ioctl(fd, IOCTL_PRIVCMD_GSI_FROM_PCIDEV, &dev_gsi); + close(fd); + + if (rc) { + PERROR("Failed to get gsi from dev"); + } else { + rc = dev_gsi.gsi; + } +#endif + + return rc; +} From patchwork Mon Jul 8 11:41:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Chen, Jiqian" X-Patchwork-Id: 13726460 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D32BBC3271E for ; Mon, 8 Jul 2024 11:42:27 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.755309.1163679 (Exim 4.92) (envelope-from ) id 1sQml7-0006IC-Nd; Mon, 08 Jul 2024 11:42:17 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 755309.1163679; Mon, 08 Jul 2024 11:42:17 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sQml7-0006Hz-Ik; Mon, 08 Jul 2024 11:42:17 +0000 Received: by outflank-mailman (input) for mailman id 755309; Mon, 08 Jul 2024 11:42:15 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sQml5-00042n-Qs for xen-devel@lists.xenproject.org; Mon, 08 Jul 2024 11:42:15 +0000 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on20601.outbound.protection.outlook.com [2a01:111:f403:2416::601]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 1dd19c7e-3d1f-11ef-8776-851b0ebba9a2; Mon, 08 Jul 2024 13:42:14 +0200 (CEST) Received: from MN2PR01CA0023.prod.exchangelabs.com (2603:10b6:208:10c::36) by IA1PR12MB8312.namprd12.prod.outlook.com (2603:10b6:208:3fc::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.35; Mon, 8 Jul 2024 11:42:09 +0000 Received: from BL02EPF0001A0FC.namprd03.prod.outlook.com (2603:10b6:208:10c:cafe::54) by MN2PR01CA0023.outlook.office365.com (2603:10b6:208:10c::36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.34 via Frontend Transport; Mon, 8 Jul 2024 11:42:09 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by BL02EPF0001A0FC.mail.protection.outlook.com (10.167.242.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7762.17 via Frontend Transport; Mon, 8 Jul 2024 11:42:08 +0000 Received: from cjq-desktop.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 8 Jul 2024 06:42:04 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1dd19c7e-3d1f-11ef-8776-851b0ebba9a2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lAJgKtTI96aLuCiwJyGre4Go/EuKZjNmVDxQyb4i8MaSIkIbODv8sM5+3zmho/umUd7wj/59QPxYw40qk1fLNLvL3JhC53CEG/W/kToEScBpYRHmvr0WMlci0ZfA17t4lsjmFPp2C0iS39soKl8X8p5NjoZJ4OK6JVSHyazrK5kuQqhOVX44Lik+DXSKC3kDQ7+HJ1oDyRhVJto6vlqTpms4eV+XghvY6MUv8LsBtB0QpmTWDV1BSAUfwz0DmMPHoJnV6nA/ofLtZJLLyo6inANK48iIvjb3WubF5tESBSP1sx5lHNbbswjJAaTH98njACY4CttsSr7jYCwC73o7mg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SJ211g/xQ9EozfgjyUM5ps8F7M097mD2TeBPX5zf5hY=; b=EYrQCx/FEjF1iWEoWCSGksyoHCEwtIlxwY8eF6dtZmkiH7VqDkj+O0iESxnKxxHvYqaBVDI+i7KDBhs9YzbAVfc+fABykwrO4gXzUDMRJwuYobkV/4YjAaC6FgmKoo4MGgrC2a8jaoBlrDm0lTt/5+pt7QXBfVSQmWge5Qsozc6sdv5i4WxGHguhxtFTuiuNFhEld09I4O4xC2dzwgEnFzy50Q55JsaQItkd81O+IW3tDEGZJ2+hiQAss2At8gEDiBV+R//MZXxogjHRHmXkuPcBCwvu6+A51o/KuixnRPQVaaxNuwb6C4qArm9SDX06flPsCKYo+APDlTd4aXm6xg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SJ211g/xQ9EozfgjyUM5ps8F7M097mD2TeBPX5zf5hY=; b=ERJ/EVDrwDhQOjZb7sfEHRoyVsmMn9AWTj/cLbdzWoY/kcOWs7uq/WAdylZ81qusdLYh+zEONoLD8olpj6kpWXhHuGrxF5lSJNjVmp6uBPKYWqGRaXJ/bk2BKrwiOiLis3ua+vCVEIM6FIcdPLqwV0O9+tM6qUTe0tOCU51m1lg= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Jiqian Chen To: CC: Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini , Anthony PERARD , "Juergen Gross" , "Daniel P . Smith" , Stewart Hildebrand , Jiqian Chen , Huang Rui Subject: [RFC XEN PATCH v12 7/7] tools: Add new function to do PIRQ (un)map on PVH dom0 Date: Mon, 8 Jul 2024 19:41:24 +0800 Message-ID: <20240708114124.407797-8-Jiqian.Chen@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240708114124.407797-1-Jiqian.Chen@amd.com> References: <20240708114124.407797-1-Jiqian.Chen@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0001A0FC:EE_|IA1PR12MB8312:EE_ X-MS-Office365-Filtering-Correlation-Id: 78a51ddc-a332-4148-7099-08dc9f42ffb5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|7416014|1800799024|36860700013; X-Microsoft-Antispam-Message-Info: 2p8uK/ppXQQw7o4Np4bHG2L5tLRHd9tYH3t4BDnshFKx9knGw2EBJGpFvyyGaU1zZbjx4CRsfuP1QuvnSKQo9r2ojBrqfK0c1Dr5XbtjzA4+phg/pYTuhqzeL/sJ2fyhPMO9XY5BPhu0MRhd5bnP3lOG2XMNMYFtWPqIE29Z/2cCeLyeLHzj5toetIEMIy1l+AETFQonDka9TGuL/xCQbUqzdXar1gqzdGgFzeHHfTgidkbkohV7egWRFH1vTALGiU7H1baSGflwzmJpb/oPA5w6Jppin1208o1hAK5HCzMwWsStp+mxWPV2hyt+C4uVV5u9UzYI7dAE8HbQ70+IZWybzgb0B42gbjdE6hWdG2BQXVYsnTDucHZXr/TxooMLzHd3BWMOeHAXXq9ctgm0SOVzU20BoydNi1IoM0G6tS79bT1ghHSycsxWTvnUowp2qAYcZxyXH5CtPMd35c+FXAdoHIm4UofX9PACEw+66hGX+ZUaLNAQFoefxgCMRN8aGlXQJPpazhMRWTtT9zxKrajjRtfDCfgNBWge85w7abS8aiCppE+Aseze8lXrITQpfbDZidHI0hjqlYvkwwWUhPwxtXcVCqDUKTnFb0FMPD1/C/JMtHJeS/VuiJy3beol+iubU/W4tWU+FNpq0QXy2RQJLs9dRHvT2HaQaRVof9UnVGS25+SJUmUgxtOxqVuP5A2Mbyh+yQfFyOrlVCLadakqmZYk2FX9a7VSblOSULWuQamS1aBE3LzYquhq2df28mk7SLuWWt5rzH14E4Xj4bVmb9AIlz3siTgwqQONt4njtBuDyYpT5eGnNxOMj2wziGbeB05MX5zeWbSkzD3CMp/QE2plnurRvdg+SJpZq/1ydx1eDfTkX6ix4ewQn25jdyfFS7QdwnwweB8yKDtJHSRyUmfG2/DRUZmUQYy/iO42bTeu5LPWuYxK7ieCnUgbPo2hYTxQXtfikgkx7U2IUMNr6YMenXHYENSyLE8rbGxyRdUe+NNBsOy1mxeSpx4GyqXyNqxcnLW6vVdj9PBoOYKa3aPsy2OpSKy4BTzkduW/LlpJoDiWyS/elGm+gFFzVo/OKHr5WlPSz1oNpfpz+UQ7ioZzyUnU7cdpud+FmFKL/LXpIGEAIxXGHTzRkBJat5vHFdjKPo9Lrk7OLOYENGbpRIwuMqeA23NmCPrHbffdu7CrMijfN/4SjG72UYStuU4/JFHbF5NsAKD25UYFbdWqpYY0wDczEef5K6Fyu/Z61A3dr+OGOagruFPdnKyZtS1zR2fPmMfIdxJ5Ttc8X5f/OI2SLBKl+UbYEe8J59C31K1eEm16AaTkz371kAESWZPXiE3/FshdLVloYcnxb4b9qNsR1jHKn2JfiyerJgzKIDwlewrUiiumlvSe1mpX X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(376014)(7416014)(1800799024)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2024 11:42:08.5881 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 78a51ddc-a332-4148-7099-08dc9f42ffb5 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0001A0FC.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8312 When dom0 is PVH, and passthrough a device to dumU, xl will use the gsi number of device to do a pirq mapping, see pci_add_dm_done->xc_physdev_map_pirq, but the gsi number is got from file /sys/bus/pci/devices//irq, that confuses irq and gsi, they are in different space and are not equal, so it will fail when mapping. To solve this issue, use xc_physdev_gsi_from_dev to get the real gsi and then to map pirq. Besides, PVH dom doesn't have PIRQ flag, it doesn't do PHYSDEVOP_map_pirq for each gsi. So grant function callstack pci_add_dm_done->XEN_DOMCTL_irq_permission will fail at function domain_pirq_to_irq. And old hypercall XEN_DOMCTL_irq_permission requires passing in pirq, it is not suitable for dom0 that doesn't have PIRQs to grant irq permission. To solve this issue, use the new hypercall XEN_DOMCTL_gsi_permission to grant the permission of irq( translate from gsi) to dumU when dom0 has no PIRQs. Signed-off-by: Jiqian Chen Signed-off-by: Huang Rui Signed-off-by: Chen Jiqian --- RFC: it needs to wait for the corresponding third patch on linux kernel side to be merged. https://lore.kernel.org/xen-devel/20240607075109.126277-4-Jiqian.Chen@amd.com/ This patch must be merged after the patch on linux kernel side --- tools/include/xenctrl.h | 5 ++ tools/libs/ctrl/xc_domain.c | 15 +++++ tools/libs/light/libxl_arch.h | 4 ++ tools/libs/light/libxl_arm.c | 10 +++ tools/libs/light/libxl_pci.c | 17 ++++++ tools/libs/light/libxl_x86.c | 111 ++++++++++++++++++++++++++++++++++ 6 files changed, 162 insertions(+) diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h index 3720e22b399a..9ff5f1810cf8 100644 --- a/tools/include/xenctrl.h +++ b/tools/include/xenctrl.h @@ -1382,6 +1382,11 @@ int xc_domain_irq_permission(xc_interface *xch, uint32_t pirq, bool allow_access); +int xc_domain_gsi_permission(xc_interface *xch, + uint32_t domid, + uint32_t gsi, + uint8_t access_flag); + int xc_domain_iomem_permission(xc_interface *xch, uint32_t domid, unsigned long first_mfn, diff --git a/tools/libs/ctrl/xc_domain.c b/tools/libs/ctrl/xc_domain.c index f2d9d14b4d9f..4c89f07e4d6e 100644 --- a/tools/libs/ctrl/xc_domain.c +++ b/tools/libs/ctrl/xc_domain.c @@ -1394,6 +1394,21 @@ int xc_domain_irq_permission(xc_interface *xch, return do_domctl(xch, &domctl); } +int xc_domain_gsi_permission(xc_interface *xch, + uint32_t domid, + uint32_t gsi, + uint8_t access_flag) +{ + struct xen_domctl domctl = { + .cmd = XEN_DOMCTL_gsi_permission, + .domain = domid, + .u.gsi_permission.gsi = gsi, + .u.gsi_permission.access_flag = access_flag, + }; + + return do_domctl(xch, &domctl); +} + int xc_domain_iomem_permission(xc_interface *xch, uint32_t domid, unsigned long first_mfn, diff --git a/tools/libs/light/libxl_arch.h b/tools/libs/light/libxl_arch.h index f88f11d6de1d..11b736067951 100644 --- a/tools/libs/light/libxl_arch.h +++ b/tools/libs/light/libxl_arch.h @@ -91,6 +91,10 @@ void libxl__arch_update_domain_config(libxl__gc *gc, libxl_domain_config *dst, const libxl_domain_config *src); +_hidden +int libxl__arch_hvm_map_gsi(libxl__gc *gc, uint32_t sbdf, uint32_t domid); +_hidden +int libxl__arch_hvm_unmap_gsi(libxl__gc *gc, uint32_t sbdf, uint32_t domid); #if defined(__i386__) || defined(__x86_64__) #define LAPIC_BASE_ADDRESS 0xfee00000 diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c index a4029e3ac810..d869bbec769e 100644 --- a/tools/libs/light/libxl_arm.c +++ b/tools/libs/light/libxl_arm.c @@ -1774,6 +1774,16 @@ void libxl__arch_update_domain_config(libxl__gc *gc, { } +int libxl__arch_hvm_map_gsi(libxl__gc *gc, uint32_t sbdf, uint32_t domid) +{ + return -1; +} + +int libxl__arch_hvm_unmap_gsi(libxl__gc *gc, uint32_t sbdf, uint32_t domid) +{ + return -1; +} + /* * Local variables: * mode: C diff --git a/tools/libs/light/libxl_pci.c b/tools/libs/light/libxl_pci.c index 96cb4da0794e..3d25997921cc 100644 --- a/tools/libs/light/libxl_pci.c +++ b/tools/libs/light/libxl_pci.c @@ -17,6 +17,7 @@ #include "libxl_osdeps.h" /* must come before any other headers */ #include "libxl_internal.h" +#include "libxl_arch.h" #define PCI_BDF "%04x:%02x:%02x.%01x" #define PCI_BDF_SHORT "%02x:%02x.%01x" @@ -1478,6 +1479,16 @@ static void pci_add_dm_done(libxl__egc *egc, fclose(f); if (!pci_supp_legacy_irq()) goto out_no_irq; + + /* + * When dom0 is PVH and mapping a x86 gsi to pirq for domU, + * should use gsi to grant irq permission. + */ + if (!libxl__arch_hvm_map_gsi(gc, pci_encode_bdf(pci), domid)) + goto pci_permissive; + else + LOGED(WARN, domid, "libxl__arch_hvm_map_gsi failed (err=%d)", errno); + sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain, pci->bus, pci->dev, pci->func); f = fopen(sysfs_path, "r"); @@ -1505,6 +1516,7 @@ static void pci_add_dm_done(libxl__egc *egc, } fclose(f); +pci_permissive: /* Don't restrict writes to the PCI config space from this VM */ if (pci->permissive) { if ( sysfs_write_bdf(gc, SYSFS_PCIBACK_DRIVER"/permissive", @@ -2229,6 +2241,11 @@ skip_bar: if (!pci_supp_legacy_irq()) goto skip_legacy_irq; + if (!libxl__arch_hvm_unmap_gsi(gc, pci_encode_bdf(pci), domid)) + goto skip_legacy_irq; + else + LOGED(WARN, domid, "libxl__arch_hvm_unmap_gsi failed (err=%d)", errno); + sysfs_path = GCSPRINTF(SYSFS_PCI_DEV"/"PCI_BDF"/irq", pci->domain, pci->bus, pci->dev, pci->func); diff --git a/tools/libs/light/libxl_x86.c b/tools/libs/light/libxl_x86.c index 60643d6f5376..e7756d323cb6 100644 --- a/tools/libs/light/libxl_x86.c +++ b/tools/libs/light/libxl_x86.c @@ -879,6 +879,117 @@ void libxl__arch_update_domain_config(libxl__gc *gc, libxl_defbool_val(src->b_info.u.hvm.pirq)); } +struct pcidev_map_pirq { + uint32_t sbdf; + uint32_t pirq; + XEN_LIST_ENTRY(struct pcidev_map_pirq) entry; +}; + +static pthread_mutex_t pcidev_pirq_mutex = PTHREAD_MUTEX_INITIALIZER; +static XEN_LIST_HEAD(, struct pcidev_map_pirq) pcidev_pirq_list = + XEN_LIST_HEAD_INITIALIZER(pcidev_pirq_list); + +int libxl__arch_hvm_map_gsi(libxl__gc *gc, uint32_t sbdf, uint32_t domid) +{ + int pirq = -1, gsi, r; + xc_domaininfo_t info; + struct pcidev_map_pirq *pcidev_pirq; + libxl_ctx *ctx = libxl__gc_owner(gc); + + r = xc_domain_getinfo_single(ctx->xch, LIBXL_TOOLSTACK_DOMID, &info); + if (r < 0) { + LOGED(ERROR, domid, "getdomaininfo failed (error=%d)", errno); + return r; + } + if ((info.flags & XEN_DOMINF_hvm_guest) && + !(info.arch_config.emulation_flags & XEN_X86_EMU_USE_PIRQ)) { + gsi = xc_physdev_gsi_from_pcidev(ctx->xch, sbdf); + if (gsi < 0) { + return ERROR_FAIL; + } + r = xc_physdev_map_pirq(ctx->xch, domid, gsi, &pirq); + if (r < 0) { + LOGED(ERROR, domid, "xc_physdev_map_pirq gsi=%d (error=%d)", + gsi, errno); + return r; + } + r = xc_domain_gsi_permission(ctx->xch, domid, gsi, 1); + if (r < 0) { + LOGED(ERROR, domid, "xc_domain_gsi_permission gsi=%d (error=%d)", + gsi, errno); + return r; + } + } else { + return ERROR_FAIL; + } + + /* Save the pirq for the usage of unmapping */ + pcidev_pirq = malloc(sizeof(struct pcidev_map_pirq)); + if (!pcidev_pirq) { + LOGED(ERROR, domid, "no memory for saving pirq of pcidev info"); + return ERROR_NOMEM; + } + pcidev_pirq->sbdf = sbdf; + pcidev_pirq->pirq = pirq; + + assert(!pthread_mutex_lock(&pcidev_pirq_mutex)); + XEN_LIST_INSERT_HEAD(&pcidev_pirq_list, pcidev_pirq, entry); + assert(!pthread_mutex_unlock(&pcidev_pirq_mutex)); + + return 0; +} + +int libxl__arch_hvm_unmap_gsi(libxl__gc *gc, uint32_t sbdf, uint32_t domid) +{ + int pirq = -1, gsi, r; + xc_domaininfo_t info; + struct pcidev_map_pirq *pcidev_pirq; + libxl_ctx *ctx = libxl__gc_owner(gc); + + r = xc_domain_getinfo_single(ctx->xch, LIBXL_TOOLSTACK_DOMID, &info); + if (r < 0) { + LOGED(ERROR, domid, "getdomaininfo failed (error=%d)", errno); + return r; + } + if ((info.flags & XEN_DOMINF_hvm_guest) && + !(info.arch_config.emulation_flags & XEN_X86_EMU_USE_PIRQ)) { + gsi = xc_physdev_gsi_from_pcidev(ctx->xch, sbdf); + if (gsi < 0) { + return ERROR_FAIL; + } + assert(!pthread_mutex_lock(&pcidev_pirq_mutex)); + XEN_LIST_FOREACH(pcidev_pirq, &pcidev_pirq_list, entry) { + if (pcidev_pirq->sbdf == sbdf) { + pirq = pcidev_pirq->pirq; + XEN_LIST_REMOVE(pcidev_pirq, entry); + free(pcidev_pirq); + break; + } + } + assert(!pthread_mutex_unlock(&pcidev_pirq_mutex)); + if (pirq < 0) { + /* pirq has been unmapped, so return directly */ + return 0; + } + r = xc_physdev_unmap_pirq(ctx->xch, domid, pirq); + if (r < 0) { + LOGED(ERROR, domid, "xc_physdev_unmap_pirq pirq=%d (error=%d)", + pirq, errno); + return r; + } + r = xc_domain_gsi_permission(ctx->xch, domid, gsi, 0); + if (r < 0) { + LOGED(ERROR, domid, "xc_domain_gsi_permission gsi=%d (error=%d)", + gsi, errno); + return r; + } + } else { + return ERROR_FAIL; + } + + return 0; +} + /* * Local variables: * mode: C