From patchwork Thu May 16 10:03:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13665970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C97EC25B79 for ; Thu, 16 May 2024 10:04:02 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.722991.1127465 (Exim 4.92) (envelope-from ) id 1s7Xxi-0005Vx-GZ; Thu, 16 May 2024 10:03:46 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 722991.1127465; Thu, 16 May 2024 10:03:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxi-0005VE-Af; Thu, 16 May 2024 10:03:46 +0000 Received: by outflank-mailman (input) for mailman id 722991; Thu, 16 May 2024 10:03:45 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxh-0005ER-3k for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:45 +0000 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on20601.outbound.protection.outlook.com [2a01:111:f403:2407::601]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9305f67a-136b-11ef-b4bb-af5377834399; Thu, 16 May 2024 12:03:42 +0200 (CEST) Received: from BYAPR02CA0030.namprd02.prod.outlook.com (2603:10b6:a02:ee::43) by SA1PR12MB8161.namprd12.prod.outlook.com (2603:10b6:806:330::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27; Thu, 16 May 2024 10:03:39 +0000 Received: from CO1PEPF000042AC.namprd03.prod.outlook.com (2603:10b6:a02:ee:cafe::94) by BYAPR02CA0030.outlook.office365.com (2603:10b6:a02:ee::43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28 via Frontend Transport; Thu, 16 May 2024 10:03:36 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000042AC.mail.protection.outlook.com (10.167.243.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 10:03:36 +0000 Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May 2024 05:03:35 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May 2024 05:03:35 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Thu, 16 May 2024 05:03:34 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9305f67a-136b-11ef-b4bb-af5377834399 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Skzxkw/gc/elR/dfcS6eEaw8+V/2BeVfRfuS34yE2wgg11K3Hag3bZQ3xPAA53zgBBRDY4YD6xI/RKOgUbsEsXVwLZ/F7/lXlhTYW092/5woUx9b2DKD39pTdnk6LQGpaPrGPbWgrqY2Gar/yckqpW44OBFTSfiOrwsG9GFPYPguDXFAx6KbbtSonRJ6ld7ZswgwT3Q4ihJ4yxcXKxvoDFaH1RInF2OzAkTsSkqiH6yYWxHJaWfik7Esy6ezB4ugHRBEBj4p2RxGbBj/mu9mivT5wLKPgkACL+yaUUMI7BXRw3nCNyE27L8NKEE5oUNx8fBKp/YKYgWshzyRDYgu5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OSVZTcTs9mm7sB/dXnHKN/ALb10cEIdfRSBxLdDzF1s=; b=O2EvHeSWLPSXIXPaLyI7eBpUdXuVw2Jd/qQZXZX2GIhEYqQMu8jBHyiY+lj+6sgSTtsxxHPDvGdE6nfcYAEshhuIf8R7rsnvQ/TR4PBtJ/MNnz/mPUaKgLRy0DrlW8dZS+54nnEPVUG1LZjAksNFXwrIBp/SxghPJ34uf5iwNH2vHnWHDcJsvtwKn+Svh8BJbLXOnD2sI+WG1+rvcaaPW2vZAvDN9KaTIL+bQd2EcAxIcamLZSVIj0QFjqLroKmL5x4nj2kuxCq6OujW88bkqa7nbpeXdSwBncTcX13vg+GY0dx1cmFRQrscoUMJ2A7bzzY2LptFuGCDUmb7LnqmRg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OSVZTcTs9mm7sB/dXnHKN/ALb10cEIdfRSBxLdDzF1s=; b=138iIMtSB3GcV36u1I678e6tFnqagssZLIsNdtqr73aBpUCJUalMvKui+0tguH0+lfwBwz/UTP+WoM4/UZvBCk8OhD7Fn4QUNf9wrmwwe62xUJOjXHlS1KpPc63rfPvYsCtuPcHbjOCNsfo+VHLdcd+knGzNtuE9yXGkHr+rkuA= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Henry Wang To: CC: Henry Wang , Stefano Stabellini , Julien Grall Subject: [PATCH v2 1/8] xen/common/dt-overlay: Fix lock issue when add/remove the device Date: Thu, 16 May 2024 18:03:23 +0800 Message-ID: <20240516100330.1433265-2-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com> References: <20240516100330.1433265-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000042AC:EE_|SA1PR12MB8161:EE_ X-MS-Office365-Filtering-Correlation-Id: ade187ed-93ef-4405-8e54-08dc758f73df X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005; X-Microsoft-Antispam-Message-Info: 2oMr5RY6wH5/euGsSa4U+iE4PAOvkIApsUxYg9XGiyXP4jQaTqpI37HJC0p4ya9dSamv2eHhJMzp/w5V39/4GlFxZxY/7JnscBIOjPCb/WuRKVmrnQO0FVth/4dDo6XXhKPJC/NnZHE5uUwjAT6gWEznNb1nXcdw7K1OGQ8jryuQWBxlZs9jMTs/s6hqwJDJCC1tfwtCcnkI5VKTSomjBGaFC5RWFtyiPLqxWcxXKVlVzKENJuy5bTyu6+cZFdgs2b50pDkyTr/PdEa9xQD+5EJycjhDkGiXKua1oxEWsEZkz5K0H4QI11wHcUwR+a0G71T7aIckJ+jmcPUh2O2HI/YlMf3B+WQmsaLSNHhPBvruTb3RbwtzWpWbZSO9oiNj6L8G71RnAEUS894cP5jTgRWTKtJWdQ2PH4dehGZK+Qn5zqUYanBsyffucJtvOznI0VYFig3DWX9OiJrVn71gBewQcIl8gtcICmoCBYMEN9KRqKmi/oRPTfAPXPpTo0yvVFjA1jpql3dwDqpJVA7dMlWozuxiJ0MThH56zaffFLAQAgibqyi8FVNEBMWotK1LqYGDDIwjZhlx+LRsAu5bg2vyEYcUcYFSjtw/lrmvc64y0rDL5PNY8bs+Ct8TaOQKXENRSj9w7deAtpfiP9MDgS8gdNpOtBeliqtUGm2CWcS73CINvujyhIEJV0k+XY1kyR7xe19oIN0OsyqjcCqDphgmyUk6Y6/MjHxrkIiDmx2cJi9jmnuMVSdGMdNOH2Q0miuCqqHsX0YHGBZvpjhoKoOMbZ00hM80jpZApKQAHvBFGhzfmIUW23uRydtLMGTg7LWZBLPw+LVpLX5RKToNBYkLp9w3P5jV/5jNHCRKcoBYV6fEUVRsTBtZHODcjEHdPn0rnJ3rrroc7QOi9hEfka4iUn+K8FCxmduKaMdBnNtaG2FhQgIFrXMkXwQWgcA7hXP4KUHbZr27+YUW1/EXmP40HIR+P2H7us53pAspGjSg03yChtqS2o0iVryoQVBYAU7DoIbk0ik+L7AQgC8a8SfNDcvCZIKBgYUprEije0zNk6Tiy9UjHb/pCR9VfEe7QN4fGDfSLINauD6P1sU9W+Mb6CENhuzKbSJKA7B9W9wLrZw9FhSfsS7j07hj2mVpl9kNwcdTOAvcGTMrYQ71MNtai3WxecOVEfk+eWb1wQ3mA6avFDTtgpLQChhBnsdqom7ttVMfDFPCz+Jngb8fq9Idni+8h8QkHyRvfewljS5EdIw5lYpyBuZffaEaADRXPpqg8jeZLUKNpE2kCQ14SY68Ba5g9m8ulNEFUob9Hai4R9g8pK/+BSt22yUWjY+/uzrBUbKqlSgy4kxJGPTyaJYhPr6boLUzm/ChOm8ACnM= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:36.3165 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ade187ed-93ef-4405-8e54-08dc758f73df X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000042AC.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB8161 If CONFIG_DEBUG=y, below assertion will be triggered: (XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146 (XEN) ----[ Xen-4.19-unstable arm64 debug=y Not tainted ]---- [...] (XEN) Xen call trace: (XEN) [<00000a0000257418>] iommu_remove_dt_device+0x8c/0xd4 (PC) (XEN) [<00000a00002573a0>] iommu_remove_dt_device+0x14/0xd4 (LR) (XEN) [<00000a000020797c>] dt-overlay.c#remove_node_resources+0x8c/0x90 (XEN) [<00000a0000207f14>] dt-overlay.c#remove_nodes+0x524/0x648 (XEN) [<00000a0000208460>] dt_overlay_sysctl+0x428/0xc68 (XEN) [<00000a00002707f8>] arch_do_sysctl+0x1c/0x2c (XEN) [<00000a0000230b40>] do_sysctl+0x96c/0x9ec (XEN) [<00000a0000271e08>] traps.c#do_trap_hypercall+0x1e8/0x288 (XEN) [<00000a0000273490>] do_trap_guest_sync+0x448/0x63c (XEN) [<00000a000025c480>] entry.o#guest_sync_slowpath+0xa8/0xd8 (XEN) (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146 (XEN) **************************************** This is because iommu_remove_dt_device() is called without taking the dt_host_lock. dt_host_lock is meant to ensure that the DT node will not disappear behind back. So fix the issue by taking the lock as soon as getting hold of overlay_node. Similar issue will be observed in adding the dtbo: (XEN) Assertion 'system_state < SYS_STATE_active || rw_is_locked(&dt_host_lock)' failed at xen-source/xen/drivers/passthrough/device_tree.c:192 (XEN) ----[ Xen-4.19-unstable arm64 debug=y Not tainted ]---- [...] (XEN) Xen call trace: (XEN) [<00000a00002594f4>] iommu_add_dt_device+0x7c/0x17c (PC) (XEN) [<00000a0000259494>] iommu_add_dt_device+0x1c/0x17c (LR) (XEN) [<00000a0000267db4>] handle_device+0x68/0x1e8 (XEN) [<00000a0000208ba8>] dt_overlay_sysctl+0x9d4/0xb84 (XEN) [<00000a000027342c>] arch_do_sysctl+0x24/0x38 (XEN) [<00000a0000231ac8>] do_sysctl+0x9ac/0xa34 (XEN) [<00000a0000274b70>] traps.c#do_trap_hypercall+0x230/0x2dc (XEN) [<00000a0000276330>] do_trap_guest_sync+0x478/0x688 (XEN) [<00000a000025e480>] entry.o#guest_sync_slowpath+0xa8/0xd8 This is because the lock is released too early. So fix the issue by releasing the lock after handle_device(). Fixes: 7e5c4a8b86f1 ("xen/arm: Implement device tree node removal functionalities") Signed-off-by: Henry Wang Reviewed-by: Julien Grall --- v2: - Take the lock as soon as getting hold of overlay_node. Also release the lock after handle_device() when adding dtbo. v1.1: - Move the unlock position before the check of rc. --- xen/common/dt-overlay.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c index 1b197381f6..9cece79067 100644 --- a/xen/common/dt-overlay.c +++ b/xen/common/dt-overlay.c @@ -429,18 +429,24 @@ static int remove_nodes(const struct overlay_track *tracker) if ( overlay_node == NULL ) return -EINVAL; + write_lock(&dt_host_lock); + rc = remove_descendant_nodes_resources(overlay_node); if ( rc ) + { + write_unlock(&dt_host_lock); return rc; + } rc = remove_node_resources(overlay_node); if ( rc ) + { + write_unlock(&dt_host_lock); return rc; + } dt_dprintk("Removing node: %s\n", overlay_node->full_name); - write_lock(&dt_host_lock); - rc = dt_overlay_remove_node(overlay_node); if ( rc ) { @@ -604,8 +610,6 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path) return rc; } - write_unlock(&dt_host_lock); - prev_node->allnext = next_node; overlay_node = dt_find_node_by_path(overlay_node->full_name); @@ -619,6 +623,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path) rc = handle_device(hardware_domain, overlay_node, p2m_mmio_direct_c, tr->iomem_ranges, tr->irq_ranges); + write_unlock(&dt_host_lock); if ( rc ) { printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n"); From patchwork Thu May 16 10:03:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13665975 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9D66FC25B7C for ; Thu, 16 May 2024 10:04:11 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.722990.1127459 (Exim 4.92) (envelope-from ) id 1s7Xxi-0005Tc-7e; Thu, 16 May 2024 10:03:46 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 722990.1127459; Thu, 16 May 2024 10:03:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxi-0005TV-3y; Thu, 16 May 2024 10:03:46 +0000 Received: by outflank-mailman (input) for mailman id 722990; Thu, 16 May 2024 10:03:45 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxh-0005T3-2T for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:45 +0000 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on20601.outbound.protection.outlook.com [2a01:111:f403:2414::601]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 939f7113-136b-11ef-909d-e314d9c70b13; Thu, 16 May 2024 12:03:44 +0200 (CEST) Received: from MW4PR03CA0337.namprd03.prod.outlook.com (2603:10b6:303:dc::12) by BY5PR12MB4115.namprd12.prod.outlook.com (2603:10b6:a03:20f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28; Thu, 16 May 2024 10:03:38 +0000 Received: from CO1PEPF000044FC.namprd21.prod.outlook.com (2603:10b6:303:dc:cafe::62) by MW4PR03CA0337.outlook.office365.com (2603:10b6:303:dc::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27 via Frontend Transport; Thu, 16 May 2024 10:03:38 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by CO1PEPF000044FC.mail.protection.outlook.com (10.167.241.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7587.0 via Frontend Transport; Thu, 16 May 2024 10:03:37 +0000 Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May 2024 05:03:37 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May 2024 05:03:36 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Thu, 16 May 2024 05:03:35 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 939f7113-136b-11ef-909d-e314d9c70b13 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ajw8jvLIdS0hWT6baNa0RZztzpKE35uPM2NqX3o+g4QDdMctbOyacesvJISS9V4lkMTWXxwUZc45CQtKpuq6f5BeQ068x3vwo43XtxTG3MW/ak25k61NzM0x9MFGUVUpL41A0B8nKP076xx/SqZgb3VOthMyw/ouCm2hBWRyZrahJOy7DFZPSl/alBmyjVx26Wm08D+/o/0vRIV1CSY9SjD8OhjUFICW3K85i5iemcMDrEV4DyjkDJ0aNjSr8iHm0Ou+K5JtqMcVmzpoaL64cvZDdackoWxGR4x4k/++Isj5eBYC7RJMmE2+vMmNmM+2SCqDRW6dnprWOAX+HPhJqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7coXZqUivStKOWByVVC93sA3+vqD9Aj0YK2CmeBB7tg=; b=m4l7p6qOF1lRreW1l3CBIqWZKzMv3E/eRQyWes6HlXU7afJOK1Os5GpzuCaaSUGhAYnC++9kNXSahM3gNStDtfr8J50Toi2GW1DJiZc21QO7V3UsfhiuIFAkAQHr8eb4bAQLZA8SWyU66qI0UQnE/48qF3+64DU+NHeO8MNjIXFLSlP+pVy5l33AZgDNDWgeAGb6MHzxZD5av5CtVneyPFtT+oQ0lt1tB8REnV7OjQipKFZBJx1nhGje82TmTISY5gehBPtGnDG3yGJBSZnbhfBnR3unTLsE6kWBYsaXcQ+ky6LdRzPpWlx7eOsPyWvQgl6VtJojMeYOETNCp0waSQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7coXZqUivStKOWByVVC93sA3+vqD9Aj0YK2CmeBB7tg=; b=wY5yQibSag/CEOhjV55+Z1bmmARMTm7lc2/3XCwvpWnXWfeFW0DK93DLHFe8d5SfCocjKaub9K/y9mMEK5Ynxkk2/B8tQi5jUw6E9383q8Nzbk6JUZFM9Xw3UPl0fkjpe9EeAHeHmccVMYVoOQ3XDRlIXUhS6pZPtRqrffuBpv8= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Henry Wang To: CC: Henry Wang , Anthony PERARD , Anthony PERARD Subject: [PATCH v2 2/8] tools/xl: Correct the help information and exit code of the dt-overlay command Date: Thu, 16 May 2024 18:03:24 +0800 Message-ID: <20240516100330.1433265-3-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com> References: <20240516100330.1433265-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044FC:EE_|BY5PR12MB4115:EE_ X-MS-Office365-Filtering-Correlation-Id: f596d05e-10a9-408c-7a0a-08dc758f74cd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004; X-Microsoft-Antispam-Message-Info: hHCAM0qA+YQejaY8Ejkz5P90V/g78QFOFlZkGkZdpZmnCNzTPawDr+0lXK/QlOTtamM87/KlPnlc8MkZjtHP/yAcc4PN9a/D2IBRgTOcozS16oDgolr2v+uXTGcBa5mFRj3nh8Y/iVq1/T+yUIJYLpRC4rHLBbulJcq+oUTbUmSSeBpCFTS9ORnTVWgN45NpQwlupFDuyIPoZ3u1LSVYHl+IQDySLn7wQyqpwzv8cDxXgFdudOL0Lx9DyxDMd1t5njMLom0BcSJt4kXLhGZX8hCoLTM6mn/GZTg4ErdSLUuzsPEKg6SGmgXKAJugQakVU0+PrBE0/WzB4orVQlkQFhEFQ505SBS+zf9ZClhdgDcDW5GTHm2JppcJ6VeUaDeWbVq0XV0x5xzQlDefMWpgMVxfRzOQJrHgqEkhbWZPQfSIl+kOTA5sOeWY6Y7waVBCbQvtpxIcI7RHp9GdPk9582wGQn2niyM7LshtjNrlK9w4PcZKEAiVOtt6Umty4ukkJJJAppExNqn9RfCwEu0t4SW3bpVFBwOM6kMk4KxjkSx4ycDQBsJcqoZ9Bmy8v2Sj8P5joVKNc36ndQTeLJclR7GgL3Os2MnluclWzvhp763uf9JEDW6VTZP5h5FJiB50P3AaXtzsvnU9EfqctCZZmtCkOfso2cOHlUYFs2m/tLiqp6qspX2l1xS/m40TIehXk/iGqrugXPfY4JDsOdkb+qQKPxywyHqyztvW/8noMMcaF2o3u6i5W9R2JL5ayNM+3Y1Wsb+l2BHA2asJFfsywu2VMVAB/yWsEMXiCvbgCX6bSsKTsNl1SfsYAVYLdTlJbIjGu10W4C/owNzJr7pcD+aExUzvbhT7o6CvYC887Y2jVliRk30xtWh+OgdjB1hJQs47O4tor6PA733nEoBw2P7sxwf3kFeV5UkytnwtRAICt6Fai1V6jtzlLFD/FJKyc0JYfYp4nh5lHzxG1PeoUrmQzaYRzBxCvVfkAavJomHZc/J2W/LyTfU8F+daaUPgv/pJqh4OF5Xf1K0ijKyWPBnze+9J9lFJmhAEVs3TzE+btbfmDrgXv7C3JuGuIYXUjd5tmO4Q5RimWd2YuEzK73v8nF3sH9js55uRjKKrclEpufHIS0KEJyUaR7Pf/TCAuhdR9Z7/BBND8CvdXAyskPEwFwHlceSvipLqcfvrE5IPhj0jWOFFuW4kXWIt1WUlhMEas8HVQElslbUi8qvA6hF3wTJRxmt6QRCekJaxno4upt57ATXgaUfjHk1YAaTocWGZod+Keusm7AmlTCMso2v4D26om5HAWW9UnZhLMtJxlvSiH69Ffxoh15YPG/B2UKTIyVsCRX4r/AdiHdCMyy25VWobm1DtJZOvDn5W6FE= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:37.8445 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f596d05e-10a9-408c-7a0a-08dc758f74cd X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044FC.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4115 Fix the name mismatch in the xl dt-overlay command, the command name should be "dt-overlay" instead of "dt_overlay". Add the missing "," in the cmdtable. Fix the exit code of the dt-overlay command, use EXIT_FAILURE instead of ERROR_FAIL. Fixes: 61765a07e3d8 ("tools/xl: Add new xl command overlay for device tree overlay support") Suggested-by: Anthony PERARD Signed-off-by: Henry Wang Reviewed-by: Jason Andryuk --- v2: - New patch --- tools/xl/xl_cmdtable.c | 2 +- tools/xl/xl_vmcontrol.c | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c index 62bdb2aeaa..1f3c6b5897 100644 --- a/tools/xl/xl_cmdtable.c +++ b/tools/xl/xl_cmdtable.c @@ -635,7 +635,7 @@ const struct cmd_spec cmd_table[] = { { "dt-overlay", &main_dt_overlay, 0, 1, "Add/Remove a device tree overlay", - "add/remove <.dtbo>" + "add/remove <.dtbo>", "-h print this help\n" }, #endif diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c index 98f6bd2e76..02575d5d36 100644 --- a/tools/xl/xl_vmcontrol.c +++ b/tools/xl/xl_vmcontrol.c @@ -1278,7 +1278,7 @@ int main_dt_overlay(int argc, char **argv) const int overlay_remove_op = 2; if (argc < 2) { - help("dt_overlay"); + help("dt-overlay"); return EXIT_FAILURE; } @@ -1302,11 +1302,11 @@ int main_dt_overlay(int argc, char **argv) fprintf(stderr, "failed to read the overlay device tree file %s\n", overlay_config_file); free(overlay_dtb); - return ERROR_FAIL; + return EXIT_FAILURE; } } else { fprintf(stderr, "overlay dtbo file not provided\n"); - return ERROR_FAIL; + return EXIT_FAILURE; } rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op); From patchwork Thu May 16 10:03:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13665974 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2AF3C25B79 for ; Thu, 16 May 2024 10:04:10 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.722992.1127479 (Exim 4.92) (envelope-from ) id 1s7Xxk-0005xn-K3; Thu, 16 May 2024 10:03:48 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 722992.1127479; Thu, 16 May 2024 10:03:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxk-0005xe-H0; Thu, 16 May 2024 10:03:48 +0000 Received: by outflank-mailman (input) for mailman id 722992; Thu, 16 May 2024 10:03:47 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxj-0005ER-DS for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:47 +0000 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on20600.outbound.protection.outlook.com [2a01:111:f403:2414::600]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 94896344-136b-11ef-b4bb-af5377834399; Thu, 16 May 2024 12:03:45 +0200 (CEST) Received: from BYAPR02CA0028.namprd02.prod.outlook.com (2603:10b6:a02:ee::41) by SJ1PR12MB6268.namprd12.prod.outlook.com (2603:10b6:a03:455::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Thu, 16 May 2024 10:03:42 +0000 Received: from CO1PEPF000042AC.namprd03.prod.outlook.com (2603:10b6:a02:ee:cafe::21) by BYAPR02CA0028.outlook.office365.com (2603:10b6:a02:ee::41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28 via Frontend Transport; Thu, 16 May 2024 10:03:42 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000042AC.mail.protection.outlook.com (10.167.243.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 10:03:42 +0000 Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May 2024 05:03:39 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May 2024 05:03:38 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Thu, 16 May 2024 05:03:37 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 94896344-136b-11ef-b4bb-af5377834399 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iNrI7o2xjMNkq21QYDZzguE6Bz7SOkDDO4Fy5LXSna4r1/7+gARcEoHgXuJiCzaonF6lqw1opEybh7lZhGeigrlhYRbgW3pfnm9R7cZ78VzeDiXz7WWDveD7KA52AmA9y/1QIfmvAFwZzDIosAFIjLyshZYsJ/ClJx3ZOWALgpYJkUcYn7Pew9VAoEB5aSFvtJ26d7/fTzY8gdXem1+g3jD/z+HbC33fMYsF055ng0NuQ3cl1xe+Yn2QCdowDkUwVW/CTDeCdpAV4/u9wlBmSOIAS7xw5hxbbi01bqp2TZiMDDv22JFVfbAAZCmwatL4pyGSrv/O3HkVdZ+s1dHWaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4FOmUBM5q170h0MhyoHdpU1Ky6atwZpLP5xhhuCrh8w=; b=N1LFR8bfk3prN+aYNEQ9M7Q/a+nG+KH16btoCgBT8w4IgCv0fJnzbz4KsiQ6PDfeYUhXBOO46WbrA2EPn6ZjD90Bk/orXgTbsmsFaIk0RjrQeEh37KE3ME3eKT1nPEWtRV6Y/qYO4qvPF/MkixKV1ruORTT1lo55gYkF6Hinq/wMh9bkOhipyE8wY8BsN1ZY9bhwPoff+Jd1B4nuDqFDOIlECK/IwzUl6wXchCGCKpzJlOwbKctWzWE6SaFkS52+RlMP7V4vHgdlDT4dI38Ume5h1shfPLyqy4Jc4a8HOkgQUc11zFYmWDPLKBv8MiWZek+Ohh0KwHYL52ES7ZN2xw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4FOmUBM5q170h0MhyoHdpU1Ky6atwZpLP5xhhuCrh8w=; b=JXJoBe4xTAOMCFxY81+f73Gq61Ikop2fzkTNhKKqgIn/SmcK2hnACZtYJlXKVsgI5WxJa9JOtmFX6oQ4FDWDZslhnTnLKAzQ81fee0wQ3hU7YoCFEklPDOJxOsWP6ONABC7RwTQ06/T8ajGyLkBgVULzWKPVTiBTBfo6IuCoLME= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Henry Wang To: CC: Henry Wang , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , "Volodymyr Babchuk" Subject: [PATCH v2 3/8] xen/arm, doc: Add a DT property to specify IOMMU for Dom0less domUs Date: Thu, 16 May 2024 18:03:25 +0800 Message-ID: <20240516100330.1433265-4-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com> References: <20240516100330.1433265-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000042AC:EE_|SJ1PR12MB6268:EE_ X-MS-Office365-Filtering-Correlation-Id: 8c8c6576-5a73-45d8-2bf5-08dc758f7767 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|82310400017|36860700004|1800799015|376005; X-Microsoft-Antispam-Message-Info: p28vscojw1AxysNMGx8aJEqRwFe5dHP0+wm4yebhfDLEBSwOCT5alRUt/amghAu91/s145djjhuge+TqvSZJ4kBDEOdKSNl/DIeXI/p2OItssMaAa5JTBUYspAli7/7nxq51fWR0ZTE3nADl4RSPgdDZ3/8LlftjnPtZwVpX4XzljWWleQX6scSLZOTYPic+PoZGwmMVzck9qhJ5cIhIWjD/WDsVNl+SdZ6NNw+XmwXQEMBY5o5xzjX30/PvV2XE4ywM0dPoeS9Iwg3oCPhC1Td00IUiqTlBlEzVdPPyyKqcrMtPEzD6utq+t9eo+zuE62N2Fy9Su5EeQMzq/r1t1v/m3wq+JWrJE9n//mqLd8AeuTFSVGb/60Ex95qXrd0ioaRGvwwyHWKooobg5GAGq1gmLyJWnPKhrSiijp0RVo7Qq0/r5e2vi2X48HW6/a40LvCby1DGrBqikUfI9M04Qdb5hOQ7zz9U5LJZt5nnQ/TD9ugk+xv/i1xx3i9xVPDQ6/SmBw8MijA/qKlN/ViVRey0SSCVA+6TuPiWbMK5u7TIcgesfg79Ct6zPFQxDWD7n4vtrsblRuqhwKwj5FOxh2q7Qr6+VsK5r/+nvagholt2ibxfTtO4agqSPusVwTA0UTMEumthvQ7BOvk5QXq+NYPD8WB3DhYDLrTr7clIibk5utgekSuJOwysbiuS7lPV11cxK11S28NU4khEE/rXAzag9X64tCOoTSPS0U7wOn6nSZYw9APBuLW3SBRtaA2tj/I3cMbCGBQAcfVwErQh0bDEjqljAr1wLx262L8P7IAGzTc5bFXCRdyN42TVmGqQIG65c4ZgZMXWa0sWyTkXYLD9gaMD9mo2neWtZVKcMSloVkwdP3nbXlYuv7nNM/qDe7Zb9KqKAuOH5OhMnMLoPaCg+zn3Gu1o49CFYuYXShKNbtsv/VIDVgOTMdEAYfstyqozVHxHP5m+kgtYVctZ6XzNH6QqvCkwtMO4U+pEmvo6fbNK3I/BVIxRyXTd70aQe69m+1d889gvadwl0Lv1QavskBNMqR6XgLnjD8m9eO7PbTBofrA5wKqo6rY6M2q+2KxzNhFdSky8KCFeoDN/cup3AGOiy1cr4e9cGYt5Lid2yD7JKrI8wHtU4vfvXhQkpBfP6iMJbIMGSOXPFzLy2lbUlTJS8tWd4TbEMCq/pluiZzZtIngpgNwInN6omMzHT7QK4+lisTuTkbtv2aQqepWyKoi+7825VIW3nchUIx6zKo3iL/PpMX92v91m0vIfWZmnaQOlc8x0YoL1HsLSFlwDo7lq6NlmchUxZ7Ny+JzJBRRa6eIUL7AIoEPdUnYQV3QaZJHTHun2XtydAMWYOFkKGD4pTjXDbMLmpZmq8WVnkkoUopqBRz00x987xTrC X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(1800799015)(376005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:42.2384 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8c8c6576-5a73-45d8-2bf5-08dc758f7767 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000042AC.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6268 There are some use cases in which the dom0less domUs need to have the XEN_DOMCTL_CDF_iommu set at the domain construction time. For example, the dynamic dtbo feature allows the domain to be assigned a device that is behind the IOMMU at runtime. For these use cases, we need to have a way to specify the domain will need the IOMMU mapping at domain construction time. Introduce a "passthrough" DT property for Dom0less DomUs following the same entry as the xl.cfg. Currently only provide two options, i.e. "enable" and "disable". Set the XEN_DOMCTL_CDF_iommu at domain construction time based on the property. Signed-off-by: Henry Wang --- v2: - New patch to replace the original patch in v1: "[PATCH 03/15] xen/arm: Always enable IOMMU" --- docs/misc/arm/device-tree/booting.txt | 13 +++++++++++++ xen/arch/arm/dom0less-build.c | 7 +++++-- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt index bbd955e9c2..61f9082553 100644 --- a/docs/misc/arm/device-tree/booting.txt +++ b/docs/misc/arm/device-tree/booting.txt @@ -260,6 +260,19 @@ with the following properties: value specified by Xen command line parameter gnttab_max_maptrack_frames (or its default value if unspecified, i.e. 1024) is used. +- passthrough + + A string property specifying whether IOMMU mappings are enabled for the + domain and hence whether it will be enabled for passthrough hardware. + Possible property values are: + + - "enabled" + IOMMU mappings are enabled for the domain. + + - "disabled" + IOMMU mappings are disabled for the domain and so hardware may not be + passed through. This option is the default if this property is missing. + Under the "xen,domain" compatible node, one or more sub-nodes are present for the DomU kernel and ramdisk. diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c index 74f053c242..1396a102e1 100644 --- a/xen/arch/arm/dom0less-build.c +++ b/xen/arch/arm/dom0less-build.c @@ -848,6 +848,7 @@ static int __init construct_domU(struct domain *d, void __init create_domUs(void) { struct dt_device_node *node; + const char *dom0less_iommu; const struct dt_device_node *cpupool_node, *chosen = dt_find_node_by_path("/chosen"); @@ -895,8 +896,10 @@ void __init create_domUs(void) panic("Missing property 'cpus' for domain %s\n", dt_node_name(node)); - if ( dt_find_compatible_node(node, NULL, "multiboot,device-tree") && - iommu_enabled ) + if ( iommu_enabled && + ((!dt_property_read_string(node, "passthrough", &dom0less_iommu) && + !strcmp(dom0less_iommu, "enabled")) || + dt_find_compatible_node(node, NULL, "multiboot,device-tree")) ) d_cfg.flags |= XEN_DOMCTL_CDF_iommu; if ( !dt_property_read_u32(node, "nr_spis", &d_cfg.arch.nr_spis) ) From patchwork Thu May 16 10:03:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13665969 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7F9A4C25B74 for ; Thu, 16 May 2024 10:04:00 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.722993.1127489 (Exim 4.92) (envelope-from ) id 1s7Xxp-0006Iq-2I; Thu, 16 May 2024 10:03:53 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 722993.1127489; Thu, 16 May 2024 10:03:53 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxo-0006Ia-UQ; Thu, 16 May 2024 10:03:52 +0000 Received: by outflank-mailman (input) for mailman id 722993; Thu, 16 May 2024 10:03:51 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxn-0005T3-SB for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:51 +0000 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on20601.outbound.protection.outlook.com [2a01:111:f403:200a::601]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 97a9dba8-136b-11ef-909d-e314d9c70b13; Thu, 16 May 2024 12:03:51 +0200 (CEST) Received: from MW4PR03CA0155.namprd03.prod.outlook.com (2603:10b6:303:8d::10) by SA1PR12MB6798.namprd12.prod.outlook.com (2603:10b6:806:25a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28; Thu, 16 May 2024 10:03:45 +0000 Received: from CO1PEPF000042A8.namprd03.prod.outlook.com (2603:10b6:303:8d:cafe::cc) by MW4PR03CA0155.outlook.office365.com (2603:10b6:303:8d::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend Transport; Thu, 16 May 2024 10:03:45 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000042A8.mail.protection.outlook.com (10.167.243.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 10:03:44 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May 2024 05:03:40 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Thu, 16 May 2024 05:03:39 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 97a9dba8-136b-11ef-909d-e314d9c70b13 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iDu6A9qbnYr0zSFV7lGoYQdZBuadHzWR2oz6a1lXG+WioqD/hnVKYFcEYobpU/O9XPC21JtAUKsWl5sSKI9f29Z79HSMTgo4FYwm3MmhfVbSz2VvqgYkup10iL/Kdy9ByYIjSRwpcWLbIQ49Rd6bshUUIQPTATxDNfpWH23eBO/Oat1sXick66oWsM9JVYMVWG1pXRohTO0ctpSz4jmBwP3XusGQ7wJYBGO619U3in3ETN+Pow7rg3uuFiJR+vngeAEKWtlLYKgrFktezX/LUOUlKd+v0bgjNebASGSDmSLlDlCthZNMZZ3DvQxaUrSexZPz/5jChGX2QqXVSEZXDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wWIi7CD7ZXy7QHzhQvNqeQnVv2B/6KurT+iAYX495cs=; b=FA8DXEaMb8ttkGqE1+eT0uFILbEeo+rjq8k6OTUUD2amOPQwm0s4xmpRB7UyVHpdD8uTrQxBPc76biOg04OtrlpwtnPSGXoTSky4wthu92KIQCNJIbjuvV4bO4CsSxpLCNE8L5iybR99fyO3BaVqGt5eBYxoVqfl5oca+A3xcI82dwmyNhSLrWwSP2wWyUO0C/m03JuLgCAJZ7r/DiSNwb0c7xIeHGtCfehd/Pq0aqhePxbYTtyWTgw3dMgfBYCTytikdbjZ6Lt6f4S5xqF/js1PJRvD0+5OZrj1oCjhYGMGxNvia5u0kUIVd4rjPVWVYhp2UwAvWZN1VmWJDVRUKA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wWIi7CD7ZXy7QHzhQvNqeQnVv2B/6KurT+iAYX495cs=; b=4g1YPQZsRLLJ8Zk163NgtuJD3JvPOvaTaMBSepjWtddhd96hhX00j7UVo8i/J7/H1lhDxnf2HCTrko4s4Bt32zBIOUhiQl6YHgLOjEw/7kotk+naJhwBD0o4sf0UIdVp3x2p6KQjdxRQhywf9cYdregRfbXYGml++UH6iQxXTZI= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Henry Wang To: CC: Henry Wang , Anthony PERARD , George Dunlap , Nick Rosbrook , Juergen Gross Subject: [PATCH v2 4/8] tools/arm: Introduce the "nr_spis" xl config entry Date: Thu, 16 May 2024 18:03:26 +0800 Message-ID: <20240516100330.1433265-5-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com> References: <20240516100330.1433265-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000042A8:EE_|SA1PR12MB6798:EE_ X-MS-Office365-Filtering-Correlation-Id: d2388b5e-93f9-4d4e-1eea-08dc758f78cc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|376005|36860700004|1800799015|82310400017; X-Microsoft-Antispam-Message-Info: sanhQ7uQGc63rD9omLVewJWBeEfHtF+cHwOzttmWVDkgzTz3vIiUgsFmxeTsoxo9MrCwoZzKpMzBc1XfpvmJxe4usytmojaqqPIq2t7ZnzgPY6+IJdhn1a3xKwGU8ttFmcioqehcV8WhyI5SSpnxXdfntwvJjgYA+f82EKOySzJaBgfOyo790BcIyPXFgvNewyz9suPAQPfibeYBVp2TS55M1yY2OcRPGxLuqp54uo9RgLfNNghM6ZI6FbA6fFni6B28tk19KFD17yylICx9g2PXOb3l0chblIKHgGc1ZOi4SSCCuzs9adKnZx73nbiiBdNhuz3w6a2BoeqQFYMsj+y81nx39xXkzs3pWKwmk8i2//ev+TGQQT3nqYvus9q0UDBiMU+NwUmaJ7tdka4g3RrN4qF+G1QZOshiajWmtui8KKS27jVLaSitYu3fsrUZQ539iNSGcWShVdX5a27DG6ztK/GnP/V5aMtMVmUNGGtf7MC6Szqir7O25wlPn+qbQ1YYmXT+gHuh/3r07RQChKEw7C+JSXnmNOCYbnvmXcb1pdI12/8w8rk/e4uUHUVUSKQwsCiFLhXYDRZpMFmqqwGixAuszootAydux5M3Tma2fKTk5rUOQZaf6A7Fs9CpEFL6eJ4FJRpxlT1HUfvUxABsL883fO0P7gbA5Y406MTWK2PmnTZNk1tZFF4F2H6P1YoC+3ABH0i6fg70Oi1ZkQMsRD+K6fY8lBpW1myIxscZekcvZRaLdzHdM7nuscJTIgRuvlJ2Ypt2+cvqJmofifNc+zG3FtGIOBZmGxMaYsSm7yWFoeKkRptVuUSGH8gfwwINnloRxIJ98aNwQtNTDRnjwxhHghXE+F8jgkW/F5K7zIgwOpebPTbA3j0STX0EYX99olfXyMnzjPoxO9SMdDksum+sycIqB1/09EpvWg8MptDeiIizC/93DHC/hzbnE9H06Bu0O3XZGLE3YgTVI9CHrKdzlD//Wo4iq0K4vx+USquSWYubMHvlqP4uxH44BavYDbx0CY4w/7NuNU9l5KwQYUatcAgnY1WLa+9BucfBXN7FoFhQ9JItYldk+Spx7ayz8tztKVW1Y4ZhQZnsOGVTctWQXYteBHSSa1yru2NlnB62hymJj3ZV6WLBKaR4dqhWbfWMzU9RvpYjadoLLvaWXW+SIKOffBXmFarN8nOALxslCVuVG9kuG6gJKf5TEtUiElF+BPnQOu6u4nykNPu2vW/RrBCVnTBvQCLkPMEGJHyJQe68yszaFlyFbouBWDbDh1HOsAZj351yR7gFGlwcMlJVz5Et985GZFQ/hU8yU60tYzezZXg74XJD5pnB9KWITWz/umcnZM98hw6x2zkbVTe/O7mwhhu7JygtJoamCL+x4kdZPc/G07hjYp6r X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(36860700004)(1800799015)(82310400017);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:44.5621 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d2388b5e-93f9-4d4e-1eea-08dc758f78cc X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000042A8.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6798 Currently, the number of SPIs allocated to the domain is only configurable for Dom0less DomUs. Xen domains are supposed to be platform agnostics and therefore the numbers of SPIs for libxl guests should not be based on the hardware. Introduce a new xl config entry for Arm to provide a method for user to decide the number of SPIs. This would help to avoid bumping the `config->arch.nr_spis` in libxl everytime there is a new platform with increased SPI numbers. Update the doc and the golang bindings accordingly. Signed-off-by: Henry Wang --- v2: - New patch to replace the original patch in v1: "[PATCH 05/15] tools/libs/light: Increase nr_spi to 160" --- docs/man/xl.cfg.5.pod.in | 11 +++++++++++ tools/golang/xenlight/helpers.gen.go | 2 ++ tools/golang/xenlight/types.gen.go | 1 + tools/libs/light/libxl_arm.c | 4 ++-- tools/libs/light/libxl_types.idl | 1 + tools/xl/xl_parse.c | 3 +++ 6 files changed, 20 insertions(+), 2 deletions(-) diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in index 8f2b375ce9..6a2d86065e 100644 --- a/docs/man/xl.cfg.5.pod.in +++ b/docs/man/xl.cfg.5.pod.in @@ -3072,6 +3072,17 @@ raised. =back +=over 4 + +=item B + +A 32-bit optional integer parameter specifying the number of SPIs (Shared +Peripheral Interrupts) to allocate for the domain. If the `nr_spis` parameter +is missing, the max number of SPIs calculated by the toolstack based on the +devices allocated for the domain will be used. + +=back + =head3 x86 =over 4 diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go index 78bdb08b15..757ccaf035 100644 --- a/tools/golang/xenlight/helpers.gen.go +++ b/tools/golang/xenlight/helpers.gen.go @@ -1154,6 +1154,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)} x.ArchArm.GicVersion = GicVersion(xc.arch_arm.gic_version) x.ArchArm.Vuart = VuartType(xc.arch_arm.vuart) x.ArchArm.SveVl = SveType(xc.arch_arm.sve_vl) +x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis) if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil { return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err) } @@ -1670,6 +1671,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)} xc.arch_arm.gic_version = C.libxl_gic_version(x.ArchArm.GicVersion) xc.arch_arm.vuart = C.libxl_vuart_type(x.ArchArm.Vuart) xc.arch_arm.sve_vl = C.libxl_sve_type(x.ArchArm.SveVl) +xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis) if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil { return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err) } diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go index ccfe18019e..b7b4ba88af 100644 --- a/tools/golang/xenlight/types.gen.go +++ b/tools/golang/xenlight/types.gen.go @@ -597,6 +597,7 @@ ArchArm struct { GicVersion GicVersion Vuart VuartType SveVl SveType +NrSpis uint32 } ArchX86 struct { MsrRelaxed Defbool diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c index 1cb89fa584..a4029e3ac8 100644 --- a/tools/libs/light/libxl_arm.c +++ b/tools/libs/light/libxl_arm.c @@ -181,8 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc, LOG(DEBUG, "Configure the domain"); - config->arch.nr_spis = nr_spis; - LOG(DEBUG, " - Allocate %u SPIs", nr_spis); + config->arch.nr_spis = max(nr_spis, d_config->b_info.arch_arm.nr_spis); + LOG(DEBUG, " - Allocate %u SPIs", config->arch.nr_spis); switch (d_config->b_info.arch_arm.gic_version) { case LIBXL_GIC_VERSION_DEFAULT: diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl index 470122e768..3f143f405d 100644 --- a/tools/libs/light/libxl_types.idl +++ b/tools/libs/light/libxl_types.idl @@ -722,6 +722,7 @@ libxl_domain_build_info = Struct("domain_build_info",[ ("arch_arm", Struct(None, [("gic_version", libxl_gic_version), ("vuart", libxl_vuart_type), ("sve_vl", libxl_sve_type), + ("nr_spis", uint32), ])), ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool), ])), diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c index ab09d0288b..4aa99029b5 100644 --- a/tools/xl/xl_parse.c +++ b/tools/xl/xl_parse.c @@ -2933,6 +2933,9 @@ skip_usbdev: } } + if (!xlu_cfg_get_long (config, "nr_spis", &l, 0)) + b_info->arch_arm.nr_spis = l; + parse_vkb_list(config, d_config); d_config->virtios = NULL; From patchwork Thu May 16 10:03:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13665976 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 79CF3C25B74 for ; Thu, 16 May 2024 10:04:11 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.722994.1127499 (Exim 4.92) (envelope-from ) id 1s7Xxq-0006Zg-Bi; Thu, 16 May 2024 10:03:54 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 722994.1127499; Thu, 16 May 2024 10:03:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxq-0006ZR-8V; Thu, 16 May 2024 10:03:54 +0000 Received: by outflank-mailman (input) for mailman id 722994; Thu, 16 May 2024 10:03:53 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxo-0005T3-VL for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:52 +0000 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on20601.outbound.protection.outlook.com [2a01:111:f403:2416::601]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 986b2861-136b-11ef-909d-e314d9c70b13; Thu, 16 May 2024 12:03:52 +0200 (CEST) Received: from MW4PR03CA0162.namprd03.prod.outlook.com (2603:10b6:303:8d::17) by PH0PR12MB7905.namprd12.prod.outlook.com (2603:10b6:510:28b::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27; Thu, 16 May 2024 10:03:48 +0000 Received: from CO1PEPF000044FA.namprd21.prod.outlook.com (2603:10b6:303:8d:cafe::b1) by MW4PR03CA0162.outlook.office365.com (2603:10b6:303:8d::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend Transport; Thu, 16 May 2024 10:03:48 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by CO1PEPF000044FA.mail.protection.outlook.com (10.167.241.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7587.0 via Frontend Transport; Thu, 16 May 2024 10:03:47 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May 2024 05:03:46 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May 2024 05:03:46 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Thu, 16 May 2024 05:03:43 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 986b2861-136b-11ef-909d-e314d9c70b13 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Yv1u04U91wvqYeyAINSCS4oMSzvqtre4QfoGR/A97lMO1m3nxbh9wtaBHZfZZO3CqPi5UoKmWQWYpClmAqWq3DMjOknDGFDi53PadQDZyUUEGgHt/o7YkV2xS31uGz3z8AwoKNk+RhgBJFEczXbmR/RWRqQCun6Lkt9Qgk7sGwcyAH4O6dqMu7HL02bjKOco12KGlOLG4m12ShST2idV/dIBCcpsktiXYCd9MQdQqW7qAFJ56yZvcL1XGEYj4i/O3V1o8c6h/1iCusImSUQMWus8Sfin683Tm1O+14euR54IKtNxz7vIOezY4rXzmyZaMmWraG7TCTCbXfSRD/AXnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mF2OBpKMBYdkRqVcGsWEJvyB5o1cjSWHP++bO6wuCaM=; b=dBPN5QNFY9M/livjelK5E63yc/YObvezt8U44Rus9Tu+ssbngS0J8sM4meJjchV7ukRIPc0AI8OVj1214lLdwaCdJ8zgEes04ZqYsKMx5WEjEbixTfDLtcguve7nXNC3E9jmtsQLKIJsNyb0hFhXDi8hiSSFq0rkpDUfqtlZMfALHfdVYnchyZkvu/pphDaoyGhfv4WePoKCWHkNDfjFBUVLFIAimyLAiZeJl3rkoIr+LO1csTX+ErOVt/NWSOzp9p3erNVQ0AoFR7j6YzET9eqR0Oi8gunVTQ7E1q0UoZqJVKENM51rEVVMmGpJlgHVFZjTtviG2uXrrDx6nQKi1w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mF2OBpKMBYdkRqVcGsWEJvyB5o1cjSWHP++bO6wuCaM=; b=G3h/kFl2E2YtmsELkOZ2ReZmJo315KXNXkkP3LMrLFk5U2VjCTPTAhcHLrjIe09l8KHSQbBYG2ZnW8WZ84C8xjPiALEcGWQYcHyUCiKZJLy56dpCoTIzUjZtr8U7KFT59tkPpiQialV7HI2ut9yWH4w4CepzXhhICrlsQEa8O98= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Henry Wang To: CC: Vikram Garhwal , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , "Volodymyr Babchuk" , Henry Wang Subject: [PATCH v2 5/8] xen/arm/gic: Allow routing/removing interrupt to running VMs Date: Thu, 16 May 2024 18:03:27 +0800 Message-ID: <20240516100330.1433265-6-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com> References: <20240516100330.1433265-1-xin.wang2@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044FA:EE_|PH0PR12MB7905:EE_ X-MS-Office365-Filtering-Correlation-Id: 66647cf5-16f1-4d7b-8f9c-08dc758f7a90 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|376005|1800799015|36860700004|82310400017; X-Microsoft-Antispam-Message-Info: w96Id55op/KRIy+G9ieLph2A179n/n4oqikxATjwFbxX3XnYpGnlwv5CBoY5gCyCBGvr3JFYYSKq0/Q2yhstBkogASEmkcK8S8K9gzi+dyc4ck6johJYoADWxRjB50yh8JSl8YmTw24hsMZTrdD63ZSMXHiQCRN6owCf8yqzuHD91RMBmwaKW4R2L5EmzTasfpZ7xn/9nxm2IE62sbqvkSD9habxBKLbRNJz4pUEsZbOLcwtxijpUjI2O6ORlC2PmcNXr0+QNLVTBn1L5Qm5yY4raHvIDD7ACD8vxX0HKwzyBx8Uez/UDseIebbZDRHG1++DCdE4/qr9PQ9Kdye3rkw9w1QqgjWC6+4AYDizC59+HszPvu75CerFZd5Y5nTX2Td4MTrKh8lNsMg0sv6ZRSnyE1aGSteVy7mmlMSAFc8dDSNQfQYw0uFLRrTpD1Jn5ycFeguWMUd9ACj7m4pHexRF6JyRdeOAzCjhKk0LFIyhcygnzEori2aJQmlYL1iqE+IWeAwuh6ryRJQ/2NzojxQ+MRw7FluIGDTOm3NF76cmi/Dqs9T33IQN4pAGTjX5yaBZOvny2jN57fwCIWzpN0Zc5X903Z+VYa9dMy93+N2k8RjkZARqxnwZSHhtqaJoZligg3xp/kKzJe99y1oO++uJXTohUhOx4iQyBlLE50j1Rbhjb5DsgvULIKxkgI4zltVHWa6pI9zNJa9QWxbfuihvvZ8rEZk+gwNcIGVaKaCIInpAlcb4YDl0Lm6qldZKByyiQ2Uu/X2KnlYXGIAsgWJLYGNwLZMaIfeovLbLresF+6pofMuVyjlaLJkT2f8MS1go8y8v9AcMJdCS1gIEIdFFhILbuGDYfQPvYeRq3Ckr1PvSzqfUMRjOC/NOtn27OKSDsX0zHQQfumKUaGmc7N9MeEA+87Xe8A2572G0gb9emUPqREyK/VyQK+ptNLaAwEN7wkzsIztvA3vLSulAMwyZrqdqUaiKHr4DsHfRaCThhvLJLRnZjAs6MNSqWz9xB2OJH19cT9C4plpO0+0Ui/xjnaq9jZrEWKntnqkSrodi5p7iLreJRPEg5TazlRfty6ZA+eXKCzWfvTU+DNMGWgbGiVcdg9qmyxPpgkx2mpvKbTAx08cQZLdSJBjEZw8wfBE4nR+TwHkMLfRoKx/hedp+UEye/hlRtJjq8n/YZ4W8da8vloPkHCWyeXsVCbSjtaGffXiEtCXX9OrBcMNkrWHkpy6yBl98QCEQ4Ri/UUImNZ+eoePvMu9RPRuOzbkflP5319/s0RumiOklX0flVFSEr5KM5uWHDKP3m2+QjVPq4ZRYjYeftd2VINhcXmo0dIdsfFVCGk4xp4m66tKL8EM54R/qU8EHJrZHzMFZjRFtYOukK5GEf1pei+4ffKj1 X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(376005)(1800799015)(36860700004)(82310400017);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:47.5272 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 66647cf5-16f1-4d7b-8f9c-08dc758f7a90 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044FA.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB7905 From: Vikram Garhwal Currently, routing/removing physical interrupts are only allowed at the domain creation/destroy time. For use cases such as dynamic device tree overlay adding/removing, the routing/removing of physical IRQ to running domains should be allowed. Removing the above-mentioned domain creation/dying check. Since this will introduce interrupt state unsync issues for cases when the interrupt is active or pending in the guest, therefore for these cases we simply reject the operation. Do it for both new and old vGIC implementations. Signed-off-by: Vikram Garhwal Signed-off-by: Stefano Stabellini Signed-off-by: Henry Wang --- v2: - Reject the case where the IRQ is active or pending in guest. --- xen/arch/arm/gic-vgic.c | 8 ++++++-- xen/arch/arm/gic.c | 15 --------------- xen/arch/arm/vgic/vgic.c | 5 +++-- 3 files changed, 9 insertions(+), 19 deletions(-) diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c index 56490dbc43..d1608415f8 100644 --- a/xen/arch/arm/gic-vgic.c +++ b/xen/arch/arm/gic-vgic.c @@ -444,14 +444,18 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq, { /* The VIRQ should not be already enabled by the guest */ if ( !p->desc && - !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) ) + !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) && + !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) && + !test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) ) p->desc = desc; else ret = -EBUSY; } else { - if ( desc && p->desc != desc ) + if ( desc && p->desc != desc && + (test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) || + test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status)) ) ret = -EINVAL; else p->desc = NULL; diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index 44c40e86de..3ebd89940a 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -135,14 +135,6 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq, ASSERT(virq < vgic_num_irqs(d)); ASSERT(!is_lpi(virq)); - /* - * When routing an IRQ to guest, the virtual state is not synced - * back to the physical IRQ. To prevent get unsync, restrict the - * routing to when the Domain is been created. - */ - if ( d->creation_finished ) - return -EBUSY; - ret = vgic_connect_hw_irq(d, NULL, virq, desc, true); if ( ret ) return ret; @@ -167,13 +159,6 @@ int gic_remove_irq_from_guest(struct domain *d, unsigned int virq, ASSERT(test_bit(_IRQ_GUEST, &desc->status)); ASSERT(!is_lpi(virq)); - /* - * Removing an interrupt while the domain is running may have - * undesirable effect on the vGIC emulation. - */ - if ( !d->is_dying ) - return -EBUSY; - desc->handler->shutdown(desc); /* EOI the IRQ if it has not been done by the guest */ diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c index b9463a5f27..785ef2b192 100644 --- a/xen/arch/arm/vgic/vgic.c +++ b/xen/arch/arm/vgic/vgic.c @@ -877,7 +877,7 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu, if ( connect ) /* assign a mapped IRQ */ { /* The VIRQ should not be already enabled by the guest */ - if ( !irq->hw && !irq->enabled ) + if ( !irq->hw && !irq->enabled && !irq->active && !irq->pending_latch ) { irq->hw = true; irq->hwintid = desc->irq; @@ -887,7 +887,8 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu, } else /* remove a mapped IRQ */ { - if ( desc && irq->hwintid != desc->irq ) + if ( desc && irq->hwintid != desc->irq && + (irq->active || irq->pending_latch) ) { ret = -EINVAL; } From patchwork Thu May 16 10:03:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13665973 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96F45C25B7A for ; Thu, 16 May 2024 10:04:09 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.722995.1127509 (Exim 4.92) (envelope-from ) id 1s7Xxu-00071L-Lc; Thu, 16 May 2024 10:03:58 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 722995.1127509; Thu, 16 May 2024 10:03:58 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxu-000714-Hx; Thu, 16 May 2024 10:03:58 +0000 Received: by outflank-mailman (input) for mailman id 722995; Thu, 16 May 2024 10:03:57 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxs-0005ER-TE for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:57 +0000 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on20607.outbound.protection.outlook.com [2a01:111:f400:7e88::607]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 99e22697-136b-11ef-b4bb-af5377834399; Thu, 16 May 2024 12:03:54 +0200 (CEST) Received: from MW4P222CA0022.NAMP222.PROD.OUTLOOK.COM (2603:10b6:303:114::27) by MW4PR12MB7216.namprd12.prod.outlook.com (2603:10b6:303:226::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55; Thu, 16 May 2024 10:03:50 +0000 Received: from CO1PEPF000044FD.namprd21.prod.outlook.com (2603:10b6:303:114:cafe::41) by MW4P222CA0022.outlook.office365.com (2603:10b6:303:114::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.28 via Frontend Transport; Thu, 16 May 2024 10:03:50 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by CO1PEPF000044FD.mail.protection.outlook.com (10.167.241.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7587.0 via Frontend Transport; Thu, 16 May 2024 10:03:50 +0000 Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May 2024 05:03:49 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May 2024 05:03:48 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Thu, 16 May 2024 05:03:46 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 99e22697-136b-11ef-b4bb-af5377834399 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SzQl4oC4aqwmsE01f/6t/ZOPpDuu2OXSOZnTZMof0AJkaz4k4e46VqcSsMgpMoQNdVoOIY/9r92ht1uSnDbZmS/kt+fdgoRaSJmJL6eggQ4O3nBWJvqwl9+URHc6UstuNbkklNvmOn6eCS1KwvsnRySGWtOyrPDil3ar02qmvwldB+tO1rbXUkIhKlF4ozM2bZ9Tf636C6yQJu7vdkw6g7kXyfZe5CP67acHtGADv5WwNLxbDuirDgpvh70TFghSih8b4CtoLtH8u3c7gQBPcBk+bny8j7TwZ5qzk/7UKC+V1c1EJFqHawJ4q9xt1/kcdSbQbi+2NBaPf36Fxigjxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rQdHNGe+exEXW3biYqAzR1o3kzypd+Kl5wlHYuzIz7g=; b=jDVarwmQNumQeCnO6vlEHV0+tGYfK0jtCfRZNR+OEaEcwDSID1rr1MtYBFxLJ2Ysocl2MAkb5xWwG1Lj9YTfQDkdbDJCLGcKH76DrUZKFRr2LvSOheMvkjm75sQKLqK2Y6oHl+ZYIZZnvHsz2UTFNB+NDasP0bIhGTUw5iDznAA4P1bYtrdANBW1kL4QPbjk1cpoZRC2l5mXGbiSdSxCC1h9SMrG2fAFuCbIZpUeX6//B4LagHk5XP2c7K/vHwqE8ytDn/K/GVvlLvXEjIXTO6G7aGqUT0cV3LtZE0lMmqTMyIEyYDCNJRv1YOn6Q6TQyhJ7HpUtdqrw8H5IhtosfA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rQdHNGe+exEXW3biYqAzR1o3kzypd+Kl5wlHYuzIz7g=; b=lvR9g0YEn//fjhoof7iZwr3o0rdpktzOe28FSOoy1Os8AkM5vjWLRkXKeAOQFSorcFaZhUHbIWdjVf2GX59EcJ7mcnYke/bL6DM6y1DEIWxygv0wI4KIzlUGHHE/wyTvQ/tvKy/0ECPPirVjk9fbPKSs8fcsIONzkLuWTCGDwsg= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Henry Wang To: CC: Henry Wang , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , "Volodymyr Babchuk" , Andrew Cooper , George Dunlap , "Jan Beulich" Subject: [PATCH v2 6/8] xen/arm: Add XEN_DOMCTL_dt_overlay DOMCTL and related operations Date: Thu, 16 May 2024 18:03:28 +0800 Message-ID: <20240516100330.1433265-7-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com> References: <20240516100330.1433265-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044FD:EE_|MW4PR12MB7216:EE_ X-MS-Office365-Filtering-Correlation-Id: 01f7cbec-ff38-4212-e416-08dc758f7c2b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|36860700004|1800799015|82310400017|376005; X-Microsoft-Antispam-Message-Info: Te+EBV2CpKCi2W/w0ahW0wrJr6MCpWgKhZ1SWwMV8+V6PEKwDkcEs/d8Zp+3kC5N7YH9r5GdbAxBzwRaNiZomJi6akssyp+zgJIGAOZWTzOVQU+EgIKjst4unpLrxCRbyRCRUGKkiNio6d9GhF9Kske1fsTH+hJ6eI4zJpT1wyQV1LwKCHDyC/hUjNjCWKvOmjGtESrkGSTmVaNrpA/h9NxYP8TXM/tD2F8tna74vbQwtoSem/zEUP/LqIvq6G7kUpdqwj84bD+ERgKl7OHwdyt47k1r0525AtrF+b1kHqhnRuDlLPD9KVQ8h/UH+C03aHe+katwI+JZAnZL6sbYzrFfCtltu8seTMtAwSgiK8+qmciCdmtmyVaS8jGNQc6S4lEWiJEbaHo2hP6fA+P3zCuGNi71X7pCwasMml7rtKZUcjJYl8hU4br2eK+LBOc8h5tkUVpA/KZnl2HCU1FEKfkvX06sB+RCqJrlQnlBN+B1s1w6d22V4WLaqa3/cM23ICHNgMeOZmpxVJ5fEQVGgytSftHAHwXciwxHUx0QERDW6j31KJEy7lPrMHUSdNjLcJhw7SJ0DjKNOJZj2bKdNfxnLzS3N5oTRlTgkkVwMLDaNi/hXuseXqRbyzY5aInvlfx/obNYPQN7WdJKFPQOMK0GKfV9mkSLNEz2yUKcAgug2RUZrdvpDIziUeOOhJexYM3BZmVlv3sFfsLAtcORE6wyX7baXpnz8GvURW2Sgf1mA9g8chZKJRwSlWnT90TvYIfKGfTNgo3wrQkAZww55r7/4NdCgwaePPuXBQ6hDxdhR9/CqvnaVn7CgANWgYTQzZpRngQDJhR0iJyEv5jplR2hY9/VndPY/lWU0r0gphI+aAHth3Ymv6gHkwcHADwnTEKDIgkpqaPOmttfEetLF6G7UBdmZT4ApP3E2pGCr6HKZV3+h2Oik5dRbNtnr472Heh/2Ou5ufOCDQYVrRFr1OSfFytV04S5iC0qrLUZg+VyAe9mi/SxK2cr49MWy8i0t4KzHT0Iz1bZhfX9TECovCLOCyT1Y1Nau/xTfmYUs2dDsko/Wv4RwX8hwkc/gKOS5TXm48H8A/dUHQIzPle6L/r9SrJSaPdrKZjH3VrNH61a2JeDdLL7MVNkXfglo5zjLIEl3RArKWQagqvoXDvKsRp+sl5uiNL6icfW6LNAr6DG0tY8AB2K37v4G0y45SrPCjocQhSLMdWic6mL7tjlItuzShmimhudlseN0ZEWQXctQ3jSjcaDpjTL2GDJvxPsh6M1zeBvT0jKsrMXcNCmJi2Bf6hfjri2MfyI8sboOLbmeaK8Hjy6GRba/onomDGgRjCuyhW2bEDewk+Vn2Tjab/3lIdOjgBmW3KOViyx5j4= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(1800799015)(82310400017)(376005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:50.2073 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 01f7cbec-ff38-4212-e416-08dc758f7c2b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044FD.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7216 In order to support the dynamic dtbo device assignment to a running VM, the add/remove of the DT overlay and the attach/detach of the device from the DT overlay should happen separately. Therefore, repurpose the existing XEN_SYSCTL_dt_overlay to only add the DT overlay to Xen device tree, instead of assigning the device to the hardware domain at the same time. Add the XEN_DOMCTL_dt_overlay with operations XEN_DOMCTL_DT_OVERLAY_{ATTACH,DETACH} to do/undo the device assignment to the domain. The hypervisor firstly checks the DT overlay passed from the toolstack is valid. Then the device nodes are retrieved from the overlay tracker based on the DT overlay. The attach/detach of the device is implemented by map/unmap the IRQ and IOMMU resources. Note that with these changes, the device de-registration from the IOMMU driver should only happen at the time when the DT overlay is removed from the Xen device tree. Signed-off-by: Henry Wang Signed-off-by: Vikram Garhwal --- v2: - New patch. --- xen/arch/arm/domctl.c | 3 + xen/common/dt-overlay.c | 415 ++++++++++++++++++++++++++++------- xen/include/public/domctl.h | 15 ++ xen/include/public/sysctl.h | 7 +- xen/include/xen/dt-overlay.h | 7 + 5 files changed, 366 insertions(+), 81 deletions(-) diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c index ad56efb0f5..12a12ee781 100644 --- a/xen/arch/arm/domctl.c +++ b/xen/arch/arm/domctl.c @@ -5,6 +5,7 @@ * Copyright (c) 2012, Citrix Systems */ +#include #include #include #include @@ -176,6 +177,8 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d, return rc; } + case XEN_DOMCTL_dt_overlay: + return dt_overlay_domctl(d, &domctl->u.dt_overlay); default: return subarch_do_domctl(domctl, d, u_domctl); } diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c index 9cece79067..593e985949 100644 --- a/xen/common/dt-overlay.c +++ b/xen/common/dt-overlay.c @@ -356,24 +356,100 @@ static int overlay_get_nodes_info(const void *fdto, char **nodes_full_path) return 0; } +static int remove_irq(unsigned long s, unsigned long e, void *data) +{ + struct domain *d = data; + int rc = 0; + + /* + * IRQ should always have access unless there are duplication of + * of irqs in device tree. There are few cases of xen device tree + * where there are duplicate interrupts for the same node. + */ + if (!irq_access_permitted(d, s)) + return 0; + /* + * TODO: We don't handle shared IRQs for now. So, it is assumed that + * the IRQs was not shared with another domain. + */ + rc = irq_deny_access(d, s); + if ( rc ) + { + printk(XENLOG_ERR "unable to revoke access for irq %ld\n", s); + return rc; + } + + rc = release_guest_irq(d, s); + if ( rc ) + { + printk(XENLOG_ERR "unable to release irq %ld\n", s); + return rc; + } + + return rc; +} + +static int remove_all_irqs(struct rangeset *irq_ranges, struct domain *d) +{ + return rangeset_report_ranges(irq_ranges, 0, ~0UL, remove_irq, d); +} + +static int remove_iomem(unsigned long s, unsigned long e, void *data) +{ + struct domain *d = data; + int rc = 0; + p2m_type_t t; + mfn_t mfn; + + mfn = p2m_lookup(d, _gfn(s), &t); + if ( mfn_x(mfn) == 0 || mfn_x(mfn) == ~0UL ) + return -EINVAL; + + rc = iomem_deny_access(d, s, e); + if ( rc ) + { + printk(XENLOG_ERR "Unable to remove %pd access to %#lx - %#lx\n", + d, s, e); + return rc; + } + + rc = unmap_mmio_regions(d, _gfn(s), e - s, _mfn(s)); + if ( rc ) + return rc; + + return rc; +} + +static int remove_all_iomems(struct rangeset *iomem_ranges, struct domain *d) +{ + return rangeset_report_ranges(iomem_ranges, 0, ~0UL, remove_iomem, d); +} + /* Check if node itself can be removed and remove node from IOMMU. */ -static int remove_node_resources(struct dt_device_node *device_node) +static int remove_node_resources(struct dt_device_node *device_node, + struct domain *d) { int rc = 0; unsigned int len; domid_t domid; - domid = dt_device_used_by(device_node); + if ( !d ) + { + domid = dt_device_used_by(device_node); - dt_dprintk("Checking if node %s is used by any domain\n", - device_node->full_name); + dt_dprintk("Checking if node %s is used by any domain\n", + device_node->full_name); - /* Remove the node if only it's assigned to hardware domain or domain io. */ - if ( domid != hardware_domain->domain_id && domid != DOMID_IO ) - { - printk(XENLOG_ERR "Device %s is being used by domain %u. Removing nodes failed\n", - device_node->full_name, domid); - return -EINVAL; + /* + * We also check if device is assigned to DOMID_IO as when a domain + * is destroyed device is assigned to DOMID_IO. + */ + if ( domid != DOMID_IO ) + { + printk(XENLOG_ERR "Device %s is being assigned to %u. Device is assigned to %d\n", + device_node->full_name, DOMID_IO, domid); + return -EINVAL; + } } /* Check if iommu property exists. */ @@ -381,9 +457,12 @@ static int remove_node_resources(struct dt_device_node *device_node) { if ( dt_device_is_protected(device_node) ) { - rc = iommu_remove_dt_device(device_node); - if ( rc < 0 ) - return rc; + if ( !list_empty(&device_node->domain_list) ) + { + rc = iommu_deassign_dt_device(d, device_node); + if ( rc < 0 ) + return rc; + } } } @@ -392,7 +471,8 @@ static int remove_node_resources(struct dt_device_node *device_node) /* Remove all descendants from IOMMU. */ static int -remove_descendant_nodes_resources(const struct dt_device_node *device_node) +remove_descendant_nodes_resources(const struct dt_device_node *device_node, + struct domain *d) { int rc = 0; struct dt_device_node *child_node; @@ -402,12 +482,12 @@ remove_descendant_nodes_resources(const struct dt_device_node *device_node) { if ( child_node->child ) { - rc = remove_descendant_nodes_resources(child_node); + rc = remove_descendant_nodes_resources(child_node, d); if ( rc ) return rc; } - rc = remove_node_resources(child_node); + rc = remove_node_resources(child_node, d); if ( rc ) return rc; } @@ -420,8 +500,7 @@ static int remove_nodes(const struct overlay_track *tracker) { int rc = 0; struct dt_device_node *overlay_node; - unsigned int j; - struct domain *d = hardware_domain; + unsigned int j, len; for ( j = 0; j < tracker->num_nodes; j++ ) { @@ -431,18 +510,15 @@ static int remove_nodes(const struct overlay_track *tracker) write_lock(&dt_host_lock); - rc = remove_descendant_nodes_resources(overlay_node); - if ( rc ) - { - write_unlock(&dt_host_lock); - return rc; - } - - rc = remove_node_resources(overlay_node); - if ( rc ) + /* Check if iommu property exists. */ + if ( dt_get_property(overlay_node, "iommus", &len) ) { - write_unlock(&dt_host_lock); - return rc; + if ( dt_device_is_protected(overlay_node) ) + { + rc = iommu_remove_dt_device(overlay_node); + if ( rc < 0 ) + return rc; + } } dt_dprintk("Removing node: %s\n", overlay_node->full_name); @@ -457,22 +533,6 @@ static int remove_nodes(const struct overlay_track *tracker) write_unlock(&dt_host_lock); } - /* Remove IRQ access. */ - if ( tracker->irq_ranges ) - { - rc = rangeset_consume_ranges(tracker->irq_ranges, irq_remove_cb, d); - if ( rc ) - return rc; - } - - /* Remove mmio access. */ - if ( tracker->iomem_ranges ) - { - rc = rangeset_consume_ranges(tracker->iomem_ranges, iomem_remove_cb, d); - if ( rc ) - return rc; - } - return rc; } @@ -536,9 +596,6 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt, xfree(entry->nodes_address); - rangeset_destroy(entry->irq_ranges); - rangeset_destroy(entry->iomem_ranges); - xfree(entry); out: @@ -620,15 +677,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path) return -EFAULT; } - rc = handle_device(hardware_domain, overlay_node, p2m_mmio_direct_c, - tr->iomem_ranges, - tr->irq_ranges); write_unlock(&dt_host_lock); - if ( rc ) - { - printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n"); - return rc; - } /* Keep overlay_node address in tracker. */ tr->nodes_address[j] = (unsigned long)overlay_node; @@ -638,9 +687,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path) } /* * Adds device tree nodes under target node. - * We use tr->dt_host_new to unflatten the updated device_tree_flattened. This - * is done to avoid the removal of device_tree generation, iomem regions mapping - * to hardware domain done by handle_node(). + * We use tr->dt_host_new to unflatten the updated device_tree_flattened. */ static long handle_add_overlay_nodes(void *overlay_fdt, uint32_t overlay_fdt_size) @@ -774,20 +821,6 @@ static long handle_add_overlay_nodes(void *overlay_fdt, goto err; } - tr->irq_ranges = rangeset_new(hardware_domain, "Overlays: Interrupts", 0); - if (tr->irq_ranges == NULL) - { - printk(XENLOG_ERR "Creating IRQ rangeset failed"); - goto err; - } - - tr->iomem_ranges = rangeset_new(hardware_domain, "Overlay: I/O Memory", 0); - if (tr->iomem_ranges == NULL) - { - printk(XENLOG_ERR "Creating IOMMU rangeset failed"); - goto err; - } - rc = add_nodes(tr, nodes_full_path); if ( rc ) { @@ -843,14 +876,205 @@ static long handle_add_overlay_nodes(void *overlay_fdt, xfree(tr->nodes_address); xfree(tr->fdt); - rangeset_destroy(tr->irq_ranges); - rangeset_destroy(tr->iomem_ranges); - xfree(tr); return rc; } +static long handle_detach_overlay_nodes(struct domain *d, + const void *overlay_fdt, + uint32_t overlay_fdt_size) +{ + int rc; + unsigned int j; + struct overlay_track *entry, *temp, *track; + bool found_entry = false; + + rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size); + if ( rc ) + return rc; + + spin_lock(&overlay_lock); + + /* + * First check if dtbo is correct i.e. it should one of the dtbo which was + * used when dynamically adding the node. + * Limitation: Cases with same node names but different property are not + * supported currently. We are relying on user to provide the same dtbo + * as it was used when adding the nodes. + */ + list_for_each_entry_safe( entry, temp, &overlay_tracker, entry ) + { + if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 ) + { + track = entry; + found_entry = true; + break; + } + } + + if ( !found_entry ) + { + rc = -EINVAL; + printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo." + " Detaching nodes is supported only for prior added dtbo.\n"); + goto out; + + } + + for ( j = 0; j < entry->num_nodes; j++ ) + { + struct dt_device_node *overlay_node; + + overlay_node = (struct dt_device_node *)entry->nodes_address[j]; + if ( overlay_node == NULL ) + { + rc = -EINVAL; + goto out; + } + + write_lock(&dt_host_lock); + rc = remove_descendant_nodes_resources(overlay_node, d); + if ( rc ) + { + write_unlock(&dt_host_lock); + goto out; + } + + rc = remove_node_resources(overlay_node, d); + if ( rc ) + { + write_unlock(&dt_host_lock); + goto out; + } + write_unlock(&dt_host_lock); + + rc = remove_all_irqs(entry->irq_ranges, d); + if ( rc ) + goto out; + + rc = remove_all_iomems(entry->iomem_ranges, d); + if ( rc ) + goto out; + } + + /* Remove IRQ access. */ + if ( entry->irq_ranges ) + { + rc = rangeset_consume_ranges(entry->irq_ranges, irq_remove_cb, d); + if ( rc ) + goto out; + } + + /* Remove mmio access. */ + if ( entry->iomem_ranges ) + { + rc = rangeset_consume_ranges(entry->iomem_ranges, iomem_remove_cb, d); + if ( rc ) + goto out; + } + + rangeset_destroy(entry->irq_ranges); + rangeset_destroy(entry->iomem_ranges); + + out: + spin_unlock(&overlay_lock); + + return rc; +} + +static long handle_attach_overlay_nodes(struct domain *d, + const void *overlay_fdt, + uint32_t overlay_fdt_size) +{ + int rc; + unsigned int j; + struct overlay_track *entry, *temp, *track; + bool found_entry = false; + + rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size); + if ( rc ) + return rc; + + spin_lock(&overlay_lock); + + /* + * First check if dtbo is correct i.e. it should one of the dtbo which was + * used when dynamically adding the node. + * Limitation: Cases with same node names but different property are not + * supported currently. We are relying on user to provide the same dtbo + * as it was used when adding the nodes. + */ + list_for_each_entry_safe( entry, temp, &overlay_tracker, entry ) + { + if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 ) + { + track = entry; + found_entry = true; + break; + } + } + + if ( !found_entry ) + { + rc = -EINVAL; + printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo." + " Attaching nodes is supported only for prior added dtbo.\n"); + goto out; + + } + + entry->irq_ranges = rangeset_new(d, "Overlays: Interrupts", 0); + if (entry->irq_ranges == NULL) + { + rc = -ENOMEM; + printk(XENLOG_ERR "Creating IRQ rangeset failed"); + goto out; + } + + entry->iomem_ranges = rangeset_new(d, "Overlay: I/O Memory", 0); + if (entry->iomem_ranges == NULL) + { + rc = -ENOMEM; + printk(XENLOG_ERR "Creating IOMMU rangeset failed"); + goto out; + } + + for ( j = 0; j < entry->num_nodes; j++ ) + { + struct dt_device_node *overlay_node; + + overlay_node = (struct dt_device_node *)entry->nodes_address[j]; + if ( overlay_node == NULL ) + { + rc = -EINVAL; + goto out; + } + + write_lock(&dt_host_lock); + rc = handle_device(d, overlay_node, p2m_mmio_direct_c, + entry->iomem_ranges, entry->irq_ranges); + write_unlock(&dt_host_lock); + if ( rc ) + { + printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n"); + goto out; + } + } + + spin_unlock(&overlay_lock); + + return 0; + + out: + spin_unlock(&overlay_lock); + + rangeset_destroy(entry->irq_ranges); + rangeset_destroy(entry->iomem_ranges); + + return rc; +} + long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op) { long ret; @@ -890,6 +1114,45 @@ long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op) return ret; } +long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op) +{ + long ret; + void *overlay_fdt; + + if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH && + op->overlay_op != XEN_DOMCTL_DT_OVERLAY_DETACH ) + return -EOPNOTSUPP; + + if ( op->overlay_fdt_size == 0 || op->overlay_fdt_size > KB(500) ) + return -EINVAL; + + if ( op->pad[0] || op->pad[1] || op->pad[2] ) + return -EINVAL; + + overlay_fdt = xmalloc_bytes(op->overlay_fdt_size); + + if ( overlay_fdt == NULL ) + return -ENOMEM; + + ret = copy_from_guest(overlay_fdt, op->overlay_fdt, op->overlay_fdt_size); + if ( ret ) + { + gprintk(XENLOG_ERR, "copy from guest failed\n"); + xfree(overlay_fdt); + + return -EFAULT; + } + + if ( op->overlay_op == XEN_DOMCTL_DT_OVERLAY_DETACH ) + ret = handle_detach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size); + else + ret = handle_attach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size); + + xfree(overlay_fdt); + + return ret; +} + /* * Local variables: * mode: C diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index a33f9ec32b..95bfb3c0e8 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -1190,6 +1190,17 @@ struct xen_domctl_vmtrace_op { typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t; DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t); +#if defined(__arm__) || defined (__aarch64__) +struct xen_domctl_dt_overlay { + XEN_GUEST_HANDLE_64(const_void) overlay_fdt; /* IN: overlay fdt. */ + uint32_t overlay_fdt_size; /* IN: Overlay dtb size. */ +#define XEN_DOMCTL_DT_OVERLAY_ATTACH 3 +#define XEN_DOMCTL_DT_OVERLAY_DETACH 4 + uint8_t overlay_op; /* IN: Attach or detach. */ + uint8_t pad[3]; /* IN: Must be zero. */ +}; +#endif + struct xen_domctl { uint32_t cmd; #define XEN_DOMCTL_createdomain 1 @@ -1277,6 +1288,7 @@ struct xen_domctl { #define XEN_DOMCTL_vmtrace_op 84 #define XEN_DOMCTL_get_paging_mempool_size 85 #define XEN_DOMCTL_set_paging_mempool_size 86 +#define XEN_DOMCTL_dt_overlay 87 #define XEN_DOMCTL_gdbsx_guestmemio 1000 #define XEN_DOMCTL_gdbsx_pausevcpu 1001 #define XEN_DOMCTL_gdbsx_unpausevcpu 1002 @@ -1339,6 +1351,9 @@ struct xen_domctl { struct xen_domctl_vuart_op vuart_op; struct xen_domctl_vmtrace_op vmtrace_op; struct xen_domctl_paging_mempool paging_mempool; +#if defined(__arm__) || defined (__aarch64__) + struct xen_domctl_dt_overlay dt_overlay; +#endif uint8_t pad[128]; } u; }; diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h index febaa4b16a..b613babdf9 100644 --- a/xen/include/public/sysctl.h +++ b/xen/include/public/sysctl.h @@ -1187,11 +1187,8 @@ DEFINE_XEN_GUEST_HANDLE(xen_sysctl_cpu_policy_t); #if defined(__arm__) || defined (__aarch64__) /* * XEN_SYSCTL_dt_overlay - * Performs addition/removal of device tree nodes under parent node using dtbo. - * This does in three steps: - * - Adds/Removes the nodes from dt_host. - * - Adds/Removes IRQ permission for the nodes. - * - Adds/Removes MMIO accesses. + * Performs addition/removal of device tree nodes under parent node using dtbo + * from dt_host. */ struct xen_sysctl_dt_overlay { XEN_GUEST_HANDLE_64(const_void) overlay_fdt; /* IN: overlay fdt. */ diff --git a/xen/include/xen/dt-overlay.h b/xen/include/xen/dt-overlay.h index c0567741ee..64c9e34a8a 100644 --- a/xen/include/xen/dt-overlay.h +++ b/xen/include/xen/dt-overlay.h @@ -14,6 +14,7 @@ #include #include #include +#include /* * overlay_track describes information about added nodes through dtbo. @@ -42,12 +43,18 @@ struct xen_sysctl_dt_overlay; #ifdef CONFIG_OVERLAY_DTB long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op); +long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op); #else #include static inline long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op) { return -EOPNOTSUPP; } +static inline long dt_overlay_domctl(struct domain *d, + struct xen_domctl_dt_overlay *op) +{ + return -EOPNOTSUPP; +} #endif #endif /* __XEN_DT_OVERLAY_H__ */ From patchwork Thu May 16 10:03:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13665977 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DAA1DC25B7B for ; Thu, 16 May 2024 10:04:11 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.722996.1127514 (Exim 4.92) (envelope-from ) id 1s7Xxv-00075v-3q; Thu, 16 May 2024 10:03:59 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 722996.1127514; Thu, 16 May 2024 10:03:59 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxu-000750-S6; Thu, 16 May 2024 10:03:58 +0000 Received: by outflank-mailman (input) for mailman id 722996; Thu, 16 May 2024 10:03:58 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxt-0005ER-TW for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:58 +0000 Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on20600.outbound.protection.outlook.com [2a01:111:f403:2405::600]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 9a731e23-136b-11ef-b4bb-af5377834399; Thu, 16 May 2024 12:03:55 +0200 (CEST) Received: from MW4PR03CA0153.namprd03.prod.outlook.com (2603:10b6:303:8d::8) by BL3PR12MB6428.namprd12.prod.outlook.com (2603:10b6:208:3b7::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.30; Thu, 16 May 2024 10:03:52 +0000 Received: from CO1PEPF000042A8.namprd03.prod.outlook.com (2603:10b6:303:8d:cafe::d7) by MW4PR03CA0153.outlook.office365.com (2603:10b6:303:8d::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.27 via Frontend Transport; Thu, 16 May 2024 10:03:51 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000042A8.mail.protection.outlook.com (10.167.243.37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 10:03:51 +0000 Received: from SATLEXMB05.amd.com (10.181.40.146) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May 2024 05:03:50 -0500 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB05.amd.com (10.181.40.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May 2024 05:03:50 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Thu, 16 May 2024 05:03:49 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9a731e23-136b-11ef-b4bb-af5377834399 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lu2btGx3SJHxfm7pkjGq8DxuXJGXF6D10E/DrZuLMG/HEaX9JRweTVtbApZv9WT01DjwKfLFzUQQWK1pVkHuNhsvPAXGZF9kKQikoBF36ZxAd9Sj2k8OgCaVy3JXzFC+dMQeI2AKuHvps0kXoj1tPvjiwmFb3anqMAAadovizni0NFEFeZDhERCuSJryj3ZMfU1ecG1K068etYljHmOde/+iD4Yd5z9BJCInwnwxVkUmiHo0Ch+DHFiTvrtDnmE4ufvAc7E0+t8Ex0SIJSelbAx2qUP0ri/GqcUi+ssGqLxzD1ysBKTs3bVLIfG5M42tuCSxFJbYWMI8tBXAeGgUBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nl/BNnEwGnkfp7ugboqdbNn2EBw+cplgSH3kLI39gkI=; b=hJgq1eHJforyIRyB/BFw6pOQvSM0F4LGPwb+T3CLIKVoS4hqNltUiG7mzDrj+U52D9cH8hQnlG7UtbT5227ZUb9j7Hl6Tb7e2w6Xzs/K40bUdqRg+MhLaDGRsAkrpOC4tbKeVqd4spRQflbfAheAivkeYmtXN3/YRlqoaHOY9jhKj/ANiro3XOu5i4/i3uRzcNbx2Nqi6DmkiRSzMmDqTVisXzdN5NeW5fAUAdz/OTpRRkSzLTzT0uMq1OXCVlgaOj+FDxi0VKs+XYkLIcmnxUse09/4O7i+rdKhCgcuPSY7Y5ea7BpSDEuc5bQOdIgVw9l6z5+YlVP7Ok/V+J/TJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nl/BNnEwGnkfp7ugboqdbNn2EBw+cplgSH3kLI39gkI=; b=eb/vQBtuMWIfNdwxD9ltn6debbnOpc+7PSbaj3lDB6p0/PYwcwmPe4UlO9oC8cDYJQ32wWKFiqXTXqnCq96MNhASRougp1MxkkFLf0jKsXhTN/6lBVO5R3UQLQE1/c2kDpKu8FbmgNdMfFDT2CAX4aYjpjMOJBY+KeLEpAY9dXo= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Henry Wang To: CC: Henry Wang , Anthony PERARD , Juergen Gross Subject: [PATCH v2 7/8] tools: Introduce the "xl dt-overlay {attach,detach}" commands Date: Thu, 16 May 2024 18:03:29 +0800 Message-ID: <20240516100330.1433265-8-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com> References: <20240516100330.1433265-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB05.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000042A8:EE_|BL3PR12MB6428:EE_ X-MS-Office365-Filtering-Correlation-Id: 1d26934f-4574-4069-9ef0-08dc758f7d19 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004; X-Microsoft-Antispam-Message-Info: TZy7jMliXsiLUTDfo09ghojyymhC1KLC0CRovvdLSdQjzUL54AxcPh/jorGvTHi4hp81jUB+BXMEtOOG5qH9Wx2AptIo83Etz1gqbVtAgtrmwWiM2Oxb32p55qxCP+IdZCDwPdVg30bpuPqdyNcNDPU5MnuiFFBsAPBtHKDlNWjFypW7tGu4Idj0fHC6FbXH2C0jvqpYio+V1U3sVXlZJiJyM13bwR9XF+2dzWhiTsDzhzkFs6owlBrwL9U9Q89lzK6gsWgEhIYeZ4yNbJa52jcGFx6exwuRlqUS5YdcgrOTDUD02Q5gNDCQl4+rOhvgBQ6Z3QAfRFrKxfMjtAY0NMHB/kACtaIMzBleS9NedG+tDR0+qrPfVSUqcZOz76VOw+GPjaYLpge6P9ruO6YXDqrt6k641UEmszBKsUVpMRub21dyzscPUF61iHhttLyzmBxQQBeulkoFg35VGQrsUBqvv0zaVvItHBB+wqvVVzCNk6Gugw2i7OU0+f9qLsAIhjFJBNa6X/li4e2HprqDppY4qD+E+adB5IJ5efqxw1wI79zti6cSqiE8L7cJW4eJZH1TfY6d2DIRQ20f5gJCESvgOEuQFtV76f9kAnz5y0U5dkihfqOUvW3BD+esTQnSiz75u3w9DacdL4vA+seCtm5lGYhQr4aGBh2MijXS2AphsCZ0ecDqI1Z8aebVjiL3YlXALdy9tYmGFBcIAnEw6C8e/T9MMqAwW1GC03lb17mB/8OaWZwJxqwtP6poabUk6ytpO2JBxKgJCSJzbrBwM9AbrborIcG2IHonTBMXSjTdeUxcriTZAOv1aW6eY3C83mKXCYeftAtZpg+LP7KtOVkh+Bm6ovJJUDmCBl/c64KkDQ9GeEMcH2+gY52S7Tiw7QDpgqxVP+dH8u+b5sOVJ3UOkO/qc7n6/zVWR3GSMRrQQuI0fLHsxcFdXnvWSDRegVWCPJ9uXeeQ7UndGQDPCdLPIKEKAjJeUH3ioW1Sxx/zenOgE5shK7eTNY6QhyUMuBhdHSx69jcOxlCDAtCPvTU31myTyPoqdZFEcWj2wbmX7RWwdB0nEzBu3tNbRZfr5xADn4FoCBYtcnKrpks3BK8agsg41s2csN0qyxTmWnxAhS0bj04IP5vmUbxJ7A0LP3YwdBVDyRkIaMXOwR+aJ6Abtrk0ckPucLjMzGVP7pxBjJbU4yCsiTAh+TSRYg3rL8VY82kCaXu33vofr21gZZEzCtTH0UjqcmHiyA5QpdImW374p4sQ/KolzOjhAShZLknXv+u7mJN7rDs5hJt04oFrYFLbZI1B6eZgUCAkDZ06ghBIt0+XY0K9ZFnKSOYH7BXSOC8CM9qIPhgr1O7SZfDWikjZE4dmzJzzhpjHgzy8dTgxeLYs95hyj+OoN0In X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:51.7809 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1d26934f-4574-4069-9ef0-08dc758f7d19 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000042A8.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6428 With the XEN_DOMCTL_dt_overlay DOMCTL added, users should be able to attach/detach devices from the provided DT overlay to domains. Support this by introducing a new set of "xl dt-overlay" commands and related documentation, i.e. "xl dt-overlay {attach,detach}". Slightly rework the command option parsing logic. Since the addition of these two commands modifies the existing libxl API libxl_dt_overlay(), also provide the backward compatible for it. Signed-off-by: Henry Wang --- v2: - New patch. --- tools/include/libxl.h | 15 ++++++++++++- tools/include/xenctrl.h | 3 +++ tools/libs/ctrl/xc_dt_overlay.c | 31 +++++++++++++++++++++++++++ tools/libs/light/libxl_dt_overlay.c | 30 ++++++++++++++++++++------ tools/xl/xl_cmdtable.c | 4 ++-- tools/xl/xl_vmcontrol.c | 33 +++++++++++++++++++---------- 6 files changed, 96 insertions(+), 20 deletions(-) diff --git a/tools/include/libxl.h b/tools/include/libxl.h index 62cb07dea6..27aab4bcee 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -2549,8 +2549,21 @@ libxl_device_pci *libxl_device_pci_list(libxl_ctx *ctx, uint32_t domid, void libxl_device_pci_list_free(libxl_device_pci* list, int num); #if defined(__arm__) || defined(__aarch64__) -int libxl_dt_overlay(libxl_ctx *ctx, void *overlay, +#define LIBXL_DT_OVERLAY_ADD 1 +#define LIBXL_DT_OVERLAY_REMOVE 2 +#define LIBXL_DT_OVERLAY_ATTACH 3 +#define LIBXL_DT_OVERLAY_DETACH 4 + +int libxl_dt_overlay(libxl_ctx *ctx, uint32_t domain_id, void *overlay, uint32_t overlay_size, uint8_t overlay_op); +#if defined(LIBXL_API_VERSION) && LIBXL_API_VERSION < 0x041900 +int libxl_dt_overlay_0x041800(libxl_ctx *ctx, void *overlay, + uint32_t overlay_size, uint8_t overlay_op); +{ + return libxl_dt_overlay(ctx, 0, overlay, overlay_size, overlay_op); +} +#define libxl_dt_overlay libxl_dt_overlay_0x041800 +#endif #endif /* diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h index 4996855944..9ceca0cffc 100644 --- a/tools/include/xenctrl.h +++ b/tools/include/xenctrl.h @@ -2657,6 +2657,9 @@ int xc_domain_cacheflush(xc_interface *xch, uint32_t domid, #if defined(__arm__) || defined(__aarch64__) int xc_dt_overlay(xc_interface *xch, void *overlay_fdt, uint32_t overlay_fdt_size, uint8_t overlay_op); +int xc_dt_overlay_domain(xc_interface *xch, void *overlay_fdt, + uint32_t overlay_fdt_size, uint8_t overlay_op, + uint32_t domain_id); #endif /* Compat shims */ diff --git a/tools/libs/ctrl/xc_dt_overlay.c b/tools/libs/ctrl/xc_dt_overlay.c index c2224c4d15..ea1da522d1 100644 --- a/tools/libs/ctrl/xc_dt_overlay.c +++ b/tools/libs/ctrl/xc_dt_overlay.c @@ -48,3 +48,34 @@ err: return err; } + +int xc_dt_overlay_domain(xc_interface *xch, void *overlay_fdt, + uint32_t overlay_fdt_size, uint8_t overlay_op, + uint32_t domain_id) +{ + int err; + struct xen_domctl domctl = { + .cmd = XEN_DOMCTL_dt_overlay, + .domain = domain_id, + .u.dt_overlay = { + .overlay_op = overlay_op, + .overlay_fdt_size = overlay_fdt_size, + } + }; + + DECLARE_HYPERCALL_BOUNCE(overlay_fdt, overlay_fdt_size, + XC_HYPERCALL_BUFFER_BOUNCE_IN); + + if ( (err = xc_hypercall_bounce_pre(xch, overlay_fdt)) ) + goto err; + + set_xen_guest_handle(domctl.u.dt_overlay.overlay_fdt, overlay_fdt); + + if ( (err = do_domctl(xch, &domctl)) != 0 ) + PERROR("%s failed", __func__); + +err: + xc_hypercall_bounce_post(xch, overlay_fdt); + + return err; +} diff --git a/tools/libs/light/libxl_dt_overlay.c b/tools/libs/light/libxl_dt_overlay.c index a6c709a6dc..9110b1efd2 100644 --- a/tools/libs/light/libxl_dt_overlay.c +++ b/tools/libs/light/libxl_dt_overlay.c @@ -41,8 +41,8 @@ static int check_overlay_fdt(libxl__gc *gc, void *fdt, size_t size) return 0; } -int libxl_dt_overlay(libxl_ctx *ctx, void *overlay_dt, uint32_t overlay_dt_size, - uint8_t overlay_op) +int libxl_dt_overlay(libxl_ctx *ctx, uint32_t domain_id, void *overlay_dt, + uint32_t overlay_dt_size, uint8_t overlay_op) { int rc; int r; @@ -57,11 +57,29 @@ int libxl_dt_overlay(libxl_ctx *ctx, void *overlay_dt, uint32_t overlay_dt_size, rc = 0; } - r = xc_dt_overlay(ctx->xch, overlay_dt, overlay_dt_size, overlay_op); - - if (r) { - LOG(ERROR, "%s: Adding/Removing overlay dtb failed.", __func__); + switch (overlay_op) + { + case LIBXL_DT_OVERLAY_ADD: + case LIBXL_DT_OVERLAY_REMOVE: + r = xc_dt_overlay(ctx->xch, overlay_dt, overlay_dt_size, overlay_op); + if (r) { + LOG(ERROR, "%s: Adding/Removing overlay dtb failed.", __func__); + rc = ERROR_FAIL; + } + break; + case LIBXL_DT_OVERLAY_ATTACH: + case LIBXL_DT_OVERLAY_DETACH: + r = xc_dt_overlay_domain(ctx->xch, overlay_dt, overlay_dt_size, + overlay_op, domain_id); + if (r) { + LOG(ERROR, "%s: Attaching/Detaching overlay dtb failed.", __func__); + rc = ERROR_FAIL; + } + break; + default: + LOG(ERROR, "%s: Invalid overlay dtb op.", __func__); rc = ERROR_FAIL; + break; } out: diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c index 1f3c6b5897..37770b20e3 100644 --- a/tools/xl/xl_cmdtable.c +++ b/tools/xl/xl_cmdtable.c @@ -634,8 +634,8 @@ const struct cmd_spec cmd_table[] = { #ifdef LIBXL_HAVE_DT_OVERLAY { "dt-overlay", &main_dt_overlay, 0, 1, - "Add/Remove a device tree overlay", - "add/remove <.dtbo>", + "Add/Remove a device tree overlay to Xen device tree, attach/detach the device to a domain", + " <.dtbo> OR <.dtbo> ", "-h print this help\n" }, #endif diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c index 02575d5d36..53d1fa3655 100644 --- a/tools/xl/xl_vmcontrol.c +++ b/tools/xl/xl_vmcontrol.c @@ -1268,32 +1268,43 @@ int main_create(int argc, char **argv) #ifdef LIBXL_HAVE_DT_OVERLAY int main_dt_overlay(int argc, char **argv) { - const char *overlay_ops = NULL; const char *overlay_config_file = NULL; void *overlay_dtb = NULL; int rc; uint8_t op; int overlay_dtb_size = 0; - const int overlay_add_op = 1; - const int overlay_remove_op = 2; + uint32_t domain_id = 0; if (argc < 2) { help("dt-overlay"); return EXIT_FAILURE; } - overlay_ops = argv[1]; - overlay_config_file = argv[2]; - - if (strcmp(overlay_ops, "add") == 0) - op = overlay_add_op; - else if (strcmp(overlay_ops, "remove") == 0) - op = overlay_remove_op; + if (strcmp(argv[optind], "add") == 0) + op = LIBXL_DT_OVERLAY_ADD; + else if (strcmp(argv[optind], "remove") == 0) + op = LIBXL_DT_OVERLAY_REMOVE; + else if (strcmp(argv[optind], "attach") == 0) + op = LIBXL_DT_OVERLAY_ATTACH; + else if (strcmp(argv[optind], "detach") == 0) + op = LIBXL_DT_OVERLAY_DETACH; else { fprintf(stderr, "Invalid dt overlay operation\n"); return EXIT_FAILURE; } + overlay_config_file = argv[optind+1]; + + if (op == LIBXL_DT_OVERLAY_ATTACH || op == LIBXL_DT_OVERLAY_DETACH) { + if (argc <= optind + 2) { + fprintf(stderr, "Missing domain ID\n"); + help("dt-overlay"); + return EXIT_FAILURE; + } else { + domain_id = strtol(argv[optind+2], NULL, 10); + } + } + if (overlay_config_file) { rc = libxl_read_file_contents(ctx, overlay_config_file, &overlay_dtb, &overlay_dtb_size); @@ -1309,7 +1320,7 @@ int main_dt_overlay(int argc, char **argv) return EXIT_FAILURE; } - rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op); + rc = libxl_dt_overlay(ctx, domain_id, overlay_dtb, overlay_dtb_size, op); free(overlay_dtb); From patchwork Thu May 16 10:03:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13665972 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 592CEC25B74 for ; Thu, 16 May 2024 10:04:08 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.722997.1127529 (Exim 4.92) (envelope-from ) id 1s7Xxw-0007aI-IV; Thu, 16 May 2024 10:04:00 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 722997.1127529; Thu, 16 May 2024 10:04:00 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxw-0007ZA-EN; Thu, 16 May 2024 10:04:00 +0000 Received: by outflank-mailman (input) for mailman id 722997; Thu, 16 May 2024 10:03:59 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s7Xxv-0005T3-4q for xen-devel@lists.xenproject.org; Thu, 16 May 2024 10:03:59 +0000 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on20601.outbound.protection.outlook.com [2a01:111:f403:200a::601]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 9cb6c228-136b-11ef-909d-e314d9c70b13; Thu, 16 May 2024 12:03:58 +0200 (CEST) Received: from PH7P220CA0105.NAMP220.PROD.OUTLOOK.COM (2603:10b6:510:32d::20) by DM4PR12MB5867.namprd12.prod.outlook.com (2603:10b6:8:66::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.29; Thu, 16 May 2024 10:03:54 +0000 Received: from CO1PEPF000042AA.namprd03.prod.outlook.com (2603:10b6:510:32d:cafe::3c) by PH7P220CA0105.outlook.office365.com (2603:10b6:510:32d::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.29 via Frontend Transport; Thu, 16 May 2024 10:03:54 +0000 Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000042AA.mail.protection.outlook.com (10.167.243.39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7587.21 via Frontend Transport; Thu, 16 May 2024 10:03:53 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 16 May 2024 05:03:52 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Thu, 16 May 2024 05:03:51 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9cb6c228-136b-11ef-909d-e314d9c70b13 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=X6cDwxO82zVZLhGS1fy16FpelQP4WNG3Ys4u65CT48A/rvuVSWZQFSn3cORmA6L6v/oLpxHRUQJ6hs718YpXuGiLiYqre2+rjnDiLWLgkCWvn4/r0bonA1Cdq+lyhCa1bemVJtNntBz3EKMekuB3zy1+nh9DWtcs0zCM4uVxT2u9fet84BH/Jg/kVdqW3J/abTA6txstnJ9c6ykvrj3C/9x769vxCBFh4Quz/9G/8Xj4jJD2l0WRvpwkPE3Vvpco9BYFLVBFCh7e3bfdRAQgfqYhUBd7NaKDm6/BxRkd5VyesAcAECA24pjlaiKi6YLRAT32qGG2BOdbo/U6DDaruA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=v/Vywe/ogvtzCaenHHy7pk7r94uYXwgaQHO2LYJtq4Y=; b=d2Ac9Yv57g5c8k4qvUqRR9iV7ICX4xZMvKKRt+EhyRK0oHE3d9x3FeXPGPUyt08kSb9Xm9RXX03IoO+LeKbGFse7Ogt+jb+q0PcI+vX2SdQsbAWfxyHxkh7UzrvNs9/4j4ZEXbuTuorgOhgHaX1G1P4oAi/+SIHNYihVUiTL3iYsPrSW9DnnBYKEbTF0TxE9kHxYL3ly3wUyC5W+UXfK0M7oyN4ohO9WLEu3cYtXz3xxLUSNG8D8yZgTi2TVTnBZ4CK4nzf8Ki8+EKSpNGh9cfiB7NCiko1z6+v6EgH5wda/j8B7Fq1TiVQqNpwDy3hfb/4lTrdZWPAozU63P0VW4w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=v/Vywe/ogvtzCaenHHy7pk7r94uYXwgaQHO2LYJtq4Y=; b=dvIZDONT3W5ainpu7h0e34Hl0u3ueCu0dg9BqYh7BlftTnMGXXKxptrQWFQcOvJFaomsHUgHWTIWeu1TAbYZgk4Ux9KeexCKzab7RnSRFNUhGfF+UREKv4+LSJ7xkj2n0D87FSpC6hvBhSNL4vp6W/xy8jxqI3mgL2VWFPAjI8w= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C From: Henry Wang To: CC: Vikram Garhwal , Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , "Volodymyr Babchuk" , Henry Wang Subject: [PATCH v2 8/8] docs: Add device tree overlay documentation Date: Thu, 16 May 2024 18:03:30 +0800 Message-ID: <20240516100330.1433265-9-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240516100330.1433265-1-xin.wang2@amd.com> References: <20240516100330.1433265-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB04.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000042AA:EE_|DM4PR12MB5867:EE_ X-MS-Office365-Filtering-Correlation-Id: ff737319-4604-4d2a-f520-08dc758f7e49 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|1800799015|82310400017|36860700004|376005; X-Microsoft-Antispam-Message-Info: AHvP9pqvWSLHwdxEjSz16tS/r58hqMsutxIYmn6IiroeSLWQyfn9kX4tpRyJAmV8/UfBjjKJm75JMrHMo941QljY9LDp+gGhLXvOLYkR2VkbcMK8ysGyEM6c8zlvdJHLaCZCZahsXb4He44Qimqaq0hfDnpXnY+9SYuU8fksT7nnTm05mOkji/3GrwgjOzKbjDurB/QrpUr9C0y6mXq+BgsAwFE4E45NEDWKtR5D2zj/vmoRNCi7sjFJvBgEqZQY8HdBcC+A8OKetwVvhBGwK/h5ZEuubLNXqCvIzLZUY3ASlwJG03P77iKvAhjZ+NRauotuj/Dr9dlNgte47Jp9jSF0UPvzt2Ywhq+H36ZnuT5OGyxe/F+30XP38lWiODlOsGz9FNd5r8328GyrP6o+vPYpAaxTClZNfVMObhmXFLM7pC+8vnZrL7UlGvqunS+fKZKNzbGGDVGdv4qrodshpStc6L9tOFgdM6DOmYJc+vIfrg4js8wLJgqAeYxcycJgV5ZoRoQ81Zwb+3JD6XDicAL//6qJ8/wzBI5im86uFr47ZLacqh/2erKSrI2DPBPXHR67hp79GM7nm39XF4lfJaL3YyXz88O6+zi2XZhiqv7BVD47hhHCZennWSGfExxZsdR7SMBkx/FG8tv5945SGSfiAbkOoPmBHREnCFmckV9rxVsuiQzXKzG84341TNepDaFaUqxaW/wyA8iZgQskrn3HBctOhmaBHpXF5iJDkB8V4RFf8mLpu0bqWliXhS7pN5Ev7sGZB99/JQLCWjy/dqNh6j3GaK3kc2XBGoLXuWtjohsnJ59VuFxnqJz2XOngKtkNBVWABJoqaSpF4OXcLz1T9IkvUufp3QF79eerG/KI3VXE8JgkK34vpt7PBTv7jtNUXQH1iHBdBQoyzs7HzeZQRP4PaBky0fTSRCH2bJx3guApuRwdrr+H6vkgWQMX5ao63aAJklHHeZPB3ywvYEDkpHs5TT904k4J05t+yiFWLOPQ3e/PNEfuWua6wLW3YDH/U1Y5xYCSNKurSSHwPjEHyV6hUL2jO9YTApdEV5L4QWYeybcX0Ue9GtP5AKT3QqMUcI61A0C0vvwFV6xlzLDweaTXs00k6egbNqaC3RlWhF/ALWCZ9bqgbF3wJa6LxZSg9MDJQITH4dIbIvtPC1NzcPinbGoc7dVthylf/Y8i7zqPxH+FNFP4pgf5dCfWbRBhJlstbnKkfKF+IM04Yo7peNMewhLYUKZJkhmhr3koGMnRC3joNAeX9MUEOjY5ry6Du3PryJOX3iVAwizV+Je8YPlmle5aGTQXcbom9kM6g1BQ6xsBTkvL5Uu+ckyb X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(36860700004)(376005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2024 10:03:53.7910 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ff737319-4604-4d2a-f520-08dc758f7e49 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000042AA.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5867 From: Vikram Garhwal Signed-off-by: Vikram Garhwal Signed-off-by: Stefano Stabellini Signed-off-by: Henry Wang --- v2: - Update the content based on the changes in this version. --- docs/misc/arm/overlay.txt | 99 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100644 docs/misc/arm/overlay.txt diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt new file mode 100644 index 0000000000..811a6de369 --- /dev/null +++ b/docs/misc/arm/overlay.txt @@ -0,0 +1,99 @@ +# Device Tree Overlays support in Xen + +Xen now supports dynamic device assignment to running domains, +i.e. adding/removing nodes (using .dtbo) to/from Xen device tree, and +attaching/detaching them to/from a running domain with given $domid. + +Dynamic node assignment works in two steps: + +## Add/Remove device tree overlay to/from Xen device tree + +1. Xen tools check the dtbo given and parse all other user provided arguments +2. Xen tools pass the dtbo to Xen hypervisor via hypercall. +3. Xen hypervisor applies/removes the dtbo to/from Xen device tree. + +## Attach/Detach device from the DT overlay to/from domain + +1. Xen tools check the dtbo given and parse all other user provided arguments +2. Xen tools pass the dtbo to Xen hypervisor via hypercall. +3. Xen hypervisor attach/detach the device to/from the user-provided $domid by + mapping/unmapping node resources in the DT overlay. + +# Examples + +Here are a few examples on how to use it. + +## Dom0 device add + +For assigning a device tree overlay to Dom0, user should firstly properly +prepare the DT overlay. More information about device tree overlays can be +found in [1]. Then, in Dom0, enter the following: + + (dom0) xl dt-overlay add overlay.dtbo + +This will allocate the devices mentioned in overlay.dtbo to Xen device tree. + +To assign the newly added device from the dtbo to Dom0: + + (dom0) xl dt-overlay attach overlay.dtbo 0 + +Next, if the user wants to add the same device tree overlay to dom0 +Linux, execute the following: + + (dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay + (dom0) cat overlay.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo + +Finally if needed, the relevant Linux kernel drive can be loaded using: + + (dom0) modprobe module_name.ko + +## Dom0 device remove + +For removing the device from Dom0, first detach the device from Dom0: + + (dom0) xl dt-overlay detach overlay.dtbo 0 + +NOTE: The user is expected to unload any Linux kernel modules which +might be accessing the devices in overlay.dtbo before detach the device. +Detaching devices without unloading the modules might result in a crash. + +Then remove the overlay from Xen device tree: + + (dom0) xl dt-overlay remove overlay.dtbo + +## DomU device add/remove + +All the nodes in dtbo will be assigned to a domain; the user will need +to prepare the dtb for the domU. For example, the `interrupt-parent` property +of the DomU overlay should be changed to the Xen hardcoded value `0xfde8`. +Below assumes the properly written DomU dtbo is `overlay_domu.dtbo`. + +User will need to create the DomU with below properties properly configured +in the xl config file: +- `iomem` +- `passthrough` (if IOMMU is needed) + +User will also need to modprobe the relevant drivers. + +Example for domU device add: + + (dom0) xl dt-overlay add overlay.dtbo # If not executed before + (dom0) xl dt-overlay attach overlay.dtbo $domid + (dom0) xl console $domid # To access $domid console + +Next, if the user needs to modify/prepare the overlay.dtbo suitable for +the domU: + + (domU) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay + (domU) cat overlay_domu.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo + +Finally, if needed, the relevant Linux kernel drive can be probed: + + (domU) modprobe module_name.ko + +Example for domU overlay remove: + + (dom0) xl dt-overlay detach overlay.dtbo $domid + (dom0) xl dt-overlay remove overlay.dtbo + +[1] https://www.kernel.org/doc/Documentation/devicetree/overlay-notes.txt