From patchwork Tue May 21 04:35:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13668904 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D7830C25B74 for ; Tue, 21 May 2024 04:36:23 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.726499.1130802 (Exim 4.92) (envelope-from ) id 1s9HEU-0007CE-8J; Tue, 21 May 2024 04:36:14 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 726499.1130802; Tue, 21 May 2024 04:36:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HEU-0007C3-4I; Tue, 21 May 2024 04:36:14 +0000 Received: by outflank-mailman (input) for mailman id 726499; Tue, 21 May 2024 04:36:12 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HES-0006eS-MX for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:12 +0000 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on20600.outbound.protection.outlook.com [2a01:111:f403:2407::600]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a52348c6-172b-11ef-909f-e314d9c70b13; Tue, 21 May 2024 06:36:10 +0200 (CEST) Received: from SA9PR13CA0045.namprd13.prod.outlook.com (2603:10b6:806:22::20) by CH3PR12MB7499.namprd12.prod.outlook.com (2603:10b6:610:142::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Tue, 21 May 2024 04:36:05 +0000 Received: from SA2PEPF00001507.namprd04.prod.outlook.com (2603:10b6:806:22:cafe::22) by SA9PR13CA0045.outlook.office365.com (2603:10b6:806:22::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.17 via Frontend Transport; Tue, 21 May 2024 04:36:05 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by SA2PEPF00001507.mail.protection.outlook.com (10.167.242.39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:04 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May 2024 23:36:02 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Mon, 20 May 2024 23:36:01 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a52348c6-172b-11ef-909f-e314d9c70b13 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZbHT8CJrUUQY5hs5537SOck+tLTRvrprlKSawMwfvSCMiwpbiKfn4KsvUGkEpLx2W1oIbUuZ5AI2j8ISgrJB/an5qViiNdtVfH6smVhqZg3ai4jEGiXcIQU+0yuwLX7B4TmT/RF7gh0CiT1Z9bNuYwJvaovfKt8wOIbugPkfP5KHAzeyuPhFvo6iXZ5bYGUXMi2JEzSfIjMcANzygWeWtobzrqzt7aYjVkN0EV6Ucv3nX2Xat/vot1DWcmjdfEUw1PBSR4xvhVvPyuYrnxS3kQrOAdUpcYTULPxc82Lp2bS+C6o9iEcbnCL4RCts4ymVn6XYMJr67meK1SO7sE2ZTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kBS9p4N9tMcJAJGNweW/eDIIQQk4+w4cUu2XcNfNYwg=; b=TQjlVF8XqVN7gQWYuaMlIM1P77lTW2lHcGKf0XbJPvH4jXv/fj4gatbk+VED5CKR0+2y5icmSeMx1MOLBv5JG8802TAizCgomcwrIjw1EYJtlHhqFcFDHoYrshWGbNFFGfZUbeaTlfp7MY/KO7XCycRbivGYUduvRvIHX0Gsta4HeV2m3IvaoTpSOALCL6y7OnYkgMnCzgEikMxv4qxks2ji8X5sbeDLNtveC6h64h37rcqzxFQfBBW4UdhtUiEnYpQSGBzJcmM8pTAbB8VEemTNPNM0xhIH/DMeArrASVXvsxLAPG8sjHb46B0GOx3SrokwlqpD12KONnsPgQKdRA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kBS9p4N9tMcJAJGNweW/eDIIQQk4+w4cUu2XcNfNYwg=; b=m6XNOcaiydaSRylGgFvLSPAujP/G2QNPXWxNeL3XajMYzEgztIjAJH8Kyhbs+xQQdl+K2KRwNycmM2n4Afsa/hBk9nbepNG2ATUcgiqW1oJNUt3egpyRd8MY3/fLp71qEyZxkQJejQA9yrwf6YogMrVgP13ENgDFDhqCi4UIJbQ= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Henry Wang To: CC: Stefano Stabellini , Julien Grall , Henry Wang , Julien Grall Subject: [PATCH v3 1/8] xen/common/dt-overlay: Fix lock issue when add/remove the device Date: Tue, 21 May 2024 12:35:50 +0800 Message-ID: <20240521043557.1580753-2-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com> References: <20240521043557.1580753-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00001507:EE_|CH3PR12MB7499:EE_ X-MS-Office365-Filtering-Correlation-Id: 227e150c-a10e-4550-9bf4-08dc794f86b6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|36860700004|82310400017|1800799015|376005; X-Microsoft-Antispam-Message-Info: sQqQ87UkyLKxgaGUSQWaQotMsMysHeA3ERrkt1ttq82ARRub86jbMIa+TVabKFatSTw5BrVfMfr9n0Ltym/gB0tTd+Gs7INIkUKy5bQ8pglJlQoxqK/h+e8vPrA+ilqa2/pVqjlbaecjvHKZDxf04RqvYJLvEEwYegxob7UrJaWHNlNvu7b1cZ+bIhGPLYWjFNJmwDscAqZ44lHVehD0Qg5u6faqtPa6DoFG229+xpjuHQ7IDRUp3oKaJiy9/hNEHQ2Yar5ee88HEMndUAC/gNMAVtglv9dYRU9ZDrGIO+D3KYEjlk1UDoczh03+P+YGs2agmLqd7rb2JWRCk90/zxWZ7ROBLk0vk7vvdu95UQhpyUUP2KICR0FSpZfRPDCCDs/9xEvXBHwmwmj+v6fzVeIFy+LZ3aEZIJ8gxBKWkFryIKECPWiGN0+LREGja6FCEojX5EX0VZ8klQWCfBIUYv1F9HB6WV7B/SaS7bnjZBGrnt0ETeAQ0ScNYO53TXD/lFQtrW7EXCeV0kgVzwAmCrSTtp2ytLDIV9TcxAMZCigcY7BufVol5IEhw+VDwQIYZcHtDQPCPsE7/gOQ5Wo0pm9bJHFZ/wjKCAS+SiQtwIyQhWfXQ58ji0Xx2EEc7bKxQZfrVyMKh0ECcUDw1XP33ijGP+UBJ2TqpIqbSPM5Yyvn+VL356p1HRx60EL4DRlNbDg6ZA7H9g6n31yWQI3mxEnH+R5zJJiLXSjXLN4kXbSURda/dCeFyosOsW7yQmohT7Fo+67IeK5/oH2GMdpSpEMFqJlv1K3k5CYvKNvoW6Okse6w5fG17Z+I09MXkgmxKVayftjKV96xrYbiaUjmYUWLcoA+uiQG0s1CbBr8PHcFYeH7FxtCxRD8/Wd2nJssHtgSs9oUpHJarssSVQztRxSBLhtEF1QNT+KgDwgei6XUxCSIRCX5rGAt6TFdTq31H+ihqMGN8e01g/bLIPNRtg7OSWcsn8ec5FVesqJPfqcJPobQleSBBoUOXIzuyhj3hAq+yU9hdeaEp5MUrTPUNJpw8yHbSZefevJ9RLNS6ToAXL1aqCHrjU+5tFIyzHBE7kHK5O1PCDnOvvGKyGJsCDgkFed+9BpcgVkILPHpTiK0Qq7EoAaA+fL3+M9VCS1wnotFL3oSSRSd9fj0nAxgQE1/1KXUkWW7FbH9ZGSKL6CS+4EeMzilOnwp8CLZkTujlMrIrHa0umRlK3wsfu91Qn2FYfT7B9yl/J6lrcmSukZrKnTaZh7ddQiVtmMsJakV+wwXDDTi0IoDP+xdFAhTA8zL/wZ5n4pI0UQkCePzJON7qmwpkjOnDXj8lROUSrrMm3tlZEnX2xx1nawnK9ZPR9K5IKYe6SPcvOWJejlapFo= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(82310400017)(1800799015)(376005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:04.8790 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 227e150c-a10e-4550-9bf4-08dc794f86b6 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00001507.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB7499 If CONFIG_DEBUG=y, below assertion will be triggered: (XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146 (XEN) ----[ Xen-4.19-unstable arm64 debug=y Not tainted ]---- [...] (XEN) Xen call trace: (XEN) [<00000a0000257418>] iommu_remove_dt_device+0x8c/0xd4 (PC) (XEN) [<00000a00002573a0>] iommu_remove_dt_device+0x14/0xd4 (LR) (XEN) [<00000a000020797c>] dt-overlay.c#remove_node_resources+0x8c/0x90 (XEN) [<00000a0000207f14>] dt-overlay.c#remove_nodes+0x524/0x648 (XEN) [<00000a0000208460>] dt_overlay_sysctl+0x428/0xc68 (XEN) [<00000a00002707f8>] arch_do_sysctl+0x1c/0x2c (XEN) [<00000a0000230b40>] do_sysctl+0x96c/0x9ec (XEN) [<00000a0000271e08>] traps.c#do_trap_hypercall+0x1e8/0x288 (XEN) [<00000a0000273490>] do_trap_guest_sync+0x448/0x63c (XEN) [<00000a000025c480>] entry.o#guest_sync_slowpath+0xa8/0xd8 (XEN) (XEN) (XEN) **************************************** (XEN) Panic on CPU 0: (XEN) Assertion 'rw_is_locked(&dt_host_lock)' failed at drivers/passthrough/device_tree.c:146 (XEN) **************************************** This is because iommu_remove_dt_device() is called without taking the dt_host_lock. dt_host_lock is meant to ensure that the DT node will not disappear behind back. So fix the issue by taking the lock as soon as getting hold of overlay_node. Similar issue will be observed in adding the dtbo: (XEN) Assertion 'system_state < SYS_STATE_active || rw_is_locked(&dt_host_lock)' failed at xen-source/xen/drivers/passthrough/device_tree.c:192 (XEN) ----[ Xen-4.19-unstable arm64 debug=y Not tainted ]---- [...] (XEN) Xen call trace: (XEN) [<00000a00002594f4>] iommu_add_dt_device+0x7c/0x17c (PC) (XEN) [<00000a0000259494>] iommu_add_dt_device+0x1c/0x17c (LR) (XEN) [<00000a0000267db4>] handle_device+0x68/0x1e8 (XEN) [<00000a0000208ba8>] dt_overlay_sysctl+0x9d4/0xb84 (XEN) [<00000a000027342c>] arch_do_sysctl+0x24/0x38 (XEN) [<00000a0000231ac8>] do_sysctl+0x9ac/0xa34 (XEN) [<00000a0000274b70>] traps.c#do_trap_hypercall+0x230/0x2dc (XEN) [<00000a0000276330>] do_trap_guest_sync+0x478/0x688 (XEN) [<00000a000025e480>] entry.o#guest_sync_slowpath+0xa8/0xd8 This is because the lock is released too early. So fix the issue by releasing the lock after handle_device(). Fixes: 7e5c4a8b86f1 ("xen/arm: Implement device tree node removal functionalities") Signed-off-by: Henry Wang Reviewed-by: Julien Grall --- v3: - Add Julien's Reviewed-by tag. v2: - Take the lock as soon as getting hold of overlay_node. Also release the lock after handle_device() when adding dtbo. v1.1: - Move the unlock position before the check of rc. --- xen/common/dt-overlay.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c index 1b197381f6..9cece79067 100644 --- a/xen/common/dt-overlay.c +++ b/xen/common/dt-overlay.c @@ -429,18 +429,24 @@ static int remove_nodes(const struct overlay_track *tracker) if ( overlay_node == NULL ) return -EINVAL; + write_lock(&dt_host_lock); + rc = remove_descendant_nodes_resources(overlay_node); if ( rc ) + { + write_unlock(&dt_host_lock); return rc; + } rc = remove_node_resources(overlay_node); if ( rc ) + { + write_unlock(&dt_host_lock); return rc; + } dt_dprintk("Removing node: %s\n", overlay_node->full_name); - write_lock(&dt_host_lock); - rc = dt_overlay_remove_node(overlay_node); if ( rc ) { @@ -604,8 +610,6 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path) return rc; } - write_unlock(&dt_host_lock); - prev_node->allnext = next_node; overlay_node = dt_find_node_by_path(overlay_node->full_name); @@ -619,6 +623,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path) rc = handle_device(hardware_domain, overlay_node, p2m_mmio_direct_c, tr->iomem_ranges, tr->irq_ranges); + write_unlock(&dt_host_lock); if ( rc ) { printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n"); From patchwork Tue May 21 04:35:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13668903 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 434E7C25B7B for ; Tue, 21 May 2024 04:36:22 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.726498.1130788 (Exim 4.92) (envelope-from ) id 1s9HES-0006lZ-UE; Tue, 21 May 2024 04:36:12 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 726498.1130788; Tue, 21 May 2024 04:36:12 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HES-0006l6-MJ; Tue, 21 May 2024 04:36:12 +0000 Received: by outflank-mailman (input) for mailman id 726498; Tue, 21 May 2024 04:36:12 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HES-0006hu-3W for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:12 +0000 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on20601.outbound.protection.outlook.com [2a01:111:f403:2414::601]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a4385c6f-172b-11ef-b4bb-af5377834399; Tue, 21 May 2024 06:36:08 +0200 (CEST) Received: from SA9PR13CA0057.namprd13.prod.outlook.com (2603:10b6:806:22::32) by DM6PR12MB4252.namprd12.prod.outlook.com (2603:10b6:5:211::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Tue, 21 May 2024 04:36:06 +0000 Received: from SA2PEPF00001507.namprd04.prod.outlook.com (2603:10b6:806:22:cafe::46) by SA9PR13CA0057.outlook.office365.com (2603:10b6:806:22::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.17 via Frontend Transport; Tue, 21 May 2024 04:36:06 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by SA2PEPF00001507.mail.protection.outlook.com (10.167.242.39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:06 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May 2024 23:36:03 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Mon, 20 May 2024 23:36:02 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a4385c6f-172b-11ef-b4bb-af5377834399 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gBWTSlySG9p9Ae6RjL0C+Kr51gF7ITt0cBlVY003oDwljuC3e9lwBPuxooeRQdBOWr3H94GjZkD41AkAb+rjvf46Eic8gferqRxss5IujoWAnoqf56bpwxcAckFBOq8f8lP/wa08KDAk6BQxP48bDJBpAjJHffOg6FwFnRYnadvEkwdpjYOgKc17RrP3ekfw6IiHIRoGMEyZxdMKJgUCLX7J/8rc3vJCX+l0RuygBaMx+i+KABiQMoa+vmhfeCQgLRZ71Q/nT8Lf4P9VzFa4MZTl19nm5Ssy3SoeewJNnqxTPMPoUiZu2B3vGX+13OSI4k5gn3YA3GC3cPRh5nCUmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6MgtBjlO1A8ilW5oFOin/8HiTsVKeBou6t2YnvlwNys=; b=oeHMEoOkkpR1cbiCvXZL4B/zvvoKPa3pGw6i4hVo0NuA7I1pOQSHJyWqia0FcCiYpzdzErUvp/phEaUtZ/ps8uz3bC2V0T47KX7FZgFcH8Szsf4dnq9zBdNf/rcOk0ZGKBSz7uGBfnj0//PzUr/bCDms0MNPbu+cs0+XGwNt1Yvqz39STI2f77t7QXfGv4cdnRiUUU0Kx+X/0iubBUFla7/MtHzeW3P0d2HtVnV/PSd0l1aDfCzPv1IWxRmTsfb3App75fGTZ6qtj13ToDdkpSXMQzXWwWa4/gmqhHQmIHmc0zSukJgDeBrD8EyVGmzTS+2AAJtFGyPZZ0HkJSfzyw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6MgtBjlO1A8ilW5oFOin/8HiTsVKeBou6t2YnvlwNys=; b=oRW02LtyEF6Ox24cpEq+YTKbIq6Hr82X3gFFA2uJZ33twsZLH+coinuNIRl/FdLFnNaz/hUnKGvf9RjBcQksFHZi3u5a4RjkHC5dwyr2NfnRWlR6q0Mt/DB3iT2ppJjcyFZtzW/5Zy6Rx76OM0pGS8COg3AtiF94FH15zRTWq94= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Henry Wang To: CC: Anthony PERARD , Henry Wang , Jason Andryuk Subject: [PATCH v3 2/8] tools/xl: Correct the help information and exit code of the dt-overlay command Date: Tue, 21 May 2024 12:35:51 +0800 Message-ID: <20240521043557.1580753-3-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com> References: <20240521043557.1580753-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00001507:EE_|DM6PR12MB4252:EE_ X-MS-Office365-Filtering-Correlation-Id: effbd565-7baf-44a3-f460-08dc794f8791 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|1800799015|36860700004|82310400017|376005; X-Microsoft-Antispam-Message-Info: cz8HrDpXqiDvLVcZTeBBGTQdK1ZTcd9yoVmYA4g/cC/7cYlKEJ9SqPkw7lUYjbHTO6IsWNsBf3/8pj1kStE+2s7Ri5acSonboWaveYOvpFpSOUJpqe+DpAfxsxTyRxmdto+5UZuJKFYZJ/NneEUDwSecZLmGYlWrKR5ToCTex4mWLCmO9ddfs4VKu1FJLpNHbPxihnD4idiyrTCdUWiOagJtY/75L4uwK0sSp1HVdbFDbJtAvC8tLImRNNUcHDlpleKU+3pnzfvkTH9sxn7nuREh1rF5otmLh2zZBhpY8MpDFJ6lYdmrzjI3w/HNqs3imEpkBzGoNLDI7Ep66RFCQOhNmCrboS+IBNDO5XcrC0jOILIaV3P3xa2TDGRmxHE8NbNg91HrsKcx/KxuN6AI8adwOkpwCoN0pCOINnNVu0SOxeeWy0TkeEG9XYg+kPtR+7s3iU6nwaedY3ub8IzGBHpCDBBB4daE06ZvYXA69i1ta7HRRzZ8EX5fZ/089ywT1Omx5ZwZ3FHOX3eXaFPtOGxdssTLzGwuQ3sKphsFWLuhjM7cN83grRw6x/khiTEn/EjGkYgt82TE479GTE0A0EiWmyLjHqlX0ZY6nXEyfEQCfHoQMPIvUPigxPNt17r6RVn07QrqoCOSLwhT/im6lGnq814NIVHQoWpEtPbXWsrBON9NWLFTHGw1qd5gdDpAjKGDyORLleLLHmQsH2sNb0yr+Mf4LBcoR4tl+MSBVSpF/nMTweJ4TLHFF0pnIznZkLKurCmRCKAfXFlyrezmGGB1QomkVFwDnAL4Gyj5hKhLi8CWlDaq6obW9Y2/EVlfJHeQEPXNDjWH3no1Nf7Uzg9l7icKHqtGWA3HbhIAS6MBUu1wy1ZWmAIdyLFWOsA8K1Tviw3ORBEvh2XlU7SsN8+fPI/KFUJLLBosfq15wWS/sXBVTBewJGOuv8pFOnvx4q1turwr9w5t4mIFTDNpTIeicvNkqUUFYtQ0hz+SQiZgxrhookwDGRHS5S3lB8BLjcj1nrJu13u4HK4WUoP/s2D+YAZruEUfiayJrJRMnKINdwxkPzCAyaQkBSojwVmKen6lBc2GeOidLYN/EhDXfWW/jNbCb5sMY4r2Z1Meby0ZK065GNws52rNr8sOWaNBhh9FMxygQa8jSxi6vsy4EviTh0RufkRR7d8VCNVHSV1ZWGHKNGbko7XOnbc0jCYVqrTM8gR8QCTMw84slNTIUgVV5uJw2KWLOQ4RREZs7O5hDnoilTjtjVLO75GO7kHZ7YL6RknP/2b3WVC7uDf9RVTFb9JC24FrDklPUfI9wnohQ2cKC1OXteqrb53bmnNLunLCt1/ufxDoBrgkRfkpX41YPcnRbHYqps3XcxfIYmY= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(36860700004)(82310400017)(376005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:06.3009 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: effbd565-7baf-44a3-f460-08dc794f8791 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00001507.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4252 Fix the name mismatch in the xl dt-overlay command, the command name should be "dt-overlay" instead of "dt_overlay". Add the missing "," in the cmdtable. Fix the exit code of the dt-overlay command, use EXIT_FAILURE instead of ERROR_FAIL. Fixes: 61765a07e3d8 ("tools/xl: Add new xl command overlay for device tree overlay support") Suggested-by: Anthony PERARD Signed-off-by: Henry Wang Reviewed-by: Jason Andryuk --- v3: - Add Jason's Reviewed-by tag. v2: - New patch --- tools/xl/xl_cmdtable.c | 2 +- tools/xl/xl_vmcontrol.c | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c index 62bdb2aeaa..1f3c6b5897 100644 --- a/tools/xl/xl_cmdtable.c +++ b/tools/xl/xl_cmdtable.c @@ -635,7 +635,7 @@ const struct cmd_spec cmd_table[] = { { "dt-overlay", &main_dt_overlay, 0, 1, "Add/Remove a device tree overlay", - "add/remove <.dtbo>" + "add/remove <.dtbo>", "-h print this help\n" }, #endif diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c index 98f6bd2e76..02575d5d36 100644 --- a/tools/xl/xl_vmcontrol.c +++ b/tools/xl/xl_vmcontrol.c @@ -1278,7 +1278,7 @@ int main_dt_overlay(int argc, char **argv) const int overlay_remove_op = 2; if (argc < 2) { - help("dt_overlay"); + help("dt-overlay"); return EXIT_FAILURE; } @@ -1302,11 +1302,11 @@ int main_dt_overlay(int argc, char **argv) fprintf(stderr, "failed to read the overlay device tree file %s\n", overlay_config_file); free(overlay_dtb); - return ERROR_FAIL; + return EXIT_FAILURE; } } else { fprintf(stderr, "overlay dtbo file not provided\n"); - return ERROR_FAIL; + return EXIT_FAILURE; } rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op); From patchwork Tue May 21 04:35:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13668906 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E0136C25B75 for ; Tue, 21 May 2024 04:36:23 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.726501.1130817 (Exim 4.92) (envelope-from ) id 1s9HEV-0007Uv-SS; Tue, 21 May 2024 04:36:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 726501.1130817; Tue, 21 May 2024 04:36:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HEV-0007Tw-MX; Tue, 21 May 2024 04:36:15 +0000 Received: by outflank-mailman (input) for mailman id 726501; Tue, 21 May 2024 04:36:13 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HET-0006eS-Ms for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:13 +0000 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on20600.outbound.protection.outlook.com [2a01:111:f403:2417::600]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id a5d4fe5f-172b-11ef-909f-e314d9c70b13; Tue, 21 May 2024 06:36:11 +0200 (CEST) Received: from SA9PR13CA0045.namprd13.prod.outlook.com (2603:10b6:806:22::20) by SA1PR12MB6775.namprd12.prod.outlook.com (2603:10b6:806:25a::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Tue, 21 May 2024 04:36:07 +0000 Received: from SA2PEPF00001507.namprd04.prod.outlook.com (2603:10b6:806:22:cafe::23) by SA9PR13CA0045.outlook.office365.com (2603:10b6:806:22::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.17 via Frontend Transport; Tue, 21 May 2024 04:36:07 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by SA2PEPF00001507.mail.protection.outlook.com (10.167.242.39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:07 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May 2024 23:36:05 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Mon, 20 May 2024 23:36:04 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a5d4fe5f-172b-11ef-909f-e314d9c70b13 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=C17Pi7tJGLw4DxXwSM9ESWg+rU7/DL+BNHCHdzx108xL3y0u1SAhsITSJG/V+hZ6e2Mw83HLbVmKSf19vVmx5Dfwd2qmvf6PJE+z52vOzDBDCxdXDijE6a8F6rOzeaAcPXlDNKUA69J27tHRj+k1vQn8hng/rJiykmgbs4dRHvvFeS3mj06/6T2uuxSLVEQxlHWyuilGtbngxRogVBKH+17igtCWP6/pbhN+JgPCDFOK5q5yBttJiTmkTXvn0QZ64+dcLoIJDoJLO1hCtUDMpTJ4cPPah0bWKjgRoHuQB21f+LBpfS6xqD1X19o7mgdMHGbobn1uIm3KU59HOWe2ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FuT4kemK+kjHrj50JMbp96wCovrhPDOGjTghim1ZSdM=; b=bjAXEwE2weqPzRyAftaegUnj/bQU5S9W+wpTYVqmqVnEGNrt3dC3BvezD/7WbBfg7Ya8Dy+Bt21s0NC0jUGbc+hJiL61lzIt5j40ODG5U3Kz2sW99tzqmbZRZhE/+Oq8aVgAm4Tg3b9Pcabh6M8Y5R1gU8qOtJDLee5z5ShoanKPr2Aqp4qgu7lpk8Nox6MVkk1iVCLUM+zjyKaoftFmHaxYwy234Xgbe+srYG94RwPL/eqX8k8CCLooFtOYfbFXdnrWrlGzUN5CRGWi1nUv58SkLCEyyY38JWNNzcl8COGHcOxtB4HPWZfKUkSPmNA6+A5U5UODmg8JCOxwQU741Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FuT4kemK+kjHrj50JMbp96wCovrhPDOGjTghim1ZSdM=; b=OyewpQFMLsLzwHRRGnuZZR+b9WXn9PLZ7dv9WAkvDvye5aTGKcelpFrl24Uiub4fZPFuJiA828qlNfOL4N3+Gn2BAZ2g67P2StzAmJ7jGcZoj9Rit+Pgn1Ogo0WVegm8eCaoiGZxdxGoBED/sXBjl9LgqJgH7e3nvCdViME6z+k= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Henry Wang To: CC: Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , "Henry Wang" Subject: [PATCH v3 3/8] xen/arm, doc: Add a DT property to specify IOMMU for Dom0less domUs Date: Tue, 21 May 2024 12:35:52 +0800 Message-ID: <20240521043557.1580753-4-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com> References: <20240521043557.1580753-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00001507:EE_|SA1PR12MB6775:EE_ X-MS-Office365-Filtering-Correlation-Id: e98ba37c-bd5c-4a95-9f03-08dc794f881e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|82310400017|36860700004|376005|1800799015; X-Microsoft-Antispam-Message-Info: OjUZlyRjaxo9UymNdsxuU/VMmz908IooKJ5Hq41wYYPUxw0jvP/J1SH0Dgzg9Qw6vUhe77juLSOdALoUTJ7MPOQ2NGQX6Oaj/zLaT+nY+J3cFE2LZCd7N50boG8Ftl0iVGrIYTH/jYcWQf20eWXBh6wj2yFECT+obqpij9ApHlPJ4piJxhmPzIU9AlVjcq/3E0QacQqHvjewJWMI1BcHeZEBjBUuB1aB9ayCQFlW62SGH4uga3qCMyvT+3ejtZOvRdVdJF/laNLdJ7hYtvyVPnc7oHyjChDvXivAUKrQASIMmShsFZEDZeWEHcqSs6qRebPdjkzCIRrpRfh4BeZkoRI8JC7CoxF3hxCrl9gi5GYKZrqrM7W0Uq2/4c+rOeE2VKZYKTioXmNUOu9QRaZSG2DnwVB1UqqLaKhBrGPeswld3orys+i+IffK34ObGxA06R+WODQvVmQyTmy55HaxVA2hvVIE/Cy8X6cqfR2aIcmlPvboh0C4zW4HN1f8SPtJ/EFGgHQrtcB76E4aYFPiemGGiktFjpSLye93VpyMgo6UFUl257Mt9jeF5TC4D87vv7JZZXpmzfN5CS+3QorHRaJ4hqn181uz+HALBFmeLw8N8faFLmV7pBlySY09CGOMxAriS3BiU6ubzPgzVQATn9tfVefES7qLSyAcVUZJTDlUEgTbF1NgpvOhe+Baxui5rCA3WcttrhGJmC9iG4cZgpdK6ec3dIzi2QwaLymNxwbpJVCiTPbE4nNncham2PJOWBJrVU3m9RdZUEzXzzV/XSlWwM49E0TedO8VAzJzB1ZcYH8J3Fm72wfXdURKiykfqw64Scaq67tFYvtE6L2l6iMRdgP6WlmZ/FTR6dUsNwewZkOeLi3324QGNga9hQUWT+2UpavGXQxrk0hvlwped46TBMVAYpnyIxKKz04Y2A00serVA/tjlu0NCZHi6k33AY7EHYA1YmwHZSWyQ0VaKU1GYFzR+p7DsOMwS4rcih1ncND5r7ymh4gks3QYcUOJPlYE66v/pMU8W5fyHST/IQoHhEdLd1+eAonoNgrZEpnOlIqsKxPYJSXq68N5DdBoJrT0SdS6oBpPVEnRCCvciINCgmzhGv3ejDNPTn2AMk1gjyU303g9Kzo37LiGvRNKPoyC7DiD8H3+3srzKGIrArFbR5bPLsJvnN77/DObSTNzSN6baxwYVR1dfbwbPZDkt/2uDRKsvjPw5jSGgaattr1ThGyioCP1498OyTnNmFuepnqRvxQSHUeCzvagMwNOqrdYetjMLT9A3PYyYk4VZC4dYXepr92uVHatVjuE36Dm8MlP5dUf+gukoZCvTw/PjcWYli7ApBV5AaOFrXOgag4xKvD9tngxtwBhG3jUZOFQp2O/kuR4lgAwSCZa+pXo X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(36860700004)(376005)(1800799015);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:07.2384 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e98ba37c-bd5c-4a95-9f03-08dc794f881e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00001507.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6775 There are some use cases in which the dom0less domUs need to have the XEN_DOMCTL_CDF_iommu set at the domain construction time. For example, the dynamic dtbo feature allows the domain to be assigned a device that is behind the IOMMU at runtime. For these use cases, we need to have a way to specify the domain will need the IOMMU mapping at domain construction time. Introduce a "passthrough" DT property for Dom0less DomUs following the same entry as the xl.cfg. Currently only provide two options, i.e. "enable" and "disable". Set the XEN_DOMCTL_CDF_iommu at domain construction time based on the property. Signed-off-by: Henry Wang --- v3: - Use a separate variable to cache the condition from the "passthrough" flag separately to improve readability. - Update the doc to explain the default condition more clearly. v2: - New patch to replace the original patch in v1: "[PATCH 03/15] xen/arm: Always enable IOMMU" --- docs/misc/arm/device-tree/booting.txt | 16 ++++++++++++++++ xen/arch/arm/dom0less-build.c | 11 +++++++++-- 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/docs/misc/arm/device-tree/booting.txt b/docs/misc/arm/device-tree/booting.txt index bbd955e9c2..f1fd069c87 100644 --- a/docs/misc/arm/device-tree/booting.txt +++ b/docs/misc/arm/device-tree/booting.txt @@ -260,6 +260,22 @@ with the following properties: value specified by Xen command line parameter gnttab_max_maptrack_frames (or its default value if unspecified, i.e. 1024) is used. +- passthrough + + A string property specifying whether IOMMU mappings are enabled for the + domain and hence whether it will be enabled for passthrough hardware. + Possible property values are: + + - "enabled" + IOMMU mappings are enabled for the domain. Note that this option is the + default if the user provides the device partial passthrough device tree + for the domain. + + - "disabled" + IOMMU mappings are disabled for the domain and so hardware may not be + passed through. This option is the default if this property is missing + and the user does not provide the device partial device tree for the domain. + Under the "xen,domain" compatible node, one or more sub-nodes are present for the DomU kernel and ramdisk. diff --git a/xen/arch/arm/dom0less-build.c b/xen/arch/arm/dom0less-build.c index 74f053c242..5830a7051d 100644 --- a/xen/arch/arm/dom0less-build.c +++ b/xen/arch/arm/dom0less-build.c @@ -848,6 +848,8 @@ static int __init construct_domU(struct domain *d, void __init create_domUs(void) { struct dt_device_node *node; + const char *dom0less_iommu; + bool iommu = false; const struct dt_device_node *cpupool_node, *chosen = dt_find_node_by_path("/chosen"); @@ -895,8 +897,13 @@ void __init create_domUs(void) panic("Missing property 'cpus' for domain %s\n", dt_node_name(node)); - if ( dt_find_compatible_node(node, NULL, "multiboot,device-tree") && - iommu_enabled ) + if ( !dt_property_read_string(node, "passthrough", &dom0less_iommu) && + !strcmp(dom0less_iommu, "enabled") ) + iommu = true; + + if ( iommu_enabled && + (iommu || dt_find_compatible_node(node, NULL, + "multiboot,device-tree")) ) d_cfg.flags |= XEN_DOMCTL_CDF_iommu; if ( !dt_property_read_u32(node, "nr_spis", &d_cfg.arch.nr_spis) ) From patchwork Tue May 21 04:35:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13668905 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 20AA9C25B7D for ; Tue, 21 May 2024 04:36:24 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.726500.1130811 (Exim 4.92) (envelope-from ) id 1s9HEV-0007Ru-GJ; Tue, 21 May 2024 04:36:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 726500.1130811; Tue, 21 May 2024 04:36:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HEV-0007Rg-Ce; Tue, 21 May 2024 04:36:15 +0000 Received: by outflank-mailman (input) for mailman id 726500; Tue, 21 May 2024 04:36:13 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HET-0006hu-G0 for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:13 +0000 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on20600.outbound.protection.outlook.com [2a01:111:f403:2415::600]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a60a278c-172b-11ef-b4bb-af5377834399; Tue, 21 May 2024 06:36:11 +0200 (CEST) Received: from SN6PR08CA0013.namprd08.prod.outlook.com (2603:10b6:805:66::26) by MN2PR12MB4255.namprd12.prod.outlook.com (2603:10b6:208:198::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Tue, 21 May 2024 04:36:09 +0000 Received: from SA2PEPF00001508.namprd04.prod.outlook.com (2603:10b6:805:66:cafe::2e) by SN6PR08CA0013.outlook.office365.com (2603:10b6:805:66::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend Transport; Tue, 21 May 2024 04:36:08 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by SA2PEPF00001508.mail.protection.outlook.com (10.167.242.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:08 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May 2024 23:36:07 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Mon, 20 May 2024 23:36:06 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a60a278c-172b-11ef-b4bb-af5377834399 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gDQjP1EqJXQPNeWNjdHFD9UehBjTc+UWgjTa9vgYXI5ZQCbOtn+60VAWSKe9LxxenbyeBXZ4ztO0My6g1GkHh7Dy3xtltz9v2nc7bPcWmnwGw0jWR7Lz/ciW83WYrESMZ8Gkf8X3w3IYTVogJjQxqSXcj3amd3fQzuiLTPewRBidgiCLSy6mQ7Mp7wSNdsn2Mm0h6U8nYb5cgHtho3vp4nLhv6+EM1RjSgoojqHA6KqzdRLPVwTekZ5wFmh50mIM8BQ5itvWDKCbF7/Yo/qQX9fmATX/x6MOiKNi7MmeCVy7/unqfu9i2W9fijRGBmClMndmBhyDg/kUEoe++zKVNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Wmo94nWRzmCLwFNiswsI5NwrIKUvcZBhuU9kpPno8cI=; b=MuTKnAyNK2o8xX7o6SDp6A891mMXTG44akktQgjKKRmyd3MFHO2qAHd+KF5M7HlpJJUICPEwh/sGFqwNzy8sdju06CoRLISMvW/T2XxRn09voqtuGreKhAjyNiG6LkPP0G+DHfA6QB1M4jyfU6tMDZjEtcNNDul4lMJn8sKnI/Hd8xPuNHkd6a/NNo8pcTGBecuL2+Cdag9aSgKc2Y0fZVd1GkVAJkMfSelmaMQcFegEqwHLuOTh3QstkZP3pFHWDgh3FvIF8gJOZpf366ghIpqtm31QqqOWtWu4USk6g9cPAynizE+T/YQxUjh9kOVS7dCjm5lBIIlSmYr1gVulSg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Wmo94nWRzmCLwFNiswsI5NwrIKUvcZBhuU9kpPno8cI=; b=VtzGigXWO1TXy7+LJWzdx/zB5q8KIXX3ilObFOh7Y64la3MLf2C5wgA75ZDrQCjK/8/q0DmT4Nu1kq4uO32j5OOGUdWts4wm1t7eQmUhKF4giGY52vQedkOV4jIwzLu3ZA250d65e9dCwq5JYEv2TUhF+wZP+4gf11rfKAl3XxU= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Henry Wang To: CC: Anthony PERARD , George Dunlap , Nick Rosbrook , "Juergen Gross" , Henry Wang Subject: [PATCH v3 4/8] tools/arm: Introduce the "nr_spis" xl config entry Date: Tue, 21 May 2024 12:35:53 +0800 Message-ID: <20240521043557.1580753-5-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com> References: <20240521043557.1580753-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00001508:EE_|MN2PR12MB4255:EE_ X-MS-Office365-Filtering-Correlation-Id: b5279e61-7b56-48d6-445c-08dc794f88da X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|1800799015|36860700004|82310400017|376005; X-Microsoft-Antispam-Message-Info: aeCy5ecntFfY3oyUBQxWx8biEGZ0cvsquN4WWY6TASVCBEwTeb33K4l/+GSOhxZvWSd3Eqd9f9+tgXLrK/fKaYSKfWoYJSET52GuFHKq/FL1UsIAOEi2HFLYQNCJsJ7V8HghPC0E11SEhYKA2kIWJ/vc3sv8vBcvlijeIiqG27C6HXsfGw+UrzwoQwJ5yordmPMya18GB6QEpN9qapYeG6/GIyJANmEBCv2OkIov2h6lctlGK96bcH5UlGVAk7gQZUoLRxT57XPwVMx9GwY1rATUWUgI5KhZ1FpR+qZCstXMWhClPcfGIpsU3hjIaxX4YqDlVNbByrmj4OK730wqyKntdephWtfDYTHok5q6Bu+Bp2eFzq05ibOYtS9G5urFJrowdhjBgxH9SWy3MUW3xz9Ik02be76RCjHH5YOCfTvZnpbzihWWFI6oIFct28UQlD+r2qZq0ngj3va17cCFoENuTV0hNSQX4e7vTw97xI8d2GGSgzZP7r3aaTmcdIs4Wd9XefPQMkEBUh8+Ao2/Tf4aEDN58SbiLz1DaYlfma0HifwOeUDQ8Q+OKEBhuKmt8FUEruyZrtiAT0wDOweYmvjsJ2qvNAYblK8/8aR6RmJE5d7HsUaA5xT5bC9zo00PS3GlD0ikGgWUw2/wBJeyxPXVo0gvV55SwenAAOEsQbqwaJu118E+w7DfVbQVfD/5oKeWHT3M0/r5kKoMkl9bySgiFPL3OTPbcMChG4SOBcO8LR1cYcZ6XE+O4rTL/mWcAYR4sP1RycZk70QKhF/HWfpDAzk0sZETUuF1Ge3XV+wtCAYF01MwGPJ2MdvsGCDWqQuloR1ukLcggf3eg7KPmzcv/QBncENRN4KdQkE8+tZhGBucDE7ltt5pNjQ2T9biCVgtb0n8Xrvy1fJsLcjvT4IcdPrrEdfV6HxLabI2JhEA58ZK7rfi2YcKaVrvGLfe03rzuA2qC0wAk63vPcP16ZX7yxMRtnm0h+0XAm8HYxyWMriN8Qf32f6FnKxoJJiVoAY3E+ZHg5fQWZYPqrqop98P76fzqo51i91xsME33XnI20QXZ/IaazhPterScFBDMTvy7YiorjU5Q9XzdEMk5/++MIgy4HooFhnjs0F3pmEY2x+UOS+J39NvyMFoCy0sq+uEvU0wAfry3sEPtTOEUgsHFKi31/cga5I9rJemX7wDZaAUDAQy5h8lmdgZh/uGhMU0kz9DL0384zv1mDxli4xKGN0OF3FkvUkGXV/eFNhokwFwVx0VPimtfaae5XLy4U1SJ/Id2jlTF22Y9DHTWLlCIoDkVQP3Ot0y+IOpQMsf+Pyk8o7wBeRd5hYrBvfVGy/neT7N/074CK58wN7mU7DcgP171BOrSA4Ro7XbjG2lj6hiU81hKRsFbO3fp+gB X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(36860700004)(82310400017)(376005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:08.4682 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b5279e61-7b56-48d6-445c-08dc794f88da X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00001508.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4255 Currently, the number of SPIs allocated to the domain is only configurable for Dom0less DomUs. Xen domains are supposed to be platform agnostics and therefore the numbers of SPIs for libxl guests should not be based on the hardware. Introduce a new xl config entry for Arm to provide a method for user to decide the number of SPIs. This would help to avoid bumping the `config->arch.nr_spis` in libxl everytime there is a new platform with increased SPI numbers. Update the doc and the golang bindings accordingly. Signed-off-by: Henry Wang Reviewed-by: Jason Andryuk --- v3: - Reword documentation to avoid ambiguity. v2: - New patch to replace the original patch in v1: "[PATCH 05/15] tools/libs/light: Increase nr_spi to 160" --- docs/man/xl.cfg.5.pod.in | 14 ++++++++++++++ tools/golang/xenlight/helpers.gen.go | 2 ++ tools/golang/xenlight/types.gen.go | 1 + tools/libs/light/libxl_arm.c | 4 ++-- tools/libs/light/libxl_types.idl | 1 + tools/xl/xl_parse.c | 3 +++ 6 files changed, 23 insertions(+), 2 deletions(-) diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in index 8f2b375ce9..416d582844 100644 --- a/docs/man/xl.cfg.5.pod.in +++ b/docs/man/xl.cfg.5.pod.in @@ -3072,6 +3072,20 @@ raised. =back +=over 4 + +=item B + +An optional 32-bit integer parameter specifying the number of SPIs (Shared +Peripheral Interrupts) to allocate for the domain. If the value specified by +the `nr_spis` parameter is smaller than the number of SPIs calculated by the +toolstack based on the devices allocated for the domain, or the `nr_spis` +parameter is not specified, the value calculated by the toolstack will be used +for the domain. Otherwise, the value specified by the `nr_spis` parameter will +be used. + +=back + =head3 x86 =over 4 diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go index b9cb5b33c7..fe5110474d 100644 --- a/tools/golang/xenlight/helpers.gen.go +++ b/tools/golang/xenlight/helpers.gen.go @@ -1154,6 +1154,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)} x.ArchArm.GicVersion = GicVersion(xc.arch_arm.gic_version) x.ArchArm.Vuart = VuartType(xc.arch_arm.vuart) x.ArchArm.SveVl = SveType(xc.arch_arm.sve_vl) +x.ArchArm.NrSpis = uint32(xc.arch_arm.nr_spis) if err := x.ArchX86.MsrRelaxed.fromC(&xc.arch_x86.msr_relaxed);err != nil { return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err) } @@ -1670,6 +1671,7 @@ return fmt.Errorf("invalid union key '%v'", x.Type)} xc.arch_arm.gic_version = C.libxl_gic_version(x.ArchArm.GicVersion) xc.arch_arm.vuart = C.libxl_vuart_type(x.ArchArm.Vuart) xc.arch_arm.sve_vl = C.libxl_sve_type(x.ArchArm.SveVl) +xc.arch_arm.nr_spis = C.uint32_t(x.ArchArm.NrSpis) if err := x.ArchX86.MsrRelaxed.toC(&xc.arch_x86.msr_relaxed); err != nil { return fmt.Errorf("converting field ArchX86.MsrRelaxed: %v", err) } diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go index 5b293755d7..c9e45b306f 100644 --- a/tools/golang/xenlight/types.gen.go +++ b/tools/golang/xenlight/types.gen.go @@ -597,6 +597,7 @@ ArchArm struct { GicVersion GicVersion Vuart VuartType SveVl SveType +NrSpis uint32 } ArchX86 struct { MsrRelaxed Defbool diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c index 1cb89fa584..a4029e3ac8 100644 --- a/tools/libs/light/libxl_arm.c +++ b/tools/libs/light/libxl_arm.c @@ -181,8 +181,8 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc, LOG(DEBUG, "Configure the domain"); - config->arch.nr_spis = nr_spis; - LOG(DEBUG, " - Allocate %u SPIs", nr_spis); + config->arch.nr_spis = max(nr_spis, d_config->b_info.arch_arm.nr_spis); + LOG(DEBUG, " - Allocate %u SPIs", config->arch.nr_spis); switch (d_config->b_info.arch_arm.gic_version) { case LIBXL_GIC_VERSION_DEFAULT: diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl index 79e9c656cc..4e65e6fda5 100644 --- a/tools/libs/light/libxl_types.idl +++ b/tools/libs/light/libxl_types.idl @@ -722,6 +722,7 @@ libxl_domain_build_info = Struct("domain_build_info",[ ("arch_arm", Struct(None, [("gic_version", libxl_gic_version), ("vuart", libxl_vuart_type), ("sve_vl", libxl_sve_type), + ("nr_spis", uint32), ])), ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool), ])), diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c index c504ab3711..e3a4800f6e 100644 --- a/tools/xl/xl_parse.c +++ b/tools/xl/xl_parse.c @@ -2935,6 +2935,9 @@ skip_usbdev: } } + if (!xlu_cfg_get_long (config, "nr_spis", &l, 0)) + b_info->arch_arm.nr_spis = l; + parse_vkb_list(config, d_config); d_config->virtios = NULL; From patchwork Tue May 21 04:35:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13668907 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E40CBC25B74 for ; Tue, 21 May 2024 04:36:26 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.726503.1130832 (Exim 4.92) (envelope-from ) id 1s9HEZ-00081t-Bg; Tue, 21 May 2024 04:36:19 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 726503.1130832; Tue, 21 May 2024 04:36:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HEZ-00081l-7k; Tue, 21 May 2024 04:36:19 +0000 Received: by outflank-mailman (input) for mailman id 726503; Tue, 21 May 2024 04:36:18 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HEY-0006hu-8Q for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:18 +0000 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on20600.outbound.protection.outlook.com [2a01:111:f403:2418::600]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a857611f-172b-11ef-b4bb-af5377834399; Tue, 21 May 2024 06:36:15 +0200 (CEST) Received: from SA1P222CA0046.NAMP222.PROD.OUTLOOK.COM (2603:10b6:806:2d0::14) by MW6PR12MB8957.namprd12.prod.outlook.com (2603:10b6:303:23a::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Tue, 21 May 2024 04:36:12 +0000 Received: from SA2PEPF00001506.namprd04.prod.outlook.com (2603:10b6:806:2d0:cafe::c3) by SA1P222CA0046.outlook.office365.com (2603:10b6:806:2d0::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend Transport; Tue, 21 May 2024 04:36:11 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by SA2PEPF00001506.mail.protection.outlook.com (10.167.242.38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:10 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May 2024 23:36:10 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Mon, 20 May 2024 23:36:08 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a857611f-172b-11ef-b4bb-af5377834399 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lZgPU7G1H+hhtuaWEuG/cMMmBo/QQ49SAG+AZvvFieqW6nQcvTRxqXdKrDBK2oWwTIgqAYyXW2HJawQa13miNOxFqlYK/NcLU9BNnnDtbAnhbktwlBWXwyWWzccQUWlaczfX8Xe+S4bDs7sE9OceNUvQyVQmYZNddeSWI9MsSzMa6tnKnLHsW7xjrHa23FIAgq5KXRB/r4O7+4KG/2TISHL4pSdKMxIJEDbKI3vS9DAJO2FpZJviUDn98kll0mW7Un4FNF4nx0bH+B1ILui13JtcZYmuWHkL7AkkQeIk74XG7xdAogJs/P1spFDp9PnHCKxfOoYtekCTdmWMp+wfnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EOJtF38SHzbKHHCO6HcbBP/3k8UEkTSkkoNvwkF1QIM=; b=QBtl44x1G/czAYBYL4NEKW+rBKOl7GF0UAaPXDJO83C9zkgJZzZwFcV5iz1OL4hQ6jiR0fVfLZyqAK4RpIOAwoxof1OYBH3T5pv/qK6SlbxCAL9lyon2lTGztVWG9zN1aO7nR96FJenXJwNDoUA3rZHfPbu/Gg1CRcCJtAp9L+8pH+Tq+ZzCjUzj0RUL4nFYfHDil3+B5y3vpnRuC6TivZLSoeine2eK7ypgBjECuYl1d+MR4n9b8KPyfhT+/7UlKXiokTsDq0KtaaoCaUTHDVlQf5z2GpH92BOPoacxQXbDGdVXMYfWe9RdMvXqNmHk5rhGMnvTL0zG1YgUzll7yg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EOJtF38SHzbKHHCO6HcbBP/3k8UEkTSkkoNvwkF1QIM=; b=S8Sm22uAopwOkW6WFnp9X+UOmz1gKQmZeaERmqNckV/Ei1KDdKMH7fKKmqLOWWEB+5H24Os1gsNfWYIAEJImkk50W1cb5htRRPGFeYD7W7/VwPNeWz0cAufpNhmAl3Znjc1tQ3LRg6NjhA/01+IaVAJbwKhgneLsIMudVo1j03U= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Henry Wang To: CC: Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , "Henry Wang" Subject: [PATCH v3 5/8] xen/arm/gic: Allow routing/removing interrupt to running VMs Date: Tue, 21 May 2024 12:35:54 +0800 Message-ID: <20240521043557.1580753-6-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com> References: <20240521043557.1580753-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00001506:EE_|MW6PR12MB8957:EE_ X-MS-Office365-Filtering-Correlation-Id: 556d5ffa-525c-4836-1cb7-08dc794f8a1b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|36860700004|376005|82310400017|1800799015; X-Microsoft-Antispam-Message-Info: +Qql7oBpC7z9QsDGgKoq0TXcutywO3ibPMhx5jjDz4u/TIaGO7etjfTbZsGyLDgMF64CKDEOrduo2EqUd0O3amflBPJzXguY/tmBp6uJrvukT6SNIfmPp3ZpSONcEMkqFLhEqwyVAKCkpimVhdOZ9MjiXqL9AEgr5Dei4jS4jW6hbeC06+BpLOhClgJ/Br1ngGSJ4vCeGABNV0SDYRNAFfmK+qHOq+M8dQYzS0n+0b60ieIjNmg0rn6VA5es5XHPomXD7eZqMUW4gobcuwoXgJRjPfL0tyaaKNTjeQMQ6IcCJ+s+M3XG42t51JbL4K8vbhmxnaWPtfqpbDwVycqRfbYzSFYIpOJcVD0LGW2zYKQlz3E3JeDzw5Ubn81UE1AKxiCsVKGzywraP7DSuVFf2NC8lMNnes40b11iZB9Uy32A5F+soo6bTzuizlRa13FBrI6of4mKSASUyePIm2SNmdTTSJItJTsgTrjGJqILLjX5YvsjneTvM8OJPcR20QCC3nv5TroNS53/KOdfcyttZya3sIfXYlFhduTiJVttzQwz9HocLEJFULDlxjyp8zifNTBMk3vrOAOdNeo+rgkQ96Teb97jGy+K54tNnriXFjg84+hi9l1vjyY8ceqV38223vtvh6ZMnJVfQ5TuFonyWJlwiYgcJ92Bt0ULXmkxbdZdetiN0q/J2wUM+bBVpNRrEGKnvdEDxovArEreRNktqaKpCibXxyq4KqFdXhZCOhVtNVzw5d145FEWl7PX+LG+AbaO9Suv60Hxehgwww3VWirIjY20UZjbpMjR7s10pBddbSJcahtTk+zwD9cXpNoWj0UelRcQA/R54EhsXavqzrUIqq2g+sk2gc2C1GA3fPvxSyBIVGBgrMNuvVXoXFUUzHXVoRGbgRhSJxXQUW/qhJLlesL0MSdjjbnv/hxxcyyALPhsj36RoxwOCPJsxzd52+kKOcP+U9K/14WHLbMEH6EhYaCLklqHyrdKOkJTG4hlMvKftswV6qpNcUzb0mScb+vCZWFUr3lKLdBfcvJI5SMjQY/ls9tbm0HmPAoPjcEXHFSGjuxUGF9bfS+CuJc8Ffipda4ivAM9c1AbYdT2TkS+vk+H/TiHiL/t9sqwo1smfpuUJnZoZchvu0pAoUetsdiXiRBoMt567GbHaUbj6240XYljz7EsSJYs9sA22HLh9WFPUUu6pL980WbgDzkeNYjEexdX+0FL4yY0fZ0pdqvCvxwHcBdSxrtJrEdrT0rXssaB1sFUR99TYl4yHi77POeDjw+4KCWrNGnDe/bXdJWAFg/zwnz7peq8ToKmebMJO8GuPynAuL9xf7OIfX0Y33FNIEofX9PybYAkmQPSYZOUoE1Sp350KCvvDeLKjjytWc6zdcBgwuOb3HYNJGYP X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(36860700004)(376005)(82310400017)(1800799015);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:10.5552 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 556d5ffa-525c-4836-1cb7-08dc794f8a1b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00001506.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR12MB8957 From: Vikram Garhwal Currently, routing/removing physical interrupts are only allowed at the domain creation/destroy time. For use cases such as dynamic device tree overlay adding/removing, the routing/removing of physical IRQ to running domains should be allowed. Removing the above-mentioned domain creation/dying check. Since this will introduce interrupt state unsync issues for cases when the interrupt is active or pending in the guest, therefore for these cases we simply reject the operation. Do it for both new and old vGIC implementations. Signed-off-by: Vikram Garhwal Signed-off-by: Stefano Stabellini Signed-off-by: Henry Wang --- v3: - Update in-code comments. - Correct the if conditions. - Add taking/releasing the vgic lock of the vcpu. v2: - Reject the case where the IRQ is active or pending in guest. --- xen/arch/arm/gic-vgic.c | 15 ++++++++++++--- xen/arch/arm/gic.c | 15 --------------- xen/arch/arm/vgic/vgic.c | 10 +++++++--- 3 files changed, 19 insertions(+), 21 deletions(-) diff --git a/xen/arch/arm/gic-vgic.c b/xen/arch/arm/gic-vgic.c index 56490dbc43..956c11ba13 100644 --- a/xen/arch/arm/gic-vgic.c +++ b/xen/arch/arm/gic-vgic.c @@ -439,24 +439,33 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *v, unsigned int virq, /* We are taking to rank lock to prevent parallel connections. */ vgic_lock_rank(v_target, rank, flags); + spin_lock(&v_target->arch.vgic.lock); if ( connect ) { - /* The VIRQ should not be already enabled by the guest */ + /* + * The VIRQ should not be already enabled by the guest nor + * active/pending in the guest. + */ if ( !p->desc && - !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) ) + !test_bit(GIC_IRQ_GUEST_ENABLED, &p->status) && + !test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) && + !test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) ) p->desc = desc; else ret = -EBUSY; } else { - if ( desc && p->desc != desc ) + if ( (desc && p->desc != desc) || + test_bit(GIC_IRQ_GUEST_VISIBLE, &p->status) || + test_bit(GIC_IRQ_GUEST_ACTIVE, &p->status) ) ret = -EINVAL; else p->desc = NULL; } + spin_unlock(&v_target->arch.vgic.lock); vgic_unlock_rank(v_target, rank, flags); return ret; diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index 44c40e86de..3ebd89940a 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -135,14 +135,6 @@ int gic_route_irq_to_guest(struct domain *d, unsigned int virq, ASSERT(virq < vgic_num_irqs(d)); ASSERT(!is_lpi(virq)); - /* - * When routing an IRQ to guest, the virtual state is not synced - * back to the physical IRQ. To prevent get unsync, restrict the - * routing to when the Domain is been created. - */ - if ( d->creation_finished ) - return -EBUSY; - ret = vgic_connect_hw_irq(d, NULL, virq, desc, true); if ( ret ) return ret; @@ -167,13 +159,6 @@ int gic_remove_irq_from_guest(struct domain *d, unsigned int virq, ASSERT(test_bit(_IRQ_GUEST, &desc->status)); ASSERT(!is_lpi(virq)); - /* - * Removing an interrupt while the domain is running may have - * undesirable effect on the vGIC emulation. - */ - if ( !d->is_dying ) - return -EBUSY; - desc->handler->shutdown(desc); /* EOI the IRQ if it has not been done by the guest */ diff --git a/xen/arch/arm/vgic/vgic.c b/xen/arch/arm/vgic/vgic.c index b9463a5f27..78554c11e2 100644 --- a/xen/arch/arm/vgic/vgic.c +++ b/xen/arch/arm/vgic/vgic.c @@ -876,8 +876,11 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu, if ( connect ) /* assign a mapped IRQ */ { - /* The VIRQ should not be already enabled by the guest */ - if ( !irq->hw && !irq->enabled ) + /* + * The VIRQ should not be already enabled by the guest nor + * active/pending in the guest + */ + if ( !irq->hw && !irq->enabled && !irq->active && !irq->pending_latch ) { irq->hw = true; irq->hwintid = desc->irq; @@ -887,7 +890,8 @@ int vgic_connect_hw_irq(struct domain *d, struct vcpu *vcpu, } else /* remove a mapped IRQ */ { - if ( desc && irq->hwintid != desc->irq ) + if ( (desc && irq->hwintid != desc->irq) || + irq->active || irq->pending_latch ) { ret = -EINVAL; } From patchwork Tue May 21 04:35:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13668910 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 040FFC25B75 for ; Tue, 21 May 2024 04:36:32 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.726505.1130848 (Exim 4.92) (envelope-from ) id 1s9HEc-0008Sc-Cy; Tue, 21 May 2024 04:36:22 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 726505.1130848; Tue, 21 May 2024 04:36:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HEc-0008QW-2g; Tue, 21 May 2024 04:36:22 +0000 Received: by outflank-mailman (input) for mailman id 726505; Tue, 21 May 2024 04:36:20 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HEa-0006eS-Bo for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:20 +0000 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on20601.outbound.protection.outlook.com [2a01:111:f400:7e88::601]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id aa4ee97c-172b-11ef-909f-e314d9c70b13; Tue, 21 May 2024 06:36:19 +0200 (CEST) Received: from SA1P222CA0035.NAMP222.PROD.OUTLOOK.COM (2603:10b6:806:2d0::10) by SA1PR12MB9247.namprd12.prod.outlook.com (2603:10b6:806:3af::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36; Tue, 21 May 2024 04:36:14 +0000 Received: from SA2PEPF00001506.namprd04.prod.outlook.com (2603:10b6:806:2d0:cafe::ed) by SA1P222CA0035.outlook.office365.com (2603:10b6:806:2d0::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7544.55 via Frontend Transport; Tue, 21 May 2024 04:36:14 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by SA2PEPF00001506.mail.protection.outlook.com (10.167.242.38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:14 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May 2024 23:36:12 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Mon, 20 May 2024 23:36:10 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: aa4ee97c-172b-11ef-909f-e314d9c70b13 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BAa3lpafPZQvr1YDjSdauNNXobXxJcpOnNsPWh5+1icXRlADKKLU1MlwpI6v5jurDAEluiO4pHZYIyEkR5Wsd4hGK+gsviZEieVVdOjFBPwqo7T7DZqAsqTvqRcs51C/waS5m6f5nrLGfgw17hkSXq8Q2G0L/NX6v46KwJ6e0ZcArZTE36WOul1aLX/hRv9LlWOIVVKXgTQzhc5S6Psy7+TsmSn7v8mYMqx7yWrcKnYSeFFAeiY371glACcOZoV60MMaY8n7/13vxzVPTJaXmX/uHZvBRu1TpFZt/+MFOarEX2J9525ExL65PxCoEcdbSx24C/Bklr2KtKv1Y7CLgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=c5asE2vIShV06rhyQR/L2eZ3PEF3qss1h6tMuTFdt4g=; b=epZ9eySRQp5IaVic1R+4fyc4tk8BzeT7norjD66eu4nZU24HfOfh6zS7oq4JjTkPZ0Km1KrGL+pjwriUtF11pejDKzdByuA4KaAG00JmgaZelzWKhMqyqx3Hs69Tn8stfd3tKonbQlHHt9M86q7ONO/TOvkqJsGmsZ0PFN8rxukgX64uGk5lFGZI6XxMJqoWXyovaST3ATxkgi+yCMTp/3VkfZCO5t7QH+F636MCwNOa5flrdVtSYPhj9hhMz0DzKahWdsGtAvyDNyyVulkOpClxr3OcsAqBTsv3w/sxdQ1zMiUcK81Haz5m48VhvUxLWbRHdTVLvCLumAsBgyePgg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=c5asE2vIShV06rhyQR/L2eZ3PEF3qss1h6tMuTFdt4g=; b=sqwYR0Mg4WjfZYWlz0NbAuuoxG9kCZT2BKUAbh4wZDAsn4pUp8kaVlCmQ7ZYpWPivvMKjUu01pkTNSvzQzGdQAFggFaDxAXZkSdyIgfZ/QkzxafRlNjN06IlStiwh2zjR2N2YacR8BEnJAJNqj1mzYqzdgCxolGaUZJ9mgjZPcY= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Henry Wang To: CC: Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Henry Wang Subject: [PATCH v3 6/8] xen/arm: Add XEN_DOMCTL_dt_overlay DOMCTL and related operations Date: Tue, 21 May 2024 12:35:55 +0800 Message-ID: <20240521043557.1580753-7-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com> References: <20240521043557.1580753-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00001506:EE_|SA1PR12MB9247:EE_ X-MS-Office365-Filtering-Correlation-Id: b7df2670-00aa-46e9-dbc3-08dc794f8c60 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|82310400017|1800799015|36860700004|376005; X-Microsoft-Antispam-Message-Info: iDPD4qsEn3+NtRctDhK14FfHQI854Qi/ehp2yAH0A0tgAd9z237TVYEWYurAPYj/LEsIBAI2XiIacSWG0BWVqZZ+LAy50vhNQ/4PR3fRqvYWAAhZRXFmhM9+DM38fOmgWyF3ItOIUx6WmwSXrP0tDJxDkAclKbYyt4i1UFMumcYFuI6w5oPWzTryYyY+6/HVoBSCkakwgcAFRWoSaRJVSn+wlwiEVo7mgEejUAbRipvW07iMi8zFabfooD5yM+8YeRAkQct5E47g+f6p43lgnIPwxQEgpCVlA4iShwcEzHFo0+UyrcPBIuJFMHZZvRCXpEWlFL275Ile00Xzbe6MaBnjCtunvzswLSLTtlbLpq4NKBxoLN0FvNUueKCOc5AwH51rv7x0amwlrDLsEHHKCTgVoBP0ctnA8sxI+8G90CCAvYxVvlcIZwDYQAdF8A2z+6MJFrTTgc+N9cZToilfKs7ZNgp7+wxXkfoZP9a2Ban4P8X213w5jlq9kSn+YqBnEdT8LpQdGXbQKEiJAW2NYGPzwRww/674yAQ1AmrWSGGzTOW5Bvked1iKN7x5T8Qx4roDocWXzObx/Hbi6drKNf0RYNZzlPDldmE8xP2twOCpdNX7+W0j/gZIiYZQla0bDXeE3DCdoakk2aHO3REGRbl52H4WQDvUqcw6gChUvj8z6nDjgvl9TbbaNaAgNonxNUAIzlOkgSpcGHDPekEKU32LJRSsbho9pPnQ0kVOnm5r2rWOROJGWMJloLRtr9Z/j9Z3NDCtkc6qCsxHNoSijKL5Iu4lGttDS0nWqd1FeKkkDK06emAjuUBIAcvaKZacEuObdOfREikaLO7HytKsGT/2g1HVSiWLDr3w81Bib5HjhWz0SREWufMzhn3/F45Ely3/3WpLOJq0qtmRZj09o7eIjcZxBUQwYg9KO/2SSS86Eoum7VSX15FuazMktsAETd1ROk2WHIRTchkzv5Uat1C9Wghe1GYyHD6nJDrM8xPI3QltJjIzq7xSK2ngpEGvbFGfxfw/ib3x/VMqQk790M9F828bQWCHxk4n/SJFfURP1LiCpjlSAA8yODaR9ldL+eG0CvDdXEOpED3NHN6tMYhx1EAx4tIDyE+8BeLn//UMLY8UyR/w2xRojwf1DCop1saZgWgh8cbE61amzsiwC0ky2mxNHBY9pDf6uFByweIR1iH/pTPikLsUaT9J4p91cIaaLI0RFuGrkDqAbsW6nBFMqNLSM/oTqm1oTGfJqTz8hux/OWbFxe8XtycR08pBvpL44PpUfLwBt/EOm7qTawVg6NhpIuquMIdE++//K1DTO2iI4w4arKJlyGUY8OsJCfna29wlND7U009N5WrQnbgjoElJ21tI8UvZtENyE7H8knDq72XMc1pP4V/C84zw X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(82310400017)(1800799015)(36860700004)(376005);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:14.3678 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b7df2670-00aa-46e9-dbc3-08dc794f8c60 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00001506.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB9247 In order to support the dynamic dtbo device assignment to a running VM, the add/remove of the DT overlay and the attach/detach of the device from the DT overlay should happen separately. Therefore, repurpose the existing XEN_SYSCTL_dt_overlay to only add the DT overlay to Xen device tree, instead of assigning the device to the hardware domain at the same time. Add the XEN_DOMCTL_dt_overlay with operations XEN_DOMCTL_DT_OVERLAY_{ATTACH,DETACH} to do/undo the device assignment to the domain. The hypervisor firstly checks the DT overlay passed from the toolstack is valid. Then the device nodes are retrieved from the overlay tracker based on the DT overlay. The attach/detach of the device is implemented by map/unmap the IRQ and IOMMU resources. Note that with these changes, the device de-registration from the IOMMU driver should only happen at the time when the DT overlay is removed from the Xen device tree. Signed-off-by: Henry Wang Signed-off-by: Vikram Garhwal --- v3: - Style fixes for arch-selection #ifdefs. - Do not include public/domctl.h, only add a forward declaration of struct xen_domctl_dt_overlay. - Extract the overlay track entry finding logic to a function, drop the unused variables. - Use op code 1&2 for XEN_DOMCTL_DT_OVERLAY_{ATTACH,DETACH}. v2: - New patch. --- xen/arch/arm/domctl.c | 3 + xen/common/dt-overlay.c | 438 +++++++++++++++++++++++++++-------- xen/include/public/domctl.h | 15 ++ xen/include/public/sysctl.h | 11 +- xen/include/xen/dt-overlay.h | 7 + 5 files changed, 367 insertions(+), 107 deletions(-) diff --git a/xen/arch/arm/domctl.c b/xen/arch/arm/domctl.c index ad56efb0f5..12a12ee781 100644 --- a/xen/arch/arm/domctl.c +++ b/xen/arch/arm/domctl.c @@ -5,6 +5,7 @@ * Copyright (c) 2012, Citrix Systems */ +#include #include #include #include @@ -176,6 +177,8 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d, return rc; } + case XEN_DOMCTL_dt_overlay: + return dt_overlay_domctl(d, &domctl->u.dt_overlay); default: return subarch_do_domctl(domctl, d, u_domctl); } diff --git a/xen/common/dt-overlay.c b/xen/common/dt-overlay.c index 9cece79067..693b6e4777 100644 --- a/xen/common/dt-overlay.c +++ b/xen/common/dt-overlay.c @@ -356,24 +356,136 @@ static int overlay_get_nodes_info(const void *fdto, char **nodes_full_path) return 0; } +/* This function should be called with the overlay_lock taken */ +static struct overlay_track * +find_track_entry_from_tracker(const void *overlay_fdt, + uint32_t overlay_fdt_size) +{ + struct overlay_track *entry, *temp; + bool found_entry = false; + + ASSERT(spin_is_locked(&overlay_lock)); + + /* + * First check if dtbo is correct i.e. it should one of the dtbo which was + * used when dynamically adding the node. + * Limitation: Cases with same node names but different property are not + * supported currently. We are relying on user to provide the same dtbo + * as it was used when adding the nodes. + */ + list_for_each_entry_safe( entry, temp, &overlay_tracker, entry ) + { + if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 ) + { + found_entry = true; + break; + } + } + + if ( !found_entry ) + { + printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo." + " Operation is supported only for prior added dtbo.\n"); + return NULL; + } + + return entry; +} + +static int remove_irq(unsigned long s, unsigned long e, void *data) +{ + struct domain *d = data; + int rc = 0; + + /* + * IRQ should always have access unless there are duplication of + * of irqs in device tree. There are few cases of xen device tree + * where there are duplicate interrupts for the same node. + */ + if (!irq_access_permitted(d, s)) + return 0; + /* + * TODO: We don't handle shared IRQs for now. So, it is assumed that + * the IRQs was not shared with another domain. + */ + rc = irq_deny_access(d, s); + if ( rc ) + { + printk(XENLOG_ERR "unable to revoke access for irq %ld\n", s); + return rc; + } + + rc = release_guest_irq(d, s); + if ( rc ) + { + printk(XENLOG_ERR "unable to release irq %ld\n", s); + return rc; + } + + return rc; +} + +static int remove_all_irqs(struct rangeset *irq_ranges, struct domain *d) +{ + return rangeset_report_ranges(irq_ranges, 0, ~0UL, remove_irq, d); +} + +static int remove_iomem(unsigned long s, unsigned long e, void *data) +{ + struct domain *d = data; + int rc = 0; + p2m_type_t t; + mfn_t mfn; + + mfn = p2m_lookup(d, _gfn(s), &t); + if ( mfn_x(mfn) == 0 || mfn_x(mfn) == ~0UL ) + return -EINVAL; + + rc = iomem_deny_access(d, s, e); + if ( rc ) + { + printk(XENLOG_ERR "Unable to remove %pd access to %#lx - %#lx\n", + d, s, e); + return rc; + } + + rc = unmap_mmio_regions(d, _gfn(s), e - s, _mfn(s)); + if ( rc ) + return rc; + + return rc; +} + +static int remove_all_iomems(struct rangeset *iomem_ranges, struct domain *d) +{ + return rangeset_report_ranges(iomem_ranges, 0, ~0UL, remove_iomem, d); +} + /* Check if node itself can be removed and remove node from IOMMU. */ -static int remove_node_resources(struct dt_device_node *device_node) +static int remove_node_resources(struct dt_device_node *device_node, + struct domain *d) { int rc = 0; unsigned int len; domid_t domid; - domid = dt_device_used_by(device_node); + if ( !d ) + { + domid = dt_device_used_by(device_node); - dt_dprintk("Checking if node %s is used by any domain\n", - device_node->full_name); + dt_dprintk("Checking if node %s is used by any domain\n", + device_node->full_name); - /* Remove the node if only it's assigned to hardware domain or domain io. */ - if ( domid != hardware_domain->domain_id && domid != DOMID_IO ) - { - printk(XENLOG_ERR "Device %s is being used by domain %u. Removing nodes failed\n", - device_node->full_name, domid); - return -EINVAL; + /* + * We also check if device is assigned to DOMID_IO as when a domain + * is destroyed device is assigned to DOMID_IO. + */ + if ( domid != DOMID_IO ) + { + printk(XENLOG_ERR "Device %s is being assigned to %u. Device is assigned to %d\n", + device_node->full_name, DOMID_IO, domid); + return -EINVAL; + } } /* Check if iommu property exists. */ @@ -381,9 +493,12 @@ static int remove_node_resources(struct dt_device_node *device_node) { if ( dt_device_is_protected(device_node) ) { - rc = iommu_remove_dt_device(device_node); - if ( rc < 0 ) - return rc; + if ( !list_empty(&device_node->domain_list) ) + { + rc = iommu_deassign_dt_device(d, device_node); + if ( rc < 0 ) + return rc; + } } } @@ -392,7 +507,8 @@ static int remove_node_resources(struct dt_device_node *device_node) /* Remove all descendants from IOMMU. */ static int -remove_descendant_nodes_resources(const struct dt_device_node *device_node) +remove_descendant_nodes_resources(const struct dt_device_node *device_node, + struct domain *d) { int rc = 0; struct dt_device_node *child_node; @@ -402,12 +518,12 @@ remove_descendant_nodes_resources(const struct dt_device_node *device_node) { if ( child_node->child ) { - rc = remove_descendant_nodes_resources(child_node); + rc = remove_descendant_nodes_resources(child_node, d); if ( rc ) return rc; } - rc = remove_node_resources(child_node); + rc = remove_node_resources(child_node, d); if ( rc ) return rc; } @@ -420,8 +536,7 @@ static int remove_nodes(const struct overlay_track *tracker) { int rc = 0; struct dt_device_node *overlay_node; - unsigned int j; - struct domain *d = hardware_domain; + unsigned int j, len; for ( j = 0; j < tracker->num_nodes; j++ ) { @@ -431,18 +546,15 @@ static int remove_nodes(const struct overlay_track *tracker) write_lock(&dt_host_lock); - rc = remove_descendant_nodes_resources(overlay_node); - if ( rc ) + /* Check if iommu property exists. */ + if ( dt_get_property(overlay_node, "iommus", &len) ) { - write_unlock(&dt_host_lock); - return rc; - } - - rc = remove_node_resources(overlay_node); - if ( rc ) - { - write_unlock(&dt_host_lock); - return rc; + if ( dt_device_is_protected(overlay_node) ) + { + rc = iommu_remove_dt_device(overlay_node); + if ( rc < 0 ) + return rc; + } } dt_dprintk("Removing node: %s\n", overlay_node->full_name); @@ -457,22 +569,6 @@ static int remove_nodes(const struct overlay_track *tracker) write_unlock(&dt_host_lock); } - /* Remove IRQ access. */ - if ( tracker->irq_ranges ) - { - rc = rangeset_consume_ranges(tracker->irq_ranges, irq_remove_cb, d); - if ( rc ) - return rc; - } - - /* Remove mmio access. */ - if ( tracker->iomem_ranges ) - { - rc = rangeset_consume_ranges(tracker->iomem_ranges, iomem_remove_cb, d); - if ( rc ) - return rc; - } - return rc; } @@ -485,8 +581,7 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt, uint32_t overlay_fdt_size) { int rc; - struct overlay_track *entry, *temp, *track; - bool found_entry = false; + struct overlay_track *entry; rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size); if ( rc ) @@ -494,31 +589,11 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt, spin_lock(&overlay_lock); - /* - * First check if dtbo is correct i.e. it should one of the dtbo which was - * used when dynamically adding the node. - * Limitation: Cases with same node names but different property are not - * supported currently. We are relying on user to provide the same dtbo - * as it was used when adding the nodes. - */ - list_for_each_entry_safe( entry, temp, &overlay_tracker, entry ) - { - if ( memcmp(entry->overlay_fdt, overlay_fdt, overlay_fdt_size) == 0 ) - { - track = entry; - found_entry = true; - break; - } - } - - if ( !found_entry ) + entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size); + if ( entry == NULL ) { rc = -EINVAL; - - printk(XENLOG_ERR "Cannot find any matching tracker with input dtbo." - " Removing nodes is supported only for prior added dtbo.\n"); goto out; - } rc = remove_nodes(entry); @@ -536,9 +611,6 @@ static long handle_remove_overlay_nodes(const void *overlay_fdt, xfree(entry->nodes_address); - rangeset_destroy(entry->irq_ranges); - rangeset_destroy(entry->iomem_ranges); - xfree(entry); out: @@ -620,15 +692,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path) return -EFAULT; } - rc = handle_device(hardware_domain, overlay_node, p2m_mmio_direct_c, - tr->iomem_ranges, - tr->irq_ranges); write_unlock(&dt_host_lock); - if ( rc ) - { - printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n"); - return rc; - } /* Keep overlay_node address in tracker. */ tr->nodes_address[j] = (unsigned long)overlay_node; @@ -638,9 +702,7 @@ static long add_nodes(struct overlay_track *tr, char **nodes_full_path) } /* * Adds device tree nodes under target node. - * We use tr->dt_host_new to unflatten the updated device_tree_flattened. This - * is done to avoid the removal of device_tree generation, iomem regions mapping - * to hardware domain done by handle_node(). + * We use tr->dt_host_new to unflatten the updated device_tree_flattened. */ static long handle_add_overlay_nodes(void *overlay_fdt, uint32_t overlay_fdt_size) @@ -774,20 +836,6 @@ static long handle_add_overlay_nodes(void *overlay_fdt, goto err; } - tr->irq_ranges = rangeset_new(hardware_domain, "Overlays: Interrupts", 0); - if (tr->irq_ranges == NULL) - { - printk(XENLOG_ERR "Creating IRQ rangeset failed"); - goto err; - } - - tr->iomem_ranges = rangeset_new(hardware_domain, "Overlay: I/O Memory", 0); - if (tr->iomem_ranges == NULL) - { - printk(XENLOG_ERR "Creating IOMMU rangeset failed"); - goto err; - } - rc = add_nodes(tr, nodes_full_path); if ( rc ) { @@ -843,14 +891,165 @@ static long handle_add_overlay_nodes(void *overlay_fdt, xfree(tr->nodes_address); xfree(tr->fdt); - rangeset_destroy(tr->irq_ranges); - rangeset_destroy(tr->iomem_ranges); - xfree(tr); return rc; } +static long handle_detach_overlay_nodes(struct domain *d, + const void *overlay_fdt, + uint32_t overlay_fdt_size) +{ + int rc; + unsigned int j; + struct overlay_track *entry; + + rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size); + if ( rc ) + return rc; + + spin_lock(&overlay_lock); + + entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size); + if ( entry == NULL ) + { + rc = -EINVAL; + goto out; + } + + for ( j = 0; j < entry->num_nodes; j++ ) + { + struct dt_device_node *overlay_node; + + overlay_node = (struct dt_device_node *)entry->nodes_address[j]; + if ( overlay_node == NULL ) + { + rc = -EINVAL; + goto out; + } + + write_lock(&dt_host_lock); + rc = remove_descendant_nodes_resources(overlay_node, d); + if ( rc ) + { + write_unlock(&dt_host_lock); + goto out; + } + + rc = remove_node_resources(overlay_node, d); + if ( rc ) + { + write_unlock(&dt_host_lock); + goto out; + } + write_unlock(&dt_host_lock); + + rc = remove_all_irqs(entry->irq_ranges, d); + if ( rc ) + goto out; + + rc = remove_all_iomems(entry->iomem_ranges, d); + if ( rc ) + goto out; + } + + /* Remove IRQ access. */ + if ( entry->irq_ranges ) + { + rc = rangeset_consume_ranges(entry->irq_ranges, irq_remove_cb, d); + if ( rc ) + goto out; + } + + /* Remove mmio access. */ + if ( entry->iomem_ranges ) + { + rc = rangeset_consume_ranges(entry->iomem_ranges, iomem_remove_cb, d); + if ( rc ) + goto out; + } + + rangeset_destroy(entry->irq_ranges); + rangeset_destroy(entry->iomem_ranges); + + out: + spin_unlock(&overlay_lock); + + return rc; +} + +static long handle_attach_overlay_nodes(struct domain *d, + const void *overlay_fdt, + uint32_t overlay_fdt_size) +{ + int rc; + unsigned int j; + struct overlay_track *entry; + + rc = check_overlay_fdt(overlay_fdt, overlay_fdt_size); + if ( rc ) + return rc; + + spin_lock(&overlay_lock); + + entry = find_track_entry_from_tracker(overlay_fdt, overlay_fdt_size); + if ( entry == NULL ) + { + rc = -EINVAL; + goto out; + } + + entry->irq_ranges = rangeset_new(d, "Overlays: Interrupts", 0); + if (entry->irq_ranges == NULL) + { + rc = -ENOMEM; + printk(XENLOG_ERR "Creating IRQ rangeset failed"); + goto out; + } + + entry->iomem_ranges = rangeset_new(d, "Overlay: I/O Memory", 0); + if (entry->iomem_ranges == NULL) + { + rc = -ENOMEM; + printk(XENLOG_ERR "Creating IOMMU rangeset failed"); + goto out; + } + + for ( j = 0; j < entry->num_nodes; j++ ) + { + struct dt_device_node *overlay_node; + + overlay_node = (struct dt_device_node *)entry->nodes_address[j]; + if ( overlay_node == NULL ) + { + rc = -EINVAL; + goto out; + } + + write_lock(&dt_host_lock); + rc = handle_device(d, overlay_node, p2m_mmio_direct_c, + entry->iomem_ranges, entry->irq_ranges); + write_unlock(&dt_host_lock); + if ( rc ) + { + printk(XENLOG_ERR "Adding IRQ and IOMMU failed\n"); + goto out; + } + } + + spin_unlock(&overlay_lock); + + return 0; + + out: + spin_unlock(&overlay_lock); + + rangeset_destroy(entry->irq_ranges); + rangeset_destroy(entry->iomem_ranges); + + return rc; +} + long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op) { long ret; @@ -890,6 +1089,45 @@ long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op) return ret; } +long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op) +{ + long ret; + void *overlay_fdt; + + if ( op->overlay_op != XEN_DOMCTL_DT_OVERLAY_ATTACH && + op->overlay_op != XEN_DOMCTL_DT_OVERLAY_DETACH ) + return -EOPNOTSUPP; + + if ( op->overlay_fdt_size == 0 || op->overlay_fdt_size > KB(500) ) + return -EINVAL; + + if ( op->pad[0] || op->pad[1] || op->pad[2] ) + return -EINVAL; + + overlay_fdt = xmalloc_bytes(op->overlay_fdt_size); + + if ( overlay_fdt == NULL ) + return -ENOMEM; + + ret = copy_from_guest(overlay_fdt, op->overlay_fdt, op->overlay_fdt_size); + if ( ret ) + { + gprintk(XENLOG_ERR, "copy from guest failed\n"); + xfree(overlay_fdt); + + return -EFAULT; + } + + if ( op->overlay_op == XEN_DOMCTL_DT_OVERLAY_DETACH ) + ret = handle_detach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size); + else + ret = handle_attach_overlay_nodes(d, overlay_fdt, op->overlay_fdt_size); + + xfree(overlay_fdt); + + return ret; +} + /* * Local variables: * mode: C diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h index a33f9ec32b..81efb39014 100644 --- a/xen/include/public/domctl.h +++ b/xen/include/public/domctl.h @@ -1190,6 +1190,17 @@ struct xen_domctl_vmtrace_op { typedef struct xen_domctl_vmtrace_op xen_domctl_vmtrace_op_t; DEFINE_XEN_GUEST_HANDLE(xen_domctl_vmtrace_op_t); +#if defined(__arm__) || defined(__aarch64__) +struct xen_domctl_dt_overlay { + XEN_GUEST_HANDLE_64(const_void) overlay_fdt; /* IN: overlay fdt. */ + uint32_t overlay_fdt_size; /* IN: Overlay dtb size. */ +#define XEN_DOMCTL_DT_OVERLAY_ATTACH 1 +#define XEN_DOMCTL_DT_OVERLAY_DETACH 2 + uint8_t overlay_op; /* IN: Attach or detach. */ + uint8_t pad[3]; /* IN: Must be zero. */ +}; +#endif + struct xen_domctl { uint32_t cmd; #define XEN_DOMCTL_createdomain 1 @@ -1277,6 +1288,7 @@ struct xen_domctl { #define XEN_DOMCTL_vmtrace_op 84 #define XEN_DOMCTL_get_paging_mempool_size 85 #define XEN_DOMCTL_set_paging_mempool_size 86 +#define XEN_DOMCTL_dt_overlay 87 #define XEN_DOMCTL_gdbsx_guestmemio 1000 #define XEN_DOMCTL_gdbsx_pausevcpu 1001 #define XEN_DOMCTL_gdbsx_unpausevcpu 1002 @@ -1339,6 +1351,9 @@ struct xen_domctl { struct xen_domctl_vuart_op vuart_op; struct xen_domctl_vmtrace_op vmtrace_op; struct xen_domctl_paging_mempool paging_mempool; +#if defined(__arm__) || defined(__aarch64__) + struct xen_domctl_dt_overlay dt_overlay; +#endif uint8_t pad[128]; } u; }; diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h index febaa4b16a..3a6e7d48f0 100644 --- a/xen/include/public/sysctl.h +++ b/xen/include/public/sysctl.h @@ -1184,14 +1184,11 @@ typedef struct xen_sysctl_cpu_policy xen_sysctl_cpu_policy_t; DEFINE_XEN_GUEST_HANDLE(xen_sysctl_cpu_policy_t); #endif -#if defined(__arm__) || defined (__aarch64__) +#if defined(__arm__) || defined(__aarch64__) /* * XEN_SYSCTL_dt_overlay - * Performs addition/removal of device tree nodes under parent node using dtbo. - * This does in three steps: - * - Adds/Removes the nodes from dt_host. - * - Adds/Removes IRQ permission for the nodes. - * - Adds/Removes MMIO accesses. + * Performs addition/removal of device tree nodes under parent node using dtbo + * from dt_host. */ struct xen_sysctl_dt_overlay { XEN_GUEST_HANDLE_64(const_void) overlay_fdt; /* IN: overlay fdt. */ @@ -1265,7 +1262,7 @@ struct xen_sysctl { struct xen_sysctl_cpu_policy cpu_policy; #endif -#if defined(__arm__) || defined (__aarch64__) +#if defined(__arm__) || defined(__aarch64__) struct xen_sysctl_dt_overlay dt_overlay; #endif uint8_t pad[128]; diff --git a/xen/include/xen/dt-overlay.h b/xen/include/xen/dt-overlay.h index c0567741ee..7f0f3741b5 100644 --- a/xen/include/xen/dt-overlay.h +++ b/xen/include/xen/dt-overlay.h @@ -39,15 +39,22 @@ struct overlay_track { }; struct xen_sysctl_dt_overlay; +struct xen_domctl_dt_overlay; #ifdef CONFIG_OVERLAY_DTB long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op); +long dt_overlay_domctl(struct domain *d, struct xen_domctl_dt_overlay *op); #else #include static inline long dt_overlay_sysctl(struct xen_sysctl_dt_overlay *op) { return -EOPNOTSUPP; } +static inline long dt_overlay_domctl(struct domain *d, + struct xen_domctl_dt_overlay *op) +{ + return -EOPNOTSUPP; +} #endif #endif /* __XEN_DT_OVERLAY_H__ */ From patchwork Tue May 21 04:35:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13668908 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 32118C25B74 for ; Tue, 21 May 2024 04:36:30 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.726504.1130842 (Exim 4.92) (envelope-from ) id 1s9HEb-0008Lb-L8; Tue, 21 May 2024 04:36:21 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 726504.1130842; Tue, 21 May 2024 04:36:21 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HEb-0008LO-HZ; Tue, 21 May 2024 04:36:21 +0000 Received: by outflank-mailman (input) for mailman id 726504; Tue, 21 May 2024 04:36:19 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HEZ-0006hu-OB for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:19 +0000 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on20600.outbound.protection.outlook.com [2a01:111:f403:2409::600]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id a91c120f-172b-11ef-b4bb-af5377834399; Tue, 21 May 2024 06:36:17 +0200 (CEST) Received: from SA1PR04CA0016.namprd04.prod.outlook.com (2603:10b6:806:2ce::21) by DS0PR12MB8504.namprd12.prod.outlook.com (2603:10b6:8:155::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Tue, 21 May 2024 04:36:15 +0000 Received: from SA2PEPF00001509.namprd04.prod.outlook.com (2603:10b6:806:2ce:cafe::3f) by SA1PR04CA0016.outlook.office365.com (2603:10b6:806:2ce::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.36 via Frontend Transport; Tue, 21 May 2024 04:36:15 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by SA2PEPF00001509.mail.protection.outlook.com (10.167.242.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:15 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May 2024 23:36:14 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Mon, 20 May 2024 23:36:13 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: a91c120f-172b-11ef-b4bb-af5377834399 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NHWCQdml5dsyd4yFmEEqFcnqeCsYYa5zf+yjVwVhDiX5MSfRcJxncv90F+LduLO9bvmyQsHmiUri+z/Kb2xBX7vLPPJUyBKRcujNLDyMO9/JHEAd/m1EJeuov9awrm+ol3BCx7WfI1aY3Shv+q5smepX1ehEbXkwl9cEtOMtGKCi8DL8ePpW8O25Hkvh360xHP5XjV51Tk1PsyKtj8fnDqYWFVLsvqXmEoszriGnT75x+btEwizXBwgo6ZGn/m0TIAlD+QBgrQWpV4cwVO34zkx3pJCKhAZgZNXskNPyYCgl0KVnk0T3rqStQ+pPkCawPhH5vyni9QpaDzIF0LUIWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BE7z9HeLzOE9JPXCOJkwYcS8CjTT2Ru5y5EpVZBDA6c=; b=i3ZuKH//kSg9znsMnvoRrbQSZyIP5KMKXf3CPo+uCxrRFIcqnodr9PW3VX98kx0la017YA0nlSaI2yQrP7iPmPPbjN4tPjVu6rgUyqXlIm1TgV/qmIKh32SMfpTVrlM/tB47h9P0MAydbuzvli9T+J6BvQzDrEbNs3zGFdZ1Owo5lyRZkP02GgG44ZdFVrl1E7s6sTuNxED2QfUGoEysvQiGNO7/HXUqvZIho99mSDo4KtV+hbBN82c7dWM4qEnhRyXW+f5jGyfBajpfYQCR0/iRjqQAuWvq/iqzuWdTxYjFzZ4paK95VIiwy9zYujf0hlFS3AG6n02Zg6zKiFPR4Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BE7z9HeLzOE9JPXCOJkwYcS8CjTT2Ru5y5EpVZBDA6c=; b=fDcBnNDMgjgGpw58GfvPp6nkvXvcuoNk2zHHI/ETgxZB+/dR5nmkh3/01WtQ7bIq7cN4K1HXjjrQAHQYrcK5Lo5zvC9/Tukq728sEtWEElUX91wAoZcBezD1h7pQyA4AJH1OMlfRa11N36FvmORZgetYhmnsaK3CA9ACi0pqQqw= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Henry Wang To: CC: Anthony PERARD , Juergen Gross , Henry Wang Subject: [PATCH v3 7/8] tools: Introduce the "xl dt-overlay {attach,detach}" commands Date: Tue, 21 May 2024 12:35:56 +0800 Message-ID: <20240521043557.1580753-8-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com> References: <20240521043557.1580753-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00001509:EE_|DS0PR12MB8504:EE_ X-MS-Office365-Filtering-Correlation-Id: 4b9fe93c-9dbc-4460-2658-08dc794f8cc2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004; X-Microsoft-Antispam-Message-Info: Fzb3V6t8baIST7lGd/94pmH7QN5oUjDSEYhvlfpybNneUdap1+G/ESLH9IlklOPHq1t/2P/HUSEYhmy263qZdy+O++CuiXaO+NGdKF5OhsixsVACXWj3cJGBd4vTMHNnFTEeT7uliTlqaGjyFaJZfIqVYNH6mXbL5GedV6ltCNNUKB9wI1LhBr4YfH6ei3l2ExajslKGVDezuthZ7dM5vJ9OaqZFIPiN3xFwaZUUKw2OreQ3IoyduyL/dF3bhQCtazN+Tlxbf8ttpO9ZKyhxWC9Pt5PWUgpZeIpMdZXGI3KqNKjsRacfbBRKUDc39fTl5vI8aUmBODFpYtz6TQCOvP1BM4X7F5BOzho2vD2VDnrFUD70k+eYAUkKiikJQuP2dRNgCQvpWa6Vrq85h8kr4xBMgZrf9An/j/zlYGa2Hs+v5gChKdbOM4J51UW4BkcKil/LG1m1YeFdeuqlyEXT35fcTJiKWwYBoXAtOJU1Ctz6u0CetiVATZZAkfxxcIyRfaELdtkq09EHI5usewpQYbpKzBcFsYLHjP5XxIGYx44SE89KFyw1cjb+C6jId87kACSCaD6l8cqmuPbXFpxhXNt8VjH8jwDcLkyMVd8OemML4+m7htmAyYCfTAw//bRmUd016JFfGpNGYg25jueLnIOvsd1BHS7UfNuRHzneNH4mBShJd20kPwLh/p7lxIaNxfVD5J8QMcX2ipbjUgCdhJso2yWVZi0vVfvRdiwPuxv7Xby4fYGBUhjPUnWXsXveWSiq0db5q9GyPxv3L9fnOLOrS6U8i9pwnDMk8Mnw0z6GX7piI185MoyBu3FbOnlEnpVRA4kfU7bz7cHJXK+KGNB7NaUGaCkcZSJv8TJfGbfjJoWDO5/4Y14xsmwCu5aPwtcXJJaiXW4bSrRz0QSWjLvOM63UVjP/exc9C8DwjTJ73R+nE5v1Mo5ZCka0DHybkAiOf36SKDy4neU6wIQquOJYJc1D68OuNOOoOdx1H8EL0hoibt0mJ8GmWI/YAeFAMS9fX5Ygu4cZbLUTA9dEfyU5M4KH9/BdfSXnCEA6FfDoo4O1+pLbQM0LNXnsBDV+LNFm5uWyiN/rgsDyNij1SEFi9mKXHmx6k/+HeHLBGRnIQnMCBwNRfMOb4z3yLXDYVnrjmAWBb4FxGRkHyTcZwrrF1EsHne4MQ5ftU6vf/9jjdzUAfIsmSDfmAmL7MWk+eqxyHr48ROXG1K+c7nTS0l5gMctDbzHAIzvEOMgw6+2YAS0edmW9HzgBwzzhsUDCgYMoxd2Yi1zyzFcQiQjVO77dn33dNFpZRWbdbDR5gKt0v3wkW8Uj5SKUjKK2Gqj1VtEXuk4uhpezehvxM1pDQPp10iwbz51ao9X/szKXkltQbNkmNnczlkndUYpnXgTL X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:15.0054 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4b9fe93c-9dbc-4460-2658-08dc794f8cc2 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00001509.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8504 With the XEN_DOMCTL_dt_overlay DOMCTL added, users should be able to attach/detach devices from the provided DT overlay to domains. Support this by introducing a new set of "xl dt-overlay" commands and related documentation, i.e. "xl dt-overlay {attach,detach}". Slightly rework the command option parsing logic. Signed-off-by: Henry Wang Reviewed-by: Jason Andryuk --- v3: - Introduce new API libxl_dt_overlay_domain() and co., instead of reusing existing API libxl_dt_overlay(). - Add in-code comments for the LIBXL_DT_OVERLAY_* macros. - Use find_domain() to avoid getting domain_id from strtol(). v2: - New patch. --- tools/include/libxl.h | 10 +++++++ tools/include/xenctrl.h | 3 +++ tools/libs/ctrl/xc_dt_overlay.c | 31 +++++++++++++++++++++ tools/libs/light/libxl_dt_overlay.c | 28 +++++++++++++++++++ tools/xl/xl_cmdtable.c | 4 +-- tools/xl/xl_vmcontrol.c | 42 ++++++++++++++++++++--------- 6 files changed, 104 insertions(+), 14 deletions(-) diff --git a/tools/include/libxl.h b/tools/include/libxl.h index 62cb07dea6..6cc6d6bf6a 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -2549,8 +2549,18 @@ libxl_device_pci *libxl_device_pci_list(libxl_ctx *ctx, uint32_t domid, void libxl_device_pci_list_free(libxl_device_pci* list, int num); #if defined(__arm__) || defined(__aarch64__) +/* Values should keep consistent with the op from XEN_SYSCTL_dt_overlay */ +#define LIBXL_DT_OVERLAY_ADD 1 +#define LIBXL_DT_OVERLAY_REMOVE 2 int libxl_dt_overlay(libxl_ctx *ctx, void *overlay, uint32_t overlay_size, uint8_t overlay_op); + +/* Values should keep consistent with the op from XEN_DOMCTL_dt_overlay */ +#define LIBXL_DT_OVERLAY_DOMAIN_ATTACH 1 +#define LIBXL_DT_OVERLAY_DOMAIN_DETACH 2 +int libxl_dt_overlay_domain(libxl_ctx *ctx, uint32_t domain_id, + void *overlay_dt, uint32_t overlay_dt_size, + uint8_t overlay_op); #endif /* diff --git a/tools/include/xenctrl.h b/tools/include/xenctrl.h index 4996855944..9ceca0cffc 100644 --- a/tools/include/xenctrl.h +++ b/tools/include/xenctrl.h @@ -2657,6 +2657,9 @@ int xc_domain_cacheflush(xc_interface *xch, uint32_t domid, #if defined(__arm__) || defined(__aarch64__) int xc_dt_overlay(xc_interface *xch, void *overlay_fdt, uint32_t overlay_fdt_size, uint8_t overlay_op); +int xc_dt_overlay_domain(xc_interface *xch, void *overlay_fdt, + uint32_t overlay_fdt_size, uint8_t overlay_op, + uint32_t domain_id); #endif /* Compat shims */ diff --git a/tools/libs/ctrl/xc_dt_overlay.c b/tools/libs/ctrl/xc_dt_overlay.c index c2224c4d15..ea1da522d1 100644 --- a/tools/libs/ctrl/xc_dt_overlay.c +++ b/tools/libs/ctrl/xc_dt_overlay.c @@ -48,3 +48,34 @@ err: return err; } + +int xc_dt_overlay_domain(xc_interface *xch, void *overlay_fdt, + uint32_t overlay_fdt_size, uint8_t overlay_op, + uint32_t domain_id) +{ + int err; + struct xen_domctl domctl = { + .cmd = XEN_DOMCTL_dt_overlay, + .domain = domain_id, + .u.dt_overlay = { + .overlay_op = overlay_op, + .overlay_fdt_size = overlay_fdt_size, + } + }; + + DECLARE_HYPERCALL_BOUNCE(overlay_fdt, overlay_fdt_size, + XC_HYPERCALL_BUFFER_BOUNCE_IN); + + if ( (err = xc_hypercall_bounce_pre(xch, overlay_fdt)) ) + goto err; + + set_xen_guest_handle(domctl.u.dt_overlay.overlay_fdt, overlay_fdt); + + if ( (err = do_domctl(xch, &domctl)) != 0 ) + PERROR("%s failed", __func__); + +err: + xc_hypercall_bounce_post(xch, overlay_fdt); + + return err; +} diff --git a/tools/libs/light/libxl_dt_overlay.c b/tools/libs/light/libxl_dt_overlay.c index a6c709a6dc..00503b76bd 100644 --- a/tools/libs/light/libxl_dt_overlay.c +++ b/tools/libs/light/libxl_dt_overlay.c @@ -69,3 +69,31 @@ out: return rc; } +int libxl_dt_overlay_domain(libxl_ctx *ctx, uint32_t domain_id, + void *overlay_dt, uint32_t overlay_dt_size, + uint8_t overlay_op) +{ + int rc; + int r; + GC_INIT(ctx); + + if (check_overlay_fdt(gc, overlay_dt, overlay_dt_size)) { + LOG(ERROR, "Overlay DTB check failed"); + rc = ERROR_FAIL; + goto out; + } else { + LOG(DEBUG, "Overlay DTB check passed"); + rc = 0; + } + + r = xc_dt_overlay_domain(ctx->xch, overlay_dt, overlay_dt_size, overlay_op, + domain_id); + if (r) { + LOG(ERROR, "%s: Attaching/Detaching overlay dtb failed.", __func__); + rc = ERROR_FAIL; + } + +out: + GC_FREE; + return rc; +} diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c index 1f3c6b5897..37770b20e3 100644 --- a/tools/xl/xl_cmdtable.c +++ b/tools/xl/xl_cmdtable.c @@ -634,8 +634,8 @@ const struct cmd_spec cmd_table[] = { #ifdef LIBXL_HAVE_DT_OVERLAY { "dt-overlay", &main_dt_overlay, 0, 1, - "Add/Remove a device tree overlay", - "add/remove <.dtbo>", + "Add/Remove a device tree overlay to Xen device tree, attach/detach the device to a domain", + " <.dtbo> OR <.dtbo> ", "-h print this help\n" }, #endif diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c index 02575d5d36..ccfeec39f1 100644 --- a/tools/xl/xl_vmcontrol.c +++ b/tools/xl/xl_vmcontrol.c @@ -1268,32 +1268,46 @@ int main_create(int argc, char **argv) #ifdef LIBXL_HAVE_DT_OVERLAY int main_dt_overlay(int argc, char **argv) { - const char *overlay_ops = NULL; const char *overlay_config_file = NULL; void *overlay_dtb = NULL; int rc; uint8_t op; int overlay_dtb_size = 0; - const int overlay_add_op = 1; - const int overlay_remove_op = 2; + uint32_t domain_id = 0; + bool domain_op = false; if (argc < 2) { help("dt-overlay"); return EXIT_FAILURE; } - overlay_ops = argv[1]; - overlay_config_file = argv[2]; - - if (strcmp(overlay_ops, "add") == 0) - op = overlay_add_op; - else if (strcmp(overlay_ops, "remove") == 0) - op = overlay_remove_op; - else { + if (strcmp(argv[optind], "add") == 0) + op = LIBXL_DT_OVERLAY_ADD; + else if (strcmp(argv[optind], "remove") == 0) + op = LIBXL_DT_OVERLAY_REMOVE; + else if (strcmp(argv[optind], "attach") == 0) { + op = LIBXL_DT_OVERLAY_DOMAIN_ATTACH; + domain_op = true; + } else if (strcmp(argv[optind], "detach") == 0) { + op = LIBXL_DT_OVERLAY_DOMAIN_DETACH; + domain_op = true; + } else { fprintf(stderr, "Invalid dt overlay operation\n"); return EXIT_FAILURE; } + overlay_config_file = argv[optind+1]; + + if (domain_op) { + if (argc <= optind + 2) { + fprintf(stderr, "Missing domain ID\n"); + help("dt-overlay"); + return EXIT_FAILURE; + } else { + domain_id = find_domain(argv[optind+2]); + } + } + if (overlay_config_file) { rc = libxl_read_file_contents(ctx, overlay_config_file, &overlay_dtb, &overlay_dtb_size); @@ -1309,7 +1323,11 @@ int main_dt_overlay(int argc, char **argv) return EXIT_FAILURE; } - rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op); + if (!domain_op) + rc = libxl_dt_overlay(ctx, overlay_dtb, overlay_dtb_size, op); + else + rc = libxl_dt_overlay_domain(ctx, domain_id, overlay_dtb, + overlay_dtb_size, op); free(overlay_dtb); From patchwork Tue May 21 04:35:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Henry Wang X-Patchwork-Id: 13668909 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 08E69C25B7B for ; Tue, 21 May 2024 04:36:31 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.726506.1130854 (Exim 4.92) (envelope-from ) id 1s9HEd-0000BB-5d; Tue, 21 May 2024 04:36:23 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 726506.1130854; Tue, 21 May 2024 04:36:23 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HEc-000091-Te; Tue, 21 May 2024 04:36:22 +0000 Received: by outflank-mailman (input) for mailman id 726506; Tue, 21 May 2024 04:36:21 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1s9HEb-0006hu-QU for xen-devel@lists.xenproject.org; Tue, 21 May 2024 04:36:21 +0000 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on20601.outbound.protection.outlook.com [2a01:111:f403:2418::601]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id aaab29f0-172b-11ef-b4bb-af5377834399; Tue, 21 May 2024 06:36:19 +0200 (CEST) Received: from SA9PR13CA0038.namprd13.prod.outlook.com (2603:10b6:806:22::13) by LV8PR12MB9184.namprd12.prod.outlook.com (2603:10b6:408:18f::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7587.35; Tue, 21 May 2024 04:36:17 +0000 Received: from SA2PEPF00001507.namprd04.prod.outlook.com (2603:10b6:806:22:cafe::c3) by SA9PR13CA0038.outlook.office365.com (2603:10b6:806:22::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7611.17 via Frontend Transport; Tue, 21 May 2024 04:36:17 +0000 Received: from SATLEXMB03.amd.com (165.204.84.17) by SA2PEPF00001507.mail.protection.outlook.com (10.167.242.39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7611.14 via Frontend Transport; Tue, 21 May 2024 04:36:17 +0000 Received: from SATLEXMB04.amd.com (10.181.40.145) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 20 May 2024 23:36:16 -0500 Received: from henry-MS-7D54.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server id 15.1.2507.35 via Frontend Transport; Mon, 20 May 2024 23:36:14 -0500 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: aaab29f0-172b-11ef-b4bb-af5377834399 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fywDq+F8ZGMrfUxKPqSqqF3oAolIRpEEbBFbzBbAa8KXVzudey5rsgsIMbNRln5vrQHdRLJ9ab5T7pk215gEzEvj0OsmDfhE1f8optPrabs0UyZOvnWuJf42OMe5FZ4qY0ovnWcH4D23pXFa0mUa4XuR2B+HMp7YC8+Q45sJqq0Mi3hUq9FeJ9HH2/UQe1qKq/dw6P4b8/JEI1kGeEp8Fu/jeWaeu9/moU+vo09wA+yWHLlvSAMMaMK93PEW80sUbWEgkZoUtJ45OyNrlU8iwhfeYaPSwuShMpH2WAUL27clBZPq8S0/OckiRJeYlDnASdMpd03zChjsImOe0+kOVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XLkeA8mEKzf4yLdB3J64v/O8UY7F2ncd41W/P3+Us/g=; b=UUub2UTTVSpd/hDLNf1Id5Kw4FolJSjfqMNtapVVOJr0CNMcYp4XXmDzQiz84aapscx3VbC0ybGwWqvDLhIKkC6woFYbNOfCdPGh4TCCq6t3L18JbWcXgtri1/g5VaVd5xq2rHrpDrayXC1RGXxnx8mO1QyFa4bBClO5NPBrpRpA6+An2BX1tr5U0W0ehABmiPadlouqzF74Vxl0pXiUyrdabfPynS5j4LbZLXSdfE7GYygv2M0EYjfhAHZcnqe0g6sKXhn2X6DpqPYXQrlJIr5bGO2moTryTgPpm1j2BBUdDZsCzudJc19YU9PPryCoFSjnQ0zvxJQnIX3yxgjN3w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XLkeA8mEKzf4yLdB3J64v/O8UY7F2ncd41W/P3+Us/g=; b=JDTEYIVXoeRRJgh+spDH81Yzkfx9TPDdd7H346WpzdFVni27wvGUP/7dE5HUaZsWJBFCG2HnpRO/FI4YAibEySljg+P7qfVlNeUlBseFSmA+LjZgcH1uZ9Byw5tniT8KGp/+OLHbj71/mfFnQLGtJOOFC2/77YDBx1dtaWUBAZU= X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB03.amd.com; pr=C From: Henry Wang To: CC: Stefano Stabellini , Julien Grall , Bertrand Marquis , Michal Orzel , Volodymyr Babchuk , "Henry Wang" Subject: [PATCH v3 8/8] docs: Add device tree overlay documentation Date: Tue, 21 May 2024 12:35:57 +0800 Message-ID: <20240521043557.1580753-9-xin.wang2@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240521043557.1580753-1-xin.wang2@amd.com> References: <20240521043557.1580753-1-xin.wang2@amd.com> MIME-Version: 1.0 Received-SPF: None (SATLEXMB03.amd.com: xin.wang2@amd.com does not designate permitted sender hosts) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SA2PEPF00001507:EE_|LV8PR12MB9184:EE_ X-MS-Office365-Filtering-Correlation-Id: 6fc71ecd-dcde-4d0a-93da-08dc794f8e08 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|1800799015|82310400017|376005|36860700004; X-Microsoft-Antispam-Message-Info: TBe6ZPZElNPFoIHFtmMPTFQQ9IZdZDGoI6LfPDW8BdhsaoX3Uy1pdWfPHH05aidU7lt3oXxEXHQjtrBCxlbQq5KOnisUzbbhiQolp1BAM3RjxFiDfFAgmhmgdL1VuA/Upb0OfbwtU+Uqs9cnOyT4UNxBuM0SIaDo5rmZqxqGqynnZzJaGt9s2kAUQ7FFoQ+n1HJmElJR7b37QqyjXhm3UE9kVsXYL7p1Yi+RZMasUSYfp/np1BtP/LgVgKhWHcsxiYGWNRgzOKTfDIXBR3lV4NUeL3c0c9uXtZORXjNw0fE5smxIfxB4LPHyyDVFrjRJ02em5GH7K31i7uxoW8awXiGFepv3jdlNn3zC8LJcGBGlPcEwHNb9IDif/kWw5PaWXKBj7XIo6eTQE0fs+vnHiKUjoGGDIo13mjg+OWlJEJxkblHavDEwq0yfIcEU3n6X0vhxQAkk1sNtmw2fe4ZNP4bwICahgWM0/bGbKGZwbWLbCHxcVbXerDMx8Dqgt1J1sOi+BjWH72GhaauoqTY4niLCxA9Z0oWbSNu5GE1o+Sv5rlnWOkcWflKFr6TA3UQDaX7/rlhIRd7vNGP+avEL/Jc/xP9GjhiaObiBey8O79FDb28sDntQwC9kGth8+gpY4o+Hc0dElZm0H62aifQ2MGmeZBhyKxR1fvw2DIG3UwwXlGQmEogDYPfxKIecucdUsidVQ1d4MdTKmiE6ohXXbIoqp6a0bpCeIucUCrbofKAgVZ/u4KtR+/1g/u1wSnr6zpc0UXfZhx3KwkVQ6X3FG1+2uFlocRz3cPjlwfbbKe+CfFJI8kJPzIXvojV3t7xseqSw5stbQuD3saALNrTWej5pI4yP3uo3OHX4bMthi+lfFFjv5WjNI5fg6CG+I8AxfHs9whFINvUnol8u9XvRc7k4UyPH+ofMRoPydf/9YnerRb2w0g9xumXvMa6aWIKj70hAIPUrp2QMyTo/AMa+gOcfcaS7rSI7/NcL8VPD+6kHbwTW9UL7LgbMHu0TfeUusEEaQKclleAMAMLw1n+VX8QWmeiduGPEB34LOE8pnQYOGHBPrKhXPe+mBxjk8OdJWSwlG9jcvvAp1iHLQ7yrbhcjXIYeY2xvyItdn4MEGKpNclZ6sTmSiNxpnO9CowvtGiLtCRgs/RH0qVSOoDg11KOjFdtPmllmxRZ0Kxg6GWK/19D1h3F4OL9iFN5WMk3aFZ9CBdcXfDJygGon7LpzZNA4oSERyvCkZhWmJzlQUDek0AVZ5hVgx+FtL+x08FshCcysfgtwZJHAzEVv/Ux9TKW4k7wzOxwqu4rl1iq9LmYmrDcjdxIdyzElOozVliSj X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB03.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230031)(1800799015)(82310400017)(376005)(36860700004);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 May 2024 04:36:17.1446 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6fc71ecd-dcde-4d0a-93da-08dc794f8e08 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB03.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF00001507.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR12MB9184 From: Vikram Garhwal Signed-off-by: Vikram Garhwal Signed-off-by: Stefano Stabellini Signed-off-by: Henry Wang --- v3: - No change. v2: - Update the content based on the changes in this version. --- docs/misc/arm/overlay.txt | 99 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100644 docs/misc/arm/overlay.txt diff --git a/docs/misc/arm/overlay.txt b/docs/misc/arm/overlay.txt new file mode 100644 index 0000000000..811a6de369 --- /dev/null +++ b/docs/misc/arm/overlay.txt @@ -0,0 +1,99 @@ +# Device Tree Overlays support in Xen + +Xen now supports dynamic device assignment to running domains, +i.e. adding/removing nodes (using .dtbo) to/from Xen device tree, and +attaching/detaching them to/from a running domain with given $domid. + +Dynamic node assignment works in two steps: + +## Add/Remove device tree overlay to/from Xen device tree + +1. Xen tools check the dtbo given and parse all other user provided arguments +2. Xen tools pass the dtbo to Xen hypervisor via hypercall. +3. Xen hypervisor applies/removes the dtbo to/from Xen device tree. + +## Attach/Detach device from the DT overlay to/from domain + +1. Xen tools check the dtbo given and parse all other user provided arguments +2. Xen tools pass the dtbo to Xen hypervisor via hypercall. +3. Xen hypervisor attach/detach the device to/from the user-provided $domid by + mapping/unmapping node resources in the DT overlay. + +# Examples + +Here are a few examples on how to use it. + +## Dom0 device add + +For assigning a device tree overlay to Dom0, user should firstly properly +prepare the DT overlay. More information about device tree overlays can be +found in [1]. Then, in Dom0, enter the following: + + (dom0) xl dt-overlay add overlay.dtbo + +This will allocate the devices mentioned in overlay.dtbo to Xen device tree. + +To assign the newly added device from the dtbo to Dom0: + + (dom0) xl dt-overlay attach overlay.dtbo 0 + +Next, if the user wants to add the same device tree overlay to dom0 +Linux, execute the following: + + (dom0) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay + (dom0) cat overlay.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo + +Finally if needed, the relevant Linux kernel drive can be loaded using: + + (dom0) modprobe module_name.ko + +## Dom0 device remove + +For removing the device from Dom0, first detach the device from Dom0: + + (dom0) xl dt-overlay detach overlay.dtbo 0 + +NOTE: The user is expected to unload any Linux kernel modules which +might be accessing the devices in overlay.dtbo before detach the device. +Detaching devices without unloading the modules might result in a crash. + +Then remove the overlay from Xen device tree: + + (dom0) xl dt-overlay remove overlay.dtbo + +## DomU device add/remove + +All the nodes in dtbo will be assigned to a domain; the user will need +to prepare the dtb for the domU. For example, the `interrupt-parent` property +of the DomU overlay should be changed to the Xen hardcoded value `0xfde8`. +Below assumes the properly written DomU dtbo is `overlay_domu.dtbo`. + +User will need to create the DomU with below properties properly configured +in the xl config file: +- `iomem` +- `passthrough` (if IOMMU is needed) + +User will also need to modprobe the relevant drivers. + +Example for domU device add: + + (dom0) xl dt-overlay add overlay.dtbo # If not executed before + (dom0) xl dt-overlay attach overlay.dtbo $domid + (dom0) xl console $domid # To access $domid console + +Next, if the user needs to modify/prepare the overlay.dtbo suitable for +the domU: + + (domU) mkdir -p /sys/kernel/config/device-tree/overlays/new_overlay + (domU) cat overlay_domu.dtbo > /sys/kernel/config/device-tree/overlays/new_overlay/dtbo + +Finally, if needed, the relevant Linux kernel drive can be probed: + + (domU) modprobe module_name.ko + +Example for domU overlay remove: + + (dom0) xl dt-overlay detach overlay.dtbo $domid + (dom0) xl dt-overlay remove overlay.dtbo + +[1] https://www.kernel.org/doc/Documentation/devicetree/overlay-notes.txt