From patchwork Mon Nov 4 14:28:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Teddy Astie X-Patchwork-Id: 13861497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B934FD132CF for ; Mon, 4 Nov 2024 14:29:08 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.830027.1244939 (Exim 4.92) (envelope-from ) id 1t7y4U-0006Xl-E1; Mon, 04 Nov 2024 14:28:46 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 830027.1244939; Mon, 04 Nov 2024 14:28:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1t7y4U-0006Xe-An; Mon, 04 Nov 2024 14:28:46 +0000 Received: by outflank-mailman (input) for mailman id 830027; Mon, 04 Nov 2024 14:28:45 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1t7y4T-0006XR-R3 for xen-devel@lists.xenproject.org; Mon, 04 Nov 2024 14:28:45 +0000 Received: from mail133-1.atl131.mandrillapp.com (mail133-1.atl131.mandrillapp.com [198.2.133.1]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 163448e9-9ab9-11ef-a0c5-8be0dac302b0; Mon, 04 Nov 2024 15:28:41 +0100 (CET) Received: from pmta13.mandrill.prod.atl01.rsglab.com (localhost [127.0.0.1]) by mail133-1.atl131.mandrillapp.com (Mailchimp) with ESMTP id 4Xhv3B6LpYzBsThh9 for ; Mon, 4 Nov 2024 14:28:38 +0000 (GMT) Received: from [37.26.189.201] by mandrillapp.com id c2a022822c8b471b9422ea789444521b; Mon, 04 Nov 2024 14:28:38 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 163448e9-9ab9-11ef-a0c5-8be0dac302b0 X-Custom-Connection: eyJyZW1vdGVpcCI6IjE5OC4yLjEzMy4xIiwiaGVsbyI6Im1haWwxMzMtMS5hdGwxMzEubWFuZHJpbGxhcHAuY29tIn0= X-Custom-Transaction: eyJpZCI6IjE2MzQ0OGU5LTlhYjktMTFlZi1hMGM1LThiZTBkYWMzMDJiMCIsInRzIjoxNzMwNzMwNTIxLjQ0MDcyNCwic2VuZGVyIjoiYm91bmNlLW1kXzMwNTA0OTYyLjY3MjhkYTE2LnYxLWMyYTAyMjgyMmM4YjQ3MWI5NDIyZWE3ODk0NDQ1MjFiQGJvdW5jZS52YXRlcy50ZWNoIiwicmVjaXBpZW50IjoieGVuLWRldmVsQGxpc3RzLnhlbnByb2plY3Qub3JnIn0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandrillapp.com; s=mte1; t=1730730518; x=1730991018; bh=RoSAtnJfw1cEYNC93zFobpr3kWwShUZYpgY8h54ln+8=; h=From:Subject:To:Cc:Message-Id:Feedback-ID:Date:MIME-Version: Content-Type:Content-Transfer-Encoding:CC:Date:Subject:From; b=IAI+VmTHFBMuSLeUuuzX87KcSO8+P7bu4Xw7WfXzAIbH1n8wmNdwnxCBEkfikvJDn bRVa4cMX88yft3z1TdS7Jz7T3KKFQbzD0hEZBWzoN/3i8P8NsmMW0WIIODCJCW+yyx ISP8d5R3SWkamOcACw5AmoUbtAE8Sv6sSyADukWDtK1AOlc1ZU5Vz6zgiYpuiRfm9w 9RVSYVhK2qOP6++Z+HFkbTgWZphNt4K8VlY00JJJRskNoVJpTJZr5HD1lfPIfvNCZn x+ryENl1S2Ft7McjTA8P7A5ec8+gBtgYs0YqCWuE397Q/MPCBuKE+h+jya01kvSwUM TWryF+RX4XWvw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vates.tech; s=mte1; t=1730730518; x=1730991018; i=teddy.astie@vates.tech; bh=RoSAtnJfw1cEYNC93zFobpr3kWwShUZYpgY8h54ln+8=; h=From:Subject:To:Cc:Message-Id:Feedback-ID:Date:MIME-Version: Content-Type:Content-Transfer-Encoding:CC:Date:Subject:From; b=N/goh2IGiAkJC8th+v8dUVwV2a3vAnwmSx5ix+k0oobSL+9q+58xgUrgbmz/X7jfj C5xO8kWpBiYTR2uLD7wawat6D2iUDeYbMcf+iT44Pu0UKam3WiSzi+jEJYMB2Ipzi7 rg0bqZ/8eDdsWuE3UO08ewR+ee8G+FW6Zcp6pVezjo6plsY53NeO3c7wHKd9wcvyKu rbKrmWW8/zwnEd7mF1XGDYZ4q+ZPHFT7lFv0O64D/Pfi2f787YCFdSBdHP0n1Ps3Qk UWZwHjpO8jp368w7rJgEpjBdFVj77evf9gjkvOHfnViuvJZJuw1zqHjNF6WhwAnSdP 419RZG82DJSTw== From: "Teddy Astie" Subject: =?utf-8?q?=5BXEN_RFC_PATCH_v4_0/5=5D_IOMMU_subsystem_redesign_and_P?= =?utf-8?q?V-IOMMU_interface?= X-Mailer: git-send-email 2.45.2 X-Bm-Disclaimer: Yes X-Bm-Milter-Handled: 4ffbd6c1-ee69-4e1b-aabd-f977039bd3e2 X-Bm-Transport-Timestamp: 1730730516687 To: xen-devel@lists.xenproject.org Cc: "Teddy Astie" , "Andrew Cooper" , "Jan Beulich" , "Julien Grall" , "Stefano Stabellini" , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= , "Lukasz Hawrylko" , "Daniel P. Smith" , =?utf-8?q?Mateusz_M=C3=B3?= =?utf-8?q?wka?= , =?utf-8?q?Marek_Marczykowski-G?= =?utf-8?q?=C3=B3recki?= Message-Id: X-Native-Encoded: 1 X-Report-Abuse: =?utf-8?q?Please_forward_a_copy_of_this_message=2C_including?= =?utf-8?q?_all_headers=2C_to_abuse=40mandrill=2Ecom=2E_You_can_also_report_?= =?utf-8?q?abuse_here=3A_https=3A//mandrillapp=2Ecom/contact/abuse=3Fid=3D30?= =?utf-8?q?504962=2Ec2a022822c8b471b9422ea789444521b?= X-Mandrill-User: md_30504962 Feedback-ID: 30504962:30504962.20241104:md Date: Mon, 04 Nov 2024 14:28:38 +0000 MIME-Version: 1.0 This work has been presented at Xen Summit 2024 during the IOMMU paravirtualization and Xen IOMMU subsystem rework design session. Operating systems may want to have access to a IOMMU in order to do DMA protection or implement certain features (e.g VFIO on Linux). VFIO support is mandatory for framework such as SPDK, which can be useful to implement an alternative storage backend for virtual machines [1]. In this patch series, we introduce in Xen the ability to manage several contexts per domain and provide a new hypercall interface to allow guests to manage IOMMU contexts. The VT-d driver is updated to support these new features. [1] Using SPDK with the Xen hypervisor - FOSDEM 2023 --- Changed in v2 : * fixed Xen crash when dumping IOMMU contexts (using X debug key) with DomUs without IOMMU * s/dettach/detach/ * removed some unused includes * fix dangling devices in contexts with detach Changed in v3 : * lock entirely map/unmap in hypercall * prevent IOMMU operations on dying contexts (fix race condition) * iommu_check_context+iommu_get_context -> iommu_get_context and check for NULL Changed in v4 : * Part of initialization logic is moved to domain or toolstack (IOMMU_init) + domain/toolstack now decides on "context count" and "pagetable pool size" + for now, all domains are able to initialize PV-IOMMU * introduce "dom0-iommu=no-dma" to make default context block all DMA (disables HAP and sync-pt), enforcing usage of PV-IOMMU for DMA Can be used to expose properly "Pre-boot DMA protection" * redesigned locking logic for contexts + contexts are accessed using iommu_get_context and released with iommu_put_context TODO: * add stub implementations for bissecting needs and non-ported IOMMU implementations * fix some issues with no-dma+PV and grants * complete "no-dma" mode (expose to toolstack, add documentation, ...) * properly define nested mode and PASID support Teddy Astie (5): docs/designs: Add a design document for PV-IOMMU docs/designs: Add a design document for IOMMU subsystem redesign IOMMU: Introduce redesigned IOMMU subsystem VT-d: Port IOMMU driver to new subsystem xen/public: Introduce PV-IOMMU hypercall interface docs/designs/iommu-contexts.md | 403 +++++++ docs/designs/pv-iommu.md | 116 ++ xen/arch/x86/domain.c | 2 +- xen/arch/x86/include/asm/arena.h | 54 + xen/arch/x86/include/asm/iommu.h | 58 +- xen/arch/x86/include/asm/pci.h | 17 - xen/arch/x86/mm/p2m-ept.c | 2 +- xen/arch/x86/pv/dom0_build.c | 4 +- xen/arch/x86/tboot.c | 4 +- xen/common/Makefile | 1 + xen/common/memory.c | 4 +- xen/common/pv-iommu.c | 539 ++++++++++ xen/drivers/passthrough/Makefile | 3 + xen/drivers/passthrough/context.c | 711 +++++++++++++ xen/drivers/passthrough/iommu.c | 396 +++---- xen/drivers/passthrough/pci.c | 117 +- xen/drivers/passthrough/quarantine.c | 49 + xen/drivers/passthrough/vtd/Makefile | 2 +- xen/drivers/passthrough/vtd/extern.h | 14 +- xen/drivers/passthrough/vtd/iommu.c | 1478 +++++++++----------------- xen/drivers/passthrough/vtd/quirks.c | 20 +- xen/drivers/passthrough/x86/Makefile | 1 + xen/drivers/passthrough/x86/arena.c | 157 +++ xen/drivers/passthrough/x86/iommu.c | 270 +++-- xen/include/hypercall-defs.c | 6 + xen/include/public/pv-iommu.h | 341 ++++++ xen/include/public/xen.h | 1 + xen/include/xen/iommu.h | 117 +- xen/include/xen/pci.h | 3 + 29 files changed, 3423 insertions(+), 1467 deletions(-) create mode 100644 docs/designs/iommu-contexts.md create mode 100644 docs/designs/pv-iommu.md create mode 100644 xen/arch/x86/include/asm/arena.h create mode 100644 xen/common/pv-iommu.c create mode 100644 xen/drivers/passthrough/context.c create mode 100644 xen/drivers/passthrough/quarantine.c create mode 100644 xen/drivers/passthrough/x86/arena.c create mode 100644 xen/include/public/pv-iommu.h