From patchwork Thu Jun 9 10:18:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12875167 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 106B8C433EF for ; Thu, 9 Jun 2022 10:18:28 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.344881.570503 (Exim 4.92) (envelope-from ) id 1nzFF1-00027r-PK; Thu, 09 Jun 2022 10:18:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 344881.570503; Thu, 09 Jun 2022 10:18:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nzFF1-00027h-MF; Thu, 09 Jun 2022 10:18:15 +0000 Received: by outflank-mailman (input) for mailman id 344881; Thu, 09 Jun 2022 10:18:14 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nzFF0-000264-9E for xen-devel@lists.xenproject.org; Thu, 09 Jun 2022 10:18:14 +0000 Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04on0611.outbound.protection.outlook.com [2a01:111:f400:fe0d::611]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 77eaa28c-e7dd-11ec-b605-df0040e90b76; Thu, 09 Jun 2022 12:18:13 +0200 (CEST) Received: from VE1PR04MB6560.eurprd04.prod.outlook.com (2603:10a6:803:122::25) by DU2PR04MB8790.eurprd04.prod.outlook.com (2603:10a6:10:2e1::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.13; Thu, 9 Jun 2022 10:18:11 +0000 Received: from VE1PR04MB6560.eurprd04.prod.outlook.com ([fe80::dfa:a64a:432f:e26b]) by VE1PR04MB6560.eurprd04.prod.outlook.com ([fe80::dfa:a64a:432f:e26b%7]) with mapi id 15.20.5332.013; Thu, 9 Jun 2022 10:18:11 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 77eaa28c-e7dd-11ec-b605-df0040e90b76 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DaR4wWkNvCjzOLg9rWoHFv8x0ogd7WgRJk9ugUcN35Y8HKoGQsEIYNP4mjuPLPXYIBKhF2FEqT/h2r+LFRuocknzytZfla8rHy7vN4dR+s/clp53R+0mIfXJA5Sm4r2zPty6IG9jOhvtkcpKq55yaxVEN5+k+DWSz0XfeSmctU1NjTuGoBNTTHtyj5lfsVCULOGhZfbjl4bKAu1l2w+Glsybz2WHkgg4dzOEGcavuijf8YKpuV2Yl3P26fClIjSkyPCTzE+J9y7MZV16lu1qIqntzt8bRasakcCMjuxcRowP0Z5zFMpO/ENv9eh2YqfQkMX3I+QTkof/XbRQuNAn1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nYq1b9sHpYOMu/7plOnkihexpK35czv31oBFyEyWWuo=; b=TVyCmsdizZX/7J0a2+6kXQOhB8eLu4WuTK7bz/DYYsaEyWy2CCPRlVXzg7H7ndMhz/y+RJg+ycs4hJors+QQapL4rW1Vy84vD1/y2D5prxbdRYqWYn/Ox6AX/LLt1oR5lf9OLcrWSvQhQbew6S80vgnhBSRgaZaMnDNfYJ4UESsixU0nflJO7vMwOwJE6bMY+2B4cH+QZ4Tbng9eAfth93aqv2RYzGFIP2IgRzhzpPPXjhWU5MNVf531EYxMnkltllisbl7gyb8WXCnf5k7hUX8iN/ZOZCPGtYidWx/9Ds0vNOqxdpJ5nnCJaHuSBI1OpZiS3bT5pvRAFuwnWwwfdw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nYq1b9sHpYOMu/7plOnkihexpK35czv31oBFyEyWWuo=; b=bEtwslfUSOAUr720RO8IdKRPY7ukKrOKcfw3p3S/M1HG2Vf7vK0kemxLji0C9O/o+eLKYHp5Q3XLYq416rBiuFmZsvw7k14ygRoqtesOPBYwQVagixG1B6Y5+WHH67rQKOS0pxVvcgFpB/OJ5r1Pi24becsqJI0XmzCA4PLzNdIkqcbLzALx2F7/QDL9J0Ve73TEmXbSEQwIaYTEmjcdq3/bEqfgx0/ux06ZfQHCCTQUrd31CyZGXFoloW7aJ9YQNpgUtoVVMfCvkkGvy2Mw5O1KOaFk0A6zAsCmCEuEWqKba6L0ECyGEl6HkMVuL81JMvyc5DpqnbExCEotYckZ3A== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: Date: Thu, 9 Jun 2022 12:18:09 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 Subject: [PATCH v6 04/12] VT-d: allow use of superpage mappings Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: AS9PR04CA0131.eurprd04.prod.outlook.com (2603:10a6:20b:531::28) To VE1PR04MB6560.eurprd04.prod.outlook.com (2603:10a6:803:122::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ff67f553-29e7-400a-d708-08da4a015afb X-MS-TrafficTypeDiagnostic: DU2PR04MB8790:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rcDKphobPpyg2XD4eV+IHvRKo/IeKj9+MWjKXYjRjUI4zcrpPYjtcKQHJSUNgBi2KIa0xWB1OxPD2v34XQfEAb5YnqIMHCp6SNaNpGExOuUAXfEZYVyZEJ0xHKrpPQ0CMu5bj9ZiTPDOstyi6c1+VXY3ROvA8Tol34bMLFjPXVpLU2N9oXEwPq1kXPH+qMlNxpuL5SmZ8XZWdE7AcW2Y70LLnSP1WyPZsgGigXo8ZTfyUWcxq3kDpc7Ky5Q5z+eZD4GkpB8R1eFrPysp9/N2U1taJulTN+Y6/TganLXK6cOv0a4r2kSs/VTryu61K8P9fGhJAmzuFUCBfv1hWJLUjuiiW+I/RtVn1aZ0k/qjZLTZenNCClkx5iRV2t7iQqg/UX1OGZl80AqoiYq7pLBTwPpIdIVH0uyHc56WCBt0Zzsm3l6Drl+0f0BV/XWAIgywXDVTq9jotdhRC6RdKO3I5cCMnNZdwwVNZqrg1Z2U/k96cPVMKM0iY220M3nQFqiAj1nY2JoUcUEC8VPXMEAK4c+aiukzwpxsZPrZBx3LGfayBe0HLzzazZne4BLAUvUshXjLSbuvhBvGejwxKrtNQtqBgSZMDnF4GAGvhiax92sXMHf2Asg72FQBjOOZDXsaWN/0rnB7Bqcm9u6u0mAxZOraNcOKdqYe9Y6wuQMmq7iC6OVWv88P3sMV77c7776kAV5r1+flklwxP828MBZSu+wGHM+PGTJAHmU7HL2vebluL0VIPpmNMA3bfzTWTGmE X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR04MB6560.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(26005)(6512007)(31696002)(6486002)(6506007)(5660300002)(186003)(66946007)(83380400001)(2906002)(86362001)(508600001)(8936002)(36756003)(2616005)(38100700002)(66476007)(66556008)(8676002)(4326008)(6916009)(316002)(54906003)(31686004)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?HS08EmAFKSxeUqGswrkfhpa0x3um?= =?utf-8?q?gyKu8PRMKOvAyUKAgye/hOf3PDL/n3K0pcg0JvZaQCyvsPeYUNIF8VMlX54b5vHXI?= =?utf-8?q?qPcRiMRdGkUUjxZhvSdnyVzP8kS0zbXibPZ9Ic2BZc82p6LGHYhJVotcJqZ+mDdcO?= =?utf-8?q?4P/wwXygFQLU0zwdqNdel/C5Ze/GFFVUO9HXjRXxSpMM/Wn8PSGkQgD/nvJphqVAO?= =?utf-8?q?xQEDk+ErDyL0ZMrZh1Qg8HKJ0B08PuG6sPZx5h9i6JpDVYvqDeL1hjA1LPEvVtRvo?= =?utf-8?q?HsLDTwmBMgEByRqnTqlvxIoGel/PbkOmCmU+tO3ML1fTOv4Idviswl5AYnx1T6eA7?= =?utf-8?q?ywQVAIbuZEK6hlPo8sBcT2aebmVtwSv2UnocUa401Il8kAPWKrKoBJMFMHss2R7nS?= =?utf-8?q?k1fJhEXjfsjPRjzPqHdzA2/xNmucYzm1hYEztnydlrW1Lofc4ebp1GZnMpzpWkwhQ?= =?utf-8?q?A0uAZKO/ADtL28II3zop1D8TTo6qyL30qJG44jp+jvU7sBVLzkhYHx8TrmKODeFMW?= =?utf-8?q?hq2nln8iFwLAF8+HzxWvHiq5g19Znnz4CgTOEaql9snnIAJq4cNy5HIb9+pn+iD7J?= =?utf-8?q?IqYaYG3Sx3MDwaq3vy4IcZTs3GL2Wp4flVKCf7YODwjjFIy4U7Q3BmJ9MDC2LGix0?= =?utf-8?q?rUAxp+zCZ+p7ZU8zjD6L7ek6zq1bp0oEueAWeUPoEAlqMSjbnv4jpey0gJHwPJko/?= =?utf-8?q?v/EmSHPiW9phjo/yKxBjhbzuNQbfuBb6H3A0zwelLUzQgGTz3ibZtezukJujNWYCj?= =?utf-8?q?hpaZp/yzGvwJ21wVrPUSnKhM9XGozKSAIWl4avRvNtx+sAPFvL74GYaGIR2QA4H4J?= =?utf-8?q?7vUnlGhYIEN4c+byFUBnjRytMdYU6YASEimjrHvvNfc4xdkuxEervMhk/vEPGn+Jr?= =?utf-8?q?nBlFNeZTmaOBqCmSHuc36O7KPSOmCd2ze0coa5GdE0YU+1DIUzKVojSk08uYMeh4I?= =?utf-8?q?LDm7BCiphhINCEARYImiheVjoMWiEWuqDssr07zU4dRPKwEzty3YLrgixPG9dfC62?= =?utf-8?q?qEj/7FP/gfP3zEMx/hfhEArI90l8q73NTRrvNE6YaeIZGcu8UsfewyLsOzaHtDeP3?= =?utf-8?q?Kl7HvqNYoekUAVRkMnuwP/ncsbj+lLxeyMyZdUH0xWpJA1CkFEs6CBN4EexMQq/46?= =?utf-8?q?XnQyPvyGCXq9LQjxzX6GhJxkQ5va/nhJmHIa3OgRqbkAiXPnhCK+fwPj5kQol7xLx?= =?utf-8?q?gk9PcJg9SUogSWqqhEOCfdsc1nRXg5leOF2WhLzXYYN3CPI63YDfqy6pGHv2P4LXk?= =?utf-8?q?K0REtO4pBJAKZkbVyzNYOh0CAaKE5c7XPIV6kBVooJVDuEZu13kY4HDjEd6kMJHY0?= =?utf-8?q?DVK0RuUFT2MKbNZvDScx31JfPBeCl6pwtNkONKaB16M38F3wblSfytM4amPMUXpwc?= =?utf-8?q?RW3imDXs8bG7UD1BocTGlxCV2BHQUtpqdLK2nLXst3UlkweFqKEZVGy9sRJLn8BbX?= =?utf-8?q?XHXsC7hIU6Wmc6VTfl0kgEeowg9oJZbEw3wLZ/TeZKjxXU1k0QdjkSa1ydjAF8qLH?= =?utf-8?q?My7TaHYlW+ID1pBpK8ef+CepDpkRlHmM6FU2UqCu+PNLVztRkaWzQduzLaKoTGH0a?= =?utf-8?q?W7I+8sTZL97HZjEXHX+HWRoPJHAWWPsp2+9jOTgRBcX0IVeMYLehSiUPhxt9TW4l3?= =?utf-8?q?wh7CCWBlpco7xgvXJZjzyjrqo6Afi7Vw=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: ff67f553-29e7-400a-d708-08da4a015afb X-MS-Exchange-CrossTenant-AuthSource: VE1PR04MB6560.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2022 10:18:10.9629 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: sUs2cUBD3ZPyLCVKWVJVXpTHP5n+ceV5RpWJk2WWBtvofpRjs1k4TN0t6nrYvjnT08S4LWf4TOB0Xo4QSqCS3g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR04MB8790 ... depending on feature availability (and absence of quirks). Also make the page table dumping function aware of superpages. Signed-off-by: Jan Beulich Reviewed-by: Kevin Tian Reviewed-by: Roger Pau Monné --- v6: Re-base over addition of "iommu=no-superpages" command line option. v5: In intel_iommu_{,un}map_page() assert page order is supported. v4: Change type of queue_free_pt()'s 1st parameter. Re-base. v3: Rename queue_free_pt()'s last parameter. Replace "level > 1" checks where possible. Tighten assertion. --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -779,18 +779,37 @@ static int __must_check cf_check iommu_f return ret; } +static void queue_free_pt(struct domain_iommu *hd, mfn_t mfn, unsigned int level) +{ + if ( level > 1 ) + { + struct dma_pte *pt = map_domain_page(mfn); + unsigned int i; + + for ( i = 0; i < PTE_NUM; ++i ) + if ( dma_pte_present(pt[i]) && !dma_pte_superpage(pt[i]) ) + queue_free_pt(hd, maddr_to_mfn(dma_pte_addr(pt[i])), + level - 1); + + unmap_domain_page(pt); + } + + iommu_queue_free_pgtable(hd, mfn_to_page(mfn)); +} + /* clear one page's page table */ static int dma_pte_clear_one(struct domain *domain, daddr_t addr, unsigned int order, unsigned int *flush_flags) { struct domain_iommu *hd = dom_iommu(domain); - struct dma_pte *page = NULL, *pte = NULL; + struct dma_pte *page = NULL, *pte = NULL, old; u64 pg_maddr; + unsigned int level = (order / LEVEL_STRIDE) + 1; spin_lock(&hd->arch.mapping_lock); - /* get last level pte */ - pg_maddr = addr_to_dma_page_maddr(domain, addr, 1, flush_flags, false); + /* get target level pte */ + pg_maddr = addr_to_dma_page_maddr(domain, addr, level, flush_flags, false); if ( pg_maddr < PAGE_SIZE ) { spin_unlock(&hd->arch.mapping_lock); @@ -798,7 +817,7 @@ static int dma_pte_clear_one(struct doma } page = (struct dma_pte *)map_vtd_domain_page(pg_maddr); - pte = page + address_level_offset(addr, 1); + pte = &page[address_level_offset(addr, level)]; if ( !dma_pte_present(*pte) ) { @@ -807,14 +826,20 @@ static int dma_pte_clear_one(struct doma return 0; } + old = *pte; dma_clear_pte(*pte); - *flush_flags |= IOMMU_FLUSHF_modified; spin_unlock(&hd->arch.mapping_lock); iommu_sync_cache(pte, sizeof(struct dma_pte)); unmap_vtd_domain_page(page); + *flush_flags |= IOMMU_FLUSHF_modified; + + if ( order && !dma_pte_superpage(old) ) + queue_free_pt(hd, maddr_to_mfn(dma_pte_addr(old)), + order / LEVEL_STRIDE); + return 0; } @@ -2092,8 +2117,12 @@ static int __must_check cf_check intel_i struct domain_iommu *hd = dom_iommu(d); struct dma_pte *page, *pte, old, new = {}; u64 pg_maddr; + unsigned int level = (IOMMUF_order(flags) / LEVEL_STRIDE) + 1; int rc = 0; + ASSERT((hd->platform_ops->page_sizes >> IOMMUF_order(flags)) & + PAGE_SIZE_4K); + /* Do nothing if VT-d shares EPT page table */ if ( iommu_use_hap_pt(d) ) return 0; @@ -2116,7 +2145,7 @@ static int __must_check cf_check intel_i return 0; } - pg_maddr = addr_to_dma_page_maddr(d, dfn_to_daddr(dfn), 1, flush_flags, + pg_maddr = addr_to_dma_page_maddr(d, dfn_to_daddr(dfn), level, flush_flags, true); if ( pg_maddr < PAGE_SIZE ) { @@ -2125,13 +2154,15 @@ static int __must_check cf_check intel_i } page = (struct dma_pte *)map_vtd_domain_page(pg_maddr); - pte = &page[dfn_x(dfn) & LEVEL_MASK]; + pte = &page[address_level_offset(dfn_to_daddr(dfn), level)]; old = *pte; dma_set_pte_addr(new, mfn_to_maddr(mfn)); dma_set_pte_prot(new, ((flags & IOMMUF_readable) ? DMA_PTE_READ : 0) | ((flags & IOMMUF_writable) ? DMA_PTE_WRITE : 0)); + if ( IOMMUF_order(flags) ) + dma_set_pte_superpage(new); /* Set the SNP on leaf page table if Snoop Control available */ if ( iommu_snoop ) @@ -2152,14 +2183,26 @@ static int __must_check cf_check intel_i *flush_flags |= IOMMU_FLUSHF_added; if ( dma_pte_present(old) ) + { *flush_flags |= IOMMU_FLUSHF_modified; + if ( IOMMUF_order(flags) && !dma_pte_superpage(old) ) + queue_free_pt(hd, maddr_to_mfn(dma_pte_addr(old)), + IOMMUF_order(flags) / LEVEL_STRIDE); + } + return rc; } static int __must_check cf_check intel_iommu_unmap_page( struct domain *d, dfn_t dfn, unsigned int order, unsigned int *flush_flags) { + /* + * While really we could unmap at any granularity, for now we assume unmaps + * are issued by common code only at the same granularity as maps. + */ + ASSERT((dom_iommu(d)->platform_ops->page_sizes >> order) & PAGE_SIZE_4K); + /* Do nothing if VT-d shares EPT page table */ if ( iommu_use_hap_pt(d) ) return 0; @@ -2515,6 +2558,7 @@ static int __init cf_check vtd_setup(voi { struct acpi_drhd_unit *drhd; struct vtd_iommu *iommu; + unsigned int large_sizes = iommu_superpages ? PAGE_SIZE_2M | PAGE_SIZE_1G : 0; int ret; bool reg_inval_supported = true; @@ -2557,6 +2601,11 @@ static int __init cf_check vtd_setup(voi cap_sps_2mb(iommu->cap) ? ", 2MB" : "", cap_sps_1gb(iommu->cap) ? ", 1GB" : ""); + if ( !cap_sps_2mb(iommu->cap) ) + large_sizes &= ~PAGE_SIZE_2M; + if ( !cap_sps_1gb(iommu->cap) ) + large_sizes &= ~PAGE_SIZE_1G; + #ifndef iommu_snoop if ( iommu_snoop && !ecap_snp_ctl(iommu->ecap) ) iommu_snoop = false; @@ -2628,6 +2677,9 @@ static int __init cf_check vtd_setup(voi if ( ret ) goto error; + ASSERT(iommu_ops.page_sizes == PAGE_SIZE_4K); + iommu_ops.page_sizes |= large_sizes; + register_keyhandler('V', vtd_dump_iommu_info, "dump iommu info", 1); return 0; @@ -2960,7 +3012,7 @@ static void vtd_dump_page_table_level(pa continue; address = gpa + offset_level_address(i, level); - if ( next_level >= 1 ) + if ( next_level && !dma_pte_superpage(*pte) ) vtd_dump_page_table_level(dma_pte_addr(*pte), next_level, address, indent + 1); else