From patchwork Mon Apr 25 08:30:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 61261C433F5 for ; Mon, 25 Apr 2022 08:30:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312479.529727 (Exim 4.92) (envelope-from ) id 1niu7E-0004TX-4v; Mon, 25 Apr 2022 08:30:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312479.529727; Mon, 25 Apr 2022 08:30:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niu7E-0004TQ-1Z; Mon, 25 Apr 2022 08:30:40 +0000 Received: by outflank-mailman (input) for mailman id 312479; Mon, 25 Apr 2022 08:30:38 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niu7C-0004TG-3k for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:30:38 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.109.102]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id fb625731-c471-11ec-a405-831a346695d4; Mon, 25 Apr 2022 10:30:37 +0200 (CEST) Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03lp2172.outbound.protection.outlook.com [104.47.51.172]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-5-N9J2nGXnOXyZKBl-HY7EaA-1; Mon, 25 Apr 2022 10:30:34 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by HE1PR0402MB3355.eurprd04.prod.outlook.com (2603:10a6:7:81::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:30:32 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:30:32 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: fb625731-c471-11ec-a405-831a346695d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650875436; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=auT+iHb+14B5DSUBAnnO1f63YotPDBxUCIQdd1daVCQ=; b=BvnAvvdXorHlNXPqO0Gcc33yToHVkXVD9ADZ1+dAoErPIiy0kgh4qaF6Pvv4cKe1AkPxD5 s0LteGnF6Rz9XMXdJEo8Wn35D0Z0Kt9jOLgzBHvAD046FMrq9Px6Cd5Qiyi2pkFc1qEs+A fX3QpQfg75zJaFjnZ/J1sjPtou8GSUQ= X-MC-Unique: N9J2nGXnOXyZKBl-HY7EaA-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=W8ISYQKoqRhrdnvlD6ztHf03SR2TYQaYd+zN3O+7vR9MdyJem5BOf1Rs1E/TGlgk1lZP2GRjarvkPA6jkanIY0taYYaJ1/+Cscd74mHyFfmGagSa1RW0Ert//Sajd+RidRoshl326qo2gpBl4Fz+5+OjTfastat8aEniOCC/XcVbC7otQEl7KsFlKqyKuV45GpfT4d+BBeZgtpA//2pBAZu9RZXA3ZbslzamTdIrH0KSutMBjkbsyn0eqy4shhBW4Y0FLq24M/v26WGRcomsHzOCxenGvoH0MZQtYOL23MLYjhZL9RhsWB/mAJ5ueMCF9mkt8qP2XzYvJBsND1Me9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=auT+iHb+14B5DSUBAnnO1f63YotPDBxUCIQdd1daVCQ=; b=KwUE1AYIyP3gjap9B32YKMnqeHp4fny8SovKioxF06tJnxP+yJpcwcM9BTJgDVOhPEqWkKz8wp/93BwT7a04GiJF5oyME0JZWwnyEVUa/uNnW2A5tFLWK86Ls52E5bRInJRLtYgHJoxGxMRzv1ed9u5721knwVbbkAEZISYti1LQ3PqS7zmU9v9bEwu1+Vn0Ek5o9hsRfB/ZS27t3L9KSeYnJHmJ6Lsf3h8nSVCR8NBtCAOi/ad9mOQHPUpy01MtWjmtNbxKO/vrDdUI58D5azA7/Y33zOfOYtVO3AS+f2OLXw9FNM4sH98YyWuOCSki6j3jyIP54XSXw7fXfeaoIA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <0dec8285-a348-53d4-f9fa-552c7c1c405f@suse.com> Date: Mon, 25 Apr 2022 10:30:33 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 01/21] AMD/IOMMU: correct potentially-UB shifts Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: AS9PR06CA0252.eurprd06.prod.outlook.com (2603:10a6:20b:45f::27) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: dd287a59-cfb3-406b-7fdf-08da2695dd0f X-MS-TrafficTypeDiagnostic: HE1PR0402MB3355:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aOZWqanjAFeSC14fiTf50/yViFalVpIbBgHXbPsvM7nQzrhMUX/ZMAOuU6ZHQ6IeCYO7sULvQrind2+zUZbbGVuUuEYp8VsxI2vI2qdfMA9x8RsRoljtchGhA2wPct5DfnALbruhBctpkmRBXkSVgrkN8lfc+8MRSwwz8fm5Mi5VxyBfrScSL3LvhBFSBAaPfYPvM7Seuc1JS6L6P0gu3UEF5wV3h9UIz2bNX6v9HO3P2WI0mc3knJylHDNePRKubuA4IRN65UW0HlDb8a+gv9WJboplD0BfF8Q7Su8LRcbAy9L0I0Fh6O9kpYNC5bTHu1FKylsaV5TmelCE0L5umW4HUQUfzfeG89Km9ZtiDGx2zUmsgAOFLDZHRJchpn7PeY1OKSJt5W5K3nV3xkXiPK1JXM1wSTUCtMh79lF2O9Eb2GemuK1HEJ2ftmpUCnnuAg35MVUec6mw04P/Xgary5D9PFts9aYadU6t5wa1cFNQCERTFL70/hXApU3P7RpT9xJVe5QyaPBceY1nShmi93w2h2var0j2bOYbNuZ/u8RcL9g9eKhpgR6iEzmSaLsYiJhrGDD/Stmx09Pxz634W16i+c4nKCXiLL3f7TfFtCMBbzc7FgOlO6YlxPVUQ9kUKC3HQOPxw/iFDacZalGs+3Ets9zJZoxB8QbmxFKwcQGzd3PQd4g2GdYtxyL0NzkzrkBJXr1hPXbrEh61uAbiXBYvW7Xepzwh0obJ+EOG0UXIc1PpI3GZbuo3u2sjjlx9 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(83380400001)(86362001)(6512007)(6506007)(31686004)(26005)(2616005)(186003)(31696002)(8936002)(4326008)(66476007)(66556008)(8676002)(38100700002)(5660300002)(2906002)(66946007)(36756003)(6486002)(508600001)(316002)(6916009)(54906003)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?wgYizlBDp9HaUUHRL3MxymWzE++U?= =?utf-8?q?fy/hC5E9biznoRC5EV6aZgLQtHLNGnTZUtFq3YVZG7ge4VVVPXdepFnRQLs6KeJx4?= =?utf-8?q?nOQWByuXQ3b98rWMXlSF/UJviscjywURX2mmPL6oA5Tn9RFzz5u89hhDOoruRGlvZ?= =?utf-8?q?8WwV2zSQAxJF6HRjJgZD/elKi2/LprSJGdrWTOJCyV+FXwcOK5iKxdpVx53QbUpYa?= =?utf-8?q?C2l6FGvuiSeVu1zBrjcpK4FNxXkx5lV5onfwupfdA1x6SmujTkD87tV+NREatDGIM?= =?utf-8?q?pOO/hGcnIcPT2rtuOK5/yxRJ28JXvrajDd0qXwUxWl6WpmDv8LO5p+2pBX/BUas/E?= =?utf-8?q?EDSqMrDVEa7GWm8PbE7zhmi3kJ8j2i6wrbbJzDLY4JHHVePnz5bU4tIP54FH6Fgao?= =?utf-8?q?o1ED6ZWLC7pz2uqR0M+5wusLhqZDrrzsQSqzqoEUSdwQoNhq4dYEa3gtydFGHAW2j?= =?utf-8?q?rJyQHIAvXJFb65Mnz0rkRNqeSaV1UXWALXJmFVNSniC4/y2S4bqwDse38wUJETlGz?= =?utf-8?q?Wr6/+vcXPH31+MrX/8MzohofFGcvWOtIQNSiLGPIhYFezGIUYjDEb3St3r8ZAfFBF?= =?utf-8?q?MdKs8Lf8zCYmXKfGQW678VeqGYtm+x4Jzh8P4SnM3i+1blFRBiEYomwXMEy/n0YOl?= =?utf-8?q?pf2wvFY0wiBwE6TA3M/HeAQ40nYfldVLqC09KzD9gkqY1zu91pgCaSqogJa0+kLkK?= =?utf-8?q?qTAL2AToHLkwgfCebywoRnmAYfVitcV4RxULwHdmR5HAkxl0/s2rGpnO+nskXyY+O?= =?utf-8?q?S2VN9okvyu7fkYY9feKt0NyGzablgNR6ZaM8LKtRjIbtrSRL8RDYU6gjuf4IqP1ZB?= =?utf-8?q?Y1K9jKJTazDaVuk8mFSFGF7qBUGz1+Lnrfzr9xQ/rMjJMHYn+Br98uSmAcA8ICw1r?= =?utf-8?q?9UHfQogyQG8bxGg6hWglZ+7jzxOMspuuke1a4yEWFXoE/dggSHURc++XAdEQSunul?= =?utf-8?q?EGtyKKmb8zOh/XoLFAGnCZngJRFnrpUn1vjo2Rulf55Dn0BrupjgGTpYSidZA3zU0?= =?utf-8?q?vqf1Hl0wXGOcVtpGOETWp2MN1n7bvXP2gDuVLIVrN6lglUxd8rDcvwKmmBsixaaZd?= =?utf-8?q?/aLMqzNxil4CtVr2V3ELoS9wYfBdj7HVEGYBDW1CEQ5dKxb6P4JdfVyRAucNT+KPF?= =?utf-8?q?gZtnLSIp2HIUkXFEC6WT1ybDo4YtIzNkdO6tCr2oBthQF7HLoZJU03zrGYEXZcym2?= =?utf-8?q?us+O4MGokZyfh0CpWdgGqIBNquhlliB+EYn4xOlQKthrs9GGJUwXT/sRjH4Rh9iQ0?= =?utf-8?q?XGnk79Cq8+bVsLZ971JIGHTVT/Y8zvdKThOszdto++TF1hAWtBQouQ+9cX/WSYz8u?= =?utf-8?q?QR9jrYn/uzBX/Vu+5h8fgOZ3XyCT4/QV/0dZ9xLToajnwHryFa+iHYE7ce7ESf1Ze?= =?utf-8?q?b8h+L1JLfLIAHftW8AOXkLDrlLro+59jXyv8nKJyw8GyCLiY4316PQ1v10WpG2RLJ?= =?utf-8?q?3ZRaCKJBcCCHgSjFBLbko5kwrgRGsHbkulWi0ufAXpu8PShPs6h4eD3y/xMezAP9p?= =?utf-8?q?sQi6EgQMk6+IXnuuLvMLtAoT+w/cAm/9hGIFFQK8/3PiQM86SDZqDhCbitaya4lrx?= =?utf-8?q?5ffZX1a8CtB0z8zh5VKkvCAZkYcVFCIrgJjSKmE12+WLcWXsOSGns0SrTrGixlKeM?= =?utf-8?q?Z8cEmsQZxQ40qrAExO4tLkRyu43zzW0Q=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: dd287a59-cfb3-406b-7fdf-08da2695dd0f X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:30:32.8515 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VIn1ipyJvC2Y/8mYpU9Fr/M1q1bw54pS1X6NgB2IkGWSeJRjW1BJpDvWCjFo95jkY2dLtPF2LH2sBzmHYKT4Vw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0402MB3355 Recent changes (likely 5fafa6cf529a ["AMD/IOMMU: have callers specify the target level for page table walks"]) have made Coverity notice a shift count in iommu_pde_from_dfn() which might in theory grow too large. While this isn't a problem in practice, address the concern nevertheless to not leave dangling breakage in case very large superpages would be enabled at some point. Coverity ID: 1504264 While there also address a similar issue in set_iommu_ptes_present(). It's not clear to me why Coverity hasn't spotted that one. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- v4: New. --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -89,11 +89,11 @@ static unsigned int set_iommu_ptes_prese bool iw, bool ir) { union amd_iommu_pte *table, *pde; - unsigned int page_sz, flush_flags = 0; + unsigned long page_sz = 1UL << (PTE_PER_TABLE_SHIFT * (pde_level - 1)); + unsigned int flush_flags = 0; table = map_domain_page(_mfn(pt_mfn)); pde = &table[pfn_to_pde_idx(dfn, pde_level)]; - page_sz = 1U << (PTE_PER_TABLE_SHIFT * (pde_level - 1)); if ( (void *)(pde + nr_ptes) > (void *)table + PAGE_SIZE ) { @@ -281,7 +281,7 @@ static int iommu_pde_from_dfn(struct dom { unsigned long mfn, pfn; - pfn = dfn & ~((1 << (PTE_PER_TABLE_SHIFT * next_level)) - 1); + pfn = dfn & ~((1UL << (PTE_PER_TABLE_SHIFT * next_level)) - 1); mfn = next_table_mfn; /* allocate lower level page table */ From patchwork Mon Apr 25 08:32:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825384 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C06AC433EF for ; Mon, 25 Apr 2022 08:32:28 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312491.529742 (Exim 4.92) (envelope-from ) id 1niu8l-0005Kd-If; Mon, 25 Apr 2022 08:32:15 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312491.529742; Mon, 25 Apr 2022 08:32:15 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niu8l-0005KW-Eb; Mon, 25 Apr 2022 08:32:15 +0000 Received: by outflank-mailman (input) for mailman id 312491; Mon, 25 Apr 2022 08:32:13 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niu8j-0005KM-OR for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:32:13 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.109.102]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 34642581-c472-11ec-8fc2-03012f2f19d4; Mon, 25 Apr 2022 10:32:12 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2057.outbound.protection.outlook.com [104.47.12.57]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-25-oUMUKYlAMVa2Yg-nFT7akw-1; Mon, 25 Apr 2022 10:32:11 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by HE1PR0402MB2923.eurprd04.prod.outlook.com (2603:10a6:3:da::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:32:09 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:32:09 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 34642581-c472-11ec-8fc2-03012f2f19d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650875532; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/iXrr5wbNhiSfeRN5CafS9CPFmFmiZ1NrYkK3k2eoBc=; b=OrXI8RrCgJYu0U36MH1G3nbd00GCoYsXRFNdWVCyKqlDLAWjZ2v/rYKAtPzu2GVqyJ3QYv ISeEhvBFztkrr/AhUWx16Ejg2Hyfoeadr+tjP3V8ajKtXsbgce/Q261GeqpYP0aiLx/MnK b7Nv+wq+BjkpArljAeWXp2ycfRRR6Rg= X-MC-Unique: oUMUKYlAMVa2Yg-nFT7akw-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JHI0KfcCU2uigwWDWFkCx6UQb18qfCQvO4Xtbxp02SKFpdwTu8bwWyGGuWZqshO0KX31eFKZ2n4ZifoUlYNNTTlY65ffS4ZS2QSK49+3R/IRuX1I12hgNnlr4O5jcC5gtglVAqdb/ZhxNJO5n+TDxsc3FWu9lAFhIvQae2tsWgkGgrU5jx40LMxAnK4wifPHdU/GICO6wX6NB7hTYDeZPHguap0WLuKxDl1a56GFcwZGIlPkN4cnmED3QllQuQVlor1tfsaRfvgu4XcRhGGEMO8/osZr8Y4PgSa9t4RRC+uM3IOfsnMkVhg6my/A+wAjVMnsAd9WqHG9OJi3wNyjHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rjdSQtedmMFCh8llXOYSf/g3U8VFnBdTE7dsIXoukzk=; b=WsFxqEPx2zlx2PsD33HJ1WnxZ2pguQ9u43ppwthKVBozbTx/1oioyitOrsQ1m76GmqlZvBpXSA9uAMtxVtEUTXLeaEIqr+xiFiMeJFeRbZvmeoFF39+hu4GH3vx52/h/QXwZ1CLcDWmWhen1+sTcCuS2DEdW5kHG4ifLnQP75WQ5nuC2dAf2OXKUZ4/4scDVnexkRFkl+UYNsnxhKmG5P+cKhvSzgwH9BcA0SygWVTqRkS3/7p+Xg7LCudqG/HkbCd2YrRiHKo5XfRutP/SP4sIEKxy9mXcUeOmgsLmTPpRCWYhwMJmRBenX88w32HKh4/65BKytgtP4qwkrPSEP6A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: Date: Mon, 25 Apr 2022 10:32:10 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 02/21] IOMMU: simplify unmap-on-error in iommu_map() Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" CC: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: AS9PR06CA0330.eurprd06.prod.outlook.com (2603:10a6:20b:45b::9) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bab5479d-ae76-4ef6-773d-08da269616bd X-MS-TrafficTypeDiagnostic: HE1PR0402MB2923:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4plGuojbvHRApdQtJBz5lbEa4KFSgYrAfWdhtKGjJzcLRz0T2jqpuT6+2YD+LiLN+70FjKx85i4q7NOy1P3/BjRQyw5zjA2E1GJXZtDLvSAgIYuXWIOr7GdVK3nUMe6grWwRaO/cpI4LkYVFFHiLNrGp9r49iTQEWnj6RaiE8KQ8dPZAopM+1fddAxazCC9IlNmdM6jvb+bdS2n7vCbJYT6z2gmPXBCcaBVfGS6/OF6kOwOEpRxicVOZrjfJ/r8j2rPRhxykfyVSujkHUnFpYG6GUAFVtccMyv6BeIpK6Fs6Cpnvt+daCzXb3h6Zaa54ZehWF15nml1RMUuAW5V0BJhEmBKmtjVdOniM8IGGVxqiyoyF3TUqUxBK5/LcftisJ69nDT7eMZ0QCqwmjo4zvcxOWeivt1CUGx3f3tvobed80iTGyJbEyabKYWlab0n1rT6Faf/myCHsw7YRyYsZmUrVIfxrnJI09zDtlWEgFAxi6Sw4Tu5b6YIFlp9Z06DRBosHA+OUcYmg7Ur5ZiUQRV/rYzFNq315+00+bLxlCvZ3/r2sTSoO6CMEdrE76GAQX5eUPnRVrwHhfC1f/kEsorxx5b6hOgY0fgfg+gLJv4FEzoF6+9w1w7P//ekXOdymEqLXdqa0d2DDMiqmImSSiAZCVaET5OTeYo19XgIzlMBJ9wMp1t5PYaxTeQwg79HR4/4BtMlDVH2ASgDM88Z5XQRpHgy8i0H/6d4tDk+F1srOAcqmiBYU8KtxxSEF37j2 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(66476007)(8676002)(316002)(86362001)(66946007)(5660300002)(4744005)(31696002)(66556008)(38100700002)(6666004)(2906002)(6506007)(2616005)(4326008)(8936002)(186003)(26005)(6512007)(54906003)(6916009)(508600001)(6486002)(36756003)(31686004)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: SRBWLxRBnBIwe8OiebPawFRHRf1xo+30UsiNPDLrhKJoYjwOHAm3mwCUFXmmF0JqRACpAXgzVquzlsoRkbQpeF/jiXmGhJgAEYjMVC0dQEUnRfun7LtrwPTRdtoImZtfHsHcif9hmYM8ay3nrMF4pJOyl9Gqjq+29+rSTISuaDPBf0Hj7I0nAmQjpRyLrSbT9yt3ZEiq169ci1CKwFe7wMYCSzb4wb22G14Q1iKxLH3AKmDvP03/r0niaPQMrzocaFaipMF3HKLVlmLjkr+JYFGctrHoAUaRJw3lAIIgEG/oAT0CClSh+Gd2Ow656lVsUeZqzThD/Amw2Le5T2Un2bZ/Dnx1mzzkIKN9i38LHBb1WZ5mItLhW1p3pefTSOfk1tKrbQHGfs/04+DXXpo4c+P6TEyZ5Z4iaxEMrlC/T4qiTpxTmlXNDu3IPOVPVPLYAy5KYN8Dby91/roKeBvxV89EI9mUn5dblwVqixnFvEmLLCygQG8cOVsRSNUVy5JyRqXr43DqvQJoM3WfyIvC65vGkFT2JWDfaqv4u45hvdYDriOHmULf4WlzHixQW88I30FQj3FP4wHowMNxMZaRovJXn3bC2x0sSoGUo1RIr3QywjGF5heal1EvFvZmKsquli7rcSqi+beWtNwCTnw+QH+uh/b0Fzy9LC62Lh96+LwFgfl+bB11IahLYPzUViphqfEIlwIs+vU8vhPCDRNcMIRs3gnjbGe5n24G9uYX+rU4IqAkl2A1lLQAGgJUESPYuQU5r4LBeThwUBQpB6mnbVwxRP6oVhLlJWKF+UtgytWo2Wx9ouiUPW6mPltuR88oEStn9KxEmRcRaIUQMoRqRWki8F1CvxSXXPgy4lDyk5bF9gsyTDcZByTz1zoWm8wK19Eo0FXCxiS8whw+vrxAmvGpgkZMZFCwsLOOBM+BtOSFWHc62SJYb+t4uNuYN4EBMcmef2kB1ozf3Vx3kB9B0fqBLtonmAa0Ri/OJNjqan1+vzKewg5hJsJ7RKqpSMgYRQdhEOM/YjaHZIqrp/uGaFCbs97zOcOKOrKAszeiZCX45ndGjiPPOCH2IElHwnKxVTrPrjjjV6vss7j4nhPhlKseCd6JDOcuRbiTAQiDNLbXBA6LSLU9QDQLVdazo8N6dm57BnzwX8ICK9R97fMVJa5aGU++C81C2k6J82yJ5Kv363x7U5sjkrsesC9qlwP/I5Tl4SburM6Xgs4x5KhzUOpvA/4VBXAiQ4U9KjpmRM0KKdAEboU+hZ2p12BIrVJzKxJ5dXFKzTaTqztBgXCRxCF1F/2aa8vyX6PwUoAroDFlWWHF/8hVEmhDeVdfZdZPPXlDtPekMzLbjECgog7pmuNvvoJdv7QDjkp3QF9kuMNQH51NmAqJ+wNuwuLzdtnO3YsdiDsP7LEbaQoP3C/sQa4cGz9759TqT6WuDNdUfuhfLZJmvHcGaeQIUSSp7deGvSBboKJFoYis2FYyh5QJOco2Ntdh2TRiLXlK0itheC4IUS8NqoQPMoNYX8mZZWlRsGx7bYt9p/TWGADg8GeAKBZCWpPMmlwHtcXAclFrRKtwOfY2c0a6WFxfW5NrHd9pJJ06Ln/7JFJQGdz/q0CB2y5/LplOylO8emOIm6hifYHIUN9LXt2KvR1yTk53OIXDLaBrMZYo9UKkmO/YS0bD50NrzIUGx++HAWNGFo3ELd9dnw9mNCKEBgij3363b37EUkrTt5YeWqwfJYLHxBQrKg== X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: bab5479d-ae76-4ef6-773d-08da269616bd X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:32:09.5905 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: td2wkBK9YkcAHiFj5mmvYuijr+lhcC4Pjzl44ZVZqzBUX58CA8SXvlhRVCvp1HNuwG7nnafQr988hMtZ6HouEQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0402MB2923 As of 68a8aa5d7264 ("iommu: make map and unmap take a page count, similar to flush") there's no need anymore to have a loop here. Suggested-by: Roger Pau Monné Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- v3: New. --- a/xen/drivers/passthrough/iommu.c +++ b/xen/drivers/passthrough/iommu.c @@ -308,11 +308,9 @@ int iommu_map(struct domain *d, dfn_t df d->domain_id, dfn_x(dfn_add(dfn, i)), mfn_x(mfn_add(mfn, i)), rc); - while ( i-- ) - /* if statement to satisfy __must_check */ - if ( iommu_call(hd->platform_ops, unmap_page, d, dfn_add(dfn, i), - flush_flags) ) - continue; + /* while statement to satisfy __must_check */ + while ( iommu_unmap(d, dfn, i, flush_flags) ) + break; if ( !is_hardware_domain(d) ) domain_crash(d); From patchwork Mon Apr 25 08:32:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4A5E7C433F5 for ; Mon, 25 Apr 2022 08:32:59 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312497.529753 (Exim 4.92) (envelope-from ) id 1niu9G-0005qH-T3; Mon, 25 Apr 2022 08:32:46 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312497.529753; Mon, 25 Apr 2022 08:32:46 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niu9G-0005qA-O8; Mon, 25 Apr 2022 08:32:46 +0000 Received: by outflank-mailman (input) for mailman id 312497; Mon, 25 Apr 2022 08:32:45 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niu9F-0005mk-4G for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:32:45 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 471218fe-c472-11ec-a405-831a346695d4; Mon, 25 Apr 2022 10:32:44 +0200 (CEST) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-ve1eur03lp2057.outbound.protection.outlook.com [104.47.9.57]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-14-1YS7kSfkOHKz2PAbquvqIg-1; Mon, 25 Apr 2022 10:32:42 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by AS8PR04MB9078.eurprd04.prod.outlook.com (2603:10a6:20b:445::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.20; Mon, 25 Apr 2022 08:32:41 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:32:41 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 471218fe-c472-11ec-a405-831a346695d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650875563; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Rx5RuDNsd53Ne+GEPe7btLL0NeSg+006AajLP6dHqSA=; b=UxFxlCMDfNGrnkimfoqRWi+DNLoTPwwWidfu2BaMhH718xxSfaShG+mRfjsMVsMxefdFe/ NNjbexzSb3EC5qfDsMPjpG8Mgb3i9yPF6RKSM7jhWH5mHIpxVn3yv7GWsxNve5YgNJWx1k O7POTFxPeAiRu5ZRs3uCkwQ2mMJA64w= X-MC-Unique: 1YS7kSfkOHKz2PAbquvqIg-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KBhbBW5x/s0aHTvFEKUtdvBdu0GqPt4dowlvnPJoFi/DiBpsYdme8jepY1Vtr0DCNuJvFR9MCgv3Wzq9F4N0bu/GTUMYx5uqAzWRIKsEwgD2yCPI6sC3dOrrwnAkCQ4+KarQfYMoI8upW8GRn3mPGkaQXvRVU1ks2wC3S/LDWRcKhJeNt5+mSBhoftlPmohH+rfQfOuLbH25JQVoUxCd5g+1AW3iBJduhmadZwu6K/NwQWBu05HTbTljnJ6OsKiCAY5l6UpPaWDl5vCiS2sDAaqteZ7+7L1CEvc4/XiyyFxgD1LNISYAY1uauLTVMYfuoPZvvUFRxw/hKI4Solhefg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9K+UqwmOPdb/UDNInYNhrwscOOzna6UxgCO3tesJ9P4=; b=S7xuwr7Xw9siPT1TtBYMAVWfX0l8Ggkz8QXwujgss4CjYsXOHIGyzmvvGfHKsZl5hHD9VUPa2nJ2zJj0jjc/EdcjRtFj0k1//Yy9wZtiSqBSs1I2SGXRl6e9WciinGq3NVFGYZxIzXd+1ejyL7Cn1GFlwCwaFcuBElI8PdjNV8gPv30CcQdPgFQeZY+TFPGQbIDggCxWLh5oAmVanp2YUNrO+Kxmj+U8GCCxIp90ybzSFZnhbf3+izewSX65czU9qzOuO4ixIfjOjDke5f0y3nmlhsKw5AJ4/2vTQFl772yFhNK9F3RHaTJKM064xT4iHTn8lA4voQAmoAA27JpBBg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: Date: Mon, 25 Apr 2022 10:32:42 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 03/21] IOMMU: add order parameter to ->{,un}map_page() hooks Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" CC: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: AS9PR06CA0308.eurprd06.prod.outlook.com (2603:10a6:20b:45b::28) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cab66417-afa1-4b28-7244-08da269629a5 X-MS-TrafficTypeDiagnostic: AS8PR04MB9078:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zR1KTmlmqVL1SSAro1SskXtFZHHmGz39JjcBdywElWsGccJhpQFdS7ASLXyVzd9bO3oyEuVCQ44/gVtQpDJgNs6jZkfQbD83C39HVjFKSdgZU8xx6Z/J2rOkOmv8Xfi1T/mQIsOqh8Xb63jNqaS3DQuY/TSsLtn6Yk5XacUUY7cRv8eE0QVwOkEbcfIEyPE5RzwwufCJKVtCRsODuQNaBxgQsbB8XQLzuWxr4PF/cMzI8I58uKtyIf7pudr0V0wR9LbooQzIbfPr67MjVaXwRZeyUJgashVHJ/BfyU8AOwlR6rPucs/dP8yrVCuq6R4x3X7D9y4OcwyF8uVh9CE1B9SF/tY1Zcu2qTF2d4UTkqjDgIYfI4i0O9GhyAL069NlbFwH2JOns90Pist1+Ra/NdoHR4PpzwvZpotT/30GHhSJhI8oU/Ge/VEohSuXz1603iVw1A8ULKsMRNsphsoDsgsdxcvhZaB+5ExQN2uNHYFA4EGVmtzevECytjwHloPLxKokxL+ndvtz6vzILp0QltRmzPEsO9eXsMW/Ug1emBWP6gtEpUICANoOt504hNEZ8yvgZP6/lfpxyA6glmGojz7OjmAIfrzLdnD4antEsJvc82XzVG+36P5YGXFU11x85+4DcggWtmmxHrQUALn6+/Np0wp69fdaO1uHMBo+rwEIeKWTdQ3eS7tEk4iuXYfElmA99SJjrhVLOpzsMFkWT2fSvfoe0U+xulkovb9rle0= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(6486002)(66946007)(26005)(36756003)(66476007)(8676002)(66556008)(6512007)(5660300002)(6506007)(31696002)(86362001)(508600001)(38100700002)(2906002)(8936002)(4326008)(83380400001)(54906003)(6916009)(316002)(31686004)(186003)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: QmTStnSxsT9vnjHfZiiZVvjc5lJopa7Rhh/CRwQFwKhyInRfSOhwydQvSF8GiaFjfL542gUYJr4wCB/Kz92IYPunRLu311Lu1loKdHOZy8WzIvzIRqCgkkCc2nZcgi6h7s6YXA6W8FXXU6vygunfuCmBZmuVhmJpIXjyMNgU63TKHNIhhbrIAik6Tx74hbHYUwgPuRlAa2b3mZ5GewaQ+Qi+xNJc4WvRanS83JsGesafItSp5cD/8OiG5ixMcGBq0z6lOhDGOmMsgsm7vlKq2qF+LKH3gCyEbqx3M6gGCOx3/6u9XwLBeZ6koauMpVhM4cTJTgdmcShdTk+tusMCMKlngjKfM/WPM/4G9m9Vj2NrYNB0xrcWAzWFBiqxJeR4wJ0YIIaDBYaRLDamXOxh0OxmSb1EcWpupZhEnlAGLmbz+kCTe7E1MRmjozyE/ii6DIKtAPXKD5cgLroFnMmEHSB+BrlHcqWgIHi4fAEgdNc82sksbNGl+qRrcliKUlP8xEAjJIsaKhsGRekbegp/mSTqEsk0kmxXEW7cp7j/Vae3+xUiQ/PAj6mtd4fUDDomMk7pUVV5imX8NW8I1iK7RUh8U08W+VxST9uvzCiFg66DS9s2MPGIocqbscqKLFdxqMfIyGzT8Z1/6lwaGDieSJ6vJRKiW2FBabmOG/G3zOMs5Lj8m0at16Ph3bV4yRcwzbGB2nOOUWMgLoVfHZaDfF202MewBaNcNjg72JmX6p5EXDVim8RjA1Iv0EA3/KK/PXeuecZ17PLwYCIfEQUfFYK7kmvQ1E+MK20MdRi41BSUMlNNOH97MmJldBIQcbLVNLEz6mTykxRRjHm/VcePh8Fgjxv0yLq90OAuZhE+w+6JjOTAzKnJXNLKWieL41pxW7DInSWZm7W2s34ONW035h55xYRBll4aKgEogse1f1jDXLogLw2xNR5uOqES76ptu8hBZiCFOnRZQ7w5agfaeeHKG67rQZH2/M5tYFdmnetJzoZirpxKedfzT5ZupfsHNZS3dHamXqnS6p1IsDi4BwPvIppFVu1cYzMMJWFkKZEFPlQ+Rsf6WL1y9+urcBDG+0+PMRSVSnOinH4awkXPEfbbADh5eRgS7t0ey6Ept9is7DkR+GkwDEjJbEPmBHJhSsMSF71CLqJycIOAexoBy745WBQ84EZ37ctx24+R9eo1MIbHk11QjHEyIxe3/fSplNA99MzLQ5Vsb68ks/qURJ8QKgo/vC7SN3qHqbibQFcIY9jC+T7GdItss4IIYaX13C2Pf45PgmZEc19hv600gpMaLpo0uET0NaLFTQ1APxSUBCL97u7GyxOVzYKE8Wbk9eSVC46lcIiQeK7B11exKB3UsQEzlI5y9Twx3DPIgfIhjPAxSI29UbFaZ63aCaLZ+7UO3fwJ5imITTv6tV0WSOI2dw64hgRlJ1YUyEEEfrs4/Af/wXTzvL95GrOgaywjhciSfmwmttvsEgMGGFi/gnMwEKOlv8xt+w+tIQtJaoJjE9cilhMS4k0rs9cDICMIN317s/AmNYLrIPN6h/yXSEBP3iLVo0OsX29N3TcwOfkI2u62es46F2XGZmGFmoUlBoEVZpseLXl9cQ6kiAufMucQRSE2cmLiAZO4YWD/iIIF+fK6yTk19Sj/WsbrRisz8Cp94nofPCG1nC+p9i9rTSSDGEmnkIgJgVzu6IBU1t+wiuiVy0E3N+oRLCWjnje5kBoliZq+ehJsKPrIJYxIcQ== X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: cab66417-afa1-4b28-7244-08da269629a5 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:32:41.3723 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 96WLL1VODuTBFo2bhFmwlGSlemoV4xSClqts33a34LYDVwCnNpMRxXZFg8lD05t7WwIaJyAZDZHoHaej1MesnA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR04MB9078 Or really, in the case of ->map_page(), accommodate it in the existing "flags" parameter. All call sites will pass 0 for now. Signed-off-by: Jan Beulich Reviewed-by: Kevin Tian Reviewed-by: Roger Pau Monné Acked-by: Julien Grall # Arm --- v4: Re-base. v3: Re-base over new earlier patch. v2: Re-base over change earlier in the series. --- a/xen/arch/arm/include/asm/iommu.h +++ b/xen/arch/arm/include/asm/iommu.h @@ -31,6 +31,7 @@ int __must_check arm_iommu_map_page(stru unsigned int flags, unsigned int *flush_flags); int __must_check arm_iommu_unmap_page(struct domain *d, dfn_t dfn, + unsigned int order, unsigned int *flush_flags); #endif /* __ARCH_ARM_IOMMU_H__ */ --- a/xen/drivers/passthrough/amd/iommu.h +++ b/xen/drivers/passthrough/amd/iommu.h @@ -245,7 +245,8 @@ int __must_check cf_check amd_iommu_map_ struct domain *d, dfn_t dfn, mfn_t mfn, unsigned int flags, unsigned int *flush_flags); int __must_check cf_check amd_iommu_unmap_page( - struct domain *d, dfn_t dfn, unsigned int *flush_flags); + struct domain *d, dfn_t dfn, unsigned int order, + unsigned int *flush_flags); int __must_check amd_iommu_alloc_root(struct domain *d); int amd_iommu_reserve_domain_unity_map(struct domain *domain, const struct ivrs_unity_map *map, --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -395,7 +395,7 @@ int cf_check amd_iommu_map_page( } int cf_check amd_iommu_unmap_page( - struct domain *d, dfn_t dfn, unsigned int *flush_flags) + struct domain *d, dfn_t dfn, unsigned int order, unsigned int *flush_flags) { unsigned long pt_mfn = 0; struct domain_iommu *hd = dom_iommu(d); --- a/xen/drivers/passthrough/arm/iommu_helpers.c +++ b/xen/drivers/passthrough/arm/iommu_helpers.c @@ -57,11 +57,13 @@ int __must_check arm_iommu_map_page(stru * The function guest_physmap_add_entry replaces the current mapping * if there is already one... */ - return guest_physmap_add_entry(d, _gfn(dfn_x(dfn)), _mfn(dfn_x(dfn)), 0, t); + return guest_physmap_add_entry(d, _gfn(dfn_x(dfn)), _mfn(dfn_x(dfn)), + IOMMUF_order(flags), t); } /* Should only be used if P2M Table is shared between the CPU and the IOMMU. */ int __must_check arm_iommu_unmap_page(struct domain *d, dfn_t dfn, + unsigned int order, unsigned int *flush_flags) { /* @@ -71,7 +73,8 @@ int __must_check arm_iommu_unmap_page(st if ( !is_domain_direct_mapped(d) ) return -EINVAL; - return guest_physmap_remove_page(d, _gfn(dfn_x(dfn)), _mfn(dfn_x(dfn)), 0); + return guest_physmap_remove_page(d, _gfn(dfn_x(dfn)), _mfn(dfn_x(dfn)), + order); } /* --- a/xen/drivers/passthrough/iommu.c +++ b/xen/drivers/passthrough/iommu.c @@ -294,6 +294,8 @@ int iommu_map(struct domain *d, dfn_t df if ( !is_iommu_enabled(d) ) return 0; + ASSERT(!IOMMUF_order(flags)); + for ( i = 0; i < page_count; i++ ) { rc = iommu_call(hd->platform_ops, map_page, d, dfn_add(dfn, i), @@ -354,7 +356,7 @@ int iommu_unmap(struct domain *d, dfn_t for ( i = 0; i < page_count; i++ ) { int err = iommu_call(hd->platform_ops, unmap_page, d, dfn_add(dfn, i), - flush_flags); + 0, flush_flags); if ( likely(!err) ) continue; --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -2163,7 +2163,7 @@ static int __must_check cf_check intel_i } static int __must_check cf_check intel_iommu_unmap_page( - struct domain *d, dfn_t dfn, unsigned int *flush_flags) + struct domain *d, dfn_t dfn, unsigned int order, unsigned int *flush_flags) { /* Do nothing if VT-d shares EPT page table */ if ( iommu_use_hap_pt(d) ) @@ -2173,7 +2173,7 @@ static int __must_check cf_check intel_i if ( iommu_hwdom_passthrough && is_hardware_domain(d) ) return 0; - return dma_pte_clear_one(d, dfn_to_daddr(dfn), 0, flush_flags); + return dma_pte_clear_one(d, dfn_to_daddr(dfn), order, flush_flags); } static int cf_check intel_iommu_lookup_page( --- a/xen/include/xen/iommu.h +++ b/xen/include/xen/iommu.h @@ -127,9 +127,10 @@ void arch_iommu_hwdom_init(struct domain * The following flags are passed to map operations and passed by lookup * operations. */ -#define _IOMMUF_readable 0 +#define IOMMUF_order(n) ((n) & 0x3f) +#define _IOMMUF_readable 6 #define IOMMUF_readable (1u<<_IOMMUF_readable) -#define _IOMMUF_writable 1 +#define _IOMMUF_writable 7 #define IOMMUF_writable (1u<<_IOMMUF_writable) /* @@ -255,6 +256,7 @@ struct iommu_ops { unsigned int flags, unsigned int *flush_flags); int __must_check (*unmap_page)(struct domain *d, dfn_t dfn, + unsigned int order, unsigned int *flush_flags); int __must_check (*lookup_page)(struct domain *d, dfn_t dfn, mfn_t *mfn, unsigned int *flags); From patchwork Mon Apr 25 08:33:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825390 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9248AC433EF for ; Mon, 25 Apr 2022 08:33:47 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312504.529764 (Exim 4.92) (envelope-from ) id 1niuA5-0006VR-8g; Mon, 25 Apr 2022 08:33:37 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312504.529764; Mon, 25 Apr 2022 08:33:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuA5-0006VK-4v; Mon, 25 Apr 2022 08:33:37 +0000 Received: by outflank-mailman (input) for mailman id 312504; Mon, 25 Apr 2022 08:33:35 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuA3-0005KM-J8 for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:33:35 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 6538076c-c472-11ec-8fc2-03012f2f19d4; Mon, 25 Apr 2022 10:33:34 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05lp2112.outbound.protection.outlook.com [104.47.17.112]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-20-uwry9y5OPUeZAKnkaGefdQ-1; Mon, 25 Apr 2022 10:33:33 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by PA4PR04MB9567.eurprd04.prod.outlook.com (2603:10a6:102:26d::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Mon, 25 Apr 2022 08:33:32 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:33:31 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 6538076c-c472-11ec-8fc2-03012f2f19d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650875614; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Z53/35ZdniZbOvRjYGCROZXOzui9qbcXDOdkOD0m+zk=; b=MBPp0o1HKTKEDby7Ewe7v6glUqF85R10qpPM+6kgugU5bg2sAEGjonAkJQ23HNOUFFrZST daTr53/TpwPMogHoMo+9qOR3nsYwyc/XZ+bn2YPMKA+W6QNkdgnVYDxG1PzyoVZbJtFgEG X3BjBL70DypzJjBU/h2Bc8qSp4RKJEk= X-MC-Unique: uwry9y5OPUeZAKnkaGefdQ-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PDQXtv204C/f9VPwvQC511Wup4uJLJK6ZrI5DSLO0lFc1xR4Lwl4g3V91MMtzhj59jgYbUROtqtgy2oCD24m79oPW+f3v6AMarRLxw/pMkbgG2pWAbcSc1PE1ids8PDSYY+mMQe4kPOmw8xQcvtujP4I5OCcQB78o7h1Nnf6E8pP2IFxpgcDuq3U/YEHoB0wkzWZAXgeqT3oEjEV0zpkgj8yVum4psd1hfozJRqNHYyakPT+KUa/TTzqxIQA63dNnPMDqgrkdoySGsR89BI5aqtWf+dy/tU0up/uPlRjUjZY0Jt6Iwc6753dGbbYIDw+1k/0gNIoJivCUKISqeLx8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Z53/35ZdniZbOvRjYGCROZXOzui9qbcXDOdkOD0m+zk=; b=MiAN1DnqOTNQw7Im0x6j2PAROhgQWsQdBY1QpE4pLFhWGwW0JzeFXnsFirBtFSLxEXNDTD55AUCtZ7lOyl26FqbScv/95kZWfXrZEbMBnicbW+VQ+vE7C8/187Y2ldBOPEIL87nMBwBnRGNGxiifH0i7yTkfqhjzGE6d6GnCq3KW5XmveB54NmmyRAI9kOddIuerUeECQN2qBTvBfWAKIDmC5gfCJeyKrDzodo8+1bzz9+p9rGi6g3Khcc7KPjCLludoSw1mMpy4npoaD6J/Eb9OJFXQVhoYLdQn9JWu3oXS8rzgi+VH5OzV/9sIvIuXUmyQj9/GPuG+3cvupNzAig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <227d0bd1-c448-6024-7b98-220271d9bf63@suse.com> Date: Mon, 25 Apr 2022 10:33:32 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 04/21] IOMMU: have iommu_{,un}map() split requests into largest possible chunks Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: AM6P194CA0040.EURP194.PROD.OUTLOOK.COM (2603:10a6:209:84::17) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cb8186a4-4d2b-4713-8fd9-08da269647c7 X-MS-TrafficTypeDiagnostic: PA4PR04MB9567:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YGQMhTH3ufYG3qJKRgFZ0mnwQxBkZ7EZQB9Y3QhlDYJ89pwbtb4dzpbMU9sgzuOI4vtDVizXbn//Y8rXi48infzq8wk9X5Zpv2OAcyEj2bsyVM/h9r01u4qBikE4Cs5dCacB8IsoN3urOLi8VJHV32PqLIjD5Kj6kVKvHR+rVCZ7kofDLPITxghibsirXafGNWWwWiCxtpycIZ4FEG/GGJ60el6s8myYoOvckW99sq4pL6tYbZh4Hsu0lwMjwG/nU5cDJLYDWEIQZCy37wrYyz57oCNB1UjOAsiOAOd+kPLghA+2mMe6qstYnzcY3Hl5ZLoHhA7PS1H0SC1gE+MQkMH8ys5ffFsqQsiaG21fSFxQNuxp7fPrVMl3KhDaym8SZ5nDZlv6biY+YiC9raImcaUaRy6Ux/83JwlFUhUWhIbuL8s5ISfXUW5l0lpZwiiXwB1ZWEq35hvlEKj480WuUHnYXVNFLQbAmgwBFTVnsxAG9QNyXMMwBVlwcCU/9JsEI0lpM6kKQw91YAPDuTuZLD7saRLX+ljm3uE8JU4FTM1+SE3P2mFlEsRls1HbqdJR1/zbVBWnzRmDN83GoeGyelX+TlpqWSiNwIZMl8hSSI/YTK/7SEyVfmzU0/SI/lRjvjLuJTWo8+ml8ebOEcI0AbTEiSHsHlSvFy8YKtGmh/TwKiXt4dLOqbvRkWIIwKFZfldZz3Pdxg99fp0UK1ZN/NthhcuRNpdp+WEtZfnLQZY= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(36756003)(31696002)(5660300002)(508600001)(31686004)(54906003)(6512007)(2616005)(66476007)(6916009)(316002)(186003)(2906002)(83380400001)(4326008)(8676002)(86362001)(38100700002)(6486002)(66556008)(26005)(66946007)(6506007)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?Z4ZL4weaO7gQ9Hsq2gn2jP1dbBmF?= =?utf-8?q?2QonIx4XbnU+74wA+aolPZZuOl8MdROC9n3GVUeI1bscMRKSXPzyAfwuzLBry+mzj?= =?utf-8?q?hFWHYbd8tewiEQ2FJI6o41yECGL5ai9AUhVaYS/TBcsfTm6lHjG/KQ+1jAW9sYM+a?= =?utf-8?q?F75VLNj5faPOCzBgmX9sbpY3e4x92TKWHMjK6zgYaU9Eb9MAAc7qeG/cawFRd/caP?= =?utf-8?q?g2I4pjK2muOoHrer1jOcT/wtSyliQpwz4Otrjl9j6oQ7n88r4ZEBpldg1/kBaNfP1?= =?utf-8?q?cLOzC6xfZ09F9KMxIA5PmwqkkkLSFlKM2xGHx5B0y8kKR9WlTewoqfyFLQSBEvjDq?= =?utf-8?q?PIsz3KMt46FS+5Zx/3u+zpv+EjX/+j+x7obZgFJotVVI/QPwUwOSQjbMo9Co+D90k?= =?utf-8?q?l1MSqJ1EYJS44tbsiWJhPfT3u3cMGLPgGPH8eVfSHenYPZ1BmnX83DYNWkViEX2Cn?= =?utf-8?q?EvEmVkx856YDZLVYiMP24QQx4avkPBPzSB5uzymP+YvX7IRqVnroNrY4ZBGQGhCPy?= =?utf-8?q?pVEGJfeDx7jp9rQ0YgjbzZG1ZLVrujWAZNvXp/KsCbnLkrteHFZ8nFAzdx5sim7C/?= =?utf-8?q?7RIJZXfE2jykkAtAnW4zXEMvJRWaq60RU1vuAPm0Vzgzjp2r6scc1vufrAYxXPxL5?= =?utf-8?q?pLUUWQQBq1i8exnU/m+8lvYBLtdoftCb3SG4zgqVZbUhZ9PE3GMqnSVyTslyBvie+?= =?utf-8?q?g9uxdrK5voIKAmhhjqFjAtdGl8Pgs8lYUO5q7nwNHVobgTSavlaXm5jlq1H+hb2Hl?= =?utf-8?q?UxtY+5ss+GJSzdhKhsFOIC6jij3pOVScBrjDP1u0MyIjwDvI6hOg/RKz950pYXH7J?= =?utf-8?q?Rk7CXgNJtmv12DQ8N1TMz4eos6UpqOOambLM843y2ITQPHlMgpmZ7a4uOBJKh81DT?= =?utf-8?q?KUx7X6o0U4scYhtk+3aC2S7ke3Sy23r2Kww4zWWRfUB6ubo9VtAywUHSofaGmNPAF?= =?utf-8?q?0wWpa492p1Rr3bYEmn2gJgnKyP4V7BpiRgxrT+TveuyOu3DSRxDztIFAF7diGKja/?= =?utf-8?q?SAs039sAsPe18CPiP9K9PztHDNr1q1dLVfDQJBGiJfv0EWWnRFPseMh6q+J6yx3Da?= =?utf-8?q?jYwdGHPckXCK/2u05K4xClKMtZfR9G/Ui2SR/jb+EC+VyGPoW4fIUCTyBAukv3Vcv?= =?utf-8?q?KuwO/CjFB2ZUNLQTFIR10UspuO2g2jJD8/eMyhoPEkgly0x6zRuVt/7zorA8RVOuw?= =?utf-8?q?Yj6T8Ybs7Z6Vk3Yy5AslJVOCP9zM6y/D1o1XMtNpKfcbcQqBUi98b6J2GJiQTW90X?= =?utf-8?q?CS5g7rsZRTfJq28PgyftgADBCHu83IMvLTnRi0SfhDawfuMNtNJBxid2DcH56gb4k?= =?utf-8?q?ngeheg9s3SXgGscJcAWWPo7xRLS7aa6trOgtptTlbnb0BYhCOrJ7hAmDnLZLHl9WK?= =?utf-8?q?rlhWGnpLdMPBCZ7DAnHuEfPZpdPRXKXfh9/UZnKeemuge/5NSQ96MBP9CaakHBPSK?= =?utf-8?q?bBT7H2CcTOs3p64nDtCeo6K91aiLOwYRZKLt6uHGEiZQ4KVOOT21rOXdAHj+sAivn?= =?utf-8?q?ILBMmPvzzPSmxfgfROdRWVdXa7hHaKRq36RjKcKSzpvUtQLQ5BQ9PXShWuTRTwSou?= =?utf-8?q?KR/8U/kPoUnUirBAcvGmBAiZ/OVOeBU6VGeXifsy1VJf3yBeUZ/hmJhgFtt/0fNVh?= =?utf-8?q?cgdHQatI9PygCfufnCdTTrx5fsy/RrsA=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: cb8186a4-4d2b-4713-8fd9-08da269647c7 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:33:31.8650 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: kMzdGho5gKzkzrISqdWMGx1hw/CHOUWgHGKUCs20jhSdc6melKevyZ+aRQbbcwXyJDdpfNBWScqO1tAquH+oPw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR04MB9567 Introduce a helper function to determine the largest possible mapping that allows covering a request (or the next part of it that is left to be processed). In order to not add yet more recurring dfn_add() / mfn_add() to the two callers of the new helper, also introduce local variables holding the values presently operated on. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- v3: Re-base over new earlier patch. --- a/xen/drivers/passthrough/iommu.c +++ b/xen/drivers/passthrough/iommu.c @@ -283,12 +283,38 @@ void iommu_domain_destroy(struct domain arch_iommu_domain_destroy(d); } -int iommu_map(struct domain *d, dfn_t dfn, mfn_t mfn, +static unsigned int mapping_order(const struct domain_iommu *hd, + dfn_t dfn, mfn_t mfn, unsigned long nr) +{ + unsigned long res = dfn_x(dfn) | mfn_x(mfn); + unsigned long sizes = hd->platform_ops->page_sizes; + unsigned int bit = find_first_set_bit(sizes), order = 0; + + ASSERT(bit == PAGE_SHIFT); + + while ( (sizes = (sizes >> bit) & ~1) ) + { + unsigned long mask; + + bit = find_first_set_bit(sizes); + mask = (1UL << bit) - 1; + if ( nr <= mask || (res & mask) ) + break; + order += bit; + nr >>= bit; + res >>= bit; + } + + return order; +} + +int iommu_map(struct domain *d, dfn_t dfn0, mfn_t mfn0, unsigned long page_count, unsigned int flags, unsigned int *flush_flags) { const struct domain_iommu *hd = dom_iommu(d); unsigned long i; + unsigned int order; int rc = 0; if ( !is_iommu_enabled(d) ) @@ -296,10 +322,15 @@ int iommu_map(struct domain *d, dfn_t df ASSERT(!IOMMUF_order(flags)); - for ( i = 0; i < page_count; i++ ) + for ( i = 0; i < page_count; i += 1UL << order ) { - rc = iommu_call(hd->platform_ops, map_page, d, dfn_add(dfn, i), - mfn_add(mfn, i), flags, flush_flags); + dfn_t dfn = dfn_add(dfn0, i); + mfn_t mfn = mfn_add(mfn0, i); + + order = mapping_order(hd, dfn, mfn, page_count - i); + + rc = iommu_call(hd->platform_ops, map_page, d, dfn, mfn, + flags | IOMMUF_order(order), flush_flags); if ( likely(!rc) ) continue; @@ -307,11 +338,10 @@ int iommu_map(struct domain *d, dfn_t df if ( !d->is_shutting_down && printk_ratelimit() ) printk(XENLOG_ERR "d%d: IOMMU mapping dfn %"PRI_dfn" to mfn %"PRI_mfn" failed: %d\n", - d->domain_id, dfn_x(dfn_add(dfn, i)), - mfn_x(mfn_add(mfn, i)), rc); + d->domain_id, dfn_x(dfn), mfn_x(mfn), rc); /* while statement to satisfy __must_check */ - while ( iommu_unmap(d, dfn, i, flush_flags) ) + while ( iommu_unmap(d, dfn0, i, flush_flags) ) break; if ( !is_hardware_domain(d) ) @@ -343,20 +373,25 @@ int iommu_legacy_map(struct domain *d, d return rc; } -int iommu_unmap(struct domain *d, dfn_t dfn, unsigned long page_count, +int iommu_unmap(struct domain *d, dfn_t dfn0, unsigned long page_count, unsigned int *flush_flags) { const struct domain_iommu *hd = dom_iommu(d); unsigned long i; + unsigned int order; int rc = 0; if ( !is_iommu_enabled(d) ) return 0; - for ( i = 0; i < page_count; i++ ) + for ( i = 0; i < page_count; i += 1UL << order ) { - int err = iommu_call(hd->platform_ops, unmap_page, d, dfn_add(dfn, i), - 0, flush_flags); + dfn_t dfn = dfn_add(dfn0, i); + int err; + + order = mapping_order(hd, dfn, _mfn(0), page_count - i); + err = iommu_call(hd->platform_ops, unmap_page, d, dfn, + order, flush_flags); if ( likely(!err) ) continue; @@ -364,7 +399,7 @@ int iommu_unmap(struct domain *d, dfn_t if ( !d->is_shutting_down && printk_ratelimit() ) printk(XENLOG_ERR "d%d: IOMMU unmapping dfn %"PRI_dfn" failed: %d\n", - d->domain_id, dfn_x(dfn_add(dfn, i)), err); + d->domain_id, dfn_x(dfn), err); if ( !rc ) rc = err; From patchwork Mon Apr 25 08:34:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825391 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E463DC433F5 for ; Mon, 25 Apr 2022 08:34:45 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312512.529775 (Exim 4.92) (envelope-from ) id 1niuAu-00076y-Hn; Mon, 25 Apr 2022 08:34:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312512.529775; Mon, 25 Apr 2022 08:34:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuAu-00076r-Ef; Mon, 25 Apr 2022 08:34:28 +0000 Received: by outflank-mailman (input) for mailman id 312512; Mon, 25 Apr 2022 08:34:26 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuAs-0005mk-D8 for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:34:26 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 8385b456-c472-11ec-a405-831a346695d4; Mon, 25 Apr 2022 10:34:25 +0200 (CEST) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05lp2106.outbound.protection.outlook.com [104.47.17.106]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-18-ToR_TdrHMMC6fmfo9YEqkA-1; Mon, 25 Apr 2022 10:34:23 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by PA4PR04MB9567.eurprd04.prod.outlook.com (2603:10a6:102:26d::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Mon, 25 Apr 2022 08:34:22 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:34:22 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8385b456-c472-11ec-a405-831a346695d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650875665; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vWBl6T/hSAOh0rV2Tz0Vc2i51OiQePKLr7zssMhmHEk=; b=iJiO3at5AyhaD0pkDvAEjgaCB5m8onj+KmNiCfFcLDWcZoMRrct/X6zKtMoJLivPRPtRu6 ap5MJEkS7H3C5i3SISR2GIBH6LsKtsEkHdlsk8B9UzLZVQijgI7kOqMVn4kwDFLsZ23oZK +1+I+Igtk3OAmSXyNwwYyJrYJndFtuc= X-MC-Unique: ToR_TdrHMMC6fmfo9YEqkA-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=V0iGxl19zlvHtS3x5PhrvJpKskoHfSjw4QFkmEUlnoe4XsXV1kd15gmM3wGd/HxYM5Q8Vy3yk0ubzTMBmdEdvRu1m4NypasjLdznmNxepBO4sGSLuPVq0v/LzibQPEls9j95zZEs0wWbBkVYP6Yt88ji60T16PnAkHWiNLSsPFzgZ8+uTLySvhCAz+RBQuTNzJJDD7Vycxn6J+9GTvSb7NvvrIO7oarSs1FaLiYoQR+EnjYBV6Dq+svZ7PChMQF/raZQmjjbiqu4LFyxzDQW4cuqVPOMPDAlEpnSjqGzLvdkPDPqegSXsbiiW+vnN/oMsPB8u64SV93QoVob2WpMxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vWBl6T/hSAOh0rV2Tz0Vc2i51OiQePKLr7zssMhmHEk=; b=k+Fk7Qn+W8Fsa4Er9wdyHjq93t096ejQv6HNToG/ihkdjbPAxKbJ3N8Wu6pKb19MyyNivpQ32CATspIUGN7MV7n56bBWTyGXdncVgTi1ekkPCAMmD61ynEsIe/1CDpliXNLg2WPYjhEmUrCp5z79BB5ll0dBYgSh8ecDu3cc3IvC73Dv1VZeg0ohCpBkZkuZajCqQ1liZ+gP604Nj6GqAjdjtnalNvC6cbnL4BSWfTo7g1Qx7MpClZK2lj2Dt38uuAKwBJAvYbZ9968XwjKTc7Ltx+G6AqZDVnK03WhiuD47pnX8dEeXtdFVeFNVUYiF6xNMz12Dm4NrVoB736yZXw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <5cb4dc1b-f6b0-89cc-e21c-a27a5daf0290@suse.com> Date: Mon, 25 Apr 2022 10:34:23 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 05/21] IOMMU/x86: restrict IO-APIC mappings for PV Dom0 Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: AS9PR0301CA0011.eurprd03.prod.outlook.com (2603:10a6:20b:468::32) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6f64066d-4dbd-41a9-cb13-08da269665f7 X-MS-TrafficTypeDiagnostic: PA4PR04MB9567:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oFkPV4GsiJRxgpE1BeGJBz8SYMYoymPfzbD69Z2LcS11JqaXwU1lENuhXfwNEIQU53YZu8RdqqL17vesN8IibrTjGHbBNEUn+0yHin/hhO8Ynp+pWC7zOyKOp9/bFzCONNiALZ8+f1xB/8q9XDtcIAzIS/yh76TUClHpBGsj6u0iZylbfvrxh3eeFswBabaU/hartMgJpA1b1bCDw4UxME1L1/XQC/JTD3PZJRCPSufqQ7spV11Q95XMkTaIkq5v3MMkdBNj77o0fInzY2SSRzRupaMK8WzcCgn3FyvogJTqssDGKpOG5Wc4ZmSyIZqK3fymW1hnawI19+0e6z52hNumEQqu+eTwtWTkjrme4MpoJGa5mhCHwvi6EG+oXwufdppHfg5VzEsrx9EsnxZ6gdm8QEx12Vu8QimQVnF6llOiEA/arDgwiQzE2U2YYPEJ+ZL0dzhCM1O5XPR+wyt2m3O8YkLGcthYbP1vrEniRfk951rr1g3vheg4rvKIrgKVWGmej3vPpqLi87VM58H8NoXTifg2P5cuo1WoD9ftxTvFvQBE07j1Hm9Lr2VHf4ZEhtQr9JkzCsXOMSWm9PC2KMweSA9pHz/68MjrGNgC0XfBHIJDDAttnqN7hfsBLdn4E/zcwJ1B8/9jhv6SISOPtmHUrkffQyhL60MF/AqHYCQEAE2tDGnYzPbcTErBJlyjNajsg/+ZdW/47gDrNEAWQu9aFT/sewfAm6LkvxC+PTA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(36756003)(31696002)(5660300002)(508600001)(31686004)(54906003)(6512007)(2616005)(66476007)(6916009)(316002)(186003)(2906002)(83380400001)(4326008)(8676002)(86362001)(38100700002)(6486002)(66556008)(26005)(66946007)(6506007)(8936002)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?CK/1eJNOlSqCrdxovQppPSdhdhf9?= =?utf-8?q?UzwDo2nQJe51YSZuJYH+vAXhGNcke8sPhryERs1vFyLZ81qvUqAnqPydJAcDuVC3L?= =?utf-8?q?+ZMFkcq1eoqImHfw2kYTbsaE3h4MKOBlSUcasGYAPL3OXz7OUqwLyPjhg8v1XNjC3?= =?utf-8?q?e+xOrwPgEtY428w3faKgswshy5QN/QfYtWsvSZC6QZdb6BygbWKwBu8onTOm/SZ91?= =?utf-8?q?8gJMVr9vIY9H9U/W/nMzgWcLTtJFm4TJI3P4PsKLh+O1m8UTL2pGqjGYgD1I34xQF?= =?utf-8?q?znuQEykCwR9wKMKE0kL8hSGbkzNmoF/JG391A5pcB7+YmM3fG6vX9WZiCYvM1REQf?= =?utf-8?q?Cc9iFVjuUAhzy+foBGuI0XfWf2xP4hkrAZscanyDqp/LJyeAEPSyCktFnHgWsCJh+?= =?utf-8?q?4jwAJ2gDmiSWkArifWVkDiVpnLHMiOJm6SaKss4LwY8PshcczQeNa9RkGgY7wO7Lt?= =?utf-8?q?JSFhWZzKkpK5kQS16V/m729hIoMcY98nenYwwgTqWGaX0HaE+IUYjM9xhVmnTBHDK?= =?utf-8?q?q3UWwTNZLtLLwf+pWbCMP1dzukNkj7q4iU/sVGK21QMq0ki7Fq+iSQ8WpftR+9hLq?= =?utf-8?q?Q85c7L9UMSx/zbX+G9+e8UtTaf/jJPHtOsmX0JLeoI/uWDXnhgF/E0Fnot8JD8lMw?= =?utf-8?q?A96I7ClkH4zYNkbrqxlAq6lcXjM/P6vGlUUo4yGKXip8mSJZDoSnCK0y1+FHmdr8F?= =?utf-8?q?MUgfX0QrSTGLo4W6yhpWJtoOqLjuEYehySwsH/UQPLToL8r1iOA2db/kKiOayc/Lv?= =?utf-8?q?aFv1BGVB4d26J6ghc0UUzcY5h34UJRj/xtlX4zD5XH6gx9vDoKLEPrjeWGzDK/0T1?= =?utf-8?q?LEvt0ZXiA+iKyPzcptKvuBG8CJ3AeAhfwUkJaqmppFLDc6p7w9VS5bKlQpYPPgl7H?= =?utf-8?q?NxNZbyGW2yaX7w4eHKOW9t5Spn4flw7gqgxgTp22Hnc7nMJiFfT8vFKc1d8uQH3Fg?= =?utf-8?q?RzYq5bB+txZUzdYsajUEmLS/hThdq077s720mPWVKqFMqHWF2sj7RDhg02zYLMlM9?= =?utf-8?q?EOHVX/VZ+RggJdJHo9TdCL/MApfslVB0S/INYHKgckh6FGRWKnNruWTkekuy75760?= =?utf-8?q?U3wd31ni6pBFimfa3uv89ghI0FcH8wjm0tcQStrI1g/wttKTeNhx8YHVc+tlXNSRX?= =?utf-8?q?Hg9ba7p0jGN47KqNXo/Ywl5x8dT3FJrgJ+dtOXtRhBAHAcAJJrvCI287zgzoLEVO6?= =?utf-8?q?2nca827Ai47PYh55VcWt7Z1V0mJS72CmxgrbPZAto3GQp+/6CNIXE9Iv0cwMylAAe?= =?utf-8?q?WkSY3qBWDNjNmNxoVLvEDitzIC1s6sAvfbJenc7bMPWF1Or3ptRDb27Wpp7DvqUTf?= =?utf-8?q?2SEweY+iWytguOyVY3/TQlZNtaOEGWo/VUM5Es2TyKJV8LeSDQ3PKdvNjOV+0YrJ9?= =?utf-8?q?HTQKwr8mvLrNhby+XhDHADCdpsb1ag2aec9KdIi0OQJUmCsC0ra1shghoNPPNM6Pi?= =?utf-8?q?rT/uNVqe5bObveslbxh9O/o2XXuVn/zkVNmlXwRmZXFH9JJVuBRzwxmgtHOsUnItd?= =?utf-8?q?4tc7hrM6MnxxGSjLhXd9z8tIGoXrhXVDWqFmI9GyqUT57kr8Mg6r+vTMSqSXrjIc1?= =?utf-8?q?c5hB1hsEKBN24Y7j9cwtDOxOM/MRUH0GSQDXKHsRn8C4kyPJfxxftw3Rl9Fb5A1hD?= =?utf-8?q?xqyOlYTgsj01dmOr68kNzylll+BXWkzw=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6f64066d-4dbd-41a9-cb13-08da269665f7 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:34:22.5596 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TciKBrwbBpsQ3N6GgoXP7HCYONyH21uWr1sC4JSnk0xhUhrH4Nw5vpj9X3loRaj71v4NcTbIYvPS/Ll4llKr8w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR04MB9567 While already the case for PVH, there's no reason to treat PV differently here, though of course the addresses get taken from another source in this case. Except that, to match CPU side mappings, by default we permit r/o ones. This then also means we now deal consistently with IO-APICs whose MMIO is or is not covered by E820 reserved regions. Signed-off-by: Jan Beulich --- [integrated] v1: Integrate into series. [standalone] v2: Keep IOMMU mappings in sync with CPU ones. --- a/xen/drivers/passthrough/x86/iommu.c +++ b/xen/drivers/passthrough/x86/iommu.c @@ -275,12 +275,12 @@ void iommu_identity_map_teardown(struct } } -static bool __hwdom_init hwdom_iommu_map(const struct domain *d, - unsigned long pfn, - unsigned long max_pfn) +static unsigned int __hwdom_init hwdom_iommu_map(const struct domain *d, + unsigned long pfn, + unsigned long max_pfn) { mfn_t mfn = _mfn(pfn); - unsigned int i, type; + unsigned int i, type, perms = IOMMUF_readable | IOMMUF_writable; /* * Set up 1:1 mapping for dom0. Default to include only conventional RAM @@ -289,44 +289,60 @@ static bool __hwdom_init hwdom_iommu_map * that fall in unusable ranges for PV Dom0. */ if ( (pfn > max_pfn && !mfn_valid(mfn)) || xen_in_range(pfn) ) - return false; + return 0; switch ( type = page_get_ram_type(mfn) ) { case RAM_TYPE_UNUSABLE: - return false; + return 0; case RAM_TYPE_CONVENTIONAL: if ( iommu_hwdom_strict ) - return false; + return 0; break; default: if ( type & RAM_TYPE_RESERVED ) { if ( !iommu_hwdom_inclusive && !iommu_hwdom_reserved ) - return false; + perms = 0; } - else if ( is_hvm_domain(d) || !iommu_hwdom_inclusive || pfn > max_pfn ) - return false; + else if ( is_hvm_domain(d) ) + return 0; + else if ( !iommu_hwdom_inclusive || pfn > max_pfn ) + perms = 0; } /* Check that it doesn't overlap with the Interrupt Address Range. */ if ( pfn >= 0xfee00 && pfn <= 0xfeeff ) - return false; + return 0; /* ... or the IO-APIC */ - for ( i = 0; has_vioapic(d) && i < d->arch.hvm.nr_vioapics; i++ ) - if ( pfn == PFN_DOWN(domain_vioapic(d, i)->base_address) ) - return false; + if ( has_vioapic(d) ) + { + for ( i = 0; i < d->arch.hvm.nr_vioapics; i++ ) + if ( pfn == PFN_DOWN(domain_vioapic(d, i)->base_address) ) + return 0; + } + else if ( is_pv_domain(d) ) + { + /* + * Be consistent with CPU mappings: Dom0 is permitted to establish r/o + * ones there, so it should also have such established for IOMMUs. + */ + for ( i = 0; i < nr_ioapics; i++ ) + if ( pfn == PFN_DOWN(mp_ioapics[i].mpc_apicaddr) ) + return rangeset_contains_singleton(mmio_ro_ranges, pfn) + ? IOMMUF_readable : 0; + } /* * ... or the PCIe MCFG regions. * TODO: runtime added MMCFG regions are not checked to make sure they * don't overlap with already mapped regions, thus preventing trapping. */ if ( has_vpci(d) && vpci_is_mmcfg_address(d, pfn_to_paddr(pfn)) ) - return false; + return 0; - return true; + return perms; } void __hwdom_init arch_iommu_hwdom_init(struct domain *d) @@ -368,15 +384,19 @@ void __hwdom_init arch_iommu_hwdom_init( for ( ; i < top; i++ ) { unsigned long pfn = pdx_to_pfn(i); + unsigned int perms = hwdom_iommu_map(d, pfn, max_pfn); int rc; - if ( !hwdom_iommu_map(d, pfn, max_pfn) ) + if ( !perms ) rc = 0; else if ( paging_mode_translate(d) ) - rc = p2m_add_identity_entry(d, pfn, p2m_access_rw, 0); + rc = p2m_add_identity_entry(d, pfn, + perms & IOMMUF_writable ? p2m_access_rw + : p2m_access_r, + 0); else rc = iommu_map(d, _dfn(pfn), _mfn(pfn), 1ul << PAGE_ORDER_4K, - IOMMUF_readable | IOMMUF_writable, &flush_flags); + perms, &flush_flags); if ( rc ) printk(XENLOG_WARNING "%pd: identity %smapping of %lx failed: %d\n", From patchwork Mon Apr 25 08:34:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825392 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6B109C433F5 for ; Mon, 25 Apr 2022 08:35:17 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312515.529786 (Exim 4.92) (envelope-from ) id 1niuBU-0007dp-SZ; Mon, 25 Apr 2022 08:35:04 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312515.529786; Mon, 25 Apr 2022 08:35:04 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuBU-0007dh-Ox; Mon, 25 Apr 2022 08:35:04 +0000 Received: by outflank-mailman (input) for mailman id 312515; Mon, 25 Apr 2022 08:35:03 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuBT-0007dM-6k for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:35:03 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 98f17de4-c472-11ec-8fc2-03012f2f19d4; Mon, 25 Apr 2022 10:35:01 +0200 (CEST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05lp2110.outbound.protection.outlook.com [104.47.18.110]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-14-TGjtCKQ_OQKiRzHTUvBcNw-1; Mon, 25 Apr 2022 10:35:00 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by AM9PR04MB8472.eurprd04.prod.outlook.com (2603:10a6:20b:417::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:34:58 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:34:58 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 98f17de4-c472-11ec-8fc2-03012f2f19d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650875701; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E1iCV10YdTuovGxw8Kec93Mw/xo3l/7mx9DOj2uTyh8=; b=J7zjSz9JwdWhrxjiuTlsMbfBpTRXnf19Nm7ir1jB3cpewXdL63uUo3LiTB+vGFY8xugr+U cjp9xF1xSKCvTrUSIob2FvWn93bnhb68qVdp9WxEcRtAkfxwAWl2rP0qyeVuL1fz9VQ4Ug DmWx1uXTpgz6ugHrR600ldp9M0bcaQs= X-MC-Unique: TGjtCKQ_OQKiRzHTUvBcNw-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=f3Sa5bfKpgTCEOpPcMYwB0jq3E04t72ZZsem0OFXfZei7itp8aMTXq5rXuMwQc+jEKq9WSLOLzjUR6roY1GI0w3ZxMwZaduDy9l/6jl3zyWzSGaGyYW4bU8JggF6zo7s0vKonKNQQDJiMdbg/mUXEdM9+kUIt5YRq3t4nb8978d1cTrSQ1eGlxK6U2vquSQMfVHkxUCekn0egr2daUbdR1mtUx3Gp5ze+y6Bz53jIS213pgRNPy3GPRc2l1ySoe+aMmbbKXWSb4lzGFRAqQhVl1dKDHQIbFFwDJkfAu5iCb5e4GjACjiVsT/B8awzgGXbA3ILGiunhMsk6hyBe5V0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=E1iCV10YdTuovGxw8Kec93Mw/xo3l/7mx9DOj2uTyh8=; b=UDxChBSa4v1D2I3/anjHuuemSccXKvAS00E+p73TmjN6Yj7DWIkDsT6zTE/SyvZM5YO0gyUT5u5LqC3RLk5zdzvm9tTf+/k96Aimwci5d0QJQqMddK9Jt9Ta8Np/51Qid/jgkOW0AifSeN9Mv0HWTb1H1pJcX81WIrb3Pj8SrRBWbXElrZh0fbtJ6R1itGrzjC52LIUK5w3X69uctAYFD8p9D73FHQi86sb5HQSUIGDxCYm1nRvanmTKCqbbEbjaAYDAfUM8R6A+SAFiw041S4OgnjjX4v7Py8tC2AkFp2xVJZjQq9LihGhzCgO8rIbfHbQ4O/g4HrFIGL8ps/YFTA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: Date: Mon, 25 Apr 2022 10:34:59 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 06/21] IOMMU/x86: perform PV Dom0 mappings in batches Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu References: In-Reply-To: X-ClientProxiedBy: AS8P251CA0028.EURP251.PROD.OUTLOOK.COM (2603:10a6:20b:2f2::24) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9bedd4c9-2a8f-46dd-9946-08da26967b77 X-MS-TrafficTypeDiagnostic: AM9PR04MB8472:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 74HVRLekGw92SypuC8yRZY8jqZ4WwKoKix1F8N/GccxYasZ6XN0oyMfbaW4dLYH8GNzqA9//ldCIVixNYeEZIiswQzyH4GtrFwYznUuphI+MQB0aEFly7RBdcDI+TV8K2xPe3Ao5ICmYKr7keq77xBxvqU4cyzW2EPsKGeJRc4VTCmk4V+nGTG0pifxsKSmxA8G4R7bJFrQeygD56OOoZkfRQoOwPv5OAZk39oDbogEKpirGcL8Zd/xcnOaYYbwjq7tUZEbk+WnOPpXZD3tBQmQPmszCL+tlx4HG5fjLDi/xFbhN1qZtUno2r0cVxMsQGOUiJeqnRWL/WIN5inArNDvxKX75DbQGw9gegOCztIJiYUuMrxncdpE8yRN7TH2yId6SjO8ULtsT49XCliGsVZgAmV7KIPF9nYR6nFAYF8KHr4OZNPyXOOUuvGGaxNjBqcCyo1gfo0YOApkvPQNKdu6lKDAjUcVlawQDhOBz0jDluAxjFE05/GdnAjTRgl2ktAINS03Ykw5eAHo9gOZPKXy3wS/Wm9WXnar1NQuqOWlyi/79OC8RYgS3B+Xd/LFNBbIMtDULwAmJxDZu2RA4ZQbxKJgFT2OZEIbE9R+qfuc3VRh8n9uH9Vb5qgGL+vXo31giNoGEa8ECAOYGLKeI9mtKE5Zo7eo0Yb+AKT72nxzH9W6fHwyscq3IssvJ5HtTRB8bO/2p8weZkcGYgfWrJ8QCre4kbX7j6meiiyZCfz4= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(86362001)(38100700002)(508600001)(31696002)(66476007)(6486002)(66556008)(26005)(36756003)(83380400001)(6506007)(31686004)(2616005)(6512007)(186003)(66946007)(2906002)(5660300002)(8936002)(8676002)(6916009)(30864003)(316002)(4326008)(54906003)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?Hma/Vyqw3y6HEs8jfASoS+tbJWxl?= =?utf-8?q?SHTJYA/bI0fdSqqPehR5nSYCv+Rg6Nu2Fzdrif38QmwFE5ipSWLqAoWqic2XkcMqF?= =?utf-8?q?NfeYkhQIYr9BVh3rxJK/3T3VPp6Np7tspMaJOVcq4O6WppWKupsgSwdRr2VrT7Tp/?= =?utf-8?q?ShLvyaa9pq5VYFRRaL7+eGdX4ZvH9brGhWujudqQgQhkDmbuga6vmkoOxBmF3HuJO?= =?utf-8?q?pvSTpJZiVDFBkItwl2vblI2fE5eABV/p5sr+n9hdrmZSQ+12mCNs/TWBA79WDDAtP?= =?utf-8?q?1y6upnuJrOL4bQ9n9Yn0Q0DOzzMt2d2Tl6oRViJuv32vCy375HSF6PZqF+TZx6Ch+?= =?utf-8?q?utgAL/MOgMALmrSWC6fcIF6tJI/rO3Plv8fMIrhLgxWkQEy/boBnw22uUoR0zg30S?= =?utf-8?q?Nsy1JJvAw6MWckos5XKActui/YxSugz0wQK3swkXDSQKC7u0p10cLZQ2jm4vnWHbx?= =?utf-8?q?p0cmdlsYrLoWJHFBiQRZgIclsZKqdnE617hjwrUf8FbKSo9OgPeaRJxRn896ZxeD+?= =?utf-8?q?GgSXBuYIiQx+1fvxfmhhYDXWuQsZxTkTuq5dzu/xnavL/fHwAwiRORwwwxK/9j1OY?= =?utf-8?q?FRlMMQXrB+h1ToUtEyqCyMEln6iuyuHWawbP16iZiu8Uj0sdrkO7ebkKpSqllwmfP?= =?utf-8?q?HJbQ37a6lpRoURQh3KN9gm6/Yqt4oQ2bBLHtMYtOVeFnRQn5i89k/1Gl9CMhQ0GWj?= =?utf-8?q?YTH5Bops7+twbPaNgEiXjeHMFS5XYMn1KfXbE44biUTewDagY/i5eU6Jy0inhGW8/?= =?utf-8?q?OwBIWJzpc5toJoUXmDbcGY8tX8BuNTlYnGu1h5hDbX6s9RbLV4046HihaqX49xTT2?= =?utf-8?q?b9Ovi8GZNJbpkwcGjP9eh2BWU7d5ztG3McKd5+QW0kOPmDbyKdkb+mB+5DWeclPrU?= =?utf-8?q?pLNGfwRXcR7Xhy6mK8bBTPuDpqjm8k/+2XUqC4D6+zIfi+1udnRICuYGDrxzgOqrD?= =?utf-8?q?619TT+KBISwaxGexH5u53+n7rrrX0ogwFF7zLYSZ3a8pMISKHwP8zzmlXRTDJ9TJU?= =?utf-8?q?nQmfpCj1JhA5a1QiCGwOl8qku5mDi6+GTuKVJPVMpw87v21J5VmuEqlJ9tQsjIWZ6?= =?utf-8?q?IervihiUERe/TLm3vtuHBETnFcGxV2qoqKIHNV+DazldXdXiwaja6GVL3q8K6VoKL?= =?utf-8?q?1jJcUGC/C3Wfb7R82aPuk0CBNAI/4oi3g+Lvzsdn602vKVbO5a5KzhKQ+4vONUWCT?= =?utf-8?q?md98YOOESJAtPW3rrjk0QkkCpTGRGMHvQ+dxkSbOV3UW3p+gXU50ZUOQ849MbXIQI?= =?utf-8?q?cXJDTGFC+x2fpnsNfTvvViAsMK7r9dpkPS13R+ti7l+RauSVdQXOR1MWqtKqJ2Mlh?= =?utf-8?q?rc2lpcKWfyG2PI7aO9XWPITujLjGhFHqafr3zcdhKA98gTIY+SuQmg66tKuKw1tRj?= =?utf-8?q?gXutmBubujzTAbFEWX8eqvrVC0hrumBs9GX82KRW32R9tXMNyJkEAVFAPpQz7OZLn?= =?utf-8?q?MERKrCkPPA5560UAr/DvAI4v+h72hs6kqM8KZ4yPqqSJHwLuvhRXHNCgs6Nsn6mlp?= =?utf-8?q?01dnF9dS2JV27ijtQJfnN+ZO2tO898AqDpZxQAUz24hcs++EVUVgrHy+x8Gd8fl3e?= =?utf-8?q?oBwqefiDNsGcQaRXIJvywfxqLjAtFxDN1NzsJYsYLK7/I0rbirCMofL9TPf+KbVFH?= =?utf-8?q?Q2MqFKCEIATBIHzuIzqu3fVJL4tLjtaA=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9bedd4c9-2a8f-46dd-9946-08da26967b77 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:34:58.5668 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2ZQLsqdP5cYEGBq8qweGyDUC+SBI2TSXgW1IzH5GDRx1Q9cf3Rryfhkyu+zONrqe/1tEG2+mbXV4UKPliCAoQw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR04MB8472 For large page mappings to be easily usable (i.e. in particular without un-shattering of smaller page mappings) and for mapping operations to then also be more efficient, pass batches of Dom0 memory to iommu_map(). In dom0_construct_pv() and its helpers (covering strict mode) this additionally requires establishing the type of those pages (albeit with zero type references). The earlier establishing of PGT_writable_page | PGT_validated requires the existing places where this gets done (through get_page_and_type()) to be updated: For pages which actually have a mapping, the type refcount needs to be 1. There is actually a related bug that gets fixed here as a side effect: Typically the last L1 table would get marked as such only after get_page_and_type(..., PGT_writable_page). While this is fine as far as refcounting goes, the page did remain mapped in the IOMMU in this case (when "iommu=dom0-strict"). Signed-off-by: Jan Beulich --- Subsequently p2m_add_identity_entry() may want to also gain an order parameter, for arch_iommu_hwdom_init() to use. While this only affects non-RAM regions, systems typically have 2-16Mb of reserved space immediately below 4Gb, which hence could be mapped more efficiently. The installing of zero-ref writable types has in fact shown (observed while putting together the change) that despite the intention by the XSA-288 changes (affecting DomU-s only) for Dom0 a number of sufficiently ordinary pages (at the very least initrd and P2M ones as well as pages that are part of the initial allocation but not part of the initial mapping) still have been starting out as PGT_none, meaning that they would have gained IOMMU mappings only the first time these pages would get mapped writably. Consequently an open question is whether iommu_memory_setup() should set the pages to PGT_writable_page independent of need_iommu_pt_sync(). I didn't think I need to address the bug mentioned in the description in a separate (prereq) patch, but if others disagree I could certainly break out that part (needing to first use iommu_legacy_unmap() then). Note that 4k P2M pages don't get (pre-)mapped in setup_pv_physmap(): They'll end up mapped via the later get_page_and_type(). As to the way these refs get installed: I've chosen to avoid the more expensive {get,put}_page_and_type(), favoring to put in place the intended type directly. I guess I could be convinced to avoid this bypassing of the actual logic; I merely think it's unnecessarily expensive. Note also that strictly speaking the iommu_iotlb_flush_all() here (as well as the pre-existing one in arch_iommu_hwdom_init()) shouldn't be needed: Actual hooking up (AMD) or enabling of translation (VT-d) occurs only afterwards anyway, so nothing can have made it into TLBs just yet. --- v3: Fold iommu_map() into (the now renamed) iommu_memory_setup(). Move iommu_unmap() into mark_pv_pt_pages_rdonly(). Adjust (split) log message in arch_iommu_hwdom_init(). --- a/xen/arch/x86/pv/dom0_build.c +++ b/xen/arch/x86/pv/dom0_build.c @@ -46,7 +46,8 @@ void __init dom0_update_physmap(bool com static __init void mark_pv_pt_pages_rdonly(struct domain *d, l4_pgentry_t *l4start, unsigned long vpt_start, - unsigned long nr_pt_pages) + unsigned long nr_pt_pages, + unsigned int *flush_flags) { unsigned long count; struct page_info *page; @@ -71,6 +72,14 @@ static __init void mark_pv_pt_pages_rdon ASSERT((page->u.inuse.type_info & PGT_type_mask) <= PGT_root_page_table); ASSERT(!(page->u.inuse.type_info & ~(PGT_type_mask | PGT_pae_xen_l2))); + /* + * Page table pages need to be removed from the IOMMU again in case + * iommu_memory_setup() ended up mapping them. + */ + if ( need_iommu_pt_sync(d) && + iommu_unmap(d, _dfn(mfn_x(page_to_mfn(page))), 1, flush_flags) ) + BUG(); + /* Read-only mapping + PGC_allocated + page-table page. */ page->count_info = PGC_allocated | 3; page->u.inuse.type_info |= PGT_validated | 1; @@ -107,11 +116,43 @@ static __init void mark_pv_pt_pages_rdon unmap_domain_page(pl3e); } +static void __init iommu_memory_setup(struct domain *d, const char *what, + struct page_info *page, unsigned long nr, + unsigned int *flush_flags) +{ + int rc; + mfn_t mfn = page_to_mfn(page); + + if ( !need_iommu_pt_sync(d) ) + return; + + rc = iommu_map(d, _dfn(mfn_x(mfn)), mfn, nr, + IOMMUF_readable | IOMMUF_writable, flush_flags); + if ( rc ) + { + printk(XENLOG_ERR "pre-mapping %s MFN [%lx,%lx) into IOMMU failed: %d\n", + what, mfn_x(mfn), mfn_x(mfn) + nr, rc); + return; + } + + /* + * For successfully established IOMMU mappings the type of the page(s) + * needs to match (for _get_page_type() to unmap upon type change). Set + * the page(s) to writable with no type ref. + */ + for ( ; nr--; ++page ) + { + ASSERT(!page->u.inuse.type_info); + page->u.inuse.type_info = PGT_writable_page | PGT_validated; + } +} + static __init void setup_pv_physmap(struct domain *d, unsigned long pgtbl_pfn, unsigned long v_start, unsigned long v_end, unsigned long vphysmap_start, unsigned long vphysmap_end, - unsigned long nr_pages) + unsigned long nr_pages, + unsigned int *flush_flags) { struct page_info *page = NULL; l4_pgentry_t *pl4e, *l4start = map_domain_page(_mfn(pgtbl_pfn)); @@ -177,6 +218,10 @@ static __init void setup_pv_physmap(stru L3_PAGETABLE_SHIFT - PAGE_SHIFT, MEMF_no_scrub)) != NULL ) { + iommu_memory_setup(d, "P2M 1G", page, + SUPERPAGE_PAGES * SUPERPAGE_PAGES, + flush_flags); + *pl3e = l3e_from_page(page, L1_PROT|_PAGE_DIRTY|_PAGE_PSE); vphysmap_start += 1UL << L3_PAGETABLE_SHIFT; continue; @@ -203,6 +248,9 @@ static __init void setup_pv_physmap(stru L2_PAGETABLE_SHIFT - PAGE_SHIFT, MEMF_no_scrub)) != NULL ) { + iommu_memory_setup(d, "P2M 2M", page, SUPERPAGE_PAGES, + flush_flags); + *pl2e = l2e_from_page(page, L1_PROT|_PAGE_DIRTY|_PAGE_PSE); vphysmap_start += 1UL << L2_PAGETABLE_SHIFT; continue; @@ -311,6 +359,7 @@ int __init dom0_construct_pv(struct doma unsigned long initrd_pfn = -1, initrd_mfn = 0; unsigned long count; struct page_info *page = NULL; + unsigned int flush_flags = 0; start_info_t *si; struct vcpu *v = d->vcpu[0]; void *image_base = bootstrap_map(image); @@ -573,6 +622,9 @@ int __init dom0_construct_pv(struct doma BUG(); } initrd->mod_end = 0; + + iommu_memory_setup(d, "initrd", mfn_to_page(_mfn(initrd_mfn)), + PFN_UP(initrd_len), &flush_flags); } printk("PHYSICAL MEMORY ARRANGEMENT:\n" @@ -606,6 +658,13 @@ int __init dom0_construct_pv(struct doma process_pending_softirqs(); + /* + * Map the full range here and then punch holes for page tables + * alongside marking them as such in mark_pv_pt_pages_rdonly(). + */ + iommu_memory_setup(d, "init-alloc", mfn_to_page(_mfn(alloc_spfn)), + alloc_epfn - alloc_spfn, &flush_flags); + mpt_alloc = (vpt_start - v_start) + pfn_to_paddr(alloc_spfn); if ( vinitrd_start ) mpt_alloc -= PAGE_ALIGN(initrd_len); @@ -690,7 +749,8 @@ int __init dom0_construct_pv(struct doma l1tab++; page = mfn_to_page(_mfn(mfn)); - if ( !page->u.inuse.type_info && + if ( (!page->u.inuse.type_info || + page->u.inuse.type_info == (PGT_writable_page | PGT_validated)) && !get_page_and_type(page, d, PGT_writable_page) ) BUG(); } @@ -719,7 +779,7 @@ int __init dom0_construct_pv(struct doma } /* Pages that are part of page tables must be read only. */ - mark_pv_pt_pages_rdonly(d, l4start, vpt_start, nr_pt_pages); + mark_pv_pt_pages_rdonly(d, l4start, vpt_start, nr_pt_pages, &flush_flags); /* Mask all upcalls... */ for ( i = 0; i < XEN_LEGACY_MAX_VCPUS; i++ ) @@ -794,7 +854,7 @@ int __init dom0_construct_pv(struct doma { pfn = pagetable_get_pfn(v->arch.guest_table); setup_pv_physmap(d, pfn, v_start, v_end, vphysmap_start, vphysmap_end, - nr_pages); + nr_pages, &flush_flags); } /* Write the phys->machine and machine->phys table entries. */ @@ -825,7 +885,9 @@ int __init dom0_construct_pv(struct doma if ( get_gpfn_from_mfn(mfn) >= count ) { BUG_ON(compat); - if ( !page->u.inuse.type_info && + if ( (!page->u.inuse.type_info || + page->u.inuse.type_info == (PGT_writable_page | + PGT_validated)) && !get_page_and_type(page, d, PGT_writable_page) ) BUG(); @@ -841,8 +903,12 @@ int __init dom0_construct_pv(struct doma #endif while ( pfn < nr_pages ) { - if ( (page = alloc_chunk(d, nr_pages - domain_tot_pages(d))) == NULL ) + count = domain_tot_pages(d); + if ( (page = alloc_chunk(d, nr_pages - count)) == NULL ) panic("Not enough RAM for DOM0 reservation\n"); + + iommu_memory_setup(d, "chunk", page, domain_tot_pages(d) - count, + &flush_flags); while ( pfn < domain_tot_pages(d) ) { mfn = mfn_x(page_to_mfn(page)); @@ -857,6 +923,10 @@ int __init dom0_construct_pv(struct doma } } + /* Use while() to avoid compiler warning. */ + while ( iommu_iotlb_flush_all(d, flush_flags) ) + break; + if ( initrd_len != 0 ) { si->mod_start = vinitrd_start ?: initrd_pfn; --- a/xen/drivers/passthrough/x86/iommu.c +++ b/xen/drivers/passthrough/x86/iommu.c @@ -347,8 +347,8 @@ static unsigned int __hwdom_init hwdom_i void __hwdom_init arch_iommu_hwdom_init(struct domain *d) { - unsigned long i, top, max_pfn; - unsigned int flush_flags = 0; + unsigned long i, top, max_pfn, start, count; + unsigned int flush_flags = 0, start_perms = 0; BUG_ON(!is_hardware_domain(d)); @@ -379,9 +379,9 @@ void __hwdom_init arch_iommu_hwdom_init( * First Mb will get mapped in one go by pvh_populate_p2m(). Avoid * setting up potentially conflicting mappings here. */ - i = paging_mode_translate(d) ? PFN_DOWN(MB(1)) : 0; + start = paging_mode_translate(d) ? PFN_DOWN(MB(1)) : 0; - for ( ; i < top; i++ ) + for ( i = start, count = 0; i < top; ) { unsigned long pfn = pdx_to_pfn(i); unsigned int perms = hwdom_iommu_map(d, pfn, max_pfn); @@ -390,20 +390,41 @@ void __hwdom_init arch_iommu_hwdom_init( if ( !perms ) rc = 0; else if ( paging_mode_translate(d) ) + { rc = p2m_add_identity_entry(d, pfn, perms & IOMMUF_writable ? p2m_access_rw : p2m_access_r, 0); + if ( rc ) + printk(XENLOG_WARNING + "%pd: identity mapping of %lx failed: %d\n", + d, pfn, rc); + } + else if ( pfn != start + count || perms != start_perms ) + { + commit: + rc = iommu_map(d, _dfn(start), _mfn(start), count, start_perms, + &flush_flags); + if ( rc ) + printk(XENLOG_WARNING + "%pd: IOMMU identity mapping of [%lx,%lx) failed: %d\n", + d, pfn, pfn + count, rc); + SWAP(start, pfn); + start_perms = perms; + count = 1; + } else - rc = iommu_map(d, _dfn(pfn), _mfn(pfn), 1ul << PAGE_ORDER_4K, - perms, &flush_flags); + { + ++count; + rc = 0; + } - if ( rc ) - printk(XENLOG_WARNING "%pd: identity %smapping of %lx failed: %d\n", - d, !paging_mode_translate(d) ? "IOMMU " : "", pfn, rc); - if (!(i & 0xfffff)) + if ( !(++i & 0xfffff) ) process_pending_softirqs(); + + if ( i == top && count ) + goto commit; } /* Use if to avoid compiler warning */ From patchwork Mon Apr 25 08:35:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825405 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E322FC433F5 for ; Mon, 25 Apr 2022 08:42:02 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312562.529874 (Exim 4.92) (envelope-from ) id 1niuI2-0004xn-T1; Mon, 25 Apr 2022 08:41:50 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312562.529874; Mon, 25 Apr 2022 08:41:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuI2-0004xg-Nz; Mon, 25 Apr 2022 08:41:50 +0000 Received: by outflank-mailman (input) for mailman id 312562; Mon, 25 Apr 2022 08:41:49 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuCB-0005mk-Vq for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:35:48 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.109.102]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id b427c536-c472-11ec-a405-831a346695d4; Mon, 25 Apr 2022 10:35:47 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2059.outbound.protection.outlook.com [104.47.12.59]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-33-GyOr7XTDMV6Y45BrJtxeOw-1; Mon, 25 Apr 2022 10:35:45 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by DBBPR04MB7641.eurprd04.prod.outlook.com (2603:10a6:10:1f7::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:35:44 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:35:44 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b427c536-c472-11ec-a405-831a346695d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650875746; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xpBRkm9Z5IBvu/NQMA+LCq2v737PicnDjxA+STnwy5s=; b=LHQLpS2dJwvVnjaLr852QknCwsaIN6y/ZBujl3QH7KjU6bU0MjKxlQ+rCsYhTWG2XWJr5N A/jxvGWDB59unjvm4HY4Ed9lbKih9CLBdjlvhP+bgK+XJNpQpWzxfkPdYrkzneKsmlLD3f e4VICLe5JwriFBA7RC6lY1ckdfjtGhU= X-MC-Unique: GyOr7XTDMV6Y45BrJtxeOw-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=clhCJyuyPVJOkk/RFMeOsimStHG7DCGYFXLt4YDz5r7m30pbzjTHaYPtIZ/z/rWV9aQm/a5m6a/TG1Hta8pypf05RuyxseGsaCRM5W1IZppl9ZglDaXJ+Ap6WL73TNDKjLkNENETX0Z3t8zIC3RJu96fW0hTarqpdGzmpCUI6iAIAcJVmtIbM7sq+weppRmThFxXkEh5OM6IlkDNUsLH8fuLcesNpDU61YZbuWXSisj0KowtDM0WlBDwoRmHNnsNNdci4j0UNTdteE3jaFHJtqFFzEdXaBNqf8zXXoxLvctq/qyn6nNxnndmsTp+YmVnkIe/YsQa5rL24OoNxw8NTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xpBRkm9Z5IBvu/NQMA+LCq2v737PicnDjxA+STnwy5s=; b=gslz2pLKZdGxIaWIJvr2T70alvPIBleVYreJxfZlcNzdaogpZkPPHKfZw8dApDN1lD7Vphr5IWxHAnsKN5bCGb8XUePSvv7MjB/6zY7cYlc/Q3qAISIhJf4+frVsVP4uxNyFCyjEsVYS+dMPLYVozEKhrVR5Gw+ROmdwpP7rCIguvOX9nmUITUwXEnGhLRV4puml+ENnSRnLAyjmIrwClZ6fTdFYZSIqLr4m8xydtoETXZ/y8tHdjmEtlrtUDQJiXSQzpIUpZ8r5d619u4CyIculyN7jjerDAgBYKaeSHjeClah8bs7dTq3gKHNX3RLoBpcUWJCHmDpcLYznsAi4dA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <1389c785-ffe3-5d8c-36f1-a923ce5250cd@suse.com> Date: Mon, 25 Apr 2022 10:35:45 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 07/21] IOMMU/x86: support freeing of pagetables Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu References: In-Reply-To: X-ClientProxiedBy: AS9PR06CA0194.eurprd06.prod.outlook.com (2603:10a6:20b:45d::29) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4786ce58-5143-4d8b-b37b-08da269696cc X-MS-TrafficTypeDiagnostic: DBBPR04MB7641:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jqOyCAWCKznMvypUKtfiWpUP2kDn3ZGwvltvR/p04I4Y0KYqdjiwQFNuJ81ToF5ofpWd91XXDnBv0eTz5Mxxx3H8JgQZtMbdntPdPMiwq8jtxrZ96+bDsUHasJXhCXVbYxdWp8YDhm1ITD6h+wEFX7BQkuhFp4wqPqvIlfn2ofCggFiJnO1K2St0+xgPm5BZ2+sCYvT8b3kj8Z3tJ+s0Q40H1mDu4vLHH0OZrd/JB+iy9RhtoAiMx62G8Ha8d97/qtPlY/wXZcXoTfgOXVcHdKEHZxfAfhB5aBkRXU6FTyRWS0gcsV3ev90zCox471CX91nhcV7miSNaqBDY3a9G9QLnjFYQrhy9q8BvTaYb3kfui0/pQfGKL2hPiFN/aiWYNoO8SMBeRveTB93PdJs4eQGIjsyWV5527679joyshpTcIh6okipFU4i5pepm58u1zY939EF+9w7CdHx3fycw608BZREFJg7KTyM3Bpo+iuZCwebCJVfwBL+QHrE2LD74Q6IIlOeQDI+ErLCc86EO/oFFGzXfK7VM0oDen7fsuS91M7zEPdE1OcEvTPMS6KCe2U/42RB/rgGJOA0Ya467H3D++eQpUiszXVk8AwDB5bKhHN6I3N4ZCBqky+Y99p9OUPt9xuo4hXCcJkIjgrCFgWd7BvIIgUmBp59A4/i2lz0atUcgCZVRH83GmN4atsTdkbvW0iqZe4ItBuY4r06TO/hAVXEOqM1StW93MseFNLzZhjm34yV7URDwKophxPQwkMC4JsFsipiX58/NBt00f6JAjArhp4hx056gi2IYikeoLB/fttGn1/LRWV1iZ52I X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(2616005)(186003)(6916009)(31696002)(86362001)(31686004)(8936002)(4326008)(8676002)(36756003)(2906002)(5660300002)(508600001)(66476007)(38100700002)(316002)(26005)(66556008)(66946007)(6506007)(6512007)(83380400001)(54906003)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?0CRs7IX1//RepZIoV96av5u+IbTM?= =?utf-8?q?s7fAA5MWCLn51aN5leWttPG9Nxt00F0Dc9xhrj+Psk1eoBgQgzw77v8oAct9Z75pX?= =?utf-8?q?hhH+NUfchP9UyG31UZXQtTPN6A3VxrSexH2ie8GTjUsDypO++Fm0UU47J0+7Im+Aq?= =?utf-8?q?qSZoRWYJLqju2H4+ujcPk9SxEwPM31D2UGngBuIDfgv9kx+micbNDQBHxZ/2M5IDs?= =?utf-8?q?NxZIzuS9hUU0YnXvYsoAPHlfJpqA5saDhtK9e/WbYVVQ1P8GcliFnY3zTAAEwLq9k?= =?utf-8?q?OdslQUMet2uJHUCsQ1n3M5b/+IF2NUYXEiAsQwe7dq9QTCDCEqBbB7WOZhOzWTn0V?= =?utf-8?q?8oIyhk6MEUlpUkKf669gWpwvZGeoTmMInPO7IxBGRcY6wB1l/A0hOj6ge6xMyA6Xp?= =?utf-8?q?08xSBpMmFgXw3AH/RyhqzcKTnGypSrzOtrzN3sFXlimq1LBAIE+aJ1FWn7l3G3BDm?= =?utf-8?q?nI3klhlYfwIPFI6f3zXC6p9lKfY9T7xJ2Jd1xmlNDJLnOCpYrVmw8hrGtmcn0W7X5?= =?utf-8?q?/qtExu3aL4AeSAKN0/V96fxaHldXxWfpsqp2uMZRW0+80mEf3yliqvPf/H/4oU+Ri?= =?utf-8?q?1gSZktnEj15IjY9YUJhlhns6NFQvlzEzexGMP9Edo7bhxj4oHftasigrT13tP6eHy?= =?utf-8?q?9E2pG09De93AEaD/6/faVIxcSvhsl70jzB5i0l7Lvp8OaBVz2u5NHgP6r4K2/vEUB?= =?utf-8?q?DtoxP88bHoDgA/X1G8iS4qMqIlpIozNLqgz8CMYgXkfMjag2y8uwujZ77Qk4DrLwr?= =?utf-8?q?pOeQUJkMYC83Nt0WtFh9Fz9/+1gVw8zxx6gFqP2xVQTKebMxHf4QiGXQ9f0zT99aB?= =?utf-8?q?BGKBeOuaz7eJthx65oqyrDndsJXWw6eQC9d3zZ5IsB7O5t9kf7K79JceZhITLUl2U?= =?utf-8?q?J8ugz6VY8pt9NdWjYqa0+MJBZOMzyKmQXTjV/r4+naJ6dQKUq6iNCKMy1Ui8nSc7J?= =?utf-8?q?8aapEDWjQINI3vRykeR8VuAIovurYkK7YRhNbvXZuWBV/Dth95khTnJGawfL2BOfO?= =?utf-8?q?AHckgNt1wVaXCkmq++MowOFsYwb6e/2gqnIn51L7rRwL16mFTCwlxlROtwbcAHr/L?= =?utf-8?q?XSuVAh3inp9XT9aqPmNJD0c0/+XzzPAog2q668rIAeU9v9u4bUwszxhJdWzM1Xc2n?= =?utf-8?q?y+jm7GBKFl0Qdq5VUGSBHG0D6NB1baff18vjMhYTZdWOsw+afVAYZ7IlZ9G14x0l/?= =?utf-8?q?HY1Qp3QAVP8iOab8L+nH8GREq+ibnroMlfxQFBI92720NbWcDT5t0tL/b9miazLjv?= =?utf-8?q?DqvNp7EaG5DBXg87+lLXs+jb/fXp/5Ojkg6FNwtRR4f7+n5XseYgYrH1j9mBtMZzl?= =?utf-8?q?AWzXRzyzcQh8lr9lMpiaFwPxV0i3gIiLPZUZZSZIL2ZaGAGdLU9YtJgYCPGY4gPLP?= =?utf-8?q?7gVRcBfWuSgcgAMYO3A7Tauix8mjQMPmhDjF+X3HcQT3slxl/Ho0mB+kbwtx+yW3R?= =?utf-8?q?7x9NwT+OtRi6PEupx4SeQiEP7qOblo80oiaG/zHhhzcc6qpVH3TThDWisteSNuPmq?= =?utf-8?q?vaqxroDqvar7/jq0ND4qBhan8UzikjmfPNB8/tHoJe9SovNrep/1ZfbeGB03OLy/l?= =?utf-8?q?Rbo8UCs0axAfkcPYVxFiN6yxlkthbMfdrXx4N9gPSF9S0ejZL5KnYoY55xyHWwQbt?= =?utf-8?q?eO6hjv1W32agXx3/dOrOEfXArDce0o6w=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4786ce58-5143-4d8b-b37b-08da269696cc X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:35:44.4231 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6ehietAuHGSpWwSk1Qjh5wrMwLnBLbCvBD9PZjVgRIwxWA6zLSizU2Hkt2RrQIjf5gI11iykMfDt2lyI3sStcA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR04MB7641 For vendor specific code to support superpages we need to be able to deal with a superpage mapping replacing an intermediate page table (or hierarchy thereof). Consequently an iommu_alloc_pgtable() counterpart is needed to free individual page tables while a domain is still alive. Since the freeing needs to be deferred until after a suitable IOTLB flush was performed, released page tables get queued for processing by a tasklet. Signed-off-by: Jan Beulich --- I was considering whether to use a softirq-tasklet instead. This would have the benefit of avoiding extra scheduling operations, but come with the risk of the freeing happening prematurely because of a process_pending_softirqs() somewhere. --- v4: Change type of iommu_queue_free_pgtable()'s 1st parameter. Re-base. v3: Call process_pending_softirqs() from free_queued_pgtables(). --- a/xen/arch/x86/include/asm/iommu.h +++ b/xen/arch/x86/include/asm/iommu.h @@ -147,6 +147,7 @@ void iommu_free_domid(domid_t domid, uns int __must_check iommu_free_pgtables(struct domain *d); struct domain_iommu; struct page_info *__must_check iommu_alloc_pgtable(struct domain_iommu *hd); +void iommu_queue_free_pgtable(struct domain_iommu *hd, struct page_info *pg); #endif /* !__ARCH_X86_IOMMU_H__ */ /* --- a/xen/drivers/passthrough/x86/iommu.c +++ b/xen/drivers/passthrough/x86/iommu.c @@ -12,6 +12,7 @@ * this program; If not, see . */ +#include #include #include #include @@ -550,6 +551,91 @@ struct page_info *iommu_alloc_pgtable(st return pg; } +/* + * Intermediate page tables which get replaced by large pages may only be + * freed after a suitable IOTLB flush. Hence such pages get queued on a + * per-CPU list, with a per-CPU tasklet processing the list on the assumption + * that the necessary IOTLB flush will have occurred by the time tasklets get + * to run. (List and tasklet being per-CPU has the benefit of accesses not + * requiring any locking.) + */ +static DEFINE_PER_CPU(struct page_list_head, free_pgt_list); +static DEFINE_PER_CPU(struct tasklet, free_pgt_tasklet); + +static void free_queued_pgtables(void *arg) +{ + struct page_list_head *list = arg; + struct page_info *pg; + unsigned int done = 0; + + while ( (pg = page_list_remove_head(list)) ) + { + free_domheap_page(pg); + + /* Granularity of checking somewhat arbitrary. */ + if ( !(++done & 0x1ff) ) + process_pending_softirqs(); + } +} + +void iommu_queue_free_pgtable(struct domain_iommu *hd, struct page_info *pg) +{ + unsigned int cpu = smp_processor_id(); + + spin_lock(&hd->arch.pgtables.lock); + page_list_del(pg, &hd->arch.pgtables.list); + spin_unlock(&hd->arch.pgtables.lock); + + page_list_add_tail(pg, &per_cpu(free_pgt_list, cpu)); + + tasklet_schedule(&per_cpu(free_pgt_tasklet, cpu)); +} + +static int cf_check cpu_callback( + struct notifier_block *nfb, unsigned long action, void *hcpu) +{ + unsigned int cpu = (unsigned long)hcpu; + struct page_list_head *list = &per_cpu(free_pgt_list, cpu); + struct tasklet *tasklet = &per_cpu(free_pgt_tasklet, cpu); + + switch ( action ) + { + case CPU_DOWN_PREPARE: + tasklet_kill(tasklet); + break; + + case CPU_DEAD: + page_list_splice(list, &this_cpu(free_pgt_list)); + INIT_PAGE_LIST_HEAD(list); + tasklet_schedule(&this_cpu(free_pgt_tasklet)); + break; + + case CPU_UP_PREPARE: + case CPU_DOWN_FAILED: + tasklet_init(tasklet, free_queued_pgtables, list); + break; + } + + return NOTIFY_DONE; +} + +static struct notifier_block cpu_nfb = { + .notifier_call = cpu_callback, +}; + +static int __init cf_check bsp_init(void) +{ + if ( iommu_enabled ) + { + cpu_callback(&cpu_nfb, CPU_UP_PREPARE, + (void *)(unsigned long)smp_processor_id()); + register_cpu_notifier(&cpu_nfb); + } + + return 0; +} +presmp_initcall(bsp_init); + bool arch_iommu_use_permitted(const struct domain *d) { /* From patchwork Mon Apr 25 08:36:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825393 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7517FC433EF for ; Mon, 25 Apr 2022 08:37:01 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312522.529797 (Exim 4.92) (envelope-from ) id 1niuDA-0008PB-E4; Mon, 25 Apr 2022 08:36:48 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312522.529797; Mon, 25 Apr 2022 08:36:48 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuDA-0008P4-A8; Mon, 25 Apr 2022 08:36:48 +0000 Received: by outflank-mailman (input) for mailman id 312522; Mon, 25 Apr 2022 08:36:46 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuD8-0008Oy-SG for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:36:46 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id d6e6387d-c472-11ec-8fc2-03012f2f19d4; Mon, 25 Apr 2022 10:36:45 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04lp2056.outbound.protection.outlook.com [104.47.13.56]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-12-gSnVVsnVNa-CbtIEbv3wKA-1; Mon, 25 Apr 2022 10:36:43 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by AM6PR0402MB3831.eurprd04.prod.outlook.com (2603:10a6:209:19::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:36:41 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:36:41 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d6e6387d-c472-11ec-8fc2-03012f2f19d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650875805; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BtcZhyZ3f1vF1sJNNT1uwgtm+sm1Ld+qz9aFACUX3vg=; b=gYkQ4TjBGzw6RvBgD/EigooTehKz7XibdXx5mTnm0q5SrGFdjN9apTNlVmOhXH76zlMn5b GVnyRrpZY5JzQrqbnY0+uOVM5R96k6yMRDXvPJh6N7Qo7AYQJQxRqvAPygTiueXUxFRulx 785+8mJkxElXusUaX1G2ZtHDe3CGeZA= X-MC-Unique: gSnVVsnVNa-CbtIEbv3wKA-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XdNXV77H1NXVLu3v1tX98yi5giqrhi1AVSWRKpIeJDImc59aqNeb1coPEGfr9PadNvQBThoYp3+JTZpR3S5Q+UL4s6CRgGdWz5qPmBvaFP8MWAK1IvqS/6ZUCFQuwSwY301oZ//Jd2XMM0gPR3K3k4q2XCrFI9Ko2TiGaSsvBYoObjAFIDyEv2fGDqfMVSRU7NxBBR11ts8OtPXie1qlOtK4WvqtAnrqWuY13Bii81DBPoyTB6aIAyq7tcRGXAJwVq+ELq+ocjNMVi+4zocQ9vHZhKNj79oZu7jDkt2bUT8ybXcmVg/QYMfTPKq+NXrvovjtrucdaJdNahZUf91Xhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BtcZhyZ3f1vF1sJNNT1uwgtm+sm1Ld+qz9aFACUX3vg=; b=GvKxR8c/dwRcqKyi34eO0hf+wIesvubCyO5d1FCLYVZV24GlKBrUaRhSjYFSmYtNjByxYrt57Gwhc2nFdQaStJxT0IKWl1Cs8uJ9lMsM+49APqZKVxUPXymZWR3QL7mfvy4KnRDVIh4NG7N1Arb3s0cYaSfl1/BNm6J/Qev1aUpB3yXsWWYZsbcldjupgGD8Z0JuqHNvCSV55/RSsTbqxfz7joIEx+0Ac70FHbbH/x9/PXAIPf4ZOxVQ8miQ9IdDykdFQBzUpY5jGZP2IrHQUB0ql9k1fZkKQjTUWRkIV0U7xTVai3JLdPH81BTSb4KzcJzB9o8HZo664s/qaLZZWw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: Date: Mon, 25 Apr 2022 10:36:42 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 08/21] AMD/IOMMU: walk trees upon page fault Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: AM5PR0101CA0011.eurprd01.prod.exchangelabs.com (2603:10a6:206:16::24) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 92e83e9d-ff7c-4cdf-3ddc-08da2696b908 X-MS-TrafficTypeDiagnostic: AM6PR0402MB3831:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xm7dGnj4lg4cYqZJgka0kXEATUj/KgewnOFgpQ1hn4GyZ9KuepI5SA64E0xUAq5nNwYL4OePaBatCPsNN8dAL8abJH4i4io9Fxw26kZSSkQl9X3d+FUwm29PvsJf32CWCcCyI6M+6vEtbBPD8wDzuHEu9LAMgXQzCO6ybIrrtcwn6OAzF9FvtjxQCPVi0xdPZYD11F5VCFiJuBBc6XI076oKYX8uvUVaAwWCMHGTxmSm1mNW/CQ30q+CnxjfW3O08t4X0rZrANGUzuFT2i603cROf1yMy4wvNf59epwjXXNrZ4uDwLGlArnZwzmTe4qxOkBpCIZBJxRVk6s5apRE3MehNAaUKmG9/MFJzvI636QHvE5ZtTZk3gYnNre4X6ux0qU/NSaBaq2hrU5an/BticdVztwy7u+uqFxdzr0UvTvbB0+ESQKNot3sEPEeDJbnjfqnBeqC7rTKriGFpBLAjPNoPYPdFgygzvT1zMTJQXal+73JVXD6Sp1H1sNdZVA530fMpAGjsD4rQphOcMoG0IsNeJzggkVS8fuaWMiHr4LzEiEG1pDVJuNiBZO0JI9XUQk4tyQpc9zXO1sx1zFfpOHjXWeb7/lS/lUWdy7g4wdR6kmN189zZxf2Osbk5j2wfup1jfBRpxfdFTIAcHrlLIFEU9NlyG4DetNEKo6Hn+sYyGg4YqixzuHA7rHa2LUR4kroaKGjybyws/Mtr0oO1HN4wQKrUfzgbdziEW8Z9N4= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(66946007)(2906002)(66556008)(4326008)(8676002)(38100700002)(5660300002)(508600001)(6486002)(6916009)(8936002)(54906003)(2616005)(31686004)(66476007)(36756003)(316002)(186003)(31696002)(86362001)(6506007)(26005)(83380400001)(6512007)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?5DPCvk6XFJWb/jPzgyvgWXEwoua8?= =?utf-8?q?MXocYjd6CJ40yUCxVvy7TA0q/t6b+zvlYbZHpQbYHAMpKEcCJEj+XzzND2a48aM6W?= =?utf-8?q?EzCimRtyTXnZDETgmd2nbMaFJERYXXuwiGTCaTGy2GiXYDxyFu64YygArdsfc6yfb?= =?utf-8?q?ZMo/LJjcsqrDePOSEH8CRuUK36/X7xLldgHM0E0Ak7ovZJfP/mgeQ76wgpd+Odhxp?= =?utf-8?q?91VXFC47mMx5UfrvgIjShfwJcOC+VP6+Gy6BOu4FiVuYpiJ9xwskcK8848kwIs5oa?= =?utf-8?q?1jwIbuozXJNThL1rlgGgfO+AFLJK4XiX3MX7o48U+AZmbKAjKSAVX7KGN1uYfBW+A?= =?utf-8?q?FD8jBVB+DAOiGcpCgaDWG2HVDalb02R8HvJaymf8G3o+FXTwNfK4NuqflpTETr6S8?= =?utf-8?q?fvJoAGY/rRIJtlO4Q38SlG0tN3UMrjngS533o03tzybh/5TmOOnkWYdSKBMC1SCnx?= =?utf-8?q?lLIeprBYVzLMWTcM/QMLbBHSUCEqhvqzzFVjIz64DPcAb8yp+ntu3jErsf9HahMwD?= =?utf-8?q?ZU1Gq2GSyoQTT83mIUCsamSi78Rxyfh9WQ/ZtsOjUtk9xcs3zTvuYr99axTjbGN2H?= =?utf-8?q?tbmHI0SGQ5B7eGWzVbZ2Anr5ngXguiyzPLdniMaR+lZ1x6npG4lD7lDP/y6kMaKXE?= =?utf-8?q?7JlXcZvbpBxrwcawgvj5pMUM28Bu350F63CA2o9NDuPTONTkA+VCuuEiumCWKsJAY?= =?utf-8?q?CSc87nuow0q5dfUc/rkKSsO+Mlu0EsPh/10rBW1DL5PXXrnp1ywMxAz5SvUNTfm8C?= =?utf-8?q?uchlVT61w/YPhVGepkudoSSyj4HJMtzYQfgAE1nCUkOclPW1DKqiQSr5+oqkIVyDT?= =?utf-8?q?bvH5LpiqPr4yR47+isUXrBhQc2Jcd/TSm81t+wkGIufcdFD9SZR+g8B2c4E59pUb9?= =?utf-8?q?TLlKrJfbdEccJYdjqogTP7z81ROVR7byxdZp4YMilGL9P/dkjhHEuxfiJOSZbzAgY?= =?utf-8?q?OoGe4/j+Ipy3RcGHDwUwo/Sma/S1o3bMAAvr+36648ylUd4SmDpcjyIPA0lcC62up?= =?utf-8?q?8Q0CTzAyZIB4cECgaVsw7w/sPBaZX3kfcmLuCEzGwuvtKr13wVYUMB70m2zYGwDZt?= =?utf-8?q?2LFp9hVg4qtTzpwSzEfxtYZbTYaSMEBSjn4mOkcRnfLntfy65JO+bU5Csz+Fyc/eP?= =?utf-8?q?/b54p+VKxG+us13IGvnsj9kR257F4WI9L8zBMUHZTZP+ntdUurCNWxykKLhG+Mlfv?= =?utf-8?q?IEVR4daf2vg4Jj0fziZQRovEmPW6eMxXt8E+u9OSp4aKzsf22NH9XweKmNkMEc1Fd?= =?utf-8?q?/yGXiePGv2vvvKAwVYZS+w8gmTwj2wbesroWd++azKOiSa7vMrlmDOoaRojVzKgDz?= =?utf-8?q?sUaXa75FHhP3+KGEj4a5JA2ve0HP5Rt72GAHmu+3ct3MzPhpyJXRUjEacLJu0xBm5?= =?utf-8?q?OyGxLMrA1/LjIyKRSFPPjUjqVoDtRfN2QsQsz8L6bfLlMKAdokVoGaT7+9Myn4gxP?= =?utf-8?q?IimeADyOGN9Wda9Qk9AslNRjcVHeF8Dqf1/YODyaiYZ6lQIC9pwQhuJ+M2oW3RC8Y?= =?utf-8?q?RB6K0ON1PCKAAm5CWrziE8FxkQbCkBnzzwMnDgumY/RkikTKnemShwFhTFPrNmlxQ?= =?utf-8?q?Dswat1xjA0+kIJoIYBdeObobX0HE0Px7jlkctsC+G3JKNVKt3y94YjBRRgi0nOuTG?= =?utf-8?q?WHkcu1i7PX93+SnJftPyIdP18mGU6Otg=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 92e83e9d-ff7c-4cdf-3ddc-08da2696b908 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:36:41.8402 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: l5c2buF642afHOntxQfEsIvR329//ygv1CjYXrIANgUDdYiUPcb8pamR7nb+4pTEpy9uuyUZswXAHuXCAjJ5ow== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR0402MB3831 This is to aid diagnosing issues and largely matches VT-d's behavior. Since I'm adding permissions output here as well, take the opportunity and also add their displaying to amd_dump_page_table_level(). Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- Note: "largely matches VT-d's behavior" includes the lack of any locking here. Adding suitable locking may not be that easy, as we'd need to determine which domain's mapping lock to acquire in addition to the necessary IOMMU lock (for the device table access), and whether that domain actually still exists. The latter is because if we really want to play safe here, imo we also need to account for the device table to be potentially corrupted / stale. --- v4: Re-base. --- a/xen/drivers/passthrough/amd/iommu.h +++ b/xen/drivers/passthrough/amd/iommu.h @@ -259,6 +259,8 @@ int __must_check cf_check amd_iommu_flus struct domain *d, dfn_t dfn, unsigned long page_count, unsigned int flush_flags); int __must_check cf_check amd_iommu_flush_iotlb_all(struct domain *d); +void amd_iommu_print_entries(const struct amd_iommu *iommu, unsigned int dev_id, + dfn_t dfn); /* device table functions */ int get_dma_requestor_id(uint16_t seg, uint16_t bdf); --- a/xen/drivers/passthrough/amd/iommu_init.c +++ b/xen/drivers/passthrough/amd/iommu_init.c @@ -575,6 +575,9 @@ static void cf_check parse_event_log_ent (flags & 0x002) ? " NX" : "", (flags & 0x001) ? " GN" : ""); + if ( iommu_verbose ) + amd_iommu_print_entries(iommu, device_id, daddr_to_dfn(addr)); + for ( bdf = 0; bdf < ivrs_bdf_entries; bdf++ ) if ( get_dma_requestor_id(iommu->seg, bdf) == device_id ) pci_check_disable_device(iommu->seg, PCI_BUS(bdf), --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -428,6 +428,50 @@ int cf_check amd_iommu_unmap_page( return 0; } +void amd_iommu_print_entries(const struct amd_iommu *iommu, unsigned int dev_id, + dfn_t dfn) +{ + mfn_t pt_mfn; + unsigned int level; + const struct amd_iommu_dte *dt = iommu->dev_table.buffer; + + if ( !dt[dev_id].tv ) + { + printk("%pp: no root\n", &PCI_SBDF(iommu->seg, dev_id)); + return; + } + + pt_mfn = _mfn(dt[dev_id].pt_root); + level = dt[dev_id].paging_mode; + printk("%pp root @ %"PRI_mfn" (%u levels) dfn=%"PRI_dfn"\n", + &PCI_SBDF(iommu->seg, dev_id), mfn_x(pt_mfn), level, dfn_x(dfn)); + + while ( level ) + { + const union amd_iommu_pte *pt = map_domain_page(pt_mfn); + unsigned int idx = pfn_to_pde_idx(dfn_x(dfn), level); + union amd_iommu_pte pte = pt[idx]; + + unmap_domain_page(pt); + + printk(" L%u[%03x] = %"PRIx64" %c%c\n", level, idx, pte.raw, + pte.pr ? pte.ir ? 'r' : '-' : 'n', + pte.pr ? pte.iw ? 'w' : '-' : 'p'); + + if ( !pte.pr ) + break; + + if ( pte.next_level >= level ) + { + printk(" L%u[%03x]: next: %u\n", level, idx, pte.next_level); + break; + } + + pt_mfn = _mfn(pte.mfn); + level = pte.next_level; + } +} + static unsigned long flush_count(unsigned long dfn, unsigned long page_count, unsigned int order) { --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -724,10 +724,11 @@ static void amd_dump_page_table_level(st mfn_to_page(_mfn(pde->mfn)), pde->next_level, address, indent + 1); else - printk("%*sdfn: %08lx mfn: %08lx\n", + printk("%*sdfn: %08lx mfn: %08lx %c%c\n", indent, "", (unsigned long)PFN_DOWN(address), - (unsigned long)PFN_DOWN(pfn_to_paddr(pde->mfn))); + (unsigned long)PFN_DOWN(pfn_to_paddr(pde->mfn)), + pde->ir ? 'r' : '-', pde->iw ? 'w' : '-'); } unmap_domain_page(table_vaddr); From patchwork Mon Apr 25 08:37:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825394 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 11BBFC433F5 for ; Mon, 25 Apr 2022 08:37:25 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312525.529807 (Exim 4.92) (envelope-from ) id 1niuDX-0000Ru-Na; Mon, 25 Apr 2022 08:37:11 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312525.529807; Mon, 25 Apr 2022 08:37:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuDX-0000Rn-KF; Mon, 25 Apr 2022 08:37:11 +0000 Received: by outflank-mailman (input) for mailman id 312525; Mon, 25 Apr 2022 08:37:10 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuDW-0008Oy-HA for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:37:10 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id e51cd92e-c472-11ec-8fc2-03012f2f19d4; Mon, 25 Apr 2022 10:37:09 +0200 (CEST) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01lp2055.outbound.protection.outlook.com [104.47.1.55]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-4-lhSQmLnTMdCSD3zsMAdqNw-1; Mon, 25 Apr 2022 10:37:07 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by AM6PR0402MB3831.eurprd04.prod.outlook.com (2603:10a6:209:19::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:37:06 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:37:06 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e51cd92e-c472-11ec-8fc2-03012f2f19d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650875829; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3svnu4oijNxO7SygTj1nIXI23Kv+KFJWpuZLxXqiiHk=; b=I3hoD7Yv80OcqzwbG1oXA6nUgQ+8Lpyt27WC50E/gad9mke2KnP3k+o5fiGiDuz/HnTuix r8BZPBU6ZYidpq5NuuRuM1P1UfxC1Vsu2TOm8LasJGNAmAH+dXY+EbuDQtgaXuuASupGY9 SKs2NMpvZgX9tA7tIwyyRKw5sYCGsSM= X-MC-Unique: lhSQmLnTMdCSD3zsMAdqNw-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Yvn6mRihD2sGvl+cQB3Q+DdgpxcOpwwUhlSBU/VWrlEGXHaKwZUPLG/wX7+CtLGFnv8YUj5LQ4LVHbQB7ZmZWR20mq+BCthfC7v2r4DTBp0DgAMfNruIkxHWR2R1vBg5AbJWEr/AHcalqARlSo/KLUCd04w5Z/04RHZjosiIL8AvyhWGJgS2ogmsGFoEs1UfiyIpDOiiguAD6RDBecGq/DCUO2udZNpaB7bi81cJvAbOwNDqma/QcfYYu0cAXDsLwxgp2UOIK+3IK72wXwcax9Q7tfjTJL7PmSoML5Z5Z7S2mw5eXbvHSFWm5+LeOTPSK30jKxfsYIDa/ZGqXdsmEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pJ/rcnHk48J2486MfEcTZdg7UhutmuOyywKelNyvVqw=; b=ZDHOkMHf20ORZOnUx7Q7vZHa8U2C2MtjwHwpd10T//5EiSFNauZMHLVqKeWNgjedTDWqjSw4EFlAGzv+SWlgqF0vWLAItJWYwJLDQzpmbRGZjQAYWQDlGqBc5wR8+AHUCwPg6OwZoz+ur6uMWpQGgUQd1ZqxJk3Ak/hMetYn0psdvYD6MLEsrE7sTHxCpJ45fAZbZDTyfQ83Z4h3WWbvxFHiAPFF+BFocsR6+bgNTZ0hmbjGosxJRZafv+SorTNcZVT6ZliUlFMJcopMuMq6IX2tXO3V5V4WevRXkJ/KuW5ZOyvBGlKVxOPhAOJ/PtO4xwJQPOW+i0/ZbmGfxB/AUw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: Date: Mon, 25 Apr 2022 10:37:06 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 09/21] AMD/IOMMU: return old PTE from {set,clear}_iommu_pte_present() Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" CC: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: AS9PR06CA0066.eurprd06.prod.outlook.com (2603:10a6:20b:464::11) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c68f9848-db3b-4065-c36a-08da2696c75e X-MS-TrafficTypeDiagnostic: AM6PR0402MB3831:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rrXWuFSVB2Ikn/Fsk7GLk4OKZXbfgWj1niR1mP5y1gIUMISAVCEYcv/lk3pypisCCKnogU4NK9uWRU3+oQptvYiHzVaGrlFCb8ByOHUDXsbnsSRkvUpk9+3KG2v7rstvqG2ZBtJR1z38h6zpZYgpsseogWjYhKQEJGm4g4YdUeISJ2yialnS/dNi4vC+weSYwDE66adtFiubACGeB/WBLZaJkONAJo9HB6j2jnZ/Q+n4RVGTFbRGeG88vhxYSNV6f31na/pf6L79wSsj8adX747IKf2Uf6MgTLBopD3lPPlFKeeq2ZqBsnka5eaXH6SgtSTttP5UpwodC9D2Sl2jdGxHtg/jUOXtgnzQ93YXC49rGW62c4Fn8FVCPNhEGhf2FGb6fXq6FAV19z812evuebkr9sbTsLDHRYP5q5s2cyTO5tGEnbNcHcbcgWNj9z++xG0Xh50i27hm6bhUQ6cOsyzkEx9t2UhrbqDgKdY2qsckTzU1qquW7KnZW0OjbHzRj/MRRoOx9l/zTCVl6fBT9c8QOSVSg8U5uDpi9teZKOjZ0cfsStH64l7n3Wjj3WIs/qxFFP9YNjNvJHUCKsjtedWOSjn4NiO+DygoI7EkdkJ4Cs/OIVPC2IA0G3dhh2iCFJmG/SSZodbr1KXiOkCqJ9BEtKSbPGvj5tcI6CVB8tGDLvdM5xlOdB+T2HiRevxHsx3oyswl/zyiB8XyMIrBOjvTSnmrtN1hhsZzA1TPf7g= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(66946007)(2906002)(66556008)(4326008)(8676002)(38100700002)(5660300002)(508600001)(6486002)(6916009)(8936002)(54906003)(2616005)(31686004)(66476007)(36756003)(316002)(186003)(31696002)(86362001)(6506007)(26005)(83380400001)(6512007)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: QpvxNlTRvthqlsTNAmRgcSHHtY4mT9GM2edATTJifFfNonCovqNEbXKSoCyZmy8WlED5BReGQJZ9G7+gyviF3P+kl/UWdbzB6KUf5WWtR8K3TbWQyZODU9u5rTGV7N7S+nA+ft/lNUo1QnVQ8/K2nAbWLBvV/CAOBeLFrJuVrtc0qUvLCfKjKTg09YSMrG8O4CrA/IxCSvPwBDQu9yojAUGAw11g0n0EQzBj4kv0XOp04H+2XZpHdWD3tUKu7s40UvKJ2/2GkuLQMJc7QEWuFpAnj/gvB5kzPifHDcXdLzPkZtrY6oiY6oIdEDwGOrJ+RXS69sZvvm36hNJDZWnobQHKsn70fOLULtIH3cR602jalycUIQDWBA9yoYqzbkrVPwLTwdLWiu1g8s3m0B6gvNqTxX3WQt+zSaVXuCIS9swqEte0aLiivmzlmObBiEPiBcYBNarDvkVxvfOhzfY66h9My4pBY9D7SGtW1NAUEFbDaWfKskt3Fou5FYdYIySB2PWOpAVEKLmACUoC8sHb69QbQ7Cf+gQ4fZnT8bRxYkhtWeW2altabXEhrCyZcq96r0thEMc1xNJUtirvmkptO03BQiOzW7dBcqpJ0Y2JBJsD7orfhWs+qrwNupd1VwbH6BsDJ571LB17vUNKQswCeYSejZLlvLoi95LNM8poCcz5wWEHw+9e7xtaSZOAYbdiKBn/PDqQUIBKN6ZIXYfqA55e/yjeukTG8B242ZNiqMr+CX+18v6IDqypQ5DYqMmfAWfoebU3SZuGfT15NGBDe/JB6dM6yZfnVY3nRczQhGWIMsaSTcfLIDVaaU9e1t4iwfiasS6XoGVmtakpHr9SH4GQsEPaFHCafh3lUS69aAVKwBhVQmjuxSSYDqh/ZwRpXY/YaDm2OmGmzUQWTm0MZ87a/updLbDssW8daip+ifzwJ8qWqoe/JHNK7SurQFVhGKwLDYNQZiNHwIbAj8tlilfZItoYB1TKqxP1FUMQ6p7RiOcabGNGUEH/DjXjJPsvdLu0l7pgWRTRscRM85BrWst1nsZdOkLs2mF60HHyFWcpJaXXGgTBc/e6UEoU9160ef0p36xLsKw0FBi5D4PQuGCOQr7OFNBVCN/y79p16SCmskB5HYqGunqWQ/203B5uU93yxqxp5TPta+uKwlyKhWEtHGbcJC3a9MkE3/zVpAnwdOC+OY1KHFWmY65qxv+3scL68DuF+EiqHGf03HGNjhVHaMITMDiXnfwu7nKvZ1u4fU1dWhoekDfD2rXEJkamHdFpx+OyPwGj0znp5fPZY4k2y04n4boj5hIxlxcrdNbzHcMOt1YhpyU0antAXc1LHEj/3h7Ho+08q0w908TwlkJPaeiXqW9YD9wFnUbUc0+ikJAOpI0575iHcY/NtbQPfyIa+0IG47bwSaZKj3L6W8ZlZdIV/OsfxIYVM7L//KEvHPHuXD1rX8trWLxv38YdrzmSfVXwpHhUwNNPZ5jkCiHyOCS/mG4s+WrugfTQzqtKgeFQyC5QlGvBX3wyPe9hyVlVes7vv/+w7iOuhvLP8DuL2OPQM/XVL+Z79bDqy5mPpE+jaennyE+Hbl73v98NIS6g7QvNcgpwoqqD0DGFxHyyMnLHxJmEmaThw/DXOmZS+EvLivUwReyjeMMKUyJRh0VCnhFbsGnBFYd6k/ZaabDqyQ/gXXOgjZSmdAyQmjtwZ0GPkHJ8k34TD70BALR08S4QvVbwE9admOLPXXvfEQ== X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: c68f9848-db3b-4065-c36a-08da2696c75e X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:37:05.9422 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5CzsvGC8W8vgJ6L7eXJO0CYSkYGJba6X/LQkt0x7tisP8jA6ws0qXwLvwWs+aF0HZbysfW7juSezZdanjJxR3w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR0402MB3831 In order to free intermediate page tables when replacing smaller mappings by a single larger one callers will need to know the full PTE. Flush indicators can be derived from this in the callers (and outside the locked regions). First split set_iommu_pte_present() from set_iommu_ptes_present(): Only the former needs to return the old PTE, while the latter (like also set_iommu_pde_present()) doesn't even need to return flush indicators. Then change return types/values and callers accordingly. Note that for subsequent changes returning merely a boolean (old.pr) is not going to be sufficient; the next_level field will also be required. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- v4: Re-base over changes earlier in the series. --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -31,30 +31,28 @@ static unsigned int pfn_to_pde_idx(unsig return idx; } -static unsigned int clear_iommu_pte_present(unsigned long l1_mfn, - unsigned long dfn) +static union amd_iommu_pte clear_iommu_pte_present(unsigned long l1_mfn, + unsigned long dfn) { - union amd_iommu_pte *table, *pte; - unsigned int flush_flags; + union amd_iommu_pte *table, *pte, old; table = map_domain_page(_mfn(l1_mfn)); pte = &table[pfn_to_pde_idx(dfn, 1)]; + old = *pte; - flush_flags = pte->pr ? IOMMU_FLUSHF_modified : 0; write_atomic(&pte->raw, 0); unmap_domain_page(table); - return flush_flags; + return old; } -static unsigned int set_iommu_pde_present(union amd_iommu_pte *pte, - unsigned long next_mfn, - unsigned int next_level, bool iw, - bool ir) +static void set_iommu_pde_present(union amd_iommu_pte *pte, + unsigned long next_mfn, + unsigned int next_level, + bool iw, bool ir) { - union amd_iommu_pte new = {}, old; - unsigned int flush_flags = IOMMU_FLUSHF_added; + union amd_iommu_pte new = {}; /* * FC bit should be enabled in PTE, this helps to solve potential @@ -68,29 +66,42 @@ static unsigned int set_iommu_pde_presen new.next_level = next_level; new.pr = true; - old.raw = read_atomic(&pte->raw); - old.ign0 = 0; - old.ign1 = 0; - old.ign2 = 0; + write_atomic(&pte->raw, new.raw); +} - if ( old.pr && old.raw != new.raw ) - flush_flags |= IOMMU_FLUSHF_modified; +static union amd_iommu_pte set_iommu_pte_present(unsigned long pt_mfn, + unsigned long dfn, + unsigned long next_mfn, + unsigned int level, + bool iw, bool ir) +{ + union amd_iommu_pte *table, *pde, old; - write_atomic(&pte->raw, new.raw); + table = map_domain_page(_mfn(pt_mfn)); + pde = &table[pfn_to_pde_idx(dfn, level)]; + + old = *pde; + if ( !old.pr || old.next_level || + old.mfn != next_mfn || + old.iw != iw || old.ir != ir ) + set_iommu_pde_present(pde, next_mfn, 0, iw, ir); + else + old.pr = false; /* signal "no change" to the caller */ - return flush_flags; + unmap_domain_page(table); + + return old; } -static unsigned int set_iommu_ptes_present(unsigned long pt_mfn, - unsigned long dfn, - unsigned long next_mfn, - unsigned int nr_ptes, - unsigned int pde_level, - bool iw, bool ir) +static void set_iommu_ptes_present(unsigned long pt_mfn, + unsigned long dfn, + unsigned long next_mfn, + unsigned int nr_ptes, + unsigned int pde_level, + bool iw, bool ir) { union amd_iommu_pte *table, *pde; unsigned long page_sz = 1UL << (PTE_PER_TABLE_SHIFT * (pde_level - 1)); - unsigned int flush_flags = 0; table = map_domain_page(_mfn(pt_mfn)); pde = &table[pfn_to_pde_idx(dfn, pde_level)]; @@ -98,20 +109,18 @@ static unsigned int set_iommu_ptes_prese if ( (void *)(pde + nr_ptes) > (void *)table + PAGE_SIZE ) { ASSERT_UNREACHABLE(); - return 0; + return; } while ( nr_ptes-- ) { - flush_flags |= set_iommu_pde_present(pde, next_mfn, 0, iw, ir); + set_iommu_pde_present(pde, next_mfn, 0, iw, ir); ++pde; next_mfn += page_sz; } unmap_domain_page(table); - - return flush_flags; } /* @@ -349,6 +358,7 @@ int cf_check amd_iommu_map_page( struct domain_iommu *hd = dom_iommu(d); int rc; unsigned long pt_mfn = 0; + union amd_iommu_pte old; spin_lock(&hd->arch.mapping_lock); @@ -385,12 +395,16 @@ int cf_check amd_iommu_map_page( } /* Install 4k mapping */ - *flush_flags |= set_iommu_ptes_present(pt_mfn, dfn_x(dfn), mfn_x(mfn), - 1, 1, (flags & IOMMUF_writable), - (flags & IOMMUF_readable)); + old = set_iommu_pte_present(pt_mfn, dfn_x(dfn), mfn_x(mfn), 1, + (flags & IOMMUF_writable), + (flags & IOMMUF_readable)); spin_unlock(&hd->arch.mapping_lock); + *flush_flags |= IOMMU_FLUSHF_added; + if ( old.pr ) + *flush_flags |= IOMMU_FLUSHF_modified; + return 0; } @@ -399,6 +413,7 @@ int cf_check amd_iommu_unmap_page( { unsigned long pt_mfn = 0; struct domain_iommu *hd = dom_iommu(d); + union amd_iommu_pte old = {}; spin_lock(&hd->arch.mapping_lock); @@ -420,11 +435,14 @@ int cf_check amd_iommu_unmap_page( if ( pt_mfn ) { /* Mark PTE as 'page not present'. */ - *flush_flags |= clear_iommu_pte_present(pt_mfn, dfn_x(dfn)); + old = clear_iommu_pte_present(pt_mfn, dfn_x(dfn)); } spin_unlock(&hd->arch.mapping_lock); + if ( old.pr ) + *flush_flags |= IOMMU_FLUSHF_modified; + return 0; } From patchwork Mon Apr 25 08:38:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825395 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4CBF3C433F5 for ; Mon, 25 Apr 2022 08:38:25 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312533.529819 (Exim 4.92) (envelope-from ) id 1niuEV-0001As-5X; Mon, 25 Apr 2022 08:38:11 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312533.529819; Mon, 25 Apr 2022 08:38:11 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuEV-0001Al-2I; Mon, 25 Apr 2022 08:38:11 +0000 Received: by outflank-mailman (input) for mailman id 312533; Mon, 25 Apr 2022 08:38:09 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuET-0001AU-P5 for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:38:09 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 0877dcb2-c473-11ec-a405-831a346695d4; Mon, 25 Apr 2022 10:38:08 +0200 (CEST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05lp2112.outbound.protection.outlook.com [104.47.18.112]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-24-9euADwJEMM6-oNQMvwzxDA-1; Mon, 25 Apr 2022 10:38:07 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by VE1PR04MB7424.eurprd04.prod.outlook.com (2603:10a6:800:1a7::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:38:05 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:38:05 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0877dcb2-c473-11ec-a405-831a346695d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650875888; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HbAbSHg5ZYzDcoXEhcNJap9YC68omZXSPDO+MrCQoo4=; b=a0B/FBbG54bSiF2MRSXM3YVxNwvPbqQGkN3pnGcbCJBq1nhn7r+BIz+F5iQzn65SE1Q0/e H1/hYbutfq4ZtLvhqhiGCLLGBffD7R+Xwbdc+lWMRU1PK/xOgnSC5Rwhxd9ETwzGDG9pbP KSlXvchJxU6jUaFp9BKoPZ3ve9JBf7k= X-MC-Unique: 9euADwJEMM6-oNQMvwzxDA-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Cbewgt8CsIo6ZPR2bkZk0GWOw4kfOiWCRFBP4svCdR4igDdcQVjgYG+wjq+diuMhe7MwFGLIBrUcsmWpU9tBRpZAptQFKot4Bb9hm67wY9J3hF2KoK9a61t5I8Qxkhr1HmPIyIGAxHvIKQ6PLk90c54pajs5FjN/Uk2uDMOXgFFn3T9UUu6v+DqM8g9k1tKe+KVo1glkWihZN4B9UAwHjQzFcWj6n8r6ADNBG0G5pYVFRBURHrcRH504ArFQNzKs1njPZFsOnZe61pcG0H9Ftbk9T70yi31sVZn/xF9KafAaqiMWt1GZJwS2HcYuo2D/+Rp+XCnwmVMuJ04qCos3EQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HbAbSHg5ZYzDcoXEhcNJap9YC68omZXSPDO+MrCQoo4=; b=AN4i23UufRgtexkGF58cmoZB5BQCTd9TvdJZGMZ++1x9LlMGs4+Y5WQJeXEeaDwVH/MhlCadrVN2Do1O92yprebqNhzq+Z4k+tSbkgjOR/h/0tP3d6FHxodxKlsaEyNaKrztmHa95GkJN3EjGVzy8F8U8Pd1kSvd5lD+GMhzaW2PFHiK4l7wISDnFkviQHGR07LObjoILRBks7/lr/0R0VJQPIQY6lOYwhi3Jb+t3Q+D6MCn0PJbdm/G/8CYlppXamvTM0u+Yfkh1kil+Ftyjm7GHMwDlq797nIiWg7j38poklIaYIvG85p+j0ZjF8D/6SD84h/qjSpijN67xMbTNQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <5866e22e-9f31-84ab-1df9-db84aa802944@suse.com> Date: Mon, 25 Apr 2022 10:38:06 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 10/21] AMD/IOMMU: allow use of superpage mappings Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: AM7PR04CA0014.eurprd04.prod.outlook.com (2603:10a6:20b:110::24) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d56cf4ea-ca11-43f8-2c34-08da2696eab8 X-MS-TrafficTypeDiagnostic: VE1PR04MB7424:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Fiut1mesAS/t0AksbdJlXJ0SSZtzAwn9hT8AUwtxdev03uSMZxhxzu544oz/pHolxu61r4ScHKBO58niWtBcUOyfcYlZoIdrmH4gp/K1AnGWzD/trDZ6R0pzyJAEplhsEmhP/AeCX9xXZx3zbVwtbeWp272bb++7PluO1r/PCN0dzd+AcA7t4AALCHMn51Gv/nzFHwENKzZckfcyCKEQCRs9aOtrheoPrhNwt1BNmyb1CzwVUxu1ZV+tpjM3IerRSiqxcwnNg8+MJwIz8o80hT8/+M/42k2nYPC6w9lhR/7zhuB8zLHtaCEwMj917GRSMvJ7ngENqDexO2m23rTiw2F5KAf9QsBux0iHdu8t3qBj2iYmWkp170uZ+xxuAJAJ0blH0Todp7arLN+id+/rX9RS1TnmAM6V250ppHyIkdOWcuN5wEgFMaZMMdDQaGFS3MsgU/EjcH4PSoYQUpRTqYLJG2oIjIhfk7M8v2H9A0Cjw3kAiSXATAIIBP27tD//B4F231tL0GCmqmNEIRUfsJaueQLqYbfp/O7G2rH87dXlUoRqQtRqabohRdqs+sHJiGkmF5L8arT6Az7yrhvyscdL3vPNFClId3LId96AaDREy4QIElkl8tIlY4e66wMyufL0juLFzAQl5T8yyXhXh7fldTEK1xACnQ8u0Jd01s+e/fOgLJIlfBPEUHhzbIUiAgI43kolJQWVgascJDehitvQRlAcku0zxqqS5ZcH2Rk= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(54906003)(31696002)(86362001)(66946007)(83380400001)(66556008)(8676002)(66476007)(38100700002)(2906002)(6916009)(186003)(316002)(2616005)(4326008)(8936002)(31686004)(36756003)(6506007)(508600001)(6486002)(6512007)(26005)(5660300002)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?1/OZyOGn8SDmLpaVacKj53o1gQl1?= =?utf-8?q?nC1tFbofV9cmn2yWEjurIFgd91usNm2Fmx76GDy7lPBYIniM1joBNAzz21eEi/YJQ?= =?utf-8?q?8E/Sq1qWSSp9JmJ70BKF5spJ1wq9qrZdZwnx8gppLwE4KGLjBXOv++VvuxyejhH3/?= =?utf-8?q?9oTASXt8aHX1HgAS01efDAjpO6i7g17pkDDPALfZUW/y6SSB7A6mrDQxBVbcueKgR?= =?utf-8?q?ELHHCclVDFS6GfpcLhh5QmqemNizD1mN6FzCNH17Cde4ZVmfy4hI/Blj091cFwPa+?= =?utf-8?q?PtDa402WC8bdB7hJ9Xu2eKBYdjI7xxBEuY99Q0pp2DuqYn0OV1UmnQEibQ5nU1ppn?= =?utf-8?q?G4sNJ0dOtB1u9F77B6KBEGMZ4AFQ10eu8k6Dj7+rJuOsKfn9N5Ju9xZ3/kmQpEHRX?= =?utf-8?q?cGpq5zgbYZA6bsFG2z0rxqeTuaSTqA3dyfwTZT4LooaqX1w8ObbE39QgeO3FRh6uX?= =?utf-8?q?+cz0z7teLBTw91iWjn97ZqwEJVmdbMU2LiU9ateWf+J8qynBc5wvyk1aR/svbtof6?= =?utf-8?q?3PjTkzaMbiovPFa4pkPGKUX9bMvavmOD8m4gawbcqGV6ojnrxBD8gGh7bairswp6F?= =?utf-8?q?qKur4nlZupGu7zc7dapLqojyiPIVLNu+sfLfXH3RxzWgJejHWuzysvOXQjkGWxHiC?= =?utf-8?q?U3Y+d03GJfrhCoNukOwx3QwA6vMjs61cuJgNgc0idmZad7WCNbqugqivwhguGguSt?= =?utf-8?q?eD5/KcW1e6Otk/ESdGZC98EzyY61NZL6JFlWQbSyt84QWSCN0tTazeAJ85ZXGSC5k?= =?utf-8?q?iy2tk10Jf+PbfpX8T54Gfb2M3KUSfbn3DVzJobmyh+hi9VtQvPcKr899t/4HEcJec?= =?utf-8?q?68536MQFbcNCK9ot1ZaTkDNZNifTBCt05DVqqmE3vKqEc/2XBSqOwxtym8xbWi/5K?= =?utf-8?q?N/Hoo25PJuY+RLQa7Mu/i6WyufbTXHn6cBqSkmF/P3C6YllhK3PpPY1Hkbiow9f/t?= =?utf-8?q?sVWZYgJhooV44e3FbqP66jf/yBFpuFx+NgNYcfqBDORQwE2xH7SQx21e9CfkJxI3V?= =?utf-8?q?fOD829OYBJUBWSuGRZS0s6/SUysW4OJjXqN/CI/iZzyM3wBGMBx6Y9gGQadxpJvu9?= =?utf-8?q?xIhvZ/GzBpFwrQxPYiNd7gN5YnCcA74flo6LHsIjy2VGsdaWNGeb+M8JL/ceIJezY?= =?utf-8?q?VWvvZoM2itkkVxjh9YBKbh5yUy4ZM6J1s1h4ta29SQdRgo6eNt1B0m4H2jB3FC1rN?= =?utf-8?q?tyUO/zqNaHlGLP5d2A3baGRCJ/paG19A1H0eZRxcQpMqhIBB5OrHjBlCPoUUN8G89?= =?utf-8?q?yQIJ9UqlSORD+qp2EO8yEoxvQLGVKyOOWF0mwtnrC4kZXRkOYIgtz9OcpLEBc8dKR?= =?utf-8?q?Ap0rexS7DllwKU+gypz9pU/Lrs1O/O5iENIZT3lBmWQAUqDlno+H8i2WrQ2wRIepi?= =?utf-8?q?wHX2BoaqmxxV82ov9kmnfcfEwMBoXPCjVGLi3pT93oNw4J3gr6JGeTYgWRMUByJ31?= =?utf-8?q?eiLpjJMIAik7s0bYhfvp7Wsliu03coR/VFIgUurIu2UB3pE6eFlVLRbppNsOBfir3?= =?utf-8?q?odgI33n0uHeRZIp6XemgHuJa4Oj1FZ7tLwpXQaI6oqh+F1ZjUqtro8nt2ocfY3EYc?= =?utf-8?q?JUNlBYWhSCzxQ9DnWRihOb/IhhTSWXG6iUtRUWjzryFxK+z9k8KrzjI1I/llavEWN?= =?utf-8?q?L5VmMwqgd7K/+yaC7DUYoM63T9QW4rMA=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: d56cf4ea-ca11-43f8-2c34-08da2696eab8 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:38:05.2328 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KSo6xq+YQlGLdbM93qG9dyJ3VhSsbd5Vboxi0sDqpRnf0RaaO16IK9bKlB8dD6nUEeR6imRzjEUM5GRcJaA+rw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB7424 No separate feature flags exist which would control availability of these; the only restriction is HATS (establishing the maximum number of page table levels in general), and even that has a lower bound of 4. Thus we can unconditionally announce 2M, 1G, and 512G mappings. (Via non-default page sizes the implementation in principle permits arbitrary size mappings, but these require multiple identical leaf PTEs to be written, which isn't all that different from having to write multiple consecutive PTEs with increasing frame numbers. IMO that's therefore beneficial only on hardware where suitable TLBs exist; I'm unaware of such hardware.) Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- I'm not fully sure about allowing 512G mappings: The scheduling-for- freeing of intermediate page tables would take quite a while when replacing a tree of 4k mappings by a single 512G one. Yet then again there's no present code path via which 512G chunks of memory could be allocated (and hence mapped) anyway, so this would only benefit huge systems where 512 1G mappings could be re-coalesced (once suitable code is in place) into a single L4 entry. And re-coalescing wouldn't result in scheduling-for-freeing of full trees of lower level pagetables. --- v4: Change type of queue_free_pt()'s 1st parameter. Re-base. v3: Rename queue_free_pt()'s last parameter. Replace "level > 1" checks where possible. --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -32,12 +32,13 @@ static unsigned int pfn_to_pde_idx(unsig } static union amd_iommu_pte clear_iommu_pte_present(unsigned long l1_mfn, - unsigned long dfn) + unsigned long dfn, + unsigned int level) { union amd_iommu_pte *table, *pte, old; table = map_domain_page(_mfn(l1_mfn)); - pte = &table[pfn_to_pde_idx(dfn, 1)]; + pte = &table[pfn_to_pde_idx(dfn, level)]; old = *pte; write_atomic(&pte->raw, 0); @@ -351,11 +352,32 @@ static int iommu_pde_from_dfn(struct dom return 0; } +static void queue_free_pt(struct domain_iommu *hd, mfn_t mfn, unsigned int level) +{ + if ( level > 1 ) + { + union amd_iommu_pte *pt = map_domain_page(mfn); + unsigned int i; + + for ( i = 0; i < PTE_PER_TABLE_SIZE; ++i ) + if ( pt[i].pr && pt[i].next_level ) + { + ASSERT(pt[i].next_level < level); + queue_free_pt(hd, _mfn(pt[i].mfn), pt[i].next_level); + } + + unmap_domain_page(pt); + } + + iommu_queue_free_pgtable(hd, mfn_to_page(mfn)); +} + int cf_check amd_iommu_map_page( struct domain *d, dfn_t dfn, mfn_t mfn, unsigned int flags, unsigned int *flush_flags) { struct domain_iommu *hd = dom_iommu(d); + unsigned int level = (IOMMUF_order(flags) / PTE_PER_TABLE_SHIFT) + 1; int rc; unsigned long pt_mfn = 0; union amd_iommu_pte old; @@ -384,7 +406,7 @@ int cf_check amd_iommu_map_page( return rc; } - if ( iommu_pde_from_dfn(d, dfn_x(dfn), 1, &pt_mfn, flush_flags, true) || + if ( iommu_pde_from_dfn(d, dfn_x(dfn), level, &pt_mfn, flush_flags, true) || !pt_mfn ) { spin_unlock(&hd->arch.mapping_lock); @@ -394,8 +416,8 @@ int cf_check amd_iommu_map_page( return -EFAULT; } - /* Install 4k mapping */ - old = set_iommu_pte_present(pt_mfn, dfn_x(dfn), mfn_x(mfn), 1, + /* Install mapping */ + old = set_iommu_pte_present(pt_mfn, dfn_x(dfn), mfn_x(mfn), level, (flags & IOMMUF_writable), (flags & IOMMUF_readable)); @@ -403,8 +425,13 @@ int cf_check amd_iommu_map_page( *flush_flags |= IOMMU_FLUSHF_added; if ( old.pr ) + { *flush_flags |= IOMMU_FLUSHF_modified; + if ( IOMMUF_order(flags) && old.next_level ) + queue_free_pt(hd, _mfn(old.mfn), old.next_level); + } + return 0; } @@ -413,6 +440,7 @@ int cf_check amd_iommu_unmap_page( { unsigned long pt_mfn = 0; struct domain_iommu *hd = dom_iommu(d); + unsigned int level = (order / PTE_PER_TABLE_SHIFT) + 1; union amd_iommu_pte old = {}; spin_lock(&hd->arch.mapping_lock); @@ -423,7 +451,7 @@ int cf_check amd_iommu_unmap_page( return 0; } - if ( iommu_pde_from_dfn(d, dfn_x(dfn), 1, &pt_mfn, flush_flags, false) ) + if ( iommu_pde_from_dfn(d, dfn_x(dfn), level, &pt_mfn, flush_flags, false) ) { spin_unlock(&hd->arch.mapping_lock); AMD_IOMMU_ERROR("invalid IO pagetable entry dfn = %"PRI_dfn"\n", @@ -435,14 +463,19 @@ int cf_check amd_iommu_unmap_page( if ( pt_mfn ) { /* Mark PTE as 'page not present'. */ - old = clear_iommu_pte_present(pt_mfn, dfn_x(dfn)); + old = clear_iommu_pte_present(pt_mfn, dfn_x(dfn), level); } spin_unlock(&hd->arch.mapping_lock); if ( old.pr ) + { *flush_flags |= IOMMU_FLUSHF_modified; + if ( order && old.next_level ) + queue_free_pt(hd, _mfn(old.mfn), old.next_level); + } + return 0; } --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -747,7 +747,7 @@ static void cf_check amd_dump_page_table } static const struct iommu_ops __initconst_cf_clobber _iommu_ops = { - .page_sizes = PAGE_SIZE_4K, + .page_sizes = PAGE_SIZE_4K | PAGE_SIZE_2M | PAGE_SIZE_1G | PAGE_SIZE_512G, .init = amd_iommu_domain_init, .hwdom_init = amd_iommu_hwdom_init, .quarantine_init = amd_iommu_quarantine_init, --- a/xen/include/xen/page-defs.h +++ b/xen/include/xen/page-defs.h @@ -21,4 +21,19 @@ #define PAGE_MASK_64K PAGE_MASK_GRAN(64K) #define PAGE_ALIGN_64K(addr) PAGE_ALIGN_GRAN(64K, addr) +#define PAGE_SHIFT_2M 21 +#define PAGE_SIZE_2M PAGE_SIZE_GRAN(2M) +#define PAGE_MASK_2M PAGE_MASK_GRAN(2M) +#define PAGE_ALIGN_2M(addr) PAGE_ALIGN_GRAN(2M, addr) + +#define PAGE_SHIFT_1G 30 +#define PAGE_SIZE_1G PAGE_SIZE_GRAN(1G) +#define PAGE_MASK_1G PAGE_MASK_GRAN(1G) +#define PAGE_ALIGN_1G(addr) PAGE_ALIGN_GRAN(1G, addr) + +#define PAGE_SHIFT_512G 39 +#define PAGE_SIZE_512G PAGE_SIZE_GRAN(512G) +#define PAGE_MASK_512G PAGE_MASK_GRAN(512G) +#define PAGE_ALIGN_512G(addr) PAGE_ALIGN_GRAN(512G, addr) + #endif /* __XEN_PAGE_DEFS_H__ */ From patchwork Mon Apr 25 08:38:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825401 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0F764C433F5 for ; Mon, 25 Apr 2022 08:38:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312538.529830 (Exim 4.92) (envelope-from ) id 1niuEz-0001ic-EL; Mon, 25 Apr 2022 08:38:41 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312538.529830; Mon, 25 Apr 2022 08:38:41 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuEz-0001iV-BH; Mon, 25 Apr 2022 08:38:41 +0000 Received: by outflank-mailman (input) for mailman id 312538; Mon, 25 Apr 2022 08:38:40 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuEy-0001TY-BJ for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:38:40 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 1aad4c05-c473-11ec-8fc2-03012f2f19d4; Mon, 25 Apr 2022 10:38:39 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04lp2055.outbound.protection.outlook.com [104.47.13.55]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-6-FnbnxG2eO5yWg5I1ukSaqQ-1; Mon, 25 Apr 2022 10:38:38 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by PAXPR04MB8638.eurprd04.prod.outlook.com (2603:10a6:102:21d::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:38:36 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:38:36 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1aad4c05-c473-11ec-8fc2-03012f2f19d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650875919; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m9iSfvMKYPfUIQdOapiU+/T/G/OgrgAKwMM8hwIFvpo=; b=EqkNxuy16guVo0OgYTCCIeK52s7phBnnaN7NsI4LyOfjM93nse67dl/aCTFYo/fm1Tw+vv iZOstjjYDkkOZ9kA+6yTRFWxH6xFiRti0gnjVUQzcdDZbbN76uAVJ+xDr+SoVRQvk6cKE3 b+JvCvfSQnwbsBVOijREzsSMIzTOiM0= X-MC-Unique: FnbnxG2eO5yWg5I1ukSaqQ-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AC9GrLuVvBD1b/TnZsGZSiVY1DbV36D8eXE/50Dd1+WHYeX8l/796QMQhbLWfQOU3iaPh1wZbM+vpTz1xeydMr30JN0qiHq71I5hkke5N9qy9Bejnok3hLUDp6btRlUxwwyZpAraJ/IIyimx1/vGYhZESEFNc/V1dovAiXTMLsKbanr5zzHiCjnwEu6vKhFUekMt3tkkCw2PaZukhotgzNpUA8LBIlWMHwynycKNudL82c33WqHb02pBH1X5OC0lQElG/MqEwODY12tAb9f9LhF6AQRncGfs+sgbyktCae7SmYujLOe9OM0np/KLo2VEyHEW2htiD+E88xmRVcV/+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=m9iSfvMKYPfUIQdOapiU+/T/G/OgrgAKwMM8hwIFvpo=; b=hRQozTN2AbYVHczUPgxM6DHCgnFgGdUqMzyWE7U+ae1FAj0rrHIdgD5yvJyf9mI0qPCCYw+DvBvwRytr857wYO5cJfTvsRQ9eMfyAk5xZ4EzgdjJIIdDmJbOwUOVx8O19gv2SkSt4WbFETSjq76zel40l5F0S5r5qjhQO/ltvQUYHHDOkT+Es39fs4y/1J+K02howFk8dOK7gRmsVztFZVps4qWp0moQTiHdZzgJIkGWOyvRIYputYeAqqxIXQ5v5MdaPfqQzuhsEDDWdu7CBxQG2WxuNyVkf4FxAA8PBqAucTfrCfoGQwFt/54pyMh3q+QgkpF3FfnxZmnAVlbnvw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: Date: Mon, 25 Apr 2022 10:38:37 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 11/21] VT-d: allow use of superpage mappings Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Kevin Tian References: In-Reply-To: X-ClientProxiedBy: AM6P192CA0016.EURP192.PROD.OUTLOOK.COM (2603:10a6:209:83::29) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 024289db-c4be-4ab4-6b62-08da2696fd57 X-MS-TrafficTypeDiagnostic: PAXPR04MB8638:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0txE0dPW/Knkt9kdhOFyIlR3d3u/lL8h3ml3s4PWP+iJ8P7ndgoQHkFE7Fx+sTdO80P6EcYKL58edE7HTg0SNAvBXJdr1lt6dBpOG8PVCrQQDNCc/9aC526DBFzaJMJmWvRWpNdUi7TsChWWyxfUi6/KQ5sIxHJ1knteJDPyKHZwQIIwwz953V0FnH1DGVsBie+gXQGU9l4T4cJSbxqVXziHo0ZzH8qwjgBeNsxAwbzOLsHKecq7+d2DOeHr//PGWw52D1kTP0SLKyZ4opPyZmFTHw/jFLBdD0+rjRVEt0mBGJeD4phtcyeoeJ+FnZXAgCh/V5897KRQUVl4PRUmJYyhbfDJr5rkkIyUKau1fpxGvjusro/X1B6MhfZDpOxmHP7Bg8uoPsX+n2pIX5fiCrNLTydRjm7+kGl5mn86fjLlhZwHsud2+HYZc36zPZB7Ca/zyOLk/k7GoqxyWc0OgB1p0JUOj1/gh9K3dp8jD84uS533KbwnPr68gPMmCcXtFbQowFGWNuuMXX2nHv/eqIGUcOhg5GiKGirlVnR7UX5/yNznT6akHzyoPgMDzDdlinoe+rvl7wsFvz3w9aLkvTwtPUa1YLRjoLPQaVh7SAkQHUoLaFLCKDfjxZp5JBFcmJtmgsk9VSpAr9U9pL56P5+/ctQeAp0tIiYsY9wPr51VVxEEJTmVhxeYd4L8nDcQipq6uI/86IO99m+UbQtYwUzEYn3CmSk5+dttvGX1WrE= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(5660300002)(31696002)(83380400001)(86362001)(316002)(8936002)(6916009)(66946007)(66476007)(66556008)(8676002)(26005)(186003)(2616005)(54906003)(4326008)(38100700002)(2906002)(6512007)(6506007)(31686004)(508600001)(6486002)(36756003)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?e9H8Ed2codkztjxdjwYfR60fsO4o?= =?utf-8?q?BjdBN4ma/dyhaMZHZFIx5TMUgEuzDDSCTqDNco2WMumbNskmT9gy8/2PSYB2Qfu2S?= =?utf-8?q?s6Fqx70qlbkQO+Zm0gThxv4VwYX3NhbjJNBl6e+WV/DST67CwixNlmM1BS08cwP1W?= =?utf-8?q?rm0cUXZ/19DmSLQ3Svpk04exnuH09EMazfkWirUbW6NXMjuNl5ELfYAyZwt3IWE5t?= =?utf-8?q?Z0L3YGBCiAt3KhHmNzl5N22FvPuC9UEaykjvvdIT44JGQz3vpADLCV9hpoNCH3cLe?= =?utf-8?q?MnFfnDgIQiODp0wSW3pp3gp25OU0lcqH4CH8e5wnUbzkZipPS7PWQozvMdsh6DPSq?= =?utf-8?q?R0oyaYfHDnB7EV8PFjwfyU+sdvtWJjpm6VsuOQnGgOKhlk1aHCla768Qdp+jvzmDc?= =?utf-8?q?6UJ7m1Ogvv8MaLN+m3o/uWlrmP86qOG0gq40MCczseMukn28VOyVTb+O3cA3sASaf?= =?utf-8?q?8kIzbJUjTc+u3+QWEdHO90tHl2dt1e7iHjJYckX3FBjhXVR3XXQy2nanhEIPr+cF1?= =?utf-8?q?j3f495UxhVqCczQuqoPzhQASpLXqJdxnV5oPwS5plekEysKrxJcUyg1BLwm3yGM6Q?= =?utf-8?q?QXg9AoZOfV/ZzinC8y0SMQjb7IBJlAyftmgHroAhSjplbLFhl04cf2/5c6sBYvUhq?= =?utf-8?q?8Qo6eMwcRyJG1TSBuI55yeN5Lfoc2khR/CmqWTqPL4+KO9Y2/ZcuTz2mUrqNaXdyv?= =?utf-8?q?b9mPC9lPTyRAaEEEouk/CAfoCaoATa6h2gf5OOap5n/iGrwiENI4I1CB4oOi4qqEC?= =?utf-8?q?eSwc2gui+Kp6w1npnK3D+LAMxKQ7SAEdo5hzb2wBcT0xIgKzqtvMXWBhEQGtcjkVB?= =?utf-8?q?Qw5GtonRTMyD5B7ruNG7hcECD2gXeydw8L65DQEYgxjyODBHnjlmr1HqNrgKxQif5?= =?utf-8?q?3DNFSzg7FhUFRvbFfnsvl6ISMAzoll1QCoEexrCXkM9P550cFLd3HEdqUoNqVGJ9z?= =?utf-8?q?1eIuBvzOyB29bKtwBPMM2NBFnI5Wi7spBqyggYdM7nKYLX+nLTcOEKXzhJrcS6HwB?= =?utf-8?q?PcWa1nsXvS/v+2qZ5DOfYdiMNqyRac7UJw8xIrGDDFgr8Fd/CI3uXqP5PKFJzHCFU?= =?utf-8?q?GQ9Rg47esiJadKIA52lozQib7beT1gUfQUzbaF2tjtTsixm0CCA6s7/Q1QjeWkI5E?= =?utf-8?q?oinKyIC4YCZXXNlfXn6CdHdhcneQ0uScnw5TFv3sQcc6nqcfF+CXG4Ym8sVAMUDGn?= =?utf-8?q?5MtoPSE1t59vHY5sJW/R7sp3BGlQcz1F7YH9SOoP3p3ZbaUERZ5BNHXXggidmmyj7?= =?utf-8?q?17gLrR9WIr8EgNeevqt8hWlMIuBJvJtxBz9KD4Xmv3JHgBcR3uTKzkeWAgy5FTKme?= =?utf-8?q?lZXkbZ/JWAueVMDFp5lKKc0RUV9VeOjasw/8MQGcLfD6ZTH8RhnevfF5yuAbvDKMD?= =?utf-8?q?MlM8bnGgue03WSBraX/Dq8kVA+GsU+IAd1FRdoEs0kczRGUr/qhgXS/6gWIykeTHD?= =?utf-8?q?00MnWUHwt+b8qpCma4HhMESVdbIJzbUs2ov//FtiMzJK4RuayBZN315WzVz54rRyp?= =?utf-8?q?RKBN6Z3CeAMHRmKND+B9vBH2ZCsNeEn+WySmRvslP6gY2Dkw3F6+AMJ2KZUIPei8S?= =?utf-8?q?KEW4mICHlDE6PJlcYsoWtYwDxN6lbcuvT9EYcKdbRCUFIO9TRh/hsJgX08+SQI+MY?= =?utf-8?q?SPtElDOtj9YiFRzxO54fJq4chR8KF6Lw=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 024289db-c4be-4ab4-6b62-08da2696fd57 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:38:36.4442 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +VzYyKTRyt6dCKW8dDbGLYv6niNWOFdSKAQ+GCN5TveojwzsCU3CvZpaHSudkuuoj5h15ylA8yUlizQ82Kkspg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR04MB8638 ... depending on feature availability (and absence of quirks). Also make the page table dumping function aware of superpages. Signed-off-by: Jan Beulich Reviewed-by: Kevin Tian Reviewed-by: Roger Pau Monné --- v4: Change type of queue_free_pt()'s 1st parameter. Re-base. v3: Rename queue_free_pt()'s last parameter. Replace "level > 1" checks where possible. Tighten assertion. --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -784,18 +784,37 @@ static int __must_check cf_check iommu_f return iommu_flush_iotlb(d, INVALID_DFN, 0, 0); } +static void queue_free_pt(struct domain_iommu *hd, mfn_t mfn, unsigned int level) +{ + if ( level > 1 ) + { + struct dma_pte *pt = map_domain_page(mfn); + unsigned int i; + + for ( i = 0; i < PTE_NUM; ++i ) + if ( dma_pte_present(pt[i]) && !dma_pte_superpage(pt[i]) ) + queue_free_pt(hd, maddr_to_mfn(dma_pte_addr(pt[i])), + level - 1); + + unmap_domain_page(pt); + } + + iommu_queue_free_pgtable(hd, mfn_to_page(mfn)); +} + /* clear one page's page table */ static int dma_pte_clear_one(struct domain *domain, daddr_t addr, unsigned int order, unsigned int *flush_flags) { struct domain_iommu *hd = dom_iommu(domain); - struct dma_pte *page = NULL, *pte = NULL; + struct dma_pte *page = NULL, *pte = NULL, old; u64 pg_maddr; + unsigned int level = (order / LEVEL_STRIDE) + 1; spin_lock(&hd->arch.mapping_lock); - /* get last level pte */ - pg_maddr = addr_to_dma_page_maddr(domain, addr, 1, flush_flags, false); + /* get target level pte */ + pg_maddr = addr_to_dma_page_maddr(domain, addr, level, flush_flags, false); if ( pg_maddr < PAGE_SIZE ) { spin_unlock(&hd->arch.mapping_lock); @@ -803,7 +822,7 @@ static int dma_pte_clear_one(struct doma } page = (struct dma_pte *)map_vtd_domain_page(pg_maddr); - pte = page + address_level_offset(addr, 1); + pte = &page[address_level_offset(addr, level)]; if ( !dma_pte_present(*pte) ) { @@ -812,14 +831,20 @@ static int dma_pte_clear_one(struct doma return 0; } + old = *pte; dma_clear_pte(*pte); - *flush_flags |= IOMMU_FLUSHF_modified; spin_unlock(&hd->arch.mapping_lock); iommu_sync_cache(pte, sizeof(struct dma_pte)); unmap_vtd_domain_page(page); + *flush_flags |= IOMMU_FLUSHF_modified; + + if ( order && !dma_pte_superpage(old) ) + queue_free_pt(hd, maddr_to_mfn(dma_pte_addr(old)), + order / LEVEL_STRIDE); + return 0; } @@ -2097,6 +2122,7 @@ static int __must_check cf_check intel_i struct domain_iommu *hd = dom_iommu(d); struct dma_pte *page, *pte, old, new = {}; u64 pg_maddr; + unsigned int level = (IOMMUF_order(flags) / LEVEL_STRIDE) + 1; int rc = 0; /* Do nothing if VT-d shares EPT page table */ @@ -2121,7 +2147,7 @@ static int __must_check cf_check intel_i return 0; } - pg_maddr = addr_to_dma_page_maddr(d, dfn_to_daddr(dfn), 1, flush_flags, + pg_maddr = addr_to_dma_page_maddr(d, dfn_to_daddr(dfn), level, flush_flags, true); if ( pg_maddr < PAGE_SIZE ) { @@ -2130,13 +2156,15 @@ static int __must_check cf_check intel_i } page = (struct dma_pte *)map_vtd_domain_page(pg_maddr); - pte = &page[dfn_x(dfn) & LEVEL_MASK]; + pte = &page[address_level_offset(dfn_to_daddr(dfn), level)]; old = *pte; dma_set_pte_addr(new, mfn_to_maddr(mfn)); dma_set_pte_prot(new, ((flags & IOMMUF_readable) ? DMA_PTE_READ : 0) | ((flags & IOMMUF_writable) ? DMA_PTE_WRITE : 0)); + if ( IOMMUF_order(flags) ) + dma_set_pte_superpage(new); /* Set the SNP on leaf page table if Snoop Control available */ if ( iommu_snoop ) @@ -2157,8 +2185,14 @@ static int __must_check cf_check intel_i *flush_flags |= IOMMU_FLUSHF_added; if ( dma_pte_present(old) ) + { *flush_flags |= IOMMU_FLUSHF_modified; + if ( IOMMUF_order(flags) && !dma_pte_superpage(old) ) + queue_free_pt(hd, maddr_to_mfn(dma_pte_addr(old)), + IOMMUF_order(flags) / LEVEL_STRIDE); + } + return rc; } @@ -2516,6 +2550,7 @@ static int __init cf_check vtd_setup(voi { struct acpi_drhd_unit *drhd; struct vtd_iommu *iommu; + unsigned int large_sizes = PAGE_SIZE_2M | PAGE_SIZE_1G; int ret; bool reg_inval_supported = true; @@ -2558,6 +2593,11 @@ static int __init cf_check vtd_setup(voi cap_sps_2mb(iommu->cap) ? ", 2MB" : "", cap_sps_1gb(iommu->cap) ? ", 1GB" : ""); + if ( !cap_sps_2mb(iommu->cap) ) + large_sizes &= ~PAGE_SIZE_2M; + if ( !cap_sps_1gb(iommu->cap) ) + large_sizes &= ~PAGE_SIZE_1G; + #ifndef iommu_snoop if ( iommu_snoop && !ecap_snp_ctl(iommu->ecap) ) iommu_snoop = false; @@ -2629,6 +2669,9 @@ static int __init cf_check vtd_setup(voi if ( ret ) goto error; + ASSERT(iommu_ops.page_sizes == PAGE_SIZE_4K); + iommu_ops.page_sizes |= large_sizes; + register_keyhandler('V', vtd_dump_iommu_info, "dump iommu info", 1); return 0; @@ -2961,7 +3004,7 @@ static void vtd_dump_page_table_level(pa continue; address = gpa + offset_level_address(i, level); - if ( next_level >= 1 ) + if ( next_level && !dma_pte_superpage(*pte) ) vtd_dump_page_table_level(dma_pte_addr(*pte), next_level, address, indent + 1); else From patchwork Mon Apr 25 08:40:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825402 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 16586C433EF for ; Mon, 25 Apr 2022 08:40:31 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312547.529841 (Exim 4.92) (envelope-from ) id 1niuGT-0003Br-U2; Mon, 25 Apr 2022 08:40:13 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312547.529841; Mon, 25 Apr 2022 08:40:13 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuGT-0003Bk-QR; Mon, 25 Apr 2022 08:40:13 +0000 Received: by outflank-mailman (input) for mailman id 312547; Mon, 25 Apr 2022 08:40:13 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuGT-0003Bc-1g for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:40:13 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.109.102]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 51ae4a43-c473-11ec-8fc2-03012f2f19d4; Mon, 25 Apr 2022 10:40:11 +0200 (CEST) Received: from EUR04-HE1-obe.outbound.protection.outlook.com (mail-he1eur04lp2055.outbound.protection.outlook.com [104.47.13.55]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-35-3F6FFF6yNJy5jC8k8S1EAA-1; Mon, 25 Apr 2022 10:40:09 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by VI1PR04MB3024.eurprd04.prod.outlook.com (2603:10a6:802:d::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.18; Mon, 25 Apr 2022 08:40:06 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:40:06 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 51ae4a43-c473-11ec-8fc2-03012f2f19d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650876011; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Rl/rdFZhS9nc5eyV5hC8ECsbWBsvtfY3/7jS7Dgl1JE=; b=EoABwmeuWlggepzpWtkjaJD32vsbewyVuKdu4X8SPf9+itK/SyDBtqJnvQ+/27em3YgeUQ bqiiVpm5eQFP4KWrBe4hmdmZY9vB5bBR6yCjVP2jNZOMwHZGgYuIEADqgCwZEL8UQkN7ht wd4mvMTxw24yTrTB7l3XIyxgM5biQRY= X-MC-Unique: 3F6FFF6yNJy5jC8k8S1EAA-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LXMyRwtIGfrvAAIAB1EYQmowUJqPgI4gjMD/0XF9WPzaYETlnEGaTN/Rq4UcAC+cDzdlUIrGEDklMl35dGPn1iRNCvWOA2j9EBpO2nnCbrokmAdEexQ93k2eQe+Hj+s8i2hXXKEtwhEg8b6THaH75vje/NKBWui/TVvC+UPPvzpJ3K6r3Nw0mjpi0sgN8MBd0b1vDOvHR2n47evm3786xzxbXNPo3mPIsaSRHnOPtWkLm36AA7TocdGGZD3R+OTpAsbA43uZKNFkOSe3DRZ7vOKic0qSK8dxX9q+sfTeDxtX/Hl5F6+YDS/FoHP70DDXxPPjr1uHVqlnVuWa5MlbBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=cKU8QBhqAL8on7QC3DOwvdl/zHdOAjDj5Tns2U4wJto=; b=ZQDFbAIlAKOsg8tpHOqv3qSiWq5BKfaGsyREXhcvIL6ArFy5OWtQDo7UR9bgKw1znN+IunRBiZHWVb18mqIbXJtOSKNksYpmfR9a2doRNJOrDmjmaNqWSjIRG8ia+kX/SdPemvGAYNLHltjxIe6Bf9+wn2g3qVsZ9F0T5vzf7fPe95NmhJ1KTpBj1H307oaYJanfrgR7AWHwOlqkAPUZqcXR5udhBhhy4+J3yBHF8TgLLFjhL7CFm3H9x4HXvnJXVlZO7XiW2Ds+CcQGDCAKo9uXzk8WEFZB31OoBwALtLt1r9Cmof7lTZUomVA1UT+BFSkeH8+zjrkamgLWYyclSA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <024c89df-9cf9-4d74-116d-ca8481dca90b@suse.com> Date: Mon, 25 Apr 2022 10:40:06 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 12/21] IOMMU: fold flush-all hook into "flush one" Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" CC: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: FR2P281CA0010.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a::20) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 82bd58dc-66e2-41ee-fbc4-08da269732bd X-MS-TrafficTypeDiagnostic: VI1PR04MB3024:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FPaoW+yJEZmfxGVw+R5ApLtETmbykN4S76uLHfqFCkHdkJGRbPxKqJg8ESda51jdqkUreXGpUeuKCbNCMPPqtd0yM9SmyC+aSPExNRgpgHCI5/asGdGTHTUamNzR2I2rLNroVU2JSE9TIQDMQAalmCOOxtzjn0A1cplR1X/6Jhdwa++DLdLxJaSxYIUNZy6ifP984huGp0P8L9scmm+lXYn4VHL0igzbyGMGvBNtoiuGuYT+yXCSJu/CkK6M0phe2hBbcTmDn9y39qKoX2JUpMlikJwjuS5OmbLdn24mSqPOEhnmRgUmPUyt4+G8PZrTWNyo50hy6nD2lpVHpZvPFOWqqmzncFOdKTRwyEvrV2e/nwwGaaZYnALMY4o37cOiORh9XxGYN9jP0+OC6JiNDxF9NsQVxwjBW3twkFndoqjqJ/UQjO3yBkUmLQqewcOH5t8N+mfKg4zboA9S65mljxTmdQ+EQg+fPnyuDJjp1Ouzwmz6lgt1toOi2v1cJAsB5RvUWL5FGJwZZVNLeuRvSmgDdP/FfhMLSo5lIcRQj0/NchA0heQsb07kM/HYgv/H87Oym3wQV+mUmXxcxDJ2iQDJI6Vp7Scw92TjtIIOqWkAmpmp4+AfACGr0Fe5wh8MRSIY9b7t9CsrTheYXXIUAaj1BhUaUyj7wEuxV2VIhPJ2NMw1h25euuXwkwG3K0pAdsd377Ifw6ex6MyRiV6SlnwNB+haB70Z6tlxqqbr0lM= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(5660300002)(86362001)(38100700002)(31696002)(508600001)(186003)(2616005)(83380400001)(8936002)(6486002)(26005)(6506007)(66946007)(6512007)(66556008)(36756003)(2906002)(6916009)(66476007)(31686004)(54906003)(4326008)(8676002)(316002)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 6wHeL+SCGwc0O6Z++agJmn05CcUEssKj/mai+ElDXgdOt6jQUGGhrnEa9Q8mpI7l62yKdNCgS0oi/OfG05fszUNAc4/YdwU8lmnEHDoafZy8B/CYh4WfKjy4jvfa/6FZIcm1O/Eh5+lrDitGPSTa+3ZTwnq8fQ44VemR3ZEcwg4C6GGxYcxNdPiyoH+Unx+d+xEdVyThwSz9ltgsxBbVMCMfbUEQ1ia1Fh1oFyVfTa0H5iqZ3e73uRx6KsaqChbNZeebEX4oGXlBsKFr8bB+21exyK08iGHsurUWihAz6UAqUTW4QH94sn/F6QAfy1p/sT+2losEMwFfJ0cCk8ZK4iZkehEK08i92U1zjK2UEsPHM5Bk3tbuGGyS0NP3EIGvtYK2Kyv2yVvdFXLt0z1gLFe/FVd+Lhao8fXzVjtUREzfEJVy219XvUn2GtI3wonbBW4Fr3MdlnGb7sNBdL7Gv9ihnw26i0KwSxhbGSy7X2A4vQqPs+m2j7VZP6YXGi03uRZALrmIWf1s/o3FXwT3zbYCd0+jpCiovVkFYaFTMzgcOi7cviWgSmnEiKukcGVADVjv47kRyaB1VpCug1kViZSdJkQu9gbfJSjSmT+CBd32JIqBo1XMaKMWLE06wMjJoAbO7Tg9vtQJCuSSPqcz4UOuIQ+cSw4WNx+0GNrya3K8J7R0kdNcz/wqy1LVLXmRibQfkKzmYD4vjWKeNb0i1eWzyw4B4rALrrRH/DGci4l33unurIzPE/Om/5lwcxBVmmEzj3GCD8zPRWHndz7IZlCzRLvbj6eBeBEgekqYDQWEdVtU47R47vdK0KfHAgohcrJTeA39sNto59bGlHT3C6WLONK9fpXAtXn8DUoQBLq3gWjwUDursmUrMPapu0aLPRfz8okzHAkXEUdoftInLNRWFKqueJEQPctQK+hULsuf/N6Oqn36teVSsTPUeGsz+X6hAPGXatkq2nDrBTNgPGn/QiZy3+hwq6wMCM9MaMmndROYDTWDqFLXo37rvZqy7cXXDkiZTcp/dTCfrqZRFfczs/WAQzA6VAO1t8/g46z3TdoJXsmItWMjo82qFnk6o/hoQF/ynHeTiCFmhcDowjr5lMkMyQIfidoXuCwGSJ+z/RnpoHhYLGe43wkTu405mxVdtIjxmj99WvW0Tj0BOpoQGsmgW1qfqnnaHOTtfR8T66Gxesh3/oz3AMBNEzYn3qz7f2vc2a4GkEBqrjeuO2WJTBRe7QYVv5s+aFXPDXbtGZkUjKiA7faN2ktybopOOzKnmp2dBjvf4kxNl47YBkZ5KPJDMTGJGa81jibjKItB8ZNqck5i/Kn+OPTmXac2nMTJkw+mL4ifJnLQUxIigydKkqwGJ5rSP+CDIe5M0QjaTPFUBxk2O2vQx6HE7XMxJ8k5/fUrg/CshONpnW7FUhQtiV97ANOlg4Q/GdSzyCoq/xruGDPUFjXta5QudSszVo9846aNXxlsK2J1uY/Z/D60zgawuPJ9UIvN4pif/LHd4bZsv4sYqvi9/ar1INrt7tO7FZ2Z/PQIl0B047KJtB93DYDyDetZAN1Nr1lYqORpITxcTVazHbHXk+uhPvTG+omtmSDZ0C7ndFnRxJeZ1C4CyHDvhkOZnug6KwolIPCo1sbdhKGb/dEYHj2zNNdMzT54f+VaH6mvlH7TqWGfedbf1wO0zSpSpeuOJi6tBy9A3uZ7+IwhX63aSYS4GJiMkZ6PBEUbhtN2rKJHk7TSMQ== X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 82bd58dc-66e2-41ee-fbc4-08da269732bd X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:40:06.0657 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uVB+Hh6GpwtGZPGomUBi9S2PMHQ5b0XT0eTEr9YF7a3Q4JuqIbi9RTjhzfMCHKrF5cf9FY2WgFTQIR+QdVIFPQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3024 Having a separate flush-all hook has always been puzzling me some. We will want to be able to force a full flush via accumulated flush flags from the map/unmap functions. Introduce a respective new flag and fold all flush handling to use the single remaining hook. Note that because of the respective comments in SMMU and IPMMU-VMSA code, I've folded the two prior hook functions into one. For SMMU-v3, which lacks a comment towards incapable hardware, I've left both functions in place on the assumption that selective and full flushes will eventually want separating. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné Reviewed-by: Oleksandr Tyshchenko # IPMMU-VMSA, SMMU-V2 Reviewed-by: Rahul Singh # SMMUv3 Acked-by: Julien Grall # Arm Reviewed-by: Kevin Tian --- TBD: What we really are going to need is for the map/unmap functions to specify that a wider region needs flushing than just the one covered by the present set of (un)maps. This may still be less than a full flush, but at least as a first step it seemed better to me to keep things simple and go the flush-all route. --- v4: Re-base. v3: Re-base over changes earlier in the series. v2: New. --- a/xen/drivers/passthrough/amd/iommu.h +++ b/xen/drivers/passthrough/amd/iommu.h @@ -258,7 +258,6 @@ int cf_check amd_iommu_get_reserved_devi int __must_check cf_check amd_iommu_flush_iotlb_pages( struct domain *d, dfn_t dfn, unsigned long page_count, unsigned int flush_flags); -int __must_check cf_check amd_iommu_flush_iotlb_all(struct domain *d); void amd_iommu_print_entries(const struct amd_iommu *iommu, unsigned int dev_id, dfn_t dfn); --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -539,15 +539,18 @@ int cf_check amd_iommu_flush_iotlb_pages { unsigned long dfn_l = dfn_x(dfn); - ASSERT(page_count && !dfn_eq(dfn, INVALID_DFN)); - ASSERT(flush_flags); + if ( !(flush_flags & IOMMU_FLUSHF_all) ) + { + ASSERT(page_count && !dfn_eq(dfn, INVALID_DFN)); + ASSERT(flush_flags); + } /* Unless a PTE was modified, no flush is required */ if ( !(flush_flags & IOMMU_FLUSHF_modified) ) return 0; - /* If the range wraps then just flush everything */ - if ( dfn_l + page_count < dfn_l ) + /* If so requested or if the range wraps then just flush everything. */ + if ( (flush_flags & IOMMU_FLUSHF_all) || dfn_l + page_count < dfn_l ) { amd_iommu_flush_all_pages(d); return 0; @@ -572,13 +575,6 @@ int cf_check amd_iommu_flush_iotlb_pages return 0; } - -int cf_check amd_iommu_flush_iotlb_all(struct domain *d) -{ - amd_iommu_flush_all_pages(d); - - return 0; -} int amd_iommu_reserve_domain_unity_map(struct domain *d, const struct ivrs_unity_map *map, --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -759,7 +759,6 @@ static const struct iommu_ops __initcons .map_page = amd_iommu_map_page, .unmap_page = amd_iommu_unmap_page, .iotlb_flush = amd_iommu_flush_iotlb_pages, - .iotlb_flush_all = amd_iommu_flush_iotlb_all, .reassign_device = reassign_device, .get_device_group_id = amd_iommu_group_id, .enable_x2apic = iov_enable_xt, --- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c +++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c @@ -1000,13 +1000,19 @@ out: } /* Xen IOMMU ops */ -static int __must_check ipmmu_iotlb_flush_all(struct domain *d) +static int __must_check ipmmu_iotlb_flush(struct domain *d, dfn_t dfn, + unsigned long page_count, + unsigned int flush_flags) { struct ipmmu_vmsa_xen_domain *xen_domain = dom_iommu(d)->arch.priv; + ASSERT(flush_flags); + if ( !xen_domain || !xen_domain->root_domain ) return 0; + /* The hardware doesn't support selective TLB flush. */ + spin_lock(&xen_domain->lock); ipmmu_tlb_invalidate(xen_domain->root_domain); spin_unlock(&xen_domain->lock); @@ -1014,16 +1020,6 @@ static int __must_check ipmmu_iotlb_flus return 0; } -static int __must_check ipmmu_iotlb_flush(struct domain *d, dfn_t dfn, - unsigned long page_count, - unsigned int flush_flags) -{ - ASSERT(flush_flags); - - /* The hardware doesn't support selective TLB flush. */ - return ipmmu_iotlb_flush_all(d); -} - static struct ipmmu_vmsa_domain *ipmmu_get_cache_domain(struct domain *d, struct device *dev) { @@ -1360,7 +1356,6 @@ static const struct iommu_ops ipmmu_iomm .hwdom_init = arch_iommu_hwdom_init, .teardown = ipmmu_iommu_domain_teardown, .iotlb_flush = ipmmu_iotlb_flush, - .iotlb_flush_all = ipmmu_iotlb_flush_all, .assign_device = ipmmu_assign_device, .reassign_device = ipmmu_reassign_device, .map_page = arm_iommu_map_page, --- a/xen/drivers/passthrough/arm/smmu.c +++ b/xen/drivers/passthrough/arm/smmu.c @@ -2649,11 +2649,17 @@ static int force_stage = 2; */ static u32 platform_features = ARM_SMMU_FEAT_COHERENT_WALK; -static int __must_check arm_smmu_iotlb_flush_all(struct domain *d) +static int __must_check arm_smmu_iotlb_flush(struct domain *d, dfn_t dfn, + unsigned long page_count, + unsigned int flush_flags) { struct arm_smmu_xen_domain *smmu_domain = dom_iommu(d)->arch.priv; struct iommu_domain *cfg; + ASSERT(flush_flags); + + /* ARM SMMU v1 doesn't have flush by VMA and VMID */ + spin_lock(&smmu_domain->lock); list_for_each_entry(cfg, &smmu_domain->contexts, list) { /* @@ -2670,16 +2676,6 @@ static int __must_check arm_smmu_iotlb_f return 0; } -static int __must_check arm_smmu_iotlb_flush(struct domain *d, dfn_t dfn, - unsigned long page_count, - unsigned int flush_flags) -{ - ASSERT(flush_flags); - - /* ARM SMMU v1 doesn't have flush by VMA and VMID */ - return arm_smmu_iotlb_flush_all(d); -} - static struct iommu_domain *arm_smmu_get_domain(struct domain *d, struct device *dev) { @@ -2864,7 +2860,6 @@ static const struct iommu_ops arm_smmu_i .add_device = arm_smmu_dt_add_device_generic, .teardown = arm_smmu_iommu_domain_teardown, .iotlb_flush = arm_smmu_iotlb_flush, - .iotlb_flush_all = arm_smmu_iotlb_flush_all, .assign_device = arm_smmu_assign_dev, .reassign_device = arm_smmu_reassign_dev, .map_page = arm_iommu_map_page, --- a/xen/drivers/passthrough/arm/smmu-v3.c +++ b/xen/drivers/passthrough/arm/smmu-v3.c @@ -3416,7 +3416,6 @@ static const struct iommu_ops arm_smmu_i .hwdom_init = arch_iommu_hwdom_init, .teardown = arm_smmu_iommu_xen_domain_teardown, .iotlb_flush = arm_smmu_iotlb_flush, - .iotlb_flush_all = arm_smmu_iotlb_flush_all, .assign_device = arm_smmu_assign_dev, .reassign_device = arm_smmu_reassign_dev, .map_page = arm_iommu_map_page, --- a/xen/drivers/passthrough/iommu.c +++ b/xen/drivers/passthrough/iommu.c @@ -478,15 +478,12 @@ int iommu_iotlb_flush_all(struct domain const struct domain_iommu *hd = dom_iommu(d); int rc; - if ( !is_iommu_enabled(d) || !hd->platform_ops->iotlb_flush_all || + if ( !is_iommu_enabled(d) || !hd->platform_ops->iotlb_flush || !flush_flags ) return 0; - /* - * The operation does a full flush so we don't need to pass the - * flush_flags in. - */ - rc = iommu_call(hd->platform_ops, iotlb_flush_all, d); + rc = iommu_call(hd->platform_ops, iotlb_flush, d, INVALID_DFN, 0, + flush_flags | IOMMU_FLUSHF_all); if ( unlikely(rc) ) { if ( !d->is_shutting_down && printk_ratelimit() ) --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -772,18 +772,21 @@ static int __must_check cf_check iommu_f struct domain *d, dfn_t dfn, unsigned long page_count, unsigned int flush_flags) { - ASSERT(page_count && !dfn_eq(dfn, INVALID_DFN)); - ASSERT(flush_flags); + if ( flush_flags & IOMMU_FLUSHF_all ) + { + dfn = INVALID_DFN; + page_count = 0; + } + else + { + ASSERT(page_count && !dfn_eq(dfn, INVALID_DFN)); + ASSERT(flush_flags); + } return iommu_flush_iotlb(d, dfn, flush_flags & IOMMU_FLUSHF_modified, page_count); } -static int __must_check cf_check iommu_flush_iotlb_all(struct domain *d) -{ - return iommu_flush_iotlb(d, INVALID_DFN, 0, 0); -} - static void queue_free_pt(struct domain_iommu *hd, mfn_t mfn, unsigned int level) { if ( level > 1 ) @@ -3185,7 +3188,6 @@ static const struct iommu_ops __initcons .resume = vtd_resume, .crash_shutdown = vtd_crash_shutdown, .iotlb_flush = iommu_flush_iotlb_pages, - .iotlb_flush_all = iommu_flush_iotlb_all, .get_reserved_device_memory = intel_iommu_get_reserved_device_memory, .dump_page_tables = vtd_dump_page_tables, }; --- a/xen/include/xen/iommu.h +++ b/xen/include/xen/iommu.h @@ -147,9 +147,11 @@ enum { _IOMMU_FLUSHF_added, _IOMMU_FLUSHF_modified, + _IOMMU_FLUSHF_all, }; #define IOMMU_FLUSHF_added (1u << _IOMMU_FLUSHF_added) #define IOMMU_FLUSHF_modified (1u << _IOMMU_FLUSHF_modified) +#define IOMMU_FLUSHF_all (1u << _IOMMU_FLUSHF_all) int __must_check iommu_map(struct domain *d, dfn_t dfn, mfn_t mfn, unsigned long page_count, unsigned int flags, @@ -281,7 +283,6 @@ struct iommu_ops { int __must_check (*iotlb_flush)(struct domain *d, dfn_t dfn, unsigned long page_count, unsigned int flush_flags); - int __must_check (*iotlb_flush_all)(struct domain *d); int (*get_reserved_device_memory)(iommu_grdm_t *, void *); void (*dump_page_tables)(struct domain *d); From patchwork Mon Apr 25 08:40:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825403 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E184BC433EF for ; Mon, 25 Apr 2022 08:41:13 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312552.529851 (Exim 4.92) (envelope-from ) id 1niuHG-0003kh-7T; Mon, 25 Apr 2022 08:41:02 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312552.529851; Mon, 25 Apr 2022 08:41:02 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuHG-0003ka-3l; Mon, 25 Apr 2022 08:41:02 +0000 Received: by outflank-mailman (input) for mailman id 312552; Mon, 25 Apr 2022 08:41:00 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuHE-0003e7-3Z for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:41:00 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.109.102]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 6dee6dec-c473-11ec-a405-831a346695d4; Mon, 25 Apr 2022 10:40:59 +0200 (CEST) Received: from EUR03-AM5-obe.outbound.protection.outlook.com (mail-am5eur03lp2050.outbound.protection.outlook.com [104.47.8.50]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-25-qKVd713AP8u-SNmZDmSHuw-1; Mon, 25 Apr 2022 10:40:56 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by VI1PR04MB3024.eurprd04.prod.outlook.com (2603:10a6:802:d::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.18; Mon, 25 Apr 2022 08:40:54 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:40:54 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 6dee6dec-c473-11ec-a405-831a346695d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650876058; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E9yc1zY4O0b+hMzg6Cm9cySZnRsS6KKgNE5TEN/QJ18=; b=CikhCdS8ajxONqf5C6XjXIWuO+bWXh8QViRTJh9R/n/MCwINRvaH949NCWwLHAfpt9QK8y 6m74sz4zgrbINCWPREuSKQC1WMnmE4zOXwzGZ2EmoSKDQAnYxVuuJPRSm1dbHMMxROcdQn uz3uLaviBln6h305CGUjSZtRHEySzxk= X-MC-Unique: qKVd713AP8u-SNmZDmSHuw-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QZ3B+4Gko45DPdGlbaO0jJGgyVkv98b2YSclDN5loclZQ9blMzSMKSWk+pym9uK8qp/qf9VJf1Jd0M+VBbhL1ey62j89w8Ncr3x+ZcPlkqCduVz7lLw+3l9ubMeRgV1qOTaBeHCs7JUp/U3EqWp5Nhaur47qBE0JLDD/BlZuyZAcujlikoQU5CVtUSf5xMEC2/hqs4DUBy+NenOtVw5RGgaAHELwVzXedxp/cxxf2JycTZgMN930vfhjJg2+rknsAYFCmR/L+iIEX4Mw2ICO6GgvcP/aw1nBuvs8feFXkG8bS0EqMmbg5vqvYpvMrboSvH9wCm++a2USIpd0mA1HNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=E9yc1zY4O0b+hMzg6Cm9cySZnRsS6KKgNE5TEN/QJ18=; b=Og3m297lwLlYjVavG2aYsKXYvsoX1W1AAH2lUQ9aqS9qz7I4pQKXqsKe3u/V6lGcoAJM7aamhP0GzibROlqHK7uO9SHkejQzIf/OKNOotGYX969gHGPcwQDu/g/iZuDpR4I2Uupu9x3El0ig4BSZyLG9S5RRPIvNX4a25aSOXcGiZ+ii6acBhOKdCA7lR0kWnSz03xuv5cY0c0kVz78LKR0IIIPZ+TCruWTkxvyK5QTCnyRiOfMniqKf2sBFtnobGlLfUhXR2moEyT/v/g/PtbRxf4icqgudNbYjI5YwEUsEUoKwL+D7UCy3qcVEMyKZREtzbSz2SkW7CxIQPH0ZCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <9d073a05-0c7d-4989-7a38-93cd5b01d071@suse.com> Date: Mon, 25 Apr 2022 10:40:55 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 13/21] IOMMU/x86: prefill newly allocate page tables Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu References: In-Reply-To: X-ClientProxiedBy: FR2P281CA0009.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a::19) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 555bda5b-d73c-403f-75b0-08da26974fb0 X-MS-TrafficTypeDiagnostic: VI1PR04MB3024:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tfcakujOkl5oXtmD+8ssxSE6hPhUjc+RpA2hOuPX6ogxJJTPSHZwCoTb3SZE4gv75gsmVA3ElV9E6nimjzLI4NI8mwCMAG4ucEPKxbKo6iLN28QeqAQyGFRdHTDPXFJMpqUiphV1HRwZ+S3nBZG3bR5lRdw2qJ/oj2VVKU1rBhL90RQo9ndzo7VtW1BJ29J9p47xkcDhwTlSLcaS5syZSVj7rH5vSDmqpbDIOp8BPCwh3DuylogMc+BAK1UdsN+nIa4p38qZyiFh0VAP2YQbntsOdkbB13/ipgiL2ULea72R+gNM0yIzX/KBp0OqL7d6kZSn5+UH65YDOSXcaFhCwgSGyaIWrFCxKfBYKyAX2xnatV5nAOGQ9wsL7+2Q3eMoDgUIlKGmpU+pjzeSjSOKa7rcOfPVwvIU56Zg1A0tOTNPfF6wLHCcVVEcDMsJw2PX4sCOl+KuAqW0dAjpDwu6N5UWKYxScJI+LbFX2PKWYlnAhkECyqbl26adq7hDw8272VwfPF4uyLu3A3m2n+6lpGy3pj6v1v+MDiMaPAPt3Rn3lCe/HrAHzoyxcREBq8bTIfCzSegkb5N/pFSuUlTfwN+6USWx9Ipv5oyywDJmUWVeCMn8KbEnXkpGCUX1iP8K0FOxz9HXJLbvCdgaDsNomwbQkJBDz5a7GVyjgRlfYnKhJeZSsCD5hvdTTZ0v2RLoyY7/y5DPrzJZc82HWijGz2Ax+DHTv/52Z83AgiKQoXM= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(5660300002)(86362001)(38100700002)(31696002)(508600001)(186003)(2616005)(83380400001)(8936002)(6486002)(26005)(6506007)(66946007)(6512007)(66556008)(36756003)(2906002)(6916009)(66476007)(31686004)(54906003)(4326008)(8676002)(316002)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?B2c/LGQhFeH0S//9dR5uvvxeZpkZ?= =?utf-8?q?wzKaK2gktfIK4X/tcLVk5Gp2U65S4Ii4LPs2MtfEekuTeFMTyJV5pcxXZd2qbgt3J?= =?utf-8?q?uUIMXQ3DG4ucGPo8/6cUrfeQfDs2ZzPzQSuC3h7uU5G3hXMuDzqEE41SqKDy8cmwN?= =?utf-8?q?qWHgyNM6GiLQbjTj9p4cdVET6EfZHDqr1eYgUsY3ay7keT+wcYO6jLyBmJrJRPh9n?= =?utf-8?q?AWrJQ0kxYwAtE8xvPBQwqcvb4pP3L98CjnyfjvFV98XhRNsN7VUD9Cno4cJeiVd0r?= =?utf-8?q?2mNN/rc29xO3hfBBLiFdIT8w7ljW+PhdEQhXaGt2GnGYgDVDmKo1qlVW97N3dYNtk?= =?utf-8?q?jL8CLqeqSc6EReiDPU3mjzZx1dqDhF03PWE4dypuPTkq2kI4iaWvym1ZHoRg3Vw1Z?= =?utf-8?q?vVQa2DkgblyRQmc20hVoFeTGtzMQyteqBDXm06BagYhQ+nKr8RrymNPq9SVI5ckNW?= =?utf-8?q?nsHQf+d5k77jCGOGioKlMeqd0btmmhhTMaXLTeTAlmbiQO4Q5umG/Ug6lPSbxnVCq?= =?utf-8?q?TMFjB/ocwuqHlg8FOubepQl97PRNovG1eAewshVEw9ezse/fkwGMvdA/yEPTwaOR6?= =?utf-8?q?7ZthzJtY9/UkM0Smb3S8Cwq+B4n1mCiBRgc9E+2kgThLEFMrWOpO/jpzXyaWRX9Jl?= =?utf-8?q?3AOG9GI/rebfqWhZnJ6Bk7LcMBczpApBAhfvpf6CDbF+Xy9Q5U2P5UeS1ahG3a2Eh?= =?utf-8?q?FlOVTkGQExZyaSreIeTsw/anTqQg0qoI4hzVGg9Tfy1+dsNUwFBHca4CYXhoEufCB?= =?utf-8?q?90jkwL2x3WF3QXUa3t62xKaTO0mIH1+5rj3Rxr+30zeiAa/WlLsfMWeaLThSFeGlz?= =?utf-8?q?afli/yS+ER1Wj3vSYGeCD+HsqoLc2ZHpB6JFbymAf1TespUUh2CPsHkphRZDQYVYA?= =?utf-8?q?6/bCjEXBwiurNzvIlE9PFZl9SXN9qAOwS6zxml27spYm+w7j5FBtXKuO5B7W93PnS?= =?utf-8?q?ECpOJAnOkfi0nFJS3Iz7hzLjML097nZkaCaagfya0uKpwDnrUSG4sCZ8v6LTxsK/1?= =?utf-8?q?76PyesQYyzB0qhFzrhf7crdaW7Dan2h/7FAQH+OsNM9g5lxIc+YRI0xqVwL+GbF+N?= =?utf-8?q?yeGt9hVe0aQcbCiaHWGjrV3pWGB7HmIINW9G71VrRmMVdjOD8McA5b2EiLNDmpegf?= =?utf-8?q?ALWlB2ZHaSqgCarGVIc/DvwU3ZfWzGJe/QovbbKdG+HPNXs0Wuycn59Kif76e9yiA?= =?utf-8?q?CUZyDKcht7EJyk5vWyOE/IOQOuPJtIYwQURWMDo1iHtmirKWdnm3vrH5tMThbdz1c?= =?utf-8?q?MVYN/EvPO6HXgW1Q+l8o2UHlauJ/wtNTdpJRhn13oris83Lum0ULqd8xRT++3OvGy?= =?utf-8?q?LCtCSFAcQlDuw3KjwrmjvLqyw5DhpqDVGCpTylU5whXhlNTGdkTEnvUkTEntMtpNE?= =?utf-8?q?ihKYGUmljOVHozIjZOLJk/pAyI4d6NOnVT9lBDMsQs+Rl0+49GSWrAfX/vHHAveor?= =?utf-8?q?pebOUorV6UjgULFvD8Ksffk7TZYLiF8S+9tvgJ+B8ucvWyH4/TXlTOIzkvNPVnuRp?= =?utf-8?q?U3iHEjqZjDvODYOxSn3hmaYLXlJ+sblyi1Y4/6YiP4JcuRdrvbF63hgggjpBH8+VJ?= =?utf-8?q?wTJe8XmSJAvjpN2Es2ZIO3IWhVNkBNaJl3vz3GNUhphmAVigngocKDFSimruDyDbX?= =?utf-8?q?KvlhAtyqmxjU1P5Ev5hH0OdlTq93SoOw=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 555bda5b-d73c-403f-75b0-08da26974fb0 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:40:54.7102 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5cWnBrVPL2oHVweV5swGEksjW6qJ/A9bBdlKBQ0B0qI2xYw/JFmwTOqtWLwSQK8rBDD5Ld9eotQe0+udFh0dXA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB3024 Page tables are used for two purposes after allocation: They either start out all empty, or they get filled to replace a superpage. Subsequently, to replace all empty or fully contiguous page tables, contiguous sub-regions will be recorded within individual page tables. Install the initial set of markers immediately after allocation. Make sure to retain these markers when further populating a page table in preparation for it to replace a superpage. The markers are simply 4-bit fields holding the order value of contiguous entries. To demonstrate this, if a page table had just 16 entries, this would be the initial (fully contiguous) set of markers: index 0 1 2 3 4 5 6 7 8 9 A B C D E F marker 4 0 1 0 2 0 1 0 3 0 1 0 2 0 1 0 "Contiguous" here means not only present entries with successively increasing MFNs, each one suitably aligned for its slot, but also a respective number of all non-present entries. Signed-off-by: Jan Beulich Reviewed-by: Kevin Tian --- An alternative to the ASSERT()s added to set_iommu_ptes_present() would be to make the function less general-purpose; it's used in a single place only after all (i.e. it might as well be folded into its only caller). While in VT-d's comment ahead of struct dma_pte I'm adjusting the description of the high bits, I'd like to note that the description of some of the lower bits isn't correct either. Yet I don't think adjusting that belongs here. --- v4: Add another comment referring to pt-contig-markers.h. Re-base. v3: Add comments. Re-base. v2: New. --- a/xen/arch/x86/include/asm/iommu.h +++ b/xen/arch/x86/include/asm/iommu.h @@ -146,7 +146,8 @@ void iommu_free_domid(domid_t domid, uns int __must_check iommu_free_pgtables(struct domain *d); struct domain_iommu; -struct page_info *__must_check iommu_alloc_pgtable(struct domain_iommu *hd); +struct page_info *__must_check iommu_alloc_pgtable(struct domain_iommu *hd, + uint64_t contig_mask); void iommu_queue_free_pgtable(struct domain_iommu *hd, struct page_info *pg); #endif /* !__ARCH_X86_IOMMU_H__ */ --- a/xen/drivers/passthrough/amd/iommu-defs.h +++ b/xen/drivers/passthrough/amd/iommu-defs.h @@ -446,11 +446,13 @@ union amd_iommu_x2apic_control { #define IOMMU_PAGE_TABLE_U32_PER_ENTRY (IOMMU_PAGE_TABLE_ENTRY_SIZE / 4) #define IOMMU_PAGE_TABLE_ALIGNMENT 4096 +#define IOMMU_PTE_CONTIG_MASK 0x1e /* The ign0 field below. */ + union amd_iommu_pte { uint64_t raw; struct { bool pr:1; - unsigned int ign0:4; + unsigned int ign0:4; /* Covered by IOMMU_PTE_CONTIG_MASK. */ bool a:1; bool d:1; unsigned int ign1:2; --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -115,7 +115,19 @@ static void set_iommu_ptes_present(unsig while ( nr_ptes-- ) { - set_iommu_pde_present(pde, next_mfn, 0, iw, ir); + ASSERT(!pde->next_level); + ASSERT(!pde->u); + + if ( pde > table ) + ASSERT(pde->ign0 == find_first_set_bit(pde - table)); + else + ASSERT(pde->ign0 == PAGE_SHIFT - 3); + + pde->iw = iw; + pde->ir = ir; + pde->fc = true; /* See set_iommu_pde_present(). */ + pde->mfn = next_mfn; + pde->pr = true; ++pde; next_mfn += page_sz; @@ -295,7 +307,7 @@ static int iommu_pde_from_dfn(struct dom mfn = next_table_mfn; /* allocate lower level page table */ - table = iommu_alloc_pgtable(hd); + table = iommu_alloc_pgtable(hd, IOMMU_PTE_CONTIG_MASK); if ( table == NULL ) { AMD_IOMMU_ERROR("cannot allocate I/O page table\n"); @@ -325,7 +337,7 @@ static int iommu_pde_from_dfn(struct dom if ( next_table_mfn == 0 ) { - table = iommu_alloc_pgtable(hd); + table = iommu_alloc_pgtable(hd, IOMMU_PTE_CONTIG_MASK); if ( table == NULL ) { AMD_IOMMU_ERROR("cannot allocate I/O page table\n"); @@ -717,7 +729,7 @@ static int fill_qpt(union amd_iommu_pte * page table pages, and the resulting allocations are always * zeroed. */ - pgs[level] = iommu_alloc_pgtable(hd); + pgs[level] = iommu_alloc_pgtable(hd, 0); if ( !pgs[level] ) { rc = -ENOMEM; @@ -775,7 +787,7 @@ int cf_check amd_iommu_quarantine_init(s return 0; } - pdev->arch.amd.root_table = iommu_alloc_pgtable(hd); + pdev->arch.amd.root_table = iommu_alloc_pgtable(hd, 0); if ( !pdev->arch.amd.root_table ) return -ENOMEM; --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c @@ -342,7 +342,7 @@ int amd_iommu_alloc_root(struct domain * if ( unlikely(!hd->arch.amd.root_table) && d != dom_io ) { - hd->arch.amd.root_table = iommu_alloc_pgtable(hd); + hd->arch.amd.root_table = iommu_alloc_pgtable(hd, 0); if ( !hd->arch.amd.root_table ) return -ENOMEM; } --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -334,7 +334,7 @@ static uint64_t addr_to_dma_page_maddr(s goto out; pte_maddr = level; - if ( !(pg = iommu_alloc_pgtable(hd)) ) + if ( !(pg = iommu_alloc_pgtable(hd, 0)) ) goto out; hd->arch.vtd.pgd_maddr = page_to_maddr(pg); @@ -376,7 +376,7 @@ static uint64_t addr_to_dma_page_maddr(s } pte_maddr = level - 1; - pg = iommu_alloc_pgtable(hd); + pg = iommu_alloc_pgtable(hd, DMA_PTE_CONTIG_MASK); if ( !pg ) break; @@ -388,12 +388,13 @@ static uint64_t addr_to_dma_page_maddr(s struct dma_pte *split = map_vtd_domain_page(pte_maddr); unsigned long inc = 1UL << level_to_offset_bits(level - 1); - split[0].val = pte->val; + split[0].val |= pte->val & ~DMA_PTE_CONTIG_MASK; if ( inc == PAGE_SIZE ) split[0].val &= ~DMA_PTE_SP; for ( offset = 1; offset < PTE_NUM; ++offset ) - split[offset].val = split[offset - 1].val + inc; + split[offset].val |= + (split[offset - 1].val & ~DMA_PTE_CONTIG_MASK) + inc; iommu_sync_cache(split, PAGE_SIZE); unmap_vtd_domain_page(split); @@ -2173,7 +2174,7 @@ static int __must_check cf_check intel_i if ( iommu_snoop ) dma_set_pte_snp(new); - if ( old.val == new.val ) + if ( !((old.val ^ new.val) & ~DMA_PTE_CONTIG_MASK) ) { spin_unlock(&hd->arch.mapping_lock); unmap_vtd_domain_page(page); @@ -3052,7 +3053,7 @@ static int fill_qpt(struct dma_pte *this * page table pages, and the resulting allocations are always * zeroed. */ - pgs[level] = iommu_alloc_pgtable(hd); + pgs[level] = iommu_alloc_pgtable(hd, 0); if ( !pgs[level] ) { rc = -ENOMEM; @@ -3109,7 +3110,7 @@ static int cf_check intel_iommu_quaranti if ( !drhd ) return -ENODEV; - pg = iommu_alloc_pgtable(hd); + pg = iommu_alloc_pgtable(hd, 0); if ( !pg ) return -ENOMEM; --- a/xen/drivers/passthrough/vtd/iommu.h +++ b/xen/drivers/passthrough/vtd/iommu.h @@ -253,7 +253,10 @@ struct context_entry { * 2-6: reserved * 7: super page * 8-11: available - * 12-63: Host physcial address + * 12-51: Host physcial address + * 52-61: available (52-55 used for DMA_PTE_CONTIG_MASK) + * 62: reserved + * 63: available */ struct dma_pte { u64 val; @@ -263,6 +266,7 @@ struct dma_pte { #define DMA_PTE_PROT (DMA_PTE_READ | DMA_PTE_WRITE) #define DMA_PTE_SP (1 << 7) #define DMA_PTE_SNP (1 << 11) +#define DMA_PTE_CONTIG_MASK (0xfull << PADDR_BITS) #define dma_clear_pte(p) do {(p).val = 0;} while(0) #define dma_set_pte_readable(p) do {(p).val |= DMA_PTE_READ;} while(0) #define dma_set_pte_writable(p) do {(p).val |= DMA_PTE_WRITE;} while(0) @@ -276,7 +280,7 @@ struct dma_pte { #define dma_pte_write(p) (dma_pte_prot(p) & DMA_PTE_WRITE) #define dma_pte_addr(p) ((p).val & PADDR_MASK & PAGE_MASK_4K) #define dma_set_pte_addr(p, addr) do {\ - (p).val |= ((addr) & PAGE_MASK_4K); } while (0) + (p).val |= ((addr) & PADDR_MASK & PAGE_MASK_4K); } while (0) #define dma_pte_present(p) (((p).val & DMA_PTE_PROT) != 0) #define dma_pte_superpage(p) (((p).val & DMA_PTE_SP) != 0) --- a/xen/drivers/passthrough/x86/iommu.c +++ b/xen/drivers/passthrough/x86/iommu.c @@ -522,11 +522,12 @@ int iommu_free_pgtables(struct domain *d return 0; } -struct page_info *iommu_alloc_pgtable(struct domain_iommu *hd) +struct page_info *iommu_alloc_pgtable(struct domain_iommu *hd, + uint64_t contig_mask) { unsigned int memflags = 0; struct page_info *pg; - void *p; + uint64_t *p; #ifdef CONFIG_NUMA if ( hd->node != NUMA_NO_NODE ) @@ -538,7 +539,29 @@ struct page_info *iommu_alloc_pgtable(st return NULL; p = __map_domain_page(pg); - clear_page(p); + + if ( contig_mask ) + { + /* See pt-contig-markers.h for a description of the marker scheme. */ + unsigned int i, shift = find_first_set_bit(contig_mask); + + ASSERT(((PAGE_SHIFT - 3) & (contig_mask >> shift)) == PAGE_SHIFT - 3); + + p[0] = (PAGE_SHIFT - 3ull) << shift; + p[1] = 0; + p[2] = 1ull << shift; + p[3] = 0; + + for ( i = 4; i < PAGE_SIZE / 8; i += 4 ) + { + p[i + 0] = (find_first_set_bit(i) + 0ull) << shift; + p[i + 1] = 0; + p[i + 2] = 1ull << shift; + p[i + 3] = 0; + } + } + else + clear_page(p); iommu_sync_cache(p, PAGE_SIZE); From patchwork Mon Apr 25 08:41:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825404 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 71D3DC433EF for ; Mon, 25 Apr 2022 08:41:40 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312558.529863 (Exim 4.92) (envelope-from ) id 1niuHg-0004Kn-Ig; Mon, 25 Apr 2022 08:41:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312558.529863; Mon, 25 Apr 2022 08:41:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuHg-0004Kc-Fb; Mon, 25 Apr 2022 08:41:28 +0000 Received: by outflank-mailman (input) for mailman id 312558; Mon, 25 Apr 2022 08:41:27 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuHf-0003e7-01 for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:41:27 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 7e336170-c473-11ec-a405-831a346695d4; Mon, 25 Apr 2022 10:41:26 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2053.outbound.protection.outlook.com [104.47.12.53]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-10-ZktRNjkqPfWbQ1YRYp5RlQ-2; Mon, 25 Apr 2022 10:41:23 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by DB8PR04MB6891.eurprd04.prod.outlook.com (2603:10a6:10:112::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:41:22 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:41:22 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 7e336170-c473-11ec-a405-831a346695d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650876085; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R8DiCPGmZfdIitLCnmZ7S3mCQH5fBWGDgn8DzVJxrZM=; b=cG5HNqJ85x5KfaghM7F4h1epa3hfnbhsd4u+khx3ZB5JZ6+D84+6O0jx9bm54GQ/P/qxsf xmgl3PpI0YXapJV5ei/DTTg1r4zNWazfHFzbIBCHjMrLX0gUPKSHcFhGjDx445mYMNlA0f /UuAgjHrNja+egtScCbMNOHYobjaz7o= X-MC-Unique: ZktRNjkqPfWbQ1YRYp5RlQ-2 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=coDZ6RpsG602xyEM7tLBo7nhvZW3Ko6pEtf1UXXXmMOcujikRDFcu7Aoh0BjUypvHQi/nVS7pLvgb2b1Pm8OtxoEWItdP42EXLRrJIGnos2xgNrZqGLR3GnEtGlOTIwSIqPi3yH7JO6l9DErytmtoOfuzsgFuu+raBzNrawSWOtDrgyDggb1QgMAufpqM6jrWfkR9WbSFZnkpZEZldkwkDzIswXnNTQ212PkNuIdxS3pOTt/P3dSTdXZCxIp+dwwc8leld9s2v2YTIIETVllWga08w398fri79N9RKFLQTsVbIPqc+JkR4n1a8p4ShqJ2SKZp/Dp4IqhYTC3Kc9//w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=R8DiCPGmZfdIitLCnmZ7S3mCQH5fBWGDgn8DzVJxrZM=; b=M3o93GbzCeH6THWaLvpzP3/l2TovQH+tocAd70n3h5+tkyqjDbH2zQ2VytALlubRohBExvTfP2o0tyIOfFGkzTk5XCWBYOzHs6gspcDoMvgpEKz9qAVvEeoMZYq9eYePDgStbhGeCEB+aSsj7OOWo6r2eI4VwX+nvMg4w5feLEQdM8Y/Yh/nDvmCvBpqPG6/ibaUOlXuPd+yKNCRbIOuaPJIpP+gBhz/RbeMSZx/fS7dbQtqUd2Z7mgG0t8UNJ2nVGLSDY3f1ZwEd4yyddatpMNXbbwdawoU9ngHJWuhwKFeifRKKd7bs2x6xZeS7GilOm0p8TbSPXWZV4tSq2fdlQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: Date: Mon, 25 Apr 2022 10:41:23 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 14/21] x86: introduce helper for recording degree of contiguity in page tables Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu References: In-Reply-To: X-ClientProxiedBy: AM6P194CA0043.EURP194.PROD.OUTLOOK.COM (2603:10a6:209:84::20) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 16e2feb1-0837-4112-01c3-08da2697606e X-MS-TrafficTypeDiagnostic: DB8PR04MB6891:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zXiGfoVkqc9pAj1Wobos8VX3wDvPd7bajFrbhXS3nJqZbhuq9LGg0a2vBL7nVq/vbjtA9B8Dl1Sp8tXqNSHNe0gFl7BOt3cbop5pV5b3WbYvg+q//Fb2sabk3ZMLTIMbAOQTuwNqxIjwxHUxuF6uzF/KbOdn9jLj3I2Nwy3xycN9dC3WjQWmI9bJ6S1QgVEXjEEzTqmmbHJzceWMy73CwuoNftvM303NAn1l7dGPaKIeLfXLMiG6qqhVv/4vlq6GhX7l2AHCdQFc9Jlk/MamVjoORqqYPWPFPCSwxX0jfKA74gYbmv0gb5uSldYWL2Tlw1AzZeK1RkqvwW+Fg+tYL+1ONzoHPIjKOd/jYKDLYjxXAmz3gdv0kvzKoJYG+Rl4STnUzrpaWEqHH9+OVz73OU+Za4NjuAyTkQ1R/tkIKF0BBnuCIhM8zkKecc4o6QFO0BL7Pn5FSabySHh+h9+MMaTz9JvAUbvffb+xNDIMtPGCzc/vbJd9N5cNXN+PTkh2gnb14v4FzpkkzYVDHphSMMbdHGETC3OIs/g366s6WZKBjhvKQ5QYsAL6ZxJzLPTkb8xOByeFtNORmF8IYLY2PnxY9qGxF2EJGvo/HQGmfyUZympB1UM4f24EeOS8LcOCnuMnvet88tSs9FP6NbJju4AiMywXJPR75LkHcJmojkdDmtZ0ma3NXBgAwcw86EHt/OsuhltnbDkUHNM1Zkuz6q3aodmk7tJdUMJHqiRNf2A= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(66556008)(26005)(66946007)(6512007)(38100700002)(508600001)(66476007)(316002)(6506007)(83380400001)(54906003)(6486002)(186003)(6916009)(86362001)(31696002)(2616005)(2906002)(5660300002)(31686004)(8936002)(36756003)(4326008)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?CNIh2Ohvpk3rJSkKbkyQkcNbzBXq?= =?utf-8?q?5dKn5VbOVUKabEjUz3LTuhsy+BhrENIRBgGqWFz3k7f3ZX1WJdiMsIIgpSseVW9Eh?= =?utf-8?q?SE+Abt4nBBL4UmoReKvnWfGVtg+EMTW9+aVlV5UldxQEVur61ApOOQno8OjnuMSOA?= =?utf-8?q?YH310KpQ8uisTElPRFurGVE/sQjUlXS9RhP2n1U3omOGAaquKQB15K8ye2bnNxM5V?= =?utf-8?q?zYCoaS2Fum0dQmDnwr1bIILoI7bRch2X9+BmcB8clqp3DltsOaaU5kP8G4TVvuN9e?= =?utf-8?q?1xoSDplHa9JlNj7loNHK+u3Bf0T5strcVnW1Far2kPCkL9bQFbo30bn5oIufhY/BS?= =?utf-8?q?5no7lLBE1s3U0bvzJJtEP1LUID3Aw/3Qb2tvrckCA3LBDtRnIAOxbJzjn9C7VF+a5?= =?utf-8?q?9c5DBPt6WEnkMTCv4x71AFIBz1Si/r9fs9rJA+xRU5NOqCfDEFWLxzBbtMrVSSy4R?= =?utf-8?q?xB4OT5gaGoGi31rEGULwCwbWFiFfQf/RcKUOd4tow8TsAFLM0CPvlQpsPvhRuXwYu?= =?utf-8?q?jJvr5qZploQb5I4g1aeaFxQdP4SD5DkOrksgVmFHnnK1NxXOuMfWMz4UawFw6SwwV?= =?utf-8?q?G9J0+9QC+FlSwCSm2a6ilBzjP8o9fi31HA90z5Pk1/BSzhZ9kFC25dWmfD7Ktk1MX?= =?utf-8?q?e4Q21uw6aCXV8y42665H5hTCjBsp5P99Ct4L6U34X1ayvuGfum5lsLEOr8FKIPGmp?= =?utf-8?q?+drTiwYT1HVoTNCMe6IrJaeFXiJ/2q2nryoLQz8MM2jV06eGipajuWFzxikfxw9HY?= =?utf-8?q?YRxY/XDDIUtVNfcJrXEwKAxbVuUnE6SUNphIGFwc/VEFNOJR3L3P2wQkJcqj+Yhs6?= =?utf-8?q?tVVLQqnkDQ3KKXaPIsZVDDitU5jt5vGHnY00j+ypJvqKas9OdFLowbmwQbPxEYq1S?= =?utf-8?q?X4Dl7BuYEP82jUVCIFTzab9nLm30T089CKnm24up3MHBZyw/t10cWewdGtK66pWcj?= =?utf-8?q?yfa6LM7P1mBTTcb7yzB61SqDQh4T8Zn8Aulmya3IXMVnoWP3i9DlAc1l+BoNxPsEF?= =?utf-8?q?wt8dj0LIikzxpEnISBAlRov3YnbuRGX8hK8+bWATzuCtCdIymu+VRRM1dto+wLV7C?= =?utf-8?q?Lj05fXhO2S06gpdr/Xugym0B7NlM40XIuOler1vrWqv63U/V1FO/aSLrptTdia+2U?= =?utf-8?q?W1OQt/f8wYtvUEcvpD1eva+7HkMqKIiGGDA86DodrKrNov6KVitBJSmzA6b32EInz?= =?utf-8?q?Xhg8V7zN2qCiYDrJmPCK6t+vayDjsKX3q9PdewjcMHnYwAnj5rW9wf0FBnTpP4Q5z?= =?utf-8?q?oewNsnnJeX8iGQAtJuZe76AQfFUVYTYScKy8e+P/h+FnUSzmxehqN6f3PrtDooJU+?= =?utf-8?q?OAHg6SVgIRTJPlQEJn6SPJKdS6a570cQ6ipPI2KMi4nK7wuCTWiibU2MQZR0RyzkF?= =?utf-8?q?OKSyTeBx6w0ZlFihLtukVxC9hc1rNmdCXGwr1UzUgzMfPhi6VkZnJXCnWtaQGKEZz?= =?utf-8?q?dbVQ4J/FGk0geQL3PaZf2HplFyQpyDtVi2GQ8gN0GmxjnOYJNV+LPyCNxFHlBoWXo?= =?utf-8?q?aNXP1DEhZCWXAMPo2ivzPf4KSvKX/acs5QWhSnlo6PCNh4P5r1h2PZL1XajdlK+/X?= =?utf-8?q?Cjt1/Ku4h+1P/Zyd/34+G65ofQyLE0VG4intUhdtp4MHsJZBhyB0diSWs00zWnxey?= =?utf-8?q?7D/Gyn4v7h9CiKebTCkHwoT97i5i6u5w=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 16e2feb1-0837-4112-01c3-08da2697606e X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:41:22.7491 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: e2I5vDMK/fGIZa0bT3yfQpKU/+SM2HnQVxQsKx/3m4fpNuRjVuhNv3vxrem1NWc5BF9rceepUr7X0ihfTYDvpg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR04MB6891 This is a re-usable helper (kind of a template) which gets introduced without users so that the individual subsequent patches introducing such users can get committed independently of one another. See the comment at the top of the new file. To demonstrate the effect, if a page table had just 16 entries, this would be the set of markers for a page table with fully contiguous mappings: index 0 1 2 3 4 5 6 7 8 9 A B C D E F marker 4 0 1 0 2 0 1 0 3 0 1 0 2 0 1 0 "Contiguous" here means not only present entries with successively increasing MFNs, each one suitably aligned for its slot, but also a respective number of all non-present entries. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- v3: Rename function and header. Introduce IS_CONTIG(). v2: New. --- /dev/null +++ b/xen/arch/x86/include/asm/pt-contig-markers.h @@ -0,0 +1,105 @@ +#ifndef __ASM_X86_PT_CONTIG_MARKERS_H +#define __ASM_X86_PT_CONTIG_MARKERS_H + +/* + * Short of having function templates in C, the function defined below is + * intended to be used by multiple parties interested in recording the + * degree of contiguity in mappings by a single page table. + * + * Scheme: Every entry records the order of contiguous successive entries, + * up to the maximum order covered by that entry (which is the number of + * clear low bits in its index, with entry 0 being the exception using + * the base-2 logarithm of the number of entries in a single page table). + * While a few entries need touching upon update, knowing whether the + * table is fully contiguous (and can hence be replaced by a higher level + * leaf entry) is then possible by simply looking at entry 0's marker. + * + * Prereqs: + * - CONTIG_MASK needs to be #define-d, to a value having at least 4 + * contiguous bits (ignored by hardware), before including this file, + * - page tables to be passed here need to be initialized with correct + * markers. + */ + +#include +#include +#include + +/* This is the same for all anticipated users, so doesn't need passing in. */ +#define CONTIG_LEVEL_SHIFT 9 +#define CONTIG_NR (1 << CONTIG_LEVEL_SHIFT) + +#define GET_MARKER(e) MASK_EXTR(e, CONTIG_MASK) +#define SET_MARKER(e, m) \ + ((void)((e) = ((e) & ~CONTIG_MASK) | MASK_INSR(m, CONTIG_MASK))) + +#define IS_CONTIG(kind, pt, i, idx, shift, b) \ + ((kind) == PTE_kind_leaf \ + ? (((pt)[i] ^ (pt)[idx]) & ~CONTIG_MASK) == (1ULL << ((b) + (shift))) \ + : !((pt)[i] & ~CONTIG_MASK)) + +enum PTE_kind { + PTE_kind_null, + PTE_kind_leaf, + PTE_kind_table, +}; + +static bool pt_update_contig_markers(uint64_t *pt, unsigned int idx, + unsigned int level, enum PTE_kind kind) +{ + unsigned int b, i = idx; + unsigned int shift = (level - 1) * CONTIG_LEVEL_SHIFT + PAGE_SHIFT; + + ASSERT(idx < CONTIG_NR); + ASSERT(!(pt[idx] & CONTIG_MASK)); + + /* Step 1: Reduce markers in lower numbered entries. */ + while ( i ) + { + b = find_first_set_bit(i); + i &= ~(1U << b); + if ( GET_MARKER(pt[i]) > b ) + SET_MARKER(pt[i], b); + } + + /* An intermediate table is never contiguous with anything. */ + if ( kind == PTE_kind_table ) + return false; + + /* + * Present entries need in-sync index and address to be a candidate + * for being contiguous: What we're after is whether ultimately the + * intermediate table can be replaced by a superpage. + */ + if ( kind != PTE_kind_null && + idx != ((pt[idx] >> shift) & (CONTIG_NR - 1)) ) + return false; + + /* Step 2: Check higher numbered entries for contiguity. */ + for ( b = 0; b < CONTIG_LEVEL_SHIFT && !(idx & (1U << b)); ++b ) + { + i = idx | (1U << b); + if ( !IS_CONTIG(kind, pt, i, idx, shift, b) || GET_MARKER(pt[i]) != b ) + break; + } + + /* Step 3: Update markers in this and lower numbered entries. */ + for ( ; SET_MARKER(pt[idx], b), b < CONTIG_LEVEL_SHIFT; ++b ) + { + i = idx ^ (1U << b); + if ( !IS_CONTIG(kind, pt, i, idx, shift, b) || GET_MARKER(pt[i]) != b ) + break; + idx &= ~(1U << b); + } + + return b == CONTIG_LEVEL_SHIFT; +} + +#undef IS_CONTIG +#undef SET_MARKER +#undef GET_MARKER +#undef CONTIG_NR +#undef CONTIG_LEVEL_SHIFT +#undef CONTIG_MASK + +#endif /* __ASM_X86_PT_CONTIG_MARKERS_H */ From patchwork Mon Apr 25 08:42:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825421 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ECC58C433F5 for ; Mon, 25 Apr 2022 08:52:04 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312614.529951 (Exim 4.92) (envelope-from ) id 1niuRi-0001wS-Os; Mon, 25 Apr 2022 08:51:50 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312614.529951; Mon, 25 Apr 2022 08:51:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuRi-0001wL-Ld; Mon, 25 Apr 2022 08:51:50 +0000 Received: by outflank-mailman (input) for mailman id 312614; Mon, 25 Apr 2022 08:51:49 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuIX-0003e7-Mc for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:42:21 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.109.102]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 9ed68a8e-c473-11ec-a405-831a346695d4; Mon, 25 Apr 2022 10:42:20 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2052.outbound.protection.outlook.com [104.47.12.52]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-29-0CaPfVyuOMuCq0QkgcbLoA-1; Mon, 25 Apr 2022 10:42:19 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by DB8PR04MB6891.eurprd04.prod.outlook.com (2603:10a6:10:112::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:42:18 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:42:18 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 9ed68a8e-c473-11ec-a405-831a346695d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650876140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mS9j+KCKEN6hv0oR924WDhW3Lh9+MjR+9IlswbKwkL8=; b=I76/w8DGOZpgW35SScLZK8Mfh3oq3E5u84hjo3ndSjfDTWyL3P4rbqlbB/25bWPxIsEIyK +wtbtZZJE6qemVuJtRFhAhzUwfo31vQjAsxcwBVAwI3bN6/bkzkIZoerO4uNbKAcTxw/G9 EGzw/qpATgJ/6PBlcdpqTy2Nl4czJpY= X-MC-Unique: 0CaPfVyuOMuCq0QkgcbLoA-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SRUjh57B/BFkSVqND4gx+M5oyLYLIjTb2ioq0JI2nOKfFKrDlC2fcypIHjbzwpogWZFibaVtm0Aj2VMyKG0kYReO4PqjH4xIpXS1qAJReuqlnU1NuJlhHb3XxPXMyk/7u1UcTBE8nv7y14Ei9KUnHRnM+FgVm6cguFicahET4e68zmW4ylxmbNN9mSiToqBNCImRsQidj0DoU4nrH7nHGfeUJI1aPam+fZaghqrfwV4mHAaWBmMXGEM3n61O0CeZM/KI/svS6hFyjj91+d96Z+ryr6i9fw0gRPd/UpY67xNbVtI7ZzXzQk+hsjBwO88hhQYocsjBMgcjN0JFnX8eRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mS9j+KCKEN6hv0oR924WDhW3Lh9+MjR+9IlswbKwkL8=; b=HS8FiBEfGvy5bXFfl5ZZJw0pwmK+B1s2/2a781d+5SFuBRlK/KfPFlx9nlRms3xVIYYpb6dR/dUnjivS3JcXy0IaAn2OJXIIRZr8kd2jJgBwxmJBdj6T+1VVXknqpZYLuqBHPgZ09qvOKQTKEmKQTjOWAO466TJk35T9e+33Dt8vDvi1I2iAdu1TKO1rA8WoKMQleikUZeMatUuTdHTZiMBX+qoShmclds+MiJcNmRBodWgiDlcumpv3t6sGA+hq+at2V3AdpLihoG0t40UCrPhJ4OgijXwX9Bcw2iVzZ54HZFFB0rD/4pU3eybiJqWBWPESWJomArl0nzg7tHS8Nw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <16f5b398-56f6-a70c-9ce0-0ad72eab5058@suse.com> Date: Mon, 25 Apr 2022 10:42:19 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 15/21] AMD/IOMMU: free all-empty page tables Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: AM5PR1001CA0046.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:206:15::23) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f3b9e395-2f89-4334-3add-08da26978160 X-MS-TrafficTypeDiagnostic: DB8PR04MB6891:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: I0gBddgZfz9sq6a4pIpAEnA7XSRVz2ByjpF0e5hz/z57IZ6KtRO6xJsoSySDSDhotybWUy9goPbPUQfJfCl47A7oFtyR84Z/FnouxFNDF/jR72MgR/D1JewmLds6CR8iFx6anCp/UXUi5GztMbOBZTVfd9fa6FMdof44zIPpZCsZ+xgG4nml0AZstfxrFKZfpnf6xqYpgPPTTa87+8KZd0CUaFQaNYkaEbqE/tdirqb8+k2FsH9Xce7YJw6l4nWwaKeejLWW0PKDm4Pf83lhXP0lwcoFniHxJk4yY8HwUXrvHp7/a1B9Zzhhj53n+PCAusdRj0YZZA2eYv1xUU6wScm0VoFh3ViR7S11cbXlwURFQ8hd75Mzdo+rspWljYqz/DINMEuvV+Tr5sMkWC0qxq4QipBBaYSdON2zdLdXNJH18xIEaQgyiiDbTA7zdYBdthslmblMqBnvyXxI/3UHW2Bx4rb4puYRtAtuUkG5PVOTD1zoyuPBFto1NT8dUNBtlBhnPy2umwpq5ScL3DoNr9T4oFnt7zxyXGCJkCgP5zT7li9PhOgGp9GovU+4L+dq+GuHt6zsuWbGkS6ZlVsxwC5OpilNKODOHzgMLyWaqtwGc0GgrNVDZLYYouPxFuX3SbO2s2GJthEJSIqMG9vvWoTmiR3RILNQ/bb7y6VVrb8+jELrA0TFCB0RO8KdHw/vrZUVC3bMEzAfjlJ29c1SqA+whRKM9GRZrf1WLZXa00g= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(66556008)(26005)(66946007)(6512007)(38100700002)(508600001)(66476007)(316002)(6506007)(83380400001)(54906003)(6486002)(186003)(6916009)(86362001)(31696002)(2616005)(2906002)(5660300002)(31686004)(8936002)(36756003)(4326008)(8676002)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?a9Rnz0ncCNhXFC3PIFDW6aSwr7Au?= =?utf-8?q?JLUgOcFOd2zXPRyNGSrM1bLfGfYfS2NpeGKL0qQkU1x8swrzCnc9ZmncvN9eK9OhK?= =?utf-8?q?J0oLIr3+v38/dnzcBALvSIpfWwB07P1+saLbRorvlUrdX4KOEYQxpAN1idqL3n8f/?= =?utf-8?q?DS+DAckYDnuO2KTdt6nlOW9m0Z8om0LOMeV4USL/B5UKHRvIeOGsJVG85Ig7OxjNN?= =?utf-8?q?4HlsUNObjqUhV2YWkhTVEbibreYnojK6OZfi6VsF/6R2274i/2ljCivEjR5SO5X8d?= =?utf-8?q?tC1Eha6LNO20nPEhPfL+96B5/DN2FM1fl0hQ2FFn0CfQZbDE0bmUpXkX141AxdOee?= =?utf-8?q?tWnFE8Bn1GOI0E9IYjVUSmvIzFbQg5yObF2y53I1nS2QxqYTbGl2o6R8rtGVn227+?= =?utf-8?q?A5OyNoiZ6vYqq97zclWnl4Ln4ebQU1kJdLcvJoQvEx0SsS+DplJS49XTGZmbhDANH?= =?utf-8?q?zundFCpsLeCOsudYr0R8SjacehKgurRX2+I97359xBdQN2jzEXkBie0tn0Cyu2PlZ?= =?utf-8?q?jgnwVTxlBMvKwt9LElMQ7EEJnh+7nuP2FqLEh4SGcqh0XM9HdWVCVqTffvxQt6hVp?= =?utf-8?q?mqwJnpvRdniI8/ApRP/VTzivTP2XnqqSJ6C6n5O+gHJrpx2D11rifcFANCx/YsAKR?= =?utf-8?q?/ou2ZoCtmBtzm51HS8ZODnnZUYLBCB3mFqF6iQLFKMRdU8Q8iFvhIPczFz7X/uVUf?= =?utf-8?q?bXyDpp+ngONAYstA3OZAx0fW3Hj4Nli00O5utif9yXwAFLYUmMBqFVWlnpZBn8WfI?= =?utf-8?q?HzPOyt26KRejmLzDDzbHMltPplrc2z3Q/aDeLMATPWnncnTJjv/SQeE3fBe+oyzq/?= =?utf-8?q?8fAHtigckLur0gAG+wYCnvIBOMfswaamaHeQBqOvg+dBB0FO+kK7FYFdgMx/3yEZj?= =?utf-8?q?Ot58PpX5kr2IiY79L2Nfhr7rDvFf/rxOY+A+2j91RCfjMP2nv9gSn1rUcjjEs0WBI?= =?utf-8?q?lx6d1eXAL4X959OB6d2rkDnrw1bCFzXDOzvXHTYtuC0XDmAypRTD3rJRog0TEEObj?= =?utf-8?q?uWVbz5woqZZ88i/OHDKkTbGU8STr6z5HKrkmUZX6Qicv7ScqgGyt94aRkJAiCTj/a?= =?utf-8?q?IGJjwyCsGHcktge3Zj0yUfLwWl3SlGNkFK6X8V7/t9Jvq2TqyED5SzcOOOqRX5UUY?= =?utf-8?q?+grj5ryW3NTC1VHfWwMnJ/wefNX2ABAnXDzAjGekns7FiNPL7YCJusfLydTlO2r24?= =?utf-8?q?pG2214pVWoCl4jPmvfmkxiBX883LJwiSdkJ90Cud9h8AW1omUzyXAyJowcY9KKAPU?= =?utf-8?q?0d/xPupN05UPB50ahJWxcVpxm+9/TvrxHPF3nHmBr0mm26LvaS2V+IL9LwUh6kMT9?= =?utf-8?q?FI+CRBYonV+mto3WzK6TBghvvDJkEjVjiOyBhI+GuU3LVPvQT9y/tPpkls+mljzYN?= =?utf-8?q?SSoi4ZNGEO/EzZVMlb1c/FgYBpqXQNDRu5rpyd0cm4rneVEIGtxzI0rz8o8Tx+QNl?= =?utf-8?q?4sGgunBYEupf7nB9foWsKpKL8iH0B/kqbqNOLMzb2HNpHvQAO2t53hSzs4bWW/kEM?= =?utf-8?q?ny9mFynfOKMd8tX0Qkp0CFn0a9p7w2qt6xkbJhwHsRPgpcOQm0Iz4lCbf953ZNj1w?= =?utf-8?q?168knHb+qfVVNd3t3tbXbK08YM0lj6YPkOZMXlTeekJ8n4wu5VIYtcdbNTllUD+wH?= =?utf-8?q?F5A1r2gnKCchAJfJU43CI1PGa2O5j8WQ=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: f3b9e395-2f89-4334-3add-08da26978160 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:42:17.9624 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Ckv8qioaFNZ4ZHa0ocqeNtfYSlenojKy/sQjR16M66MjYgxEcIqLVoMJWYNO4mVpbqd2BZSFVp1uDjSRcmY3iQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB8PR04MB6891 When a page table ends up with no present entries left, it can be replaced by a non-present entry at the next higher level. The page table itself can then be scheduled for freeing. Note that while its output isn't used there yet, pt_update_contig_markers() right away needs to be called in all places where entries get updated, not just the one where entries get cleared. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- v4: Re-base over changes earlier in the series. v3: Re-base over changes earlier in the series. v2: New. --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -21,6 +21,9 @@ #include "iommu.h" +#define CONTIG_MASK IOMMU_PTE_CONTIG_MASK +#include + /* Given pfn and page table level, return pde index */ static unsigned int pfn_to_pde_idx(unsigned long pfn, unsigned int level) { @@ -33,16 +36,20 @@ static unsigned int pfn_to_pde_idx(unsig static union amd_iommu_pte clear_iommu_pte_present(unsigned long l1_mfn, unsigned long dfn, - unsigned int level) + unsigned int level, + bool *free) { union amd_iommu_pte *table, *pte, old; + unsigned int idx = pfn_to_pde_idx(dfn, level); table = map_domain_page(_mfn(l1_mfn)); - pte = &table[pfn_to_pde_idx(dfn, level)]; + pte = &table[idx]; old = *pte; write_atomic(&pte->raw, 0); + *free = pt_update_contig_markers(&table->raw, idx, level, PTE_kind_null); + unmap_domain_page(table); return old; @@ -85,7 +92,11 @@ static union amd_iommu_pte set_iommu_pte if ( !old.pr || old.next_level || old.mfn != next_mfn || old.iw != iw || old.ir != ir ) + { set_iommu_pde_present(pde, next_mfn, 0, iw, ir); + pt_update_contig_markers(&table->raw, pfn_to_pde_idx(dfn, level), + level, PTE_kind_leaf); + } else old.pr = false; /* signal "no change" to the caller */ @@ -322,6 +333,9 @@ static int iommu_pde_from_dfn(struct dom smp_wmb(); set_iommu_pde_present(pde, next_table_mfn, next_level, true, true); + pt_update_contig_markers(&next_table_vaddr->raw, + pfn_to_pde_idx(dfn, level), + level, PTE_kind_table); *flush_flags |= IOMMU_FLUSHF_modified; } @@ -347,6 +361,9 @@ static int iommu_pde_from_dfn(struct dom next_table_mfn = mfn_x(page_to_mfn(table)); set_iommu_pde_present(pde, next_table_mfn, next_level, true, true); + pt_update_contig_markers(&next_table_vaddr->raw, + pfn_to_pde_idx(dfn, level), + level, PTE_kind_table); } else /* should never reach here */ { @@ -474,8 +491,24 @@ int cf_check amd_iommu_unmap_page( if ( pt_mfn ) { + bool free; + /* Mark PTE as 'page not present'. */ - old = clear_iommu_pte_present(pt_mfn, dfn_x(dfn), level); + old = clear_iommu_pte_present(pt_mfn, dfn_x(dfn), level, &free); + + while ( unlikely(free) && ++level < hd->arch.amd.paging_mode ) + { + struct page_info *pg = mfn_to_page(_mfn(pt_mfn)); + + if ( iommu_pde_from_dfn(d, dfn_x(dfn), level, &pt_mfn, + flush_flags, false) ) + BUG(); + BUG_ON(!pt_mfn); + + clear_iommu_pte_present(pt_mfn, dfn_x(dfn), level, &free); + *flush_flags |= IOMMU_FLUSHF_all; + iommu_queue_free_pgtable(hd, pg); + } } spin_unlock(&hd->arch.mapping_lock); From patchwork Mon Apr 25 08:42:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825406 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A5CC3C433F5 for ; Mon, 25 Apr 2022 08:43:07 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312577.529885 (Exim 4.92) (envelope-from ) id 1niuJ6-0005m4-76; Mon, 25 Apr 2022 08:42:56 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312577.529885; Mon, 25 Apr 2022 08:42:56 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuJ6-0005lx-3j; Mon, 25 Apr 2022 08:42:56 +0000 Received: by outflank-mailman (input) for mailman id 312577; Mon, 25 Apr 2022 08:42:55 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuJ5-0005as-70 for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:42:55 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id b29b928a-c473-11ec-8fc2-03012f2f19d4; Mon, 25 Apr 2022 10:42:54 +0200 (CEST) Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-vi1eur04lp2057.outbound.protection.outlook.com [104.47.14.57]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-38-GNVznJqEMCyrHfr4cwFnUw-1; Mon, 25 Apr 2022 10:42:51 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by AM6PR04MB6647.eurprd04.prod.outlook.com (2603:10a6:20b:f2::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:42:49 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:42:49 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b29b928a-c473-11ec-8fc2-03012f2f19d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650876174; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=F76uggdKpcuSYHZPwCK0FV2JMpLEoy075SIPYzCN2oY=; b=aHt0+UEKQ5szrRaM/h7CXa47hjLmRn1XOA9ssQzOkcVIYIrXHiEz0uRpdNfjmTdcEXnFIw DI1dMvwvY/FkFGZqH6Sb1bv7Z7zW/w6QwDJeKG18o+PXr+wSEml6gyRqZblSHrO3EdDucw h2PMLOLjfEU7rNDkQVre18RRe9uavs0= X-MC-Unique: GNVznJqEMCyrHfr4cwFnUw-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IwIt75MDAsiF5vuXMEKM7Qs+6xXxJvstqdESHtOtEwDLFLjwSSEFbGjSRN97SO7fthl7A00hE/kQdhmu4/fnuQ8CULnCFlXWgDkL/odcVYm+sviWr5/OBkc56EvarFy+JyiNXunjq/wK4+0B6MkcQfXNFvaIGkXRj3FUPeEM8d3AaP/k7Ktk5+xfSUSURSnWMAIJLDktKyNb96SgacdOPoJVovH0P6Cau75VKGOAY8cLchySdky7j2AYWFok0aTjLmdeyEFvVEKW3aGiaCOlYSQU7g7QtLhz7NpzDiusy+7bVE6LcEoC1LGVybKdqJIGO6ELqasXif/BYww+Dx/9uQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=F76uggdKpcuSYHZPwCK0FV2JMpLEoy075SIPYzCN2oY=; b=CMkyhM7LTzpt/ZYD1AKt5b6zsuaGrv43WnK6Zjz7u6vRl9zfWCFmTmkaFMDjIPHHRcM+Ak2136f9pEDzP2r0cehOsB1BH5HCuWCWUIjMSnYKcgb8cwD+K/y/QY8ut2IKHmmRxBjyT5YjjZn2acCnU69Jg+eTVL6mzPRaYgpHzTvRknrQqSYZAvkFpVVRZZiZaq7nvo/ESTe6RIR4m84sEnClCO3F/G98gFoyu5sBSX55UIoMgj2+gO8WFbIGqruxF0PRmsRBlCA6XCMXwTLCfYLlyx8K5JrU6qrdppfRuU+AyvnqfJbRg+TdsOBOcOtUoVinldcYz922sv8xETHUGg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: Date: Mon, 25 Apr 2022 10:42:50 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 16/21] VT-d: free all-empty page tables Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Kevin Tian References: In-Reply-To: X-ClientProxiedBy: AS9PR04CA0033.eurprd04.prod.outlook.com (2603:10a6:20b:46a::8) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1f36f1ab-8f0d-44cf-4b65-08da269793cc X-MS-TrafficTypeDiagnostic: AM6PR04MB6647:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +sZCJwKu+kKqohWo2wnpRtkG+226O4gDKlAtsuJIqnk6FjMeovxdmo9YulYupX9OPLn6n5NO0Hut7derq1vXpL6swurkfUaU/VJ3MICXKVkbaXw1gR0hWKgVsfdnzFoynS7dKGn+r5acRHdsoVEauALfhsqfnzK00kOIfk/tM2c6c4bevJ0noXrpJRdXpzUgTdATF30uSaLV6qhH0B74J+ZKs9ueRvFeBZqwKzZdzqC27CK+XPzxBsuYt8EuF6nvIUOLo1hXVHeq+pwZ1GwmkdbvsWzdgjmiWPz/k4dtZ6OhrOZoLDoiKSsY/ZxLcChd4h2MJBy8d/6jZWp1TCnVogvDP0ibWdD26nZmD/r+kZHzAc7V/8t5aTb7rMZz+ST44TAF1VrXT0yFawWMn0VsYcEeUokz5NYe5emXbp6K/jqUPtTmuwJo/l620EsFUw/WMdlhq6Y5WSyIq/uP84T4y+jaqX+yRcTE4pmKjeCO9RLKDUJhq4HWnCma+HasPxv3X/AwCJDcSxhjaffLF2pAO0cPQsgk77pIKXQPDnh158H3gLku2rXUq6pYpR7sHEnEhVpMF0jTIC1RbXRLDkjY43E51cAVdzw/Mr3V8Fdpy7XqbbDymywUB4+amsZLvRAOJ2Pd3xKPuQv2Jps+ZoxTnfMQad7Xo6RFTTQKsuQpgYtC4vhfhP1wwDISLIL7l8EHKWAlQ1WkEL49c5X/JdziyYliCRB+KW1iTPMDn3D4WdQ= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(6512007)(66556008)(66476007)(4326008)(2616005)(31696002)(8676002)(38100700002)(86362001)(26005)(6666004)(6506007)(6486002)(2906002)(83380400001)(5660300002)(508600001)(66946007)(36756003)(316002)(8936002)(186003)(31686004)(6916009)(54906003)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?h/6wSbtJ0v/TQ9fUp//L8KRfmcu5?= =?utf-8?q?RJcrOU9XgAUGX8BrYgEUZczmB/fo9RxeDYjXmC7ow61JXKUfsT21RBRlgO/jEfklr?= =?utf-8?q?r8/s9uvQ6dmDDvvmNH4U6dngM/nb/PdpAYJ98rvSM/ua4vNgA4DVZAYdeei+DuN7R?= =?utf-8?q?HkPgaB2zqGRMkwekqYEZj/L07WcxhR3zNFIl1wTdeQs/Jk8aCeHHQBRESSdiH7jbs?= =?utf-8?q?ebmusCuoNs7h4SoVUqVdm2QeCA2q/4+pfhuOUsigSVvGgn3VryIpjFM1xOxu4hH1o?= =?utf-8?q?iyGB5gigr/85Uy5PwaDQpdQHS5akZdBi5xCT2Ut8cJ7VLlXHiyfPt5F2/ODLskydX?= =?utf-8?q?W31e0tMp5jhWO7ChA9HSjrkaosEHaTAgRDl4xd984Fj8F25zNHQOdQt2ayirm2E6U?= =?utf-8?q?N1w9mu8dAu9wsm8fwAsWwXA0x82KVU7XiFY56Q1n9kLG4NvspG0cPTVGb0oeuAq5H?= =?utf-8?q?bU0t2TA8lsuTo8JBvmmE/3uQETJ8U1Tboxs6knDY2axeqrI8E9O6OTrnICEDJbfj5?= =?utf-8?q?i3oTQy47WrKIH9ahAod6gLN3vhqPtdnExdCu7HKH1Fr5Mu2p7yogB6o83Lny1rKt1?= =?utf-8?q?lXlws2ffT9Nf2lwhAVm9giNxeE+j7xxDTdV2hoWCB+zbSfnvALnvkyhsjnq1iyInm?= =?utf-8?q?16XusvZdaxNjKSGMN9eTYJb3fj6p9Jw4M057qYl52o8CO/nwZZan1sw+KMAzVhbzS?= =?utf-8?q?vwRrMeGOS8aEbRYMKnARzerArG0Enuh4Vd4eTZxGsgDQ+EFLI2lopow0zn1KtuY+E?= =?utf-8?q?pqD3mR1GWxawACXnd7+Qv64dDFnvweTF+eKuXLMSfmdzEYZ2YPcmc8SrJvaROWRxQ?= =?utf-8?q?eQmVzLge3aNQwgmkueyCowpL5n85ughsalAcRW0wwOAZdcvo7B0gEC4dswChIJ3Aw?= =?utf-8?q?Kcq00NyVbJ9zAZXq11KEAhHAbrQebBDjc81hjRpM6Y+Pu9hsL1yAg+DOOXtvIsJhk?= =?utf-8?q?2Apzk51J63LpE/tJEOjsiDEQHyjffbbYygIinwQy/7vX6T5lUgtwD/w2kyVv02Huh?= =?utf-8?q?0/i4+zsWSUMnTIoMHkXGUvTi73oWy/S0cPlFEyzQ4oK483mCkvbNXwqFngq6nQoYF?= =?utf-8?q?l9XhBfndoRNFvQnONdZQ41ofZWKEy+0TvQGum3+W7ZlByEbQ9ryeQOvZNaEPUZ118?= =?utf-8?q?eiSYppVxpl++mZYwFCxhRXhl64hj4sr5ujOnxQPtlVDFv+4pPgEqJzYtef0BwnFQU?= =?utf-8?q?ZBOEh2RvTS9VGuw/RofbC9vZyZ8M/PzBdvkv3LiX0Rmrci1D1hjT6UOdujzdg0AtN?= =?utf-8?q?uzbVAB0c/kp+3nzutlnIqXOxS//ZtVRpvNHa7xLJufA1EOhVID3/mQodWUBHqnBZM?= =?utf-8?q?zKHC2h8UjJurF6Zp+vfPUV374t0rvbID1N7yxvLKh8gh5RtOnk9QzbAUwi36p6rnW?= =?utf-8?q?zRTeAapTTwHf3i+smStLvqk6Rg6yxYL4QTYJmutBnCiiSVfziODOcDIK5CHq9DfhB?= =?utf-8?q?BR4ycXBiDVyqPB6mROZHXHE0/tjaIiyAR50c90knuvOcXywWI15n3IR0hSJ1x1kBB?= =?utf-8?q?LYgk0FfhJsWBjEcG18AFcM7AAtOitf8fnKFhUwCrSDZdAcyNwAQLtY4eJlbZDgfMZ?= =?utf-8?q?Ak9OhH80fFGiTSHBzDhjFNzYFWai5m60Oj5QCoSJTwFJcDCNgGlR6wuaTVXQRX5oJ?= =?utf-8?q?3aFN1Meui681kClCkkSOdw7q0fpuF2Hw=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1f36f1ab-8f0d-44cf-4b65-08da269793cc X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:42:48.9933 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jsmPMr0Ur35rlFPrzoOZ5DbnsRGaLvtm54hMftZ30/8sLRqFmrAOKhr7z/nR7CubGlwix4WOz85X9hPsdgTFgw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR04MB6647 When a page table ends up with no present entries left, it can be replaced by a non-present entry at the next higher level. The page table itself can then be scheduled for freeing. Note that while its output isn't used there yet, pt_update_contig_markers() right away needs to be called in all places where entries get updated, not just the one where entries get cleared. Note further that while pt_update_contig_markers() updates perhaps several PTEs within the table, since these are changes to "avail" bits only I do not think that cache flushing would be needed afterwards. Such cache flushing (of entire pages, unless adding yet more logic to me more selective) would be quite noticable performance-wise (very prominent during Dom0 boot). Signed-off-by: Jan Beulich Reviewed-by: Kevin Tian Reviewed-by: Roger Pau Monné --- v4: Re-base over changes earlier in the series. v3: Properly bound loop. Re-base over changes earlier in the series. v2: New. --- The hang during boot on my Latitude E6410 (see the respective code comment) was pretty close after iommu_enable_translation(). No errors, no watchdog would kick in, just sometimes the first few pixel lines of the next log message's (XEN) prefix would have made it out to the screen (and there's no serial there). It's been a lot of experimenting until I figured the workaround (which I consider ugly, but halfway acceptable). I've been trying hard to make sure the workaround wouldn't be masking a real issue, yet I'm still wary of it possibly doing so ... My best guess at this point is that on these old IOMMUs the ignored bits 52...61 aren't really ignored for present entries, but also aren't "reserved" enough to trigger faults. This guess is from having tried to set other bits in this range (unconditionally, and with the workaround here in place), which yielded the same behavior. --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -43,6 +43,9 @@ #include "vtd.h" #include "../ats.h" +#define CONTIG_MASK DMA_PTE_CONTIG_MASK +#include + /* dom_io is used as a sentinel for quarantined devices */ #define QUARANTINE_SKIP(d, pgd_maddr) ((d) == dom_io && !(pgd_maddr)) #define DEVICE_DOMID(d, pdev) ((d) != dom_io ? (d)->domain_id \ @@ -405,6 +408,9 @@ static uint64_t addr_to_dma_page_maddr(s write_atomic(&pte->val, new_pte.val); iommu_sync_cache(pte, sizeof(struct dma_pte)); + pt_update_contig_markers(&parent->val, + address_level_offset(addr, level), + level, PTE_kind_table); } if ( --level == target ) @@ -837,9 +843,31 @@ static int dma_pte_clear_one(struct doma old = *pte; dma_clear_pte(*pte); + iommu_sync_cache(pte, sizeof(*pte)); + + while ( pt_update_contig_markers(&page->val, + address_level_offset(addr, level), + level, PTE_kind_null) && + ++level < min_pt_levels ) + { + struct page_info *pg = maddr_to_page(pg_maddr); + + unmap_vtd_domain_page(page); + + pg_maddr = addr_to_dma_page_maddr(domain, addr, level, flush_flags, + false); + BUG_ON(pg_maddr < PAGE_SIZE); + + page = map_vtd_domain_page(pg_maddr); + pte = &page[address_level_offset(addr, level)]; + dma_clear_pte(*pte); + iommu_sync_cache(pte, sizeof(*pte)); + + *flush_flags |= IOMMU_FLUSHF_all; + iommu_queue_free_pgtable(hd, pg); + } spin_unlock(&hd->arch.mapping_lock); - iommu_sync_cache(pte, sizeof(struct dma_pte)); unmap_vtd_domain_page(page); @@ -2182,8 +2210,21 @@ static int __must_check cf_check intel_i } *pte = new; - iommu_sync_cache(pte, sizeof(struct dma_pte)); + + /* + * While the (ab)use of PTE_kind_table here allows to save some work in + * the function, the main motivation for it is that it avoids a so far + * unexplained hang during boot (while preparing Dom0) on a Westmere + * based laptop. + */ + pt_update_contig_markers(&page->val, + address_level_offset(dfn_to_daddr(dfn), level), + level, + (hd->platform_ops->page_sizes & + (1UL << level_to_offset_bits(level + 1)) + ? PTE_kind_leaf : PTE_kind_table)); + spin_unlock(&hd->arch.mapping_lock); unmap_vtd_domain_page(page); From patchwork Mon Apr 25 08:43:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825407 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6D1E4C433F5 for ; Mon, 25 Apr 2022 08:43:34 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312583.529896 (Exim 4.92) (envelope-from ) id 1niuJV-0006Kr-K0; Mon, 25 Apr 2022 08:43:21 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312583.529896; Mon, 25 Apr 2022 08:43:21 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuJV-0006Kk-Gt; Mon, 25 Apr 2022 08:43:21 +0000 Received: by outflank-mailman (input) for mailman id 312583; Mon, 25 Apr 2022 08:43:20 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuJU-0005as-5B for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:43:20 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.109.102]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id c179822f-c473-11ec-8fc2-03012f2f19d4; Mon, 25 Apr 2022 10:43:19 +0200 (CEST) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-ve1eur03lp2055.outbound.protection.outlook.com [104.47.9.55]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-3-b_RJjmPEOLKgT5WnXpataQ-1; Mon, 25 Apr 2022 10:43:18 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by AM0PR04MB6098.eurprd04.prod.outlook.com (2603:10a6:208:141::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:43:15 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:43:15 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: c179822f-c473-11ec-8fc2-03012f2f19d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650876199; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o5dMhpfznMlqVzr+kJ/53g9HZAHAFuJCF88KGG0p/Pw=; b=gskIEYe9IOm7q3QfsHWYC2um8rRth94hdRAjXOeqLzzgOlwkvnjnTtnJ3qyM3ZYwNpcL7i XXfZ0yFtaVjMo+1Cw6RWnPhv0CqOBmpH9/wALLZ1Pj+sqc844dOQAtCp799P6Pj9JlMs6b dNoRD+ioyrDskObMH6tsdZfh7vw++zc= X-MC-Unique: b_RJjmPEOLKgT5WnXpataQ-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZftQTbEYgdFI+6kFvVAR1LxW+gdwYPevxgRcxbAjkJH2xeMrt1lhDdShjj4WxGNyXpSN8/wAYkXtG3mNdNvLiSiLehSEP1GOWXE7OjIFRwb8kL+waytULHoEbRruTU5us+n3Zp5XGJfkLAhGj3Mkf/aD/7K+HGjvICVAmXDXazDxdLVvnBxFr0HBJW5gQ5IiX54qaWTzMNagqB0pNz/bSqyOmiLXIFmBMvSajgYZSa9Ofn/DrLFImniq8DO46z2FsH5wV9XUAHA+tlVcNDWtNkdTFBzvF7veo47iWWYTsqrh3htOuju3MwoMm682VsCRL/yLNM+0ZiAie4PQIaPDXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=o5dMhpfznMlqVzr+kJ/53g9HZAHAFuJCF88KGG0p/Pw=; b=hPrybism78QkDSlKfu4XCcxGX0U2vKBDRgDyZNGShWVF9DJlzW21kuN/2eom/yaoHbd6QkU2k1e/B+s+yoPc0B2lk5x9RgUItu2UTUcGjf8JsgYoO70kT/W8Yz5j8sB1TuWApNa/GJDFERrf07Eje1SkYjYHr/OHIqIayTomzPrI4sRTSnPw5GF6JhmHb1pu56K5gVcNFEpWvc8VwvaeDgwMs1I03wx3UrFjHo4tqOSYppSSkezTBcUZWagYru0351ZSQ7+YUn5o0HiXmu4d1r/5j5MjR4BvBfA/EdJ/scN3tm0BXRvLb7TKkkgoOdIoc/hdmC1nHPxbUl4rtm3Yyg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: Date: Mon, 25 Apr 2022 10:43:16 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 17/21] AMD/IOMMU: replace all-contiguous page tables by superpage mappings Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: AM6PR10CA0100.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:209:8c::41) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0f182e83-4d2b-4a78-34b6-08da2697a387 X-MS-TrafficTypeDiagnostic: AM0PR04MB6098:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ytsBkCaiyoBdFjY91ut+WHsBc5lfa6NiEnTAJYQGLPRDVwVIshYUIGksbHMJmMwvoBhJKIDKjXXXXtW06f6og/+du/wk0zu2/8OqHw4opGJMo8fqCpit2HjL6DrzVkr5XO909B2v/jl+yb29yqWC/kE+2amqVj5BpRKJoc533Mt/MZyvDNP8AZAxYRNmaWNu2+jIi7g+AVw1mIoy5qqKmCmkCcrrp1ppHuV65kPVtarNyBpAuChiI0YkbK/BkzeaXM4rWkRhh1wWil/KBhe8BJsYlcFcR1bwfb8KQRtp34aEfKtZOShA9pAAdJi228p80QvWqochXLt/2tbvZtsaE5N9W84360ojKDwSl/z7ISzBoaRNO2arq47qReNO0FoLbk66JHVYiQzRX1XDs0FbyFPu95Z0fz9mNYA2RhKl9i6HyAwDhZ0rhqWbJ2oT/FtT4EIwQwO3bMdoyDvUCSKZXXlv0QsH5WAR1p+GDeQ0ljJqN4zunCwgGLjbDGDQioa5IQixe/vbZc6heBBl70NgBZVhVjFn/82Y1XdVeEyyDgrxjKCun+E4PI0CEg6ZgA7UJNAY+pnsSnW1Y2wF3Dvl8ivLJddl+xVMLnNUxCXGTVf+FWYT6HuCEBLC9hB20gPSWpiVIPVIHrnSmfNVSJFlntt4znH8KIzFGGvClAfOBZ2kqO7xf4M9cNl8voKRQ6JIkFifhMDqBT/Kqum6SaiX3hbT8nwu5qvgEeyas9rDFcE= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(5660300002)(2616005)(508600001)(8936002)(86362001)(6506007)(6512007)(2906002)(26005)(31696002)(38100700002)(6486002)(186003)(83380400001)(66476007)(8676002)(66556008)(66946007)(54906003)(316002)(31686004)(36756003)(6916009)(4326008)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?fQXg3czNMoecGfip8u4on5dsB1Cv?= =?utf-8?q?hhD6zfHa3PORa+nhMXjni12iOb0hiewSsW4r3a3cpdgKP3hFeOxQVMxUsLniu4PQh?= =?utf-8?q?Lt6bWQpPKxetD7bFvzQdI+iGIQuQY0zyWIAnIK7/ScHXN1tY6U5POLO4put996ur6?= =?utf-8?q?Cdi/Y8+VMEAUaBkBcKO97dJmV4a4pqFebWDR3cG//8eJZJxXOATMYRZhUW6Q9DWwn?= =?utf-8?q?M3b8M0lKSTy7NmANPxETSsm7NLjvxzWavwwj1e9URWcHQbaPI7efY6BFWEmUe0wLZ?= =?utf-8?q?isDbKDqI7srOJDOX2nAlE6NA3Jia+C7hWGHPyT3SJyyjWRnOIRGl9I5KtpZeVmuJp?= =?utf-8?q?8wyDxA1SBUv6YuY0VmMxx/wW2QXNdI4G7XB8ADfkKjB5Ll4w0MVt7G9lM3vrxazb4?= =?utf-8?q?zV3dQ1CRod3SL2qOlYWE0Ez4TuXNFgSF6WM4eMOipBYCu9+dHdhhunyXhny5RBVw9?= =?utf-8?q?LzbpLQ6SH+eBJtRPltHEEnbYztvEbS/L8WLN4niN84HHUDTBp5TmHLEwPOGnu8b2N?= =?utf-8?q?vFGIsJS4gJTddgbtMor0LC2gqelx5wn19SPlPPbqut29iNxgRpkUhAlxHs5Ue/r5f?= =?utf-8?q?9yaKygFZOz610Fn6OIDSQx3PVi9Pt85q9V1LEDmh4QXsoKCu7IVvKWJeXRhdLjg/Q?= =?utf-8?q?FUN/ldwLts5jFhiGf0C8Tw4vHp2tCnYS2UJRDCJgm+TOjyOVfMOK2ERzPWxSgBNM3?= =?utf-8?q?gdehAehcQMP03oDnfHur0CblAPGoKuMJd2SFWvw2IoxNgzmHu8NsQOlpnp/gX/i1V?= =?utf-8?q?OugFEETd61jur2Tnkm/LQpEPm1uHGlpBukD7y4P3Klzo/M903K8kioM4btpRG3YaI?= =?utf-8?q?V7PO8iEWSPw52t4K4uijt5gDd37YATRx6XgtCGAf5001BfbaF+7ALqqYht2BqGv3G?= =?utf-8?q?EP361AaOXrv1KiCzYgadc1rfy50rN0St/twwpvtBEGMThfz/dC9Xnk2XBk/PywqrZ?= =?utf-8?q?lL/fzTPc4NEva6vBlToCf4iDbctZXeP8tsKM5E0LlIlmU9C05nhL7DkMHMOjFIXdA?= =?utf-8?q?7LqCxG8a56/3WKdHqMJLyuBpYvTx7NKJtmabRBPuscXwW654JCNloUzP3RI4H8Z+g?= =?utf-8?q?3ekZ5Y1VUa32OV0ImK3zVgYOQHZkgsxC61szGBAQdaXDePP9LJAsQgHMWzuomq8D3?= =?utf-8?q?uPpvUx/I25UjTEclkASV93noOpBlfgCvMZ9g+OixuXqfnEsu3Ww5Nd+2+ka3bESPO?= =?utf-8?q?K4mngothbC2KkLgiQuhSDicf0vRwuqNaoQ77esa52vRul3RwIJx1iqzyUySfPzm+Y?= =?utf-8?q?txbOqCDodfzr+U+ArkyEKRqC0LBLXz5IbWWqevelQ0y6ZL2vpDUxbyo22PseHUH3e?= =?utf-8?q?xC8FfWhgPwEeb5qshZsuuT13KAErVnJYDzLnToNzCFjgLuAT1nNPgkJi24wjNS/W1?= =?utf-8?q?JxHznX36/Ikr+jOPqEZwXXtkHmARqARlVgn/80+ulsSWPlnc1yfE0M4lZFMFE2y+7?= =?utf-8?q?dRyuxDOEBoU/BUi1R71X4CagBcp5bha1MPZ4ZY9UXosvg8vOTC7/h2ksRVu+Stwyr?= =?utf-8?q?73MAIlW2h7mJvJg9yucA7h/K1K87TKINLwh4CqSvFJ4r7ObLGf2OzllU0Wyuw1jYo?= =?utf-8?q?0RPzmBYmcDRjAl2Wt874BrGFLM/u2qVABOvf4Gu/dOoy+kfFAxQ+xuFNtt4t+VPj6?= =?utf-8?q?oKwdlcdFA43wLMfwXz6D9fhIRe3gVkZg=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0f182e83-4d2b-4a78-34b6-08da2697a387 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:43:15.2748 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: agMlyq3K+fhi5YuXGvetiyiMKMi6oafnFHppyh2d4gNy0WJhVsZb9SsFfVrYfUX3Q7gO7gM4O8p66QuC5Jrj2w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR04MB6098 When a page table ends up with all contiguous entries (including all identical attributes), it can be replaced by a superpage entry at the next higher level. The page table itself can then be scheduled for freeing. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- Unlike the freeing of all-empty page tables, this causes quite a bit of back and forth for PV domains, due to their mapping/unmapping of pages when they get converted to/from being page tables. It may therefore be worth considering to delay re-coalescing a little, to avoid doing so when the superpage would otherwise get split again pretty soon. But I think this would better be the subject of a separate change anyway. Of course this could also be helped by more "aware" kernel side behavior: They could avoid immediately mapping freed page tables writable again, in anticipation of re-using that same page for another page table elsewhere. --- v4: Re-base over changes earlier in the series. v3: New. --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -81,7 +81,8 @@ static union amd_iommu_pte set_iommu_pte unsigned long dfn, unsigned long next_mfn, unsigned int level, - bool iw, bool ir) + bool iw, bool ir, + bool *contig) { union amd_iommu_pte *table, *pde, old; @@ -94,11 +95,15 @@ static union amd_iommu_pte set_iommu_pte old.iw != iw || old.ir != ir ) { set_iommu_pde_present(pde, next_mfn, 0, iw, ir); - pt_update_contig_markers(&table->raw, pfn_to_pde_idx(dfn, level), - level, PTE_kind_leaf); + *contig = pt_update_contig_markers(&table->raw, + pfn_to_pde_idx(dfn, level), + level, PTE_kind_leaf); } else + { old.pr = false; /* signal "no change" to the caller */ + *contig = false; + } unmap_domain_page(table); @@ -407,6 +412,7 @@ int cf_check amd_iommu_map_page( { struct domain_iommu *hd = dom_iommu(d); unsigned int level = (IOMMUF_order(flags) / PTE_PER_TABLE_SHIFT) + 1; + bool contig; int rc; unsigned long pt_mfn = 0; union amd_iommu_pte old; @@ -447,8 +453,26 @@ int cf_check amd_iommu_map_page( /* Install mapping */ old = set_iommu_pte_present(pt_mfn, dfn_x(dfn), mfn_x(mfn), level, - (flags & IOMMUF_writable), - (flags & IOMMUF_readable)); + flags & IOMMUF_writable, + flags & IOMMUF_readable, &contig); + + while ( unlikely(contig) && ++level < hd->arch.amd.paging_mode ) + { + struct page_info *pg = mfn_to_page(_mfn(pt_mfn)); + unsigned long next_mfn; + + if ( iommu_pde_from_dfn(d, dfn_x(dfn), level, &pt_mfn, flush_flags, + false) ) + BUG(); + BUG_ON(!pt_mfn); + + next_mfn = mfn_x(mfn) & (~0UL << (PTE_PER_TABLE_SHIFT * (level - 1))); + set_iommu_pte_present(pt_mfn, dfn_x(dfn), next_mfn, level, + flags & IOMMUF_writable, + flags & IOMMUF_readable, &contig); + *flush_flags |= IOMMU_FLUSHF_modified | IOMMU_FLUSHF_all; + iommu_queue_free_pgtable(hd, pg); + } spin_unlock(&hd->arch.mapping_lock); From patchwork Mon Apr 25 08:43:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825411 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 65B39C433F5 for ; Mon, 25 Apr 2022 08:44:12 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312588.529907 (Exim 4.92) (envelope-from ) id 1niuK0-0006wu-U9; Mon, 25 Apr 2022 08:43:52 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312588.529907; Mon, 25 Apr 2022 08:43:52 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuK0-0006wn-Po; Mon, 25 Apr 2022 08:43:52 +0000 Received: by outflank-mailman (input) for mailman id 312588; Mon, 25 Apr 2022 08:43:51 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuJy-0005as-Th for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:43:51 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id d3cfee6a-c473-11ec-8fc2-03012f2f19d4; Mon, 25 Apr 2022 10:43:49 +0200 (CEST) Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01lp2054.outbound.protection.outlook.com [104.47.2.54]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-6-KWNPR1rNMOqmff3-3asZWQ-1; Mon, 25 Apr 2022 10:43:46 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by AM5PR04MB2977.eurprd04.prod.outlook.com (2603:10a6:206:a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:43:44 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:43:44 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d3cfee6a-c473-11ec-8fc2-03012f2f19d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650876229; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Ab4csObLsDUSWIHGU1Rqn2fo95dC4MwB105VQU6jBM=; b=kn/O4h13ccJFKgaYAjqdu5fZJ84HjjJ6DUjM69fBlyBzHxPTfaVvqNRSHylPD2P0b5nyjk snmaxc895CHDlCHzMIW7m+E9chm+9/oqQ1VuLcWVzUA31JaWipQJU62Uo5FJKHCCLVhGVs nB7yKUpDzmY+ObDwAN9JnpHfGpOHRz4= X-MC-Unique: KWNPR1rNMOqmff3-3asZWQ-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=c3sSYpQUGEe8oMQuLwpvxYE7IALALsYNSrxzciq44FAKisvIRZ21WAj4BL4oTKi8QCpEWU0nnfwqf2ofQVeAE2LWvgTcKUNpoII0ZDez+rWwtJz9dsY1WlciEu+eB1oSXMlwnQUQFX3g3bE7qqtWz9MQM6aVJ5+bfv2Z8gvFEmcq8RPw11SsYprDCRHsQrERLK8IHEO3SIsxvg/lbca+BxO4AcwSBSnqbIy76ZvoBFh65yKuEwwYPVYJVa5hhSfNO9q+rXZGAB6/Y7P+NHdWP6n9O3suJZWmRWm9CTfZOVpvqIHpsTrUSTeZxz+Rr8bBNU0BU67xRIctdAP4V1+ivw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2Ab4csObLsDUSWIHGU1Rqn2fo95dC4MwB105VQU6jBM=; b=jtjTfb2Xv1Lwdusn/BqZi1ZpIp9lm18sqYVKaQv1dFoH9nReYaLSGs6LLAVl09Ywaz+i+PaPCtDXT4oIVCaTdgbvfkoj09R1xV3tNDujoh74pI8lfwszPjf4uSbo5+dwl6ycQrzh0lL2w50jbEp3JzXksnhBenr3bsHUFaThNk+IhD/vES4jpSXNdLEEx5eSgjUwigMS+yBeMX4QWMH36BOIvIKfGdCEgMc5TOxoGgZJYQbQlMku6KYnONiZAz0ABzv1YbDB8MklTazAukg5pELcBGh7bCH3f4vMmMZLvbfd0ajb8C0kDb6Xq3N2hXApCEEYkhiFTht+DCP1HVNdKg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <98553b89-6296-9e4c-4677-9201cd7cdeef@suse.com> Date: Mon, 25 Apr 2022 10:43:45 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 18/21] VT-d: replace all-contiguous page tables by superpage mappings Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Kevin Tian References: In-Reply-To: X-ClientProxiedBy: AM7PR02CA0025.eurprd02.prod.outlook.com (2603:10a6:20b:100::35) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 31b50537-6199-46fd-b8f1-08da2697b4e7 X-MS-TrafficTypeDiagnostic: AM5PR04MB2977:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nzZBCILlrNZDOJWCc8bd7Frfg87ffeUqCgCeHxSU/SOuETRVFnnxO26+GxIkSky9IMiWMcK+y7GaRJ7ULk+3kuZSoacbqSbNGrH9MhbvNnuBGm8tf/0DTPoQDd0Z+lWEZ+dnDOd7OmPL0mxBW5eQNWu269Ls0lQv6Tob7hx5iyBN5sbCdUjrzD+n61lMJiJndDYvitL1MkyKTn0K4gEqQrprtsD80KyzRSJxmrQQYDNSWQ/u59BJi8pRQsplCgE9ADDnNoT1S98r+zjmSPEb1+DIh/ypwcTLSBngctxh3vvCRqKHG/hefKf+VM0vIhtFDmoBcLsG4mP3AAaC9dneR7VxN4O6pKLk5AVyIJzAfeK/pwNdjXq+pa8mF3wEur6MRXOELIYiKhVAdZdopgtizEXeFTZMErC4itRFj0DsHBSloL/Yb3cJ/CxyxhdR/a/m6jEo+ZZayo1AlG1Wo8LPEDX4afM/L3z3zs0vM4/Wqgl4WIp3J+cNk3Nmf8UswRdY3XLfwmyX5091J18OoB3h15dSgTJPhYW6+ZUcjvdIyVZK7oB/zGp5Fft1xDIwVgSzCePAZ2WbaCEcjpki4OGfN/ZLwxhVEmN+CziKKWjw0/opH2mbM8fAmSFocUANfoDHCaJ3hyYKULCo6Ma08ETTiDZ9TdDf52jMArHSo64hxkeOH0sikZV76kVDocybJW9RkX4XZ427fHe4itvTvXhZHnZoX1Syb/1Laj3FcX4vBuc= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(316002)(6916009)(508600001)(38100700002)(26005)(6512007)(6486002)(31696002)(5660300002)(83380400001)(36756003)(86362001)(8936002)(31686004)(2616005)(2906002)(66476007)(66556008)(54906003)(8676002)(4326008)(6506007)(186003)(66946007)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?oEAb4d+VhUrsgvquzv5Z6vrWScYo?= =?utf-8?q?FlKX7ZeNP7OBLMzKFrhvPrh7WOPcN2wGwot9vEVZbJrVAqJwRBUNZdPTC9Ajcsp95?= =?utf-8?q?YoMfsWsr5UF1zP3yYTS8PCUEpMS0MX2+15P8BnbwbhPhDFFcrOj/1cwlPkmn6ogCK?= =?utf-8?q?xhbaH2/MUPekEMatq5OdVZA6jQL3m7iCitURc0Oo/0CIl5VtVPw0G2U80Yvn6BtnS?= =?utf-8?q?hla/lt2ecpN43DWfdXf3SGb/RKQ0eP+pM9ifRtzP2MyQZYjyNFF0ZHivwG59JkDsf?= =?utf-8?q?NfIYzs24cPilD5ORzBWII5drREyMbgNq8doOalKpUVCMk3qSBpC4vFevodcdbuUsb?= =?utf-8?q?o71tjh5nCBThaRsHHjrWJ6K/3emv7dj98cf1psOxjIguzonuoVQBnXDW1rZyHbeXN?= =?utf-8?q?zzJKc+T0icAutYHnxeh6ve3dOeceYW3KL/PIBfKOjBNxtKi7hi7bTsKIW+6WTwzvV?= =?utf-8?q?LArUZMVLDrhmhpLcv7JjC8EOtz4n9l8/4/8MkgyqUPD+8tcBR/ztdX+E+yPylzqGx?= =?utf-8?q?OVsOj2m0L7h2//N+uqeiWCnnYdfTJTAuDZqgeserf2X6S+OSHPysfP/6Swxh3CIGE?= =?utf-8?q?xHWtTeJSztg86vajYuHX00xkz9tkd/HBn0+E8uyO0ei0AaIlSF8v0s+DlAt7pvyi5?= =?utf-8?q?LDOR3MqqtTHO+QYbXO7ByWvtBsl6lKHsleC/rgxJxdPAKRS22MCwzhccq9U0o2/1g?= =?utf-8?q?+zxgXoCMm9ge6Y0LlJMDdqDmXC+dcgpL6gtLDFu+dcV6ytjIqzQTOquOmZBnIC/8I?= =?utf-8?q?UVA4hZEm//R5EPim1haJ7JUJuAdyW6RpqKZ9lNAWyaAXYdP1baXr4D6BPtSli63Fo?= =?utf-8?q?50msr2DDzcS013z9HUHsHyugP6JxChBHZW9MVAnHnxrdvBcC62THhXN916mdYNXlE?= =?utf-8?q?d6bs+k8tTqZbSVDYHbICVycut+fwK9hOmHTobyLokJMSvVNAdDjQTABiPefrNWOrG?= =?utf-8?q?q2YKcTHdsdFufg2pUwmDshe/bJsthi/SYQjmrlWP97YoO/PTjNDAJfbwXfIk1LpGI?= =?utf-8?q?Zd3bVzw4OCwtD2tK00Si+aTB4MS+RqLVBDzrVb1kuwPNfDnRccfmRCeEmkG8y6t66?= =?utf-8?q?lp5Y/4KolzB0P/7gMTPP0xs9b4RaQyDPYTS3Zl4eRYuq0IN6UD9m+aBtdq08IZdfE?= =?utf-8?q?id2whaBlUQGqeT2toiwNnGiZ9DKr90ISy7P25ryDTEp4EqYzHwLcQV45WejzrxEOv?= =?utf-8?q?KBDzidd8eOBpoc6N2h7cQQ9VpEGddgFsuWZjLwEvmqpEBEW+dcyHSWPhnkG+sPOrn?= =?utf-8?q?QL8P2CcQoWBOccT/1taPtN8SMjlQ7D7w4w+aGx8BCw36R7J4p9iyISAlrbtjNaVV+?= =?utf-8?q?911m+go5Akm/roNvK4Hw/LD/6nSnFTGayv4E2N1vwESg+Mh/H0XkzQjYz0UPKhWuL?= =?utf-8?q?e8HOOj9xlBi7D6YVcL7/0i4mFQ7kz64jtcczeEPCcKwSOE516Ned2V/yGWTdD2hg0?= =?utf-8?q?qJf0F4CsAcNy22JVFf4KmlYFe1p50Dmb7fpu/Lc/Pv5kJ7420YiWJDqAWL1ymVNaK?= =?utf-8?q?0tRJ8XW5FLf/hOn2RXIr8TdslgMLmLZ/VN6KYp1Sq4cjdpy2TaTbkCK9NWdwO7yJ+?= =?utf-8?q?M5VuQIidS7emvLE4VDdbLkehQUlQCJq8QQAeoXE6y826zliHJ3advQ7K+COQuGTaN?= =?utf-8?q?qF0iUmGn9C9myRAOncRPSHP0HzY/XHCw=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 31b50537-6199-46fd-b8f1-08da2697b4e7 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:43:44.4286 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QnGPBIna7w5QCA+on4z0BrWmh88J8zzk6jnLn8NuK7xx2FVT0XPddEUHzw6i8DuoGABAlL7oU1NzJe5kpb99OQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR04MB2977 When a page table ends up with all contiguous entries (including all identical attributes), it can be replaced by a superpage entry at the next higher level. The page table itself can then be scheduled for freeing. The adjustment to LEVEL_MASK is merely to avoid leaving a latent trap for whenever we (and obviously hardware) start supporting 512G mappings. Signed-off-by: Jan Beulich Reviewed-by: Kevin Tian --- Unlike the freeing of all-empty page tables, this causes quite a bit of back and forth for PV domains, due to their mapping/unmapping of pages when they get converted to/from being page tables. It may therefore be worth considering to delay re-coalescing a little, to avoid doing so when the superpage would otherwise get split again pretty soon. But I think this would better be the subject of a separate change anyway. Of course this could also be helped by more "aware" kernel side behavior: They could avoid immediately mapping freed page tables writable again, in anticipation of re-using that same page for another page table elsewhere. --- v4: Re-base over changes earlier in the series. v3: New. --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -2216,14 +2216,35 @@ static int __must_check cf_check intel_i * While the (ab)use of PTE_kind_table here allows to save some work in * the function, the main motivation for it is that it avoids a so far * unexplained hang during boot (while preparing Dom0) on a Westmere - * based laptop. + * based laptop. This also has the intended effect of terminating the + * loop when super pages aren't supported anymore at the next level. */ - pt_update_contig_markers(&page->val, - address_level_offset(dfn_to_daddr(dfn), level), - level, - (hd->platform_ops->page_sizes & - (1UL << level_to_offset_bits(level + 1)) - ? PTE_kind_leaf : PTE_kind_table)); + while ( pt_update_contig_markers(&page->val, + address_level_offset(dfn_to_daddr(dfn), level), + level, + (hd->platform_ops->page_sizes & + (1UL << level_to_offset_bits(level + 1)) + ? PTE_kind_leaf : PTE_kind_table)) ) + { + struct page_info *pg = maddr_to_page(pg_maddr); + + unmap_vtd_domain_page(page); + + new.val &= ~(LEVEL_MASK << level_to_offset_bits(level)); + dma_set_pte_superpage(new); + + pg_maddr = addr_to_dma_page_maddr(d, dfn_to_daddr(dfn), ++level, + flush_flags, false); + BUG_ON(pg_maddr < PAGE_SIZE); + + page = map_vtd_domain_page(pg_maddr); + pte = &page[address_level_offset(dfn_to_daddr(dfn), level)]; + *pte = new; + iommu_sync_cache(pte, sizeof(*pte)); + + *flush_flags |= IOMMU_FLUSHF_modified | IOMMU_FLUSHF_all; + iommu_queue_free_pgtable(hd, pg); + } spin_unlock(&hd->arch.mapping_lock); unmap_vtd_domain_page(page); --- a/xen/drivers/passthrough/vtd/iommu.h +++ b/xen/drivers/passthrough/vtd/iommu.h @@ -232,7 +232,7 @@ struct context_entry { /* page table handling */ #define LEVEL_STRIDE (9) -#define LEVEL_MASK ((1 << LEVEL_STRIDE) - 1) +#define LEVEL_MASK (PTE_NUM - 1UL) #define PTE_NUM (1 << LEVEL_STRIDE) #define level_to_agaw(val) ((val) - 2) #define agaw_to_level(val) ((val) + 2) From patchwork Mon Apr 25 08:44:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825422 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EED2FC433F5 for ; Mon, 25 Apr 2022 08:52:24 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312621.529962 (Exim 4.92) (envelope-from ) id 1niuS6-0002Wk-2e; Mon, 25 Apr 2022 08:52:14 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312621.529962; Mon, 25 Apr 2022 08:52:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuS5-0002Wb-Uq; Mon, 25 Apr 2022 08:52:13 +0000 Received: by outflank-mailman (input) for mailman id 312621; Mon, 25 Apr 2022 08:52:12 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuKN-0005as-UA for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:44:16 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id e2a41c9e-c473-11ec-8fc2-03012f2f19d4; Mon, 25 Apr 2022 10:44:14 +0200 (CEST) Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05lp2111.outbound.protection.outlook.com [104.47.18.111]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-4-K12lP_fTMSycn3GH1ByPbg-1; Mon, 25 Apr 2022 10:44:11 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by AM6PR04MB5736.eurprd04.prod.outlook.com (2603:10a6:20b:a8::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:44:10 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:44:10 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: e2a41c9e-c473-11ec-8fc2-03012f2f19d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650876254; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Zlfuedf2q4YXfgbpQ1MWK6FkdjlHZbNpmqaza4t7ui0=; b=ZqC6XIIk0MO9pqRpiZnKWC4KSX4zb7ptRtfPjJKkB+4+PWgM1PTdSjAzBAqiPAJSIB0FQ0 4yvio6tmaohuD4Zy2SNbPruBFyXEqP6KrGMYliZGzDHm1M4PinqeB7ku3hiIDvEsZ3V8Xv 6cvk5SQdcqjGn8l/p3Udiehc2iJVmIM= X-MC-Unique: K12lP_fTMSycn3GH1ByPbg-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PiH9++EK85PyX3TRC6isqEvNAa5XVolEvvmHWXnptDiTS1v9VzfUzNfQnH+UY5IOIn3CuoQbdhMzVBr+Cvk92DvGvjgYD723kjRYw4813rlAHveluyz9owTP5ZK9wpcGQFtE92xgvZ7TorU/XyYrlT3sSmqsmh4fNoy9ichJcnmvZZWcF75RApPFBZZvT/5/CenULZL/6qROR2wMJ3sV6Fs+FCh1uRotFnn6e5R0gzeYDMoEkalonLR2vuPvko0IbKnwiwnMBLW2UyefsYwzB2FXjMpPDh92dWXWLxrfSdKdXmHWn2FaQivZ/S06VdvYyFowFf/yTsUzr5qeKKrcog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Zlfuedf2q4YXfgbpQ1MWK6FkdjlHZbNpmqaza4t7ui0=; b=fRR/fjHaKZvyftUmmyoxpGxqm0f7bBjHCOaE4R5nWWaR3AuNFlfbO3/2Luf4mSlAwVUYPRRG2ck+0vdPllauErqFP4f0wYwbZWYWYUenQyF3FfqyxbfCcGBN+xD3oYHUKTCxb/3fHEcRSQwkNdHibjUdOMkpsNBlldgyeNHfTcd4/qOlbjbU6XLjqejG+hcfNkjEXL7r9Oo2TjKgKmAJ6mxIm0wQEYLXpGYvv7vWciPNSQhpcfDJZLQfYwpvqTh0dHaMwfEYxL4PPrnEO5sRqp9bQRE5SXeVlDQSBMwyaEOKbrwayG5nBA5eVSDh51fCR2jY5h25w+MhtMAEdfavqw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: Date: Mon, 25 Apr 2022 10:44:11 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 19/21] IOMMU/x86: add perf counters for page table splitting / coalescing Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu References: In-Reply-To: X-ClientProxiedBy: AS8PR04CA0071.eurprd04.prod.outlook.com (2603:10a6:20b:313::16) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 312f22fd-5081-46bf-c97b-08da2697c446 X-MS-TrafficTypeDiagnostic: AM6PR04MB5736:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rR0LqJokPUj3LZ21D+k1gdv+6H/uRaLBQ/QkXNTEBMSch/Pho5Tl4jyPYHWOIpuoeT/hZmdHE1ZIt4klNhcq95rp7i3PDo80X+SfxQuJcUJ9uQ3Z4rXvTfUeCZ4n/MDLYNOturgTKri4MUg3xp5tv+Av9cDpF8Xg7wstLEhXWDGu3SywURFgOWzTyabEsMpBiW2H7sr832oNJO+z48ETDrkqiuAv+tfuRPQ80FSH8Brs50FHlKtgxx2GgI6KafGMvw9dgtiKxfqX8P2fYj1k9pP7yWsXkKAHuDDMFpyssolZCWcNCC+HLWFQtSTJViN1//XYIoqKokfhlpknISXO8dYXGLPZcmz+jmh7y6hjynfJvRR2e92uBMCkPf56NxJOl7eHBCq+7GmHQDl/TeQq7WZM4u1GjNt+j6+iPU56M2Tm3GYBkPHa2tnRytvG9kAAD5Hp7Un9HHRJo51vvusUkG8v7K7/CPXbWmuliEX31KHePM0mqiBONf9kiVakUG6aM6cK5HsFmQedSNLjtfdB6Dkv5naxMBjtXNMDtAELxGaUI/OXb610q9Hi+J4BZDZANot/c4wdqZRLtqosd+Y5jhaA8dMmOPHzzsaDvyk2Kq/B2HIkW1Tx91Gr4Fs7Bn93N7ETe4Otrbxrq6IA7sRTDfxha12bUHV3SeSGuOGZc4xJBeua4U8jOec0IoFtDBBQ1tGhWtYbJ4KKp0XlsSqpdpZHGnAfzi3p+Yye8JjA2Lg= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(54906003)(6916009)(8936002)(6512007)(31686004)(83380400001)(26005)(316002)(5660300002)(36756003)(6506007)(186003)(2616005)(2906002)(508600001)(86362001)(31696002)(38100700002)(6486002)(66556008)(8676002)(4326008)(66476007)(66946007)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?rcnNX2fzrr1dOun2GKkZDsFCiJOd?= =?utf-8?q?Fr+lTi2lNlgQKDMTcFOsgub5pga5pBTghx+ahvC7N19QbgUHVmfF2DKr5gS5D6BGW?= =?utf-8?q?3JrvMW8LINt2RNJaoW5GZafEbdftQReiysGqa48eOfxQnjAEDhM27yiPr0PdeXOyI?= =?utf-8?q?h5Qhejlyy10fqQwUIcMr6ibj1mNWwj3Cg48lFu0qRlkcD3aeGrw+5RpD9MOF9PRa8?= =?utf-8?q?TSOaIm7UHMziogDZcXXWuaSXfBV3V8eRmHndYlF01zWJaJL+m8eeNC64pAx+AQqDY?= =?utf-8?q?dXBf2BwJl7iiw9c6IazlgAChxjo9VGW5a1sBmFXaMJLdHckon/Y1s3aqQSdDrSe7m?= =?utf-8?q?65camcek9rLRbOxnwEXeF/a5d/zJcWml5/MOGLdS38guXmr46bWbggQMy7ozuHmSw?= =?utf-8?q?AUCzKfY697dQwwtAOfQ2Qu7o3pBuVEPEfmhOi2LHsAYMyz0XXfpuvoqIKsQNdiyx9?= =?utf-8?q?ZRMYKWaRfuh6jMyENN7qoBz2JnoC6TaKm6rGgmXGYc2GpozrBVFZ480y85ENLGcC8?= =?utf-8?q?InMd3LpDTQ/f8fmJx+r0k50EDq3cDsrif/cz8It2jZ9XYx/s2CIt8IINFddrY8Uge?= =?utf-8?q?GwePMZ5jJpYouMmbaRcGEn56YEd5E3QsbzI1w7/YKVtN0nLsfJr6Ca1EyCPHsO0zO?= =?utf-8?q?sZX5XndKgUkQDkjpgsmArlqP5e2FNcAqN7rtNshXWH42ZvZSgKFzWaEiDYMTJYPoF?= =?utf-8?q?vA0zsqVSApsD8SHu1u3acunjb8sN0zsE0q5f0HlACSd5eh/vh/NxXGJfv89RIVBJn?= =?utf-8?q?edkZ5ZqwGRDsnUfWybSOHTL/O2ogRWuV4Dv/6vkHNLzLI766VSMuQmUiO2gJGubu8?= =?utf-8?q?mjXaI/5ZNj95aqUwHE3NR8iTrGJ/+Y7pKR+l5AojI6B/zxWWvWQ5LAtxprDxjPskg?= =?utf-8?q?/mjbjmZDund9p8rYIGP2Sa9NxxSHUwVA0dloNLhwsZ/j5GFjqrPmqRG5q3KiCnL93?= =?utf-8?q?jAT32YmWBNYyu2jFx36cNXNPgbFFvYMY8TvZaB9pKF+78khcqX5R5nA3eNA+HnyiF?= =?utf-8?q?62+0WWT21NYtiEe8QNm+1MGf3XTXKKMM+zUtnl5k/FAnzzzi2zUDs/oIUW5DjymD4?= =?utf-8?q?pRwMLHQQDa+rvEtnoXuG1dhrVJOlNAxgcX1Ln+0XSfKTQdkHYVMae/n+RikqSDFwb?= =?utf-8?q?kImPfhD08uerzKNT1CnsePDKpC8aiRSHMKPKZvOEm/QQ2MHDKiheQFNRrinM9ek5y?= =?utf-8?q?ARlXQwgRo8dGvmuJvWx+7Am16daw9n/RkDUnFgCUeUm1VYTwsimwtSmjBGVVijDD8?= =?utf-8?q?GwVMKAFAle/+olo/oPYe+9E5FQLXXmtn8DrEK6jKzZI1jc+gYhTMQZPTc+PTUBDsa?= =?utf-8?q?YDBU0f/gFQ4cEtL9k+X/SLMA3invP0bZt7/VlWjAFqGN2AYrPyTwEz9bS59+VceQX?= =?utf-8?q?S2hx2/ZPIaynW/E3T8qnvl8Mnuxw1WknO8Y/HtlKVICRwyP55n2b2019j2Xo9zxCw?= =?utf-8?q?dWGqTNLvtYFbLJhT2cXnvZIbIiEVa83PQE97naHvk6qkN7UbdJcgOahjs5fP4JZbY?= =?utf-8?q?H89ELiDyOV19JTDR7hf/L156V7AH9IIvJJRjnkc/O/vhGMCzrLhJzxO6CEx2cA5ih?= =?utf-8?q?d4R+eHV1vgHf5Kv2PMOZh/x1dTg4895P9qsHGASW6pNSQrkL3Pp9o6QQEYSdfR9qD?= =?utf-8?q?Z8GHCSjKzF/jfKDJGti5nnW7m94JyEIw=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 312f22fd-5081-46bf-c97b-08da2697c446 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:44:10.2342 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: wmGHL2u8w04+ko7s6OOlm/kavCHzUpHAjWSk1g4TDrkMbTQdl7w5p9EBiQ2EPgvktGMS0MuoVV9CqrMc0LoXsA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR04MB5736 Signed-off-by: Jan Beulich Reviewed-by: Kevin tian Reviewed-by: Roger Pau Monné --- v3: New. --- a/xen/arch/x86/include/asm/perfc_defn.h +++ b/xen/arch/x86/include/asm/perfc_defn.h @@ -125,4 +125,7 @@ PERFCOUNTER(realmode_exits, "vmexit PERFCOUNTER(pauseloop_exits, "vmexits from Pause-Loop Detection") +PERFCOUNTER(iommu_pt_shatters, "IOMMU page table shatters") +PERFCOUNTER(iommu_pt_coalesces, "IOMMU page table coalesces") + /*#endif*/ /* __XEN_PERFC_DEFN_H__ */ --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -343,6 +343,8 @@ static int iommu_pde_from_dfn(struct dom level, PTE_kind_table); *flush_flags |= IOMMU_FLUSHF_modified; + + perfc_incr(iommu_pt_shatters); } /* Install lower level page table for non-present entries */ @@ -472,6 +474,7 @@ int cf_check amd_iommu_map_page( flags & IOMMUF_readable, &contig); *flush_flags |= IOMMU_FLUSHF_modified | IOMMU_FLUSHF_all; iommu_queue_free_pgtable(hd, pg); + perfc_incr(iommu_pt_coalesces); } spin_unlock(&hd->arch.mapping_lock); @@ -532,6 +535,7 @@ int cf_check amd_iommu_unmap_page( clear_iommu_pte_present(pt_mfn, dfn_x(dfn), level, &free); *flush_flags |= IOMMU_FLUSHF_all; iommu_queue_free_pgtable(hd, pg); + perfc_incr(iommu_pt_coalesces); } } --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -404,6 +404,8 @@ static uint64_t addr_to_dma_page_maddr(s if ( flush_flags ) *flush_flags |= IOMMU_FLUSHF_modified; + + perfc_incr(iommu_pt_shatters); } write_atomic(&pte->val, new_pte.val); @@ -865,6 +867,7 @@ static int dma_pte_clear_one(struct doma *flush_flags |= IOMMU_FLUSHF_all; iommu_queue_free_pgtable(hd, pg); + perfc_incr(iommu_pt_coalesces); } spin_unlock(&hd->arch.mapping_lock); @@ -2244,6 +2247,7 @@ static int __must_check cf_check intel_i *flush_flags |= IOMMU_FLUSHF_modified | IOMMU_FLUSHF_all; iommu_queue_free_pgtable(hd, pg); + perfc_incr(iommu_pt_coalesces); } spin_unlock(&hd->arch.mapping_lock); From patchwork Mon Apr 25 08:44:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825412 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 41C65C433F5 for ; Mon, 25 Apr 2022 08:44:53 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312594.529918 (Exim 4.92) (envelope-from ) id 1niuKo-0007Yd-7c; Mon, 25 Apr 2022 08:44:42 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312594.529918; Mon, 25 Apr 2022 08:44:42 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuKo-0007YW-4W; Mon, 25 Apr 2022 08:44:42 +0000 Received: by outflank-mailman (input) for mailman id 312594; Mon, 25 Apr 2022 08:44:40 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuKm-0006Gx-Pj for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:44:40 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.109.102]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id f1c49a42-c473-11ec-a405-831a346695d4; Mon, 25 Apr 2022 10:44:40 +0200 (CEST) Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01lp2059.outbound.protection.outlook.com [104.47.2.59]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-39-amMb_LFZOMu6XEUgtkPj9g-1; Mon, 25 Apr 2022 10:44:38 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by AM5PR04MB2977.eurprd04.prod.outlook.com (2603:10a6:206:a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:44:37 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:44:37 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: f1c49a42-c473-11ec-a405-831a346695d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650876279; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VKOf5D5gFHYc8NiI8kFeZJqK9EWiOdTIy+CLucDl34s=; b=V9WX0lV0t2sza+nXm5ROHZ5dF/dsKoPBlSMFulcSTFGgWlqjo8m9uqKIHyv0bJgtNFSAKd ZZt1yrx/fJVm1GosY0GD7ayTovFpimClo0NodGWSRzuC1fCRKjjDpFI/ptIMD7oRSN3JvZ WjQENp91kO78ObeQp0HZroxjnbBw1Gk= X-MC-Unique: amMb_LFZOMu6XEUgtkPj9g-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DFKomPZ0ehnH2Tv7nR1rxT+Thg+o1r1+ZOU22OVWCYKdsYc+V3bVmmC7nI7WXSOSwLCNW9fy3wUSyYDt5oXW3MB/IXkpoe6s+XIrlNEgNoSHWtWSRNOOyWchT3VPrGIiD19j7K5kjpCxpK87xf1yV684dmEr8HmC5EWpqV9ZKG9gbMf0rUrvkJG+M1m58cmeEE+1fzRS1PFLU6MpIRx5A7zhkPTAQfhDoGS0RyTO1qRKzSpC8pgO6QnaXiZIDb0Uakx5WY+ZmrI9SMdapC940hSVyZ4V4Jo8zb5egnp0ynrXKSjVRiJg01aF/AvhLBQ1BmZ8LqCG2zLRgWPaO3T8Lw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VKOf5D5gFHYc8NiI8kFeZJqK9EWiOdTIy+CLucDl34s=; b=S7aFEpYdwpOcgky23RyKTWbkFXfV+8rfPhhvKiiwi/55q5PgdrPkY0V7mii3l7J+bHimPqFme3we4ukKnPpS8LFMsPQKmTWbnfB3m7UyKQOcRJgqjONfK0o9Xj1eNB5p6gNnMFVVvUgpEdl0BvCc/39mv23S6kay7IReWJEGx+0fXZiTXBM7PszCAcd35EWBbdpyMetrR9Rt9YA8LL006CO+8FhXTBJXf95n9cdwnNyGc+H2eHF3WmHG4zqoX5KbKLyScV7AQYHXRN7BHwncFKkK/4Dwfy6NJEOW7nHWD2ZVcg5gNSfDmGOHiaA8AT/8PFC8dxb8ZxTdDAA1QIjRpA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: Date: Mon, 25 Apr 2022 10:44:38 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 20/21] VT-d: fold iommu_flush_iotlb{,_pages}() Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Kevin Tian References: In-Reply-To: X-ClientProxiedBy: AM6PR04CA0024.eurprd04.prod.outlook.com (2603:10a6:20b:92::37) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 70d23c8a-15b3-4f4d-dfad-08da2697d46d X-MS-TrafficTypeDiagnostic: AM5PR04MB2977:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zClWtB2CE9VAH9kzgPKpN/kq1Dx1PNtOm0os0MInwnVppE4seVQOFtVb5OFr/MG8lQK5wBQ2xHsRRe6lG4ong4gYWkVRE5hPHXlZLzydgGniVLjbzKSC6ZfJCkRQlAG7RNB6HyR7a6YN7vq6hJ0MvleWZkW00r9Yo1voVU+9uzap24JrNTRs7vg7S4XNb2pF6fJT45vs5IKT2YCSiF7Xuv5RQOikicMaOSEtzxj6Fl/oKsw6ps97UArEkt9bmWLbU7TndWY8sQAvAbbNkCnVAbLScZTIOlpdo6ITl/oKpJBWotKvjBmcOvBvIiXfSkq4BAj8vNaoiTbwXcjxlLOZGuP0ZhR1RQiLqFtaBYtvngIeB/aaZ3/eTQDojxG4PlTudw2eiLu4Xylo+a4XDl+wNIo1uHzbjReMfbF7NXbpMSmFe+C7wY69ZlYMTOSqOlXTbBIW4GwJl7S2von9anyHxWiPW+MTWxK6QteCyjP7clekbrVgdPcViMT5PNi6Emy3oK9U+nIXAqCin7gEbLbIHF/YJZRrofIEgZ5g0DLR/XI3vdajh2QvtsuNa9nMLSJ9ckWaA5ItXsNVREJqeRABr5NUC5Wffe1F27qXaCEjHrchYmXbvJJWBYZ8SHW+kke6aNjxXZYGjG7TJCPQyf+6y3K3o3wuix6MxpZ5fZ3k1CH61cMuJNql4x7QvBzvxOm4ORVIYrnNcHHvaHvEqwzoHty3HKjkiMKMa3ASnc/9YZo= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(316002)(6916009)(508600001)(38100700002)(26005)(6512007)(6486002)(31696002)(5660300002)(83380400001)(36756003)(86362001)(8936002)(31686004)(2616005)(2906002)(66476007)(66556008)(54906003)(8676002)(4326008)(6506007)(186003)(66946007)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?7pNbTMYPpnHhB6ORh1J4ymH9WW63?= =?utf-8?q?7jyth5d5V0zuXDuCIRBnqXz1yotSIRunXO32IzxUYCdpY9G2azaZsRxzWsyDXsmVN?= =?utf-8?q?URvxPZWfMCuQEdO2nb9sMk67zNCw95+g8/47Fzi5mUEwbXpCJI2BYkhQazJR2lAHn?= =?utf-8?q?2zGy/1tgRSQDrSfKIKFKcke95NdzSQA0ptOZChgT3J4wABde35oomX8ef/v6LW/ci?= =?utf-8?q?WXSbhErPfeTZ7IOuG+HjKMyUumUcySdcW56B3l5ZP/25IvxrY7xuvDCQsAx2tCu7D?= =?utf-8?q?iOQ64JtE3A7QP2rpyNCtqVaVf7zeeeXbFUutv/BkFl/D52mYaQRFuQn25UefJTZOV?= =?utf-8?q?SNNN6vAZCpOVpeEtS3CmSUx5CPM9hrC+cJtzT3tdDuSLvL7nvDypVNQy/yxcwYtH1?= =?utf-8?q?JohUQ/IYfx1c/LRqP8Pt0CxbrLNIRAHn1Ch5ZtdxR8Qp3O/B3c88KUkwktwajoqXi?= =?utf-8?q?YLA3iAedeEG7SA3kysXPAABYlHR5O1S4UmStzv3FYXZ0UdqcI7NKtUcKC8lWOttKb?= =?utf-8?q?N8DAbuuORt0PYTDBY4hDdAKu7aR3l23UvYqaI4rCCvRuhtLogM6EIhO3BZ84fkrDe?= =?utf-8?q?fFADowZziVsaQnuplpInJjkMJwy/A4e941gH2FJoeGTnPL6w/aq5HV5qdEIvqQBXf?= =?utf-8?q?QYfu7ILz50CDqolXRBDwTpwN/Cg3ZX+tBXI9dRFUuYsc3fFVVi2PjoH0vJDhqv56v?= =?utf-8?q?bi5zzHwQG8posdJUWF+y92/F/72Hl8JobSesz4CJIchQKCAjWhTQACFKaWojYmNYt?= =?utf-8?q?mADk0xAV+BZE99RkEcLbChhVxH4ceE8dlAfj/RKrbOYoaI5G3pq6B7aAXCa4NVMrq?= =?utf-8?q?w1s3pDXGDeJ7xUldZfpuJSMrCHV/K/yzrniua/LdCYNBj+vJ/kD9qvTAcyzUDmR9U?= =?utf-8?q?OeDV1JE8kfoxj80vSYICv8gBeFKJLOFwJUwC0CvlaTvm18z8Q2lY4RNDIC1D8ZpyC?= =?utf-8?q?K7I8VscK0KQlERdVyEyMgmShcURMjfhrTnGWviu9Anjk4H9xzUo5Toq+REPaAu8Rn?= =?utf-8?q?W4VtmTX3OKoMm3U57LAGfqJ0q5w/L9/JEKL0qovRAkpLbzhWuKMg9PYV7nl+almcL?= =?utf-8?q?FaukvHnRmvu3ZlVbuHVy5NbkJ4tdTX0DmDF1gc9C1cIAtXkoS6yJ3tHKnsKDWLdFV?= =?utf-8?q?TGOOaOWtb+vK1trqPFa4yMsDZuCaNy5xz0xuhXQzapZBfjuSn+LCmuhje6ULg3nt1?= =?utf-8?q?/qOXlfyG8MLBZ3Upz9z9M/zIi0CI3xxRVKjOcGWAud2yZpyaKUlpRAoX/0hSEyx+o?= =?utf-8?q?0ACf8Zkrf5VMB50hHxhO5FFZ+Q7kTE7FCSajofwLkL9mhQ3g9r49tuOG2ORk5520j?= =?utf-8?q?Bfw5L6pwypabHIQbpMMDpLIoxOHsfgcyfkNobjotPQYMAHsnWbVW4kz6QWIRGn+Sa?= =?utf-8?q?yeY+A/NdluMRH3xrW/yHouA0FaQwYHIcVVXZ1BrvvHa+ORsp3as9YDowpwewY/tIC?= =?utf-8?q?Bp10JfQqzEBzBX4ebo+EoadzUwMcyGOgf7rksUpa7MZQUHx/xAB9rmSh2WequmiKq?= =?utf-8?q?4m8+uAiw1zvtHYTgGcJRwIAPn7mJmMzgoZFq5Hv30WuqeDcP9gc5cP4FyMYXigFb1?= =?utf-8?q?DoDoE4InzLF9bPfT1SU5+O5rT8u/xIzJIt1q1dNEPb4yWuSGa02cHPEhdUwCG0kXf?= =?utf-8?q?UTCeAHmKU2ZxuW884EWV2qG4Wk0UYt4w=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 70d23c8a-15b3-4f4d-dfad-08da2697d46d X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:44:37.3171 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: k7fAhFoevg0hQXWekoohtYGS79unbsX5B0enOzQDCkaNq6Bo8GhownF1cPEFJG0IIjzLwpci9yftLEm6kAb7cg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR04MB2977 With iommu_flush_iotlb_all() gone, iommu_flush_iotlb_pages() is merely a wrapper around the not otherwise called iommu_flush_iotlb(). Fold both functions. No functional change intended. Signed-off-by: Jan Beulich Reviewed-by: Kevin Tian Reviewed-by: Roger Pau Monné --- v4: New. --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -728,9 +728,9 @@ static int __must_check iommu_flush_all( return rc; } -static int __must_check iommu_flush_iotlb(struct domain *d, dfn_t dfn, - bool_t dma_old_pte_present, - unsigned long page_count) +static int __must_check cf_check iommu_flush_iotlb(struct domain *d, dfn_t dfn, + unsigned long page_count, + unsigned int flush_flags) { struct domain_iommu *hd = dom_iommu(d); struct acpi_drhd_unit *drhd; @@ -739,6 +739,17 @@ static int __must_check iommu_flush_iotl int iommu_domid; int ret = 0; + if ( flush_flags & IOMMU_FLUSHF_all ) + { + dfn = INVALID_DFN; + page_count = 0; + } + else + { + ASSERT(page_count && !dfn_eq(dfn, INVALID_DFN)); + ASSERT(flush_flags); + } + /* * No need pcideves_lock here because we have flush * when assign/deassign device @@ -765,7 +776,7 @@ static int __must_check iommu_flush_iotl rc = iommu_flush_iotlb_psi(iommu, iommu_domid, dfn_to_daddr(dfn), get_order_from_pages(page_count), - !dma_old_pte_present, + !(flush_flags & IOMMU_FLUSHF_modified), flush_dev_iotlb); if ( rc > 0 ) @@ -777,25 +788,6 @@ static int __must_check iommu_flush_iotl return ret; } -static int __must_check cf_check iommu_flush_iotlb_pages( - struct domain *d, dfn_t dfn, unsigned long page_count, - unsigned int flush_flags) -{ - if ( flush_flags & IOMMU_FLUSHF_all ) - { - dfn = INVALID_DFN; - page_count = 0; - } - else - { - ASSERT(page_count && !dfn_eq(dfn, INVALID_DFN)); - ASSERT(flush_flags); - } - - return iommu_flush_iotlb(d, dfn, flush_flags & IOMMU_FLUSHF_modified, - page_count); -} - static void queue_free_pt(struct domain_iommu *hd, mfn_t mfn, unsigned int level) { if ( level > 1 ) @@ -3254,7 +3246,7 @@ static const struct iommu_ops __initcons .suspend = vtd_suspend, .resume = vtd_resume, .crash_shutdown = vtd_crash_shutdown, - .iotlb_flush = iommu_flush_iotlb_pages, + .iotlb_flush = iommu_flush_iotlb, .get_reserved_device_memory = intel_iommu_get_reserved_device_memory, .dump_page_tables = vtd_dump_page_tables, }; From patchwork Mon Apr 25 08:45:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825413 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 298C5C433F5 for ; Mon, 25 Apr 2022 08:45:27 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312600.529928 (Exim 4.92) (envelope-from ) id 1niuLK-0008Be-NK; Mon, 25 Apr 2022 08:45:14 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312600.529928; Mon, 25 Apr 2022 08:45:14 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuLK-0008BX-KF; Mon, 25 Apr 2022 08:45:14 +0000 Received: by outflank-mailman (input) for mailman id 312600; Mon, 25 Apr 2022 08:45:13 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuLJ-0007xd-DJ for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:45:13 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.111.102]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 04f4af2b-c474-11ec-8fc2-03012f2f19d4; Mon, 25 Apr 2022 10:45:12 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2057.outbound.protection.outlook.com [104.47.12.57]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-30-883xrxsdMsqnka3G6lzhlA-1; Mon, 25 Apr 2022 10:45:11 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by AM6PR0402MB3589.eurprd04.prod.outlook.com (2603:10a6:209:3::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:45:09 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:45:09 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 04f4af2b-c474-11ec-8fc2-03012f2f19d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650876312; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jXroAWCzWdcYKf4JkOt1e3tm2yhGlKfIfFmJbhMkY+A=; b=kr7KnmwIPS0SFGtkzRGxOcuaquHDQMxy0Lrudl11hcm1TcEjrmkizs8kXqIRGIHSTPJqhf 8V4g1WhjsU1gx1tiHB5psTZX9/meI+KkktqWQifKM5PWTX5hsTbt/PnscP/MDj3yBQF3jT FlCyG0Y5RJ5bDFy500i5oxW+nCRGKXg= X-MC-Unique: 883xrxsdMsqnka3G6lzhlA-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HInOMlI4Bg+9oFbVpgietINA1BzkEzKrxHWbmndiaqi9B2OYq9MmpV9uHvIn5vMOiZ/s4hFehxZ/hKgpgilB2YXr85ajz9FaL5dHYcbti/5Mtwhe2jv0HObKRK1kkasPcqcu9NmZgM/PSBb3y6NdeDNDkMK5Olu02U/iCX4W+XzOsstZvCyDtHdGIJSAU2SgRdvrXXusN+zqLW/5SLvjX55+bFVdiukUqeEonmFAAA8wiS3onyomkvSWqwBAKrrB0O3svKrdUS1llNOoAQUJNC/iO2cDHBT89XPjxp3AqaulG1uMnr+2pCsVvCy7W0tGbm/Ons5m0Pbx8lMSiVK3cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jXroAWCzWdcYKf4JkOt1e3tm2yhGlKfIfFmJbhMkY+A=; b=UJmktIb5WKgJhWQl2o7vw3K/tgnDvU/9G/Rm3SnqXQCiSIRtaNcuYU11TtrmNCoWACN7o8F+QOl5FpEE5nKqDa8hJks1V6n01PiUuxx+rcDoG0TX6GxakFG3DcWrMykV84mArem5Nb0MOrl/1qk00ttzwdD0Cwf8vtecea7VLpxGsYZs98EULABok0++/L5TW7dP85vexLSJVFdKu4vjr/bMrHo+05soCwwLlwceVzyOAHuXdDunsv0gS8HPoXx6wXkOv8IKtoPK4x8NZJrZM8YU7g3VTyd5ETVqDUVniiTrqeAoBBwme9WbmOm83IiTIWT4mses2pRL+JkDF2xu4g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <4a24a85e-267f-9de5-4009-b32b9ab8aa0d@suse.com> Date: Mon, 25 Apr 2022 10:45:10 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 21/21] VT-d: fold dma_pte_clear_one() into its only caller Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Kevin Tian References: In-Reply-To: X-ClientProxiedBy: AM6P194CA0064.EURP194.PROD.OUTLOOK.COM (2603:10a6:209:84::41) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: abf36fe3-1522-4f9d-a676-08da2697e768 X-MS-TrafficTypeDiagnostic: AM6PR0402MB3589:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yPzY50RVEYMGVEFRJHYq9HhHq4nVfj0OeqWX5vARnB16yOCSpV6ggN25iBkI8FrV8bzRAVbeYign74vdpzDp3lcVf1VIS4tY7c+uBQmKp89oy89E8guMr0AXZ5rEwLax1L6ACdVA2uTeHOU3x15ZhGyeVRtO7DhYMXIVFeNf/vus7iUbsQSxXMOzkAwFTMMag5qdriUpRG5riTzS6F/1KE0ZV6E3mRFDrAGG6wxtjQ/KEkNwCycV/ncYcivEyFv/7mEsgC0mD6hWQNV9fTd6ZU0UHQGArnxgO4NIQiY8g37w7WoTX/JDe7ozL0gGHzb0tx9JaB5+Tf8t/JYqqjsVVdbseFFpFCUHXCekUR6DgyOfC5rwsl0F9gj/ifzLEjEntzilbNQAUhSoBDhzTDXEdHbxfNPT1eyOefJliQshLxEFCkQYIxcCKq1rVroKCJsHgLsLfJW4A2xx19bElrlD5L4Xb0WZ4ebJ3u3YanwhZq03B5ZUetM7Pq1su/12/hJbzYkOD/X2+RCJJ0WCeaEBzCVTacOVTwsxkZJWU18ZhA0hvxHrOQgmHA5Dqwc75/8NvE1OWIYLP6r9omoJ7MqDwpThlp1RCHIZbH1J5pQCHawW05iiGVAm++R0g7GUqQPxJsUJrwSP/15z3i05Qi18TE+NO/+yRh3Kd0wpURWpO7ARYt58F6PBz8HH8WeX8j0ZldUwc1IQ0YtpYM5IS+MJwAtVHh+pf+zoIPnDIFytukQ= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(86362001)(54906003)(6486002)(5660300002)(508600001)(8936002)(2906002)(31686004)(83380400001)(2616005)(186003)(8676002)(4326008)(66946007)(66476007)(66556008)(26005)(6512007)(6506007)(6666004)(316002)(6916009)(31696002)(38100700002)(36756003)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?dkxfX/HE82776VzMYbOo8yq3YGAS?= =?utf-8?q?TreBT/+7v/ldNlxAERYgeH3aa0y5FwhPNUOIPR26sIdV3Lte+yQHAj4HZ9U/9zbTb?= =?utf-8?q?iKy6C8l4+XBh7rqVGD4rNwnigse0mkYsP7+ubnPqWqO/AeWe4j0jtmwxgfoehe0U7?= =?utf-8?q?vuszn1+4JF71rZgeQTAtdkJaIwGMNo+7MX6AXqebQFXjf4ZWCiocZXLWXAQP2cHNE?= =?utf-8?q?fMVwujdOaGLoxevVAXenWJUNEZyVlPSBHU0TZ+hjyJFW7v1s4IeM/VyL1OLJgfdus?= =?utf-8?q?9vOGhbqZtjTVAPnQineKqHyd2h4Q2VA8MsSFzPmt4F2wN7pXpq+JcklHZndLdiJ2y?= =?utf-8?q?c0dQ2T9FM0RTLzfqVIfQXe9QEen67eUgWPfCyPVbEqJsoREHT+HZ5GM3+SXtGG6HK?= =?utf-8?q?FvA4QkUS4VFu8FH6uyFDuvgX3SHojVPeLvm2oT2ttW1/1mnv7Hd5gmDJAs/9GRDtc?= =?utf-8?q?X6hqJiOC+JifKa2/8xliE8I5FTvjz1J4gpcCWdOrgv22DmPrcybDCOz9iq2FvUTlu?= =?utf-8?q?ijo58y2Ch9bfKxFJBBnf1NmbF0xK7mo2IugwMYW3akRXGY0vg0nennjpqhqv4QAnh?= =?utf-8?q?8ycWeTRnXfLZUMYNEFp0IuAeQ0iubgFNeh811wveAiShH7mOE5myE3GzWcZtozKw9?= =?utf-8?q?Eb5J+08ZIQSgebVwFI+zHGD1+3qkcrOlGTMPQDi1YCTTD2J58rMZMPl8cf7xVj+eX?= =?utf-8?q?v4hP6tDF0yOAqq14lxeEi/ZD9hoQg+fzFApi73vk8CJIvWj+gFba0sWQZrPDa1rRT?= =?utf-8?q?sa+SibVNqEVtrejNgbKmlL5URAeUwZiP/UsRkGOI/l8LjusuOjqn0KXjXh3xqVMKv?= =?utf-8?q?RwSA/nLtnlMohtiJZFrTAsZmCTMxEZhhnyIzluyergnYOSoeJc2pubpieWdJZ/Sa4?= =?utf-8?q?vf1+VsfnO9Uerw1ARNKhzHPjwIsumeJ2JbesdKwgESKEpQty6dbY+kT6/vAFuHTFc?= =?utf-8?q?/Y77qM6/dEZ2Kq5n/dcYRaRVwRqb/QccgBYDB8xjf8yAdMisMWMPiHgrRnAvkHcuH?= =?utf-8?q?UDbGH10a6flNB0ea7qbK21pnsAAopmtqvQmJfJJT9dAL/RJ+Gl3F8lGAWgNbObdLu?= =?utf-8?q?ZzkkuTC62kuOKhesTtmTrK1lQlEc4A4CIY2R196YFyHsBeHDf1iSXcLwYqFT252gu?= =?utf-8?q?ZVKM/qvHEOocGMF9u1C3bXgZEYISAnE+ji1aeFJEitnBjt6DGKQwoIPODGIgJG+YK?= =?utf-8?q?wcAH0v/AbsV0dh0AsBFO1u8fOzjHryi7nsEIecsR2L/1/3EqJgle0usPNlwfWWlST?= =?utf-8?q?uRB6d27uBoCdYFb6srISBx3wHm7+D3gQBwpNueYRGJUmKALM9lykN4O5hgBz9XSi9?= =?utf-8?q?2mtpqFawbwueK0YB+YwqcHqHaM5FeoWr84fQo3kY8fATq0QLmDHPVdaaFR22N7pSz?= =?utf-8?q?8gxEnxNuUFjstQfjl4GP1ZV1dJWPhQyqjx/VduXfSYKr2dDsJzg1HJ7aIF78Q3Mmv?= =?utf-8?q?XWP1WxQRdO/vPVgqd0H13AarzSU4ldDp19SZ+BCzRtNEaoysAoz0HL3XgmAR7ZGWz?= =?utf-8?q?f02idkt1pf5MlCoNVh0cS/S1ARM/WAYrsria6PSj/LbZsfYsG1QwMArLwv/K6AG4W?= =?utf-8?q?InMSJDf8hHrhrTTv8gB6nS4ZbQP0YxQsPYGcOxC5u1b7cnC0LunLQTunyrrclfzS0?= =?utf-8?q?SRL4tNZA/r29RjEsOzIWoacVve4C3WtA=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: abf36fe3-1522-4f9d-a676-08da2697e768 X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:45:09.1596 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: WTxGIlSfltNLOmE5j3Ebd+5bNFOtLGcC3NzX3aoayucvjIQmLH6Rw/FcV2ERoBJekHWtd19Yw2TBvDOyCxM1/w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR0402MB3589 This way intel_iommu_unmap_page() ends up quite a bit more similar to intel_iommu_map_page(). No functional change intended. Signed-off-by: Jan Beulich Reviewed-by: Kevin Tian Reviewed-by: Roger Pau Monné --- v4: New. --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -806,75 +806,6 @@ static void queue_free_pt(struct domain_ iommu_queue_free_pgtable(hd, mfn_to_page(mfn)); } -/* clear one page's page table */ -static int dma_pte_clear_one(struct domain *domain, daddr_t addr, - unsigned int order, - unsigned int *flush_flags) -{ - struct domain_iommu *hd = dom_iommu(domain); - struct dma_pte *page = NULL, *pte = NULL, old; - u64 pg_maddr; - unsigned int level = (order / LEVEL_STRIDE) + 1; - - spin_lock(&hd->arch.mapping_lock); - /* get target level pte */ - pg_maddr = addr_to_dma_page_maddr(domain, addr, level, flush_flags, false); - if ( pg_maddr < PAGE_SIZE ) - { - spin_unlock(&hd->arch.mapping_lock); - return pg_maddr ? -ENOMEM : 0; - } - - page = (struct dma_pte *)map_vtd_domain_page(pg_maddr); - pte = &page[address_level_offset(addr, level)]; - - if ( !dma_pte_present(*pte) ) - { - spin_unlock(&hd->arch.mapping_lock); - unmap_vtd_domain_page(page); - return 0; - } - - old = *pte; - dma_clear_pte(*pte); - iommu_sync_cache(pte, sizeof(*pte)); - - while ( pt_update_contig_markers(&page->val, - address_level_offset(addr, level), - level, PTE_kind_null) && - ++level < min_pt_levels ) - { - struct page_info *pg = maddr_to_page(pg_maddr); - - unmap_vtd_domain_page(page); - - pg_maddr = addr_to_dma_page_maddr(domain, addr, level, flush_flags, - false); - BUG_ON(pg_maddr < PAGE_SIZE); - - page = map_vtd_domain_page(pg_maddr); - pte = &page[address_level_offset(addr, level)]; - dma_clear_pte(*pte); - iommu_sync_cache(pte, sizeof(*pte)); - - *flush_flags |= IOMMU_FLUSHF_all; - iommu_queue_free_pgtable(hd, pg); - perfc_incr(iommu_pt_coalesces); - } - - spin_unlock(&hd->arch.mapping_lock); - - unmap_vtd_domain_page(page); - - *flush_flags |= IOMMU_FLUSHF_modified; - - if ( order && !dma_pte_superpage(old) ) - queue_free_pt(hd, maddr_to_mfn(dma_pte_addr(old)), - order / LEVEL_STRIDE); - - return 0; -} - static int iommu_set_root_entry(struct vtd_iommu *iommu) { u32 sts; @@ -2261,6 +2192,12 @@ static int __must_check cf_check intel_i static int __must_check cf_check intel_iommu_unmap_page( struct domain *d, dfn_t dfn, unsigned int order, unsigned int *flush_flags) { + struct domain_iommu *hd = dom_iommu(d); + daddr_t addr = dfn_to_daddr(dfn); + struct dma_pte *page = NULL, *pte = NULL, old; + uint64_t pg_maddr; + unsigned int level = (order / LEVEL_STRIDE) + 1; + /* Do nothing if VT-d shares EPT page table */ if ( iommu_use_hap_pt(d) ) return 0; @@ -2269,7 +2206,62 @@ static int __must_check cf_check intel_i if ( iommu_hwdom_passthrough && is_hardware_domain(d) ) return 0; - return dma_pte_clear_one(d, dfn_to_daddr(dfn), order, flush_flags); + spin_lock(&hd->arch.mapping_lock); + /* get target level pte */ + pg_maddr = addr_to_dma_page_maddr(d, addr, level, flush_flags, false); + if ( pg_maddr < PAGE_SIZE ) + { + spin_unlock(&hd->arch.mapping_lock); + return pg_maddr ? -ENOMEM : 0; + } + + page = map_vtd_domain_page(pg_maddr); + pte = &page[address_level_offset(addr, level)]; + + if ( !dma_pte_present(*pte) ) + { + spin_unlock(&hd->arch.mapping_lock); + unmap_vtd_domain_page(page); + return 0; + } + + old = *pte; + dma_clear_pte(*pte); + iommu_sync_cache(pte, sizeof(*pte)); + + while ( pt_update_contig_markers(&page->val, + address_level_offset(addr, level), + level, PTE_kind_null) && + ++level < min_pt_levels ) + { + struct page_info *pg = maddr_to_page(pg_maddr); + + unmap_vtd_domain_page(page); + + pg_maddr = addr_to_dma_page_maddr(d, addr, level, flush_flags, false); + BUG_ON(pg_maddr < PAGE_SIZE); + + page = map_vtd_domain_page(pg_maddr); + pte = &page[address_level_offset(addr, level)]; + dma_clear_pte(*pte); + iommu_sync_cache(pte, sizeof(*pte)); + + *flush_flags |= IOMMU_FLUSHF_all; + iommu_queue_free_pgtable(hd, pg); + perfc_incr(iommu_pt_coalesces); + } + + spin_unlock(&hd->arch.mapping_lock); + + unmap_vtd_domain_page(page); + + *flush_flags |= IOMMU_FLUSHF_modified; + + if ( order && !dma_pte_superpage(old) ) + queue_free_pt(hd, maddr_to_mfn(dma_pte_addr(old)), + order / LEVEL_STRIDE); + + return 0; } static int cf_check intel_iommu_lookup_page(