From patchwork Thu Jun 9 10:19:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12875172 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6D81DC433EF for ; Thu, 9 Jun 2022 10:19:57 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.344904.570525 (Exim 4.92) (envelope-from ) id 1nzFGS-0003UR-GQ; Thu, 09 Jun 2022 10:19:44 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 344904.570525; Thu, 09 Jun 2022 10:19:44 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nzFGS-0003UK-Di; Thu, 09 Jun 2022 10:19:44 +0000 Received: by outflank-mailman (input) for mailman id 344904; Thu, 09 Jun 2022 10:19:43 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nzFGR-0003U3-MK for xen-devel@lists.xenproject.org; Thu, 09 Jun 2022 10:19:43 +0000 Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-ve1eur03on0615.outbound.protection.outlook.com [2a01:111:f400:fe09::615]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ad4bb61e-e7dd-11ec-bd2c-47488cf2e6aa; Thu, 09 Jun 2022 12:19:42 +0200 (CEST) Received: from VE1PR04MB6560.eurprd04.prod.outlook.com (2603:10a6:803:122::25) by PAXPR04MB8559.eurprd04.prod.outlook.com (2603:10a6:102:216::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5332.13; Thu, 9 Jun 2022 10:19:41 +0000 Received: from VE1PR04MB6560.eurprd04.prod.outlook.com ([fe80::dfa:a64a:432f:e26b]) by VE1PR04MB6560.eurprd04.prod.outlook.com ([fe80::dfa:a64a:432f:e26b%7]) with mapi id 15.20.5332.013; Thu, 9 Jun 2022 10:19:41 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ad4bb61e-e7dd-11ec-bd2c-47488cf2e6aa ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OSG56MYUeDTBjUZtKwynHQfPfGEbVYGWxTbP6fxJb2xBFGwpc8RPGlHipIOoRbTs2z+84O98tlYeDS2415adZqc9yrH9Q7JATFDiNQJTn+cgu67jCEQ6z8B2YbkGwi/sjFkXJOyhZmx518gTx5qOw7hdOHB2yEjHsW8If/Qdh4TaN0QToQNwIiWM5QY5qR/skCVaR19bxGYnbfGyvzEM/7Kn3I8rh0ptF0ZpNw9OLAWlY7HRVK/aMFD4agRFBbO8toP3iuG0x1nr/c5+kzM4ZPjxw4sEdnqapD6eTZdMq8kqkLTqqCfypLul0OXwEG4f/fpKR+StgEmfRHU7YESrfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=i1SRbMZ5vtUn4A75NysmPVCeWEjl1KBkL8Pu3NNkQeY=; b=QUtiBQexOROx22dYmxZBzyjk2js3wpx89FbfdPCf8rMCZtTY50PYyAyB+ifoyFLuhYDYXnkckF5gQkED1WsaZOJYRsL6abetr1Zy/GOkmCcFQA0dNXgctURr3uHKXRmWUuRRgXPYzr0g4uah9tMo3zuJdQZ7nfONHnORaffzeQ/qKm5BMJhnTsfSOmxKBXiJehseRrf5/abjQ5HdChlhGK7H+dFxJDoDbtV+jex4dg0Y2iqQ9w1pkKP92KgWRT7S25qBZqmhnmwHocEKw7dhF4XOqKAzjTTJYanip2LunAoT7xLQSGZpaYo07BFoA0HUztibU4pce5LTa23nycInhg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=i1SRbMZ5vtUn4A75NysmPVCeWEjl1KBkL8Pu3NNkQeY=; b=4RD3TPjgOKvP/ILG99eZi6mlFo40A8sfUPofFnLfdgg/N5df/3FsMxA+6EzC80EDGAotxahVFTMwceuAUmGik0JsTQ59oTTbqMdQuf+7i1bdIZt3rv9nEnrMDDXAnjf4nrIX/gITLkwnrKBU/mO2UNlwbhulFc7FfgTQqe4Ky/Ns3PZGkaLRX0InBOvJFhbdzb6/vnD/tXDteyzM16UBGuIcElAHTjfAgHEw42sDbX50YR4EfIWk7KIa8+OgdEIgGTYyrjsD7gDBzE7xr9gpxYyzYddi7OR5hI49kBx8N5uwhaUbd090v2i2FP6zPn+DqIUFh22AahTa+1k+je9Z3w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <730648db-7a2b-ce0c-675d-cddca116474f@suse.com> Date: Thu, 9 Jun 2022 12:19:39 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 Subject: [PATCH v6 07/12] AMD/IOMMU: free all-empty page tables Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= References: In-Reply-To: X-ClientProxiedBy: AM7PR03CA0028.eurprd03.prod.outlook.com (2603:10a6:20b:130::38) To VE1PR04MB6560.eurprd04.prod.outlook.com (2603:10a6:803:122::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0449f426-36e3-4ae6-e953-08da4a0190d1 X-MS-TrafficTypeDiagnostic: PAXPR04MB8559:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: RVsT2Ku1jdNEZbZ0N+GhSIS70NTG05KKVgjDtwjuB34wV2kaSpVewDZmpdz8jOyTXae42iLJB7/oNVZlQRissuEHdWF2JimEAsyf3Q7NhMvl6tAa5ERLjvovoFnjRbWcksIvIhGPStuqelZlQhtmgmPSQouZ6vC7dPqEdGvKr30FR3KF5SiUBKXDacTehgZ+y92wDCDf8HdKODs+2Fj28QC8gehcwFgAXM1nWBK739/MrVfX1rNdUA2miU9tPaoSRa6xyx59k0t7n58CpHu+GQm6VQGqag8duZfUmrjY+Aw/5sZJpTJfdr0A++NGVSLs4rgz+9Aqqk5PuJ452xVVcmDOTj0o4o8WbQ6LBTEOYLh7vO00pFueCv3aHmFfvCC1OB5oUyilArrQEzIx51y4vP2kJvN6qgIo8HnBUW0gUqkt2uqcg9lm5ZsGUAvyD0MVi/ivwiEeJB/kVI30qUDhs1MWbhV6Sqe2sSscrCcvZlNpyNae88SKgPEkOcXC5HaNFa7OTFtslJDvmUrgqx1zV7gnVpwP3IEzY2i12PPpSQIlsBH4oREhdiHY2g3MiK7Ifc7LSwXxbeo4mrL6fhay9ZlaN2Oj5EfBWggkodtIwHyc06w8jPHtRdz6CNjJGS1zldcGCujLiz84cMC6wRIibu8cwJt8vO21Ix7tJKgwzolm0ZiYtbhT6DiRaXf+RDTxrsam9fbAckj2ZTWka5KAAYi3pK3DtU0ouutCHus0mcA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR04MB6560.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(316002)(36756003)(66476007)(66556008)(66946007)(54906003)(6916009)(8676002)(4326008)(83380400001)(31686004)(6486002)(86362001)(5660300002)(6506007)(2906002)(508600001)(186003)(2616005)(38100700002)(8936002)(6512007)(26005)(31696002)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?Vm7lDTbNI51Shg6HZO/smIfqDRZK?= =?utf-8?q?wiKnknfQKSdP2ZUsFQNewTC1HBatCPgDrbOYEinwkUefKVK9Ts9zrbgvj0LVdHfP/?= =?utf-8?q?LLfWElhLD0GKMwAi2jxI+aoi6a0nL+cuiQsN5SDu2Obedec2lix4F5pug5SLCftRs?= =?utf-8?q?S7mmU3Cr22sZ9SW68yK/UwMkUOnb2T4SjErmqb8cY7555WLZooVPuRJQyUeCIq5HN?= =?utf-8?q?gwJ0kVilWAZAHxoqlGs8nbgCeCpM5MsLbAUfQghFjS2QWIqkbX+CqszTQWUzTOQoN?= =?utf-8?q?kN9URgf1+aMdH+KiRM2RV7LXf2qKooL9wn2JJl5XPUSxlhFqPGnIheSTrI8ZyY4ax?= =?utf-8?q?M/OfCzZx3wei++yqNgrEUNnCoj+yV3qfr/DnvKH1tmYcRUr/+EqcSN+beKdWwloal?= =?utf-8?q?QMHf6gtdEknibV+X4BeRQqKjQG0clcPEWUgCCwSkx64rZFw4BU5A5N2TAOWSedarY?= =?utf-8?q?fBRdDrTodTwch0hd2oFxCXn0gRHUjO/FTSb+udJPBsKLWDLyGer6Krv/f7O5H3I2C?= =?utf-8?q?Abdkvi/LiJVSswAL8dYQKwPlrg14DqLx3aIkDQmNN/gH3+Fa+YMiy8FRkG7lj+/JT?= =?utf-8?q?oH8aE+d242L9JUuUWdK7lRbyyO70588e6SVRp8fZwOABXNHfuodPlHX5TN87l19WS?= =?utf-8?q?9HOVZRq4wePmIellO4RNrxTSRAi+9St1gU02vOyzK4msR40M2CAHuzSo/Vm9HpBrK?= =?utf-8?q?nwAn3Gtof10ZAuA8a8XmbJ/MYqQEd41VkHs0t9TRZcO7MrKCxkiQb5JzHqtC/xO6z?= =?utf-8?q?zfdnjApW1TTdtfOAubUZXfF+Glv2DMC0LyLxkcA71uRBoqwRL1C7/ieEEAlhgqqQM?= =?utf-8?q?LLdIg0ZbikyZA6EvvJ7P+DgAVNwZziHYqyQql1//tstkpv9uRw9eyc/x/EG7VpTC4?= =?utf-8?q?NyL0v5TKbXJGxEY34WgNVFA2SVCKajSCDqD6ZvnGrt1tEQ5PkwOZRJ+WNAbuQXH4t?= =?utf-8?q?YV+v4sAYloBTZZcZ4a7c97OlRmLlsq4iii+WCDEvM811EyRxGZxBRCkQ3rpUQc3S5?= =?utf-8?q?kTKe5i+pMmEXxYzWoB7ntpQGDPknzDOTaY/YZrxDz5yS7sUmr+487SoGXmAW+9zc6?= =?utf-8?q?SB9eV75KslzON/ti5eNOnYwHSg5HTwHhnfAnrsXfH368h2kedImyd2vk0Kfu9DSF2?= =?utf-8?q?cRxA/HVLbRpR5QsC3/K9EjCF/AwIr4VHs9PlbfpXzCoq6D1rZBd5fGuy6A2hOL+Xt?= =?utf-8?q?PEHApUIls3nRi8vx2vD6JV6lUqzoX0WApXCwKfuSIvXK/6r+6xhG6rdmEkgXc/ser?= =?utf-8?q?TDqAZZyRnUimwckAj2UD50rJ3aVrIBHg0ThXVI890aRURaLRY2wSDI4YNAb6JM13V?= =?utf-8?q?tpCAJ6ZecX0hst3bG3SXyfWwsNj/FAPi95yDmvfZjzSWqny/96aaN/zhtRso5RxUD?= =?utf-8?q?RV+sspIU64cLcVbzv7RmUlgncTvDM3aS3sQ27r+Qw8D2znMk7jhYWfmBhu/ImnlBa?= =?utf-8?q?AcVWmBaTM3KZCQsNlVBHBoEBbb/OI+ph20D3dOHn5blYuAB89a45Pf+iiiCbzVz5y?= =?utf-8?q?owG1NwaWHjdkPQy3tSTkPtn46XYzpWiYSSpTqUKjVoItMAIEpKz3Qp0J+DEkuLoUR?= =?utf-8?q?OBE46JMI7q00B1Oh9f73etlXEyxQI7/BMucLCMh/8HyosvLPMggVKMIu6Z4TC/jLv?= =?utf-8?q?lqnnhQAU4VLINjKgmm6icEmQsaP3XucQ=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0449f426-36e3-4ae6-e953-08da4a0190d1 X-MS-Exchange-CrossTenant-AuthSource: VE1PR04MB6560.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jun 2022 10:19:41.2851 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: X59Sn3qNiA4CQ9aWhDZL50U5FAq0MK/9X1RIsdI9z5SMoTXLTUE5kh4u+29rMXaG7rr+SrZCHimj5UB44Eijsw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR04MB8559 When a page table ends up with no present entries left, it can be replaced by a non-present entry at the next higher level. The page table itself can then be scheduled for freeing. Note that while its output isn't used there yet, pt_update_contig_markers() right away needs to be called in all places where entries get updated, not just the one where entries get cleared. Signed-off-by: Jan Beulich Reviewed-by: Roger Pau Monné --- v5: Re-base over changes earlier in the series. v4: Re-base over changes earlier in the series. v3: Re-base over changes earlier in the series. v2: New. --- a/xen/drivers/passthrough/amd/iommu_map.c +++ b/xen/drivers/passthrough/amd/iommu_map.c @@ -21,6 +21,7 @@ #include "iommu.h" +#define CONTIG_MASK IOMMU_PTE_CONTIG_MASK #include /* Given pfn and page table level, return pde index */ @@ -35,16 +36,20 @@ static unsigned int pfn_to_pde_idx(unsig static union amd_iommu_pte clear_iommu_pte_present(unsigned long l1_mfn, unsigned long dfn, - unsigned int level) + unsigned int level, + bool *free) { union amd_iommu_pte *table, *pte, old; + unsigned int idx = pfn_to_pde_idx(dfn, level); table = map_domain_page(_mfn(l1_mfn)); - pte = &table[pfn_to_pde_idx(dfn, level)]; + pte = &table[idx]; old = *pte; write_atomic(&pte->raw, 0); + *free = pt_update_contig_markers(&table->raw, idx, level, PTE_kind_null); + unmap_domain_page(table); return old; @@ -87,7 +92,11 @@ static union amd_iommu_pte set_iommu_pte if ( !old.pr || old.next_level || old.mfn != next_mfn || old.iw != iw || old.ir != ir ) + { set_iommu_pde_present(pde, next_mfn, 0, iw, ir); + pt_update_contig_markers(&table->raw, pfn_to_pde_idx(dfn, level), + level, PTE_kind_leaf); + } else old.pr = false; /* signal "no change" to the caller */ @@ -326,6 +335,9 @@ static int iommu_pde_from_dfn(struct dom smp_wmb(); set_iommu_pde_present(pde, next_table_mfn, next_level, true, true); + pt_update_contig_markers(&next_table_vaddr->raw, + pfn_to_pde_idx(dfn, level), + level, PTE_kind_table); *flush_flags |= IOMMU_FLUSHF_modified; } @@ -351,6 +363,9 @@ static int iommu_pde_from_dfn(struct dom next_table_mfn = mfn_x(page_to_mfn(table)); set_iommu_pde_present(pde, next_table_mfn, next_level, true, true); + pt_update_contig_markers(&next_table_vaddr->raw, + pfn_to_pde_idx(dfn, level), + level, PTE_kind_table); } else /* should never reach here */ { @@ -487,8 +502,24 @@ int cf_check amd_iommu_unmap_page( if ( pt_mfn ) { + bool free; + /* Mark PTE as 'page not present'. */ - old = clear_iommu_pte_present(pt_mfn, dfn_x(dfn), level); + old = clear_iommu_pte_present(pt_mfn, dfn_x(dfn), level, &free); + + while ( unlikely(free) && ++level < hd->arch.amd.paging_mode ) + { + struct page_info *pg = mfn_to_page(_mfn(pt_mfn)); + + if ( iommu_pde_from_dfn(d, dfn_x(dfn), level, &pt_mfn, + flush_flags, false) ) + BUG(); + BUG_ON(!pt_mfn); + + clear_iommu_pte_present(pt_mfn, dfn_x(dfn), level, &free); + *flush_flags |= IOMMU_FLUSHF_all; + iommu_queue_free_pgtable(hd, pg); + } } spin_unlock(&hd->arch.mapping_lock);