From patchwork Wed Dec 21 13:25:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 13078749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CE5F7C10F1B for ; Wed, 21 Dec 2022 13:25:37 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.467883.726934 (Exim 4.92) (envelope-from ) id 1p7z67-0003pP-Sg; Wed, 21 Dec 2022 13:25:27 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 467883.726934; Wed, 21 Dec 2022 13:25:27 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p7z67-0003pI-Ph; Wed, 21 Dec 2022 13:25:27 +0000 Received: by outflank-mailman (input) for mailman id 467883; Wed, 21 Dec 2022 13:25:26 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p7z66-0003Di-2u for xen-devel@lists.xenproject.org; Wed, 21 Dec 2022 13:25:26 +0000 Received: from EUR02-DB5-obe.outbound.protection.outlook.com (mail-db5eur02on2089.outbound.protection.outlook.com [40.107.249.89]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id ecf178b1-8132-11ed-91b6-6bf2151ebd3b; Wed, 21 Dec 2022 14:25:24 +0100 (CET) Received: from VE1PR04MB6560.eurprd04.prod.outlook.com (2603:10a6:803:122::25) by AS8PR04MB8610.eurprd04.prod.outlook.com (2603:10a6:20b:425::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5924.16; Wed, 21 Dec 2022 13:25:21 +0000 Received: from VE1PR04MB6560.eurprd04.prod.outlook.com ([fe80::4da2:ea8b:e71e:b8d8]) by VE1PR04MB6560.eurprd04.prod.outlook.com ([fe80::4da2:ea8b:e71e:b8d8%4]) with mapi id 15.20.5924.016; Wed, 21 Dec 2022 13:25:21 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ecf178b1-8132-11ed-91b6-6bf2151ebd3b ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QKS3DpjHRFu7eTVeb1my0sbj8LQjQlJ9u0VLMI0ZWhqQtriYFB6uQjKMDQPbZbvSdNEvN1rHw5M+b7Z1RDNEUS6y/Ku9I2aU/cv0VbnJDD6n16Y7QZgoV0E5pMbgiGgJE26D5gSCkqFBmc5AwHFdev0jwZuA4xSDhi5OGs48nxGV7E7oM7a4FzFSe8Aaihg3Y2pjp5cOtANET0mEnxQrVXUY98JQpvjStKHL1vNE+Mfp0E1U6qXpF3oIWpIilZDwsz7ETLtlR8TiWrBYmYBljosX+x5qgPIrrK7RKfJQRSlvW7RxIu9iK07FWzoBicFmdarIO+9ddW+9zJttXZMwVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jRn3ptmgQ2T1waA/6n6LdzW2l6WqDKNqZIo99H92QZU=; b=Vl/6bAiSq1G0APxADm88/y65y0chLV3lEQCeBpPivgt44AWhIpgnkUEjX1U8YLZSZiWH73LYaLY74bq5TyrZR+0CgrCDsXL1CA6WtMcaK63Mfr98fQaEVY3RaVNRqtATwN/Hx30QEFlDkCJ0ejqnCYlugvhzGdTE7vw526vw5Gx0/Q0EVjopOS/kUGZl9XKDNAs/uibTJAg4sm3/+23cp2SpaDG7ZDQKUoShS7BTsYFZpgkdQZGnhVjsKzag2cahMYlapnGFTEoX4Ym9AFAMee7zjp1MA86u7gff0ZErL89cKdxg49xmhXLHritrOy8j6eMz5uw9pbjZ8/MYuyvQVg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jRn3ptmgQ2T1waA/6n6LdzW2l6WqDKNqZIo99H92QZU=; b=HXsirKOkW0PC57tQ7AHFFEZD84QzctJije7da3nhF0vMo0Wmt6/6BpCtD+dzlqPoytwYeS7L8KNBzG9LBrNT/ddI0guphKWmdsqjaAo1WOuVHqvkKuKT6tcDnOn42cOFJ00p5A7nwOMLe/HsdczKc+JSCF7UD8bV9e9TQNqtFW2oS6bqSlhBK+4sS4xcolCzFmhbyAplGeihgFIGMs7xY8sn1s62kvWUFHg7S/aO1R4bDLOYCmARgJUdBhjwG8ql1J1YI5rQwYI4ovpSKnNIUv8laHYMGp0Q6go2Gzq3TsM+w93Y3BMEpkR9/xrdWG4b2jPsrsv5kXNFThi6xyHyeA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <8d519e00-83c6-aee9-e7ba-523aa4265e1e@suse.com> Date: Wed, 21 Dec 2022 14:25:19 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.6.0 Subject: [PATCH 2/8] x86/paging: fold most HAP and shadow final teardown Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Wei Liu , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , George Dunlap , Tim Deegan References: In-Reply-To: X-ClientProxiedBy: FR2P281CA0041.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:92::12) To VE1PR04MB6560.eurprd04.prod.outlook.com (2603:10a6:803:122::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: VE1PR04MB6560:EE_|AS8PR04MB8610:EE_ X-MS-Office365-Filtering-Correlation-Id: 1cd98313-74f6-44aa-0ea2-08dae356cf9f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yUc+bDBZJI2axKq2+y8wY5mDQh4mPnR6FW1wUPRcBGj+Jh3P6yMz1WeKr3uyTJ0y2d4+CqIjNmt3XyH32FR4zyEfKbm52Zdf9dshak2Uw/8/vbtLSt1qFummkt7BpSU71blFDQR6TI8d4qXvm7kIF0UJo5/PnwJ6+751+olpCipVPZzbo4IW476+dKf4b91FNxLMJcZdmhatRPcvYkYsSbQ/5b2NanwfbY1aSPM8iZi8isOxARV8FpvdY4Ek771EfCIErjfGYfLPHDfgOH2GCMWi4T0bl//jaoydrFv8iDKOsJFenrCPnEaHMchq+KYTBvzM/1zZie7Vve7NrXIVXgBM2OVIltnzhKHB2jSpRljfIOa4bzQo7oMuiVEn8PdzpR5SAjgiPiv3bnEY5GsVqUFWJjQ/Xr9j9doO2r8hFws+B2sunSFT8nVSDFoAcBzX3cw6Tdi2aG8V26iShM7qOqQGSuZfnDJr/aY0+O6wEvuCu/NsUfvxomDMaf1mYf3LdAgp1+YO8G4z/OiIjEOVkHwp5OGR8Z/HnDjItdxTHc26yTg6k0v5HuosuMuDe/27/XyGMi1YLGUh9/9ItC0JAywZG2fQ3L81Jovw079M0Ac/oiWzIqDEsksa9RvQsdEWFJ2PEZ5xuPM4OuT4qGnF8SbVT0cqqBn9qvcNG4WSOAuBIZ0M2yuEuNOQkTbvzHCU6kMbKbSkV3wX5DAApkg1w9U3Hx+mF30QQDpiPO1SjIo= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VE1PR04MB6560.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(346002)(136003)(396003)(376002)(39860400002)(366004)(451199015)(31686004)(8936002)(8676002)(66476007)(86362001)(4326008)(66556008)(5660300002)(66946007)(316002)(2906002)(41300700001)(6916009)(54906003)(2616005)(83380400001)(31696002)(6512007)(186003)(6506007)(26005)(38100700002)(6486002)(478600001)(36756003)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?TsPqo/V8T6yFXo5afynZeT1dBuS2?= =?utf-8?q?hEaVHMBEKl2Ee+hSqyoD3TZjI+Lw2zRwnXWaE8WYN+n+2slmVrbwZXSfll3d9g3tu?= =?utf-8?q?hlkEzRvil8JE4AXU1RDL1bB1BuZDwnjOcGFNFlQEhq3GslmunJ3KRiSEJ2IGAd0+K?= =?utf-8?q?frRc6NRgPbJrAqIT4Ybox1F9Wzj4DKfn899gEkLw7qlgMFo5KXUEY98WgMhvpx7aA?= =?utf-8?q?n7vs4jDVFR0jrpIJjNpXAyXQqWB/22GDkdF1HMsCp+giiBUZK0pLpgYtgtWttw5Lh?= =?utf-8?q?qLkm7kQB91AxkJAzfjd4GNcFoIyrI0Y+pnfacZafUWuVCl+rhvNIpqXeEJmATz4k9?= =?utf-8?q?QJZYf3UT/jo8vuH+aYO4SWldyxFpBVuE9UH+5SxW3Oc6FuYifDVOfmAkOA+i3qVir?= =?utf-8?q?2RuqMP0o+BpaEpS6+9ety11fKuaXGs86PJxEsEaf0SYPdvL1zgPII+ZCLZjmLIRIC?= =?utf-8?q?f3gxM139AXltFJ9sEDXqcyOINnj59jZ5HFd1V8nO/Yte44B7u2ixffVAZCGO/rhVG?= =?utf-8?q?qmDRPYrpmRpT+QxTRL/rilGoz5hmBE0k5KUu9gq8qhE4CSRXBo9ZTWttmoYu4s41+?= =?utf-8?q?3euNBYXbzUYVWg+S6vS33LOGf0G+3h0ha/Tqfs5hF0ACUMoPpasSo9InbgP/9HDZ9?= =?utf-8?q?DI24rEVYbbBKlAH51i2s8N2UT/5oKByF9oCGymrYf3eGomB+TivfbqDyb9zNi+f3t?= =?utf-8?q?D5tetxzoRJKHFE92ZHbbJRwXi+ml2B40yOziQCjyQ2j5x1+hlTO/rj3quKpus+Pp1?= =?utf-8?q?azAOVQU7gSgtkizkQ23mgSDRTUi6LtQCNMc1NuAbdUU4kPZcMx12iS+AxlDsQpa4c?= =?utf-8?q?Z1fAOyFAkJ+e9kuwd5ImACvRVOS09YZqC82d/UOt9WVst2dNd4YSIVmHi7K7FvBtb?= =?utf-8?q?HlB/pXR4j5cbNmXRbJlCWv3pdRVYTKFEAQbvVmhc+589gkJZYVNfws+pL3heuJ2IO?= =?utf-8?q?mHnTeKTJR5ieaNbPAc157L+0PPLT38pIEDrLYy4Mz6T8QmeWh0olBXeIeCQammrD5?= =?utf-8?q?Iv6l+zaiuT+cXtQiNs0wy3uYYrbNogdYzmBw0OQhHPQI7+3vRmTKGQGzBTjnVhJnI?= =?utf-8?q?SROxZJi93BjBCsvUQNXS8dSK0a7AN2zk64nN6rC2jA11HfyJbrQBHF/4A9/mLYel9?= =?utf-8?q?g0jkknsifypU+vjFZS9UntffZdy9WFqWuu+I7VLLLDviwbcV7xP9omyWqYgIpdndt?= =?utf-8?q?/etO90cqNLyHtJ+9IW9dHGwCCKdcw3GgcEJd/a6isfU+Gt5m1oMzX2zvumy9h9m2j?= =?utf-8?q?Iv5a2Kyfmedwrc6Wi4rEFUlK2oCxANuWGBGwR4W5beX/ubPaFhZgo1RYeA7EV1dQf?= =?utf-8?q?9Jgb9OIZRKM6l7hTN+PPcDG1LgCx29h7wT801uhYhBUQWpXLHXGx+slOkmdN37p6T?= =?utf-8?q?//2HkAv8UDpV99gf2WJMn3InEciT+uwweJPM/ho/XTMNeJxN+4ZX70pMV5jBIdoM4?= =?utf-8?q?5ckar5xaFrAAx/XE8VYrggBKRAnSKt14uPBjhMLTolPagGHm5Vq3zZAna+lZWUNSY?= =?utf-8?q?piMTU4OLatmU?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1cd98313-74f6-44aa-0ea2-08dae356cf9f X-MS-Exchange-CrossTenant-AuthSource: VE1PR04MB6560.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Dec 2022 13:25:21.7440 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Wf+fYUmmBlmLwlp0Jrg/bEaqydI5Hz3LyjuBo/GD+GrKj25x3GWDY/X8FRY/H+zAFYWp8bcgv8z6IYNXYUwBPg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR04MB8610 HAP does a few things beyond what's common, which are left there at least for now. Common operations, however, are moved to paging_final_teardown(), allowing shadow_final_teardown() to go away. While moving (and hence generalizing) the respective SHADOW_PRINTK() drop the logging of total_pages from the 2nd instance - the value is necessarily zero after {hap,shadow}_set_allocation(). Signed-off-by: Jan Beulich --- The remaining parts of hap_final_teardown() could be moved as well, at the price of a CONFIG_HVM conditional. I wasn't sure whether that was deemed reasonable. --- a/xen/arch/x86/include/asm/shadow.h +++ b/xen/arch/x86/include/asm/shadow.h @@ -78,9 +78,6 @@ int shadow_domctl(struct domain *d, void shadow_vcpu_teardown(struct vcpu *v); void shadow_teardown(struct domain *d, bool *preempted); -/* Call once all of the references to the domain have gone away */ -void shadow_final_teardown(struct domain *d); - void sh_remove_shadows(struct domain *d, mfn_t gmfn, int fast, int all); /* Adjust shadows ready for a guest page to change its type. */ --- a/xen/arch/x86/mm/hap/hap.c +++ b/xen/arch/x86/mm/hap/hap.c @@ -268,8 +268,8 @@ static void hap_free(struct domain *d, m /* * For dying domains, actually free the memory here. This way less work is - * left to hap_final_teardown(), which cannot easily have preemption checks - * added. + * left to paging_final_teardown(), which cannot easily have preemption + * checks added. */ if ( unlikely(d->is_dying) ) { @@ -552,18 +552,6 @@ void hap_final_teardown(struct domain *d for (i = 0; i < MAX_NESTEDP2M; i++) { p2m_teardown(d->arch.nested_p2m[i], true, NULL); } - - if ( d->arch.paging.total_pages != 0 ) - hap_teardown(d, NULL); - - p2m_teardown(p2m_get_hostp2m(d), true, NULL); - /* Free any memory that the p2m teardown released */ - paging_lock(d); - hap_set_allocation(d, 0, NULL); - ASSERT(d->arch.paging.p2m_pages == 0); - ASSERT(d->arch.paging.free_pages == 0); - ASSERT(d->arch.paging.total_pages == 0); - paging_unlock(d); } void hap_vcpu_teardown(struct vcpu *v) --- a/xen/arch/x86/mm/paging.c +++ b/xen/arch/x86/mm/paging.c @@ -842,10 +842,46 @@ int paging_teardown(struct domain *d) /* Call once all of the references to the domain have gone away */ void paging_final_teardown(struct domain *d) { - if ( hap_enabled(d) ) + bool hap = hap_enabled(d); + + PAGING_PRINTK("%pd final teardown starts. Pages total = %u, free = %u, p2m = %u\n", + d, d->arch.paging.total_pages, + d->arch.paging.free_pages, d->arch.paging.p2m_pages); + + if ( hap ) hap_final_teardown(d); + + /* + * Double-check that the domain didn't have any paging memory. + * It is possible for a domain that never got domain_kill()ed + * to get here with its paging allocation intact. + */ + if ( d->arch.paging.total_pages ) + { + if ( hap ) + hap_teardown(d, NULL); + else + shadow_teardown(d, NULL); + } + + /* It is now safe to pull down the p2m map. */ + p2m_teardown(p2m_get_hostp2m(d), true, NULL); + + /* Free any paging memory that the p2m teardown released. */ + paging_lock(d); + + if ( hap ) + hap_set_allocation(d, 0, NULL); else - shadow_final_teardown(d); + shadow_set_allocation(d, 0, NULL); + + PAGING_PRINTK("%pd final teardown done. Pages free = %u, p2m = %u\n", + d, d->arch.paging.free_pages, d->arch.paging.p2m_pages); + ASSERT(!d->arch.paging.p2m_pages); + ASSERT(!d->arch.paging.free_pages); + ASSERT(!d->arch.paging.total_pages); + + paging_unlock(d); p2m_final_teardown(d); } --- a/xen/arch/x86/mm/shadow/common.c +++ b/xen/arch/x86/mm/shadow/common.c @@ -1232,7 +1232,7 @@ void shadow_free(struct domain *d, mfn_t /* * For dying domains, actually free the memory here. This way less - * work is left to shadow_final_teardown(), which cannot easily have + * work is left to paging_final_teardown(), which cannot easily have * preemption checks added. */ if ( unlikely(dying) ) @@ -2940,35 +2940,6 @@ out: } } -void shadow_final_teardown(struct domain *d) -/* Called by arch_domain_destroy(), when it's safe to pull down the p2m map. */ -{ - SHADOW_PRINTK("dom %u final teardown starts." - " Shadow pages total = %u, free = %u, p2m=%u\n", - d->domain_id, d->arch.paging.total_pages, - d->arch.paging.free_pages, d->arch.paging.p2m_pages); - - /* Double-check that the domain didn't have any shadow memory. - * It is possible for a domain that never got domain_kill()ed - * to get here with its shadow allocation intact. */ - if ( d->arch.paging.total_pages != 0 ) - shadow_teardown(d, NULL); - - /* It is now safe to pull down the p2m map. */ - p2m_teardown(p2m_get_hostp2m(d), true, NULL); - /* Free any shadow memory that the p2m teardown released */ - paging_lock(d); - shadow_set_allocation(d, 0, NULL); - SHADOW_PRINTK("dom %u final teardown done." - " Shadow pages total = %u, free = %u, p2m=%u\n", - d->domain_id, d->arch.paging.total_pages, - d->arch.paging.free_pages, d->arch.paging.p2m_pages); - ASSERT(d->arch.paging.p2m_pages == 0); - ASSERT(d->arch.paging.free_pages == 0); - ASSERT(d->arch.paging.total_pages == 0); - paging_unlock(d); -} - static int shadow_one_bit_enable(struct domain *d, u32 mode) /* Turn on a single shadow mode feature */ {