From patchwork Mon Apr 25 08:35:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 12825405 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E322FC433F5 for ; Mon, 25 Apr 2022 08:42:02 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.312562.529874 (Exim 4.92) (envelope-from ) id 1niuI2-0004xn-T1; Mon, 25 Apr 2022 08:41:50 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 312562.529874; Mon, 25 Apr 2022 08:41:50 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuI2-0004xg-Nz; Mon, 25 Apr 2022 08:41:50 +0000 Received: by outflank-mailman (input) for mailman id 312562; Mon, 25 Apr 2022 08:41:49 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1niuCB-0005mk-Vq for xen-devel@lists.xenproject.org; Mon, 25 Apr 2022 08:35:48 +0000 Received: from de-smtp-delivery-102.mimecast.com (de-smtp-delivery-102.mimecast.com [194.104.109.102]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id b427c536-c472-11ec-a405-831a346695d4; Mon, 25 Apr 2022 10:35:47 +0200 (CEST) Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04lp2059.outbound.protection.outlook.com [104.47.12.59]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id de-mta-33-GyOr7XTDMV6Y45BrJtxeOw-1; Mon, 25 Apr 2022 10:35:45 +0200 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) by DBBPR04MB7641.eurprd04.prod.outlook.com (2603:10a6:10:1f7::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.21; Mon, 25 Apr 2022 08:35:44 +0000 Received: from DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f]) by DU2PR04MB8616.eurprd04.prod.outlook.com ([fe80::5cb0:5195:4203:7c2f%9]) with mapi id 15.20.5186.021; Mon, 25 Apr 2022 08:35:44 +0000 X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: b427c536-c472-11ec-a405-831a346695d4 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=mimecast20200619; t=1650875746; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xpBRkm9Z5IBvu/NQMA+LCq2v737PicnDjxA+STnwy5s=; b=LHQLpS2dJwvVnjaLr852QknCwsaIN6y/ZBujl3QH7KjU6bU0MjKxlQ+rCsYhTWG2XWJr5N A/jxvGWDB59unjvm4HY4Ed9lbKih9CLBdjlvhP+bgK+XJNpQpWzxfkPdYrkzneKsmlLD3f e4VICLe5JwriFBA7RC6lY1ckdfjtGhU= X-MC-Unique: GyOr7XTDMV6Y45BrJtxeOw-1 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=clhCJyuyPVJOkk/RFMeOsimStHG7DCGYFXLt4YDz5r7m30pbzjTHaYPtIZ/z/rWV9aQm/a5m6a/TG1Hta8pypf05RuyxseGsaCRM5W1IZppl9ZglDaXJ+Ap6WL73TNDKjLkNENETX0Z3t8zIC3RJu96fW0hTarqpdGzmpCUI6iAIAcJVmtIbM7sq+weppRmThFxXkEh5OM6IlkDNUsLH8fuLcesNpDU61YZbuWXSisj0KowtDM0WlBDwoRmHNnsNNdci4j0UNTdteE3jaFHJtqFFzEdXaBNqf8zXXoxLvctq/qyn6nNxnndmsTp+YmVnkIe/YsQa5rL24OoNxw8NTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xpBRkm9Z5IBvu/NQMA+LCq2v737PicnDjxA+STnwy5s=; b=gslz2pLKZdGxIaWIJvr2T70alvPIBleVYreJxfZlcNzdaogpZkPPHKfZw8dApDN1lD7Vphr5IWxHAnsKN5bCGb8XUePSvv7MjB/6zY7cYlc/Q3qAISIhJf4+frVsVP4uxNyFCyjEsVYS+dMPLYVozEKhrVR5Gw+ROmdwpP7rCIguvOX9nmUITUwXEnGhLRV4puml+ENnSRnLAyjmIrwClZ6fTdFYZSIqLr4m8xydtoETXZ/y8tHdjmEtlrtUDQJiXSQzpIUpZ8r5d619u4CyIculyN7jjerDAgBYKaeSHjeClah8bs7dTq3gKHNX3RLoBpcUWJCHmDpcLYznsAi4dA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=suse.com; dmarc=pass action=none header.from=suse.com; dkim=pass header.d=suse.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=suse.com; Message-ID: <1389c785-ffe3-5d8c-36f1-a923ce5250cd@suse.com> Date: Mon, 25 Apr 2022 10:35:45 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: [PATCH v4 07/21] IOMMU/x86: support freeing of pagetables Content-Language: en-US From: Jan Beulich To: "xen-devel@lists.xenproject.org" Cc: Andrew Cooper , Paul Durrant , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu References: In-Reply-To: X-ClientProxiedBy: AS9PR06CA0194.eurprd06.prod.outlook.com (2603:10a6:20b:45d::29) To DU2PR04MB8616.eurprd04.prod.outlook.com (2603:10a6:10:2db::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4786ce58-5143-4d8b-b37b-08da269696cc X-MS-TrafficTypeDiagnostic: DBBPR04MB7641:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jqOyCAWCKznMvypUKtfiWpUP2kDn3ZGwvltvR/p04I4Y0KYqdjiwQFNuJ81ToF5ofpWd91XXDnBv0eTz5Mxxx3H8JgQZtMbdntPdPMiwq8jtxrZ96+bDsUHasJXhCXVbYxdWp8YDhm1ITD6h+wEFX7BQkuhFp4wqPqvIlfn2ofCggFiJnO1K2St0+xgPm5BZ2+sCYvT8b3kj8Z3tJ+s0Q40H1mDu4vLHH0OZrd/JB+iy9RhtoAiMx62G8Ha8d97/qtPlY/wXZcXoTfgOXVcHdKEHZxfAfhB5aBkRXU6FTyRWS0gcsV3ev90zCox471CX91nhcV7miSNaqBDY3a9G9QLnjFYQrhy9q8BvTaYb3kfui0/pQfGKL2hPiFN/aiWYNoO8SMBeRveTB93PdJs4eQGIjsyWV5527679joyshpTcIh6okipFU4i5pepm58u1zY939EF+9w7CdHx3fycw608BZREFJg7KTyM3Bpo+iuZCwebCJVfwBL+QHrE2LD74Q6IIlOeQDI+ErLCc86EO/oFFGzXfK7VM0oDen7fsuS91M7zEPdE1OcEvTPMS6KCe2U/42RB/rgGJOA0Ya467H3D++eQpUiszXVk8AwDB5bKhHN6I3N4ZCBqky+Y99p9OUPt9xuo4hXCcJkIjgrCFgWd7BvIIgUmBp59A4/i2lz0atUcgCZVRH83GmN4atsTdkbvW0iqZe4ItBuY4r06TO/hAVXEOqM1StW93MseFNLzZhjm34yV7URDwKophxPQwkMC4JsFsipiX58/NBt00f6JAjArhp4hx056gi2IYikeoLB/fttGn1/LRWV1iZ52I X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DU2PR04MB8616.eurprd04.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(366004)(2616005)(186003)(6916009)(31696002)(86362001)(31686004)(8936002)(4326008)(8676002)(36756003)(2906002)(5660300002)(508600001)(66476007)(38100700002)(316002)(26005)(66556008)(66946007)(6506007)(6512007)(83380400001)(54906003)(6486002)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?0CRs7IX1//RepZIoV96av5u+IbTM?= =?utf-8?q?s7fAA5MWCLn51aN5leWttPG9Nxt00F0Dc9xhrj+Psk1eoBgQgzw77v8oAct9Z75pX?= =?utf-8?q?hhH+NUfchP9UyG31UZXQtTPN6A3VxrSexH2ie8GTjUsDypO++Fm0UU47J0+7Im+Aq?= =?utf-8?q?qSZoRWYJLqju2H4+ujcPk9SxEwPM31D2UGngBuIDfgv9kx+micbNDQBHxZ/2M5IDs?= =?utf-8?q?NxZIzuS9hUU0YnXvYsoAPHlfJpqA5saDhtK9e/WbYVVQ1P8GcliFnY3zTAAEwLq9k?= =?utf-8?q?OdslQUMet2uJHUCsQ1n3M5b/+IF2NUYXEiAsQwe7dq9QTCDCEqBbB7WOZhOzWTn0V?= =?utf-8?q?8oIyhk6MEUlpUkKf669gWpwvZGeoTmMInPO7IxBGRcY6wB1l/A0hOj6ge6xMyA6Xp?= =?utf-8?q?08xSBpMmFgXw3AH/RyhqzcKTnGypSrzOtrzN3sFXlimq1LBAIE+aJ1FWn7l3G3BDm?= =?utf-8?q?nI3klhlYfwIPFI6f3zXC6p9lKfY9T7xJ2Jd1xmlNDJLnOCpYrVmw8hrGtmcn0W7X5?= =?utf-8?q?/qtExu3aL4AeSAKN0/V96fxaHldXxWfpsqp2uMZRW0+80mEf3yliqvPf/H/4oU+Ri?= =?utf-8?q?1gSZktnEj15IjY9YUJhlhns6NFQvlzEzexGMP9Edo7bhxj4oHftasigrT13tP6eHy?= =?utf-8?q?9E2pG09De93AEaD/6/faVIxcSvhsl70jzB5i0l7Lvp8OaBVz2u5NHgP6r4K2/vEUB?= =?utf-8?q?DtoxP88bHoDgA/X1G8iS4qMqIlpIozNLqgz8CMYgXkfMjag2y8uwujZ77Qk4DrLwr?= =?utf-8?q?pOeQUJkMYC83Nt0WtFh9Fz9/+1gVw8zxx6gFqP2xVQTKebMxHf4QiGXQ9f0zT99aB?= =?utf-8?q?BGKBeOuaz7eJthx65oqyrDndsJXWw6eQC9d3zZ5IsB7O5t9kf7K79JceZhITLUl2U?= =?utf-8?q?J8ugz6VY8pt9NdWjYqa0+MJBZOMzyKmQXTjV/r4+naJ6dQKUq6iNCKMy1Ui8nSc7J?= =?utf-8?q?8aapEDWjQINI3vRykeR8VuAIovurYkK7YRhNbvXZuWBV/Dth95khTnJGawfL2BOfO?= =?utf-8?q?AHckgNt1wVaXCkmq++MowOFsYwb6e/2gqnIn51L7rRwL16mFTCwlxlROtwbcAHr/L?= =?utf-8?q?XSuVAh3inp9XT9aqPmNJD0c0/+XzzPAog2q668rIAeU9v9u4bUwszxhJdWzM1Xc2n?= =?utf-8?q?y+jm7GBKFl0Qdq5VUGSBHG0D6NB1baff18vjMhYTZdWOsw+afVAYZ7IlZ9G14x0l/?= =?utf-8?q?HY1Qp3QAVP8iOab8L+nH8GREq+ibnroMlfxQFBI92720NbWcDT5t0tL/b9miazLjv?= =?utf-8?q?DqvNp7EaG5DBXg87+lLXs+jb/fXp/5Ojkg6FNwtRR4f7+n5XseYgYrH1j9mBtMZzl?= =?utf-8?q?AWzXRzyzcQh8lr9lMpiaFwPxV0i3gIiLPZUZZSZIL2ZaGAGdLU9YtJgYCPGY4gPLP?= =?utf-8?q?7gVRcBfWuSgcgAMYO3A7Tauix8mjQMPmhDjF+X3HcQT3slxl/Ho0mB+kbwtx+yW3R?= =?utf-8?q?7x9NwT+OtRi6PEupx4SeQiEP7qOblo80oiaG/zHhhzcc6qpVH3TThDWisteSNuPmq?= =?utf-8?q?vaqxroDqvar7/jq0ND4qBhan8UzikjmfPNB8/tHoJe9SovNrep/1ZfbeGB03OLy/l?= =?utf-8?q?Rbo8UCs0axAfkcPYVxFiN6yxlkthbMfdrXx4N9gPSF9S0ejZL5KnYoY55xyHWwQbt?= =?utf-8?q?eO6hjv1W32agXx3/dOrOEfXArDce0o6w=3D=3D?= X-OriginatorOrg: suse.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4786ce58-5143-4d8b-b37b-08da269696cc X-MS-Exchange-CrossTenant-AuthSource: DU2PR04MB8616.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 08:35:44.4231 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6ehietAuHGSpWwSk1Qjh5wrMwLnBLbCvBD9PZjVgRIwxWA6zLSizU2Hkt2RrQIjf5gI11iykMfDt2lyI3sStcA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR04MB7641 For vendor specific code to support superpages we need to be able to deal with a superpage mapping replacing an intermediate page table (or hierarchy thereof). Consequently an iommu_alloc_pgtable() counterpart is needed to free individual page tables while a domain is still alive. Since the freeing needs to be deferred until after a suitable IOTLB flush was performed, released page tables get queued for processing by a tasklet. Signed-off-by: Jan Beulich --- I was considering whether to use a softirq-tasklet instead. This would have the benefit of avoiding extra scheduling operations, but come with the risk of the freeing happening prematurely because of a process_pending_softirqs() somewhere. --- v4: Change type of iommu_queue_free_pgtable()'s 1st parameter. Re-base. v3: Call process_pending_softirqs() from free_queued_pgtables(). --- a/xen/arch/x86/include/asm/iommu.h +++ b/xen/arch/x86/include/asm/iommu.h @@ -147,6 +147,7 @@ void iommu_free_domid(domid_t domid, uns int __must_check iommu_free_pgtables(struct domain *d); struct domain_iommu; struct page_info *__must_check iommu_alloc_pgtable(struct domain_iommu *hd); +void iommu_queue_free_pgtable(struct domain_iommu *hd, struct page_info *pg); #endif /* !__ARCH_X86_IOMMU_H__ */ /* --- a/xen/drivers/passthrough/x86/iommu.c +++ b/xen/drivers/passthrough/x86/iommu.c @@ -12,6 +12,7 @@ * this program; If not, see . */ +#include #include #include #include @@ -550,6 +551,91 @@ struct page_info *iommu_alloc_pgtable(st return pg; } +/* + * Intermediate page tables which get replaced by large pages may only be + * freed after a suitable IOTLB flush. Hence such pages get queued on a + * per-CPU list, with a per-CPU tasklet processing the list on the assumption + * that the necessary IOTLB flush will have occurred by the time tasklets get + * to run. (List and tasklet being per-CPU has the benefit of accesses not + * requiring any locking.) + */ +static DEFINE_PER_CPU(struct page_list_head, free_pgt_list); +static DEFINE_PER_CPU(struct tasklet, free_pgt_tasklet); + +static void free_queued_pgtables(void *arg) +{ + struct page_list_head *list = arg; + struct page_info *pg; + unsigned int done = 0; + + while ( (pg = page_list_remove_head(list)) ) + { + free_domheap_page(pg); + + /* Granularity of checking somewhat arbitrary. */ + if ( !(++done & 0x1ff) ) + process_pending_softirqs(); + } +} + +void iommu_queue_free_pgtable(struct domain_iommu *hd, struct page_info *pg) +{ + unsigned int cpu = smp_processor_id(); + + spin_lock(&hd->arch.pgtables.lock); + page_list_del(pg, &hd->arch.pgtables.list); + spin_unlock(&hd->arch.pgtables.lock); + + page_list_add_tail(pg, &per_cpu(free_pgt_list, cpu)); + + tasklet_schedule(&per_cpu(free_pgt_tasklet, cpu)); +} + +static int cf_check cpu_callback( + struct notifier_block *nfb, unsigned long action, void *hcpu) +{ + unsigned int cpu = (unsigned long)hcpu; + struct page_list_head *list = &per_cpu(free_pgt_list, cpu); + struct tasklet *tasklet = &per_cpu(free_pgt_tasklet, cpu); + + switch ( action ) + { + case CPU_DOWN_PREPARE: + tasklet_kill(tasklet); + break; + + case CPU_DEAD: + page_list_splice(list, &this_cpu(free_pgt_list)); + INIT_PAGE_LIST_HEAD(list); + tasklet_schedule(&this_cpu(free_pgt_tasklet)); + break; + + case CPU_UP_PREPARE: + case CPU_DOWN_FAILED: + tasklet_init(tasklet, free_queued_pgtables, list); + break; + } + + return NOTIFY_DONE; +} + +static struct notifier_block cpu_nfb = { + .notifier_call = cpu_callback, +}; + +static int __init cf_check bsp_init(void) +{ + if ( iommu_enabled ) + { + cpu_callback(&cpu_nfb, CPU_UP_PREPARE, + (void *)(unsigned long)smp_processor_id()); + register_cpu_notifier(&cpu_nfb); + } + + return 0; +} +presmp_initcall(bsp_init); + bool arch_iommu_use_permitted(const struct domain *d) { /*