From patchwork Wed Jul 26 15:10:03 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oleksandr Tyshchenko X-Patchwork-Id: 9865267 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id BD50760382 for ; Wed, 26 Jul 2017 15:12:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C3AF027CAF for ; Wed, 26 Jul 2017 15:12:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B721B286A9; Wed, 26 Jul 2017 15:12:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id F304327CAF for ; Wed, 26 Jul 2017 15:12:48 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1daNxJ-00005U-00; Wed, 26 Jul 2017 15:10:32 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1daNxH-0008VY-Aj for xen-devel@lists.xenproject.org; Wed, 26 Jul 2017 15:10:31 +0000 Received: from [85.158.139.211] by server-3.bemta-5.messagelabs.com id D7/86-02033-6E0B8795; Wed, 26 Jul 2017 15:10:30 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrDIsWRWlGSWpSXmKPExsVyMfS6i+7DDRW RBod3CFt83zKZyYHR4/CHKywBjFGsmXlJ+RUJrBm/+jQKLoZUtG+ewNbA+Ni9i5GLQ0hgBqPE j7kLmEAcFoGXLBLXGmaCORIC/awSHw+vZu9i5ARy4iQOfrwGZVdKzFj/iRHEFhJQkni9czMTx Kg5TBILe+8xgSTYBAwk9r97AtYgAlR0b9VksCJmgR5GiZcHfzGDJIQFwiU2vH4GZrMIqEp8vX SDDcTmFXCS2DLlB9Q2OYmb5zqBajg4OAWcJZ53KEEsdpJY/Xwh6wRGgQWMDKsYNYpTi8pSi3S NzPSSijLTM0pyEzNzdA0NTPVyU4uLE9NTcxKTivWS83M3MQJDq56BgXEH4+3JfocYJTmYlER5 J5lWRArxJeWnVGYkFmfEF5XmpBYfYpTh4FCS4OVaD5QTLEpNT61Iy8wBBjlMWoKDR0mENwgkz VtckJhbnJkOkTrFaMlx5cq6L0wcUw5sB5KvJvz/xiTEkpeflyolzssO0iAA0pBRmgc3DhaJlx hlpYR5GRkYGIR4ClKLcjNLUOVfMYpzMCoJ85aCTOHJzCuB2/oK6CAmoIPmzCgFOagkESEl1cD oMGd35GrnW9lX5Fs/PT7U61G/gc9j2pwDq7+f3f3tJJeChS+/8RcH9W7/J92fXLZLFM6dvMnT Nqd6DU9SEN+nHeu/7fzzRvbj08veSxT2f8wtuuQZppMfKTanw089m1vQY7bH939q3c4lHVabV qQcMHJk0S7QTkjTe9dr87p1T9fqyHN7DzIosRRnJBpqMRcVJwIAoJalnb8CAAA= X-Env-Sender: olekstysh@gmail.com X-Msg-Ref: server-10.tower-206.messagelabs.com!1501081824!76196416!1 X-Originating-IP: [209.85.215.68] X-SpamReason: No, hits=0.5 required=7.0 tests=BODY_RANDOM_LONG X-StarScan-Received: X-StarScan-Version: 9.4.25; banners=-,-,- X-VirusChecked: Checked Received: (qmail 60378 invoked from network); 26 Jul 2017 15:10:25 -0000 Received: from mail-lf0-f68.google.com (HELO mail-lf0-f68.google.com) (209.85.215.68) by server-10.tower-206.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 26 Jul 2017 15:10:25 -0000 Received: by mail-lf0-f68.google.com with SMTP id t128so6583369lff.3 for ; Wed, 26 Jul 2017 08:10:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=tukd/eSJK+3QwrtZY6NqSlDqpw/MDVEMp1Y5dyqKNyQ=; b=uhb2OGoR7CPBpPLKIZ1rr3GLgxlZkwGMq24VeWO9Uz45WUiLCDyLe4B/2vKHW+WrTD 1j29Hq+SCo9JoJnx5Q+w69CS4Fh+gpqOUzPPlzBM6KBFE5MosMuv9TlcRicq28/ZDitO su6GI69KweoEdhCxxdPgjhxAeTPum3tsztVRKbxmHK8alfrdBVW+JlhbteNJhwQ8tu4X xsKS3AlR33M2x+k+IvLP90WrmtUMwEIr74kOiIFHXl1VTrGK8TGt/7zvAlQIbqUlrXq+ Gh2rgmnwM2EGek58aYW2dqxD1qqzK1A3UI/erRH5iqQIBcr+owsm0Ofe959sxh4XU/dV 7T2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=tukd/eSJK+3QwrtZY6NqSlDqpw/MDVEMp1Y5dyqKNyQ=; b=HwfF/J7UrxXHzofpoLmsqX0wCE8VenIVaDvLHFo4OzZcfBQXcnsQhaLoWERFZWtuRt q0Y5CqkNSgjMgmj42HSlIa4sCaCT7FcS6xAgdmxyA8K8atkMJDfnzonlSUChJ4sGJHPD RhK+ljESq2XmRHDJhZJMH744Q5B/WvuuYxfetpbBjvm3EXqkVVTly8MhjUvH0doPYWVN NMLX4tYWeE4qFnbt4nazZmU8P4cNneY509MuZoPYt9tCJZjJVYstN+N9ZyhiB3PSxu0e f+RN7WYg6SCxsKBaY6hS6FeL7tLNEooayfLLtc6Ga5Q87xE8L2bijsb2akLLEo8t+yEG Guxw== X-Gm-Message-State: AIVw113D7O2wBkbY9yTElACBut/GcHeugroIHBmF1gZllJulFUNFxoZd enpiRxJoM+eTkBAv X-Received: by 10.25.229.211 with SMTP id i80mr485937lfk.146.1501081824206; Wed, 26 Jul 2017 08:10:24 -0700 (PDT) Received: from otyshchenko.kyiv.epam.com (ll-52.209.223.85.sovam.net.ua. [85.223.209.52]) by smtp.gmail.com with ESMTPSA id d203sm3202565lfg.59.2017.07.26.08.10.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 26 Jul 2017 08:10:23 -0700 (PDT) From: Oleksandr Tyshchenko To: xen-devel@lists.xenproject.org Date: Wed, 26 Jul 2017 18:10:03 +0300 Message-Id: <1501081804-4882-7-git-send-email-olekstysh@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1501081804-4882-1-git-send-email-olekstysh@gmail.com> References: <1501081804-4882-1-git-send-email-olekstysh@gmail.com> Cc: Oleksandr Tyshchenko , Julien Grall , Stefano Stabellini Subject: [Xen-devel] [RFC PATCH v1 6/7] iommu/arm: ipmmu-vmsa: Deallocate page table asynchronously X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Oleksandr Tyshchenko This is the PoC how to optimize page table deallocation sequence by splitting it into separate chunks. Use iommu_pt_cleanup_list to queue pages that need to be handled and freed next time. Use free_page_table platform callback to dequeue pages. Signed-off-by: Oleksandr Tyshchenko CC: Julien Grall CC: Stefano Stabellini --- xen/drivers/passthrough/arm/io-pgtable-arm.c | 94 +++++++++++++++++++++++++--- xen/drivers/passthrough/arm/io-pgtable.c | 5 +- xen/drivers/passthrough/arm/io-pgtable.h | 4 +- xen/drivers/passthrough/arm/ipmmu-vmsa.c | 33 ++++++++-- 4 files changed, 119 insertions(+), 17 deletions(-) diff --git a/xen/drivers/passthrough/arm/io-pgtable-arm.c b/xen/drivers/passthrough/arm/io-pgtable-arm.c index c98caa3..7673fda 100644 --- a/xen/drivers/passthrough/arm/io-pgtable-arm.c +++ b/xen/drivers/passthrough/arm/io-pgtable-arm.c @@ -254,6 +254,10 @@ struct arm_lpae_io_pgtable { /* Xen: We deal with domain pages. */ struct page_info *pgd; + /* Xen: To indicate that deallocation sequence is in progress. */ + bool_t cleanup; + /* Xen: To count allocated domain pages. */ + unsigned int page_count; }; typedef u64 arm_lpae_iopte; @@ -329,7 +333,7 @@ static void __arm_lpae_set_pte(arm_lpae_iopte *ptep, arm_lpae_iopte pte, #endif static struct page_info *__arm_lpae_alloc_pages(size_t size, gfp_t gfp, - struct io_pgtable_cfg *cfg) + struct arm_lpae_io_pgtable *data) { struct page_info *pages; unsigned int order = get_order_from_bytes(size); @@ -342,15 +346,21 @@ static struct page_info *__arm_lpae_alloc_pages(size_t size, gfp_t gfp, for (i = 0; i < (1 << order); i ++) clear_and_clean_page(pages + i); + data->page_count += (1<page_count <= 0); + free_domheap_pages(pages, order); + + data->page_count -= (1<pad" variable for + * storing "data" pointer we need during deallocation sequence. The current + * free_page_table platform callback carries the only one "page" argument. + * To perform required calculations with the current (generic) allocator + * implementation we are highly interested in the following fields: + * - data->levels + * - data->pg_shift + * - data->pgd_size + * But, this necessity might be avoided if we integrate allocator code with + * IPMMU-VMSA driver. And these variables will turn into the + * corresponding #define-s. + */ +static void __arm_lpae_free_next_pgtable(struct arm_lpae_io_pgtable *data, + int lvl, struct page_info *page) +{ + if (!data->cleanup) { + /* + * We are here during normal page table maintenance. Just call + * __arm_lpae_free_pgtable(), what we actually had to call. + */ + __arm_lpae_free_pgtable(data, lvl, page); + } else { + /* + * The page table deallocation sequence is in progress. Use some fields + * in struct page_info to pass arguments we will need during handling + * this page back. Queue page to list. + */ + PFN_ORDER(page) = lvl; + page->pad = (u64)&data->iop.ops; + + spin_lock(&iommu_pt_cleanup_lock); + page_list_add_tail(page, &iommu_pt_cleanup_list); + spin_unlock(&iommu_pt_cleanup_lock); + } +} + /* Xen: We deal with domain pages. */ static void __arm_lpae_free_pgtable(struct arm_lpae_io_pgtable *data, int lvl, struct page_info *page) @@ -553,19 +603,41 @@ static void __arm_lpae_free_pgtable(struct arm_lpae_io_pgtable *data, int lvl, if (!pte || iopte_leaf(pte, lvl)) continue; - __arm_lpae_free_pgtable(data, lvl + 1, iopte_deref(pte, data)); + __arm_lpae_free_next_pgtable(data, lvl + 1, iopte_deref(pte, data)); } unmap_domain_page(start); - __arm_lpae_free_pages(page, table_size, &data->iop.cfg); + __arm_lpae_free_pages(page, table_size, data); } -static void arm_lpae_free_pgtable(struct io_pgtable *iop) +/* + * We added extra "page" argument since we want to know what page is processed + * at the moment and should be freed. + * */ +static void arm_lpae_free_pgtable(struct io_pgtable *iop, struct page_info *page) { struct arm_lpae_io_pgtable *data = io_pgtable_to_data(iop); + int lvl; - __arm_lpae_free_pgtable(data, ARM_LPAE_START_LVL(data), data->pgd); - kfree(data); + if (!data->cleanup) { + /* Start page table deallocation sequence from the first level. */ + data->cleanup = true; + lvl = ARM_LPAE_START_LVL(data); + } else { + /* Retrieve the level to continue deallocation sequence from. */ + lvl = PFN_ORDER(page); + PFN_ORDER(page) = 0; + page->pad = 0; + } + + __arm_lpae_free_pgtable(data, lvl, page); + + /* + * Seems, we have already deallocated all pages, so it is time + * to release unfreed resource. + */ + if (!data->page_count) + kfree(data); } /* Xen: We deal with domain pages. */ @@ -889,8 +961,12 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie) cfg->arm_lpae_s1_cfg.mair[0] = reg; cfg->arm_lpae_s1_cfg.mair[1] = 0; + /* Just to be sure */ + data->cleanup = false; + data->page_count = 0; + /* Looking good; allocate a pgd */ - data->pgd = __arm_lpae_alloc_pages(data->pgd_size, GFP_KERNEL, cfg); + data->pgd = __arm_lpae_alloc_pages(data->pgd_size, GFP_KERNEL, data); if (!data->pgd) goto out_free_data; diff --git a/xen/drivers/passthrough/arm/io-pgtable.c b/xen/drivers/passthrough/arm/io-pgtable.c index bfc7020..e25d731 100644 --- a/xen/drivers/passthrough/arm/io-pgtable.c +++ b/xen/drivers/passthrough/arm/io-pgtable.c @@ -77,7 +77,7 @@ struct io_pgtable_ops *alloc_io_pgtable_ops(enum io_pgtable_fmt fmt, * It is the IOMMU driver's responsibility to ensure that the page table * is no longer accessible to the walker by this point. */ -void free_io_pgtable_ops(struct io_pgtable_ops *ops) +void free_io_pgtable_ops(struct io_pgtable_ops *ops, struct page_info *page) { struct io_pgtable *iop; @@ -86,5 +86,6 @@ void free_io_pgtable_ops(struct io_pgtable_ops *ops) iop = container_of(ops, struct io_pgtable, ops); io_pgtable_tlb_flush_all(iop); - io_pgtable_init_table[iop->fmt]->free(iop); + iop->cookie = NULL; + io_pgtable_init_table[iop->fmt]->free(iop, page); } diff --git a/xen/drivers/passthrough/arm/io-pgtable.h b/xen/drivers/passthrough/arm/io-pgtable.h index fb81fcf..df0e21b 100644 --- a/xen/drivers/passthrough/arm/io-pgtable.h +++ b/xen/drivers/passthrough/arm/io-pgtable.h @@ -144,7 +144,7 @@ struct io_pgtable_ops *alloc_io_pgtable_ops(enum io_pgtable_fmt fmt, * * @ops: The ops returned from alloc_io_pgtable_ops. */ -void free_io_pgtable_ops(struct io_pgtable_ops *ops); +void free_io_pgtable_ops(struct io_pgtable_ops *ops, struct page_info *page); /* @@ -201,7 +201,7 @@ static inline void io_pgtable_tlb_sync(struct io_pgtable *iop) */ struct io_pgtable_init_fns { struct io_pgtable *(*alloc)(struct io_pgtable_cfg *cfg, void *cookie); - void (*free)(struct io_pgtable *iop); + void (*free)(struct io_pgtable *iop, struct page_info *page); }; extern struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s1_init_fns; diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c b/xen/drivers/passthrough/arm/ipmmu-vmsa.c index e54b507..2a04800 100644 --- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c +++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c @@ -708,8 +708,8 @@ static void ipmmu_tlb_flush_all(void *cookie) { struct ipmmu_vmsa_domain *domain = cookie; - /* Xen: Just return if context_id has non-existent value */ - if (domain->context_id >= domain->root->num_ctx) + /* Xen: Just return if context is absent or context_id has non-existent value */ + if (!domain || domain->context_id >= domain->root->num_ctx) return; ipmmu_tlb_invalidate(domain); @@ -796,7 +796,9 @@ static int ipmmu_domain_init_context(struct ipmmu_vmsa_domain *domain) */ ret = ipmmu_domain_allocate_context(domain->root, domain); if (ret < 0) { - free_io_pgtable_ops(domain->iop); + /* Pass root page table for this domain as an argument. */ + free_io_pgtable_ops(domain->iop, + maddr_to_page(domain->cfg.arm_lpae_s1_cfg.ttbr[0])); return ret; } @@ -2193,7 +2195,12 @@ static void ipmmu_vmsa_destroy_domain(struct iommu_domain *io_domain) * been detached. */ ipmmu_domain_destroy_context(domain); - free_io_pgtable_ops(domain->iop); + /* + * Pass root page table for this domain as an argument. + * This call will lead to start deallocation sequence. + */ + free_io_pgtable_ops(domain->iop, + maddr_to_page(domain->cfg.arm_lpae_s1_cfg.ttbr[0])); } kfree(domain); @@ -2383,6 +2390,17 @@ static int ipmmu_vmsa_domain_init(struct domain *d, bool use_iommu) return 0; } +/* + * Seems, there is one more page we need to process. So, retrieve + * the pointer and go on deallocation sequence. + */ +static void ipmmu_vmsa_free_page_table(struct page_info *page) +{ + struct io_pgtable_ops *ops = (struct io_pgtable_ops *)page->pad; + + free_io_pgtable_ops(ops, page); +} + static void __hwdom_init ipmmu_vmsa_hwdom_init(struct domain *d) { } @@ -2404,6 +2422,12 @@ static void ipmmu_vmsa_domain_teardown(struct domain *d) ASSERT(list_empty(&xen_domain->contexts)); xfree(xen_domain); dom_iommu(d)->arch.priv = NULL; + /* + * After this point we have all domain resources deallocated, except + * page table which we will deallocate asynchronously. The IOMMU code + * provides us with iommu_pt_cleanup_list and free_page_table platform + * callback what we actually going to use. + */ } static int __must_check ipmmu_vmsa_map_pages(struct domain *d, @@ -2462,6 +2486,7 @@ static void ipmmu_vmsa_dump_p2m_table(struct domain *d) static const struct iommu_ops ipmmu_vmsa_iommu_ops = { .init = ipmmu_vmsa_domain_init, .hwdom_init = ipmmu_vmsa_hwdom_init, + .free_page_table = ipmmu_vmsa_free_page_table, .teardown = ipmmu_vmsa_domain_teardown, .iotlb_flush = ipmmu_vmsa_iotlb_flush, .assign_device = ipmmu_vmsa_assign_dev,