From patchwork Tue Jul 30 07:27:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13746818 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72269C3DA49 for ; Tue, 30 Jul 2024 07:22:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D35E26B0089; Tue, 30 Jul 2024 03:22:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CE5116B008A; Tue, 30 Jul 2024 03:22:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B86316B008C; Tue, 30 Jul 2024 03:22:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 96AAE6B0089 for ; Tue, 30 Jul 2024 03:22:29 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 0A7C78013C for ; Tue, 30 Jul 2024 07:22:29 +0000 (UTC) X-FDA: 82395576018.06.E0A75F5 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf26.hostedemail.com (Postfix) with ESMTP id 60B3B140014 for ; Tue, 30 Jul 2024 07:22:26 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Vja3T545; spf=pass (imf26.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1722324074; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9GhNSv1i2xQPHGPrJ2SZXGsmCYJWwUPS35RMZccTDpo=; b=O0JWQdrjnBGrlyscxVHE34vup8UwpvBC+T5T96r0luPvwigsv/du0VAgyol0CD+BZQQbsQ Awm7iowv0FIWmjiJSqDJpZzckp+J08H8JA8/TYo1fLWs8wpOthdCjgFJMU6Fs9ozaiJKHc GQvOZtc00s84xf4Y/Vh+u6KA900XEMg= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Vja3T545; spf=pass (imf26.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1722324074; a=rsa-sha256; cv=none; b=wUousZt7CdgoXa1q1F6HPJ0LdM/Bbf3uMXl0qFNg61nDvy/AF+Eej/KpC0dlnPLtlGLq5n uU0mUU8LYseVdb1fSdfAXptOeewIzX1Bqgh5oEGJOnzBC1X4ZZm0sFB/DVWNmiEoyNO4ch BPnvZkRWoaxzngXNJJGKRpx9CBCDadw= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 3321061DAC; Tue, 30 Jul 2024 07:22:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B5824C4AF09; Tue, 30 Jul 2024 07:22:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722324144; bh=HFILjDyuOkFueGdpeZ+expeCNrW1pem0IxRTIfXPmJE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Vja3T5454h4Q+VtrSRmXah9kscyuQ0sdUaWF4cc6GQNdR4jZLqcvzt/NApj4QHVhI pqrlzT+GWHsnAid/s7t2kxwJYFUtrUPlxfokjC/Txb/CQVYpaKg0WAWtESCPzRq71x NXeojkPNmEJrPbsYGdO5TaC1GuWCiXQv76PuZ6FU1/c+ZAWYWlluAS+r9mBDcGbOt5 1D/+y4VcHKQVl6RBo7gAGoNGP9Euq87+ecG3V1+RQ5tu2ME9dlPBD4ssUxQBUeIFyI uYUT3ZAY3PQJ/mREouQl40/x4dEbUyqiFsmeMNBhlEBFnnYo1M91KLOZ77KIeR/1sW TD6wdpb7tGFTQ== From: alexs@kernel.org To: Will Deacon , "Aneesh Kumar K . V" , Nick Piggin , Peter Zijlstra , Russell King , Catalin Marinas , Brian Cain , WANG Xuerui , Geert Uytterhoeven , Jonas Bonn , Stefan Kristiansson , Stafford Horne , Michael Ellerman , Naveen N Rao , Paul Walmsley , Albert Ou , Thomas Gleixner , Borislav Petkov , Dave Hansen , x86@kernel.org, "H . Peter Anvin" , Andy Lutomirski , Bibo Mao , Baolin Wang , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Qi Zheng , Vishal Moola , "Aneesh Kumar K . V" , Kemeng Shi , Lance Yang , Peter Xu , Barry Song , linux-s390@vger.kernel.org Cc: Guo Ren , Christophe Leroy , Palmer Dabbelt , Mike Rapoport , Oscar Salvador , Alexandre Ghiti , Jisheng Zhang , Samuel Holland , Anup Patel , Josh Poimboeuf , Breno Leitao , Alexander Gordeev , Gerald Schaefer , Hugh Dickins , David Hildenbrand , Ryan Roberts , Matthew Wilcox , Alex Shi , "Naveen N . Rao" Subject: [RFC PATCH 11/18] mm/pgtable: introduce ptdesc_pfn and use ptdesc in free_pte_range() Date: Tue, 30 Jul 2024 15:27:12 +0800 Message-ID: <20240730072719.3715016-1-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240730064712.3714387-1-alexs@kernel.org> References: <20240730064712.3714387-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 60B3B140014 X-Stat-Signature: pkj7s76tms4pa4pwzhjmkyxjftb4bmpt X-Rspam-User: X-HE-Tag: 1722324146-82959 X-HE-Meta: U2FsdGVkX1+lA8NKVLJ2sZOt6d0mlFOxvu3uMvMxa8IJ+U/+ZC5jf2U61eBJw9cO9nnM0t48LFgCbvFdUuEozsOdaTyiVWOQP8LDqtp4K7K9xVSV3YSstzhMQw6zPRTA09HcyHahHNEcNlo54mvR+1FFCsqCkOztZLQYdMdNAD9/pYyF24c2AZi3gA7rlzzeyJp+BgQa0Bi5wuhs1RLmjAFfu6Xy3zWqEchiLcxx6qAFl6ef8ZHhFR8G3TaunqhzAEECYZ2wck2ahU8vY93JsvxCWVdAyQiLebGr9MaNVo0X4tkSglm+xVLbBIVzYHpI42qvB2DlGq+VSxGLnvC7aP7mUUdyr1KWZnzhZqkVYEqo7RZFGtnvpybvYA9qgQIUBpIoVEEZL2GB+FUNFsk0GsfeIpS1EBj7Uo74ii1T+J6XeOhGJIMZfp/jy97Dm/VWYynQg6dJurmZaPKa44wiItO0lQcl+m9LYtQ5wjYbqZkvT4dZpWLE2UZFiL2ynIoRNpRRoIgO7w0ChG2czUbD4tmKxug4nS9w5JwdZ+5UWWiPHeQc2OtGqs9WKaoJ9gMU2dKaP5xHm9L2MEFOYv75RrWt86iT8VXsEfVJsL5pym267EkfGQiILzozz8hmybuCSGNi5g4ZYuDe7toJX3vi+FlNI8jVyqJH9GuRSRJ52d36eimfIlmkMKh85E7HvR58TLEFkHsPZifgPTpR7ROEedBCiBcfkS6pArNGILV8xRneIfWaP6EOb5kgN4Ko9/D2zlKlg5qBUlgFomemfviRb+lxCd5AT9RzgjkuJBijOwI1GJjPppMYam7QYARUrM+VDHvGxwa+DhBwLb75hDuxVIdngcnsxQySBYp4nWgmkHCoAlV4EbAYRkXi4mcgWRPmpQVGdvVqopZJXDydZ/2uSZ1O1s7XSDPfw8C68ZNikbmWX/IACNL3GBB4xaYdDIlxEtvpK9HLqisJgw+xQso ZD6+scsb qP9ARVKihQkSPc80Odrh08Jr/Wc36iRW1c4yd39d1INPtQIUZ1lJeHs81Q09pj1lNxqR43vhRG71tjvyDd+e5488zebMtgbuvITmM50sxucSLHJBeSGcNkE7Csdx17/XQGxgRF/MlQBhDAhaOU5IyeHG4FQVsSehzmQEuYuqPVR1aHVdAD6Kjq2JbbcaJ7v+zNDv6fnwUXFK/dQnGfLw3iPtKTIfcY7L4RzFHGqekgZ9BrvOCNrfexLhTWm+uEDeDYGPkIcCS8//6f7toaarkR6kDyRpCwx93yctvQ9rra4LTfDKU1CLUdQ3+FNbBCX1BUsko0ElN2Ip7lrHJYhDrFXnxjyhgNGU8ccG2MaYeAy3cnGu+K4cIJVyEdMU2LPHZIPs/zKCD+iCvWX2BIgJGvPnktqgvu62p43XOQPP9pZYLP60bf8DBguQdt+0mITx3KKguWPpQfmisqMN7paXmcc0lQER24T9k8iNZRW3qc2Cq0VV6G623WY4qL/Yrhl5YSekwQSKYPol1FqvmbHXFAh8AgW2NUsywz0i27sLjvo9SyklpG9pmxtUDOpybwAV3OVHpevhse1/YiGlXok/iOYeIS+kDbufpwb4R+6VMi7hSpO7xe+TOliPxsky6IH+DgHLMZ6AOo1/2z4ZME41Tvxe185eEpJ7fyvRaE+rQTBV5st5SrVpupYMWSxQp1ZBZJ5ofXxzH+7YThnhInafCkMeVlLkYJGuO/ege840ojZGDfLMRzoP+jUNvVj6Yss4A94tv/Qg8JZqIiKc5LwBFDGoVq/J1OM3xnwcR+IsCQNodic3QZdapjkvrol+mJ7X5tCGezK4eWEzG0zYqTopYZ30uODwFVbQ5XMW+WUvsIXYlRHwTVpnE0rBWbSB/vhcPHxM65svXolGfshZjKOiVxxCklXstoEuWdAcwI8DZNbwkUgWPy+Q/tPDrt78NJuJSDT4ScHbL5Gc3tdQ9Hhrz/xMwNTQi qzg8+DpD EZYjvjLXzs3PzsFz0OLAgCIgWHdgG+iBcZsKhCidABqnAUrxbpGjA/VPxaMa6xVXahky337WkAKYEGG5YHwpv3B8ov1RybRLbifXR0L3kWV23r/UJ+hL2BI3qfzzC5GO3Dta50NJOf/yrulrHsslC+25qV2AzXpD6kyWvWwuf9UTyK6uj+XRfXXyuIygx9p7Yv+x/v7828qfE0LCsDIuvQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alex Shi Replace pgtable_t by ptdesc in free_pte_range and it's callee pte_free_tlb series functions. And save some converters now. We have to use type casting for pmd_pgtable() instead of page_ptdesc() helper since different arch has different type of pgtable_t. btw, we can not simplify pmd_ptdesc() via replace pmd_pgtable_page by pmd_page, since some arch may have no pmd_page yet. Signed-off-by: Alex Shi Cc: Anup Patel Cc: Samuel Holland Cc: Jisheng Zhang Cc: Alexandre Ghiti Cc: Oscar Salvador Cc: Palmer Dabbelt Cc: Guo Ren Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-openrisc@vger.kernel.org Cc: linux-m68k@lists.linux-m68k.org Cc: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-mm@kvack.org Cc: linux-arch@vger.kernel.org Cc: Andy Lutomirski Cc: H. Peter Anvin Cc: x86@kernel.org Cc: Dave Hansen Cc: Borislav Petkov Cc: Thomas Gleixner Cc: Naveen N. Rao Cc: Christophe Leroy Cc: Michael Ellerman Cc: Stafford Horne Cc: Stefan Kristiansson Cc: Jonas Bonn Cc: Geert Uytterhoeven Cc: Catalin Marinas Cc: Russell King Cc: Peter Zijlstra Cc: Nick Piggin Cc: Aneesh Kumar K.V Cc: Will Deacon Cc: Breno Leitao Cc: Josh Poimboeuf Cc: Vishal Moola Cc: Mike Rapoport --- arch/arm/include/asm/tlb.h | 4 +--- arch/arm64/include/asm/tlb.h | 4 +--- arch/csky/include/asm/pgalloc.h | 4 ++-- arch/hexagon/include/asm/pgalloc.h | 4 ++-- arch/loongarch/include/asm/pgalloc.h | 4 ++-- arch/m68k/include/asm/motorola_pgalloc.h | 4 ++-- arch/openrisc/include/asm/pgalloc.h | 4 ++-- arch/powerpc/include/asm/book3s/32/pgalloc.h | 2 +- arch/powerpc/include/asm/book3s/64/pgalloc.h | 2 +- arch/riscv/include/asm/pgalloc.h | 8 +++----- arch/x86/include/asm/pgalloc.h | 4 ++-- arch/x86/mm/pgtable.c | 6 +++--- include/linux/mm.h | 14 ++++++++++++++ mm/memory.c | 3 ++- 14 files changed, 38 insertions(+), 29 deletions(-) diff --git a/arch/arm/include/asm/tlb.h b/arch/arm/include/asm/tlb.h index f40d06ad5d2a..ed6aa4255518 100644 --- a/arch/arm/include/asm/tlb.h +++ b/arch/arm/include/asm/tlb.h @@ -37,10 +37,8 @@ static inline void __tlb_remove_table(void *_table) #include static inline void -__pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, unsigned long addr) +__pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *ptdesc, unsigned long addr) { - struct ptdesc *ptdesc = page_ptdesc(pte); - pagetable_pte_dtor(ptdesc); #ifndef CONFIG_ARM_LPAE diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h index a947c6e784ed..cee7234af6e7 100644 --- a/arch/arm64/include/asm/tlb.h +++ b/arch/arm64/include/asm/tlb.h @@ -77,11 +77,9 @@ static inline void tlb_flush(struct mmu_gather *tlb) last_level, tlb_level); } -static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, +static inline void __pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *ptdesc, unsigned long addr) { - struct ptdesc *ptdesc = page_ptdesc(pte); - pagetable_pte_dtor(ptdesc); tlb_remove_ptdesc(tlb, ptdesc); } diff --git a/arch/csky/include/asm/pgalloc.h b/arch/csky/include/asm/pgalloc.h index 9c84c9012e53..b24b4611436e 100644 --- a/arch/csky/include/asm/pgalloc.h +++ b/arch/csky/include/asm/pgalloc.h @@ -63,8 +63,8 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm) #define __pte_free_tlb(tlb, pte, address) \ do { \ - pagetable_pte_dtor(page_ptdesc(pte)); \ - tlb_remove_page_ptdesc(tlb, page_ptdesc(pte)); \ + pagetable_pte_dtor(pte); \ + tlb_remove_page_ptdesc(tlb, pte); \ } while (0) extern void pagetable_init(void); diff --git a/arch/hexagon/include/asm/pgalloc.h b/arch/hexagon/include/asm/pgalloc.h index 55988625e6fb..a3e082e54b74 100644 --- a/arch/hexagon/include/asm/pgalloc.h +++ b/arch/hexagon/include/asm/pgalloc.h @@ -89,8 +89,8 @@ static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, #define __pte_free_tlb(tlb, pte, addr) \ do { \ - pagetable_pte_dtor((page_ptdesc(pte))); \ - tlb_remove_page_ptdesc((tlb), (page_ptdesc(pte))); \ + pagetable_pte_dtor((pte)); \ + tlb_remove_page_ptdesc((tlb), (pte)); \ } while (0) #endif diff --git a/arch/loongarch/include/asm/pgalloc.h b/arch/loongarch/include/asm/pgalloc.h index 4e2d6b7ca2ee..c96d7160babc 100644 --- a/arch/loongarch/include/asm/pgalloc.h +++ b/arch/loongarch/include/asm/pgalloc.h @@ -46,8 +46,8 @@ extern pgd_t *pgd_alloc(struct mm_struct *mm); #define __pte_free_tlb(tlb, pte, address) \ do { \ - pagetable_pte_dtor(page_ptdesc(pte)); \ - tlb_remove_page_ptdesc((tlb), page_ptdesc(pte)); \ + pagetable_pte_dtor(pte); \ + tlb_remove_page_ptdesc((tlb), pte); \ } while (0) #ifndef __PAGETABLE_PMD_FOLDED diff --git a/arch/m68k/include/asm/motorola_pgalloc.h b/arch/m68k/include/asm/motorola_pgalloc.h index f6bb375971dc..f9ee5ec4574d 100644 --- a/arch/m68k/include/asm/motorola_pgalloc.h +++ b/arch/m68k/include/asm/motorola_pgalloc.h @@ -44,10 +44,10 @@ static inline void pte_free(struct mm_struct *mm, struct ptdesc *ptdesc) free_pointer_table(ptdesc_page(ptdesc), TABLE_PTE); } -static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pgtable, +static inline void __pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *ptdesc, unsigned long address) { - free_pointer_table(pgtable, TABLE_PTE); + free_pointer_table(ptdesc_page(ptdesc), TABLE_PTE); } diff --git a/arch/openrisc/include/asm/pgalloc.h b/arch/openrisc/include/asm/pgalloc.h index c6a73772a546..2251d940c3d8 100644 --- a/arch/openrisc/include/asm/pgalloc.h +++ b/arch/openrisc/include/asm/pgalloc.h @@ -68,8 +68,8 @@ extern pte_t *pte_alloc_one_kernel(struct mm_struct *mm); #define __pte_free_tlb(tlb, pte, addr) \ do { \ - pagetable_pte_dtor(page_ptdesc(pte)); \ - tlb_remove_page_ptdesc((tlb), (page_ptdesc(pte))); \ + pagetable_pte_dtor(pte); \ + tlb_remove_page_ptdesc((tlb), (pte)); \ } while (0) #endif diff --git a/arch/powerpc/include/asm/book3s/32/pgalloc.h b/arch/powerpc/include/asm/book3s/32/pgalloc.h index dd4eb3063175..a435c84d1f9a 100644 --- a/arch/powerpc/include/asm/book3s/32/pgalloc.h +++ b/arch/powerpc/include/asm/book3s/32/pgalloc.h @@ -64,7 +64,7 @@ static inline void __tlb_remove_table(void *_table) pgtable_free(table, shift); } -static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t table, +static inline void __pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *table, unsigned long address) { pgtable_free_tlb(tlb, table, 0); diff --git a/arch/powerpc/include/asm/book3s/64/pgalloc.h b/arch/powerpc/include/asm/book3s/64/pgalloc.h index eb7d2ca59f62..675eca34fe40 100644 --- a/arch/powerpc/include/asm/book3s/64/pgalloc.h +++ b/arch/powerpc/include/asm/book3s/64/pgalloc.h @@ -167,7 +167,7 @@ static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd, *pmd = __pmd(__pgtable_ptr_val(pte_page) | PMD_VAL_BITS); } -static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t table, +static inline void __pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *table, unsigned long address) { pgtable_free_tlb(tlb, table, PTE_INDEX); diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h index f52264304f77..63596efcd528 100644 --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -183,13 +183,11 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, #endif /* __PAGETABLE_PMD_FOLDED */ -static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, +static inline void __pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *pte, unsigned long addr) { - struct ptdesc *ptdesc = page_ptdesc(pte); - - pagetable_pte_dtor(ptdesc); - riscv_tlb_remove_ptdesc(tlb, ptdesc); + pagetable_pte_dtor(pte); + riscv_tlb_remove_ptdesc(tlb, pte); } #endif /* CONFIG_MMU */ diff --git a/arch/x86/include/asm/pgalloc.h b/arch/x86/include/asm/pgalloc.h index 497c757b5b98..06a9a5867a86 100644 --- a/arch/x86/include/asm/pgalloc.h +++ b/arch/x86/include/asm/pgalloc.h @@ -53,9 +53,9 @@ extern void pgd_free(struct mm_struct *mm, pgd_t *pgd); extern struct ptdesc *pte_alloc_one(struct mm_struct *); -extern void ___pte_free_tlb(struct mmu_gather *tlb, struct page *pte); +extern void ___pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *pte); -static inline void __pte_free_tlb(struct mmu_gather *tlb, struct page *pte, +static inline void __pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *pte, unsigned long address) { ___pte_free_tlb(tlb, pte); diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index c27d15cd01b9..3cf9c0d25dbd 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -50,10 +50,10 @@ static int __init setup_userpte(char *arg) } early_param("userpte", setup_userpte); -void ___pte_free_tlb(struct mmu_gather *tlb, struct page *pte) +void ___pte_free_tlb(struct mmu_gather *tlb, struct ptdesc *pte) { - pagetable_pte_dtor(page_ptdesc(pte)); - paravirt_release_pte(page_to_pfn(pte)); + pagetable_pte_dtor(pte); + paravirt_release_pte(ptdesc_pfn(pte)); paravirt_tlb_remove_table(tlb, pte); } diff --git a/include/linux/mm.h b/include/linux/mm.h index 381750f41767..7424f964dff3 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2859,6 +2859,20 @@ static inline bool pagetable_is_reserved(struct ptdesc *pt) return folio_test_reserved(ptdesc_folio(pt)); } +/** + * ptdesc_pfn - Return the Page Frame Number of a ptdesc. + * @ptdesc: The ptdesc. + * + * A ptdesc may contain multiple pages. The pages have consecutive + * Page Frame Numbers. + * + * Return: The Page Frame Number of the first page in the ptdesc. + */ +static inline unsigned long ptdesc_pfn(struct ptdesc *ptdesc) +{ + return page_to_pfn(ptdesc_page(ptdesc)); +} + /** * pagetable_alloc - Allocate pagetables * @gfp: GFP flags diff --git a/mm/memory.c b/mm/memory.c index 3014168e7296..27c2f63b7487 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -189,7 +189,8 @@ void mm_trace_rss_stat(struct mm_struct *mm, int member) static void free_pte_range(struct mmu_gather *tlb, pmd_t *pmd, unsigned long addr) { - pgtable_t token = pmd_pgtable(*pmd); + struct ptdesc *token = (struct ptdesc *)pmd_pgtable(*pmd); + pmd_clear(pmd); pte_free_tlb(tlb, token, addr); mm_dec_nr_ptes(tlb->mm);