From patchwork Mon Mar 17 14:16:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14019345 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97E57C28B30 for ; Mon, 17 Mar 2025 14:21:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3385D280003; Mon, 17 Mar 2025 10:21:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2E7F1280001; Mon, 17 Mar 2025 10:21:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 163B8280003; Mon, 17 Mar 2025 10:21:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 97AB9280001 for ; Mon, 17 Mar 2025 10:21:37 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 81BC94DD79 for ; Mon, 17 Mar 2025 14:21:38 +0000 (UTC) X-FDA: 83231256276.02.88F073F Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf24.hostedemail.com (Postfix) with ESMTP id 4329F180010 for ; Mon, 17 Mar 2025 14:21:36 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf24.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742221296; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/qdT7+4ubPgbgEWg/jUXvx/IzFk/7KwkKCTNSsj59Nw=; b=3ZjOFuqCxJ59gXcX4vQespVxHzLXZZzzJ93ePztTPJdkSCc8tNZacZx0Cu5GDxGmiuQH/n r0fCDVJj/BMxR1tYl/Cz1XUujImzJpyiGKMhHa7E9vabe+X/LAm2AMXMwWZ+58RK3lAqDU kg5BWoVieJds7LUd1Oyi6lGuJgpfgtQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742221296; a=rsa-sha256; cv=none; b=sEbUDNgezUKivnQTxVYXLMw5G7ozl7wZTUPN/ESkre+eKtfigHKl/5QAMV0c5EcTby3V6o Cr7gNOKSxRn3WBdW4ZHWgCK2gLUlqrilLA22HDUZ+muNiiGttMdE3po4ZN5uRGq6e7L2T5 Jj8kLXmp+piRcykzD6l5aMgLslsjyyc= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf24.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 42C2F1688; Mon, 17 Mar 2025 07:21:44 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EFD3D3F63F; Mon, 17 Mar 2025 07:21:30 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org Subject: [PATCH 01/11] mm: Pass mm down to pagetable_{pte,pmd}_ctor Date: Mon, 17 Mar 2025 14:16:50 +0000 Message-ID: <20250317141700.3701581-2-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250317141700.3701581-1-kevin.brodsky@arm.com> References: <20250317141700.3701581-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 4329F180010 X-Stat-Signature: n63yy1iqyaajbrmryyjkyjwgzbiuuipf X-HE-Tag: 1742221296-891004 X-HE-Meta: U2FsdGVkX18W1KS7iijiF0psOeHdNg+hXYk7TZcgbVrcSg7qVFjkCgT4LJjynR5mz+2Yx06C00Kh4zUvwd9LGXJOBKbbd5wDsd5rk4JwrNBrKZQN/eAwmXkZDjHuS8gt3Zgk5Xxh0ElbhJp/fpTXfNheduOPj4fDYKEar8FtCPEL2AwEnOaCcu7FSfqGL4dJ0J3xruVqgQ5/Ns9AVxoGM9Pr1A104V72SlKPmU5Q3jLSGXApzeg9tiLNv0lX3qx19AMjuF/Pa1lcqVLk0HNiw2GdnX9yBw1eSr199kgM94kdsdnTqGSMP9CUZt17rxWCs4WRUs6rOfViLcETaviEKAqYlsf4BKycafJDXiFBXk3pYoRdxoDQp4jyAuPgG0cEdhF1sk4cb82vWlGhdGIWeLMKNdM6elOWJXqlRaYsM6H+yrdM5ZKdr/k7Yy7eRU/uuAs5/ZTdIzrgCabsbxlSRdYbzzxhMk0u0SyMuGpdGPyzIYl00Zn/WcHEsgrPgqWnIDAecovZHOvQX8HRXoAm2r9v96mDNXXZxAb+6W/UFFd8KHT1DK7tD4PyGNkdviTNbSeeiqXgKgsYP5Kb4ZMzquK2fDFb13xB0YYFLGpdTfolNVbUUGPI6P8JUCDKJ91/WEtoQoD6Q/PB/Fufp/4yY2Yn2mqdHPVtBQXOw5GyYcv+sk4hW9aJh1qwM0iApN9O3ayXbb5PVH2/TfEsD6FqdC4Lb3cNytmwqCfyw3FFI5n66lNa8zSE10cMQWITH5pW2D9JdGMvb/n6z3zt3RXQ0iZQhl0yWuKZevotuja+82wxg+THFFuq4l8ggKycQE8HZCxD3g5Uk1QFyc1FxH70ZH/lR+ZMNx+FI0w/2XggCE91Fqk5PhJoWa+hPnCybCI0rENUpzdiL3XOenaETrRt6suscd+ibDTfifdLFQS+74H588DPrDQfKZiYZmdfbTiArRxqNVdXp93JzhTfNZ0 oIO4KLm1 KPPqeefnLP3XIk95Mfp5AekyWEJ+G584PMFpa7iT8pirLzdnOC+vwaib2oxzW9E+YClyvmb8vyS+hw+aMW2WU2sXuJf0VhR1zWurVpR+rxj3jUpgBTC82GUyAok5Ne6mfdDmqdJzW3vFz8ykbhOJlhe0ZkQDTL0wGTh6su7Ytlq4C6hT7RapDCsakUUMDF85uVNjyKKiN+TwRLSw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In preparation for calling constructors for all kernel page tables while eliding unnecessary ptlock initialisation, let's pass down the associated mm to the PTE/PMD level ctors. (These are the two levels where ptlocks are used.) In most cases the mm is already around at the point of calling the ctor so we simply pass it down. This is however not the case for special page table allocators: * arch/arm/mm/mmu.c * arch/arm64/mm/mmu.c * arch/riscv/mm/init.c In those cases, the page tables being allocated are either for standard kernel memory (init_mm) or special page directories, which may not be associated to any mm. For now let's pass NULL as mm; this will be refined where possible in future patches. No functional change in this patch. Signed-off-by: Kevin Brodsky --- arch/arm/mm/mmu.c | 2 +- arch/arm64/mm/mmu.c | 4 ++-- arch/loongarch/include/asm/pgalloc.h | 2 +- arch/m68k/include/asm/mcf_pgalloc.h | 2 +- arch/m68k/include/asm/motorola_pgalloc.h | 10 +++++----- arch/m68k/mm/motorola.c | 6 +++--- arch/mips/include/asm/pgalloc.h | 2 +- arch/parisc/include/asm/pgalloc.h | 2 +- arch/powerpc/mm/book3s64/pgtable.c | 2 +- arch/powerpc/mm/pgtable-frag.c | 2 +- arch/riscv/mm/init.c | 4 ++-- arch/s390/include/asm/pgalloc.h | 2 +- arch/s390/mm/pgalloc.c | 2 +- arch/sparc/mm/init_64.c | 2 +- arch/sparc/mm/srmmu.c | 2 +- arch/x86/mm/pgtable.c | 2 +- include/asm-generic/pgalloc.h | 4 ++-- include/linux/mm.h | 6 ++++-- 18 files changed, 30 insertions(+), 28 deletions(-) diff --git a/arch/arm/mm/mmu.c b/arch/arm/mm/mmu.c index f02f872ea8a9..edb7f56b7c91 100644 --- a/arch/arm/mm/mmu.c +++ b/arch/arm/mm/mmu.c @@ -735,7 +735,7 @@ static void *__init late_alloc(unsigned long sz) void *ptdesc = pagetable_alloc(GFP_PGTABLE_KERNEL & ~__GFP_HIGHMEM, get_order(sz)); - if (!ptdesc || !pagetable_pte_ctor(ptdesc)) + if (!ptdesc || !pagetable_pte_ctor(NULL, ptdesc)) BUG(); return ptdesc_to_virt(ptdesc); } diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 1dfe1a8efdbe..437d4977bcf5 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -494,9 +494,9 @@ static phys_addr_t pgd_pgtable_alloc(int shift) * folded, and if so pagetable_pte_ctor() becomes nop. */ if (shift == PAGE_SHIFT) - BUG_ON(!pagetable_pte_ctor(ptdesc)); + BUG_ON(!pagetable_pte_ctor(NULL, ptdesc)); else if (shift == PMD_SHIFT) - BUG_ON(!pagetable_pmd_ctor(ptdesc)); + BUG_ON(!pagetable_pmd_ctor(NULL, ptdesc)); return pa; } diff --git a/arch/loongarch/include/asm/pgalloc.h b/arch/loongarch/include/asm/pgalloc.h index 7211dff8c969..57b7dbcee259 100644 --- a/arch/loongarch/include/asm/pgalloc.h +++ b/arch/loongarch/include/asm/pgalloc.h @@ -72,7 +72,7 @@ static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address) if (!ptdesc) return NULL; - if (!pagetable_pmd_ctor(ptdesc)) { + if (!pagetable_pmd_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/m68k/include/asm/mcf_pgalloc.h b/arch/m68k/include/asm/mcf_pgalloc.h index 4c648b51e7fd..465a71101b7d 100644 --- a/arch/m68k/include/asm/mcf_pgalloc.h +++ b/arch/m68k/include/asm/mcf_pgalloc.h @@ -48,7 +48,7 @@ static inline pgtable_t pte_alloc_one(struct mm_struct *mm) if (!ptdesc) return NULL; - if (!pagetable_pte_ctor(ptdesc)) { + if (!pagetable_pte_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/m68k/include/asm/motorola_pgalloc.h b/arch/m68k/include/asm/motorola_pgalloc.h index 5abe7da8ac5a..1091fb0affbe 100644 --- a/arch/m68k/include/asm/motorola_pgalloc.h +++ b/arch/m68k/include/asm/motorola_pgalloc.h @@ -15,7 +15,7 @@ enum m68k_table_types { }; extern void init_pointer_table(void *table, int type); -extern void *get_pointer_table(int type); +extern void *get_pointer_table(struct mm_struct *mm, int type); extern int free_pointer_table(void *table, int type); /* @@ -26,7 +26,7 @@ extern int free_pointer_table(void *table, int type); static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) { - return get_pointer_table(TABLE_PTE); + return get_pointer_table(mm, TABLE_PTE); } static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) @@ -36,7 +36,7 @@ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) static inline pgtable_t pte_alloc_one(struct mm_struct *mm) { - return get_pointer_table(TABLE_PTE); + return get_pointer_table(mm, TABLE_PTE); } static inline void pte_free(struct mm_struct *mm, pgtable_t pgtable) @@ -53,7 +53,7 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pgtable, static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address) { - return get_pointer_table(TABLE_PMD); + return get_pointer_table(mm, TABLE_PMD); } static inline int pmd_free(struct mm_struct *mm, pmd_t *pmd) @@ -75,7 +75,7 @@ static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd) static inline pgd_t *pgd_alloc(struct mm_struct *mm) { - return get_pointer_table(TABLE_PGD); + return get_pointer_table(mm, TABLE_PGD); } diff --git a/arch/m68k/mm/motorola.c b/arch/m68k/mm/motorola.c index 73651e093c4d..6ab3ef39ba7a 100644 --- a/arch/m68k/mm/motorola.c +++ b/arch/m68k/mm/motorola.c @@ -139,7 +139,7 @@ void __init init_pointer_table(void *table, int type) return; } -void *get_pointer_table(int type) +void *get_pointer_table(struct mm_struct *mm, int type) { ptable_desc *dp = ptable_list[type].next; unsigned int mask = list_empty(&ptable_list[type]) ? 0 : PD_MARKBITS(dp); @@ -164,10 +164,10 @@ void *get_pointer_table(int type) * m68k doesn't have SPLIT_PTE_PTLOCKS for not having * SMP. */ - pagetable_pte_ctor(virt_to_ptdesc(page)); + pagetable_pte_ctor(mm, virt_to_ptdesc(page)); break; case TABLE_PMD: - pagetable_pmd_ctor(virt_to_ptdesc(page)); + pagetable_pmd_ctor(mm, virt_to_ptdesc(page)); break; case TABLE_PGD: pagetable_pgd_ctor(virt_to_ptdesc(page)); diff --git a/arch/mips/include/asm/pgalloc.h b/arch/mips/include/asm/pgalloc.h index 26c7a6ede983..415237af4029 100644 --- a/arch/mips/include/asm/pgalloc.h +++ b/arch/mips/include/asm/pgalloc.h @@ -65,7 +65,7 @@ static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address) if (!ptdesc) return NULL; - if (!pagetable_pmd_ctor(ptdesc)) { + if (!pagetable_pmd_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/parisc/include/asm/pgalloc.h b/arch/parisc/include/asm/pgalloc.h index 2ca74a56415c..3b84ee93edaa 100644 --- a/arch/parisc/include/asm/pgalloc.h +++ b/arch/parisc/include/asm/pgalloc.h @@ -39,7 +39,7 @@ static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address) ptdesc = pagetable_alloc(gfp, PMD_TABLE_ORDER); if (!ptdesc) return NULL; - if (!pagetable_pmd_ctor(ptdesc)) { + if (!pagetable_pmd_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/pgtable.c index ce64abea9e3e..83afc4e72088 100644 --- a/arch/powerpc/mm/book3s64/pgtable.c +++ b/arch/powerpc/mm/book3s64/pgtable.c @@ -423,7 +423,7 @@ static pmd_t *__alloc_for_pmdcache(struct mm_struct *mm) ptdesc = pagetable_alloc(gfp, 0); if (!ptdesc) return NULL; - if (!pagetable_pmd_ctor(ptdesc)) { + if (!pagetable_pmd_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/powerpc/mm/pgtable-frag.c b/arch/powerpc/mm/pgtable-frag.c index 713268ccb1a0..387e9b1fe12c 100644 --- a/arch/powerpc/mm/pgtable-frag.c +++ b/arch/powerpc/mm/pgtable-frag.c @@ -61,7 +61,7 @@ static pte_t *__alloc_for_ptecache(struct mm_struct *mm, int kernel) ptdesc = pagetable_alloc(PGALLOC_GFP | __GFP_ACCOUNT, 0); if (!ptdesc) return NULL; - if (!pagetable_pte_ctor(ptdesc)) { + if (!pagetable_pte_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 15b2eda4c364..703c3648cfa9 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -409,7 +409,7 @@ static phys_addr_t __meminit alloc_pte_late(uintptr_t va) { struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL & ~__GFP_HIGHMEM, 0); - BUG_ON(!ptdesc || !pagetable_pte_ctor(ptdesc)); + BUG_ON(!ptdesc || !pagetable_pte_ctor(NULL, ptdesc)); return __pa((pte_t *)ptdesc_address(ptdesc)); } @@ -489,7 +489,7 @@ static phys_addr_t __meminit alloc_pmd_late(uintptr_t va) { struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL & ~__GFP_HIGHMEM, 0); - BUG_ON(!ptdesc || !pagetable_pmd_ctor(ptdesc)); + BUG_ON(!ptdesc || !pagetable_pmd_ctor(NULL, ptdesc)); return __pa((pmd_t *)ptdesc_address(ptdesc)); } diff --git a/arch/s390/include/asm/pgalloc.h b/arch/s390/include/asm/pgalloc.h index b19b6ed2ab53..cd65b93d87e9 100644 --- a/arch/s390/include/asm/pgalloc.h +++ b/arch/s390/include/asm/pgalloc.h @@ -98,7 +98,7 @@ static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long vmaddr) if (!table) return NULL; crst_table_init(table, _SEGMENT_ENTRY_EMPTY); - if (!pagetable_pmd_ctor(virt_to_ptdesc(table))) { + if (!pagetable_pmd_ctor(mm, virt_to_ptdesc(table))) { crst_table_free(mm, table); return NULL; } diff --git a/arch/s390/mm/pgalloc.c b/arch/s390/mm/pgalloc.c index 30387a6e98ff..35b0ab6fab24 100644 --- a/arch/s390/mm/pgalloc.c +++ b/arch/s390/mm/pgalloc.c @@ -170,7 +170,7 @@ unsigned long *page_table_alloc(struct mm_struct *mm) ptdesc = pagetable_alloc(GFP_KERNEL, 0); if (!ptdesc) return NULL; - if (!pagetable_pte_ctor(ptdesc)) { + if (!pagetable_pte_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c index 05882bca5b73..cd60a0a8ca0e 100644 --- a/arch/sparc/mm/init_64.c +++ b/arch/sparc/mm/init_64.c @@ -2899,7 +2899,7 @@ pgtable_t pte_alloc_one(struct mm_struct *mm) if (!ptdesc) return NULL; - if (!pagetable_pte_ctor(ptdesc)) { + if (!pagetable_pte_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/arch/sparc/mm/srmmu.c b/arch/sparc/mm/srmmu.c index dd32711022f5..f8fb4911d360 100644 --- a/arch/sparc/mm/srmmu.c +++ b/arch/sparc/mm/srmmu.c @@ -350,7 +350,7 @@ pgtable_t pte_alloc_one(struct mm_struct *mm) page = pfn_to_page(__nocache_pa((unsigned long)ptep) >> PAGE_SHIFT); spin_lock(&mm->page_table_lock); if (page_ref_inc_return(page) == 2 && - !pagetable_pte_ctor(page_ptdesc(page))) { + !pagetable_pte_ctor(mm, page_ptdesc(page))) { page_ref_dec(page); ptep = NULL; } diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index 1fef5ad32d5a..2d20f021c50b 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -249,7 +249,7 @@ static int preallocate_pmds(struct mm_struct *mm, pmd_t *pmds[], int count) if (!ptdesc) failed = true; - if (ptdesc && !pagetable_pmd_ctor(ptdesc)) { + if (ptdesc && !pagetable_pmd_ctor(mm, ptdesc)) { pagetable_free(ptdesc); ptdesc = NULL; failed = true; diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h index 892ece4558a2..e164ca66f0f6 100644 --- a/include/asm-generic/pgalloc.h +++ b/include/asm-generic/pgalloc.h @@ -70,7 +70,7 @@ static inline pgtable_t __pte_alloc_one_noprof(struct mm_struct *mm, gfp_t gfp) ptdesc = pagetable_alloc_noprof(gfp, 0); if (!ptdesc) return NULL; - if (!pagetable_pte_ctor(ptdesc)) { + if (!pagetable_pte_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } @@ -137,7 +137,7 @@ static inline pmd_t *pmd_alloc_one_noprof(struct mm_struct *mm, unsigned long ad ptdesc = pagetable_alloc_noprof(gfp, 0); if (!ptdesc) return NULL; - if (!pagetable_pmd_ctor(ptdesc)) { + if (!pagetable_pmd_ctor(mm, ptdesc)) { pagetable_free(ptdesc); return NULL; } diff --git a/include/linux/mm.h b/include/linux/mm.h index 8483e09aeb2c..d92c16f6cfa2 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3015,7 +3015,8 @@ static inline void pagetable_dtor_free(struct ptdesc *ptdesc) pagetable_free(ptdesc); } -static inline bool pagetable_pte_ctor(struct ptdesc *ptdesc) +static inline bool pagetable_pte_ctor(struct mm_struct *mm, + struct ptdesc *ptdesc) { if (!ptlock_init(ptdesc)) return false; @@ -3121,7 +3122,8 @@ static inline spinlock_t *pmd_lock(struct mm_struct *mm, pmd_t *pmd) return ptl; } -static inline bool pagetable_pmd_ctor(struct ptdesc *ptdesc) +static inline bool pagetable_pmd_ctor(struct mm_struct *mm, + struct ptdesc *ptdesc) { if (!pmd_ptlock_init(ptdesc)) return false; From patchwork Mon Mar 17 14:16:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14019346 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4EBA4C28B30 for ; Mon, 17 Mar 2025 14:21:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F1E3B280004; Mon, 17 Mar 2025 10:21:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EA6D1280001; Mon, 17 Mar 2025 10:21:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2112280004; Mon, 17 Mar 2025 10:21:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B2FE1280001 for ; Mon, 17 Mar 2025 10:21:43 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 828AE55C61 for ; Mon, 17 Mar 2025 14:21:44 +0000 (UTC) X-FDA: 83231256528.30.D20B075 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP id 3094F40021 for ; Mon, 17 Mar 2025 14:21:40 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742221301; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FkrrTiKic5g/SkGADpTJcYzWJvDsKPZ0Ua0I8vQ0aUw=; b=MzlcFMEwq1fyhE0yQk3klyEutCBAOdo1MJwWCP7jupp7IFZ1deLTEjCXKkbX1WhS2ms+k6 6wmblhqfZiDLOnCGKaJD/vBIE/XgfIjJ7gj2bOGvh1B+/bpMJEHoNE1tfqa+NtR1dBc7VI khVeU5ZpYXrAgLZNoLqF/gE/FODbWkE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742221301; a=rsa-sha256; cv=none; b=CglAHESkp1G/ooN5yDicBAned/LWmF0ReK2yXJE2rBGzMAx+HBzNIoodIGpFDi1Ijnp1pi mRG1itMkNYrBkE8bDE+ve9EGJA8EnOSMg4ZVzEOUKtFzS8gT5CH78xFwIDFILGdIHXyI6S ejhed4pAVUUKApwGMPtDgUMCk07/G/4= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CF25D22FA; Mon, 17 Mar 2025 07:21:48 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A78843F63F; Mon, 17 Mar 2025 07:21:35 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org Subject: [PATCH 02/11] mm: Call ctor/dtor for kernel PTEs Date: Mon, 17 Mar 2025 14:16:51 +0000 Message-ID: <20250317141700.3701581-3-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250317141700.3701581-1-kevin.brodsky@arm.com> References: <20250317141700.3701581-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3094F40021 X-Stat-Signature: tnijosxkt4fzow1a9kiwokfieus9nuex X-HE-Tag: 1742221300-116014 X-HE-Meta: U2FsdGVkX18mJXJ5qHIYQqhtpizedAix/dUCwDjxnbZp6gH3icRfN77Cc0vjIU4Rr32xZ9a9q7ev7u7z0tll8NKfee0FkAXhsR9DEaBryWod1Kot4bYPSrhatLw+uIfsJn1bi5TMYiDPZ7AnWxQCBqMp1u+0ISmaZIPIiDMW7am1FzwxG6YU0eRIP27icrlIpztJABZ/Yq4DsM5UtNA9rvINEgcFi8Z5+6r0rGKTZwGUYg+vUChkSF3hIViMfpCks+7AD77x12875fo8Sjs26hUcHC1Ii3Qsn1DNKQNonetSiBP8FD080vk3o3WQ+m82fNfGJApLUCTsPYc81zVnne0UiKFizX2NtSEZCT6kcAOs7YBozkhmbM/RisZKf2I0AbWAsUPRk/h7yk8btzwTHEja2ZndjcQS23oeh4Bda6LFyFb+LBkFebGC3lJyPSBlggo4YyHqHiKFw+GHuQFfzPWoDFB8eXvoAakFQ/yHstzBqE3tPh95zpaiVU9nXJpY6X0Y1p8DMM9mFhdqWprRxK2N4AsXuMehvIoO6t9yx5by3NCMKFy35eiHAd7Gf0LOdtQsGdBMtdpUNA/7NxGysDJn9DrEILn8f6izDovOp/Q2QQZR9+kihPfet9lhZe9w+n2piVvwGgnJG1ZDXU63zwcbOq0Ciz1yN2y86ms/HCKc6CP0u4xzd4nrwgctclNONnO4U6ELWdUn0SJsZNf7YM1eJEm4XYa2y1jv+CF6opI3dc1bp6t/c4hmrnQOYNQhhKPUks0Q7O48OKY637ZhU5a/UvvnX22znNCZONV/HALKTCieTfMW+YvtdJLRI/Mhlz0TZwPmPTeVpXqdJGwP+SAqXtfoF/WNp8JiEUuFhMZmmc4o21oqwG98u66Xv+XIdiUgq9OJZrSOR5E/yxbeINivKc2AQ2Xtkok1pIuTa0mcFZbfT9fUIpfpx2F68oAbB9kUmeDi8VxLGhpyssw 6Z9TnVmz s3v02ZtX0qWvDLuYKBuBxY5SDsFC6ob357abXW0YkYuu+z2+qmudCQPZEcmsHYAXRu9iluL4NQ2rtKmOaYYCbBhaaghGvlTG/zLxxMC8zwa9QuYi/NcIPIw658iRZvAcDy2JIXIpwcmuS5J3qBPdMFE/7RajBbB6GeosqtFh8VIFjhOR3flGA6lxvaw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since [1], constructors/destructors are expected to be called for all page table pages, at all levels and for both user and kernel pgtables. There is however one glaring exception: kernel PTEs are managed via separate helpers (pte_alloc_kernel/pte_free_kernel), which do not call the [cd]tor, at least not in the generic implementation. The most obvious reason for this anomaly is that init_mm is special-cased not to use split page table locks. As a result calling ptlock_init() for PTEs associated with init_mm would be wasteful, potentially resulting in dynamic memory allocation. However, pgtable [cd]tors perform other actions - currently related to accounting/statistics, and potentially more functionally significant in the future. Now that pagetable_pte_ctor() is passed the associated mm, we can make it skip the call to ptlock_init() for init_mm; this allows us to call the ctor from pte_alloc_one_kernel() too. This is matched by a call to the pgtable destructor in pte_free_kernel(); no special-casing is needed on that path, as ptlock_free() is already called unconditionally. (ptlock_free() is a no-op unless a ptlock was allocated for the given PTP.) This patch ensures that all architectures that rely on call the [cd]tor for kernel PTEs. pte_free_kernel() cannot be overridden so changing the generic implementation is sufficient. pte_alloc_one_kernel() can be overridden using __HAVE_ARCH_PTE_ALLOC_ONE_KERNEL, and a few architectures implement it by calling the page allocator directly. We amend those so that they call the generic __pte_alloc_one_kernel() instead, if possible, ensuring that the ctor is called. A few architectures do not use ; those will be taken care of separately. [1] https://lore.kernel.org/linux-mm/20250103184415.2744423-1-kevin.brodsky@arm.com/ Signed-off-by: Kevin Brodsky --- arch/csky/include/asm/pgalloc.h | 2 +- arch/microblaze/mm/pgtable.c | 2 +- arch/openrisc/mm/ioremap.c | 2 +- include/asm-generic/pgalloc.h | 7 ++++++- include/linux/mm.h | 2 +- 5 files changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/csky/include/asm/pgalloc.h b/arch/csky/include/asm/pgalloc.h index bf8400c28b5a..288dca0d160a 100644 --- a/arch/csky/include/asm/pgalloc.h +++ b/arch/csky/include/asm/pgalloc.h @@ -29,7 +29,7 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) pte_t *pte; unsigned long i; - pte = (pte_t *) __get_free_page(GFP_KERNEL); + pte = __pte_alloc_one_kernel(mm); if (!pte) return NULL; diff --git a/arch/microblaze/mm/pgtable.c b/arch/microblaze/mm/pgtable.c index 9f73265aad4e..e96dd1b7aba4 100644 --- a/arch/microblaze/mm/pgtable.c +++ b/arch/microblaze/mm/pgtable.c @@ -245,7 +245,7 @@ unsigned long iopa(unsigned long addr) __ref pte_t *pte_alloc_one_kernel(struct mm_struct *mm) { if (mem_init_done) - return (pte_t *)__get_free_page(GFP_KERNEL | __GFP_ZERO); + return __pte_alloc_one_kernel(mm); else return memblock_alloc_try_nid(PAGE_SIZE, PAGE_SIZE, MEMBLOCK_LOW_LIMIT, diff --git a/arch/openrisc/mm/ioremap.c b/arch/openrisc/mm/ioremap.c index 8e63e86251ca..3b352f97fecb 100644 --- a/arch/openrisc/mm/ioremap.c +++ b/arch/openrisc/mm/ioremap.c @@ -36,7 +36,7 @@ pte_t __ref *pte_alloc_one_kernel(struct mm_struct *mm) pte_t *pte; if (likely(mem_init_done)) { - pte = (pte_t *)get_zeroed_page(GFP_KERNEL); + pte = __pte_alloc_one_kernel(mm); } else { pte = memblock_alloc_or_panic(PAGE_SIZE, PAGE_SIZE); } diff --git a/include/asm-generic/pgalloc.h b/include/asm-generic/pgalloc.h index e164ca66f0f6..3c8ec3bfea44 100644 --- a/include/asm-generic/pgalloc.h +++ b/include/asm-generic/pgalloc.h @@ -23,6 +23,11 @@ static inline pte_t *__pte_alloc_one_kernel_noprof(struct mm_struct *mm) if (!ptdesc) return NULL; + if (!pagetable_pte_ctor(mm, ptdesc)) { + pagetable_free(ptdesc); + return NULL; + } + return ptdesc_address(ptdesc); } #define __pte_alloc_one_kernel(...) alloc_hooks(__pte_alloc_one_kernel_noprof(__VA_ARGS__)) @@ -48,7 +53,7 @@ static inline pte_t *pte_alloc_one_kernel_noprof(struct mm_struct *mm) */ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) { - pagetable_free(virt_to_ptdesc(pte)); + pagetable_dtor_free(virt_to_ptdesc(pte)); } /** diff --git a/include/linux/mm.h b/include/linux/mm.h index d92c16f6cfa2..ee31ffd7ead2 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3018,7 +3018,7 @@ static inline void pagetable_dtor_free(struct ptdesc *ptdesc) static inline bool pagetable_pte_ctor(struct mm_struct *mm, struct ptdesc *ptdesc) { - if (!ptlock_init(ptdesc)) + if (mm != &init_mm && !ptlock_init(ptdesc)) return false; __pagetable_ctor(ptdesc); return true; From patchwork Mon Mar 17 14:16:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14019347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B2F5C282EC for ; Mon, 17 Mar 2025 14:21:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B1B6F280005; Mon, 17 Mar 2025 10:21:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AA4AE280001; Mon, 17 Mar 2025 10:21:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D1A6280005; Mon, 17 Mar 2025 10:21:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 71CA2280001 for ; Mon, 17 Mar 2025 10:21:45 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id DF43E1A1F83 for ; Mon, 17 Mar 2025 14:21:46 +0000 (UTC) X-FDA: 83231256612.04.8C17F75 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf04.hostedemail.com (Postfix) with ESMTP id 43DA640003 for ; Mon, 17 Mar 2025 14:21:45 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf04.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742221305; a=rsa-sha256; cv=none; b=q4pHDZrGgoH2fRIT0DFLm1L1COh5lLBxF9uqM9ebDshKCfj0t2UkeUQlJ7mLS3Pb3plfAO 7tZFjzPhaK77RSk5jpNewNlR2YldJRIabtX1kfvQyMeWshTXEFflDlT02n3DpGzahOlaua Ujq7IBm+CvrcdGHBkFi/LlzYhAdPE+Q= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf04.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742221305; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tEBbZMuK7OXxaJSao4EKWfZ44UXf7uN/ZmWU7nnmgv8=; b=xpU3svoHSsU5DcVi963qmUn3kgGJDdspA+4/9014pqw3kG+iRuzK0a3tVeId1xE3Jbiqsn HH9B3fbtIho9TsO0MDYasov40iXoFJkdqakQEmc18vbtWdnya84YUuDjM7NUCTgFZUvx7A +ooHRHPWZisIa/1XMqVp565eQBmyw7o= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 67A932573; Mon, 17 Mar 2025 07:21:53 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3FEA43F63F; Mon, 17 Mar 2025 07:21:40 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org Subject: [PATCH 03/11] m68k: mm: Call ctor/dtor for kernel PTEs Date: Mon, 17 Mar 2025 14:16:52 +0000 Message-ID: <20250317141700.3701581-4-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250317141700.3701581-1-kevin.brodsky@arm.com> References: <20250317141700.3701581-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 43DA640003 X-Stat-Signature: gkzyadfqhyou4g3oj51cfxm73czkgh9g X-Rspamd-Server: rspam06 X-HE-Tag: 1742221305-413094 X-HE-Meta: U2FsdGVkX1/mD9cSpAi9lP+6DvO/1GyhcYN9p1QMgR+guJWgfZ3pAIJMfssdj1nxFDStRIAQQ3Lp/7DjLvYEpBCSEMR5XqFUQvJ1Ws/jWimIyw720NBLw6uKJrhWVJd5hrS2AgG4P1Trb7ONIw/C7z2F8sX55JBuExTNbtX1g8JJLJhD1CyQsOXufxL+s1CPq+4reL2xdukaN0o+JDLsdCBv6/8qpOu9fIKRQr9Cik6M7Ws/1kRP+HCUoE2lU3Wyz9emFKZpdCnq+Yv5+/qW/GgmQAIQ8WuX/c2W1V2lAJ85suz+qLOvOaJNiWVMQ99Cu68gxq01MLnvHudnhKBTuTOov89lxUZ9RjZSQAU/5RD2xgFDh1csiqyTtpsGCXmRIdjb83PV630bMgYUIC5L57cThhwjHKFwdXVOpJldslLuC9F2Rek+TteMTsDZNs+b60u2q0jhJAHFKfE3c4NSjNpG5gV5VWleYDubSyDeLHlZqcyH4cHtydx+L3gms4jVrIWXakKVXndPlxictQ8hRIXrL87OwHvFCES1rKHwfGYOSv2qF3bHj70D4XJFtaj7GBMukGeaRJGz2iItNLnfFkoCwwI5rC0JqOSGAZ1WkGhitV0ffDoiKBNweXfKZl0RzVBc81cFeKbocYNe0yFASWfYGEC+f3fmKCF4o/QMK0hhtNZcg1vqi2gqCecu3XtodVGlwuiSegCd9myK1e0owSOEStmfqPUS4kK4RT/xiUbThMskstrc2Di+JflvTXqgP7XAsaBaoN4SjmliEtA6LU74nnf+vJ4P1A+Li2d7J7lFggq8pcGXjcl5yZeyTjGg4unW+Vy7YR/CqfQfjSwcOscwCZYGZW9UBMwE3B1xgxrKyQLxFEj6W+b122Ye2wnOZnDi75wXc58FrPbo8X4+yS+iF5xsk0INIt57lvrCqzXX2lbqVjXh6bBiNXeS1LuBnrmYXBl/plZZH/D9Svv f3bGG7vk c2hCs6bbjP3ZvhD5KwoUgvjTxz6uroVqKVm/cr3mDa28XQ7BA4kmC/BrFLHZ6gFgMYg1LZuammRfwEfmS/y3uLsoVdsu0iSDUaygJ2JrEEzLgxoXhuSujWslMrXfwXYqsOvEHvd+wCSey9HvtUxJ2lXAwYQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The generic implementation of pte_{alloc_one,free}_kernel now calls the [cd]tor. Align the m68k/ColdFire implementation of those functions by calling the [cd]tor explicitly. Signed-off-by: Kevin Brodsky --- arch/m68k/include/asm/mcf_pgalloc.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/m68k/include/asm/mcf_pgalloc.h b/arch/m68k/include/asm/mcf_pgalloc.h index 465a71101b7d..fc5454d37da3 100644 --- a/arch/m68k/include/asm/mcf_pgalloc.h +++ b/arch/m68k/include/asm/mcf_pgalloc.h @@ -7,7 +7,7 @@ static inline void pte_free_kernel(struct mm_struct *mm, pte_t *pte) { - pagetable_free(virt_to_ptdesc(pte)); + pagetable_dtor_free(virt_to_ptdesc(pte)); } extern const char bad_pmd_string[]; @@ -19,6 +19,10 @@ static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) if (!ptdesc) return NULL; + if (!pagetable_pte_ctor(mm, ptdesc)) { + pagetable_free(ptdesc); + return NULL; + } return ptdesc_address(ptdesc); } From patchwork Mon Mar 17 14:16:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14019348 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73D2AC35FF3 for ; Mon, 17 Mar 2025 14:21:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2428F280006; Mon, 17 Mar 2025 10:21:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1CDCF280001; Mon, 17 Mar 2025 10:21:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 046B5280006; Mon, 17 Mar 2025 10:21:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DBEFA280001 for ; Mon, 17 Mar 2025 10:21:50 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 388B2141F97 for ; Mon, 17 Mar 2025 14:21:52 +0000 (UTC) X-FDA: 83231256864.22.E0626C1 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf04.hostedemail.com (Postfix) with ESMTP id DC2804001D for ; Mon, 17 Mar 2025 14:21:49 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742221310; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/BkpJ0cTKq4ChrXalGvhAQpS6e0Bp2XA+877e60Q9Ds=; b=casQlAUkKaJ8rvRkClIekiymZxYR7bkKde4J3m6VsWlW1VApIDeq/aG51vWZgcbsaHpw4t qmbJHmT8o9/bGevF6e0KN3SrALjut9s7h3sd0SWEQefWpNd2ihpJ9NgTbPz4cHh3FkYA8Z XWsHU10JxzV2ligmF0v+90T9qAOB274= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742221310; a=rsa-sha256; cv=none; b=rKOZmD/C9UBTOCstJNnCkf0yxgV0FsaFoCcquQgOjEfj8CYiLsuQ9HcJe+8UZG4tWJaxu8 ilFeL390RbfaY6y8qziteLkFrzYgUjrCjo/+emxIwehQEL9pGIPEx4fwmYCm4D0NPaULco ZMY7L/UdI7NfRkDGTUYaXBqdq0NtESk= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 005532576; Mon, 17 Mar 2025 07:21:58 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CC9AA3F63F; Mon, 17 Mar 2025 07:21:44 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org Subject: [PATCH 04/11] powerpc: mm: Call ctor/dtor for kernel PTEs Date: Mon, 17 Mar 2025 14:16:53 +0000 Message-ID: <20250317141700.3701581-5-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250317141700.3701581-1-kevin.brodsky@arm.com> References: <20250317141700.3701581-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: DC2804001D X-Stat-Signature: xeob7pt67zzj8yrsap4ubjnu9hjxebq7 X-HE-Tag: 1742221309-105431 X-HE-Meta: U2FsdGVkX19vK02TTiRiqeHJ2lE+6Hz/cqAPvZteAPzNIx5Kd+ywGVJyste6xnbbr+FmzBhcSjeGP5Cz00Vp//oWjbmsSNjotoSwK7nqAP31cIEUFbyo+rJvrmaFWWnsGLVcfqUMx4x58baRfRT/2qn31jypRueQ6oiGv3A7iqg9zyWuKaGnhrzAga8RSxhdkTLSh9BZSo8uqar3iRFWSvmrKSwl5vUsLyLO1JWmO45y67QTENLqQMpFTe5ofyCPDiONcKtkcSuY5Of/J6gBAMMPu2aJQ1MC5wCjdggMPARk+z4ZOpAeidmxxTJkE9WgYvab9ZhXB4D8s/jb/eprOSmc6aQca6GPxsPfqOtQhosPTC0jHkqMgvYOd0CFfWUbK+plyY16muw0ky+AYQDgf1RmzlJkjATuuRXa5ZIj8rUZI2VEqRMIKwIFass+BoI2qpD4DCec+6iAemltBMoaiNbpwMKNoXDwtVVfqCO8U8AHA2eXdvqcSGaakgu8xwgjL9Nk9FZBk3AchLFNwmcKQeyI16NYNJ5h/3THI3xa5V7wFG0Ml6NGbFpLARBVPqaZYVmP9nTvJNT3++8WVVOOzBMFuTt/pcpTM9aBB8dMmQZuNL/M2O3pWtpN4QdDLfSl1Gso/UF0zuD8iS7YpfNzqN158t5XI4irGBEqdQaTzKlEJ2YddehN4IY9ww/+WHsdjK2nrUD2pFjUYPvcNmzsm3wXMhheXfLIi7F9OemKBjJaOiR+1V374jFdkvqeUEv8KoD6PfvP97qaM2/sNlVDIR3SEpkKIKpiR4DvaZMqst4MypQXir6S67LOq20kEdajUYFJwJQQ8lBvXryJob9dB+QarTGxboHiu3U1PH9YQfjV8TNoxL52PyC0xORzyq2gKJ2U6AwIfYfmWl1z74NCwxFErTmN60r9bnTFhlFObpupiyTryvtVWY4sbkanu40qXAPrSvulECuY3Kyy3HQ ONJXMfzo med5l9cfAHvrjnKUc5VxxPIQ9KoV2j3m1UkrYIpyzZpYUtUB1yy86nt95PqXimWPKKjzyh24edxRC46KSJtwBy+4GeLIeyZsP+p3jr6I2dMVA6L6jKEnsB6L2YJXXoySMTvU07qZSOjZYjJP2XGvasE1J3g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The generic implementation of pte_{alloc_one,free}_kernel now calls the [cd]tor, without initialising the ptlock needlessly as pagetable_pte_ctor() skips it for init_mm. On powerpc, all functions related to PTE allocation are implemented by common helpers, which are passed a boolean to differentiate user from kernel pgtables. This patch aligns the powerpc implementation with the generic one by calling pagetable_pte_[cd]tor() unconditionally in those helpers. Signed-off-by: Kevin Brodsky --- arch/powerpc/mm/pgtable-frag.c | 30 +++++++++++++----------------- 1 file changed, 13 insertions(+), 17 deletions(-) diff --git a/arch/powerpc/mm/pgtable-frag.c b/arch/powerpc/mm/pgtable-frag.c index 387e9b1fe12c..77e55eac16e4 100644 --- a/arch/powerpc/mm/pgtable-frag.c +++ b/arch/powerpc/mm/pgtable-frag.c @@ -56,19 +56,17 @@ static pte_t *__alloc_for_ptecache(struct mm_struct *mm, int kernel) { void *ret = NULL; struct ptdesc *ptdesc; + gfp_t gfp = PGALLOC_GFP; - if (!kernel) { - ptdesc = pagetable_alloc(PGALLOC_GFP | __GFP_ACCOUNT, 0); - if (!ptdesc) - return NULL; - if (!pagetable_pte_ctor(mm, ptdesc)) { - pagetable_free(ptdesc); - return NULL; - } - } else { - ptdesc = pagetable_alloc(PGALLOC_GFP, 0); - if (!ptdesc) - return NULL; + if (!kernel) + gfp |= __GFP_ACCOUNT; + + ptdesc = pagetable_alloc(gfp, 0); + if (!ptdesc) + return NULL; + if (!pagetable_pte_ctor(mm, ptdesc)) { + pagetable_free(ptdesc); + return NULL; } atomic_set(&ptdesc->pt_frag_refcount, 1); @@ -124,12 +122,10 @@ void pte_fragment_free(unsigned long *table, int kernel) BUG_ON(atomic_read(&ptdesc->pt_frag_refcount) <= 0); if (atomic_dec_and_test(&ptdesc->pt_frag_refcount)) { - if (kernel) - pagetable_free(ptdesc); - else if (folio_test_clear_active(ptdesc_folio(ptdesc))) - call_rcu(&ptdesc->pt_rcu_head, pte_free_now); - else + if (kernel || !folio_test_clear_active(ptdesc_folio(ptdesc))) pte_free_now(&ptdesc->pt_rcu_head); + else + call_rcu(&ptdesc->pt_rcu_head, pte_free_now); } } From patchwork Mon Mar 17 14:16:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14019349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9FA3C35FF3 for ; Mon, 17 Mar 2025 14:21:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6BABE280007; Mon, 17 Mar 2025 10:21:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 61C4C280001; Mon, 17 Mar 2025 10:21:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 497ED280007; Mon, 17 Mar 2025 10:21:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 2DA6E280001 for ; Mon, 17 Mar 2025 10:21:55 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B7212C1F38 for ; Mon, 17 Mar 2025 14:21:56 +0000 (UTC) X-FDA: 83231257032.08.E1D24C2 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf23.hostedemail.com (Postfix) with ESMTP id C2108140016 for ; Mon, 17 Mar 2025 14:21:54 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf23.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742221315; a=rsa-sha256; cv=none; b=3qpDGJCAWzEVq9l74hzCA246D00heL669Qku0Z7I54m8yn6WOUzGjdlnb702IhGA7QxlkR KGEkyaHUPOib02dduMY2Jd3y07NFjf43C6TU5Nc7luitnr3z7Q3FNrn5xSV6CoZpU6Dffc +ak5jnwYs2A8zKaMUmctlpLgDB6BPB8= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf23.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742221314; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=o+UbYltWTL9nsXjK30sfubrpExxRG1UfppgmpoTi2qk=; b=KGgFZYQD6+Nn5wjhCJ8dG4kPoz/b0MviaJHybjjxCcpaMQLw+LydrUuRA4Sc4SIn8b3/ds 6IVJVeYAuGWS6ZQD3Bypzt0GZ9MV+TR8znTXeCipini+WtnAC9ymr7OWKXzONu2VXFc4lF dlX3GD44ZT+E9XaarQ3pVv5HjZxY8ss= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8C87A25DC; Mon, 17 Mar 2025 07:22:02 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 64BEE3F63F; Mon, 17 Mar 2025 07:21:49 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org Subject: [PATCH 05/11] sparc64: mm: Call ctor/dtor for kernel PTEs Date: Mon, 17 Mar 2025 14:16:54 +0000 Message-ID: <20250317141700.3701581-6-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250317141700.3701581-1-kevin.brodsky@arm.com> References: <20250317141700.3701581-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: C2108140016 X-Stat-Signature: z59eusr94hp4sykooc7pc4qf9sw5hwgm X-Rspamd-Server: rspam06 X-HE-Tag: 1742221314-591007 X-HE-Meta: U2FsdGVkX1+ZJuw4UKY+pt/T5WAk9hwTfG6AfmAsNC97r7LAswwaoPDHmiNnJ6sfpe1RxOhFNydULDWjuW1QA8Vw01ODGgCUH5ndMY0kWT6fWhk+ZnWuAHIxDWUxu8ZBhFCGtUadGXRXnYyMa12NN9n82Gu/nH03BDsU5bo3RU2nZFkIkDh0cEs23c/HO/SV25kzgVviqs1UjiPZHsTknm5HZuUFrGZgL+4ukdou5JnhF4+3RYysS7wpncTF585V+2hYrSu2Fmwqz16nfXcDX0Rdw3IRwx0EaHbw33V42HPDmEv6oRgpFTWJohMBm48bfL3s37b3z8xugzT+8VzbxVVhpP2GyivY42v4nzlMK7KXL/E1jskdhXAQH4R7c8RSXnY3CuGQXq6I1fm113Sd2x+NKLxO83DylO6O8MeTtzglwLMG7JLASeNCy5RjAHGzdEV1hVRu04bKSQz9VCfRcK03M3xjcnTL0pM1Ut4eSZ4Ap4TLzR01wCEuLJmxAhHZm4puJ9VhPrJOMsVxST0yqWxP7pe1xr3R8DL6+Un+vEZhEEmMbtgdQ3nCa6r6UmoIvVjMz+MVHt5F1PJLCTm/+AvLjxEiiMJzA/xnJimUk49Lkbcyixe6Oa6i4DOUAxJ67u05VsCTAK9Pc5MBzTrfdPded/U3l4g9WZa/eO5skEfJKfOe57s2BI0h23DzVze5/HiHToqJ+0FUDtC9DvzXTtOxfd2Budvb4P8lSjXIditytBQ48eDczVJFoaaifFRzQg6hWXFxZJX47zO4RdbDrY11yzYrf5sF0djPaxTtPQW4NhU2hk7p3oeVYn6eitJgKy826lpN8TFuTJrrym0lvQCOgeQi76aWSviepwFHbHrBAv8pAAf5fj6fBDQwYul1kOnY6wevYMqlEdUT3DBg7h+BMszPV/+irkAxALhlynp7lBmREfDb1kCuPGfsBe3eHfuzQHyqE85L34Q6gIw qzr+yhUO TzQwuOXEV9MlJnGBost8ywX1F34hccoM5dVp31bNW432opIgM4oVoN7fD42snTzdlPtuCRd+U2NJgsPB6ayp94fuUw7S+8c/kee8gnngVr3DzGS1pYP+HByPKDfes1RMoYuyuKOjSwJrKQe5kELNRGdPfUw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The generic implementation of pte_{alloc_one,free}_kernel now calls the [cd]tor, without initialising the ptlock needlessly as pagetable_pte_ctor() skips it for init_mm. Align sparc64 with the generic implementation by ensuring pagetable_pte_[cd]tor() are called for kernel PTEs. As a result the kernel and user alloc/free functions have the same implementation, and since pgtable_t is defined as pte_t *, we can have both call a common helper. Signed-off-by: Kevin Brodsky --- arch/sparc/mm/init_64.c | 27 +++++++++++++-------------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/arch/sparc/mm/init_64.c b/arch/sparc/mm/init_64.c index cd60a0a8ca0e..0a1b2b1adad4 100644 --- a/arch/sparc/mm/init_64.c +++ b/arch/sparc/mm/init_64.c @@ -2882,18 +2882,7 @@ void __flush_tlb_all(void) : : "r" (pstate)); } -pte_t *pte_alloc_one_kernel(struct mm_struct *mm) -{ - struct page *page = alloc_page(GFP_KERNEL | __GFP_ZERO); - pte_t *pte = NULL; - - if (page) - pte = (pte_t *) page_address(page); - - return pte; -} - -pgtable_t pte_alloc_one(struct mm_struct *mm) +static pte_t *__pte_alloc_one(struct mm_struct *mm) { struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL | __GFP_ZERO, 0); @@ -2906,9 +2895,14 @@ pgtable_t pte_alloc_one(struct mm_struct *mm) return ptdesc_address(ptdesc); } -void pte_free_kernel(struct mm_struct *mm, pte_t *pte) +pte_t *pte_alloc_one_kernel(struct mm_struct *mm) { - free_page((unsigned long)pte); + return __pte_alloc_one(mm); +} + +pgtable_t pte_alloc_one(struct mm_struct *mm) +{ + return __pte_alloc_one(mm); } static void __pte_free(pgtable_t pte) @@ -2919,6 +2913,11 @@ static void __pte_free(pgtable_t pte) pagetable_free(ptdesc); } +void pte_free_kernel(struct mm_struct *mm, pte_t *pte) +{ + __pte_free(pte); +} + void pte_free(struct mm_struct *mm, pgtable_t pte) { __pte_free(pte); From patchwork Mon Mar 17 14:16:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14019350 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64653C28B30 for ; Mon, 17 Mar 2025 14:22:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2066F280008; Mon, 17 Mar 2025 10:22:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 190BC280001; Mon, 17 Mar 2025 10:22:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 00929280008; Mon, 17 Mar 2025 10:21:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id CDC4C280001 for ; Mon, 17 Mar 2025 10:21:59 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 61131AB77E for ; Mon, 17 Mar 2025 14:22:01 +0000 (UTC) X-FDA: 83231257242.27.0B89A68 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf28.hostedemail.com (Postfix) with ESMTP id 69629C0006 for ; Mon, 17 Mar 2025 14:21:59 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf28.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742221319; a=rsa-sha256; cv=none; b=ojTqtix6L1tQOSryvTr/K/JR/jZ0brP6WVVRDIiSna5VuSD9UGaB4ye/6P/UwUzWMDHkP2 vUEW+QflxFLXedC63Nf1C6EOlDmI2lUs7GBquUWzfWLUIO6l/RNSwZFoDHK0EJ/4PtI4+g yJBlGCN5lvjDtOIE9FQklo3iClrugoM= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf28.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742221319; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Fo5GoDaPm0gIjYVqCGzA2mr+USoIG153jiLEzNpCVDE=; b=63WRq6Hd7erlNCFqTqYK14c3/uI1f+ajMFxu7NoNA4lyuqNzmGCFPtoQsExY4FjEQZKh7t LN89dt8P7ShiZHmIBF4Wujl8MJ0FbMiEVSfsTu7PGLTXExYXVOp/9MBBazzCyeNrb84eq4 O7ER8UdhIS5AwGECA6TYcg3Gb7NpNQU= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 24CC326BE; Mon, 17 Mar 2025 07:22:07 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F102B3F63F; Mon, 17 Mar 2025 07:21:53 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org Subject: [PATCH 06/11] mm: Skip ptlock_init() for kernel PMDs Date: Mon, 17 Mar 2025 14:16:55 +0000 Message-ID: <20250317141700.3701581-7-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250317141700.3701581-1-kevin.brodsky@arm.com> References: <20250317141700.3701581-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspam-User: X-Stat-Signature: sqw1utdcs8u4yqm7q6storg4z4n56qax X-Rspamd-Queue-Id: 69629C0006 X-HE-Tag: 1742221319-879489 X-HE-Meta: U2FsdGVkX18KCOw7BAM/808qJ1EaPR6rj1iumM6t3u2XJh9JrBy5tKzww1e/XCzmpVIFCjXLopIzpEH0GnaLbyOsCVheeKvL75+ZTsrDj1T5y7hbiDoZ27LmTepZg5WhkB5/FLmTTXY9/IZ2S92V7kAwEM0+4RrR1yxPdYcUidHBaSddcPk6oQQRGjhccdnxIv3RNAh94zH/f7VSPsqY3BDpLz7axGDsRfne9PgJwb424LOa2XAy9Mih75bNPILr28U5ItrMnI9B6B2mYzZxFhPkk9vyGZ6WyNZkLaeThaHCZt9x63Vnz+umlZWT0AUSHQm1vhNhnUwq3hGzpFUoOvTPysi8tmYTGEHfGJi0+QETcc78kyTH2nmtmFWKBZ0lhJDfYpVK9oDeq6GDdcIM5Z5mYNnrZPv/diz1yQNacF2XjlKFbA4BWYQaSFtAK7iW7pZ8WKqPisdnMB2Ioungs4HK4A1oFb5TDAAGPTtwjDx3V5ew5nFo62S0AgQ2KwF9JquT6WZkPPI04ePA+PjeuWJP8as28QvUo7rG3OHLC3OxavYm/enC5o0V8ScrY5zS2V25CtpABlJkbHSAaFNp558ZKlWVoFmVQVL7fD+Yxrb042aT65RsIQz6no072fXeDxxTEynq5YXzwA0qDHRJhpwYEbBcxds+JyOldy5ZuAIK21GtOp1VEYGXDPLg9iBwAtO/Nh1UWmb8r+J0E4KJ40gughlkvOwCBJBhnZ07Oy6Bo9uyShKujTVzqSyiXshD/Ex6XIUDD0AlSxEJbTInC9IoQT+DWD/rrFb306gGgGeBEPm2iyAOWoQFAKkIke/7uRLeRPqXbL8/Hzz4my2cN/qF7s7TDUgFh6qVEORwR2En+wtXLSe8Qt5nZtdH5Z9Vxpg0EwQkCw1jHwGLm5A6ZJLdqn1CJ2ol13SbpslwUwf/kGmR6cs7bP0hqlLjvZXd/0e573e7oN0MhJlFYFC +qm4fZ7J S7Y7e7OTNYJlwkQVMxZ1u/eV0xgqQ3XS25eZ9foCHNfkTU6+1vE8vwZmPUaVr+2R1VbTDG05fil4YEF6JSdNOioOwQXh5i4zbdHEoheGAu+1MAwAfM09ig6DR3VbjlCTIxILJv7FN4jvzAX62s14IymAI5umej15fYYvLbqZDbeVpVzqTXqUC3X8rOKZL8HLnNwOMF+sgmkkM70o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Split page table locks are not used for pgtables associated to init_mm, at any level. pte_alloc_kernel() does not call ptlock_init() as a result. There is however no separate alloc/free functions for kernel PMDs, and pmd_ptlock_init() is called unconditionally. When ALLOC_SPLIT_PTLOCKS is true (e.g. 32-bit architectures or if CONFIG_PREEMPT_RT is selected), this results in unnecessary dynamic memory allocation every time a kernel PMD is allocated. Now that pagetable_pmd_ctor() is passed the associated mm, we can easily remove this overhead by skipping pmd_ptlock_init() if the pgtable is associated to init_mm. No special-casing is needed on the dtor path, as ptlock_free() is already called unconditionally for all levels. (ptlock_free() is a no-op unless a ptlock was allocated for the given PTP.) Signed-off-by: Kevin Brodsky --- include/linux/mm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index ee31ffd7ead2..4759da9cd633 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3125,7 +3125,7 @@ static inline spinlock_t *pmd_lock(struct mm_struct *mm, pmd_t *pmd) static inline bool pagetable_pmd_ctor(struct mm_struct *mm, struct ptdesc *ptdesc) { - if (!pmd_ptlock_init(ptdesc)) + if (mm != &init_mm && !pmd_ptlock_init(ptdesc)) return false; ptdesc_pmd_pts_init(ptdesc); __pagetable_ctor(ptdesc); From patchwork Mon Mar 17 14:16:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14019351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BE7CC282EC for ; Mon, 17 Mar 2025 14:22:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BF9B3280009; Mon, 17 Mar 2025 10:22:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B8304280001; Mon, 17 Mar 2025 10:22:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9FEC9280009; Mon, 17 Mar 2025 10:22:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 82A83280001 for ; Mon, 17 Mar 2025 10:22:04 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 01E6BAB675 for ; Mon, 17 Mar 2025 14:22:05 +0000 (UTC) X-FDA: 83231257452.01.FA0889B Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf22.hostedemail.com (Postfix) with ESMTP id F0A5AC0029 for ; Mon, 17 Mar 2025 14:22:03 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf22.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742221324; a=rsa-sha256; cv=none; b=A3QD245xacER7te+8H1Jwo/3ARDOavRyJtt1lwoA8nvX9Kz2UPaYFNV/nS9F9qEgSp8X9s SJzkaePYaj4nDoZtJpstaMOrj0KTHY6ohq04nBzh6jYNSHGGBleS33mXVtWtyKk1O3YUR1 ceeQN8jI1Nr5F4mL4P8ANbGyNprc9wU= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf22.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742221324; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e0qUgaUrYwFOh7rWnfnoc0fr3tb5PvfK77peSjWpZHE=; b=qibjs6YoI6r0jsA+zWLbw5+s1HtFnLmfygRUKSaEbLA6LCkPwZlYofA9PJ5n463/4pKu5m d86SL/hVEMbyyS/zk/2MqmelsIsYlOVhPvBTF4G0i4ubfuyXTnv2zXtaRmveEbVb3rL+Hy ZYLLV/m2tZpy2J2fjD1zF36CF5yeXCw= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B17952936; Mon, 17 Mar 2025 07:22:11 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8A11D3F63F; Mon, 17 Mar 2025 07:21:58 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org Subject: [PATCH 07/11] arm64: mm: Use enum to identify pgtable level instead of *_SHIFT Date: Mon, 17 Mar 2025 14:16:56 +0000 Message-ID: <20250317141700.3701581-8-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250317141700.3701581-1-kevin.brodsky@arm.com> References: <20250317141700.3701581-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Stat-Signature: paf6rzpqxpxi7mt5sow1da6crtebdtzh X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: F0A5AC0029 X-Rspam-User: X-HE-Tag: 1742221323-322213 X-HE-Meta: U2FsdGVkX1+V0/M47ulfPz6lnK21U/79HCPe5CFJze6k6hNevJaZ7Zt03d6vBHUwAAiHDiVLWr2vVJthFU8ZiMWQnUX4+4x20RVJ9+V1mDiym2W/xeSLiT0+3uoUNkvzq4O2fjw/IkpF6FdmAkYIuE+3OPV3+vDX8/DGVBh7cPnwAbm3MhKH4XRLq6Zzl8JNbEKdmvCOLDZqmU6MtCqC4jcVKwmM1ey5dofTOj0337WKgKHM3F5EWKKGW2wSqhKiTru0ihnAr96SkyzN3lRuuN25W/N8ikNflhbZVixEJD8uuMPv3JeZKzLp+PKuQ+ZfjNPTVm+TFbFBn8YPSTD6ZKWKT+SajNd+6SRC+0dy0J9c28NZHkI+h6bldqZsPva/cluPKgkH4DEA70ODhR+rqPWID+wDAx05EsoaW/jZYpZ0S+UTAFIxVD1K/15/NIv+ffJT8npdnbFnL/8viq3GJMiMloehPXjhZshi09dPRsQWfR/v5PXlIpg5NlpwLscm85+bymrFGpHnWEjjofMNdtV5eqZPKUw+JusYx2e4X8x0wcw9lwYL6caoPYKUSfIS7elQGfoCxo7RgXle3ASr9JNVam6ZM8fc8jaLfI3BlmNJOSIVDe+hH9L18L7cpcWN4Pe1rfCO+uDg7+wXg2af7q6GPZ4sdXuf/c2eQOLJd3PuGKO2NEa1oBfo3TrxxQzUje/9xX0zC60jG1GggZXgAVONG5rm4f7aNqeujr0mVnbLvhnBlWQLx8kmhVtqOpCsICs6ZgxZ7bvSq08ymP0hcx91IPhDsBLtYa6jqukc4oNBXEqJOuclPctcIq9UKGqUshVnot5+10sH+W8wMhoO/D4OpEWCHncN085ELdf0ZVyKtDJOolz7BtBuT9iOepUN+Z+ex0Fzh+Pu/MosjydblWTQIrd/9AQh4rXzCNBBOTeGFNM+A88KauXRs0+AX8Mnvg8GkKty29YRk0gMAtS csFeZ7Ym ZVpI39qITIRpLzCQ3Pf4DB2IV+oprRTtEfUIT1E+ft2ln0Dbi6L52uhrdT0EUySNcYe3TZo65Ac2H8k0eVzeBA1uhJjL8ftUROv92Hk+HqzpshAUJja7ECrNRrhk7U7nD49/FN0Th4w8RzVN/rDtwA/URKS/T7TKXsIRgFDFYh9TnhkpAGA9hzQs+zK/jQxRF20O/ylSSJt8vNBU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Commit 90292aca9854 ("arm64: mm: use appropriate ctors for page tables") introduced pgtable ctor calls in pgd_pgtable_alloc(). To identify the pgtable level and call the appropriate ctor, the *_SHIFT value associated with the pgtable level is used. However, those values do not unambiguously identify a level, because if a given level is folded, the *_SHIFT value will be equal to that of the upper level (e.g. PMD_SHIFT == PUD_SHIFT if PMD is folded). As things stand, there is probably not much damaged done by calling the ctor for a different level, and ARCH_ENABLE_SPLIT_PMD_PTLOCK is only selected if PMD isn't folded (so we don't needlessly initialise pmd_ptlock). Still, this is pretty confusing, and it would get even more confusing when adding ctor calls for the remaining levels. Let's simplify all this by using an enum to identify the pgtable level instead; this way folding becomes irrelevant. This is inspired by one of the m68k pgtable allocators (arch/m68k/include/asm/motorola_pgalloc.h). Signed-off-by: Kevin Brodsky --- arch/arm64/mm/mmu.c | 54 +++++++++++++++++++++++++++------------------ 1 file changed, 33 insertions(+), 21 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 437d4977bcf5..a7292ce9d7b8 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -46,6 +46,13 @@ #define NO_CONT_MAPPINGS BIT(1) #define NO_EXEC_MAPPINGS BIT(2) /* assumes FEAT_HPDS is not used */ +enum pgtable_type { + TABLE_PTE, + TABLE_PMD, + TABLE_PUD, + TABLE_P4D, +}; + u64 kimage_voffset __ro_after_init; EXPORT_SYMBOL(kimage_voffset); @@ -107,7 +114,7 @@ pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, } EXPORT_SYMBOL(phys_mem_access_prot); -static phys_addr_t __init early_pgtable_alloc(int shift) +static phys_addr_t __init early_pgtable_alloc(enum pgtable_type pgtable_type) { phys_addr_t phys; @@ -192,7 +199,7 @@ static void init_pte(pte_t *ptep, unsigned long addr, unsigned long end, static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(enum pgtable_type), int flags) { unsigned long next; @@ -207,7 +214,7 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, if (flags & NO_EXEC_MAPPINGS) pmdval |= PMD_TABLE_PXN; BUG_ON(!pgtable_alloc); - pte_phys = pgtable_alloc(PAGE_SHIFT); + pte_phys = pgtable_alloc(TABLE_PTE); ptep = pte_set_fixmap(pte_phys); init_clear_pgtable(ptep); ptep += pte_index(addr); @@ -243,7 +250,7 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, static void init_pmd(pmd_t *pmdp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), int flags) + phys_addr_t (*pgtable_alloc)(enum pgtable_type), int flags) { unsigned long next; @@ -277,7 +284,8 @@ static void init_pmd(pmd_t *pmdp, unsigned long addr, unsigned long end, static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), int flags) + phys_addr_t (*pgtable_alloc)(enum pgtable_type), + int flags) { unsigned long next; pud_t pud = READ_ONCE(*pudp); @@ -294,7 +302,7 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, if (flags & NO_EXEC_MAPPINGS) pudval |= PUD_TABLE_PXN; BUG_ON(!pgtable_alloc); - pmd_phys = pgtable_alloc(PMD_SHIFT); + pmd_phys = pgtable_alloc(TABLE_PMD); pmdp = pmd_set_fixmap(pmd_phys); init_clear_pgtable(pmdp); pmdp += pmd_index(addr); @@ -325,7 +333,7 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, static void alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(enum pgtable_type), int flags) { unsigned long next; @@ -339,7 +347,7 @@ static void alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long end, if (flags & NO_EXEC_MAPPINGS) p4dval |= P4D_TABLE_PXN; BUG_ON(!pgtable_alloc); - pud_phys = pgtable_alloc(PUD_SHIFT); + pud_phys = pgtable_alloc(TABLE_PUD); pudp = pud_set_fixmap(pud_phys); init_clear_pgtable(pudp); pudp += pud_index(addr); @@ -383,7 +391,7 @@ static void alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long end, static void alloc_init_p4d(pgd_t *pgdp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(enum pgtable_type), int flags) { unsigned long next; @@ -397,7 +405,7 @@ static void alloc_init_p4d(pgd_t *pgdp, unsigned long addr, unsigned long end, if (flags & NO_EXEC_MAPPINGS) pgdval |= PGD_TABLE_PXN; BUG_ON(!pgtable_alloc); - p4d_phys = pgtable_alloc(P4D_SHIFT); + p4d_phys = pgtable_alloc(TABLE_P4D); p4dp = p4d_set_fixmap(p4d_phys); init_clear_pgtable(p4dp); p4dp += p4d_index(addr); @@ -427,7 +435,7 @@ static void alloc_init_p4d(pgd_t *pgdp, unsigned long addr, unsigned long end, static void __create_pgd_mapping_locked(pgd_t *pgdir, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(enum pgtable_type), int flags) { unsigned long addr, end, next; @@ -455,7 +463,7 @@ static void __create_pgd_mapping_locked(pgd_t *pgdir, phys_addr_t phys, static void __create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + phys_addr_t (*pgtable_alloc)(enum pgtable_type), int flags) { mutex_lock(&fixmap_lock); @@ -468,10 +476,11 @@ static void __create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys, extern __alias(__create_pgd_mapping_locked) void create_kpti_ng_temp_pgd(pgd_t *pgdir, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), int flags); + phys_addr_t (*pgtable_alloc)(enum pgtable_type), + int flags); #endif -static phys_addr_t __pgd_pgtable_alloc(int shift) +static phys_addr_t __pgd_pgtable_alloc(enum pgtable_type pgtable_type) { /* Page is zeroed by init_clear_pgtable() so don't duplicate effort. */ void *ptr = (void *)__get_free_page(GFP_PGTABLE_KERNEL & ~__GFP_ZERO); @@ -480,23 +489,26 @@ static phys_addr_t __pgd_pgtable_alloc(int shift) return __pa(ptr); } -static phys_addr_t pgd_pgtable_alloc(int shift) +static phys_addr_t pgd_pgtable_alloc(enum pgtable_type pgtable_type) { - phys_addr_t pa = __pgd_pgtable_alloc(shift); + phys_addr_t pa = __pgd_pgtable_alloc(pgtable_type); struct ptdesc *ptdesc = page_ptdesc(phys_to_page(pa)); /* * Call proper page table ctor in case later we need to * call core mm functions like apply_to_page_range() on * this pre-allocated page table. - * - * We don't select ARCH_ENABLE_SPLIT_PMD_PTLOCK if pmd is - * folded, and if so pagetable_pte_ctor() becomes nop. */ - if (shift == PAGE_SHIFT) + switch (pgtable_type) { + case TABLE_PTE: BUG_ON(!pagetable_pte_ctor(NULL, ptdesc)); - else if (shift == PMD_SHIFT) + break; + case TABLE_PMD: BUG_ON(!pagetable_pmd_ctor(NULL, ptdesc)); + break; + default: + break; + } return pa; } From patchwork Mon Mar 17 14:16:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14019352 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F36A6C35FF9 for ; Mon, 17 Mar 2025 14:22:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A5946280002; Mon, 17 Mar 2025 10:22:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B8A3280001; Mon, 17 Mar 2025 10:22:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83356280002; Mon, 17 Mar 2025 10:22:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 663B0280001 for ; Mon, 17 Mar 2025 10:22:08 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 017B257F8F for ; Mon, 17 Mar 2025 14:22:10 +0000 (UTC) X-FDA: 83231257620.25.6E87D63 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf03.hostedemail.com (Postfix) with ESMTP id 320542000D for ; Mon, 17 Mar 2025 14:22:07 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf03.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742221328; a=rsa-sha256; cv=none; b=vqBhGqdJC0EonaRwklx93JDP0e3OVNoLAIIMWfHHiYL15dxfQj+OGEvOi9cKrZmsdcrDqj SI2pu0i0PXGQ87mtsj8SFH44HZ+lWSs3w4zqPUK3IxuvOsaA5ogK/s8i4NVLipqrTJAczQ zjPoBzvQD7MWcuhInkIfMzBi2Ki1b4E= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf03.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742221328; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=x0oPRrgKcuuA+3U0YyqK7D70URxIq3yDHbaKBG07nKg=; b=ZX/isGSnTGSkU4Er6+3/q3HEc4XP3soSEwtPvNQ3eDbgssJ4T5rA//9bpbcSNdfwngko6e qVUE7WSwjsvxpEQGUig8O8EQbwXUJUDIa8DiKBXFlRbgVlloYuj9vrnsDcKzjI1ef3Xrk7 so7/vNJIASeQdg+nWOoZP2xWYg2QWFg= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 49B6813D5; Mon, 17 Mar 2025 07:22:16 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2274D3F63F; Mon, 17 Mar 2025 07:22:03 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org Subject: [PATCH 08/11] arm64: mm: Always call PTE/PMD ctor in __create_pgd_mapping() Date: Mon, 17 Mar 2025 14:16:57 +0000 Message-ID: <20250317141700.3701581-9-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250317141700.3701581-1-kevin.brodsky@arm.com> References: <20250317141700.3701581-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Stat-Signature: mji58orojn7pa11hzp7kn1jm7y6wkty4 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 320542000D X-Rspam-User: X-HE-Tag: 1742221327-66807 X-HE-Meta: U2FsdGVkX19VDuLdFUNoIVKVg2JWoRInFxu8Hdzb2igaSZCsmkHcL+av6T3e8dOrkey7AMgJ23RFWFJS6WHRdKhenI2184MSVG4jFS7Wg2nRsVLTJEunf8oFe/drAJuCUpukkdQoznmZnYYFgowtA0oT9L6nWVuS/P8hCAaOK62/ctrbvsi3IaCAiZQ4fq6xkSGJc1vTJcAPeMGgq0Oy9rUBOkrwSEzgUI0YAMTF2DemPRuoJ+SGRPes2r2tLCVMfl6OfCFs36SuKjVp6dVnPktpTJgT1i16+aR5v7/LsCw8DSOGOluuV17LccO3UhS1WCFdRks8nIXzAOgBkb9rA8xd247oo9As69Acat2aRs2meF0Pc+knI9h2R96msL0BVNSdEDpUkctTALwIu3qVdvUttfDWbkVCg1TKj3c4uQcTdHgZvHsXpEYVzvdQT9v9MilkYAu+O0DnwA8Fy1l4Ly9F58858NytbO9rTRh5fcGr3BY2lA203deM1dCoODUq12dhMYjkhuVZojFoeHUkIwBhf7vJvMv72s8ACI4YKhtMCTq5AsYCOr82U5nhkka2q6jTuo5MovjlDwHJrYryr80iYFUKPnSPrHanPIMVn6KlWVS7R6vfwVhfOoHjvkJfcLkq22E0u0MP7uQSNFMQWPG1oLAvm4dxE00ETOlDSyH0y+xqvXZJFmD3ZUNBIaH8nDI130+jr6dhLsTNM4/NBh0xhJ0OMYnHDUsk036rBYAI7VzsDW2MJ069IcqWNiEJ5S4wD7Bw9A67Lb3aQ7TvB3f+LU4bffXAdDdv8VphUZtOHqWz5jyjYUkyuL4FrNdarJh71fRkHtDpju+9ScbDd78FH47FG2sDaNpZFUJzKgXiwuyLYor8T6NEZNbn4wQnd2JKQYzR/DXjxLDt6a92FU+Fokz46sICpOcx3zB4RyBvRLTpzijkgWnX+jmYtit0k+TI/+9GLrls1/XJO/C 8EotYlGX WwT2K+b99KtzemNE+Zc1upwiSp/d75aDLAuOeI+Y5R1ldsfTHA1kHqc/cZxAt+SlwVP9A/Lk+awnxNJg9Sg3hYip18WaD3Aa4xcLSj+CMAKANAyB0FnTNeo2u9IK8F7rIuQilig5X1gngA3VdNaHYbb/rqvDgcXYcr7gyvuUe8CBNBq2C3KdXcF1BhJqDEtokM0LglRRHUyrbRWU2xo3OOdvKbdknEMvym3wpc0KvUByEDuY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: TL;DR: always call the PTE/PMD ctor, passing the appropriate mm to skip ptlock_init() if unneeded. __create_pgd_mapping() is used for creating different kinds of mappings, and may allocate page table pages if passed an allocator callback. There are currently three such cases: 1. create_pgd_mapping(), which is used to create the EFI mapping 2. arch_add_memory() 3. map_entry_trampoline() 1. uses pgd_pgtable_alloc() as allocator callback, which calls the PTE/PMD ctor, while 2. and 3. use __pgd_pgtable_alloc(), which does not. The rationale is most likely that pgtables associated with init_mm do not make use of split page table locks, and it is therefore unnecessary to initialise them by calling the ctor. 2. operates on swapper_pg_dir so the allocated pgtables are clearly associated with init_mm, this is arguably the case for 3. too (the trampoline mapping is never modified so ptlocks are anyway irrelevant). 1. corresponds to efi_mm so ptlocks need to be initialised in that case. We are now moving towards calling the ctor for all page tables, even those associated with init_mm. pagetable_{pte,pmd}_ctor() have become aware of the associated mm so that the ptlock initialisation can be skipped for init_mm. This patch therefore amends the allocator callbacks so that the PTE/PMD ctor are always called, with an appropriate mm pointer to avoid unnecessary ptlock overhead. Modifying the prototype of the allocator callbacks to take the mm and propagating that pointer all the way down would be pretty invasive. Instead: * __pgd_pgtable_alloc() (cases 2. and 3. above) is replaced with pgd_pgtable_alloc_init_mm(), resulting in the ctors being called with &init_mm. This is the main functional change in this patch; the ptlock still isn't initialised, but other ctor actions (e.g. accounting-related) are now carried out for those allocated pgtables. * pgd_pgtable_alloc() (case 1. above) is replaced with pgd_pgtable_alloc_special_mm(), resulting in the ctors being called with NULL as mm. No functional change here; NULL essentially means "not init_mm", and the ptlock is still initialised. __pgd_pgtable_alloc() is now the common implementation of those two helpers. While at it we switch it to using pagetable_alloc() like standard pgtable allocator functions and remove the comment regarding ctor calls (ctors are now always expected to be called). Signed-off-by: Kevin Brodsky --- arch/arm64/mm/mmu.c | 41 +++++++++++++++++++++-------------------- 1 file changed, 21 insertions(+), 20 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index a7292ce9d7b8..accb0a33c59f 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -480,31 +480,22 @@ void create_kpti_ng_temp_pgd(pgd_t *pgdir, phys_addr_t phys, unsigned long virt, int flags); #endif -static phys_addr_t __pgd_pgtable_alloc(enum pgtable_type pgtable_type) +static phys_addr_t __pgd_pgtable_alloc(struct mm_struct *mm, + enum pgtable_type pgtable_type) { /* Page is zeroed by init_clear_pgtable() so don't duplicate effort. */ - void *ptr = (void *)__get_free_page(GFP_PGTABLE_KERNEL & ~__GFP_ZERO); + struct ptdesc *ptdesc = pagetable_alloc(GFP_PGTABLE_KERNEL & ~__GFP_ZERO, 0); + phys_addr_t pa; - BUG_ON(!ptr); - return __pa(ptr); -} - -static phys_addr_t pgd_pgtable_alloc(enum pgtable_type pgtable_type) -{ - phys_addr_t pa = __pgd_pgtable_alloc(pgtable_type); - struct ptdesc *ptdesc = page_ptdesc(phys_to_page(pa)); + BUG_ON(!ptdesc); + pa = page_to_phys(ptdesc_page(ptdesc)); - /* - * Call proper page table ctor in case later we need to - * call core mm functions like apply_to_page_range() on - * this pre-allocated page table. - */ switch (pgtable_type) { case TABLE_PTE: - BUG_ON(!pagetable_pte_ctor(NULL, ptdesc)); + BUG_ON(!pagetable_pte_ctor(mm, ptdesc)); break; case TABLE_PMD: - BUG_ON(!pagetable_pmd_ctor(NULL, ptdesc)); + BUG_ON(!pagetable_pmd_ctor(mm, ptdesc)); break; default: break; @@ -513,6 +504,16 @@ static phys_addr_t pgd_pgtable_alloc(enum pgtable_type pgtable_type) return pa; } +static phys_addr_t pgd_pgtable_alloc_init_mm(enum pgtable_type pgtable_type) +{ + return __pgd_pgtable_alloc(&init_mm, pgtable_type); +} + +static phys_addr_t pgd_pgtable_alloc_special_mm(enum pgtable_type pgtable_type) +{ + return __pgd_pgtable_alloc(NULL, pgtable_type); +} + /* * This function can only be used to modify existing table entries, * without allocating new levels of table. Note that this permits the @@ -542,7 +543,7 @@ void __init create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; __create_pgd_mapping(mm->pgd, phys, virt, size, prot, - pgd_pgtable_alloc, flags); + pgd_pgtable_alloc_special_mm, flags); } static void update_mapping_prot(phys_addr_t phys, unsigned long virt, @@ -756,7 +757,7 @@ static int __init map_entry_trampoline(void) memset(tramp_pg_dir, 0, PGD_SIZE); __create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS, entry_tramp_text_size(), prot, - __pgd_pgtable_alloc, NO_BLOCK_MAPPINGS); + pgd_pgtable_alloc_init_mm, NO_BLOCK_MAPPINGS); /* Map both the text and data into the kernel page table */ for (i = 0; i < DIV_ROUND_UP(entry_tramp_text_size(), PAGE_SIZE); i++) @@ -1362,7 +1363,7 @@ int arch_add_memory(int nid, u64 start, u64 size, flags |= NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; __create_pgd_mapping(swapper_pg_dir, start, __phys_to_virt(start), - size, params->pgprot, __pgd_pgtable_alloc, + size, params->pgprot, pgd_pgtable_alloc_init_mm, flags); memblock_clear_nomap(start, size); From patchwork Mon Mar 17 14:16:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14019353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98D0CC35FF3 for ; Mon, 17 Mar 2025 14:22:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5566728000A; Mon, 17 Mar 2025 10:22:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4DDB5280001; Mon, 17 Mar 2025 10:22:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3816F28000A; Mon, 17 Mar 2025 10:22:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 197F6280001 for ; Mon, 17 Mar 2025 10:22:13 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 9FE3382011 for ; Mon, 17 Mar 2025 14:22:14 +0000 (UTC) X-FDA: 83231257788.29.9DECB00 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP id B0DC44001A for ; Mon, 17 Mar 2025 14:22:12 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742221332; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IFWyEFvD7taIrIds3ww1RZCtlo3XiqYGCEZ44ZgHvMg=; b=nyVDlchlketc9mtrRbjcKkPoMaVTW4mGmn6zjSxT1JQKSQZkSoDKie4x4o35HFBB0IWcdX 3V62F+6OJAYHFuEL3jLGWo+EgowuPDDYd5vHaxyvkkJaojt7jEu7sb8bybfO1htKGNlhfG FPYQoXD8OlpN/yHbr+goN1i7koJWdWg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742221332; a=rsa-sha256; cv=none; b=sZTkBp476vpvR5A8zRqttSWZjg9nA6elrP7Hf2Ui0FicvJU4oHh5sFhR1Psk7xOuGgGRxO RBeec24ZtiJJdZlt/wYO8jkougS4zHRDmCihgGnW9n1ITkRwzlRhhSqrnKy1jC3flH77N1 aniYYzWn/yaQih3sjdeqWJLe/3HovSY= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D64111688; Mon, 17 Mar 2025 07:22:20 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id AEC523F63F; Mon, 17 Mar 2025 07:22:07 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org Subject: [PATCH 09/11] riscv: mm: Clarify ctor mm argument in alloc_{pte,pmd}_late Date: Mon, 17 Mar 2025 14:16:58 +0000 Message-ID: <20250317141700.3701581-10-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250317141700.3701581-1-kevin.brodsky@arm.com> References: <20250317141700.3701581-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B0DC44001A X-Stat-Signature: mnb8tduf68tg3faqaxadxh5u7tk5f7s4 X-HE-Tag: 1742221332-110409 X-HE-Meta: U2FsdGVkX1+qR+ipCHhHwvidlpe83MO6C6QL8LQ3Ngquph39ZMb0wlH8bt8TII/9wU8SFtuurvV86NbzAYqI/pJ8ZB0AMawUSJhXe03fOkltUTP9iNA2Al/Rrj9M+PavPcLxumHcp23jIpf9uXUWRU1yyKDBKvUQu7vEyU8f+MyqQkyXee5NmDH/7AK0pkk/7apvxHYMrzYuIZG8weXcUwipv8oMTxLhrznEU/AEEWg1y1KHR+J2L9a/er+qocFsu455DGgWBXH/bOXK/T5GOEY5M+j1ch9di3YZynYG4HaTFn0Lk3M5GJ0XTcXufCfSIs2Vn16st0A16HMNszg3QoNkHkbwXCRikEuYK0nMNfmo3bOxKfn3grwmXOHuiPX6EdIXHo7XYYVFOMsS2BZwiOfaCIUiAJ3s9G/Z1l0H6zBbVZveGkXNmDhQIrFnSKXx9wfPbzSuTzWP9YekBXAq//fn2vYE0zu8cbmpEvdc+IwSG4SWqXZ7vxKe6Yn/1/fWmklmMRF/Bvmb3gVqlh5KDMUUB9UdaXPknl+1+pRziSZlqb8jbuImM06ZEdlaiFrgIa6esCR8qjE0qrr0eBjBYmrSU3DPpLLANAV3PObvRob1ja1U7pl1zgXZWyZLgfjH5Ui4/0glsyilG0vKssHA7sXaUTeR08fXkyw8K0k8MjQAFIMAJrGOj3AIRGHuO3kJZhhLv2yoGbR/NorJ2xj1TXygul1Dn0K5AV6o1pMBs+VoY8w8ZaMRSuYevDpCw6P2J4z1ScmR6NyUoQc4Fw8xyfQOrdosnFIYE6GkQdoBn1hl7tl4boNVV8pJC9uk1DEcPREP6CmHieTZMZUe4iJDvnUfcnFt9mQsVrxepC1d8qD3xMjLINwEYM9sMkzkMBArT8jhxBozjn+ql9fauzM8yppbI9kZb3R3njMMAkRd6+NqP1aD6XqZOBMChJuWBbJdxVjm8kun/DpAewok3i9 2WueNl35 RcjHKWBZJHfVQ11j3SvNSLBQRW5oxHWvIiNLa1by9DIHyBuXeKHcWUxlWyxek9FzGwrW60/pXLDkunu3+igbQeeX0cet1AM6n2IMTjGKXT3Id9D6o1xEZ8a9GMHSFDw0yqCuxImwmdXXmo882sWKvLPVqZeMt6dDxr+UV0TBdS87w3853FU21L5m8X6CbKstZSRVV0Ji+oum9tLk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: pagetable_{pte,pmd}_ctor(mm, ptdesc) skip the ptlock initialisation if mm is &init_mm. To avoid unnecessary overhead, it is therefore preferable to pass the actual mm associated to the PTE/PMD. Unfortunately, this proves challenging for alloc_{pte,pmd}_late() as the associated mm is not available at the point where they are called - in fact not even top-level functions like create_pgd_mapping() are passed the mm. As a result they both call the ctor with NULL as mm; this is safe but potentially wasteful. This is not a new situation, but let's add a couple of comments to clarify it. Signed-off-by: Kevin Brodsky --- arch/riscv/mm/init.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index 703c3648cfa9..fb18940113f2 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -409,6 +409,11 @@ static phys_addr_t __meminit alloc_pte_late(uintptr_t va) { struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL & ~__GFP_HIGHMEM, 0); + /* + * We do not know which mm the PTE page is associated to at this point. + * Passing NULL to the ctor is the safe option, though it may result + * in unnecessary work (e.g. initialising the ptlock for init_mm). + */ BUG_ON(!ptdesc || !pagetable_pte_ctor(NULL, ptdesc)); return __pa((pte_t *)ptdesc_address(ptdesc)); } @@ -489,6 +494,7 @@ static phys_addr_t __meminit alloc_pmd_late(uintptr_t va) { struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL & ~__GFP_HIGHMEM, 0); + /* See comment in alloc_pte_late() regarding NULL passed the ctor */ BUG_ON(!ptdesc || !pagetable_pmd_ctor(NULL, ptdesc)); return __pa((pmd_t *)ptdesc_address(ptdesc)); } From patchwork Mon Mar 17 14:16:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14019354 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 218CDC28B30 for ; Mon, 17 Mar 2025 14:22:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C15C628000B; Mon, 17 Mar 2025 10:22:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B9D4A280001; Mon, 17 Mar 2025 10:22:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A403128000B; Mon, 17 Mar 2025 10:22:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 859E7280001 for ; Mon, 17 Mar 2025 10:22:17 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id CC33E161FC4 for ; Mon, 17 Mar 2025 14:22:18 +0000 (UTC) X-FDA: 83231257956.09.68B44DF Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf13.hostedemail.com (Postfix) with ESMTP id 5FF4B2000C for ; Mon, 17 Mar 2025 14:22:17 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf13.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742221337; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=g+3zGhkVnCLWy34Fe/Jcig7Y+t9bV+3UdTW8ay+DTF4=; b=MPz9b1vHmc1Eboo99dcagAnUNaO2MbHYWlyku9Am92AbQeZoyUMvVdfQccnjzPawCmY96j yoNM7rTzUN7W/19wBoSAcUxCJXUgZblAJg9SPuaV7TgCihRPDv6EX9iYr1lByFeDe202K+ kE59CmYDnpP+HHb7uaM1RsP/mQQRMEI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742221337; a=rsa-sha256; cv=none; b=n2U+KONrsQS0z6pKN6HBUo4Mk9gtepe3YB94kOS89SRhnYIqBc0di3gskpUsg5nQSD9z86 C+Waj+9ZbKWIb4tGso5mxQkgqbUwXIMFVH0Vx9eNThaK7yWXJtyjhnJEtPtMkugdX6Ua+Z MYLMCPxVS4l/f/Lq2vnO/eOnThtE9y4= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf13.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6E78C22FA; Mon, 17 Mar 2025 07:22:25 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 46C2C3F63F; Mon, 17 Mar 2025 07:22:12 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org Subject: [PATCH 10/11] arm64: mm: Call PUD/P4D ctor in __create_pgd_mapping() Date: Mon, 17 Mar 2025 14:16:59 +0000 Message-ID: <20250317141700.3701581-11-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250317141700.3701581-1-kevin.brodsky@arm.com> References: <20250317141700.3701581-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 5FF4B2000C X-Stat-Signature: mqcber4n4nkuoak487ewx6843zjap3oa X-HE-Tag: 1742221337-892875 X-HE-Meta: U2FsdGVkX1/VlTv3A95NPqeTs7+lk2m+8KKucLybPuOURRkuG3Q/Wzpj4Co6X/A+W4TyhnbIAEjsuKupXnp7EjWv8YTYy0ltpdfOCoWLevrC8QeX3rvR4yEDmFiNBwKCzkTUj5+w7oycxW2mWdDP4aETVot6+QIoKSjwVGBSqhGSINmg6HApv0HhDIz4iPpSR6HnDe8EsWi3pdcqz9ugNZU4Z3E6TsFo0N5+VKuIEFMlJOWXIoRpVRKLe+3jQ8jY0TM92pSWpmalsr3ZSeyeURw+1fsJpg2x6m+ki7r4g5vilcGAuobfaOwOvkfXlsoZwq5GJCtgsrtcb2Ba4ywbW08u87Xj/7EzFULqhK9zQav2duBEdm0dgsSiN4tbtmwMAdSU6kKUE4zkE2WG6IoY5M9MjMqLUfd6MVbl5H561I56SsDLj3geBxuYS+Mt3K/rMafHJLE8NCYXJOf7Lg9lFD8LZ+1XyTdsuq97zlYL7rA9970Ukle3l01o9FcKuUaG2taWq4jPOEi23KW1qKf1xgA5AwtIsPCDVSJ95gt91APiTa6mJwr/Djo0BvlAhVcJzCfg5khuGmUPL754UUj6m177FhIylyc92I+d7+YgKwzwN7W5jbl4AtvopCjbxz1SslDKn4f1q9jpm7DZsdVtl78Qysbz23cpgkU8vnaBM+t5t8ZVXn+zqjvxkEI632WfkFMZ/Kgg4BIDhv7m3AsXn425vGcSyZCV572WPbx2aC9E3u/IDl0BXiyLwHug1b35EOGehlmkPDCmITRyJreV3u/KRdh6vR1qAq58WysICHvOFX4SfcLZsihkFqNHum1G6S/YPU9On/ymyLHsJWGYr0CZ4S737uhESLjt1FpGsQjVAmPVFuLIoY0uqkYC1Fa7ghxcFUybic8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Constructors for PUD/P4D-level pgtables were recently introduced. They should be called for all pgtables; make sure they are called for special kernel mappings created by __create_pgd_mapping() too. Signed-off-by: Kevin Brodsky --- arch/arm64/mm/mmu.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index accb0a33c59f..10bf39654e77 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -497,7 +497,11 @@ static phys_addr_t __pgd_pgtable_alloc(struct mm_struct *mm, case TABLE_PMD: BUG_ON(!pagetable_pmd_ctor(mm, ptdesc)); break; - default: + case TABLE_PUD: + pagetable_pud_ctor(ptdesc); + break; + case TABLE_P4D: + pagetable_p4d_ctor(ptdesc); break; } From patchwork Mon Mar 17 14:17:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kevin Brodsky X-Patchwork-Id: 14019355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AB2FC282EC for ; Mon, 17 Mar 2025 14:22:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 080D428000C; Mon, 17 Mar 2025 10:22:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 00802280001; Mon, 17 Mar 2025 10:22:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D9DFD28000C; Mon, 17 Mar 2025 10:22:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id BC0EC280001 for ; Mon, 17 Mar 2025 10:22:22 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 572031CD33D for ; Mon, 17 Mar 2025 14:22:24 +0000 (UTC) X-FDA: 83231258208.06.8BBD2FC Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf20.hostedemail.com (Postfix) with ESMTP id 2BD351C0015 for ; Mon, 17 Mar 2025 14:22:21 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742221342; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FjqsABZix1moSeU8RQR7pgSF9B7lHOGmA+zkeUwNAF8=; b=l6gQtKfQsPuKv45U+YACa5YJGFuZ5nTRcjpUPx1MR9Uyf0gBokF1fjFGct/dxtVha2k8JL /q64G2SDz6ejHH7NqbOWKLfTUMIeTJSDFLSfOBwTsLtp5ePJ55xnVP20f8DWGJqr6V6Mq0 G6buex/11rIri5TI3+6Imu2DniAWB2E= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of kevin.brodsky@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=kevin.brodsky@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742221342; a=rsa-sha256; cv=none; b=39gEirTRHufXxZRmbuySg57K7e4F0S5i1VsdGNxZWZZk//QGzKzrvGHZEks851KcebuVrx Y3sn5vUpLqnhInb6hZpZ+NKTRKPhYnM16kyskoDGaFs0R8cHMZqM637JePqDs0XHh0rLHv ULK0ixh5ER/JadEjqnk7IA+t0FuBbcM= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 06A762573; Mon, 17 Mar 2025 07:22:30 -0700 (PDT) Received: from e123572-lin.arm.com (e123572-lin.cambridge.arm.com [10.1.194.54]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D35C43F63F; Mon, 17 Mar 2025 07:22:16 -0700 (PDT) From: Kevin Brodsky To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Kevin Brodsky , Albert Ou , Andreas Larsson , Andrew Morton , Catalin Marinas , Dave Hansen , "David S. Miller" , Geert Uytterhoeven , Linus Walleij , Madhavan Srinivasan , Mark Rutland , Matthew Wilcox , Michael Ellerman , "Mike Rapoport (IBM)" , Palmer Dabbelt , Paul Walmsley , Peter Zijlstra , Qi Zheng , Ryan Roberts , Will Deacon , Yang Shi , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-openrisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, sparclinux@vger.kernel.org Subject: [PATCH 11/11] riscv: mm: Call PUD/P4D ctor in special kernel pgtable alloc Date: Mon, 17 Mar 2025 14:17:00 +0000 Message-ID: <20250317141700.3701581-12-kevin.brodsky@arm.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20250317141700.3701581-1-kevin.brodsky@arm.com> References: <20250317141700.3701581-1-kevin.brodsky@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 2BD351C0015 X-Rspamd-Server: rspam03 X-Stat-Signature: xwetj6tr43ftqhkxxafx7hcm9sbdr8ci X-HE-Tag: 1742221341-671368 X-HE-Meta: U2FsdGVkX1+qnUnOy1Ghcpu9TIpoid2eqzQ1OqhRJFrjZWb7EbcJtazGOAB393WacWfeF3UNbqgf30BY+7hPxva7EB5yOqXNlDzXyMVZsTTJ1I3mCR2J+boFw5sKH5QvRKxs6HeBdu8hDPAPHrx61s/PTVH1Lskt/PhDPWr0nCNzY6nlKp8TLvTrS5qM8M8zD+aEwaAz80sVDUxFdBfDDefu4q1r/bWQhzhTAhBJPy7trlVaMpifBpD4qYlJKLf8KA3e0VSf5tcG8ou9qy6ejdf4TXM7elRTkJErLpm/Jp/3M+2nfqfvtFTBOo/gI0Vqu64OM37RMN9UijSEBmdrfmj+lVOtS38ia5pd/nfzWDerENzTC/PINwurjBHcJZmk/4yjgQt59/mYPQgQ14Hix0sZVk94i8SMkgP2LIT0ULgDcqg5uf+dNLSkY4WUfxKqwcNSQv/sJiJo1bTedz6PSuE26CtZmD9xJL9g99Uw4xR2WptTe2nAat6WBVGTJh+Gulb2nvvFLvco/ZaZiZhGHnygHzYKcECwVic1spT3CabIDCAOkT9JJAkSRDCVc2mR7Dg7nPbwJZvq40YV0D/TSGusf6wb5bLwWhM9IM/5ejTy8JNNyXiziF+cGsmxOwFJrcCsERgoczqQNKhuZWPynqKCpUDOYruaLlMqZtZDPgJ+hin0ACklqWcYRKHYlk9cJiNHJonFlVdyi5a04BMaaxBx52nr6L+QJOYHnx10itZZhK5+M365VoJlCCVj2dUpPnE+lRumn7GhrXPTE0owVkYUbCcUSTIrvhA0Vmg6By2EOkaqg64pN7JrQmDgC67ieKjYZATz28MBpzSgvNy2SB+oxNOMPsfa9lv3ASbikxPV5dCuL8uuFxn7wBaEAkM9ElGaOSCWZWWHNVtDD+D6+eYslqfeE17bcbzYxlQRx7I8F+RmKHJFNZt86NafkvSQE7rBXWIjioSFavCLEMD SqxCX/5U OyhABgAozlEl1iBXwp3SgqLN9+DzZIdo7zhsxtzaW1NVoe76lvIPNhOIpa2sJfAEjIRCbpYdlYpVtnrOE01YXCY3vlSUKQd5YJClVNtGSwfJu6NvGmAove4PPj6LXG6Gn5Y18WUh7iqXYx92T4/Mizj17IQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Constructors for PUD/P4D-level pgtables were recently introduced. They should be called for all pgtables; make sure they are called for special kernel mappings created by create_pgd_mapping() too. While at it also switch to using pagetable_alloc() like in alloc_{pte,pmd}_late(). Signed-off-by: Kevin Brodsky --- arch/riscv/mm/init.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index fb18940113f2..dc2715f3fd00 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -557,11 +557,11 @@ static phys_addr_t __init alloc_pud_fixmap(uintptr_t va) static phys_addr_t __meminit alloc_pud_late(uintptr_t va) { - unsigned long vaddr; + struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL, 0); - vaddr = __get_free_page(GFP_KERNEL); - BUG_ON(!vaddr); - return __pa(vaddr); + BUG_ON(!ptdesc); + pagetable_pud_ctor(ptdesc); + return __pa((pud_t *)ptdesc_address(ptdesc)); } static p4d_t *__init get_p4d_virt_early(phys_addr_t pa) @@ -595,11 +595,11 @@ static phys_addr_t __init alloc_p4d_fixmap(uintptr_t va) static phys_addr_t __meminit alloc_p4d_late(uintptr_t va) { - unsigned long vaddr; + struct ptdesc *ptdesc = pagetable_alloc(GFP_KERNEL, 0); - vaddr = __get_free_page(GFP_KERNEL); - BUG_ON(!vaddr); - return __pa(vaddr); + BUG_ON(!ptdesc); + pagetable_p4d_ctor(ptdesc); + return __pa((p4d_t *)ptdesc_address(ptdesc)); } static void __meminit create_pud_mapping(pud_t *pudp, uintptr_t va, phys_addr_t pa, phys_addr_t sz,