From patchwork Mon Jun 10 05:54:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 13691508 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BB83C27C5E for ; Mon, 10 Jun 2024 05:55:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D265A6B00A1; Mon, 10 Jun 2024 01:55:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CAF146B00A3; Mon, 10 Jun 2024 01:55:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B02B26B00A4; Mon, 10 Jun 2024 01:55:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8EEB16B00A1 for ; Mon, 10 Jun 2024 01:55:29 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id EC040C0D8B for ; Mon, 10 Jun 2024 05:55:28 +0000 (UTC) X-FDA: 82213916736.07.959B862 Received: from pegase1.c-s.fr (pegase1.c-s.fr [93.17.236.30]) by imf26.hostedemail.com (Postfix) with ESMTP id D027D14000A for ; Mon, 10 Jun 2024 05:55:26 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.236.30 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu; dmarc=pass (policy=quarantine) header.from=csgroup.eu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717998927; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Xnap6QF/MKf6QAqysV3GvrdUulr8iCU56pdyhGUPre8=; b=oT1wN4D+2ooXGzeDH7o0g0ssmsUWhD+o8AcgM7nE/HN+voHlO9sutRsxfeHN6djj+rp/fP MUuLaMLmjIq04HFoG56PrJf+jWKJOMuO+Y2GBxOYgDpZaw2PIXTnp6eGMvZEP5DhWZ/9io /vpUg2J6iSaGyGmM/IVzupQkhfDkE1o= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717998927; a=rsa-sha256; cv=none; b=HHPVdxoQ2RAFrrCYqQ4+eWy1zW+ZA19aKb2fEQtBpeYItmeJBDMK0w6kXbs4Cvx2zs1xX4 hkR2G1EJ7XFhtjwsG6S4rtL/V6uUygeyFuxaxbyxPRQOCm8nri0PtRQ5mex4rggSoVBvNY mwcVVvsvvGk6ejvVv749C575EdRYsCs= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of christophe.leroy@csgroup.eu designates 93.17.236.30 as permitted sender) smtp.mailfrom=christophe.leroy@csgroup.eu; dmarc=pass (policy=quarantine) header.from=csgroup.eu Received: from localhost (mailhub3.si.c-s.fr [192.168.12.233]) by localhost (Postfix) with ESMTP id 4VyLcR0r0wz9v4H; Mon, 10 Jun 2024 07:55:03 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aCiUBYeJtwWr; Mon, 10 Jun 2024 07:55:03 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 4VyLcL1MHkz9v53; Mon, 10 Jun 2024 07:54:58 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 2ABCB8B764; Mon, 10 Jun 2024 07:54:58 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id aZaDYQFEsWdC; Mon, 10 Jun 2024 07:54:58 +0200 (CEST) Received: from PO20335.idsi0.si.c-s.fr (unknown [172.25.230.108]) by messagerie.si.c-s.fr (Postfix) with ESMTP id ECD138B76C; Mon, 10 Jun 2024 07:54:57 +0200 (CEST) From: Christophe Leroy To: Andrew Morton , Jason Gunthorpe , Peter Xu , Oscar Salvador , Michael Ellerman , Nicholas Piggin Cc: Christophe Leroy , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org Subject: [PATCH v5 06/18] powerpc/mm: Allow hugepages without hugepd Date: Mon, 10 Jun 2024 07:54:51 +0200 Message-ID: X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1717998887; l=6864; i=christophe.leroy@csgroup.eu; s=20211009; h=from:subject:message-id; bh=OHpbr2EW8Ehxc4cGWtKirj+0lOuzXeICYvz9BBmzSDA=; b=hA87SGuiRA2PPRakE9jIhSBJ4zY9sUUbYP2+o8p6/WhEyK5l5NyrDfO7geaWjnft/kDOXBOE5 xRIZWcFqBltB8PBCaarB30z1z7EXi5uYEGQSBd4JOjAkE1INOn4Kwwo X-Developer-Key: i=christophe.leroy@csgroup.eu; a=ed25519; pk=HIzTzUj91asvincQGOFx6+ZF5AoUuP9GdOtQChs7Mm0= X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: D027D14000A X-Rspam-User: X-Stat-Signature: j3j4wsohs71adkzjy63b9xjmup3brrge X-HE-Tag: 1717998926-590435 X-HE-Meta: U2FsdGVkX192KUmtTfJ4nes5K0/13PROaK0ywJtLSun+UEc/p/Sbx0wZJ12QhaaCbxX7+CA88KZDH5fedD+NLdPLBowlcqKdEPTFJ7BmGYxWkUL9S63f3XpZZIFcZUBFlWq+zo/3Gyli1XWGsKQSM9AGNbxDbwiGdDEalhM3HBlyKfycECWUZaUHMesW9hWRRwC/PDhP7Zp9MuLmBNRCi1hPrjRYh7e32N7PItvDRMnm9Kp7gTugmMqQCZ2oc0pX19TzQWzBgOjGShhc0FsHtlvtnpnQuIm3lmG6x8j9XMnHy7Nt1qqXqnKxuQcLzQNjlwoqlFrRwROCkJ9Swu6hrkgp6y++8S73GNyky6LwWZkeg8yRDrR9j4mYyxo/6L1teRRXmsWUMNC8noONJ0xYaur3AsyV9xuwH4ZSOFQF/p07aoRuR+dkxRyN9T1Qbiur2aTTsBPrheOANUDeRTd8+xo/psKa+GMTEFeOGvyklrhsuLnL8iHFzMGqkOc5pxdMNRTd+6sUVaIcCTxz85pIo2yQVO5pJi9xl9eTxa7PCFFa3fjslmthwnrMCi1Sq+zIQ03Xe3HmjqJgXUDbb91JXvkouoBo8NrKZVEglUlFQi0U8SOnWC0nzMVx8HgBLdY4gfbyVB4lP3EOfhxY2x651fIIZmFKjY4xJNFCsb9QBB3SUwA0kgxuTCBFsjFByPtSxRsgUsf8MVX8bHsr6TdztdonObMiIi6oDJVGgDlgqe5edkZK7RXDKPtT/f3aRiX1fMkMVTE5FDOe6eOaGEVs/wAb6947BSYq6vMBoYVbtPPV0T+u/rgam5HJPWrcke4uuv0+snvMBSVKR/Fu+8WuBRHsmO4KHfpA+gp9HipHAbiul4NpEtbYpgVhBzH66RLc4s866KpBT8UWt5xMKlwu3RV1b+8yHYoCZuSu6L3AUAG4zheN4ZCtKqij6iOMPYdcQ4DNycCXKCX0Rt5RD32 FT8GW7AG 2X7UxETaqnj4SS/rtHlKnsll4UHgnop8F4UBZXvnzHuHRQlfWBc+i9E89gCDJRiwmSCQZxeMTtj9Usi7ssPzYOIbg+ehK1W4DE6d/8LrGoj5wB0+noRq8nZYUxC0M/xw5KZyn43hTGhVQBnQWWdXJC+1yvyNsav5nelzL X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In preparation of implementing huge pages on powerpc 8xx without hugepd, enclose hugepd related code inside an ifdef CONFIG_ARCH_HAS_HUGEPD This also allows removing some stubs. Signed-off-by: Christophe Leroy Reviewed-by: Oscar Salvador --- v3: - Prepare huge_pte_alloc() for full standard topology, not only for 2-level - Reordered last part of huge_pte_alloc() v4: - Rebased of v6.10-rc1 v5: - Moved the Kconfig split in this patch. --- arch/powerpc/Kconfig | 1 - arch/powerpc/include/asm/book3s/32/pgalloc.h | 2 -- arch/powerpc/include/asm/hugetlb.h | 10 ++---- arch/powerpc/include/asm/nohash/pgtable.h | 2 +- arch/powerpc/mm/hugetlbpage.c | 33 ++++++++++++++++++++ arch/powerpc/mm/pgtable.c | 2 ++ arch/powerpc/platforms/Kconfig.cputype | 3 ++ 7 files changed, 41 insertions(+), 12 deletions(-) diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index c88c6d46a5bc..b60b6e991227 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -135,7 +135,6 @@ config PPC select ARCH_HAS_DMA_MAP_DIRECT if PPC_PSERIES select ARCH_HAS_FORTIFY_SOURCE select ARCH_HAS_GCOV_PROFILE_ALL - select ARCH_HAS_HUGEPD if HUGETLB_PAGE select ARCH_HAS_KCOV select ARCH_HAS_KERNEL_FPU_SUPPORT if PPC64 && PPC_FPU select ARCH_HAS_MEMBARRIER_CALLBACKS diff --git a/arch/powerpc/include/asm/book3s/32/pgalloc.h b/arch/powerpc/include/asm/book3s/32/pgalloc.h index dc5c039eb28e..dd4eb3063175 100644 --- a/arch/powerpc/include/asm/book3s/32/pgalloc.h +++ b/arch/powerpc/include/asm/book3s/32/pgalloc.h @@ -47,8 +47,6 @@ static inline void pgtable_free(void *table, unsigned index_size) } } -#define get_hugepd_cache_index(x) (x) - static inline void pgtable_free_tlb(struct mmu_gather *tlb, void *table, int shift) { diff --git a/arch/powerpc/include/asm/hugetlb.h b/arch/powerpc/include/asm/hugetlb.h index ea71f7245a63..79176a499763 100644 --- a/arch/powerpc/include/asm/hugetlb.h +++ b/arch/powerpc/include/asm/hugetlb.h @@ -30,10 +30,12 @@ static inline int is_hugepage_only_range(struct mm_struct *mm, } #define is_hugepage_only_range is_hugepage_only_range +#ifdef CONFIG_ARCH_HAS_HUGEPD #define __HAVE_ARCH_HUGETLB_FREE_PGD_RANGE void hugetlb_free_pgd_range(struct mmu_gather *tlb, unsigned long addr, unsigned long end, unsigned long floor, unsigned long ceiling); +#endif #define __HAVE_ARCH_HUGE_PTEP_GET_AND_CLEAR static inline pte_t huge_ptep_get_and_clear(struct mm_struct *mm, @@ -67,14 +69,6 @@ static inline void flush_hugetlb_page(struct vm_area_struct *vma, { } -#define hugepd_shift(x) 0 -static inline pte_t *hugepte_offset(hugepd_t hpd, unsigned long addr, - unsigned pdshift) -{ - return NULL; -} - - static inline void __init gigantic_hugetlb_cma_reserve(void) { } diff --git a/arch/powerpc/include/asm/nohash/pgtable.h b/arch/powerpc/include/asm/nohash/pgtable.h index f5f39d4f03c8..e7fc1314c23e 100644 --- a/arch/powerpc/include/asm/nohash/pgtable.h +++ b/arch/powerpc/include/asm/nohash/pgtable.h @@ -340,7 +340,7 @@ static inline void __set_pte_at(struct mm_struct *mm, unsigned long addr, #define pgprot_writecombine pgprot_noncached_wc -#ifdef CONFIG_HUGETLB_PAGE +#ifdef CONFIG_ARCH_HAS_HUGEPD static inline int hugepd_ok(hugepd_t hpd) { #ifdef CONFIG_PPC_8xx diff --git a/arch/powerpc/mm/hugetlbpage.c b/arch/powerpc/mm/hugetlbpage.c index 594a4b7b2ca2..20fad59ff9f5 100644 --- a/arch/powerpc/mm/hugetlbpage.c +++ b/arch/powerpc/mm/hugetlbpage.c @@ -42,6 +42,7 @@ pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr, unsigned long s return __find_linux_pte(mm->pgd, addr, NULL, NULL); } +#ifdef CONFIG_ARCH_HAS_HUGEPD static int __hugepte_alloc(struct mm_struct *mm, hugepd_t *hpdp, unsigned long address, unsigned int pdshift, unsigned int pshift, spinlock_t *ptl) @@ -193,6 +194,36 @@ pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, return hugepte_offset(*hpdp, addr, pdshift); } +#else +pte_t *huge_pte_alloc(struct mm_struct *mm, struct vm_area_struct *vma, + unsigned long addr, unsigned long sz) +{ + p4d_t *p4d; + pud_t *pud; + pmd_t *pmd; + + addr &= ~(sz - 1); + + p4d = p4d_offset(pgd_offset(mm, addr), addr); + if (!mm_pud_folded(mm) && sz >= P4D_SIZE) + return (pte_t *)p4d; + + pud = pud_alloc(mm, p4d, addr); + if (!pud) + return NULL; + if (!mm_pmd_folded(mm) && sz >= PUD_SIZE) + return (pte_t *)pud; + + pmd = pmd_alloc(mm, pud, addr); + if (!pmd) + return NULL; + + if (sz >= PMD_SIZE) + return (pte_t *)pmd; + + return pte_alloc_huge(mm, pmd, addr); +} +#endif #ifdef CONFIG_PPC_BOOK3S_64 /* @@ -248,6 +279,7 @@ int __init alloc_bootmem_huge_page(struct hstate *h, int nid) return __alloc_bootmem_huge_page(h, nid); } +#ifdef CONFIG_ARCH_HAS_HUGEPD #ifndef CONFIG_PPC_BOOK3S_64 #define HUGEPD_FREELIST_SIZE \ ((PAGE_SIZE - sizeof(struct hugepd_freelist)) / sizeof(pte_t)) @@ -505,6 +537,7 @@ void hugetlb_free_pgd_range(struct mmu_gather *tlb, } } while (addr = next, addr != end); } +#endif bool __init arch_hugetlb_valid_size(unsigned long size) { diff --git a/arch/powerpc/mm/pgtable.c b/arch/powerpc/mm/pgtable.c index bce8a8619589..9010973f036c 100644 --- a/arch/powerpc/mm/pgtable.c +++ b/arch/powerpc/mm/pgtable.c @@ -496,8 +496,10 @@ pte_t *__find_linux_pte(pgd_t *pgdir, unsigned long ea, if (!hpdp) return NULL; +#ifdef CONFIG_ARCH_HAS_HUGEPD ret_pte = hugepte_offset(*hpdp, ea, pdshift); pdshift = hugepd_shift(*hpdp); +#endif out: if (hpage_shift) *hpage_shift = pdshift; diff --git a/arch/powerpc/platforms/Kconfig.cputype b/arch/powerpc/platforms/Kconfig.cputype index b2d8c0da2ad9..0c3c69ebf19f 100644 --- a/arch/powerpc/platforms/Kconfig.cputype +++ b/arch/powerpc/platforms/Kconfig.cputype @@ -37,6 +37,7 @@ config PPC_85xx config PPC_8xx bool "Freescale 8xx" + select ARCH_HAS_HUGEPD if HUGETLB_PAGE select ARCH_SUPPORTS_HUGETLBFS select FSL_SOC select PPC_KUEP @@ -98,6 +99,7 @@ config PPC_BOOK3S_64 select ARCH_ENABLE_HUGEPAGE_MIGRATION if HUGETLB_PAGE && MIGRATION select ARCH_ENABLE_SPLIT_PMD_PTLOCK select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE + select ARCH_HAS_HUGEPD if HUGETLB_PAGE select ARCH_SUPPORTS_HUGETLBFS select ARCH_SUPPORTS_NUMA_BALANCING select HAVE_MOVE_PMD @@ -290,6 +292,7 @@ config PPC_BOOK3S config PPC_E500 select FSL_EMB_PERFMON bool + select ARCH_HAS_HUGEPD if HUGETLB_PAGE select ARCH_SUPPORTS_HUGETLBFS if PHYS_64BIT || PPC64 select PPC_SMP_MUXED_IPI select PPC_DOORBELL