From patchwork Wed Apr 28 16:46:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 12229727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3755DC433ED for ; Wed, 28 Apr 2021 16:48:15 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CBD6761400 for ; Wed, 28 Apr 2021 16:48:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CBD6761400 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=csgroup.eu Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:MIME-Version:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Date:Cc:To:Subject:From:References:In-Reply-To: Message-Id:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=gljFihEeffXXmbJpGDinurWBjVdKKyymeGPxgYRlHdA=; b=Yd7eYIXQMwWc/PczsekzHuWIgt kmF2EszEG0S39FsiUCPOjm4zCqFUBSJNrYLe+vmeSGXIaXhLlhVwCJB9JOEaEq8kBxlP+UkfR9srx 4NBbUtN/ILG5wIfjTCgopyrNXLOOtUGzmYAT868+0KY6LKTWDRz/h/G9pUg20cc2OG1LsAg2+dW39 Kf2hTmysHq4TCgqhMspgZrRpaQCBlNISKjK8lXFuI9fK/4cTOy0dt6GdoEyZNkE1mZ9lK8rjXfkaY QUEN8d+LBr9TE5KrtfBl6NnPxUjgFPWvAcG8JWeg29O5S9pe+1lmV/JcmiiNcQIsJSrXqYO4SrYQn WWhWqCVw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lbnL1-003suz-8X; Wed, 28 Apr 2021 16:46:59 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lbnKO-003ske-Cl for linux-arm-kernel@desiato.infradead.org; Wed, 28 Apr 2021 16:46:20 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Date:Cc:To:Subject:From:References: In-Reply-To:Message-Id:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description; bh=6MX3VwPke2SIwio7S7OVpEGeWLjZK0tHOe0790O0QjA=; b=dRHs945ITgQZ4cqGg9FcBmTUxk TkPv0D5glVUo3YK/IYFVYY6lna87ODDICBHP4342I1Q6c1L5vvURRN6Dx/mPpdk/EJVEkaUwrCbSu RLnb6peWsV27ISENmHr9itB4kXW4nS068U14q6cEUhyJvBDc3MDa7/G5R6Q/5PG7GV5RAASDQq3z9 0/Ml5tMUrr6e75BKqVZv0m4sWAbsa4ZcgsrBDDHnGmUfVEGtxrVzIUKXXiqrjJsenJL4o3DLzumfZ +HHu2mcWUpEvQSEQHz4KOqHzUI6CPR7J0ZuTkt6ktIPX3mr2/G0ZvJolLeUK4qifeSRfZIVzIrE1e WWgN2eyg==; Received: from pegase1.c-s.fr ([93.17.236.30]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lbnKJ-0001tB-BL for linux-arm-kernel@lists.infradead.org; Wed, 28 Apr 2021 16:46:17 +0000 Received: from localhost (mailhub3.si.c-s.fr [192.168.12.233]) by localhost (Postfix) with ESMTP id 4FVkzS3hGZz9tcY; Wed, 28 Apr 2021 18:46:12 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hTMb2D2g1dlP; Wed, 28 Apr 2021 18:46:12 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 4FVkzS2dXkz9tcV; Wed, 28 Apr 2021 18:46:12 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 2C11B8B837; Wed, 28 Apr 2021 18:46:12 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id 7CIbKb9Hwl0l; Wed, 28 Apr 2021 18:46:12 +0200 (CEST) Received: from po15610vm.idsi0.si.c-s.fr (unknown [192.168.4.90]) by messagerie.si.c-s.fr (Postfix) with ESMTP id E3DD98B831; Wed, 28 Apr 2021 18:46:11 +0200 (CEST) Received: by po15610vm.idsi0.si.c-s.fr (Postfix, from userid 0) id C29716428C; Wed, 28 Apr 2021 16:46:11 +0000 (UTC) Message-Id: In-Reply-To: References: From: Christophe Leroy Subject: [RFC PATCH v1 1/4] mm/ioremap: Fix iomap_max_page_shift To: Andrew Morton , Nicholas Piggin , Mike Kravetz , Mike Rapoport Cc: linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, sparclinux@vger.kernel.org, linux-mm@kvack.org Date: Wed, 28 Apr 2021 16:46:11 +0000 (UTC) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210428_094615_560636_2645EB6F X-CRM114-Status: GOOD ( 11.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org iomap_max_page_shift is expected to contain a page shift, so it can't be a 'bool', has to be an 'unsigned int' And fix the default values: P4D_SHIFT is when huge iomap is allowed. Signed-off-by: Christophe Leroy --- mm/ioremap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/ioremap.c b/mm/ioremap.c index d1dcc7e744ac..2f7193c6a99e 100644 --- a/mm/ioremap.c +++ b/mm/ioremap.c @@ -16,16 +16,16 @@ #include "pgalloc-track.h" #ifdef CONFIG_HAVE_ARCH_HUGE_VMAP -static bool __ro_after_init iomap_max_page_shift = PAGE_SHIFT; +static unsigned int __ro_after_init iomap_max_page_shift = P4D_SHIFT; static int __init set_nohugeiomap(char *str) { - iomap_max_page_shift = P4D_SHIFT; + iomap_max_page_shift = PAGE_SHIFT; return 0; } early_param("nohugeiomap", set_nohugeiomap); #else /* CONFIG_HAVE_ARCH_HUGE_VMAP */ -static const bool iomap_max_page_shift = PAGE_SHIFT; +static const unsigned int iomap_max_page_shift = PAGE_SHIFT; #endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */ int ioremap_page_range(unsigned long addr, From patchwork Wed Apr 28 16:46:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 12229733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FD72C433B4 for ; Wed, 28 Apr 2021 16:49:28 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CEFCF6105A for ; Wed, 28 Apr 2021 16:49:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CEFCF6105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=csgroup.eu Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:MIME-Version:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Date:Cc:To:Subject:From:References:In-Reply-To: Message-Id:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=sV95w7FxmLBKo4zQqW/xZ6nN11UMxkgRvsktmtZz0pU=; b=J3UV1RezGRRfy+YsbGPEAODkeF mU1b0x5ajmL0YB6j/tZnbdIMjGMf21MfJQu4ui6+8GgLImWP+ieNR4cnQz0hxjsDDMXiZU0kY/02P zAN1folwgW55kb0KjDhacg29SAU4DxXZZuHdkq7xmIl3JfuLkwR+90Cs+I1uI3+qoZW0oQFZn+3li Av+zrCrSgoX5SjYbggGyblBxQjUSGfC6lNJDoarLpH2lauHPtpG1RH3wDjt/4A8J5iKW6nT9Ym2GA Z6txACWBwLKCEhX8pbX62vgDJlIIGPfA2Pu0veqI9KRt0P7Sg3FgDafZebEdXEiG1bpvhE0r7QoFV no0vEdCQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lbnLg-003t7L-E6; Wed, 28 Apr 2021 16:47:40 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lbnKO-003slC-Cm for linux-arm-kernel@desiato.infradead.org; Wed, 28 Apr 2021 16:46:20 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Date:Cc:To:Subject:From:References: In-Reply-To:Message-Id:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description; bh=SggqbV0I5iNV4F25jtJ5ywPzBaxXmzI1AicRdaDPoog=; b=tuq48A8bb+CqWsxzldNG1w9lwO E4hXbUherFfVlKm+ZdulVWhjqqPjFk/QoTVmqJE2j4GFZ0JGVS3g0P1SCSj/JBDKssn1/lq90NanJ hvsuJUWolnWiD4JLRWfKY2CrRYycksf2p3xjIJ3ETX3uOmdLkeMWfbYMmWUXfyAVHXUJv+7KVBZ4g ZA5oG2qwo+W4aeHtHsvvxkjqbSl7NN97bQAFE1bPhUXpogwjbWgElA+z1z6eOdWNIbZF752J6L9Wf 7kxKj48iTwnN2fM2Bl34443QKdO9g5p4Fn43he+706+kRJkC12ixaz/6E7yPTf1Xpw5IIprchGKqr 0Pf+llrw==; Received: from pegase1.c-s.fr ([93.17.236.30]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lbnKJ-0001tQ-SW for linux-arm-kernel@lists.infradead.org; Wed, 28 Apr 2021 16:46:18 +0000 Received: from localhost (mailhub3.si.c-s.fr [192.168.12.233]) by localhost (Postfix) with ESMTP id 4FVkzT4m2Dz9tcc; Wed, 28 Apr 2021 18:46:13 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4U65XGU88BaS; Wed, 28 Apr 2021 18:46:13 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 4FVkzT3pV5z9tcV; Wed, 28 Apr 2021 18:46:13 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 6985C8B837; Wed, 28 Apr 2021 18:46:13 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id 1Av2VT33WRBs; Wed, 28 Apr 2021 18:46:13 +0200 (CEST) Received: from po15610vm.idsi0.si.c-s.fr (unknown [192.168.4.90]) by messagerie.si.c-s.fr (Postfix) with ESMTP id E9A9D8B831; Wed, 28 Apr 2021 18:46:12 +0200 (CEST) Received: by po15610vm.idsi0.si.c-s.fr (Postfix, from userid 0) id CA0F76428C; Wed, 28 Apr 2021 16:46:12 +0000 (UTC) Message-Id: In-Reply-To: References: From: Christophe Leroy Subject: [RFC PATCH v1 2/4] mm/hugetlb: Change parameters of arch_make_huge_pte() To: Andrew Morton , Nicholas Piggin , Mike Kravetz , Mike Rapoport Cc: linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, sparclinux@vger.kernel.org, linux-mm@kvack.org Date: Wed, 28 Apr 2021 16:46:12 +0000 (UTC) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210428_094616_099900_5FA64676 X-CRM114-Status: GOOD ( 15.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org At the time being, arch_make_huge_pte() has the following prototype: pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, struct page *page, int writable); vma is used to get the pages shift or size. vma is also used on Sparc to get vm_flags. page is not used. writable is not used. In order to use this function without a vma, and replace vma by shift and flags. Also remove the used parameters. Signed-off-by: Christophe Leroy Acked-by: Mike Kravetz --- arch/arm64/include/asm/hugetlb.h | 3 +-- arch/arm64/mm/hugetlbpage.c | 5 ++--- arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h | 5 ++--- arch/sparc/include/asm/pgtable_64.h | 3 +-- arch/sparc/mm/hugetlbpage.c | 6 ++---- include/linux/hugetlb.h | 4 ++-- mm/hugetlb.c | 6 ++++-- mm/migrate.c | 4 +++- 8 files changed, 17 insertions(+), 19 deletions(-) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 5abf91e3494c..1242f71937f8 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -23,8 +23,7 @@ static inline void arch_clear_hugepage_flags(struct page *page) } #define arch_clear_hugepage_flags arch_clear_hugepage_flags -extern pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, - struct page *page, int writable); +pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT extern void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 58987a98e179..23505fc35324 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -339,10 +339,9 @@ pte_t *huge_pte_offset(struct mm_struct *mm, return NULL; } -pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, - struct page *page, int writable) +pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) { - size_t pagesize = huge_page_size(hstate_vma(vma)); + size_t pagesize = 1UL << shift; if (pagesize == CONT_PTE_SIZE) { entry = pte_mkcont(entry); diff --git a/arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h b/arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h index 39be9aea86db..64b6c608eca4 100644 --- a/arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h +++ b/arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h @@ -66,10 +66,9 @@ static inline void huge_ptep_set_wrprotect(struct mm_struct *mm, } #ifdef CONFIG_PPC_4K_PAGES -static inline pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, - struct page *page, int writable) +static inline pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) { - size_t size = huge_page_size(hstate_vma(vma)); + size_t size = 1UL << shift; if (size == SZ_16K) return __pte(pte_val(entry) & ~_PAGE_HUGE); diff --git a/arch/sparc/include/asm/pgtable_64.h b/arch/sparc/include/asm/pgtable_64.h index 550d3904de65..2cd80a0a9795 100644 --- a/arch/sparc/include/asm/pgtable_64.h +++ b/arch/sparc/include/asm/pgtable_64.h @@ -377,8 +377,7 @@ static inline pgprot_t pgprot_noncached(pgprot_t prot) #define pgprot_noncached pgprot_noncached #if defined(CONFIG_HUGETLB_PAGE) || defined(CONFIG_TRANSPARENT_HUGEPAGE) -extern pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, - struct page *page, int writable); +pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte static inline unsigned long __pte_default_huge_mask(void) { diff --git a/arch/sparc/mm/hugetlbpage.c b/arch/sparc/mm/hugetlbpage.c index 04d8790f6c32..0f49fada2093 100644 --- a/arch/sparc/mm/hugetlbpage.c +++ b/arch/sparc/mm/hugetlbpage.c @@ -177,10 +177,8 @@ static pte_t hugepage_shift_to_tte(pte_t entry, unsigned int shift) return sun4u_hugepage_shift_to_tte(entry, shift); } -pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, - struct page *page, int writeable) +pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) { - unsigned int shift = huge_page_shift(hstate_vma(vma)); pte_t pte; pte = hugepage_shift_to_tte(entry, shift); @@ -188,7 +186,7 @@ pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, #ifdef CONFIG_SPARC64 /* If this vma has ADI enabled on it, turn on TTE.mcd */ - if (vma->vm_flags & VM_SPARC_ADI) + if (flags & VM_SPARC_ADI) return pte_mkmcd(pte); else return pte_mknotmcd(pte); diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index b92f25ccef58..24f47981c166 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -710,8 +710,8 @@ static inline void arch_clear_hugepage_flags(struct page *page) { } #endif #ifndef arch_make_huge_pte -static inline pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, - struct page *page, int writable) +static inline pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, + vm_flags_t flags) { return entry; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 3db405dea3dc..396285b16dd8 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3856,6 +3856,7 @@ static pte_t make_huge_pte(struct vm_area_struct *vma, struct page *page, int writable) { pte_t entry; + unsigned int shift = huge_page_shift(hstate_vma(vma)); if (writable) { entry = huge_pte_mkwrite(huge_pte_mkdirty(mk_huge_pte(page, @@ -3866,7 +3867,7 @@ static pte_t make_huge_pte(struct vm_area_struct *vma, struct page *page, } entry = pte_mkyoung(entry); entry = pte_mkhuge(entry); - entry = arch_make_huge_pte(entry, vma, page, writable); + entry = arch_make_huge_pte(entry, shift, vma->vm_flags); return entry; } @@ -5250,10 +5251,11 @@ unsigned long hugetlb_change_protection(struct vm_area_struct *vma, } if (!huge_pte_none(pte)) { pte_t old_pte; + unsigned int shift = huge_page_shift(hstate_vma(vma)); old_pte = huge_ptep_modify_prot_start(vma, address, ptep); pte = pte_mkhuge(huge_pte_modify(old_pte, newprot)); - pte = arch_make_huge_pte(pte, vma, NULL, 0); + pte = arch_make_huge_pte(pte, shift, vma->vm_flags); huge_ptep_modify_prot_commit(vma, address, ptep, old_pte, pte); pages++; } diff --git a/mm/migrate.c b/mm/migrate.c index b234c3f3acb7..49ee64cd2ff3 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -226,8 +226,10 @@ static bool remove_migration_pte(struct page *page, struct vm_area_struct *vma, #ifdef CONFIG_HUGETLB_PAGE if (PageHuge(new)) { + unsigned int shift = huge_page_shift(hstate_vma(vma)); + pte = pte_mkhuge(pte); - pte = arch_make_huge_pte(pte, vma, new, 0); + pte = arch_make_huge_pte(pte, shift, vma->vm_flags); set_huge_pte_at(vma->vm_mm, pvmw.address, pvmw.pte, pte); if (PageAnon(new)) hugepage_add_anon_rmap(new, vma, pvmw.address); From patchwork Wed Apr 28 16:46:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 12229725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE4E9C433ED for ; Wed, 28 Apr 2021 16:48:08 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 96C6261400 for ; Wed, 28 Apr 2021 16:48:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 96C6261400 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=csgroup.eu Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:MIME-Version:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Date:Cc:To:Subject:From:References:In-Reply-To: Message-Id:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ILMUKAFQDCjhPRCYuiUf6c1zMzX5QwLIv63B09Ww8hM=; b=KH3DJ7tnwgUjZhyzYWlzzhkx5B ba3IltlFV+oqLAHjxLEoe+B5oNXku8qvq4PYZPx/7N6maRCHjZYRGk1HRChCK2kuL/JXXEiERYnmq V6qBq4RlvZN28B6gzdYBboPoBux2T/mYJy0N1zarIb/Kz+P3BCNACnT5nqec17sbfC/pXSg/hiK40 2s4c7LFUfATLw6nplKptdyC94sqQeCT1AMkF4hnldfSedpABL0/JFR7s0jW9Jf//nemjWw6g4zJzf 9jLLJDNUq7EmcqR58UuiFJZDEkOU0sYwkfBROtXRpLBPIoZnsD76d/Lz9rnE2Gq/5jn2Qieeo8Zmi CCLln+pg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lbnKR-003smI-Bp; Wed, 28 Apr 2021 16:46:23 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lbnKO-003sl4-Ck for linux-arm-kernel@desiato.infradead.org; Wed, 28 Apr 2021 16:46:20 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Date:Cc:To:Subject:From:References: In-Reply-To:Message-Id:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description; bh=3c/TtkM67c77ItVQ+SLXXad1Mr4YPF1Elee4oXHVo3w=; b=ol0dL0RIq8TiHnVloxFyN8d9Ba Gh+ccd0do7CkPd6QP3I0bZY5oNg+iZ1CcGTEzyk2HH6vujonJWqVZrd8p9o657CBodSaG/HzcVHVQ 9ys6Yf7JLnKtQmC0oEyJu5NO2swkQGUlG96OmhDBnWAWMTfiZOpMTMFMgNqyOg1REEqNGNcZyrFsr m+wBAPmyxGGL/4PBwElDIjyB3LNfd0V1r6gEnvcLxnS7fmsQjfbrU0pJl12dkpUS9y3+71iN3II9R eNEM5jmimUoJbvRWFypBATdPHlDAw318OZIxZ52aMJ0Yh57n3hYjA35ls1aeTPio+HYThMq8XzR+k FUCVp9gg==; Received: from pegase1.c-s.fr ([93.17.236.30]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lbnKK-0001tu-PT for linux-arm-kernel@lists.infradead.org; Wed, 28 Apr 2021 16:46:18 +0000 Received: from localhost (mailhub3.si.c-s.fr [192.168.12.233]) by localhost (Postfix) with ESMTP id 4FVkzV4qlxz9tcX; Wed, 28 Apr 2021 18:46:14 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rGxpY2iI8zAH; Wed, 28 Apr 2021 18:46:14 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 4FVkzV3vmDz9tcV; Wed, 28 Apr 2021 18:46:14 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 6F0128B837; Wed, 28 Apr 2021 18:46:14 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id 4PWl6TcHRv8j; Wed, 28 Apr 2021 18:46:14 +0200 (CEST) Received: from po15610vm.idsi0.si.c-s.fr (unknown [192.168.4.90]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 083158B831; Wed, 28 Apr 2021 18:46:14 +0200 (CEST) Received: by po15610vm.idsi0.si.c-s.fr (Postfix, from userid 0) id D8D336428C; Wed, 28 Apr 2021 16:46:13 +0000 (UTC) Message-Id: <6f482b6722f7a891ce335db7e02ecbdb42574e8b.1619628001.git.christophe.leroy@csgroup.eu> In-Reply-To: References: From: Christophe Leroy Subject: [RFC PATCH v1 3/4] mm/pgtable: Add stubs for {pmd/pub}_{set/clear}_huge To: Andrew Morton , Nicholas Piggin , Mike Kravetz , Mike Rapoport Cc: linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, sparclinux@vger.kernel.org, linux-mm@kvack.org Date: Wed, 28 Apr 2021 16:46:13 +0000 (UTC) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210428_094616_999680_8CE26440 X-CRM114-Status: UNSURE ( 8.16 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org For architectures with no PMD and/or no PUD, add stubs similar to what we have for architectures without P4D. Signed-off-by: Christophe Leroy --- include/linux/pgtable.h | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index 46b13780c2c8..d41474a2d255 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -1371,10 +1371,34 @@ static inline int p4d_clear_huge(p4d_t *p4d) } #endif /* !__PAGETABLE_P4D_FOLDED */ +#ifndef __PAGETABLE_PUD_FOLDED int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot); -int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot); int pud_clear_huge(pud_t *pud); +#else +static inline int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot) +{ + return 0; +} +static inline int pud_clear_huge(pud_t *pud) +{ + return 0; +} +#endif /* !__PAGETABLE_PUD_FOLDED */ + +#ifndef __PAGETABLE_PMD_FOLDED +int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot); int pmd_clear_huge(pmd_t *pmd); +#else +static inline int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot) +{ + return 0; +} +static inline int pmd_clear_huge(pmd_t *pmd) +{ + return 0; +} +#endif /* !__PAGETABLE_PMD_FOLDED */ + int p4d_free_pud_page(p4d_t *p4d, unsigned long addr); int pud_free_pmd_page(pud_t *pud, unsigned long addr); int pmd_free_pte_page(pmd_t *pmd, unsigned long addr); From patchwork Wed Apr 28 16:46:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Leroy X-Patchwork-Id: 12229731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB555C433B4 for ; Wed, 28 Apr 2021 16:49:01 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4290861400 for ; Wed, 28 Apr 2021 16:49:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4290861400 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=csgroup.eu Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:MIME-Version:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Date:Cc:To:Subject:From:References:In-Reply-To: Message-Id:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ZoilsuG7dDN+pzUcl7uvESLdX69MnJNyzK/ObVYuLr4=; b=SKojLg5T4xGf0e6vD5lKyCKOXS y38tuRI7dn0HF5gnKSuqrLMz6hfetDahlIFQKraIHSZ4yPtABN9ax7zxPY2uEFCV70i6DbA1DSpR2 TRMQxU3lPMMgDz9PCVkYEja5Fir1ix9i/zIa/EWOEVN6DAOgYMnDPSZ8CmQpYly4bh8jMOzTe/cbu aYFbjG1qIlsV6kl2DMRI1JGOiUXjx3Ql8O0wEPetNqoIWFBNBcUS2PITqtqL46xndixsNzRxTAexi 0rPhzzFZGQv39R+EhjL1jBo1OM1cOXlCwvSU+OK6qLxO3j7SjXzTSKmnRukkQAGjbK3xQHQYoMP2T 3IzDwGYw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lbnLK-003t0O-IH; Wed, 28 Apr 2021 16:47:18 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lbnKO-003slF-Et for linux-arm-kernel@desiato.infradead.org; Wed, 28 Apr 2021 16:46:20 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Date:Cc:To:Subject:From:References: In-Reply-To:Message-Id:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description; bh=ckf7TO5MthGHy60uDbbTZcbx90jtIg80ZtW/Jcmv+kA=; b=efcm68xQZMFUKtZQwAr9vHwm32 wgivxLTemO2rM17kA1JqLcuwfUSrnwpIRQaLhK8pL526y+/XQavzzp+FhQfXSyBa67nGOo0nDzbv7 AFAYe4G+kuIkp7+HHceD8WyWR65AbclTNNKaLhd+XQzkA9x4qrc8BACV71cH/JYXktZEcBQmGKpgU SlyCIQOYKW6mx251K95nYcqZP5kEalZ53nhrzRbM3wUbe5+2UpIgshM+TFMADmTGB4iIrCUgSgie8 +78zW0J16pFECYwxzDA+R20XstcF7ULvtFDvzF7FIZ1v9PXemzs7LJFSoJhydEWbBNd7+yFmsameo L1GDiqbA==; Received: from pegase1.c-s.fr ([93.17.236.30]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lbnKL-0001uQ-G7 for linux-arm-kernel@lists.infradead.org; Wed, 28 Apr 2021 16:46:19 +0000 Received: from localhost (mailhub3.si.c-s.fr [192.168.12.233]) by localhost (Postfix) with ESMTP id 4FVkzW5bBHz9tcd; Wed, 28 Apr 2021 18:46:15 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from pegase1.c-s.fr ([192.168.12.234]) by localhost (pegase1.c-s.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ignT_pQ-z5aE; Wed, 28 Apr 2021 18:46:15 +0200 (CEST) Received: from messagerie.si.c-s.fr (messagerie.si.c-s.fr [192.168.25.192]) by pegase1.c-s.fr (Postfix) with ESMTP id 4FVkzW4fqhz9tcV; Wed, 28 Apr 2021 18:46:15 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 555F78B837; Wed, 28 Apr 2021 18:46:15 +0200 (CEST) X-Virus-Scanned: amavisd-new at c-s.fr Received: from messagerie.si.c-s.fr ([127.0.0.1]) by localhost (messagerie.si.c-s.fr [127.0.0.1]) (amavisd-new, port 10023) with ESMTP id z1HHXWlZ7xmI; Wed, 28 Apr 2021 18:46:15 +0200 (CEST) Received: from po15610vm.idsi0.si.c-s.fr (unknown [192.168.4.90]) by messagerie.si.c-s.fr (Postfix) with ESMTP id 0AD2A8B831; Wed, 28 Apr 2021 18:46:15 +0200 (CEST) Received: by po15610vm.idsi0.si.c-s.fr (Postfix, from userid 0) id DF8946428C; Wed, 28 Apr 2021 16:46:14 +0000 (UTC) Message-Id: <15cea17065678a4e9019a20b7d011ca6eb205de6.1619628001.git.christophe.leroy@csgroup.eu> In-Reply-To: References: From: Christophe Leroy Subject: [RFC PATCH v1 4/4] mm/vmalloc: Add support for huge pages on VMAP and VMALLOC for powerpc 8xx To: Andrew Morton , Nicholas Piggin , Mike Kravetz , Mike Rapoport Cc: linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, sparclinux@vger.kernel.org, linux-mm@kvack.org Date: Wed, 28 Apr 2021 16:46:14 +0000 (UTC) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210428_094617_721246_5A1ED374 X-CRM114-Status: GOOD ( 16.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org powerpc 8xx has 4 page sizes: - 4k - 16k - 512k - 8M At the time being, vmalloc and vmap only support huge pages which are leaf at PMD level. Here the PMD level is 4M, it doesn't correspond to any supported page size. For the time being, implement use of 16k and 512k pages which is done at PTE level. Support of 8M pages will be implemented later, it requires use of hugepd tables. Signed-off-by: Christophe Leroy --- arch/powerpc/Kconfig | 3 +- mm/vmalloc.c | 74 ++++++++++++++++++++++++++++++++++++++------ 2 files changed, 66 insertions(+), 11 deletions(-) diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 27e88c38fdf7..b443716f7413 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -188,7 +188,8 @@ config PPC select GENERIC_TIME_VSYSCALL select GENERIC_VDSO_TIME_NS select HAVE_ARCH_AUDITSYSCALL - select HAVE_ARCH_HUGE_VMAP if PPC_BOOK3S_64 && PPC_RADIX_MMU + select HAVE_ARCH_HUGE_VMAP if (PPC_BOOK3S_64 && PPC_RADIX_MMU) || PPC_8xx + select HAVE_ARCH_HUGE_VMALLOC if PPC_8xx select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if PPC32 && PPC_PAGE_SHIFT <= 14 diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 5d96fee17226..1f9f9be8ec01 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -36,6 +36,7 @@ #include #include #include +#include #include #include @@ -81,12 +82,55 @@ static void free_work(struct work_struct *w) } /*** Page table manipulation functions ***/ +static int vmap_try_huge_pte(pte_t *ptep, unsigned long addr, unsigned long end, + u64 pfn, pgprot_t prot, unsigned int max_page_shift) +{ + unsigned long size = end - addr; + pte_t pte; + + if (!IS_ENABLED(CONFIG_PPC_8xx)) + return 0; + + if (PAGE_SIZE == SZ_16K && size < SZ_512K) + return 0; + + if (size < SZ_16K) + return 0; + + if (max_page_shift < 14) + return 0; + + if (size > SZ_512K) + size = SZ_512K; + + if (max_page_shift < 19 && size > SZ_16K) + size = SZ_16K; + + if (!IS_ALIGNED(addr, size)) + return 0; + + if (!IS_ALIGNED(PFN_PHYS(pfn), size)) + return 0; + + if (pte_present(*ptep)) + return 0; + + pte = pfn_pte(pfn, prot); + pte = pte_mkhuge(pte); + pte = arch_make_huge_pte(pte, ilog2(size), 0); + + set_huge_pte_at(&init_mm, addr, ptep, pte); + + return PFN_DOWN(size); +} + static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot, - pgtbl_mod_mask *mask) + unsigned int max_page_shift, pgtbl_mod_mask *mask) { pte_t *pte; u64 pfn; + int npages; pfn = phys_addr >> PAGE_SHIFT; pte = pte_alloc_kernel_track(pmd, addr, mask); @@ -94,9 +138,14 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, return -ENOMEM; do { BUG_ON(!pte_none(*pte)); - set_pte_at(&init_mm, addr, pte, pfn_pte(pfn, prot)); - pfn++; - } while (pte++, addr += PAGE_SIZE, addr != end); + + npages = vmap_try_huge_pte(pte, addr, end, pfn, prot, max_page_shift); + if (!npages) { + set_pte_at(&init_mm, addr, pte, pfn_pte(pfn, prot)); + npages = 1; + } + pfn += npages; + } while (pte += npages, addr += PAGE_SIZE * npages, addr != end); *mask |= PGTBL_PTE_MODIFIED; return 0; } @@ -145,7 +194,7 @@ static int vmap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, continue; } - if (vmap_pte_range(pmd, addr, next, phys_addr, prot, mask)) + if (vmap_pte_range(pmd, addr, next, phys_addr, prot, max_page_shift, mask)) return -ENOMEM; } while (pmd++, phys_addr += (next - addr), addr = next, addr != end); return 0; @@ -2881,8 +2930,7 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, return NULL; } - if (vmap_allow_huge && !(vm_flags & VM_NO_HUGE_VMAP) && - arch_vmap_pmd_supported(prot)) { + if (vmap_allow_huge && !(vm_flags & VM_NO_HUGE_VMAP)) { unsigned long size_per_node; /* @@ -2895,11 +2943,17 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, size_per_node = size; if (node == NUMA_NO_NODE) size_per_node /= num_online_nodes(); - if (size_per_node >= PMD_SIZE) { + if (arch_vmap_pmd_supported(prot) && size_per_node >= PMD_SIZE) { shift = PMD_SHIFT; - align = max(real_align, 1UL << shift); - size = ALIGN(real_size, 1UL << shift); + } else if (IS_ENABLED(CONFIG_PPC_8xx)) { + if (size_per_node >= SZ_512K) { + shift = 19; + } else if (size_per_node >= SZ_16K) { + shift = 14; + } } + align = max(real_align, 1UL << shift); + size = ALIGN(real_size, 1UL << shift); } again: