From patchwork Sat Apr 10 09:56:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12195553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E88CC433B4 for ; Sat, 10 Apr 2021 10:00:02 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5AFCA610A2 for ; Sat, 10 Apr 2021 10:00:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5AFCA610A2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=L0zyYUeQRi4j223GCr7vmlf2gD0btLKjdkGVgPVK3TI=; b=AhizcQR9+V8sK3/sblX58EF0U J4M2SlC2Qb8/5lgkWXbaCwiHq5WWOwcCDSO+9E7qv5IQKL9j4n/YnxhHwVPaeRBTmxCJ8TdDIhxbc EeuzHGmOngJw4p1jbCDg4f6h5f5cYJpnFKkbvXLXphnCtduvvDr8qU/Q/h7NLiPuZ+Ek4Cw3tHKQz ilDZz6fkPzoZvAK4meWZdDLfh3TooOCeCTIIcFSHAqMluFS+M5ZyjlLYCNIkUDB0PT0rIk35rWB3k 7wQ0615YCfnF9rtMVsFluNQT6xNOY3nPIbBO7s9o2rNlEsGxhXRAMUHK28Up9QHO32/bAu5TCl7MP BARulIh/g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lVANi-002bZu-Vx; Sat, 10 Apr 2021 09:58:23 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lVAMw-002bQu-6x for linux-arm-kernel@desiato.infradead.org; Sat, 10 Apr 2021 09:57:37 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=JDkzlZvoMVLGV8peQuSm7JE3RB4/MO3qP78CDl7IL/c=; b=p1D001DJrqQxDomX0mfhS5x8/7 stMX+L21yPDak7EJHh0SMCSmXsb2WC712ByJlZPZYMeaMlYViIBPRr+ZUf2oF/jVe1qBlNfzYLLWy XGOQCtVodVHB/lCwUPyc36mkXuuUvkDqiQjDt6hr0lLoOD4Lcp/iPcsa6hV7IFoZYMoUwpv5HDoMR OB9t8H+ONRsa9n6kwGxwmnVe2Ax8Mr4LTKQFiUW1WCP09T64HffQGbRpMd5Xh7p6IP4HHIhdPjNWk LzpylXf4fiK9oU/CNavg5geV00irwVi8vwDiCRA7rMAfU2h5upL1bpqRABOsedWxDWvqbPx6DfYfG QJiCeqbA==; Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lVAMt-0051ZX-FM for linux-arm-kernel@lists.infradead.org; Sat, 10 Apr 2021 09:57:33 +0000 Received: by mail-pl1-x629.google.com with SMTP id g10so3915540plt.8 for ; Sat, 10 Apr 2021 02:57:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JDkzlZvoMVLGV8peQuSm7JE3RB4/MO3qP78CDl7IL/c=; b=XrA0DyEWHBIP2L6R0yhONYUZYVjAbbX10OqRyaxKxj/GV4VhaU0tVE4m2s3ZccBgti Ab73jHB0laNe//6k/GxWnJ9m6tXPuiQFsChV9Mqym9GD3r64oDSDp6mfEAvmEwuuZ2ak K8kyP+/5gFf2tjOn16Joe9iLEqIUHqwSVkRh2yYPRX2xpxPbUguWcNsIN7J9/nJF5GMV oepXeH48jWXcobqnpCxlw8BZB9xvY9okev5VNBA9AApjtHqgFQVDp5am0BgIFbkeehkW IEr8YINFjdHvDGuYe9LmIDZLESLa8rF+ZyXtWji7QsXbQgsPGgJWzILEzYUsVPUTt2hX EnHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JDkzlZvoMVLGV8peQuSm7JE3RB4/MO3qP78CDl7IL/c=; b=QUwtuK6BZcdN4YhbeLRe7/eY7tAURrL6a+//VxkbDWNhln8gfjL/1pPWNgDZQq0OFu Qd0xapfIE8Tp15nG6J4bzEvkazt28EzjpTFjdxqglB2rheZTt5wVYpPpOi56gDpMPwW8 U1CjtlQSf4MlcrnhLSMfS7ODXu8VfNCj1sIGrr6OubUu23hYbEvRIZW5SzpaCez83joD Fdp0/KJO+5s0UdIM0CHPgIJHlhUY7rBLIifMfczW0318AoJBQOHfUc6b6HvUviM+YUTT RiZbI0aGIDJIvbZBapwrzplHS1TNFcEen7IMNdsEqu0j6hRc+WA/lUuDlFR9OKZ+MqAV nk8Q== X-Gm-Message-State: AOAM533TBfDYVClxBmB8RtxPkGjzliB9EpPRLIf2ytKcfgeYKPemziS5 +xSZ9Z47AlDHgLh2k979cBzNP44/CROI X-Google-Smtp-Source: ABdhPJwdCf6loTjnYxNSOtTu5FsK+3m0YsvnEW/TBL+wY/ti4tMsGH45VXUSuiMhwxOTmX7rUCBVkA== X-Received: by 2002:a17:90a:7e4:: with SMTP id m91mr17281497pjm.46.1618048650577; Sat, 10 Apr 2021 02:57:30 -0700 (PDT) Received: from x1pad.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id m14sm3836322pfh.89.2021.04.10.02.57.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 10 Apr 2021 02:57:30 -0700 (PDT) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org Cc: Pingfan Liu , Catalin Marinas , Will Deacon , Marc Zyngier , Kristina Martsenko , James Morse , Steven Price , Jonathan Cameron , Pavel Tatashin , Anshuman Khandual , Atish Patra , Mike Rapoport , Logan Gunthorpe , Mark Brown Subject: [RFC 4/8] arm64/mm: enable __create_pgd_mapping() to run across different pgtable Date: Sat, 10 Apr 2021 17:56:50 +0800 Message-Id: <20210410095654.24102-5-kernelfans@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210410095654.24102-1-kernelfans@gmail.com> References: <20210410095654.24102-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210410_025731_533991_408BBD87 X-CRM114-Status: GOOD ( 14.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org PUP/PMD/PTE fixmap can not be shared across different pgtable concurrently. Also change the return type from phys_addr_t to unsigned long, since allocator may return virtual address directly. Signed-off-by: Pingfan Liu Cc: Catalin Marinas Cc: Will Deacon Cc: Marc Zyngier Cc: Kristina Martsenko Cc: James Morse Cc: Steven Price Cc: Jonathan Cameron Cc: Pavel Tatashin Cc: Anshuman Khandual Cc: Atish Patra Cc: Mike Rapoport Cc: Logan Gunthorpe Cc: Mark Brown To: linux-arm-kernel@lists.infradead.org --- arch/arm64/include/asm/pgalloc.h | 6 +++++- arch/arm64/mm/mmu.c | 6 +++--- arch/arm64/mm/mmu_include.c | 31 ++++++++++++++++++++----------- 3 files changed, 28 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h index 42f602528b90..6e9f1e218300 100644 --- a/arch/arm64/include/asm/pgalloc.h +++ b/arch/arm64/include/asm/pgalloc.h @@ -83,7 +83,7 @@ pmd_populate(struct mm_struct *mm, pmd_t *pmdp, pgtable_t ptep) } #define pmd_pgtable(pmd) pmd_page(pmd) -typedef phys_addr_t (*pgtable_alloc)(unsigned long shift, void *data); +typedef unsigned long (*pgtable_alloc)(unsigned long shift, void *data); extern void __create_pgd_mapping_extend(pgd_t *pgdir, unsigned int entries_cnt, phys_addr_t phys, @@ -93,4 +93,8 @@ extern void __create_pgd_mapping_extend(pgd_t *pgdir, void *info, int flags); +#define NO_BLOCK_MAPPINGS BIT(0) +#define NO_CONT_MAPPINGS BIT(1) +#define NO_FIXMAP BIT(2) + #endif diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 0f183aaf98c9..628752c3cfd0 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -86,7 +86,7 @@ pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, } EXPORT_SYMBOL(phys_mem_access_prot); -static phys_addr_t __init early_pgtable_alloc(unsigned long unused_a, void *unused_b) +static unsigned long __init early_pgtable_alloc(unsigned long unused_a, void *unused_b) { phys_addr_t phys; void *ptr; @@ -113,7 +113,7 @@ static phys_addr_t __init early_pgtable_alloc(unsigned long unused_a, void *unus return phys; } -static phys_addr_t __pgd_pgtable_alloc(unsigned long unused_a, void *unused_b) +static unsigned long __pgd_pgtable_alloc(unsigned long unused_a, void *unused_b) { void *ptr = (void *)__get_free_page(GFP_PGTABLE_KERNEL); BUG_ON(!ptr); @@ -123,7 +123,7 @@ static phys_addr_t __pgd_pgtable_alloc(unsigned long unused_a, void *unused_b) return __pa(ptr); } -static phys_addr_t pgd_pgtable_alloc(unsigned long shift, void *unused) +static unsigned long pgd_pgtable_alloc(unsigned long shift, void *unused) { phys_addr_t pa = __pgd_pgtable_alloc(shift, unused); diff --git a/arch/arm64/mm/mmu_include.c b/arch/arm64/mm/mmu_include.c index 371afc7d4502..adad0f93cd53 100644 --- a/arch/arm64/mm/mmu_include.c +++ b/arch/arm64/mm/mmu_include.c @@ -1,8 +1,5 @@ // SPDX-License-Identifier: GPL-2.0-only -#define NO_BLOCK_MAPPINGS BIT(0) -#define NO_CONT_MAPPINGS BIT(1) - static bool pgattr_change_is_safe(u64 old, u64 new) { /* @@ -38,11 +35,14 @@ static bool pgattr_change_is_safe(u64 old, u64 new) } static void init_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, - phys_addr_t phys, pgprot_t prot) + phys_addr_t phys, pgprot_t prot, int flags) { pte_t *ptep; - ptep = pte_set_fixmap_offset(pmdp, addr); + if (likely(!(flags & NO_FIXMAP))) + ptep = pte_set_fixmap_offset(pmdp, addr); + else + ptep = pte_offset_kernel(pmdp, addr); do { pte_t old_pte = READ_ONCE(*ptep); @@ -58,7 +58,8 @@ static void init_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, phys += PAGE_SIZE; } while (ptep++, addr += PAGE_SIZE, addr != end); - pte_clear_fixmap(); + if (likely(!(flags & NO_FIXMAP))) + pte_clear_fixmap(); } static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, @@ -91,7 +92,7 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, (flags & NO_CONT_MAPPINGS) == 0) __prot = __pgprot(pgprot_val(prot) | PTE_CONT); - init_pte(pmdp, addr, next, phys, __prot); + init_pte(pmdp, addr, next, phys, __prot, flags); phys += next - addr; } while (addr = next, addr != end); @@ -106,7 +107,10 @@ static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end, unsigned long next; pmd_t *pmdp; - pmdp = pmd_set_fixmap_offset(pudp, addr); + if (likely(!(flags & NO_FIXMAP))) + pmdp = pmd_set_fixmap_offset(pudp, addr); + else + pmdp = pmd_offset(pudp, addr); do { pmd_t old_pmd = READ_ONCE(*pmdp); @@ -133,7 +137,8 @@ static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end, phys += next - addr; } while (pmdp++, addr = next, addr != end); - pmd_clear_fixmap(); + if (likely(!(flags & NO_FIXMAP))) + pmd_clear_fixmap(); } static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, @@ -207,7 +212,10 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, } BUG_ON(p4d_bad(p4d)); - pudp = pud_set_fixmap_offset(p4dp, addr); + if (likely(!(flags & NO_FIXMAP))) + pudp = pud_set_fixmap_offset(p4dp, addr); + else + pudp = pud_offset(p4dp, addr); do { pud_t old_pud = READ_ONCE(*pudp); @@ -236,7 +244,8 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, phys += next - addr; } while (pudp++, addr = next, addr != end); - pud_clear_fixmap(); + if (likely(!(flags & NO_FIXMAP))) + pud_clear_fixmap(); } static void __create_pgd_mapping(pgd_t *pgdir, unsigned int entries_cnt, phys_addr_t phys,