From patchwork Sat Apr 10 09:56:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12195551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03316C433ED for ; Sat, 10 Apr 2021 09:59:52 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 76181610A2 for ; Sat, 10 Apr 2021 09:59:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 76181610A2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=C8hFvXZq5b+JKuKKC8JmzA4XyZC9FApKhIAHtyNphGQ=; b=KrrCbhdeVVbsp8hjY8DyxLakt hUbHpAxLt5r0QUegaXMbwQ1eCDeqfRwbznIO6qWeymPIQJwZXORchmHw8U4UDRcJjjBFLuktpVQgv RYTDkAhOQCx+jCwjFBE/0+c9wwMcjr7RHqwMWK9QFWG/mlLflxNHcspxss/qfSPehOfwvXoFPcdTR 2pqCXrLxIPpbcx/cH7kaCyUDlaZcAmcapRoiRqwbtwvE8uJzCL81cT7mDnTKzUeD9gM3C6Ty0o2fi IBRdmtRPZj1QvvAyRED5kiiyzB7ptLMzldMeqW9EtQ9GxIsPF9z2UYZLDoKKXcHskGdAkuslou/Na icC6YKiiA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lVANP-002bVJ-Er; Sat, 10 Apr 2021 09:58:03 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lVAMr-002bPb-8f for linux-arm-kernel@desiato.infradead.org; Sat, 10 Apr 2021 09:57:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=KI1OOnbo35577kxAOfgRYQQwPrpJtshmmXLoljeLfjU=; b=CnHjsDKVhxVdshNTrBKDACyzSV MmvtkEaDGWii7itdWFYYnBGfYFntDqtvQWr6C5iUtixaIHvpcL6KvNorpF/O+zVxdg6a5xQIQc5SP 0vc3loTCThK8Husd3R5rF7F2LRo3TlaiBM6T6Ui0NsSfSGIweOYshWvKNkgR3/vfZqgbDk9qC6Tdq Xrz0+gixd8kZX0uCKMbsYLRIAT87rLAwdbn9LX7zDsSf+ArJwzK5fxHlpoxRzeh4mQ0bapfiYkO30 MDIuOKe+V7d+ZbUa0G+RLvaKJ/Cd+dQ3XLiV4I6201iX6Kw4q6B0ys37Lr6tFXZx07wqOgNrUf6m9 eTQzGwxA==; Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lVAMo-0051ZE-5N for linux-arm-kernel@lists.infradead.org; Sat, 10 Apr 2021 09:57:28 +0000 Received: by mail-pg1-x531.google.com with SMTP id z16so5680709pga.1 for ; Sat, 10 Apr 2021 02:57:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KI1OOnbo35577kxAOfgRYQQwPrpJtshmmXLoljeLfjU=; b=gUF8s1Auju0vLdkI1fxNI9ZwtajfwOv0TknHMqhjkTfsCdcGUXjRES1gxAc6q6/H/7 Q5YgnH7Qfvmp3oIeTz00lk6BetDnyMUYC+042aJz2DMDO9DzpIBLdqSxunNBjL2o6MBb mQv4hzE3MxiC3yypxTgr5X1bpg+jz10cDbpjLyULfLSVOIFLO64QNN22PCV8xgoie8qV maN6rJTAx1zTKTo4YsTBxpS2SnJsw2A6hFXguCJvo472INT2h5AtdoRfbnljojdW4vQB aaRgBbi+rBCSYN5VHxROxddrJ8w0bNRIZAoT5MHJSoWyV6A3s64NovG/3fKpml/JFCHy nSmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KI1OOnbo35577kxAOfgRYQQwPrpJtshmmXLoljeLfjU=; b=dTs5bpNAJUTyiZP8J4WBtKfC5TOkUJLx57qeYE8FxuXyD2UhOKLhDiq9yYDlXgO1ig uDuZuWFiYObBXdx4HrGbtYXMK+cexJqpsGvXx4b87Ym71CQn7Ycy1fPQn+1CMi5e07jN lP1qrO7uhw7jP25CRvG9QiR4Z7qluYnhQyVs+CqDGmQyFYBrWtf5LS/2ZGGhFxEvopu4 LQLeqwnFAfK9H5GiJhCCZmjTOoTAeruQb7ZD87E6qEv0OCHZRHA+IWaN+GSk2AI1ihhA yjo5Y8yHdQGTS66Mmdt4Qdci2RP/YJ6TDpsi1dLQwZ5QnBk5lF3gGCb5PQMpjdvUqu3i Z1SA== X-Gm-Message-State: AOAM532W/XxrdFjGNs33bAlhxeHfGPFuWl4y6DKncKgQzUzMARFVRedV UdQ2B3zlZeDiLDzvNnR349gOxKMSx1xC X-Google-Smtp-Source: ABdhPJx13Gq1meu7XxZKN1DctJehlJT1d7ZId5rUJf75GpwhKSe0GcZXh4I+/olTkVX4542ZiX0ITQ== X-Received: by 2002:a05:6a00:24ca:b029:23e:e09:8ed0 with SMTP id d10-20020a056a0024cab029023e0e098ed0mr15932657pfv.20.1618048645194; Sat, 10 Apr 2021 02:57:25 -0700 (PDT) Received: from x1pad.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id m14sm3836322pfh.89.2021.04.10.02.57.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 10 Apr 2021 02:57:24 -0700 (PDT) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org Cc: Pingfan Liu , Catalin Marinas , Will Deacon , Marc Zyngier , Kristina Martsenko , James Morse , Steven Price , Jonathan Cameron , Pavel Tatashin , Anshuman Khandual , Atish Patra , Mike Rapoport , Logan Gunthorpe , Mark Brown Subject: [RFC 3/8] arm64/mm: change __create_pgd_mapping() prototype to accept extra info for allocator Date: Sat, 10 Apr 2021 17:56:49 +0800 Message-Id: <20210410095654.24102-4-kernelfans@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210410095654.24102-1-kernelfans@gmail.com> References: <20210410095654.24102-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210410_025726_232810_B3DEB401 X-CRM114-Status: GOOD ( 17.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Incoming allocator needs extra info to get the memory pool info. Signed-off-by: Pingfan Liu Cc: Catalin Marinas Cc: Will Deacon Cc: Marc Zyngier Cc: Kristina Martsenko Cc: James Morse Cc: Steven Price Cc: Jonathan Cameron Cc: Pavel Tatashin Cc: Anshuman Khandual Cc: Atish Patra Cc: Mike Rapoport Cc: Logan Gunthorpe Cc: Mark Brown To: linux-arm-kernel@lists.infradead.org --- arch/arm64/include/asm/pgalloc.h | 5 +++- arch/arm64/mm/idmap_mmu.c | 5 ++-- arch/arm64/mm/mmu.c | 31 +++++++++++++------------ arch/arm64/mm/mmu_include.c | 39 +++++++++++++++++++------------- 4 files changed, 46 insertions(+), 34 deletions(-) diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h index 555792921af0..42f602528b90 100644 --- a/arch/arm64/include/asm/pgalloc.h +++ b/arch/arm64/include/asm/pgalloc.h @@ -83,11 +83,14 @@ pmd_populate(struct mm_struct *mm, pmd_t *pmdp, pgtable_t ptep) } #define pmd_pgtable(pmd) pmd_page(pmd) +typedef phys_addr_t (*pgtable_alloc)(unsigned long shift, void *data); + extern void __create_pgd_mapping_extend(pgd_t *pgdir, unsigned int entries_cnt, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + pgtable_alloc allocator, + void *info, int flags); #endif diff --git a/arch/arm64/mm/idmap_mmu.c b/arch/arm64/mm/idmap_mmu.c index 7e9a4f4017d3..9d9fb77ce0e9 100644 --- a/arch/arm64/mm/idmap_mmu.c +++ b/arch/arm64/mm/idmap_mmu.c @@ -35,10 +35,11 @@ void __create_pgd_mapping_extend(pgd_t *pgdir, unsigned int entries_cnt, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + pgtable_alloc allocator, + void *info, int flags) { - __create_pgd_mapping(pgdir, entries_cnt, phys, virt, size, prot, pgtable_alloc, flags); + __create_pgd_mapping(pgdir, entries_cnt, phys, virt, size, prot, allocator, info, flags); } #endif diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 30afd6ed275f..0f183aaf98c9 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -86,7 +86,7 @@ pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, } EXPORT_SYMBOL(phys_mem_access_prot); -static phys_addr_t __init early_pgtable_alloc(int shift) +static phys_addr_t __init early_pgtable_alloc(unsigned long unused_a, void *unused_b) { phys_addr_t phys; void *ptr; @@ -113,7 +113,7 @@ static phys_addr_t __init early_pgtable_alloc(int shift) return phys; } -static phys_addr_t __pgd_pgtable_alloc(int shift) +static phys_addr_t __pgd_pgtable_alloc(unsigned long unused_a, void *unused_b) { void *ptr = (void *)__get_free_page(GFP_PGTABLE_KERNEL); BUG_ON(!ptr); @@ -123,9 +123,9 @@ static phys_addr_t __pgd_pgtable_alloc(int shift) return __pa(ptr); } -static phys_addr_t pgd_pgtable_alloc(int shift) +static phys_addr_t pgd_pgtable_alloc(unsigned long shift, void *unused) { - phys_addr_t pa = __pgd_pgtable_alloc(shift); + phys_addr_t pa = __pgd_pgtable_alloc(shift, unused); /* * Call proper page table ctor in case later we need to @@ -154,7 +154,8 @@ int idmap_extend_pgtable; void create_idmap(pgd_t *pgdir, phys_addr_t phys, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + pgtable_alloc allocator, + void *info, int flags) { u64 ptrs_per_pgd = idmap_ptrs_per_pgd; @@ -162,13 +163,13 @@ void create_idmap(pgd_t *pgdir, phys_addr_t phys, #if CONFIG_IDMAP_PGTABLE_EXPAND if (idmap_extend_pgtable) __create_pgd_mapping_extend(pgdir, ptrs_per_pgd, - phys, phys, size, prot, pgtable_alloc, flags); + phys, phys, size, prot, allocator, info, flags); else __create_pgd_mapping(pgdir, ptrs_per_pgd, - phys, phys, size, prot, pgtable_alloc, flags); + phys, phys, size, prot, allocator, info, flags); #else __create_pgd_mapping(pgdir, ptrs_per_pgd, - phys, phys, size, prot, pgtable_alloc, flags); + phys, phys, size, prot, allocator, info, flags); #endif } @@ -186,7 +187,7 @@ static void __init create_mapping_noalloc(phys_addr_t phys, unsigned long virt, return; } __create_pgd_mapping(init_mm.pgd, PTRS_PER_PGD, phys, virt, size, prot, NULL, - NO_CONT_MAPPINGS); + NULL, NO_CONT_MAPPINGS); } void __init create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, @@ -201,7 +202,7 @@ void __init create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; __create_pgd_mapping(mm->pgd, PTRS_PER_PGD, phys, virt, size, prot, - pgd_pgtable_alloc, flags); + pgd_pgtable_alloc, NULL, flags); } static void update_mapping_prot(phys_addr_t phys, unsigned long virt, @@ -214,7 +215,7 @@ static void update_mapping_prot(phys_addr_t phys, unsigned long virt, } __create_pgd_mapping(init_mm.pgd, PTRS_PER_PGD, phys, virt, size, prot, NULL, - NO_CONT_MAPPINGS); + NULL, NO_CONT_MAPPINGS); /* flush the TLBs after updating live kernel mappings */ flush_tlb_kernel_range(virt, virt + size); @@ -224,7 +225,7 @@ static void __init __map_memblock(pgd_t *pgdp, phys_addr_t start, phys_addr_t end, pgprot_t prot, int flags) { __create_pgd_mapping(pgdp, start, PTRS_PER_PGD, __phys_to_virt(start), end - start, - prot, early_pgtable_alloc, flags); + prot, early_pgtable_alloc, NULL, flags); } void __init mark_linear_text_alias_ro(void) @@ -325,7 +326,7 @@ static void __init map_kernel_segment(pgd_t *pgdp, void *va_start, void *va_end, BUG_ON(!PAGE_ALIGNED(size)); __create_pgd_mapping(pgdp, PTRS_PER_PGD, pa_start, (unsigned long)va_start, size, prot, - early_pgtable_alloc, flags); + early_pgtable_alloc, NULL, flags); if (!(vm_flags & VM_NO_GUARD)) size += PAGE_SIZE; @@ -369,7 +370,7 @@ static int __init map_entry_trampoline(void) /* Map only the text into the trampoline page table */ memset(tramp_pg_dir, 0, PGD_SIZE); __create_pgd_mapping(tramp_pg_dir, PTRS_PER_PGD, pa_start, TRAMP_VALIAS, PAGE_SIZE, - prot, __pgd_pgtable_alloc, 0); + prot, __pgd_pgtable_alloc, NULL, 0); /* Map both the text and data into the kernel page table */ __set_fixmap(FIX_ENTRY_TRAMP_TEXT, pa_start, prot); @@ -1261,7 +1262,7 @@ int arch_add_memory(int nid, u64 start, u64 size, flags = NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; __create_pgd_mapping(swapper_pg_dir, PTRS_PER_PGD, start, __phys_to_virt(start), - size, params->pgprot, __pgd_pgtable_alloc, + size, params->pgprot, __pgd_pgtable_alloc, NULL, flags); memblock_clear_nomap(start, size); diff --git a/arch/arm64/mm/mmu_include.c b/arch/arm64/mm/mmu_include.c index 1cf5af7e2aeb..371afc7d4502 100644 --- a/arch/arm64/mm/mmu_include.c +++ b/arch/arm64/mm/mmu_include.c @@ -64,7 +64,8 @@ static void init_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + pgtable_alloc allocator, + void *info, int flags) { unsigned long next; @@ -73,8 +74,8 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, BUG_ON(pmd_sect(pmd)); if (pmd_none(pmd)) { phys_addr_t pte_phys; - BUG_ON(!pgtable_alloc); - pte_phys = pgtable_alloc(PAGE_SHIFT); + BUG_ON(!allocator); + pte_phys = allocator(PAGE_SHIFT, info); __pmd_populate(pmdp, pte_phys, PMD_TYPE_TABLE); pmd = READ_ONCE(*pmdp); } @@ -98,7 +99,9 @@ static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), int flags) + pgtable_alloc allocator, + void *info, + int flags) { unsigned long next; pmd_t *pmdp; @@ -122,7 +125,7 @@ static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end, READ_ONCE(pmd_val(*pmdp)))); } else { alloc_init_cont_pte(pmdp, addr, next, phys, prot, - pgtable_alloc, flags); + allocator, info, flags); BUG_ON(pmd_val(old_pmd) != 0 && pmd_val(old_pmd) != READ_ONCE(pmd_val(*pmdp))); @@ -136,7 +139,9 @@ static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end, static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), int flags) + pgtable_alloc allocator, + void *info, + int flags) { unsigned long next; pud_t pud = READ_ONCE(*pudp); @@ -147,8 +152,8 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, BUG_ON(pud_sect(pud)); if (pud_none(pud)) { phys_addr_t pmd_phys; - BUG_ON(!pgtable_alloc); - pmd_phys = pgtable_alloc(PMD_SHIFT); + BUG_ON(!allocator); + pmd_phys = allocator(PMD_SHIFT, info); __pud_populate(pudp, pmd_phys, PUD_TYPE_TABLE); pud = READ_ONCE(*pudp); } @@ -164,7 +169,7 @@ static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, (flags & NO_CONT_MAPPINGS) == 0) __prot = __pgprot(pgprot_val(prot) | PTE_CONT); - init_pmd(pudp, addr, next, phys, __prot, pgtable_alloc, flags); + init_pmd(pudp, addr, next, phys, __prot, allocator, info, flags); phys += next - addr; } while (addr = next, addr != end); @@ -184,7 +189,8 @@ static inline bool use_1G_block(unsigned long addr, unsigned long next, static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, phys_addr_t phys, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + pgtable_alloc allocator, + void *info, int flags) { unsigned long next; @@ -194,8 +200,8 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, if (p4d_none(p4d)) { phys_addr_t pud_phys; - BUG_ON(!pgtable_alloc); - pud_phys = pgtable_alloc(PUD_SHIFT); + BUG_ON(!allocator); + pud_phys = allocator(PUD_SHIFT, info); __p4d_populate(p4dp, pud_phys, PUD_TYPE_TABLE); p4d = READ_ONCE(*p4dp); } @@ -222,7 +228,7 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, READ_ONCE(pud_val(*pudp)))); } else { alloc_init_cont_pmd(pudp, addr, next, phys, prot, - pgtable_alloc, flags); + allocator, info, flags); BUG_ON(pud_val(old_pud) != 0 && pud_val(old_pud) != READ_ONCE(pud_val(*pudp))); @@ -236,7 +242,8 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, static void __create_pgd_mapping(pgd_t *pgdir, unsigned int entries_cnt, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, - phys_addr_t (*pgtable_alloc)(int), + pgtable_alloc allocator, + void *info, int flags) { unsigned long addr, end, next; @@ -261,8 +268,8 @@ static void __create_pgd_mapping(pgd_t *pgdir, unsigned int entries_cnt, phys_ad do { next = pgd_addr_end(addr, end); - alloc_init_pud(pgdp, addr, next, phys, prot, pgtable_alloc, - flags); + alloc_init_pud(pgdp, addr, next, phys, prot, allocator, + info, flags); phys += next - addr; } while (pgdp++, addr = next, addr != end); }