From patchwork Mon May 31 08:45:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12288913 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE37BC47082 for ; Mon, 31 May 2021 08:49:00 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AB0B361186 for ; Mon, 31 May 2021 08:49:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AB0B361186 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=3++co9pY2xWjjNWlKwB2SOC9wf5RCYfgxnwjh8fMikk=; b=NnunOvas1EUX4g VEeL2S8PatFarH+Jbcjk79QUqXhzD3sQUWGcNS4JvHeJO+jJqOH7BMQLq2l+ywAyAnUPL3GIaDf8D auztr1+mAX24YwpJEKpng4uT3ou88n3XEPN/zblU8vS2r2nju4AQ7JswLUpQNklAsX8bHtJHIz3n2 J8VavC5wqkvXWb7ugt8SLI1bOONHY90iChPMS/cjGo2Pv22ncTF8X3FYno894JoUHQHtDhhnNYkZO nEnsVQE/JPfRRjN8hrgaVUVs2a9Y/IQU8PhmkQwPRamzMLj6MSO0XatEaSzVggJ3uqTFDbj2qqyo3 AhSYjBfqno0J/MuP95jw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lndZg-00BWHP-3Y; Mon, 31 May 2021 08:47:04 +0000 Received: from mail-pj1-x102e.google.com ([2607:f8b0:4864:20::102e]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lndZF-00BWB7-AD for linux-arm-kernel@lists.infradead.org; Mon, 31 May 2021 08:46:38 +0000 Received: by mail-pj1-x102e.google.com with SMTP id g24so6204128pji.4 for ; Mon, 31 May 2021 01:46:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xxogjRX7faegKuuduNieefIAKpNTkkEuy15CEkfsdKM=; b=qNhAIjwqusqrhi7tyyZ4nB3kvTWecd+lMMq0uuszQ29RYh5qcLhWhPzQANHlkZAWMh /+nJ1jJVMd8mvFQfJdh8Q3Awg2USNcEXX+9DV/NHcGH6Yr/KcYTe47N5Ic01cbzb7jhs 9i6Ty38B/T35i6PHE2KS8bdKpXWXyh8n8+rVnbNaIxXbxYxOcUB7xJw1Y8HG3Kwle7Yi L6gL8If4MUmI23xHwjT07vH+IOUNkLsSo7JJ8Z0cceQjQu5bGJbsVURaBDiuj4sZ2Qn2 EfhS9teaCHG/aniDx6/SdyW8l4wJA8PRNXl+JGg9Zo8JF7OoZCTojjlyXbii4v6wDiKK oXRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xxogjRX7faegKuuduNieefIAKpNTkkEuy15CEkfsdKM=; b=AXUwNqk/kicNq/ClvPcEpBj22RFS9R1X32j/oMMlrdgbwbyfUbQbidgGrP/WCNgZZh KJNushQ/o8eeC8dUDvZj5qzbvdX2bLCLNNMRjbfKAZXmeQ4p3A26LHvjz1RmAUWDNfnV vo5aZjS3QMh/ujdP2PXCoKIKjT6ByEa/Vuik/nHJxd1x6m5F8S2Tc5RtOwJfspLoX3pu G5a+mr5ei/RFHEBa6KT4rDMr4x+WmtKxRmBkFFk7rwVW3WFMPbfDF5NIMn2HlotK82wy Gy98IjtN7nJ6sOMDcizNq0AhZLXosdi5Hgy8dEjWk8P6m1kDP3vLz4SUzPHf6P4mM0+Z lXqA== X-Gm-Message-State: AOAM532PK6dmFUdqUasaO9FWhQ6xLepjU1IQB41gfDes1t8vtjXM9nfJ wwnzTXZVeDTmJwmJMCCKxaLvqdZ/sja8mdI= X-Google-Smtp-Source: ABdhPJwxd9jVht6YICfWbYYQH4vDsde2c/msYUsIhHJMAMivejKu/fs1jCPT3zX3b8zOkIL1p4b6uA== X-Received: by 2002:a17:902:9685:b029:ef:70fd:a5a2 with SMTP id n5-20020a1709029685b02900ef70fda5a2mr19658561plp.20.1622450796714; Mon, 31 May 2021 01:46:36 -0700 (PDT) Received: from qualcomm-amberwing-rep-18.khw4.lab.eng.bos.redhat.com (nat-pool-bos-t.redhat.com. [66.187.233.206]) by smtp.gmail.com with ESMTPSA id fs24sm5125677pjb.6.2021.05.31.01.46.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 May 2021 01:46:36 -0700 (PDT) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org Cc: Pingfan Liu , Catalin Marinas , Will Deacon , Ard Biesheuvel , Marc Zyngier , Kristina Martsenko , James Morse , Steven Price , Jonathan Cameron , Pavel Tatashin , Anshuman Khandual , Atish Patra , Mike Rapoport , Logan Gunthorpe , Mark Brown Subject: [PATCHv3 4/5] arm64/mm: make __create_pgd_mapping() capable to handle pgtable's paddr Date: Mon, 31 May 2021 04:45:39 -0400 Message-Id: <20210531084540.78546-5-kernelfans@gmail.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20210531084540.78546-1-kernelfans@gmail.com> References: <20210531084540.78546-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210531_014637_392515_A01B2BB1 X-CRM114-Status: GOOD ( 14.63 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This patch is the last preparation for calling __create_pgd_mapping() from head.S. Under mmu-offset situation, __create_pgd_mapping() handles paddr of each pgtable level. All of pud_t */pmd_t */pte_t * points to paddr, and they should be handled carefully to avoid the involvement of __va(). Signed-off-by: Pingfan Liu Cc: Catalin Marinas Cc: Will Deacon Cc: Ard Biesheuvel Cc: Marc Zyngier Cc: Kristina Martsenko Cc: James Morse Cc: Steven Price Cc: Jonathan Cameron Cc: Pavel Tatashin Cc: Anshuman Khandual Cc: Atish Patra Cc: Mike Rapoport Cc: Logan Gunthorpe Cc: Mark Brown To: linux-arm-kernel@lists.infradead.org --- arch/arm64/mm/mmu.c | 39 ++++++++++++++++++++++++++++++++++----- 1 file changed, 34 insertions(+), 5 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index fe16f235d1fa..5f717552b524 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -189,6 +189,12 @@ static void init_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, pte_t *ptep; ptep = pte_set_fixmap_offset(pmdp, addr); + if (likely(!(flags & BOOT_HEAD))) + ptep = pte_set_fixmap_offset(pmdp, addr); + else + /* for head.S, there is no __va() */ + ptep = (pte_t *)__pmd_to_phys(*pmdp) + pte_index(addr); + do { pte_t old_pte = READ_ONCE(*ptep); @@ -204,7 +210,8 @@ static void init_pte(pmd_t *pmdp, unsigned long addr, unsigned long end, phys += PAGE_SIZE; } while (ptep++, addr += PAGE_SIZE, addr != end); - pte_clear_fixmap(); + if (likely(!(flags & BOOT_HEAD))) + pte_clear_fixmap(); } static void alloc_init_cont_pte(pmd_t *pmdp, unsigned long addr, @@ -253,7 +260,17 @@ static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end, unsigned long next; pmd_t *pmdp; - pmdp = pmd_set_fixmap_offset(pudp, addr); + if (likely(!(flags & BOOT_HEAD))) { + pmdp = pmd_set_fixmap_offset(pudp, addr); + } else { +#if CONFIG_PGTABLE_LEVELS > 2 + /* for head.S, there is no __va() */ + pmdp = (pmd_t *)__pud_to_phys(*pudp) + pmd_index(addr); +#else + pmdp = (pmd_t *)pudp; +#endif + } + do { pmd_t old_pmd = READ_ONCE(*pmdp); @@ -280,7 +297,8 @@ static void init_pmd(pud_t *pudp, unsigned long addr, unsigned long end, phys += next - addr; } while (pmdp++, addr = next, addr != end); - pmd_clear_fixmap(); + if (likely(!(flags & BOOT_HEAD))) + pmd_clear_fixmap(); } static void alloc_init_cont_pmd(pud_t *pudp, unsigned long addr, @@ -359,7 +377,17 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, } SAFE_BUG_ON(flags, p4d_bad(p4d)); - pudp = pud_set_fixmap_offset(p4dp, addr); + if (likely(!(flags & BOOT_HEAD))) { + pudp = pud_set_fixmap_offset(p4dp, addr); + } else { +#if CONFIG_PGTABLE_LEVELS > 3 + /* for head.S, there is no __va() */ + pudp = (pud_t *)__p4d_to_phys(*p4dp) + pud_index(addr); +#else + pudp = (pud_t *)p4dp; +#endif + } + do { pud_t old_pud = READ_ONCE(*pudp); @@ -388,7 +416,8 @@ static void alloc_init_pud(pgd_t *pgdp, unsigned long addr, unsigned long end, phys += next - addr; } while (pudp++, addr = next, addr != end); - pud_clear_fixmap(); + if (likely(!(flags & BOOT_HEAD))) + pud_clear_fixmap(); } static void __create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys,