From patchwork Mon Oct 14 10:58:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834714 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 277BED1A43B for ; Mon, 14 Oct 2024 11:01:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4252B6B00DA; Mon, 14 Oct 2024 07:01:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 30D736B00DC; Mon, 14 Oct 2024 07:01:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1AF996B00DD; Mon, 14 Oct 2024 07:01:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id EC1766B00DA for ; Mon, 14 Oct 2024 07:01:50 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5A92D160D40 for ; Mon, 14 Oct 2024 11:01:42 +0000 (UTC) X-FDA: 82671917370.13.D3565FF Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf13.hostedemail.com (Postfix) with ESMTP id A51D320003 for ; Mon, 14 Oct 2024 11:01:42 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903637; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Pem7/e2IY8K92Dd4GqzgmCKk29IF+hVKC5zPWZhX1eg=; b=XFVPCjj/4CrU6zui5wfQ5CaGdXwKF6NdPljkq/zi7KmUmSnjkG33kWm0AwuVTH6uVBaRNy brSOzCGz2Nptut77BgKXqazVKU8zNpKadwVBWOFxyRjTJ8LR/xjbXORzJG13DJ78RgVTUj b+HPGfiBaIVeUCWvhdrJ2vvG7Ct8XlI= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903637; a=rsa-sha256; cv=none; b=hnU/PYU8xPbW8RLU5SfRXeG/ZN4UPWkguM33xAod7EVMiom4TI57mYfVpnrLorp6lz0vav 5p+gaQt3voL9eNZcBgQ52TWqe1dXwi76zwVARf/ufTMb2TXNjWshOL4QSvbCYbW/CEW7su mPWeaDCgAJFmCHJ85cTQobjkpaOYAuY= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0E2C6176A; Mon, 14 Oct 2024 04:02:18 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0FACB3F51B; Mon, 14 Oct 2024 04:01:45 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 46/57] arm64: Generalize fixmap for boot-time page size Date: Mon, 14 Oct 2024 11:58:53 +0100 Message-ID: <20241014105912.3207374-46-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: A51D320003 X-Stat-Signature: oqehapmzwqq9hdgqec5tp6yp3ez6cxfh X-HE-Tag: 1728903702-38123 X-HE-Meta: U2FsdGVkX19i7S2Fz0g0n2OvXaxdCm4hSRbKaBqz8SxJuvxs6ZW0KDs0AihbaVzn2d5gPYU4FLpERz1o//Vn2pfR/KZRXri4wMkuwI7ViyKClQXA+RpFSH2MQNoWYrROf3HGfq4skGCFqqDgs+kCgUPjoHzm1hftq7SxJ4L15Yd1uj+lEsCrGsPbdfhmS7fwsKrVfACBfct4+03JlQ7YcxRK6y8rERgCesqs3SE0bWdQih6K5thpQ/XT79LeJVchKhX/qUAjhL/hV4pkD8oVuVSmH+EPoVbkudVHAJrRA/MvNiyBEy49cblk7nu3I5V00ngwhJckws2q0HEan6n2748Y6qIAwC/a77enXE+8eYGGSav9AJFjJBceUHoTKo66kn6aCUv/Tr0syoxpYN7uK7RtALaYpar/B1mzgE52HdqYTj1HaHw5hf8T/jmM3z85sZqmM8uNPzu6y4MnsoFWRy4XFkwNPBe2w2gcEdRfIereJQWZWfMCwQTnesVuf3DFXr8LxbB3sHyJxmB61YnSuWFTi9P8aii50qAwHl2MInrSLDeBp2a5A+iUqjk4xUZTK2t6vxDRHs/HU1yKLBSx9DwKFjG0agMGlBcZT5yWm4HOeV0dJfaGmJfvCc+3Pwy8rzaqibCg77GmYGBi7Wq08QvmWQnjlVFTawxWisCe0GoPCJhjXB/7D3/6vVcPHdWZOjdL4S6ytVjLMbwYc/Q3dYtOBTZfQKrX3hmXLHGpr+ILdbyXQRVvIE3EhfWQnexIuzR5nYSQi2PIktiTSXbZfWraGXeFR2HB63vrWJ/wf8l5A5C1v3sWm9uQcjZNnXIWXL387ZFM7/mIupRxmoh2ddYzGekfmcK4gc4Iut8LxtX2tBsPp9/vuEey3/eTpkqX7q67+GweiRmwJJ2z95MqnS8vwkjFASSsL0iJslWx8GEMMlNT9bnIl2i1SrrLqYj7FEh3WJugTPvbYY/kMgb 0XFCq+rt 8J/NPMmer5aKMLF8JtDS4lsXAPuYGOxlhLkkyAMiXTwdNrmmznYno7UtJapi/QmeWmTuRSZbc0lTA3BFmf8/tV7mZ4uTn6k82IFCdBX3QJdxiLRP9TEzlseg2rJssUrtxGVerLk/HTLBXUleNLsxMvexOeC9gIpPkQRK2FdWM5eIqq9hZGmCsg5Grng== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Some fixmap fixed address slots previously depended on PAGE_SIZE (i.e. to determine how many slots were required to cover a given size). Since we require the fixed address slots to be compile-time constant, let's work out the worst case number of required slots when page size is PAGE_SIZE_MIN instead. Additionally, let's determine the worst-case number of PTE tables we require and statically allocate enough memory. For compile-time page size builds, the end result is the same as it was previously. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/fixmap.h | 12 ++++++++---- arch/arm64/mm/fixmap.c | 34 ++++++++++++++++++++++----------- 2 files changed, 31 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixmap.h index 9a496d54dfe6e..c73fd3c1334ff 100644 --- a/arch/arm64/include/asm/fixmap.h +++ b/arch/arm64/include/asm/fixmap.h @@ -43,7 +43,7 @@ enum fixed_addresses { * whether it crosses any page boundary. */ FIX_FDT_END, - FIX_FDT = FIX_FDT_END + DIV_ROUND_UP(MAX_FDT_SIZE, PAGE_SIZE) + 1, + FIX_FDT = FIX_FDT_END + DIV_ROUND_UP(MAX_FDT_SIZE, PAGE_SIZE_MIN) + 1, FIX_EARLYCON_MEM_BASE, FIX_TEXT_POKE0, @@ -79,7 +79,7 @@ enum fixed_addresses { * Temporary boot-time mappings, used by early_ioremap(), * before ioremap() is functional. */ -#define NR_FIX_BTMAPS (SZ_256K / PAGE_SIZE) +#define NR_FIX_BTMAPS (SZ_256K / PAGE_SIZE_MIN) #define FIX_BTMAPS_SLOTS 7 #define TOTAL_FIX_BTMAPS (NR_FIX_BTMAPS * FIX_BTMAPS_SLOTS) @@ -101,8 +101,12 @@ enum fixed_addresses { #define FIXADDR_SIZE (__end_of_permanent_fixed_addresses << PAGE_SHIFT) #define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE) -#define FIXADDR_TOT_SIZE (__end_of_fixed_addresses << PAGE_SHIFT) -#define FIXADDR_TOT_START (FIXADDR_TOP - FIXADDR_TOT_SIZE) +#define __FIXADDR_TOT_SIZE(page_shift) \ + (__end_of_fixed_addresses << (page_shift)) +#define __FIXADDR_TOT_START(page_shift) \ + (FIXADDR_TOP - __FIXADDR_TOT_SIZE(page_shift)) +#define FIXADDR_TOT_SIZE __FIXADDR_TOT_SIZE(PAGE_SHIFT) +#define FIXADDR_TOT_START __FIXADDR_TOT_START(PAGE_SHIFT) #define FIXMAP_PAGE_IO __pgprot(PROT_DEVICE_nGnRE) diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index 15ce3253ad359..a0dcf2375ccb4 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -17,27 +17,39 @@ #include /* ensure that the fixmap region does not grow down into the PCI I/O region */ -static_assert(FIXADDR_TOT_START > PCI_IO_END); +static_assert(__FIXADDR_TOT_START(PAGE_SHIFT_MAX) > PCI_IO_END); -#define NR_BM_PTE_TABLES \ - SPAN_NR_ENTRIES(FIXADDR_TOT_START, FIXADDR_TOP, PMD_SHIFT) -#define NR_BM_PMD_TABLES \ - SPAN_NR_ENTRIES(FIXADDR_TOT_START, FIXADDR_TOP, PUD_SHIFT) +#define FIXMAP_LEVEL(page_shift, lvl, vstart, vend) \ + SPAN_NR_ENTRIES(vstart, vend, PGTABLE_LEVEL_SHIFT(page_shift, lvl)) -static_assert(NR_BM_PMD_TABLES == 1); +#define FIXMAP_PAGES(page_shift, level) \ + FIXMAP_LEVEL(page_shift, level, \ + __FIXADDR_TOT_START(page_shift), FIXADDR_TOP) + +#define FIXMAP_SIZE(page_shift, level) \ + (FIXMAP_PAGES(page_shift, level) * (UL(1) << (page_shift))) + +#define FIXMAP_PTE_SIZE_MAX \ + MAX_IF_HAVE_PGSZ(FIXMAP_SIZE(ARM64_PAGE_SHIFT_4K, 2), \ + FIXMAP_SIZE(ARM64_PAGE_SHIFT_16K, 2), \ + FIXMAP_SIZE(ARM64_PAGE_SHIFT_64K, 2)) + +static_assert(FIXMAP_PAGES(ARM64_PAGE_SHIFT_4K, 1) == 1); +static_assert(FIXMAP_PAGES(ARM64_PAGE_SHIFT_16K, 1) == 1); +static_assert(FIXMAP_PAGES(ARM64_PAGE_SHIFT_64K, 1) == 1); #define __BM_TABLE_IDX(addr, shift) \ (((addr) >> (shift)) - (FIXADDR_TOT_START >> (shift))) #define BM_PTE_TABLE_IDX(addr) __BM_TABLE_IDX(addr, PMD_SHIFT) -static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __page_aligned_bss; -static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss __maybe_unused; -static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss __maybe_unused; +static pte_t bm_pte[FIXMAP_PTE_SIZE_MAX / sizeof(pte_t)] __page_aligned_bss; +static pmd_t bm_pmd[MAX_PTRS_PER_PMD] __page_aligned_bss __maybe_unused; +static pud_t bm_pud[MAX_PTRS_PER_PUD] __page_aligned_bss __maybe_unused; static inline pte_t *fixmap_pte(unsigned long addr) { - return &bm_pte[BM_PTE_TABLE_IDX(addr)][pte_index(addr)]; + return &bm_pte[BM_PTE_TABLE_IDX(addr) * PTRS_PER_PTE + pte_index(addr)]; } static void __init early_fixmap_init_pte(pmd_t *pmdp, unsigned long addr) @@ -46,7 +58,7 @@ static void __init early_fixmap_init_pte(pmd_t *pmdp, unsigned long addr) pte_t *ptep; if (pmd_none(pmd)) { - ptep = bm_pte[BM_PTE_TABLE_IDX(addr)]; + ptep = &bm_pte[BM_PTE_TABLE_IDX(addr) * PTRS_PER_PTE]; __pmd_populate(pmdp, __pa_symbol(ptep), PMD_TYPE_TABLE); } }