From patchwork Mon Oct 14 10:58:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 888D8D1A443 for ; Mon, 14 Oct 2024 10:59:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 073B36B0085; Mon, 14 Oct 2024 06:59:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 024446B0088; Mon, 14 Oct 2024 06:59:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE0596B0089; Mon, 14 Oct 2024 06:59:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B960F6B0085 for ; Mon, 14 Oct 2024 06:59:32 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 0B3E5C0D50 for ; Mon, 14 Oct 2024 10:59:24 +0000 (UTC) X-FDA: 82671911616.16.6E03A75 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP id 87C7640011 for ; Mon, 14 Oct 2024 10:59:25 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903414; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/u/yhefR/UmecQ7GGPQ2XFbto27FbbkmaIyMhrqWbOE=; b=ZTZ8KO0buUIkMED/3FljgLWdNwvXa7D0NF6kXMdObNrSdtQ3m51c2ShAgBfqty/TxrKkd+ 04YnrNEVN01kA/X8U47DDCEELwc4lBT8gwOsauylC0AtFuOB0MubH9w+QdLYNOzB9g8mUv rBIxShhTGccdEzfIZi/uQY4UHJquaUI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903414; a=rsa-sha256; cv=none; b=7h5KP4zYa6MwZEpYn2rdhuXO+urRkq2Fk8+UwIeh/wwhFYWrZMjKUT3yqcbxtXcgkCBAuT 4cMc/ddF5ZhYnSXHLrn3Ce1wxPaH7QBe1WCzVmCvH1NDwOG1aJqHhyW82My4knGIXNFvA0 sdaTDvkRzWB9Hp6yeE/wLx1ZQ1WMYg8= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EEC901007; Mon, 14 Oct 2024 03:59:58 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9E3473F51B; Mon, 14 Oct 2024 03:59:21 -0700 (PDT) From: Ryan Roberts To: "David S. Miller" , "James E.J. Bottomley" , Andreas Larsson , Andrew Morton , Anshuman Khandual , Anton Ivanov , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Chris Zankel , Dave Hansen , David Hildenbrand , Dinh Nguyen , Geert Uytterhoeven , Greg Marsden , Helge Deller , Huacai Chen , Ingo Molnar , Ivan Ivanov , Johannes Berg , John Paul Adrian Glaubitz , Jonas Bonn , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Max Filippov , Miroslav Benes , Rich Felker , Richard Weinberger , Stafford Horne , Stefan Kristiansson , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , Yoshinori Sato , x86@kernel.org Cc: Ryan Roberts , linux-alpha@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-kernel@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-um@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, sparclinux@vger.kernel.org Subject: [RFC PATCH v1 01/57] mm: Add macros ahead of supporting boot-time page size selection Date: Mon, 14 Oct 2024 11:58:08 +0100 Message-ID: <20241014105912.3207374-1-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105514.3206191-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 87C7640011 X-Stat-Signature: bibh4yqcoybonxit7isy5c9o8da86b69 X-Rspam-User: X-HE-Tag: 1728903565-674945 X-HE-Meta: U2FsdGVkX18tnwWI61uv7gEQR+al0zm2VpUL5E8ErWYVYJO5K3qpTto78VgkJnVdDwmTT4E1lM/5Y0Mb0deF/J8iBFNmIZ7wMcq2rrJI2cw4Uc6QK44lKKb9pekPJPEVe0h/Qm5VD0a954vbnSCGMQ630Ai9gqcI35Kwd/g7qpSKoUGAPfKrtIKNSQ7Zq5uXuBly5mmr0NFa4fSGexT5g3JZNk5U3QCfuUP/6xUno2tIoiM0+Mjv+yuvJ4Bjl8XdjfIbbLa/PZvleQm6vT4bLJAt9AToNX0tuvBSEiOHDUn4N3J0C9zzrwCHxEn3eOUC98sW/zHU1NE8dEGupRZavVA67KTJCK9FZUgqkB4Qf/ZcBHILJDnc30Wu4mT8FOuUpo3aHDUuA2Vbm44ZxDTSm5AqQriG7x7R8ToJw55nIx8XfzyHxRIvLNCNxHR/ZRBuBcKAMhl2iz7hI5atxohzM/UbjibakCNUp2ErKcLwi8Yq9SBqEfNf/WfF4gkNoiFE+8ehU4ZEQUnFBlSiOHznCQM/IyOxqTY2MREMsT+A2PNO60TQS3DflVm7SfvTNDXyA7rCLDBOqVm5Y72R7eBBpEfp765wmPCSAZXP2tQqW9twjBHlaLk2jcFF42P9Tgh+IFT8DXAIuG821hiPEeBB7yd7Twirl/h01jogoO4G6TghV9ARXLscQbtg5Tix6zXcrDx9wz9BizcRqzQWGGq55rXTUuDfo2Rvkz7Vj6LLDHrlPvwiGmGFw09/bqz2Nhp5+uDMsac19TeWe4iDKwd2FZeVHkxxykwwSQA+gm/X07J1iFf2tUj/ZnYVxARidRmeRCTNaejmLL8gC2xbocHPju9sZT3lTsktHxN3quV9fma6x7yvXqgUB2IBmZbcXDqaaqdApDVG6xUiEV0hgnzvXWzjXDWpi6+SMtPWCVuKaJNXiLCaXGFLtaWIS/jaRdfIx8PvPB1PyCYufXGwYkD bG7hBGxN s8OF8XFiWjjqB0eaP4Dcuwy8jL0QqKfnCCSabsfXSXwrPuTBU7Y640yS36Iwq6ZdL3INbFnVlUFi5zZr00WaxsUyyFkOGLCoWmlDQvzx5VQJ2cirDM9Y5WdmU+tBOpWdhwf15QIO52t25cYSndzj9/ZY7xArg26TDyenB2OROHbqGmSBleOJvV6IX/g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: arm64 can support multiple base page sizes. Instead of selecting a page size at compile time, as is done today, we will make it possible to select the desired page size on the command line. In this case PAGE_SHIFT and it's derivatives, PAGE_SIZE and PAGE_MASK (as well as a number of other macros related to or derived from PAGE_SHIFT, but I'm not worrying about those yet), are no longer compile-time constants. So the code base needs to cope with that. As a first step, introduce MIN and MAX variants of these macros, which express the range of possible page sizes. These are always compile-time constants and can be used in many places where PAGE_[SHIFT|SIZE|MASK] were previously used where a compile-time constant is required. (Subsequent patches will do that conversion work). When the arch/build doesn't support boot-time page size selection, the MIN and MAX variants are equal and everything resolves as it did previously. Additionally, introduce DEFINE_GLOBAL_PAGE_SIZE_VAR[_CONST]() which wrap global variable defintions so that for boot-time page size selection builds, the variable being wrapped is initialized at boot-time, instead of compile-time. This is done by defining a function to do the assignment, which has the "constructor" attribute. Constructor is preferred over initcall, because when compiling a module, the module is limited to a single initcall but constructors are unlimited. For built-in code, constructors are now called earlier to guarrantee that the variables are initialized by the time they are used. Any arch that wants to enable boot-time page size selection will need to select CONFIG_CONSTRUCTORS. These new macros need to be available anywhere PAGE_SHIFT and friends are available. Those are defined via asm/page.h (although some arches have a sub-include that defines them). Unfortunately there is no reliable asm-generic header we can easily piggy-back on, so let's define a new one, pgtable-geometry.h, which we include near where each arch defines PAGE_SHIFT. Ugh. ------- Most of the problems that need to be solved over the next few patches fall into these broad categories, which are all solved with the help of these new macros: 1. Assignment of values derived from PAGE_SIZE in global variables For boot-time page size builds, we must defer the initialization of these variables until boot-time, when the page size is known. See DEFINE_GLOBAL_PAGE_SIZE_VAR[_CONST]() as described above. 2. Define static storage in units related to PAGE_SIZE This static storage will be defined according to PAGE_SIZE_MAX. 3. Define size of struct so that it is related to PAGE_SIZE The struct often contains an array that is sized to fill the page. In this case, use a flexible array with dynamic allocation. In other cases, the struct fits exactly over a page, which is a header (e.g. swap file header). In this case, remove the padding, and manually determine the struct pointer within the page. 4. BUILD_BUG_ON() with values derived from PAGE_SIZE In most cases, we can change these to compare againt the appropriate limit (either MIN or MAX). In other cases, we must change these to run-time BUG_ON(). 5. Ensure page alignment of static data structures Align instead to PAGE_SIZE_MAX. 6. #ifdeffery based on PAGE_SIZE Often these can be changed to c code constructs. e.g. a macro that returns a different value depending on page size can be changed to use the ternary operator and the compiler will dead code strip it for the compile-time constant case and runtime evaluate it for the non-const case. Or #if/#else/#endif within a function can be converted to c if/else blocks, which are also dead code stripped for the const case. Sometimes we can change the c-preprocessor logic to use the appropriate MIN/MAX limit. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/alpha/include/asm/page.h | 1 + arch/arc/include/asm/page.h | 1 + arch/arm/include/asm/page.h | 1 + arch/arm64/include/asm/page-def.h | 2 + arch/csky/include/asm/page.h | 3 ++ arch/hexagon/include/asm/page.h | 2 + arch/loongarch/include/asm/page.h | 2 + arch/m68k/include/asm/page.h | 1 + arch/microblaze/include/asm/page.h | 1 + arch/mips/include/asm/page.h | 1 + arch/nios2/include/asm/page.h | 2 + arch/openrisc/include/asm/page.h | 1 + arch/parisc/include/asm/page.h | 1 + arch/powerpc/include/asm/page.h | 2 + arch/riscv/include/asm/page.h | 1 + arch/s390/include/asm/page.h | 1 + arch/sh/include/asm/page.h | 1 + arch/sparc/include/asm/page.h | 3 ++ arch/um/include/asm/page.h | 2 + arch/x86/include/asm/page_types.h | 2 + arch/xtensa/include/asm/page.h | 1 + include/asm-generic/pgtable-geometry.h | 71 ++++++++++++++++++++++++++ init/main.c | 5 +- 23 files changed, 107 insertions(+), 1 deletion(-) create mode 100644 include/asm-generic/pgtable-geometry.h diff --git a/arch/alpha/include/asm/page.h b/arch/alpha/include/asm/page.h index 70419e6be1a35..d0096fb5521b8 100644 --- a/arch/alpha/include/asm/page.h +++ b/arch/alpha/include/asm/page.h @@ -88,5 +88,6 @@ typedef struct page *pgtable_t; #include #include +#include #endif /* _ALPHA_PAGE_H */ diff --git a/arch/arc/include/asm/page.h b/arch/arc/include/asm/page.h index def0dfb95b436..8d56549db7a33 100644 --- a/arch/arc/include/asm/page.h +++ b/arch/arc/include/asm/page.h @@ -6,6 +6,7 @@ #define __ASM_ARC_PAGE_H #include +#include #ifdef CONFIG_ARC_HAS_PAE40 diff --git a/arch/arm/include/asm/page.h b/arch/arm/include/asm/page.h index 62af9f7f9e963..417aa8533c718 100644 --- a/arch/arm/include/asm/page.h +++ b/arch/arm/include/asm/page.h @@ -191,5 +191,6 @@ extern int pfn_valid(unsigned long); #include #include +#include #endif diff --git a/arch/arm64/include/asm/page-def.h b/arch/arm64/include/asm/page-def.h index 792e9fe881dcf..d69971cf49cd2 100644 --- a/arch/arm64/include/asm/page-def.h +++ b/arch/arm64/include/asm/page-def.h @@ -15,4 +15,6 @@ #define PAGE_SIZE (_AC(1, UL) << PAGE_SHIFT) #define PAGE_MASK (~(PAGE_SIZE-1)) +#include + #endif /* __ASM_PAGE_DEF_H */ diff --git a/arch/csky/include/asm/page.h b/arch/csky/include/asm/page.h index 0ca6c408c07f2..95173d57adc8b 100644 --- a/arch/csky/include/asm/page.h +++ b/arch/csky/include/asm/page.h @@ -92,4 +92,7 @@ static inline unsigned long virt_to_pfn(const void *kaddr) #include #endif /* !__ASSEMBLY__ */ + +#include + #endif /* __ASM_CSKY_PAGE_H */ diff --git a/arch/hexagon/include/asm/page.h b/arch/hexagon/include/asm/page.h index 8a6af57274c2d..ba7ad5231695f 100644 --- a/arch/hexagon/include/asm/page.h +++ b/arch/hexagon/include/asm/page.h @@ -139,4 +139,6 @@ static inline unsigned long virt_to_pfn(const void *kaddr) #endif /* ifdef __ASSEMBLY__ */ #endif /* ifdef __KERNEL__ */ +#include + #endif diff --git a/arch/loongarch/include/asm/page.h b/arch/loongarch/include/asm/page.h index e85df33f11c77..9862e8fb047a6 100644 --- a/arch/loongarch/include/asm/page.h +++ b/arch/loongarch/include/asm/page.h @@ -123,4 +123,6 @@ extern int __virt_addr_valid(volatile void *kaddr); #endif /* !__ASSEMBLY__ */ +#include + #endif /* _ASM_PAGE_H */ diff --git a/arch/m68k/include/asm/page.h b/arch/m68k/include/asm/page.h index 8cfb84b499751..4df4681b02194 100644 --- a/arch/m68k/include/asm/page.h +++ b/arch/m68k/include/asm/page.h @@ -60,5 +60,6 @@ extern unsigned long _ramend; #include #include +#include #endif /* _M68K_PAGE_H */ diff --git a/arch/microblaze/include/asm/page.h b/arch/microblaze/include/asm/page.h index 8810f4f1c3b02..abc23c3d743bd 100644 --- a/arch/microblaze/include/asm/page.h +++ b/arch/microblaze/include/asm/page.h @@ -142,5 +142,6 @@ static inline const void *pfn_to_virt(unsigned long pfn) #include #include +#include #endif /* _ASM_MICROBLAZE_PAGE_H */ diff --git a/arch/mips/include/asm/page.h b/arch/mips/include/asm/page.h index 4609cb0326cf3..3d91021538f02 100644 --- a/arch/mips/include/asm/page.h +++ b/arch/mips/include/asm/page.h @@ -227,5 +227,6 @@ static inline unsigned long kaslr_offset(void) #include #include +#include #endif /* _ASM_PAGE_H */ diff --git a/arch/nios2/include/asm/page.h b/arch/nios2/include/asm/page.h index 0722f88e63cc7..2e5f93beb42b7 100644 --- a/arch/nios2/include/asm/page.h +++ b/arch/nios2/include/asm/page.h @@ -97,4 +97,6 @@ extern struct page *mem_map; #endif /* !__ASSEMBLY__ */ +#include + #endif /* _ASM_NIOS2_PAGE_H */ diff --git a/arch/openrisc/include/asm/page.h b/arch/openrisc/include/asm/page.h index 1d5913f67c312..a0da2a9842241 100644 --- a/arch/openrisc/include/asm/page.h +++ b/arch/openrisc/include/asm/page.h @@ -88,5 +88,6 @@ static inline unsigned long virt_to_pfn(const void *kaddr) #include #include +#include #endif /* __ASM_OPENRISC_PAGE_H */ diff --git a/arch/parisc/include/asm/page.h b/arch/parisc/include/asm/page.h index 4bea2e95798f0..2a75496237c09 100644 --- a/arch/parisc/include/asm/page.h +++ b/arch/parisc/include/asm/page.h @@ -173,6 +173,7 @@ extern int npmem_ranges; #include #include +#include #include #define PAGE0 ((struct zeropage *)absolute_pointer(__PAGE_OFFSET)) diff --git a/arch/powerpc/include/asm/page.h b/arch/powerpc/include/asm/page.h index 83d0a4fc5f755..4601c115b6485 100644 --- a/arch/powerpc/include/asm/page.h +++ b/arch/powerpc/include/asm/page.h @@ -300,4 +300,6 @@ static inline unsigned long kaslr_offset(void) #include #endif /* __ASSEMBLY__ */ +#include + #endif /* _ASM_POWERPC_PAGE_H */ diff --git a/arch/riscv/include/asm/page.h b/arch/riscv/include/asm/page.h index 7ede2111c5917..e5af7579e45bf 100644 --- a/arch/riscv/include/asm/page.h +++ b/arch/riscv/include/asm/page.h @@ -204,5 +204,6 @@ static __always_inline void *pfn_to_kaddr(unsigned long pfn) #include #include +#include #endif /* _ASM_RISCV_PAGE_H */ diff --git a/arch/s390/include/asm/page.h b/arch/s390/include/asm/page.h index 16e4caa931f1f..42157e7690a77 100644 --- a/arch/s390/include/asm/page.h +++ b/arch/s390/include/asm/page.h @@ -275,6 +275,7 @@ static inline unsigned long virt_to_pfn(const void *kaddr) #include #include +#include #define AMODE31_SIZE (3 * PAGE_SIZE) diff --git a/arch/sh/include/asm/page.h b/arch/sh/include/asm/page.h index f780b467e75d7..09533d46ef033 100644 --- a/arch/sh/include/asm/page.h +++ b/arch/sh/include/asm/page.h @@ -162,5 +162,6 @@ typedef struct page *pgtable_t; #include #include +#include #endif /* __ASM_SH_PAGE_H */ diff --git a/arch/sparc/include/asm/page.h b/arch/sparc/include/asm/page.h index 5e44cdf2a8f2b..4327fe2bfa010 100644 --- a/arch/sparc/include/asm/page.h +++ b/arch/sparc/include/asm/page.h @@ -9,4 +9,7 @@ #else #include #endif + +#include + #endif diff --git a/arch/um/include/asm/page.h b/arch/um/include/asm/page.h index 9ef9a8aedfa66..f26011808f514 100644 --- a/arch/um/include/asm/page.h +++ b/arch/um/include/asm/page.h @@ -119,4 +119,6 @@ extern unsigned long uml_physmem; #define __HAVE_ARCH_GATE_AREA 1 #endif +#include + #endif /* __UM_PAGE_H */ diff --git a/arch/x86/include/asm/page_types.h b/arch/x86/include/asm/page_types.h index 52f1b4ff0cc16..6d2381342047f 100644 --- a/arch/x86/include/asm/page_types.h +++ b/arch/x86/include/asm/page_types.h @@ -71,4 +71,6 @@ extern void initmem_init(void); #endif /* !__ASSEMBLY__ */ +#include + #endif /* _ASM_X86_PAGE_DEFS_H */ diff --git a/arch/xtensa/include/asm/page.h b/arch/xtensa/include/asm/page.h index 4db56ef052d22..86952cb32af23 100644 --- a/arch/xtensa/include/asm/page.h +++ b/arch/xtensa/include/asm/page.h @@ -200,4 +200,5 @@ static inline unsigned long ___pa(unsigned long va) #endif /* __ASSEMBLY__ */ #include +#include #endif /* _XTENSA_PAGE_H */ diff --git a/include/asm-generic/pgtable-geometry.h b/include/asm-generic/pgtable-geometry.h new file mode 100644 index 0000000000000..358e729a6ac37 --- /dev/null +++ b/include/asm-generic/pgtable-geometry.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef ASM_GENERIC_PGTABLE_GEOMETRY_H +#define ASM_GENERIC_PGTABLE_GEOMETRY_H + +#if defined(PAGE_SHIFT_MAX) && defined(PAGE_SIZE_MAX) && defined(PAGE_MASK_MAX) && \ + defined(PAGE_SHIFT_MIN) && defined(PAGE_SIZE_MIN) && defined(PAGE_MASK_MIN) +/* Arch supports boot-time page size selection. */ +#elif defined(PAGE_SHIFT_MAX) || defined(PAGE_SIZE_MAX) || defined(PAGE_MASK_MAX) || \ + defined(PAGE_SHIFT_MIN) || defined(PAGE_SIZE_MIN) || defined(PAGE_MASK_MIN) +#error Arch must define all or none of the boot-time page size macros +#else +/* Arch does not support boot-time page size selection. */ +#define PAGE_SHIFT_MIN PAGE_SHIFT +#define PAGE_SIZE_MIN PAGE_SIZE +#define PAGE_MASK_MIN PAGE_MASK +#define PAGE_SHIFT_MAX PAGE_SHIFT +#define PAGE_SIZE_MAX PAGE_SIZE +#define PAGE_MASK_MAX PAGE_MASK +#endif + +/* + * Define a global variable (scalar or struct), whose value is derived from + * PAGE_SIZE and friends. When PAGE_SIZE is a compile-time constant, the global + * variable is simply defined with the static value. When PAGE_SIZE is + * determined at boot-time, a pure initcall is registered and run during boot to + * initialize the variable. + * + * @type: Unqualified type. Do not include "const"; implied by macro variant. + * @name: Variable name. + * @...: Initialization value. May be scalar or initializer. + * + * "static" is declared by placing "static" before the macro. + * + * Example: + * + * struct my_struct { + * int a; + * char b; + * }; + * + * static DEFINE_GLOBAL_PAGE_SIZE_VAR(struct my_struct, my_variable, { + * .a = 10, + * .b = 'e', + * }); + */ +#if PAGE_SIZE_MIN != PAGE_SIZE_MAX +#define __DEFINE_GLOBAL_PAGE_SIZE_VAR(type, name, attrib, ...) \ + type name attrib; \ + static int __init __attribute__((constructor)) __##name##_init(void) \ + { \ + name = (type)__VA_ARGS__; \ + return 0; \ + } + +#define DEFINE_GLOBAL_PAGE_SIZE_VAR(type, name, ...) \ + __DEFINE_GLOBAL_PAGE_SIZE_VAR(type, name, , __VA_ARGS__) + +#define DEFINE_GLOBAL_PAGE_SIZE_VAR_CONST(type, name, ...) \ + __DEFINE_GLOBAL_PAGE_SIZE_VAR(type, name, __ro_after_init, __VA_ARGS__) +#else /* PAGE_SIZE_MIN == PAGE_SIZE_MAX */ +#define __DEFINE_GLOBAL_PAGE_SIZE_VAR(type, name, attrib, ...) \ + type name attrib = __VA_ARGS__; \ + +#define DEFINE_GLOBAL_PAGE_SIZE_VAR(type, name, ...) \ + __DEFINE_GLOBAL_PAGE_SIZE_VAR(type, name, , __VA_ARGS__) + +#define DEFINE_GLOBAL_PAGE_SIZE_VAR_CONST(type, name, ...) \ + __DEFINE_GLOBAL_PAGE_SIZE_VAR(const type, name, , __VA_ARGS__) +#endif + +#endif /* ASM_GENERIC_PGTABLE_GEOMETRY_H */ diff --git a/init/main.c b/init/main.c index 206acdde51f5a..ba1515eb20b9d 100644 --- a/init/main.c +++ b/init/main.c @@ -899,6 +899,8 @@ static void __init early_numa_node_init(void) #endif } +static __init void do_ctors(void); + asmlinkage __visible __init __no_sanitize_address __noreturn __no_stack_protector void start_kernel(void) { @@ -910,6 +912,8 @@ void start_kernel(void) debug_objects_early_init(); init_vmlinux_build_id(); + do_ctors(); + cgroup_init_early(); local_irq_disable(); @@ -1360,7 +1364,6 @@ static void __init do_basic_setup(void) cpuset_init_smp(); driver_init(); init_irq_proc(); - do_ctors(); do_initcalls(); } From patchwork Mon Oct 14 10:58:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834670 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 369C8D1A43B for ; Mon, 14 Oct 2024 10:59:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C0D6A6B0088; Mon, 14 Oct 2024 06:59:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B970E6B0089; Mon, 14 Oct 2024 06:59:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A5D5A6B008A; Mon, 14 Oct 2024 06:59:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8AF136B0088 for ; Mon, 14 Oct 2024 06:59:35 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 2B0EF120D1F for ; Mon, 14 Oct 2024 10:59:28 +0000 (UTC) X-FDA: 82671911742.23.D05B948 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf12.hostedemail.com (Postfix) with ESMTP id 4E3A440012 for ; Mon, 14 Oct 2024 10:59:30 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf12.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903458; a=rsa-sha256; cv=none; b=sJkN/Aq5obejdtif+pRwv/Be0BtzDuhtKCLaOVKaJQNPHXYFxl3SQs1VX0Tao3xX2QfQop YO1NUccpvOGYiv3P9raWB0F5qeQiSnD3TUN6vL2n+F+z0vGeJXm6D8xDz1R0DGmvCyY1Yt kBeJnXMUUxnQRtx0GH41O6ECaLUwa70= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf12.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903458; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O6/Jx7UMSGM5GsZJBCvRZxEmHJTJyTPnGhAVFa7W4Fs=; b=fHa6CcQcO73f81xufrfq2PY3oqwmbGArikLWTx31gII7Dh6cXEdpBuii93inXQTaxbUqK7 ZoPz9VK1jBZqNqe9k1+/F8AsNw2BjB55BcfYzRHQHY0mgjYHjNv+aWpMYhJuVRTdFX83pJ 1ZgJP512Af/nl+fbO0ZZteLy0ndTDtE= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 443921684; Mon, 14 Oct 2024 04:00:02 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9E5CD3F51B; Mon, 14 Oct 2024 03:59:29 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Arnd Bergmann , Catalin Marinas , Christoph Lameter , David Hildenbrand , Dennis Zhou , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Tejun Heo , Will Deacon Cc: Ryan Roberts , linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 02/57] vmlinux: Align to PAGE_SIZE_MAX Date: Mon, 14 Oct 2024 11:58:09 +0100 Message-ID: <20241014105912.3207374-2-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 4E3A440012 X-Stat-Signature: tbc5uxduqczthkrh6168ybmgtrpbwh8i X-Rspam-User: X-HE-Tag: 1728903570-714041 X-HE-Meta: U2FsdGVkX18HaFFsc+OlNNGf38w29spNKG87c79LiCIjrVCT3pQDfV54rk+oVQ6SkEWjVI4AzQWpHR/HwnFz/1s6YiYq4UvJ8UOJxOgI2ey2xFB2AawSyRq17z2UtXVEjkO7ZHaildLp97MBWxU3K17Luh9s0HniMclnrqhN2THyRUyIl7dUxSreMPPL0YpXCPhZ/fi2fX2RqtH0gQMa6CB+eN9xmTfGyCHdKbB6zLzgMQae5WVbLubAl4EfVY1odLyYE5W0EycziZNCWMsWIdaJyzDkWyxG7hsirzwgjYj2cwqfLhLo/on1z3XFuKTVodK9InPycZAx8KmJQ3K6hpdlJUZysnbdhcv7P3hACQmq1LruR9rDqMtwSlj8U0JLyQytyYwev29RAJ9jF2OeS/B1yfY9sfo3fwZhgkNw7eocwmhpvvXe38MzBfxoIcO5RYXC2sNrTDLVaCF59atO669Z/XXkux936hkohh8EaYB7GkiRoAlpCbOnuU+FwhmAx1LP6D86hA118e/Qt58LzuYpGYOC9mFsdMCFaf81UFvsaB/Ut3Yo1G9dJ8UrA5IA0My+iabbrfq6RZR05LODK2BUEvGd6+8s0wzsYl6pTHI/fNu0FUmw+SR3BDarJyzL4XsRPs+Z7RKCJDTBT6pcE2R2w/Uiwcr3l1YlapEtZkcvEnTI+Ziton0ki8yWwomACtLjAi+KPOV1RVGGtS17BO1wXay5EFiaUQoKBZK/AmA0WbjaWzgYPmoenJ4Ve2wE74ZzbbKOUjLWC5atz3C65iQHdB+uDKnFyy7SyGay8f65hTvfnO5EDcrAL9Eg/lgAtxBvLSU1Si7RJODsHqW0+WHUT93h9y2Qq4tuxGCzleJN3Z1q6ek55Ku5VTjKa32NUbEHINyiJsrJ4RBXVoSNzy0mWq2GXi+M7BMjQgS8+tTaN79iZ1BDAUV/G1G5XiehtAZursu0sd/crvLH/KE SbznX4j2 8a/p4MTOo5XumoTYYGgKPrshHrrp0c6wYJC9jvqQN88xeZKs1ov8AmVBK0T45VglLjuKnaHnnehIe1XFGTqzzxIIG5oVkU6LO/LUM4WMVizIuMgTVzvSukQ5s5SsJHzpnejjL+KgWKluceyclJRD+dLhh73nANiYQ1qJZq077V5EqnlXCEIrKLFxyYqjP+QaORms09OnegsVK5k9a3DdyVCm/yWfPsY/c67U1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Increase alignment of structures requiring at least PAGE_SIZE alignment to PAGE_SIZE_MAX. For compile-time PAGE_SIZE, PAGE_SIZE_MAX == PAGE_SIZE so there is no change. For boot-time PAGE_SIZE, PAGE_SIZE_MAX is the largest selectable page size. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ include/asm-generic/vmlinux.lds.h | 32 +++++++++++++++---------------- include/linux/linkage.h | 4 ++-- include/linux/percpu-defs.h | 4 ++-- 3 files changed, 20 insertions(+), 20 deletions(-) diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index 1ae44793132a8..5727f883001bb 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -13,7 +13,7 @@ * . = START; * __init_begin = .; * HEAD_TEXT_SECTION - * INIT_TEXT_SECTION(PAGE_SIZE) + * INIT_TEXT_SECTION(PAGE_SIZE_MAX) * INIT_DATA_SECTION(...) * PERCPU_SECTION(CACHELINE_SIZE) * __init_end = .; @@ -23,7 +23,7 @@ * _etext = .; * * _sdata = .; - * RO_DATA(PAGE_SIZE) + * RO_DATA(PAGE_SIZE_MAX) * RW_DATA(...) * _edata = .; * @@ -371,10 +371,10 @@ * Data section helpers */ #define NOSAVE_DATA \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ __nosave_begin = .; \ *(.data..nosave) \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ __nosave_end = .; #define PAGE_ALIGNED_DATA(page_align) \ @@ -733,9 +733,9 @@ . = ALIGN(bss_align); \ .bss : AT(ADDR(.bss) - LOAD_OFFSET) { \ BSS_FIRST_SECTIONS \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ *(.bss..page_aligned) \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ *(.dynbss) \ *(BSS_MAIN) \ *(COMMON) \ @@ -950,9 +950,9 @@ */ #ifdef CONFIG_AMD_MEM_ENCRYPT #define PERCPU_DECRYPTED_SECTION \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ *(.data..percpu..decrypted) \ - . = ALIGN(PAGE_SIZE); + . = ALIGN(PAGE_SIZE_MAX); #else #define PERCPU_DECRYPTED_SECTION #endif @@ -1030,7 +1030,7 @@ #define PERCPU_INPUT(cacheline) \ __per_cpu_start = .; \ *(.data..percpu..first) \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ *(.data..percpu..page_aligned) \ . = ALIGN(cacheline); \ *(.data..percpu..read_mostly) \ @@ -1075,16 +1075,16 @@ * PERCPU_SECTION - define output section for percpu area, simple version * @cacheline: cacheline size * - * Align to PAGE_SIZE and outputs output section for percpu area. This + * Align to PAGE_SIZE_MAX and outputs output section for percpu area. This * macro doesn't manipulate @vaddr or @phdr and __per_cpu_load and * __per_cpu_start will be identical. * - * This macro is equivalent to ALIGN(PAGE_SIZE); PERCPU_VADDR(@cacheline,,) + * This macro is equivalent to ALIGN(PAGE_SIZE_MAX); PERCPU_VADDR(@cacheline,,) * except that __per_cpu_load is defined as a relative symbol against * .data..percpu which is required for relocatable x86_32 configuration. */ #define PERCPU_SECTION(cacheline) \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ .data..percpu : AT(ADDR(.data..percpu) - LOAD_OFFSET) { \ __per_cpu_load = .; \ PERCPU_INPUT(cacheline) \ @@ -1102,15 +1102,15 @@ * All sections are combined in a single .data section. * The sections following CONSTRUCTORS are arranged so their * typical alignment matches. - * A cacheline is typical/always less than a PAGE_SIZE so + * A cacheline is typical/always less than a PAGE_SIZE_MAX so * the sections that has this restriction (or similar) - * is located before the ones requiring PAGE_SIZE alignment. - * NOSAVE_DATA starts and ends with a PAGE_SIZE alignment which + * is located before the ones requiring PAGE_SIZE_MAX alignment. + * NOSAVE_DATA starts and ends with a PAGE_SIZE_MAX alignment which * matches the requirement of PAGE_ALIGNED_DATA. * * use 0 as page_align if page_aligned data is not used */ #define RW_DATA(cacheline, pagealigned, inittask) \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ .data : AT(ADDR(.data) - LOAD_OFFSET) { \ INIT_TASK_DATA(inittask) \ NOSAVE_DATA \ diff --git a/include/linux/linkage.h b/include/linux/linkage.h index 5c8865bb59d91..68aa9775fce51 100644 --- a/include/linux/linkage.h +++ b/include/linux/linkage.h @@ -36,8 +36,8 @@ __stringify(name)) #endif -#define __page_aligned_data __section(".data..page_aligned") __aligned(PAGE_SIZE) -#define __page_aligned_bss __section(".bss..page_aligned") __aligned(PAGE_SIZE) +#define __page_aligned_data __section(".data..page_aligned") __aligned(PAGE_SIZE_MAX) +#define __page_aligned_bss __section(".bss..page_aligned") __aligned(PAGE_SIZE_MAX) /* * For assembly routines. diff --git a/include/linux/percpu-defs.h b/include/linux/percpu-defs.h index 8efce7414fad6..89c7f430015ba 100644 --- a/include/linux/percpu-defs.h +++ b/include/linux/percpu-defs.h @@ -156,11 +156,11 @@ */ #define DECLARE_PER_CPU_PAGE_ALIGNED(type, name) \ DECLARE_PER_CPU_SECTION(type, name, "..page_aligned") \ - __aligned(PAGE_SIZE) + __aligned(PAGE_SIZE_MAX) #define DEFINE_PER_CPU_PAGE_ALIGNED(type, name) \ DEFINE_PER_CPU_SECTION(type, name, "..page_aligned") \ - __aligned(PAGE_SIZE) + __aligned(PAGE_SIZE_MAX) /* * Declaration/definition used for per-CPU variables that must be read mostly. From patchwork Mon Oct 14 10:58:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73677D1A43B for ; Mon, 14 Oct 2024 10:59:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 075F96B0089; Mon, 14 Oct 2024 06:59:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 04B306B008A; Mon, 14 Oct 2024 06:59:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E2DEB6B008C; Mon, 14 Oct 2024 06:59:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C58056B0089 for ; Mon, 14 Oct 2024 06:59:38 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id E44C1160D11 for ; Mon, 14 Oct 2024 10:59:29 +0000 (UTC) X-FDA: 82671911910.02.9929020 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf04.hostedemail.com (Postfix) with ESMTP id 8DCA540005 for ; Mon, 14 Oct 2024 10:59:28 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903434; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VpicSstlstrg5d6N29OH0psgAbIQb3i7oiCjbYwYKQQ=; b=PLYJwZo6gwz4+WiFr8Wnkitb7K6qzK8SXwjVN0wJcwkQN2t+/RMe9bNHsE7ZDeXGd5koF4 Y6q4N4mIHVWL7biDqkygAvJlPnMj2EAAZ1rkaq6fdMKqr1F42jSchoBO0m9dAtgReYqx57 x0+T2DrPGCFX/i6rngFoMglgDpm4hmk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903434; a=rsa-sha256; cv=none; b=w8uZK/Y18h2378Y/rjvGPm19A9oLf4VAfjY6aH9U5MzQsyJA1jAkOv0ufj2Bi/hNZAR8sP ly4xYZfX6xT4zn3upEsyXp4npkUh5SRjBvPvfsp8Jq9GeFcc3A237Sj4aKt92wuHOZF7uV 6++sZXfbLB5i69KBReGvXJQ6FqkQrGw= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; spf=pass (imf04.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8FB151424; Mon, 14 Oct 2024 04:00:05 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E9C283F51B; Mon, 14 Oct 2024 03:59:32 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Johannes Weiner , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Michal Hocko , Miroslav Benes , Roman Gushchin , Shakeel Butt , Will Deacon Cc: Ryan Roberts , cgroups@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 03/57] mm/memcontrol: Fix seq_buf size to save memory when PAGE_SIZE is large Date: Mon, 14 Oct 2024 11:58:10 +0100 Message-ID: <20241014105912.3207374-3-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Stat-Signature: eynhgpszzqhe5ckbwoz3smf47pknt4ro X-Rspamd-Queue-Id: 8DCA540005 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1728903568-861662 X-HE-Meta: U2FsdGVkX18XNc0QUoA7sHjis0JkxQp+VSyFqQtKnidhdo1smSJCv+Hl6sgAoDbpZf/6/+41k5VUKobpYaM76R8tVccDhdvgbO2lZ90S3Gh2NznCtFuEzqd1kEjs/PXmZvIufPYYisxnTrmCV3k/Kw3e/XdHExmCGJgmN/biMoufRQKCtpvAoEpumPWmVKt6oDH9dfol12olnXfxLPKcwYzOVsBB7QJk8SXL62aPGr3nFTyUrdxaCOKfp2sYP+tdQcqdoHy5LF1BauuVXU0tPl8kc7YQ1+dGzV1EcmkejhKP1rAFafnA+wtLCKJrftnxJotxOvIWm78QFVZiNeN2vjk6kLpgBccRLotEcPoLQIuunkwj71k3NGooKXpCY26fpmUQTu83xzDhWHhox7uf5jTAEyhsfvkbp2PSYzpVzYbB+HumK2dCtOJFKoHa7btkS4GuV1mMdbKq+KKFpKqygfii+eR6fZkSMMkDgppMwn8iinh3OlFAQNHMT0vJEwimKi9F08K00Z+pIMSoZoLEUvk2I2ajZSPi+GmSg/GZdHmFRBLTVXp48UGO/xfbRF2Ln3EEJiOybmXM013xEZmSm3rKdpC30ag+wxF9dlRa7pcKYQD/sVUhajZ85CmT2/D5j1EdqFiXkiAZsO3FEzCaWjaLT7A8Ja8woloLw0gEuxjsfEhwEgDiXVUlo6hBNsmMc4M6DCujjOPiIOyilOiioe+GcYuubU/YRzQiqpgKAqqT41bDqRLI7c1tK50iWN1SKsxWlQP/itMwIpVuMOxKST0gH07LT5rG1X/5qOlWH/s6724/wUx0VvM9bSLevVblxNa/R8qfASxrjTSUbFBCYwkTU0OnDijA+ejTwAyBL/FZkz4TL/Wqmky0dp4QW6Aujos8O6yqc78ryIZ7nRDXSGFR1Elj478P4rkdDDnN9K4F6NFf+6mVD5DyWTGihB4wA45KZikSXNeBv8TOpr6 0r34HfKZ WNWpoxxGxhqJw13FQDhsKERIrIZjQ0zZ+mdTVNntR5YVZ1SBMQB/MvaDdkXwskQXlMIbserFtWX6uDJPQuH0X565nAKDGYr0+nvXV6tb9sad0rYXblEfTAc19YYS2mSMKIrvjMC9HVIFimz+ZQwbiV10tFW6beCy8EyNVt1ZUihQSgiCjFHvWS9RL7d9y4ggtb3URmETSFAs1gk6ONQpFtcPUPZolJaF0b68U X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Previously the seq_buf used for accumulating the memory.stat output was sized at PAGE_SIZE. But the amount of output is invariant to PAGE_SIZE; If 4K is enough on a 4K page system, then it should also be enough on a 64K page system, so we can save 60K om the static buffer used in mem_cgroup_print_oom_meminfo(). Let's make it so. This also has the beneficial side effect of removing a place in the code that assumed PAGE_SIZE is a compile-time constant. So this helps our quest towards supporting boot-time page size selection. Signed-off-by: Ryan Roberts Acked-by: Johannes Weiner Acked-by: Shakeel Butt Acked-by: Michal Hocko Acked-by: Roman Gushchin --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ mm/memcontrol.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index d563fb515766b..c5f9195f76c65 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -95,6 +95,7 @@ static DECLARE_WAIT_QUEUE_HEAD(memcg_cgwb_frn_waitq); #define THRESHOLDS_EVENTS_TARGET 128 #define SOFTLIMIT_EVENTS_TARGET 1024 +#define SEQ_BUF_SIZE SZ_4K static inline bool task_is_dying(void) { @@ -1519,7 +1520,7 @@ void mem_cgroup_print_oom_context(struct mem_cgroup *memcg, struct task_struct * void mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg) { /* Use static buffer, for the caller is holding oom_lock. */ - static char buf[PAGE_SIZE]; + static char buf[SEQ_BUF_SIZE]; struct seq_buf s; lockdep_assert_held(&oom_lock); @@ -1545,7 +1546,7 @@ void mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg) pr_info("Memory cgroup stats for "); pr_cont_cgroup_path(memcg->css.cgroup); pr_cont(":"); - seq_buf_init(&s, buf, sizeof(buf)); + seq_buf_init(&s, buf, SEQ_BUF_SIZE); memory_stat_format(memcg, &s); seq_buf_do_printk(&s, KERN_INFO); } @@ -4158,12 +4159,12 @@ static int memory_events_local_show(struct seq_file *m, void *v) int memory_stat_show(struct seq_file *m, void *v) { struct mem_cgroup *memcg = mem_cgroup_from_seq(m); - char *buf = kmalloc(PAGE_SIZE, GFP_KERNEL); + char *buf = kmalloc(SEQ_BUF_SIZE, GFP_KERNEL); struct seq_buf s; if (!buf) return -ENOMEM; - seq_buf_init(&s, buf, PAGE_SIZE); + seq_buf_init(&s, buf, SEQ_BUF_SIZE); memory_stat_format(memcg, &s); seq_puts(m, buf); kfree(buf); From patchwork Mon Oct 14 10:58:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834672 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA213D1A443 for ; Mon, 14 Oct 2024 10:59:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 83DEC6B008A; Mon, 14 Oct 2024 06:59:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C7866B008C; Mon, 14 Oct 2024 06:59:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 690106B0092; Mon, 14 Oct 2024 06:59:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 4D5476B008A for ; Mon, 14 Oct 2024 06:59:41 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5C5A38075C for ; Mon, 14 Oct 2024 10:59:34 +0000 (UTC) X-FDA: 82671911910.13.79D0A95 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP id 60FC240010 for ; Mon, 14 Oct 2024 10:59:34 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903391; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kYeTTVqmvopusG+SkQLqUutllx+fKzmiPfJpsd8f/kQ=; b=kRFgBwa1uksiX+zMwFszoVmrtbLNmPmwim+9QMSdsea1H29WSSctPcr/5fKvkK2ZiazW6o AB8GCg2dJK/igUPe/Kxa4t5WmOKcq1CWDW8fJsYqOTr7ZgvIDN7vSx5kmEPtHDq5oCNWjB ec2qtBrlVZOF4ilE/NNa3FgaKFeGvt8= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903391; a=rsa-sha256; cv=none; b=3RaWhNKSzxLfPSv6R+ZA7FvS2JWC2ZEMAjVZY9GLbg+saRQIGxfCFoFQWcwKxAbUd8JdDf 5oS8l/u4hzUzkxkn0t3DVgS/qaZZ77/JuG67pS6OWyxXhf7PcKMt9Sx8JLWewgcuv3gYL0 r8rHGqk/rpxMae8m8x3IzqvVcBuUyvE= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 405271688; Mon, 14 Oct 2024 04:00:08 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 403163F51B; Mon, 14 Oct 2024 03:59:36 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 04/57] mm/page_alloc: Make page_frag_cache boot-time page size compatible Date: Mon, 14 Oct 2024 11:58:11 +0100 Message-ID: <20241014105912.3207374-4-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 60FC240010 X-Stat-Signature: eu519u4fxnqkzpk9sr1j4zwba5bc4p6h X-Rspam-User: X-HE-Tag: 1728903574-993617 X-HE-Meta: U2FsdGVkX1+JqLx+wMzBeRkiDlq7cXGYBk9VrNRDAZMabR4QQf1dxvvb7rtszRbPDlDWkb4KXZ56lhKokmTk9W7sa8AKHixe0+yWvqU07uJ7ACc2ojWZjTuNS4rLyhVja/PaKE+TUMVxByp1mrRcYq4pleMpJFbkrXUX51ODSAJCQBeZRdyYMLL9tbun9xBiZz9AWygjeCIdq8kdfZy4A4MvMqsWPVwX6kl2qmE+CqAgOIeZVXfjqNI6Ap4Yb3+pXfnbCoPvYNIVcM3Y24ZxIUWuqT70lyRHxywgomr4XT2PhnAGj4Utp4BEPSV2ieJd2PU+r2xkyLkFm/IucQFFFgpPSLC+cGOr2kx0cVtR2nfo5J1z9NlUTMQ3Se21VquPtzYUMb30rO31+3/9rSCwzd2Knjev0hjhzFQTkwvr+zLRj0Jl+Jdh9m6tfVrD1HvnmPI+Pd1WZWa5IdV/UdbaW0S/9AdYVSspPtHUfjnzpjkc7zaZvqfiFG9M8qVvOblvYnlBo2M85uDrHc3c/pb53JmkLwd22vpjKdRN5przfBNruEux0ZQ6sC1ibv0y/EYLiF9lvQMkgogXAx74XnQUsiiQK5n8KleNzf1bRS7Kldc41Z/wLI3Elia67iX0m4YaxkF1E/l55PGbtxoQKueedJ/5qhtW2WQ8KzEY1aYcOgacbfWg4tN6y23V4NlA4GpKQ85je2aNo4d/Tff0qnM1rgtpLtTuBrZmDGi3Q27mY9lVrDoWjlDi5vOh+nNRmdMcavjR428nxgIxXN8YysA5qsBHxVsiKHIrbzLZzgLXouNczhfGA1oT8Tb3en7rUA/xML0a1a0lExbCSpNVjjE0UhIAulYDaLHuBx3JLJcHuNj/LSv2uvof3IP0ELyf4eU4Z8HkicJiEEFHd3NYWE5VLVk1AOSc/Ce2zbxDlM6lW6kTWnl1av24LUS8yz/Oc4EDlrpUYJopeuqinQMZXU2 VUn+18ob 30o+0fEUK3Py61RZmUkgf5ibT+5igoGVJZwquBj22KsSqIWkzX+m1Bjir6g6XC9TiOCCFuaXHiLsYLw316tunZ8pV12mMLif6bVO5noglMoTffpa4IzhMLcB7GvJ0kt4VWu46aq7DWW3ZVSSz2E42kBbnp6jA04OwN5pZZcnxxdx9+1BsVIWT7DLQFsTjq/e5VXerVIi9Otn6TlcFrtLApykF3cc3LGKSCkTU X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: "struct page_frag_cache" has some optimizations that depend on page size. Let's refactor it a bit so that those optimizations can be determined at run-time for the case where page size is a boot-time parameter. For compile-time page size, the compiler should dead code strip and the result is very similar to before. One wrinkle is that we don't know if we need the size member until runtime. So remove the ifdeffery and always define offset as u32 (needed if PAGE_SIZE is >= 64K) and size as u16 (only used when PAGE_SIZE <= 32K). We move the members around a bit so that the overall size of the struct remains the same; 24 bytes for 64-bit and 16 bytes on 32 bit. Signed-off-by: Ryan Roberts page_alloc Acked-by: Vlastimil Babka --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ include/linux/mm_types.h | 13 ++++++------- mm/page_alloc.c | 31 ++++++++++++++++++------------- 2 files changed, 24 insertions(+), 20 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 4854249792545..0844ed7cfaa53 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -544,16 +544,15 @@ static inline void *folio_get_private(struct folio *folio) struct page_frag_cache { void * va; -#if (PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE) - __u16 offset; - __u16 size; -#else - __u32 offset; -#endif /* we maintain a pagecount bias, so that we dont dirty cache line * containing page->_refcount every time we allocate a fragment. */ - unsigned int pagecnt_bias; + unsigned int pagecnt_bias; + __u32 offset; + /* size only used when PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE, in which + * case PAGE_FRAG_CACHE_MAX_SIZE is 32K and 16 bits is sufficient. + */ + __u16 size; bool pfmemalloc; }; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 91ace8ca97e21..8678103b1b396 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -4822,13 +4822,18 @@ static struct page *__page_frag_cache_refill(struct page_frag_cache *nc, struct page *page = NULL; gfp_t gfp = gfp_mask; -#if (PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE) - gfp_mask = (gfp_mask & ~__GFP_DIRECT_RECLAIM) | __GFP_COMP | - __GFP_NOWARN | __GFP_NORETRY | __GFP_NOMEMALLOC; - page = alloc_pages_node(NUMA_NO_NODE, gfp_mask, - PAGE_FRAG_CACHE_MAX_ORDER); - nc->size = page ? PAGE_FRAG_CACHE_MAX_SIZE : PAGE_SIZE; -#endif + if (PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE) { + gfp_mask = (gfp_mask & ~__GFP_DIRECT_RECLAIM) | __GFP_COMP | + __GFP_NOWARN | __GFP_NORETRY | __GFP_NOMEMALLOC; + page = alloc_pages_node(NUMA_NO_NODE, gfp_mask, + PAGE_FRAG_CACHE_MAX_ORDER); + /* + * Cast to silence warning due to 16-bit nc->size. Not real + * because PAGE_SIZE only less than PAGE_FRAG_CACHE_MAX_SIZE + * when PAGE_FRAG_CACHE_MAX_SIZE is 32K. + */ + nc->size = (__u16)(page ? PAGE_FRAG_CACHE_MAX_SIZE : PAGE_SIZE); + } if (unlikely(!page)) page = alloc_pages_node(NUMA_NO_NODE, gfp, 0); @@ -4870,10 +4875,10 @@ void *__page_frag_alloc_align(struct page_frag_cache *nc, if (!page) return NULL; -#if (PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE) /* if size can vary use size else just use PAGE_SIZE */ - size = nc->size; -#endif + if (PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE) + size = nc->size; + /* Even if we own the page, we do not use atomic_set(). * This would break get_page_unless_zero() users. */ @@ -4897,10 +4902,10 @@ void *__page_frag_alloc_align(struct page_frag_cache *nc, goto refill; } -#if (PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE) /* if size can vary use size else just use PAGE_SIZE */ - size = nc->size; -#endif + if (PAGE_SIZE < PAGE_FRAG_CACHE_MAX_SIZE) + size = nc->size; + /* OK, page count is 0, we can safely set it */ set_page_count(page, PAGE_FRAG_CACHE_MAX_SIZE + 1); From patchwork Mon Oct 14 10:58:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2C30D1A45A for ; Mon, 14 Oct 2024 10:59:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 557316B0082; Mon, 14 Oct 2024 06:59:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5074A6B0083; Mon, 14 Oct 2024 06:59:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F6C76B008C; Mon, 14 Oct 2024 06:59:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 212826B0082 for ; Mon, 14 Oct 2024 06:59:44 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D599B140D1B for ; Mon, 14 Oct 2024 10:59:35 +0000 (UTC) X-FDA: 82671912120.16.9F74464 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf28.hostedemail.com (Postfix) with ESMTP id 170DAC0009 for ; Mon, 14 Oct 2024 10:59:35 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903509; a=rsa-sha256; cv=none; b=zaad3BYU6OMGCOfrYXXrwwXAhqlJFLGtCqs2FFsS/wbamxRIDmyltHaylG0JiV33FJrV/Y edv559BDwCmD5r5xLGcTv150st6KQRRJNtvuyPZzvtoqkz5ssjaApqdX+eQyOQeZH4u00W ZleGXfiLab0FaUKmwiNQ85YkyZ4ikms= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903509; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qPt13oBtsQSK+UGOOEWOc/EwkFI9q79bukNoe+ZJFuA=; b=C29wg4eNLloR/QrJvrGCt+hZhvKO0bAcHB/ORgk9PKuxrfggr4Yu462rNt4gL9I3HaBLAq t3doBdD5gnA+YaHrh9vFngv5uOvGuni2kGg9DhBJJZFodObPmvK2PInZ10W+doraxRBzip lFGRF2rhMAsZYknZphsmhoEVxTT2Kb8= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E3E091684; Mon, 14 Oct 2024 04:00:10 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E47583F51B; Mon, 14 Oct 2024 03:59:38 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 05/57] mm: Avoid split pmd ptl if pmd level is run-time folded Date: Mon, 14 Oct 2024 11:58:12 +0100 Message-ID: <20241014105912.3207374-5-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Stat-Signature: xcuco3a8m7a9eqnj6h6hfw9wjymw9icb X-Rspamd-Queue-Id: 170DAC0009 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1728903575-642657 X-HE-Meta: U2FsdGVkX19d5QI2hyIreMKRHPC54Q8zP11rr3IkoByiDWSgYosDlJzsWAwAsMnkHzOUzdK3v5xLyYtY6GYmEkGn8FAh92DJCyhIoXq9B/B2QIyM65jYZVYx0voKKgCWxsHyVMXuhveIk1QWYxjSUFrwSgC5awNUTcROrMpzlMXBU+UvuR74+lXw4zY9h2NyNp+6bM/KuInyU6NsKAEGpxGqySyTvOsf2zSl4NOyPJTt63q3TNSZOADsSVC8wjkD3J7UVsPYz5VVNAsUOju+GGtDJYCuAbj02XW26ecFPlLaFGkPvDmXTomu802Z3OiX/Pb2Kzh4dXQKoJ3z5ExT7/00bWlLg8x3K+bogdnbJ9HsUmgUbqYHR//UD6Qd3tNVvxj6413vBrmr8ihgN+vvBMTvVhYT5v/q1bGoPLJVm4XV0K1Nn+VZfW7o1SkqyafLqBiVJL/UhvS2UCiNUjH1rsMtmNxUNjg59VNwSKbcxmQhCJe/Gf2Y3UyjFu09Y0G2cb0KeFjSMC9CUxHv9orWFFkda/X7smElOpX7XWLMKbyYvr4VmDBRk6A+QQOGxU11k1qOCRWKW9k0lG+0yQWE8tNrcgU6lK3uM76iz0CEK1Qp7fjgm3bB94u+mnZM/McKu1uih6IbPDKcy908TCInU8l74Td0bAzwZAwcX8l6Gs+cjHRi3P3/2HCRIuT1gRgqbM0ZD7GtbfDzRoz0oiQUafnvOECGBymVoRyf76Q3vy1QK3i0EyRo7Jzz2gD0V4J0iuGi/ztV+HYz2i9svGY9MnpvvhgzOj3yBGvWg9r3cErLL+voz8K4hWwdHsclKWQ/YGI3lUrjPsGXvC8L0xMxLFn/FAlB6X15gQ9QdrTozDvWPVLonR53M9HbKUJlPSlHZLN0i+dQWNZgWkmZ+q3tpf4FnYnSdXv7QghEbh48cDBDhYipmbUM5XtVR2Q8Bg5NiWgFBQ1ST/+VzyHotUs vTV/Ca6D vPm2oedOwWg5UK9tEVpB7KK6Xg/c/CjGt0dxoaqfkD6EhyZp4f2pBdOmk3HwGkrCmRQkU1wdJLTnQC5qxsz6kik97aRsYTuU6MSpL7qBst/7+egrn6ih7UWvPza0oopgXnB6w4C6It8h2AlcjoBGv29lkmQ+MbR0gkZ2gKApcP+I9xIgeryTXBbCnww== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If there are only 2 levels of translation, the first level (pgd) may not be an entire page and so does not have a ptdesc backing it (this may be true on arm64 depending on the VA size and page size). Even if it is an entire page and does therefore have an entire ptdesc, pagetable_pmd_ctor() won't be called for the ptdesc (since it's a pgd not pmd table) and so the per-ptdec ptl fields won't be initialised. To date this has been fine; the arch knows at compile time if it needs to fold the pmd level and in this case does not select CONFIG_ARCH_ENABLE_SPLIT_PMD_PTLOCK. However, if the number of levels are not known at compile time (as is the case for boot-time page size selection), we want to be able to choose at boot whether to use split pmd ptls in the pmd's ptdesc or simply fall back to the lock in the mm_struct. So let's make that change; when CONFIG_ARCH_ENABLE_SPLIT_PMD_PTLOCK is selected, determine if it should be used at run-time based on mm_pmd_folded(). This sets us up for arm64 to support boot-time page size selection. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ include/linux/mm.h | 15 ++++++++++++++- include/linux/mm_types.h | 2 +- kernel/fork.c | 4 ++-- 3 files changed, 17 insertions(+), 4 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 1470736017168..09a840517c23a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3037,6 +3037,8 @@ static inline struct ptdesc *pmd_ptdesc(pmd_t *pmd) static inline spinlock_t *pmd_lockptr(struct mm_struct *mm, pmd_t *pmd) { + if (mm_pmd_folded(mm)) + return &mm->page_table_lock; return ptlock_ptr(pmd_ptdesc(pmd)); } @@ -3056,7 +3058,18 @@ static inline void pmd_ptlock_free(struct ptdesc *ptdesc) ptlock_free(ptdesc); } -#define pmd_huge_pte(mm, pmd) (pmd_ptdesc(pmd)->pmd_huge_pte) +static inline pgtable_t *__pmd_huge_pte(struct mm_struct *mm, pmd_t *pmd) +{ +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + if (mm_pmd_folded(mm)) + return &mm->pmd_huge_pte; + return &pmd_ptdesc(pmd)->pmd_huge_pte; +#else + return NULL; +#endif +} + +#define pmd_huge_pte(mm, pmd) (*__pmd_huge_pte(mm, pmd)) #else diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 0844ed7cfaa53..87dc6de7b7baf 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -946,7 +946,7 @@ struct mm_struct { #ifdef CONFIG_MMU_NOTIFIER struct mmu_notifier_subscriptions *notifier_subscriptions; #endif -#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) pgtable_t pmd_huge_pte; /* protected by page_table_lock */ #endif #ifdef CONFIG_NUMA_BALANCING diff --git a/kernel/fork.c b/kernel/fork.c index cc760491f2012..ea472566d4fcc 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -832,7 +832,7 @@ static void check_mm(struct mm_struct *mm) pr_alert("BUG: non-zero pgtables_bytes on freeing mm: %ld\n", mm_pgtables_bytes(mm)); -#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) VM_BUG_ON_MM(mm->pmd_huge_pte, mm); #endif } @@ -1276,7 +1276,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p, RCU_INIT_POINTER(mm->exe_file, NULL); mmu_notifier_subscriptions_init(mm); init_tlb_flush_pending(mm); -#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) mm->pmd_huge_pte = NULL; #endif mm_init_uprobes_state(mm); From patchwork Mon Oct 14 10:58:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834674 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 843C6D1A443 for ; Mon, 14 Oct 2024 10:59:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 24A596B0083; Mon, 14 Oct 2024 06:59:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1FA1B6B008C; Mon, 14 Oct 2024 06:59:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0C13A6B0092; Mon, 14 Oct 2024 06:59:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E08556B0083 for ; Mon, 14 Oct 2024 06:59:48 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8701EAB9F6 for ; Mon, 14 Oct 2024 10:59:33 +0000 (UTC) X-FDA: 82671912288.30.5D74BC7 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf02.hostedemail.com (Postfix) with ESMTP id 74D0280009 for ; Mon, 14 Oct 2024 10:59:35 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903556; a=rsa-sha256; cv=none; b=x3zwwsc1AEmixtReGa0c7DsvkNLLpD5XGq6FwSIcKUJIYX/LqGN07i1iApg87on4KAhpLt W2i3aa4gcEi6UL9rvWcYJ9S5/6nQmlZV8WhiD1uvHVe9BtjCCxaK3Dw0Jae8b7H1hVIsuw +48D3WXCImfpVvC9ehS3xnxj0bjF6tU= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903556; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cG9KRB9iAdom5fH60FL/Vpswjxk6D87RpKBF0kA0w7Q=; b=q7piNsZMCubpcu13j4wUwdDpGhnjK7EyIbiyHBzZQfSBlT64Cdp5I4uzaMbDURwoNiaUVU gm3Lmt4Hfa4ew8gydiRR/X08fylfKl8HluNyuV0sEaHaPOuTu4aG1Gl+QGRYocZVHd+fzq zPn0zXnkk2pYYURYAvX3UnVochf0eH8= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7DC7F1424; Mon, 14 Oct 2024 04:00:15 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 979593F51B; Mon, 14 Oct 2024 03:59:41 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , Christoph Lameter , David Hildenbrand , David Rientjes , Greg Marsden , Ivan Ivanov , Johannes Weiner , Joonsoo Kim , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Michal Hocko , Miquel Raynal , Miroslav Benes , Pekka Enberg , Richard Weinberger , Shakeel Butt , Vignesh Raghavendra , Vlastimil Babka , Will Deacon Cc: Ryan Roberts , cgroups@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-mtd@lists.infradead.org Subject: [RFC PATCH v1 06/57] mm: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:13 +0100 Message-ID: <20241014105912.3207374-6-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 5ao9kk5mnp6m8jpj6ezhs4wappacd16y X-Rspamd-Queue-Id: 74D0280009 X-Rspamd-Server: rspam02 X-HE-Tag: 1728903575-341861 X-HE-Meta: U2FsdGVkX1/GmwKyeJ2500NLoVvVVEyNRXdZ5nEHZhylf1YtnvE2aUUOriU5Iu85ZbJxGqVMSK2ipKSh0MvvGI2hE0MfhOB/gpyWvtdhVg/7kOvFIYrrlxR8pH7w6l60SV3In6WgdBL8356KDhWcjOxXscA4Lw0ednlAlzLQTsklPHqqxfQNJQTaWtFfggQE9XdyME7Qk769rcc54akzwyNW9vJKSgVRjqI9UXep2YmK4V+/wg7pIo7ImuiEOJKCYNMyt+ezYtJg77k+zCGJHgcHiLd5eDL4EVNOYDaz+SuHgC6o0tMRPL35w93LX9fLmKLy4mzEr/p8oUyFKbXROtE/2DFQywUfy7MNAbtyjW2TY77RQMJNYDTt+VCKsU52sYHChpOt+gGjaCglpsk8I0+YAP/yjmja7LYUTJ6t48wlxY/A6kC7sI5ppgVbF/Zw8ClpBu2+DJWmsIvrnRBJnJSh1xJW6SZ4Jba0rp6NUNXKMh5Lk7dPfZtChi2/SxmNZ8g7hbDP0ouMznPGA+7WYDJgKiu3ZtHZOX1XNjnZ1kgYpoXLbW99Ha6Yl/EyZlPmowndUYBiQY5RDEYnbDdSguLq7ZctYSu9kBs4RIH0jJAq5hH/rTi433ZvNbnB60w34pzujuOIeWPNZWHLtTC8QAOf2JWYCyfvHwOlWf6U9gXHTb3aNWf3h5CrAvj0ptei+Xhpco570aTRRvHfo912+vXmMhQG2ZYg8ZJprKCiAdH3B2uUZrAaOCq29xyzUP/B2lpfKN6IBxHnwsPBvklV/nFlBChoW7rx1PLg4o9P2hrAhYhu0wxvv2/Y7IBoWVvabotnL15loUiUZTaYu1ocgLoeGtQ7HEZZOPc+buMU077rQ1ak+/n+zA3moJXmbfG4QGFNhVMTpySG+MZAQ1QR3NKVCZOnycwxEg3AXaU16PmJV40enRM57mDLHE+reF0GonjJv0uQsd2AQa5RvBm nioAEW7d qV2msTN2cH/o6fFfAmWnS0f4fwp+8VbpLpH9rbxsDt+Zht/6+MQk1TjXKFVvI+tsv5bLK542SnWaAZaEAy4kj3W3qRpefDYdnmH5yqCrT8Wt8WEFd9MJ3Ap7wp40EMVtOB1Ta1uUIBmiTd4VDK6xeDMFN7axeGQ4nSP5tRLgym+qeJeu3KSClY3VpvWtNc4vkQ5Uuik2dlJOOFKlG8BKMKpXavsEu2U6HnYvM X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Refactor "struct vmap_block" to use a flexible array for used_mmap since VMAP_BBMAP_BITS is not a compile time constant for the boot-time page size case. Update various BUILD_BUG_ON() instances to check against appropriate page size limit. Re-define "union swap_header" so that it's no longer exactly page-sized. Instead define a flexible "magic" array with a define which tells the offset to where the magic signature begins. Consider page size limit in some CPP condditionals. Wrap global variables that are initialized with PAGE_SIZE derived values using DEFINE_GLOBAL_PAGE_SIZE_VAR() so their initialization can be deferred for boot-time page size builds. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ drivers/mtd/mtdswap.c | 4 ++-- include/linux/mm.h | 2 +- include/linux/mm_types_task.h | 2 +- include/linux/mmzone.h | 3 ++- include/linux/slab.h | 7 ++++--- include/linux/swap.h | 17 ++++++++++++----- include/linux/swapops.h | 6 +++++- mm/memcontrol.c | 2 +- mm/memory.c | 4 ++-- mm/mmap.c | 2 +- mm/page-writeback.c | 2 +- mm/slub.c | 2 +- mm/sparse.c | 2 +- mm/swapfile.c | 2 +- mm/vmalloc.c | 7 ++++--- 15 files changed, 39 insertions(+), 25 deletions(-) diff --git a/drivers/mtd/mtdswap.c b/drivers/mtd/mtdswap.c index 680366616da24..7412a32708114 100644 --- a/drivers/mtd/mtdswap.c +++ b/drivers/mtd/mtdswap.c @@ -1062,13 +1062,13 @@ static int mtdswap_auto_header(struct mtdswap_dev *d, char *buf) { union swap_header *hd = (union swap_header *)(buf); - memset(buf, 0, PAGE_SIZE - 10); + memset(buf, 0, SWAP_HEADER_MAGIC); hd->info.version = 1; hd->info.last_page = d->mbd_dev->size - 1; hd->info.nr_badpages = 0; - memcpy(buf + PAGE_SIZE - 10, "SWAPSPACE2", 10); + memcpy(buf + SWAP_HEADER_MAGIC, "SWAPSPACE2", 10); return 0; } diff --git a/include/linux/mm.h b/include/linux/mm.h index 09a840517c23a..49c2078354e6e 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2927,7 +2927,7 @@ static inline spinlock_t *pte_lockptr(struct mm_struct *mm, pmd_t *pmd) static inline spinlock_t *ptep_lockptr(struct mm_struct *mm, pte_t *pte) { BUILD_BUG_ON(IS_ENABLED(CONFIG_HIGHPTE)); - BUILD_BUG_ON(MAX_PTRS_PER_PTE * sizeof(pte_t) > PAGE_SIZE); + BUILD_BUG_ON(MAX_PTRS_PER_PTE * sizeof(pte_t) > PAGE_SIZE_MAX); return ptlock_ptr(virt_to_ptdesc(pte)); } diff --git a/include/linux/mm_types_task.h b/include/linux/mm_types_task.h index a2f6179b672b8..c356897d5f41c 100644 --- a/include/linux/mm_types_task.h +++ b/include/linux/mm_types_task.h @@ -37,7 +37,7 @@ struct page; struct page_frag { struct page *page; -#if (BITS_PER_LONG > 32) || (PAGE_SIZE >= 65536) +#if (BITS_PER_LONG > 32) || (PAGE_SIZE_MAX >= 65536) __u32 offset; __u32 size; #else diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 1dc6248feb832..cd58034b82c81 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -1744,6 +1744,7 @@ static inline bool movable_only_nodes(nodemask_t *nodes) */ #define PA_SECTION_SHIFT (SECTION_SIZE_BITS) #define PFN_SECTION_SHIFT (SECTION_SIZE_BITS - PAGE_SHIFT) +#define PFN_SECTION_SHIFT_MIN (SECTION_SIZE_BITS - PAGE_SHIFT_MAX) #define NR_MEM_SECTIONS (1UL << SECTIONS_SHIFT) @@ -1753,7 +1754,7 @@ static inline bool movable_only_nodes(nodemask_t *nodes) #define SECTION_BLOCKFLAGS_BITS \ ((1UL << (PFN_SECTION_SHIFT - pageblock_order)) * NR_PAGEBLOCK_BITS) -#if (MAX_PAGE_ORDER + PAGE_SHIFT) > SECTION_SIZE_BITS +#if (MAX_PAGE_ORDER + PAGE_SHIFT_MAX) > SECTION_SIZE_BITS #error Allocator MAX_PAGE_ORDER exceeds SECTION_SIZE #endif diff --git a/include/linux/slab.h b/include/linux/slab.h index eb2bf46291576..11c6ff3a12579 100644 --- a/include/linux/slab.h +++ b/include/linux/slab.h @@ -347,7 +347,7 @@ static inline unsigned int arch_slab_minalign(void) */ #define __assume_kmalloc_alignment __assume_aligned(ARCH_KMALLOC_MINALIGN) #define __assume_slab_alignment __assume_aligned(ARCH_SLAB_MINALIGN) -#define __assume_page_alignment __assume_aligned(PAGE_SIZE) +#define __assume_page_alignment __assume_aligned(PAGE_SIZE_MIN) /* * Kmalloc array related definitions @@ -358,6 +358,7 @@ static inline unsigned int arch_slab_minalign(void) * (PAGE_SIZE*2). Larger requests are passed to the page allocator. */ #define KMALLOC_SHIFT_HIGH (PAGE_SHIFT + 1) +#define KMALLOC_SHIFT_HIGH_MAX (PAGE_SHIFT_MAX + 1) #define KMALLOC_SHIFT_MAX (MAX_PAGE_ORDER + PAGE_SHIFT) #ifndef KMALLOC_SHIFT_LOW #define KMALLOC_SHIFT_LOW 3 @@ -426,7 +427,7 @@ enum kmalloc_cache_type { NR_KMALLOC_TYPES }; -typedef struct kmem_cache * kmem_buckets[KMALLOC_SHIFT_HIGH + 1]; +typedef struct kmem_cache * kmem_buckets[KMALLOC_SHIFT_HIGH_MAX + 1]; extern kmem_buckets kmalloc_caches[NR_KMALLOC_TYPES]; @@ -524,7 +525,7 @@ static __always_inline unsigned int __kmalloc_index(size_t size, /* Will never be reached. Needed because the compiler may complain */ return -1; } -static_assert(PAGE_SHIFT <= 20); +static_assert(PAGE_SHIFT_MAX <= 20); #define kmalloc_index(s) __kmalloc_index(s, true) #include diff --git a/include/linux/swap.h b/include/linux/swap.h index ba7ea95d1c57a..e85df0332979f 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -132,10 +132,17 @@ static inline int current_is_kswapd(void) * bootbits... */ union swap_header { - struct { - char reserved[PAGE_SIZE - 10]; - char magic[10]; /* SWAP-SPACE or SWAPSPACE2 */ - } magic; + /* + * Exists conceptually, but since PAGE_SIZE may not be known at compile + * time, we must access through pointer arithmetic at run time. + * + * struct { + * char reserved[PAGE_SIZE - 10]; + * char magic[10]; SWAP-SPACE or SWAPSPACE2 + * } magic; + */ +#define SWAP_HEADER_MAGIC (PAGE_SIZE - 10) + char magic[1]; struct { char bootbits[1024]; /* Space for disklabel etc. */ __u32 version; @@ -201,7 +208,7 @@ struct swap_extent { * Max bad pages in the new format.. */ #define MAX_SWAP_BADPAGES \ - ((offsetof(union swap_header, magic.magic) - \ + ((SWAP_HEADER_MAGIC - \ offsetof(union swap_header, info.badpages)) / sizeof(int)) enum { diff --git a/include/linux/swapops.h b/include/linux/swapops.h index cb468e418ea11..890fe6a3e6702 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -34,10 +34,14 @@ */ #ifdef MAX_PHYSMEM_BITS #define SWP_PFN_BITS (MAX_PHYSMEM_BITS - PAGE_SHIFT) +#define SWP_PFN_BITS_MAX (MAX_PHYSMEM_BITS - PAGE_SHIFT_MIN) #else /* MAX_PHYSMEM_BITS */ #define SWP_PFN_BITS min_t(int, \ sizeof(phys_addr_t) * 8 - PAGE_SHIFT, \ SWP_TYPE_SHIFT) +#define SWP_PFN_BITS_MAX min_t(int, \ + sizeof(phys_addr_t) * 8 - PAGE_SHIFT_MIN, \ + SWP_TYPE_SHIFT) #endif /* MAX_PHYSMEM_BITS */ #define SWP_PFN_MASK (BIT(SWP_PFN_BITS) - 1) @@ -519,7 +523,7 @@ static inline struct folio *pfn_swap_entry_folio(swp_entry_t entry) static inline bool is_pfn_swap_entry(swp_entry_t entry) { /* Make sure the swp offset can always store the needed fields */ - BUILD_BUG_ON(SWP_TYPE_SHIFT < SWP_PFN_BITS); + BUILD_BUG_ON(SWP_TYPE_SHIFT < SWP_PFN_BITS_MAX); return is_migration_entry(entry) || is_device_private_entry(entry) || is_device_exclusive_entry(entry) || is_hwpoison_entry(entry); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c5f9195f76c65..4b17bec566fbd 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -4881,7 +4881,7 @@ static int __init mem_cgroup_init(void) * to work fine, we should make sure that the overfill threshold can't * exceed S32_MAX / PAGE_SIZE. */ - BUILD_BUG_ON(MEMCG_CHARGE_BATCH > S32_MAX / PAGE_SIZE); + BUILD_BUG_ON(MEMCG_CHARGE_BATCH > S32_MAX / PAGE_SIZE_MIN); cpuhp_setup_state_nocalls(CPUHP_MM_MEMCQ_DEAD, "mm/memctrl:dead", NULL, memcg_hotplug_cpu_dead); diff --git a/mm/memory.c b/mm/memory.c index ebfc9768f801a..14b5ef6870486 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4949,8 +4949,8 @@ vm_fault_t finish_fault(struct vm_fault *vmf) return ret; } -static unsigned long fault_around_pages __read_mostly = - 65536 >> PAGE_SHIFT; +static __DEFINE_GLOBAL_PAGE_SIZE_VAR(unsigned long, fault_around_pages, + __read_mostly, 65536 >> PAGE_SHIFT); #ifdef CONFIG_DEBUG_FS static int fault_around_bytes_get(void *data, u64 *val) diff --git a/mm/mmap.c b/mm/mmap.c index d0dfc85b209bb..d9642aba07ac4 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2279,7 +2279,7 @@ int expand_downwards(struct vm_area_struct *vma, unsigned long address) } /* enforced gap between the expanding stack and other mappings. */ -unsigned long stack_guard_gap = 256UL< PFN_SECTION_SHIFT); + BUILD_BUG_ON(SECTION_MAP_LAST_BIT > PFN_SECTION_SHIFT_MIN); BUG_ON(coded_mem_map & ~SECTION_MAP_MASK); return coded_mem_map; } diff --git a/mm/swapfile.c b/mm/swapfile.c index 38bdc439651ac..6311a1cc7e46b 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -2931,7 +2931,7 @@ static unsigned long read_swap_header(struct swap_info_struct *p, unsigned long swapfilepages; unsigned long last_page; - if (memcmp("SWAPSPACE2", swap_header->magic.magic, 10)) { + if (memcmp("SWAPSPACE2", &swap_header->magic[SWAP_HEADER_MAGIC], 10)) { pr_err("Unable to find swap-space signature\n"); return 0; } diff --git a/mm/vmalloc.c b/mm/vmalloc.c index a0df1e2e155a8..b4fbba204603c 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2497,12 +2497,12 @@ struct vmap_block { spinlock_t lock; struct vmap_area *va; unsigned long free, dirty; - DECLARE_BITMAP(used_map, VMAP_BBMAP_BITS); unsigned long dirty_min, dirty_max; /*< dirty range */ struct list_head free_list; struct rcu_head rcu_head; struct list_head purge; unsigned int cpu; + unsigned long used_map[]; }; /* Queue of free and dirty vmap blocks, for allocation and flushing purposes */ @@ -2600,11 +2600,12 @@ static void *new_vmap_block(unsigned int order, gfp_t gfp_mask) unsigned long vb_idx; int node, err; void *vaddr; + size_t size; node = numa_node_id(); - vb = kmalloc_node(sizeof(struct vmap_block), - gfp_mask & GFP_RECLAIM_MASK, node); + size = struct_size(vb, used_map, BITS_TO_LONGS(VMAP_BBMAP_BITS)); + vb = kmalloc_node(size, gfp_mask & GFP_RECLAIM_MASK, node); if (unlikely(!vb)) return ERR_PTR(-ENOMEM); From patchwork Mon Oct 14 10:58:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A62CED1A43B for ; Mon, 14 Oct 2024 10:59:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 381DB6B008C; Mon, 14 Oct 2024 06:59:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 333C36B0092; Mon, 14 Oct 2024 06:59:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1D27B6B0093; Mon, 14 Oct 2024 06:59:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 01EAC6B008C for ; Mon, 14 Oct 2024 06:59:51 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 4133F1C611C for ; Mon, 14 Oct 2024 10:59:43 +0000 (UTC) X-FDA: 82671912330.07.02965D4 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf10.hostedemail.com (Postfix) with ESMTP id 50955C0019 for ; Mon, 14 Oct 2024 10:59:47 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf10.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903449; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hXo3oIPvtAqxU/+rzneWWG3oIuEw/H2AD2/H28YBHxs=; b=OUkC3mubqjMAInnftgsSuXrzGNuiE84C7meaS2nr4L5JY24sIbqQC6DanW3Gb7p8y5EZOk zKTZLRHIIr998bdxVUuNzemC4Yj2NkU0A4ZH486STP1lvl+h2JZYPlb/q4XtDk38gs/UYY ywHpcNq65d4lED8s4HwDf9mEMTQCWys= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903449; a=rsa-sha256; cv=none; b=yroq/1/nO1YUEQDkvFkxmd9mAGKaj2+bvAHTlhwEwN2ERV0d3JLUeH4avGJExmpthYfnfV wTmttBxObfIj3fd5knoT2r7Y6bAUGiqGdYnxJtkdzU3WD/ohmCLQ/yvp6TyjOhikGEar/f nuDk1rGWn3Py7WUwQs8/lliw5ivvpHA= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf10.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 130DB1684; Mon, 14 Oct 2024 04:00:19 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2FB0B3F51B; Mon, 14 Oct 2024 03:59:46 -0700 (PDT) From: Ryan Roberts To: "Theodore Ts'o" , Alexander Viro , Andreas Dilger , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , Christian Brauner , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , OGAWA Hirofumi , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 07/57] fs: Introduce MAX_BUF_PER_PAGE_SIZE_MAX for array sizing Date: Mon, 14 Oct 2024 11:58:14 +0100 Message-ID: <20241014105912.3207374-7-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 50955C0019 X-Stat-Signature: xpzgjsybu41xc76tpjjnoryj9ngf1dni X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1728903587-779265 X-HE-Meta: U2FsdGVkX1+Rw+GFacgjdkFf4xXeOtf3yzck3geUXn76k3y0dPjlF9aTjw5ybeCjL9JfDQIE5T8424jdUyIhV/u4/atgyt+4qpdraQ9/iYNq9Tc4aIFm6tg3K/owE4sg2tWDVcYIMdbtTY3XYnpTbV5hKH2q2OL8317+AXf1tFy5NF16eV9zUWenx8u9iWX4uQTTWy8S5NgqsKtiLO8uvq6+S2E8TJk/K7x80e2b0ki0aDBHjg3HMy/EW4YAPHs97OGDBO+GPLaeWxqaMs5+3VB0D60liY99Z5/aKyM5XgMl99/99VxgQgK/CmVJi/2ja67TsFhsIq4fLJzPs1sKpIJUtYYj4kGn8UY8dlDk5R3KDDEfP4SChQTB+C4Rga+HXLhiaa2j13vWEc4J4+f+u0OMXd2b+ed3zzMD+97dYd5P+8QKemZpOeTn3sEO1V/ATIorag5aGABjL9Mf0LOsiA1C9oEYS9DyglZmttA/D+EVgcM1s4eQfa/BP06qU8YxZwrPrd+Tf5EfO7Y069RAwn+0lYkKOG5SAZJoZBlk3B9DanP/cDMxG4gC0hoPetxGI5eWjh39O1PaZvhQiAyavzZhNmzgpB+ZXt41iC9CoPdXsScXOodwcTAoQMZvvya246H6SSwuBnZw7ti+3Mo4Q29Y86VagXFoFM3BbsoBmuYiNrmcmXyiLSIeHzRkWXQvynYFcwWOUPo4DsAc2ETBmKdu2DHqTVLQnCgNSLriEslV7omv785kOvy6k76j5q4OIewfu7bZQC0jbRjgZ1HMcYdAWbh311rE07FLKMXmOZglXrG2XrjNsiW/2PUvmpG6WUq1EHccdxCX7ZSsZ8vU4sS2gUfy6/v6uvKWVU/QJcCjTumHOtnzeKV+La5L2XVp1t3bLCWujhUMxyvEFvS8aYBZWUhyN8jHgCaxjjMTYd4OApXMz6iqMTuMIPqpx7v2VSlRBaJfmmv7GLhST7r MFcr13RK 2ansm4/y9gZki0vu3G2KBnEV3pCK43PrrxGvaIg5l7JbPtm7ZC6ISTRmQvxjW8oE9Uk6/NzrLl/69SvLyzgHQAxeAVQKzQAiAz55wwLr/hGTAuOCV3wLx1KYhl+wWJSocx+jIA8mouWXSPgaW9YgpTrkTfyVYJdUBUh7Xz1grw9K39p/HNMc7JIjaw6gJNGcQsx/gNdjndMfbOaUvMXzgiCu58uwxhmW5ONgX X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Code that previously defined arrays with MAX_BUF_PER_PAGE will no longer work with boot-time page selection because PAGE_SIZE is not known at compile-time. Introduce MAX_BUF_PER_PAGE_SIZE_MAX for this purpose, which is the requirement in the limit when PAGE_SIZE_MAX is the selected page size. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ fs/buffer.c | 2 +- fs/ext4/move_extent.c | 2 +- fs/ext4/readpage.c | 2 +- fs/fat/dir.c | 4 ++-- fs/fat/fatent.c | 4 ++-- include/linux/buffer_head.h | 1 + 6 files changed, 8 insertions(+), 7 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index e55ad471c5306..f00542ad43a5c 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -2371,7 +2371,7 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) { struct inode *inode = folio->mapping->host; sector_t iblock, lblock; - struct buffer_head *bh, *head, *arr[MAX_BUF_PER_PAGE]; + struct buffer_head *bh, *head, *arr[MAX_BUF_PER_PAGE_SIZE_MAX]; size_t blocksize; int nr, i; int fully_mapped = 1; diff --git a/fs/ext4/move_extent.c b/fs/ext4/move_extent.c index 204f53b236229..68304426c6f45 100644 --- a/fs/ext4/move_extent.c +++ b/fs/ext4/move_extent.c @@ -172,7 +172,7 @@ mext_page_mkuptodate(struct folio *folio, unsigned from, unsigned to) { struct inode *inode = folio->mapping->host; sector_t block; - struct buffer_head *bh, *head, *arr[MAX_BUF_PER_PAGE]; + struct buffer_head *bh, *head, *arr[MAX_BUF_PER_PAGE_SIZE_MAX]; unsigned int blocksize, block_start, block_end; int i, err, nr = 0, partial = 0; BUG_ON(!folio_test_locked(folio)); diff --git a/fs/ext4/readpage.c b/fs/ext4/readpage.c index 8494492582abe..5808d85096aeb 100644 --- a/fs/ext4/readpage.c +++ b/fs/ext4/readpage.c @@ -221,7 +221,7 @@ int ext4_mpage_readpages(struct inode *inode, sector_t block_in_file; sector_t last_block; sector_t last_block_in_file; - sector_t blocks[MAX_BUF_PER_PAGE]; + sector_t blocks[MAX_BUF_PER_PAGE_SIZE_MAX]; unsigned page_block; struct block_device *bdev = inode->i_sb->s_bdev; int length; diff --git a/fs/fat/dir.c b/fs/fat/dir.c index acbec5bdd5210..f3e96ecf21c92 100644 --- a/fs/fat/dir.c +++ b/fs/fat/dir.c @@ -1146,7 +1146,7 @@ int fat_alloc_new_dir(struct inode *dir, struct timespec64 *ts) { struct super_block *sb = dir->i_sb; struct msdos_sb_info *sbi = MSDOS_SB(sb); - struct buffer_head *bhs[MAX_BUF_PER_PAGE]; + struct buffer_head *bhs[MAX_BUF_PER_PAGE_SIZE_MAX]; struct msdos_dir_entry *de; sector_t blknr; __le16 date, time; @@ -1213,7 +1213,7 @@ static int fat_add_new_entries(struct inode *dir, void *slots, int nr_slots, { struct super_block *sb = dir->i_sb; struct msdos_sb_info *sbi = MSDOS_SB(sb); - struct buffer_head *bhs[MAX_BUF_PER_PAGE]; + struct buffer_head *bhs[MAX_BUF_PER_PAGE_SIZE_MAX]; sector_t blknr, start_blknr, last_blknr; unsigned long size, copy; int err, i, n, offset, cluster[2]; diff --git a/fs/fat/fatent.c b/fs/fat/fatent.c index 1db348f8f887a..322cf5b8e5590 100644 --- a/fs/fat/fatent.c +++ b/fs/fat/fatent.c @@ -469,7 +469,7 @@ int fat_alloc_clusters(struct inode *inode, int *cluster, int nr_cluster) struct msdos_sb_info *sbi = MSDOS_SB(sb); const struct fatent_operations *ops = sbi->fatent_ops; struct fat_entry fatent, prev_ent; - struct buffer_head *bhs[MAX_BUF_PER_PAGE]; + struct buffer_head *bhs[MAX_BUF_PER_PAGE_SIZE_MAX]; int i, count, err, nr_bhs, idx_clus; BUG_ON(nr_cluster > (MAX_BUF_PER_PAGE / 2)); /* fixed limit */ @@ -557,7 +557,7 @@ int fat_free_clusters(struct inode *inode, int cluster) struct msdos_sb_info *sbi = MSDOS_SB(sb); const struct fatent_operations *ops = sbi->fatent_ops; struct fat_entry fatent; - struct buffer_head *bhs[MAX_BUF_PER_PAGE]; + struct buffer_head *bhs[MAX_BUF_PER_PAGE_SIZE_MAX]; int i, err, nr_bhs; int first_cl = cluster, dirty_fsinfo = 0; diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h index 14acf1bbe0ce6..5dff4837b76cd 100644 --- a/include/linux/buffer_head.h +++ b/include/linux/buffer_head.h @@ -41,6 +41,7 @@ enum bh_state_bits { }; #define MAX_BUF_PER_PAGE (PAGE_SIZE / 512) +#define MAX_BUF_PER_PAGE_SIZE_MAX (PAGE_SIZE_MAX / 512) struct page; struct buffer_head; From patchwork Mon Oct 14 10:58:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834676 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0573CD1A43B for ; Mon, 14 Oct 2024 10:59:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8FFDE6B0092; Mon, 14 Oct 2024 06:59:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B1D16B0093; Mon, 14 Oct 2024 06:59:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 778446B0095; Mon, 14 Oct 2024 06:59:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5CC306B0092 for ; Mon, 14 Oct 2024 06:59:55 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id CCBE9C0D43 for ; Mon, 14 Oct 2024 10:59:46 +0000 (UTC) X-FDA: 82671912666.29.F0E4F18 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf28.hostedemail.com (Postfix) with ESMTP id 5B20EC0009 for ; Mon, 14 Oct 2024 10:59:47 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf28.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903547; a=rsa-sha256; cv=none; b=E0MrBsrsB4ukwC0jdeDcwlVJ9isM1tGdrijQ5Dl+JJSeWJ6xygfr5GtFNa65tySiGXRIad W0sOYEJZYQgmViwiaQimqy1amoiiG6M6naq0QgtvCMTE6WqLmoTgWCbpNogW/26mfClk7W YJWq9k/mKa/MRB73vcXaWyMw6rNMD0o= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf28.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903547; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dafiuhHFBOL0KU3UgDvSqFL+BWETMThHwzJyb2+CQGw=; b=TVaSU9Ozh3J2L+BA+pwDjSm0NTrS3Sy+RcUP88TcI/hD0wp9PM4s3F7MA31vLlPwcz8BJ1 88yyzj1/Ap5AQ43t7kVR7QEY78vgZpZeObXY3w5LwITiJSyZE5wbPCM7LWi5fEijj/8VSZ pMOKghNBRBCYAw+adr3guQPV/ciYmlw= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1FE2A1688; Mon, 14 Oct 2024 04:00:22 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B86113F51B; Mon, 14 Oct 2024 03:59:49 -0700 (PDT) From: Ryan Roberts To: Alexander Viro , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , Christian Brauner , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 08/57] fs: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:15 +0100 Message-ID: <20241014105912.3207374-8-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 5B20EC0009 X-Rspamd-Server: rspam01 X-Stat-Signature: n9ucenmxzf96i3csbyijn55acizczpa6 X-HE-Tag: 1728903587-520758 X-HE-Meta: U2FsdGVkX1/O4Fy1DQIQzz6ZwRqD48DzR77iUazOhkQCnQACJFFxb0VAU4hLMmMJ2o3iUPMCRSy7em0V+B+zkN4JTM9LVoG/FL02RcjSPtCqvREZHafL56R6CMPDTq76McPm25q937QcXSqfFGjdRo+jdy9coKB26aU/Ly13+gLI2e6DfAujrT7NBrdeeuD3dhb1wJa3d4n7U/pZs0NJZYd4OodS0T9GghhxJPbrIXJ8TRHYcGFe3YGHpjLfuDybNasZdjmdxZb+h7od3OFQ/DAfPefue424AhEIwQJOw1OOOcr1/9h/T6sruhrEoixVhX0aoYICPFKU5cl+BsxeeyOs1pee/yRwdR8ih26Kr9kbaE8d5KPgWuc0HQpyqmQaWvuHX74PmoHkxir147k9EP8ULn22ce2qyIKLcaT16qF4OVE/Y4EYeXzwrymRcNKtM1Fieq2apxSfgdRjurQegs5Oh9cfLE6Z4p8NBEa+6UdozXojb3UKltLtY2F5/LWP9HFArVj9g95Ss9VlS3YkRpJc42vFFVXeAu5JjvMSn9hsKHO8dsLuuJNSFJs28FYZSON6cWPJ/+wXthMQQoFFusVk2XnPhpIcWIUYTFOTvv9/4/+zcTp8oPDzGtZK3EDsrHAm5x4Fl9LklfAlvXDtuiV0RwviWT0qCGn7e0asgLAD86I68GrqyNSRZyJpYBcSFTkG5MHQnwA/c3KWccBc0peS0ipDiXtwgVEnSwAOafCJnfsLt7dNXggHpidX3ELwG4ezHWWSKqH2Od8IFMd1GeZu91TAa7jyiRmnB2md4+fhe4J7JJNhzbMDjSt6wt+X1BMEa8hxXnlBdBcLTyF5w3OeY5WAgoaP08BRTVQxHQmuVJwGXZ05TV+qTSeDTv3zAtqmku8ncx1NriVnM6rVO0H5VcYuSXlzfF596/CpHywkixfoLf/imNKufK3ydz+RqS+QRscEZJkef+72mHl C1PtIfZh OoKAwmywTyda/reD22oE4cn1L86boj537MUVjZFkrFsHf+8T57r9oY4ZpmEPe0YnrCTmGqPKG1IlDK1/IDpPCb5eu3KdSKkkVMRJbLULaoIdOat8lfOOrGmF+5s4ug9q93R6gj9AcwnS6euySQbUGX5iS1tQCbDQ0EoliELKqdXiW1HP3kE3RGc9oPAXr5E8IQeqoeMCbxUN0QfVwxzaTGFDriAGj6W9KIUlG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. In binfmt_elf, convert CPP conditional to C ternary operator; this will be folded to the same code by the compiler when in compile-time page size mode, but will also work for runtime evaluation in boot-time page size mode. In coredump, modify __dump_skip() to emit zeros in blocks of PAGE_SIZE_MIN. This resolves to the previous PAGE_SIZE for compile-time page size, but that doesn't work for boot-time page size. PAGE_SIZE_MIN is preferred here over PAGE_SIZE_MAX to save memory. Wrap global variables that are initialized with PAGE_SIZE derived values using DEFINE_GLOBAL_PAGE_SIZE_VAR() so their initialization can be deferred for boot-time page size builds. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ fs/binfmt_elf.c | 11 ++++------- fs/coredump.c | 8 ++++---- 2 files changed, 8 insertions(+), 11 deletions(-) diff --git a/fs/binfmt_elf.c b/fs/binfmt_elf.c index 19fa49cd9907f..e439d36c43c7e 100644 --- a/fs/binfmt_elf.c +++ b/fs/binfmt_elf.c @@ -84,11 +84,8 @@ static int elf_core_dump(struct coredump_params *cprm); #define elf_core_dump NULL #endif -#if ELF_EXEC_PAGESIZE > PAGE_SIZE -#define ELF_MIN_ALIGN ELF_EXEC_PAGESIZE -#else -#define ELF_MIN_ALIGN PAGE_SIZE -#endif +#define ELF_MIN_ALIGN \ + (ELF_EXEC_PAGESIZE > PAGE_SIZE ? ELF_EXEC_PAGESIZE : PAGE_SIZE) #ifndef ELF_CORE_EFLAGS #define ELF_CORE_EFLAGS 0 @@ -98,7 +95,7 @@ static int elf_core_dump(struct coredump_params *cprm); #define ELF_PAGEOFFSET(_v) ((_v) & (ELF_MIN_ALIGN-1)) #define ELF_PAGEALIGN(_v) (((_v) + ELF_MIN_ALIGN - 1) & ~(ELF_MIN_ALIGN - 1)) -static struct linux_binfmt elf_format = { +static DEFINE_GLOBAL_PAGE_SIZE_VAR(struct linux_binfmt, elf_format, { .module = THIS_MODULE, .load_binary = load_elf_binary, .load_shlib = load_elf_library, @@ -106,7 +103,7 @@ static struct linux_binfmt elf_format = { .core_dump = elf_core_dump, .min_coredump = ELF_EXEC_PAGESIZE, #endif -}; +}); #define BAD_ADDR(x) (unlikely((unsigned long)(x) >= TASK_SIZE)) diff --git a/fs/coredump.c b/fs/coredump.c index 7f12ff6ad1d3e..203f2a158246e 100644 --- a/fs/coredump.c +++ b/fs/coredump.c @@ -825,7 +825,7 @@ static int __dump_emit(struct coredump_params *cprm, const void *addr, int nr) static int __dump_skip(struct coredump_params *cprm, size_t nr) { - static char zeroes[PAGE_SIZE]; + static char zeroes[PAGE_SIZE_MIN]; struct file *file = cprm->file; if (file->f_mode & FMODE_LSEEK) { if (dump_interrupted() || @@ -834,10 +834,10 @@ static int __dump_skip(struct coredump_params *cprm, size_t nr) cprm->pos += nr; return 1; } else { - while (nr > PAGE_SIZE) { - if (!__dump_emit(cprm, zeroes, PAGE_SIZE)) + while (nr > PAGE_SIZE_MIN) { + if (!__dump_emit(cprm, zeroes, PAGE_SIZE_MIN)) return 0; - nr -= PAGE_SIZE; + nr -= PAGE_SIZE_MIN; } return __dump_emit(cprm, zeroes, nr); } From patchwork Mon Oct 14 10:58:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834677 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0D5CD1A43B for ; Mon, 14 Oct 2024 10:59:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3F83E6B0093; Mon, 14 Oct 2024 06:59:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 35A966B0095; Mon, 14 Oct 2024 06:59:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 221BD6B0096; Mon, 14 Oct 2024 06:59:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id F38026B0093 for ; Mon, 14 Oct 2024 06:59:57 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 3A00C1C6129 for ; Mon, 14 Oct 2024 10:59:49 +0000 (UTC) X-FDA: 82671912792.29.10001B9 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf15.hostedemail.com (Postfix) with ESMTP id 2B8B9A0004 for ; Mon, 14 Oct 2024 10:59:49 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf15.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903565; a=rsa-sha256; cv=none; b=Zft4J5m2Y15+KOmy7Am64OHuLbPy00/f4ymWTRQyrZ9gXylUrMZMEkBCwPOYoMnFqOWq/B ee9MVPih1HG1cW5o6jaCv0V7aPJXaObeqoU9zXF+NIhsY84S2xm7XPvhQMVXP8ilKRkMjn FLv+JDGVWT8FGjKMKc5/FR8oaJkkKgo= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf15.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903565; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HQvPCF1x2W8nC78duvmTyHGJn39OjXD+rSjYpslxs/Y=; b=dzxeM5yQbQ24yW/QI8VQXIwua1hI7/cq1i0A35CU0iq3ihz5mQe2VxotxWVPbXuRYPTPi9 rjccB0x413eGIFUw+psTeg8jjj9BqKmchyO1T6TRePSEmB2mr2CW48Al9iORbW7GjuIDvV KS7qY5tNv/CBXetdpFdRk3k/XkvDpxI= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2D6B51424; Mon, 14 Oct 2024 04:00:25 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C5CEC3F51B; Mon, 14 Oct 2024 03:59:52 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anna Schumaker , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Trond Myklebust , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-nfs@vger.kernel.org Subject: [RFC PATCH v1 09/57] fs/nfs: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:16 +0100 Message-ID: <20241014105912.3207374-9-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: tiurocajj8bihqejywr5nm3xzci1ha8z X-Rspamd-Queue-Id: 2B8B9A0004 X-Rspamd-Server: rspam02 X-HE-Tag: 1728903589-944605 X-HE-Meta: U2FsdGVkX1/+4W2M9F4X6ry2bq345IO+q/Y6g3wLl8efUdEp+y5+rGcQ8INdTkT1fCZ/5bddNCMPG9SsxR0NdhawG62+RMOadtNLrrS0oc8Ii9+44f+zKGCZV0zrJKQF3Gn3TOeoeV/RD0uIb8NaHSCXb8qD8JTWK6GvyT0krtqCSf1JCDe1dhsEHiVHoa0c+RhM7rAu12EKiVeENNK6GMFwwqFddTfK9IQ/yBHg86WyoQyiXXfC9CsTo9EGPITSOi3n42DGIDJRKydKKFj8Xz9sVGFslJows07ZKOs795+HgvnkRFlRin/jpPZZk1NfJClkt/8/9gLNi6k8eLIAsRFXPyN7OXWcIdfLzFViSRuwAHFQQ+fGRmJq51Q8UPQmJ217MKxJbcjbwuHA7nyyigQVEa94N59/PjheZcgJ5wJhrV8a2GDPKYZgPUh1VVA2WN33RWTpjMkoEAI2wEaAuXMvHyicpA61K8C133uoZEM1CPIQQLbezdeC+NiwMbKxHl3aHYK5LQo+NQvRjlwmsyjncgEb+v2R4J2LYKv1FWxMuul/LWWLWC/1WaHDcijvFBsODxADoGaEWuLaXvXUTpw4+dNfvhHxo5469PhYrKcwrV0l4XZxU3PPrLQe7liC8esgQ99+OCuTFI9npZtkCjAVk34Yceki2S72mEj1h+D+hpIwLGs2fquxDun31QyyVjyoICl9QSVzHUR/eBqWQB24sPLiQBMYF+Huv/gPYjBeFxxnGrcaXqYGedJWRdHpnRWolyCuZqYD44d+nJZAllsTo7dbBTzO7TpvnE6gKriWHnu04AtZZCmBxqnUtp2kGb4OX3Zn95U09glFgENOY/3Xi4hUuMWQL4XqPh2z8dx0ZO/HTaG8LKOoI413HvqXo1AtjB3bsBN5pPww0Zd2JsrC9bKJiWg53x0wsQlR0wOtvAu9/MrNSNv5/JBdSiOcXe4QHlmLKbFuLcAZ0Sa rz5jFsoL kHPs3MQ9C6H6ZMY8lm3gYxKd8Z8nUbEMFAefIrot8T2LSt6rpaapzY5AZw7/j7X4WLFKC0Lx1ueBceHzU7k4JD/U3OwuheI6rY1YfrbaaZNNJL3/1WMeDFCPFXLGAlfrTTFi+WrA5ywWIz8PMfMmaL+bKLGM9p7EFye0n68yNEb6IOxmWa0p1mXnHSYDndxwJyMZkjF13RcSPJqf9WJDfk3m3A4tenRsOGRLh X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Calculation of NFS4ACL_MAXPAGES and NFS4XATTR_MAXPAGES are modified to give max pages when page size is at the minimum. BUILD_BUG_ON() is modified to test against the min page size, which implicitly also applies to all other page sizes. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ fs/nfs/nfs42proc.c | 2 +- fs/nfs/nfs42xattr.c | 2 +- fs/nfs/nfs4proc.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/fs/nfs/nfs42proc.c b/fs/nfs/nfs42proc.c index 28704f924612c..c600574105c63 100644 --- a/fs/nfs/nfs42proc.c +++ b/fs/nfs/nfs42proc.c @@ -1161,7 +1161,7 @@ int nfs42_proc_clone(struct file *src_f, struct file *dst_f, return err; } -#define NFS4XATTR_MAXPAGES DIV_ROUND_UP(XATTR_SIZE_MAX, PAGE_SIZE) +#define NFS4XATTR_MAXPAGES DIV_ROUND_UP(XATTR_SIZE_MAX, PAGE_SIZE_MIN) static int _nfs42_proc_removexattr(struct inode *inode, const char *name) { diff --git a/fs/nfs/nfs42xattr.c b/fs/nfs/nfs42xattr.c index b6e3d8f77b910..734177eb44889 100644 --- a/fs/nfs/nfs42xattr.c +++ b/fs/nfs/nfs42xattr.c @@ -183,7 +183,7 @@ nfs4_xattr_alloc_entry(const char *name, const void *value, uint32_t flags; BUILD_BUG_ON(sizeof(struct nfs4_xattr_entry) + - XATTR_NAME_MAX + 1 > PAGE_SIZE); + XATTR_NAME_MAX + 1 > PAGE_SIZE_MIN); alloclen = sizeof(struct nfs4_xattr_entry); if (name != NULL) { diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c index b8ffbe52ba15a..3c3622f46d3e0 100644 --- a/fs/nfs/nfs4proc.c +++ b/fs/nfs/nfs4proc.c @@ -5928,7 +5928,7 @@ static bool nfs4_server_supports_acls(const struct nfs_server *server, * it's OK to put sizeof(void) * (XATTR_SIZE_MAX/PAGE_SIZE) bytes on * the stack. */ -#define NFS4ACL_MAXPAGES DIV_ROUND_UP(XATTR_SIZE_MAX, PAGE_SIZE) +#define NFS4ACL_MAXPAGES DIV_ROUND_UP(XATTR_SIZE_MAX, PAGE_SIZE_MIN) int nfs4_buf_to_pages_noslab(const void *buf, size_t buflen, struct page **pages) From patchwork Mon Oct 14 10:58:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834678 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAC15D1A43B for ; Mon, 14 Oct 2024 11:00:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72AE86B0095; Mon, 14 Oct 2024 07:00:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6DB1B6B0096; Mon, 14 Oct 2024 07:00:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57C186B0098; Mon, 14 Oct 2024 07:00:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3BDFF6B0095 for ; Mon, 14 Oct 2024 07:00:01 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8BF7E160CDD for ; Mon, 14 Oct 2024 10:59:52 +0000 (UTC) X-FDA: 82671912876.04.90F950C Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf08.hostedemail.com (Postfix) with ESMTP id 1AA26160003 for ; Mon, 14 Oct 2024 10:59:54 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903553; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4YDRe8auf3W9gPGvw4mfS6WcApteYPOWnCg7H4VxM6g=; b=Xy/GJIOLMR3+ESwEA7yjyw4hl8/O8mAcUuQkMJq9RHpqOyye2XyMs00eo2X2DAZJVOFjkW 8FDKUHTihEzQXNh85XFiiuf9pZRypm1NingmKObTsnmY73CJx8w/eYKgZEVxKLlY/r2YWi R50s8E/+MZEwtWOheKM4PomsQrvOIUY= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903553; a=rsa-sha256; cv=none; b=KSR/4LC0f+Vadz6+MxJd15kMJTIZ+nKrBBk2ZnHWlS19cJoSEFGJWukoRyvufJZsfyuXHp WBQ+JIEiCJTm4VOOqPqFv+n41+eqkdHMBJRTEvYlcOxdQ80VFlBIcEya/2XY4ZDT0v9vNG CVqi+RCTGHvbT/HG9xdqS4OUsL5zNY0= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 432CD1684; Mon, 14 Oct 2024 04:00:28 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D33E63F51B; Mon, 14 Oct 2024 03:59:55 -0700 (PDT) From: Ryan Roberts To: "Theodore Ts'o" , Andreas Dilger , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-ext4@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 10/57] fs/ext4: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:17 +0100 Message-ID: <20241014105912.3207374-10-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: mf1fu3pzchctmf5u9d4tsuyatpptnqqc X-Rspamd-Queue-Id: 1AA26160003 X-Rspamd-Server: rspam11 X-HE-Tag: 1728903594-229930 X-HE-Meta: U2FsdGVkX19/sl/0FAs5cU+0IkxrvePIOTBNZowi3Q06cOY7j6x3Hagrwlk9J/qmGIJXG2fbTINSVzeSJ8GHazh0NXyLMBBH+WvrrLqZ7D77PhZJAzacoMYN+Y1bPvz78W+hvKt1QWWcCcy2q8PFbyXE63Gyu8guxIYE9caW2M5f8eE7iwFnpVtDwZwL0duua4oLabQ9cfth+Zg+KWIGhKX4WuqGEX8VKyjN5RZfmgrl06k8auwL7AqoTGQ51aJPN6zI0cStmekZTnO/0mZOceASwevF9XFiBD4l31/GQ3WBaKRnTrb/aPD0lURZ+hYuzfnYpDge6s61yFD+MQcckpdz6joP2S1RGZfWFs6Fo6qN8ZgiPa2BbyhtWZZOcQHVDg0AHzxw3VQg8dUmXKp14O2i3P8v0Jlc8EIIx6tOFl7NVgj2rAVC6JG6UAxNzQ8bXcxt1hGRZaHCzp5XlQb+EKx5Z2B2v0EHKWkCVuczI6BnOroEhnyX8Wg31KZFrsPF6GbS+EPVeBjgX/MZ4qQZqA3tqQdLG/vkbW1Xri/TMi1JxYcaP2gvhnPrIm+JDptfOa99lT8cwQc4FOSVVF/eqvGPJXOxoIs8HI8Db0ZSXojyojAAzDDLfNTXdu4pC5XiCNyRAZ9/IY4CypFshr2sOhV1/mOekTE3xTWV/UeeCzljB3ugPtqdOPsxlWG7Pam7FJXYNh1/m7iPgP/Gg+V8n5c1jSIrH/hXnJCduF/j8UAb6u379ETdGAwT/HHEglnN3dg+spqM0SQ+oF8bhi3DEWaogZqRiiMdHtH5YCtdt4vQBpQ4+q7KFMeDYZS16szX71H5qpFwVXowgN2Bz7TfuwZGvfy5vp+Zk/ylrlB5H1yUVeM6uY1LFcik1CXOUTjB3ju8zho6mtDaNFh1JzatAv5c78eCuAVdFR+0gIS+2qNvUiuggu5DfEQXdX7h4T6Hfq20euNi4O9PT1DdMpr /46Jd2pN TM5kz1uMVjKximJAKqf+vEhXxOWwBCbUH8OLUk4btcv4kxtwaldVfcm5VSMYIDux1I9ypsWM4BfvAQILu9o65i7+13Omus7rdgidkBSTaBS7MxUbq+/GfVFP4jIjvcNZrw8y42DgxNdoJJ9ypN7o3TLctsiGcvWwleme+dJN/mrBGAc+oKIfZwMGu655Bx/PS+h+zD1zWM9UGlIz69pUW6aXoiD+rg/isC0mz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Convert CPP PAGE_SIZE conditionals to C if/else. For compile-time page size, the compiler will strip the dead part, and for boot-time page size, the condition will be evaluated at run time. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ fs/ext4/ext4.h | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index 08acd152261ed..1a6dbd925024a 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -2415,31 +2415,31 @@ ext4_rec_len_from_disk(__le16 dlen, unsigned blocksize) { unsigned len = le16_to_cpu(dlen); -#if (PAGE_SIZE >= 65536) - if (len == EXT4_MAX_REC_LEN || len == 0) - return blocksize; - return (len & 65532) | ((len & 3) << 16); -#else - return len; -#endif + if (PAGE_SIZE >= 65536) { + if (len == EXT4_MAX_REC_LEN || len == 0) + return blocksize; + return (len & 65532) | ((len & 3) << 16); + } else { + return len; + } } static inline __le16 ext4_rec_len_to_disk(unsigned len, unsigned blocksize) { BUG_ON((len > blocksize) || (blocksize > (1 << 18)) || (len & 3)); -#if (PAGE_SIZE >= 65536) - if (len < 65536) + if (PAGE_SIZE >= 65536) { + if (len < 65536) + return cpu_to_le16(len); + if (len == blocksize) { + if (blocksize == 65536) + return cpu_to_le16(EXT4_MAX_REC_LEN); + else + return cpu_to_le16(0); + } + return cpu_to_le16((len & 65532) | ((len >> 16) & 3)); + } else { return cpu_to_le16(len); - if (len == blocksize) { - if (blocksize == 65536) - return cpu_to_le16(EXT4_MAX_REC_LEN); - else - return cpu_to_le16(0); } - return cpu_to_le16((len & 65532) | ((len >> 16) & 3)); -#else - return cpu_to_le16(len); -#endif } /* From patchwork Mon Oct 14 10:58:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834679 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D7ACD1A443 for ; Mon, 14 Oct 2024 11:00:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 334676B0085; Mon, 14 Oct 2024 07:00:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 30B436B0096; Mon, 14 Oct 2024 07:00:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1ACA76B0098; Mon, 14 Oct 2024 07:00:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id F29876B0085 for ; Mon, 14 Oct 2024 07:00:04 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id CA84E120D73 for ; Mon, 14 Oct 2024 10:59:57 +0000 (UTC) X-FDA: 82671912876.07.14B9DDB Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf08.hostedemail.com (Postfix) with ESMTP id BDD1E160003 for ; Mon, 14 Oct 2024 10:59:58 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf08.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903462; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B2YC/X3UwnXWGexJl2uM83krn8GDdx7g25hAIge8UU4=; b=4hxsbNDiNXIMzSj3Ubpr3Pz/4Lopf4bfjfJ4lOHEO2U95mj0qvmqqeQP9TZ7ldNk/KdzxI CmZ73qFAomC8Z+YKU/KtVqS5ERDXj1O4yKXrBwrY7uklOPiTGLjU5Z9euHXHo4Y9IL4ljU jDV5Dkz46LrjHNB9EfXPx7X1BPNRJEI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903462; a=rsa-sha256; cv=none; b=LgSb7WenxwobXVIXLAkMPCzXa73iI/K/Hkoj+6ev+1KdOyHclLU1VvYDq+B3FLveGu2rCX h3M+XcdgtJL6OmCPq5vQ0nJSJrUvRd0cLvQWv99jRzlaGBe1QMUaOupsAt0GyCDSxKJt63 uoIkrQkkXJn5IFVeDDACcadetFSjL5A= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf08.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E3B391424; Mon, 14 Oct 2024 04:00:31 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E14783F51B; Mon, 14 Oct 2024 03:59:58 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Andrey Ryabinin , Anshuman Khandual , Ard Biesheuvel , Arnd Bergmann , Catalin Marinas , David Hildenbrand , Greg Marsden , Ingo Molnar , Ivan Ivanov , Juri Lelli , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Peter Zijlstra , Vincent Guittot , Will Deacon Cc: Ryan Roberts , kasan-dev@googlegroups.com, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 11/57] fork: Permit boot-time THREAD_SIZE determination Date: Mon, 14 Oct 2024 11:58:18 +0100 Message-ID: <20241014105912.3207374-11-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: BDD1E160003 X-Stat-Signature: 7quwmh649fjh9wqhx1r6wbas9wyb5cz4 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1728903598-940022 X-HE-Meta: U2FsdGVkX1/xROh/yNM4HtPpK4B3TFB2NXwFmk8h+c/v9UTU7YeIippLVfw54sQxzs2Dhm1dbCa3c3a4XhNiVe46/GW40kDax7fbGCXTpywPh1cfIcJHbi3QWy35X9a88B5d3GEWvdK3gqasQkyEwPVf1ah6/TqzDREziixExLVulPBl1uJ+Hv3v/25B27Ap6yeTlnkkfxUSTYy44tZA2lxt1CBg7nif6obl3TK6v2uEKI8gOqviaXOehIaMBV8I87ansnDptZv6ey3xLnh86y3OtmvdHKT5VYS9MplMOuxEnidAPemqPVYSlXuuh218pV1H2myWanz3H0VQljC8aFYMz4Mjg9f2yyC3BxVcFsM9ULkfq8TjyeWPl0/4nUHXI6jO5nDovx9ooqbuhgi9zVy+FY3bJglLTXi0YxxYl2nFkoS0m6QwxW0hUfp8OltOh7DMJs4IHqPVS19wrcp20ARkGa9TkBAY6GB7Dp8sQRpMZeE3BX+Gopshdmr9dtJhj62epMuLjUxxIIxv8TCIN0aE5x5fw4vrXJ338sKJ0+gMCRYbAEZYnXgZcQUVXnFUEyj4f5EBCUMA7g+XMHQLkqtoFAX7z6xOOhvGdn5lBJo4+DvXmRNZp2gZrt4IQJt8odbg8aY8u3nw2wq0dB+BjhcDzU7HCN98ce3iyyyHgGrCKLVWhNiwzbkHtWNhD22cCEVd6iGazpVHKdspYhxHxplOLw0P4q7832BdwSCKn8YtO7B5C7F7sYr3ZWYTwAFhCqdthhyKLxVrLh5DkAr8SKvXxtGFqDQmBQA3dGSTJ6stv5ksFWX/q5ynw6h1BPw7y4BkiwZgKoDFrcxCYY0vb7Llmiv5gI7l/pS+TMxbM1uWOO5HKQlOda6S29cUZ4/TwvTc2AYzvgkxMqCf8BbdZDa4E/LnV4TFfMOiAqJjPMb8hAJegNnjSAU3uF3inDaXr45/Q7wJQjqXXTQ7qfN 82z3ogUw 4N6vLum4pi4r+6wK6+KtRLDZXqAQV1BeFGI+NbrdL/0xbvWC3OqAm/QOzdquVKU+pLJ+oae1lOdQv0vbKMVGhmus2DPlzlMsnShufAqTSZ1uybaLjXYIAtvn7JJvTDkO3VLBeyYpayDLTzGRZOKpi8GcuvdUMhR+XAhb0qfLYcLJZU24TqvZOnNVknQ2Dyyn8Q+o9d2SJGS9ubPsy/5uEkA5rZs9XrH+CYasFktj2kZ10MiDYeZyvuwpe5Zw9TZJ4Thf+XHJjy7cxEXzFo7Wg/WhTzDQtL6G/hLONWcue8auDeds= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: THREAD_SIZE defines the size of a kernel thread stack. To date, it has been set at compile-time. However, when using vmap stacks, the size must be a multiple of PAGE_SIZE, and given we are in the process of supporting boot-time page size, we must also do the same for THREAD_SIZE. The alternative would be to define THREAD_SIZE for the largest supported page size, but this would waste memory when using a smaller page size. For example, arm64 requires THREAD_SIZE to be 16K, but when using 64K pages and a vmap stack, we must increase the size to 64K. If we required 64K when 4K or 16K page size was in use, we would waste 48K per kernel thread. So let's refactor to allow THREAD_SIZE to not be a compile-time constant. THREAD_SIZE_MAX (and THREAD_ALIGN_MAX) are introduced to manage the limits, as is done for PAGE_SIZE. When THREAD_SIZE is a compile-time constant, behaviour and code size should be equivalent. Signed-off-by: Ryan Roberts Acked-by: Vlastimil Babka --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ include/asm-generic/vmlinux.lds.h | 6 ++- include/linux/sched.h | 4 +- include/linux/thread_info.h | 10 ++++- init/main.c | 2 +- kernel/fork.c | 67 +++++++++++-------------------- mm/kasan/report.c | 3 +- 6 files changed, 42 insertions(+), 50 deletions(-) diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index 5727f883001bb..f19bab7a2e8f9 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -56,6 +56,10 @@ #define LOAD_OFFSET 0 #endif +#ifndef THREAD_SIZE_MAX +#define THREAD_SIZE_MAX THREAD_SIZE +#endif + /* * Only some architectures want to have the .notes segment visible in * a separate PT_NOTE ELF Program Header. When this happens, it needs @@ -398,7 +402,7 @@ init_stack = .; \ KEEP(*(.data..init_task)) \ KEEP(*(.data..init_thread_info)) \ - . = __start_init_stack + THREAD_SIZE; \ + . = __start_init_stack + THREAD_SIZE_MAX; \ __end_init_stack = .; #define JUMP_TABLE_DATA \ diff --git a/include/linux/sched.h b/include/linux/sched.h index f8d150343d42d..3de4f655ee492 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1863,14 +1863,14 @@ union thread_union { #ifndef CONFIG_THREAD_INFO_IN_TASK struct thread_info thread_info; #endif - unsigned long stack[THREAD_SIZE/sizeof(long)]; + unsigned long stack[THREAD_SIZE_MAX/sizeof(long)]; }; #ifndef CONFIG_THREAD_INFO_IN_TASK extern struct thread_info init_thread_info; #endif -extern unsigned long init_stack[THREAD_SIZE / sizeof(unsigned long)]; +extern unsigned long init_stack[THREAD_SIZE_MAX / sizeof(unsigned long)]; #ifdef CONFIG_THREAD_INFO_IN_TASK # define task_thread_info(task) (&(task)->thread_info) diff --git a/include/linux/thread_info.h b/include/linux/thread_info.h index 9ea0b28068f49..a7ccc448cd298 100644 --- a/include/linux/thread_info.h +++ b/include/linux/thread_info.h @@ -74,7 +74,15 @@ static inline long set_restart_fn(struct restart_block *restart, } #ifndef THREAD_ALIGN -#define THREAD_ALIGN THREAD_SIZE +#define THREAD_ALIGN THREAD_SIZE +#endif + +#ifndef THREAD_SIZE_MAX +#define THREAD_SIZE_MAX THREAD_SIZE +#endif + +#ifndef THREAD_ALIGN_MAX +#define THREAD_ALIGN_MAX max(THREAD_ALIGN, THREAD_SIZE_MAX) #endif #define THREADINFO_GFP (GFP_KERNEL_ACCOUNT | __GFP_ZERO) diff --git a/init/main.c b/init/main.c index ba1515eb20b9d..4dc28115fdf57 100644 --- a/init/main.c +++ b/init/main.c @@ -797,7 +797,7 @@ void __init __weak smp_prepare_boot_cpu(void) { } -# if THREAD_SIZE >= PAGE_SIZE +#ifdef CONFIG_VMAP_STACK void __init __weak thread_stack_cache_init(void) { } diff --git a/kernel/fork.c b/kernel/fork.c index ea472566d4fcc..cbc3e73f9b501 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -184,13 +184,7 @@ static inline void free_task_struct(struct task_struct *tsk) kmem_cache_free(task_struct_cachep, tsk); } -/* - * Allocate pages if THREAD_SIZE is >= PAGE_SIZE, otherwise use a - * kmemcache based allocator. - */ -# if THREAD_SIZE >= PAGE_SIZE || defined(CONFIG_VMAP_STACK) - -# ifdef CONFIG_VMAP_STACK +#ifdef CONFIG_VMAP_STACK /* * vmalloc() is a bit slow, and calling vfree() enough times will force a TLB * flush. Try to minimize the number of calls by caching stacks. @@ -343,46 +337,21 @@ static void free_thread_stack(struct task_struct *tsk) tsk->stack_vm_area = NULL; } -# else /* !CONFIG_VMAP_STACK */ +#else /* !CONFIG_VMAP_STACK */ -static void thread_stack_free_rcu(struct rcu_head *rh) -{ - __free_pages(virt_to_page(rh), THREAD_SIZE_ORDER); -} - -static void thread_stack_delayed_free(struct task_struct *tsk) -{ - struct rcu_head *rh = tsk->stack; - - call_rcu(rh, thread_stack_free_rcu); -} - -static int alloc_thread_stack_node(struct task_struct *tsk, int node) -{ - struct page *page = alloc_pages_node(node, THREADINFO_GFP, - THREAD_SIZE_ORDER); - - if (likely(page)) { - tsk->stack = kasan_reset_tag(page_address(page)); - return 0; - } - return -ENOMEM; -} - -static void free_thread_stack(struct task_struct *tsk) -{ - thread_stack_delayed_free(tsk); - tsk->stack = NULL; -} - -# endif /* CONFIG_VMAP_STACK */ -# else /* !(THREAD_SIZE >= PAGE_SIZE || defined(CONFIG_VMAP_STACK)) */ +/* + * Allocate pages if THREAD_SIZE is >= PAGE_SIZE, otherwise use a + * kmemcache based allocator. + */ static struct kmem_cache *thread_stack_cache; static void thread_stack_free_rcu(struct rcu_head *rh) { - kmem_cache_free(thread_stack_cache, rh); + if (THREAD_SIZE >= PAGE_SIZE) + __free_pages(virt_to_page(rh), THREAD_SIZE_ORDER); + else + kmem_cache_free(thread_stack_cache, rh); } static void thread_stack_delayed_free(struct task_struct *tsk) @@ -395,7 +364,16 @@ static void thread_stack_delayed_free(struct task_struct *tsk) static int alloc_thread_stack_node(struct task_struct *tsk, int node) { unsigned long *stack; - stack = kmem_cache_alloc_node(thread_stack_cache, THREADINFO_GFP, node); + struct page *page; + + if (THREAD_SIZE >= PAGE_SIZE) { + page = alloc_pages_node(node, THREADINFO_GFP, THREAD_SIZE_ORDER); + stack = likely(page) ? page_address(page) : NULL; + } else { + stack = kmem_cache_alloc_node(thread_stack_cache, + THREADINFO_GFP, node); + } + stack = kasan_reset_tag(stack); tsk->stack = stack; return stack ? 0 : -ENOMEM; @@ -409,13 +387,16 @@ static void free_thread_stack(struct task_struct *tsk) void thread_stack_cache_init(void) { + if (THREAD_SIZE >= PAGE_SIZE) + return; + thread_stack_cache = kmem_cache_create_usercopy("thread_stack", THREAD_SIZE, THREAD_SIZE, 0, 0, THREAD_SIZE, NULL); BUG_ON(thread_stack_cache == NULL); } -# endif /* THREAD_SIZE >= PAGE_SIZE || defined(CONFIG_VMAP_STACK) */ +#endif /* CONFIG_VMAP_STACK */ /* SLAB cache for signal_struct structures (tsk->signal) */ static struct kmem_cache *signal_cachep; diff --git a/mm/kasan/report.c b/mm/kasan/report.c index b48c768acc84d..57c877852dbc6 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -365,8 +365,7 @@ static inline bool kernel_or_module_addr(const void *addr) static inline bool init_task_stack_addr(const void *addr) { return addr >= (void *)&init_thread_union.stack && - (addr <= (void *)&init_thread_union.stack + - sizeof(init_thread_union.stack)); + (addr <= (void *)&init_thread_union.stack + THREAD_SIZE); } static void print_address_description(void *addr, u8 tag, From patchwork Mon Oct 14 10:58:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834680 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2DC1D1A45A for ; Mon, 14 Oct 2024 11:00:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72D5A6B0098; Mon, 14 Oct 2024 07:00:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6DE256B0099; Mon, 14 Oct 2024 07:00:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52F2D6B009A; Mon, 14 Oct 2024 07:00:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 36AFB6B0098 for ; Mon, 14 Oct 2024 07:00:08 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 87242160D24 for ; Mon, 14 Oct 2024 10:59:59 +0000 (UTC) X-FDA: 82671913212.14.2AB551B Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf26.hostedemail.com (Postfix) with ESMTP id E011314000E for ; Mon, 14 Oct 2024 11:00:01 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903464; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4UO2yeUO5mzgxVnueQx6h7oEw8kU90IKNX4PaGU1b4Y=; b=4ovNdwC9hRcllV3NX+KCFP/ycwRZHJgJyUwOzMQJ3LRqRLpX6/iLrVx+EVForw/AqlPXri D6qhQXKV5duv+8VBRn6tFumijihYfpX5JQxYvdhbIN/71sBaIsusmh+XclNwGTyijyX1nb 2+LyaQNbHiWpML+adfihPfveJmrDDiY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903464; a=rsa-sha256; cv=none; b=NriuhLHhTW4im81GxJOyDaWcAAto97Udv8B+mHkhXZICRiwUb2TmdlicxJ5EsveIOuAH1x ISoecCKQbv18mvKYBwed6qCcBS96NyRIi2QsQdzbCuKHrkB3DnmHkYtkuDPUVBtuEw7+4B Bg/Imc89/oyXjG0nhSNSp2zYLnMwBdg= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3AD141684; Mon, 14 Oct 2024 04:00:35 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 94D563F51B; Mon, 14 Oct 2024 04:00:02 -0700 (PDT) From: Ryan Roberts To: =?utf-8?q?Michal_Koutn=C3=BD?= , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Johannes Weiner , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Tejun Heo , Will Deacon , Zefan Li Cc: Ryan Roberts , cgroups@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 12/57] cgroup: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:19 +0100 Message-ID: <20241014105912.3207374-12-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Stat-Signature: m5xod69ntk457u196m1k4aba9tjdk1j8 X-Rspamd-Queue-Id: E011314000E X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1728903601-193610 X-HE-Meta: U2FsdGVkX1+FSkt0Z89QNGi08Y3z3oGkOt54fQH9Bf+8WXhVjmvxN8TckHBvmEp+DFnqEASFrWjTxn5mMW/ZEWz4AmlT8CXDoZwCyORqyIQV44T5exAJfF2X3izxUzfJke/VtC8wqwc9CB+YaYIAoPBX7q53lPzWCSQzxMQTnXt3h9lDowfVweJ5P8d2M9SXnkd2jY6d/MdqdJVkK1+AuZoJfypaSWsf4UbERHmfODW9h+xFYRTmYJXIP1SsmJgpF1QeY3zo3MYuheE5gfefvl2S7lgZtEYt36jmUeVPBvi2zTDwjNorIaRZiMvqALk2CAshTnzBDG24QSRve7aSzEi3mS971XvvM3Hk5VUJclFtkZznKvz0OLSsJt39T8fMm+cORCexNrmsA/rb8aR6uX/KT5/Izc8aEKZlF1Ihwpc8btxwbeudEt2QhEqnC4im+EzktBe5D357f79noq78KijA0b5YN8Tz7ML9V7qR86z/NrX9V6cdimPl1vV2P1FyjE6RdULy+nyyjh2jWmP2YYvz5Zhgb8IX8rvnOsFGblOEyqIeLMyRp1DuP9yozvRgUs5T/MDn8YmKGPUmVYFfZnNEVTErZudWSQxl+hm22+jmxt9yUUr0xIedSrcdQX161JZcASrsKJaZKKIcR4fbcHoSBDzC00HmzV8zHgV2ye3SyvgwdZharMI8fPkrgt2wE/drwfMAw0nj0YL/fBybAz0wGmWDPlR4E2PsnGnObZ1lS9NI+6dV/uNV2qLXawggcizpzSr8XV7o4OLlqMDkcnYJLgkQ/OTFt9EjBF8v6YW5R8bzAqEve9gxMghF8ufAVbhX659cqvPim6Cs08vYbMEuF6b1JIuFsBNY8E18UY/+5LHBQzSV2IuyN5NuJ/zTPXVuyuS4WF+V6CuopQNSKFlOH8U28v4evoNucmHpwvHq5Y+eMMqUIXVB5bnQhYNAfoAK2QmXrENgcI2uMIw 8zPZ/pOW tWtA+fXAt45L9PCejeDfLwBceKBfl1laS1u80IhVhXhBEunNNeyo0Kxmizvd3YhmPHJSu8hJk3jqwQy7WIBRLzuCna6Ev/h0qp/JmXn9GNrEwW0O41Mc05T0k3DIIg5ow4Ee0F7/LF/3XG5WFmhO0I/7a3ovo8TVWEEDbtxWycAXaoN9zDHepHmgpMkqeVEEeUvljdJ7Zz8Q9DtJeGOY39FiQXvcsCSdW4cZj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Wrap global variables that are initialized with PAGE_SIZE derived values using DEFINE_GLOBAL_PAGE_SIZE_VAR() so their initialization can be deferred for boot-time page size builds. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ kernel/cgroup/cgroup.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index c8e4b62b436a4..1e9c96210821d 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -4176,16 +4176,16 @@ static int cgroup_seqfile_show(struct seq_file *m, void *arg) return 0; } -static struct kernfs_ops cgroup_kf_single_ops = { +static DEFINE_GLOBAL_PAGE_SIZE_VAR(struct kernfs_ops, cgroup_kf_single_ops, { .atomic_write_len = PAGE_SIZE, .open = cgroup_file_open, .release = cgroup_file_release, .write = cgroup_file_write, .poll = cgroup_file_poll, .seq_show = cgroup_seqfile_show, -}; +}); -static struct kernfs_ops cgroup_kf_ops = { +static DEFINE_GLOBAL_PAGE_SIZE_VAR(struct kernfs_ops, cgroup_kf_ops, { .atomic_write_len = PAGE_SIZE, .open = cgroup_file_open, .release = cgroup_file_release, @@ -4195,7 +4195,7 @@ static struct kernfs_ops cgroup_kf_ops = { .seq_next = cgroup_seqfile_next, .seq_stop = cgroup_seqfile_stop, .seq_show = cgroup_seqfile_show, -}; +}); static void cgroup_file_notify_timer(struct timer_list *timer) { From patchwork Mon Oct 14 10:58:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834681 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F24CED1A443 for ; Mon, 14 Oct 2024 11:00:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 864FD6B009A; Mon, 14 Oct 2024 07:00:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 814206B009B; Mon, 14 Oct 2024 07:00:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68EE36B009C; Mon, 14 Oct 2024 07:00:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 48FA26B009A for ; Mon, 14 Oct 2024 07:00:11 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 602F81A0D2B for ; Mon, 14 Oct 2024 10:59:57 +0000 (UTC) X-FDA: 82671913254.05.25A95B8 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP id 5C4C840010 for ; Mon, 14 Oct 2024 11:00:04 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903504; a=rsa-sha256; cv=none; b=ezb1w/DUcpcoKjmMerSqXx6Q7eUhUpRAH+yoHmm6bcwD5GwmKS5hLNBwIcPD1zT27V79Au 8NAuFQe274G6IlMijgqIDLbwrE4eJtJOHePOv040tIlkwPJQsvKdr19hfhCTiURTszG49y qc5Z5XlXU0KDylg7e27045/e+PMAXS0= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903504; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TNFFxQG6M4Wg0iZv/cwkxpYlIfMOJbdGyBVBjoAfo3E=; b=VeS+0shZ2zt1+Q3j5x9aytIx26vi0nUlW03OOjh6/74Cb7+snHjdb+H62/smiaznI+bjEx H47ssDinbFsXxa/6bTBgKS2HgbvTRjEE5eXLwXZ8vBM4KFFu1ig44Zdky6AfYQzYkSu0iB xU7lPSKmRt8ukF4vFEitLyuXXQSh+hU= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 478EC1688; Mon, 14 Oct 2024 04:00:38 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DF8813F51B; Mon, 14 Oct 2024 04:00:05 -0700 (PDT) From: Ryan Roberts To: Alexei Starovoitov , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , Daniel Borkmann , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 13/57] bpf: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:20 +0100 Message-ID: <20241014105912.3207374-13-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5C4C840010 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: h4fowakb7gr38z9q65bst4juwoaht9n8 X-HE-Tag: 1728903604-274712 X-HE-Meta: U2FsdGVkX1/maSfe4cDILVz7HaqtWpDPTvTUVYSt59axDF64Tn9TeCBQcQWjwrgVEGiuh/x89/pbu1WUy2YViXdzk10hLrBgybAGjtPsKhMz1IxQj9a6hmb2UWzBq2CTw90XJX0yLypAnfGIeWsyctnu3Z+6wSBSWDB5MVw3ZWNixiGmCrlKakv1WxxkS6DTCfChbKOUNGuJARa8McgK/D43PsF6AR3Eez/x0RZbwDkRdkV5z392qal/MsceRJTmzAcKPmtCc/fxKil0oprkDFT3E/IIAyQG/hEGdz4910m16uc9m6J+fdzIcoZLR7VK8jWj6NidHpsQgnRDlVpfD4RMwmWBUidhOognQyKr4gi4vnE2MTnit34TTsQAnBpRrMbVk9TSEJY5VIaOEeqt9nymJDFpGaFHmE3sFzGB7XI+wHSwScVAB98Kc8LowIH+7qBCBjnlNQU/Lzn7VwhE1HMZPrbtbCdO2Th1TNouDK17uZDDWW1Bw6bgdYe4ChlqoZ8ZtDF/p8z9+VQl2PQfqAg+n4KH5cniwFWmbb7SiQwtlcqKOEr8NVwG9RQJI7NgyeaDZgSlHjPzrezcad+lmvK4zEIhIX4/JnyqwwAiJJrO4BWu2YHc+PgmC6BdRMFW2R+3g+2IGnRmVrK4tj6HtaYrkSOysthhc/sv2hrwfbQjAzIY/av7jGQIWyd7e4xgjlGcfka5MumDY+p1+ctBDwxKAYtlOiALYknmuYtX+/VCZskcueGhbFyUs/HmsWXKBTSM/tno88oY8TqELz0kN+QM8Brm9sKwBDsRTyI2F26N8Mw5Rs1jselmGqFOTZYDZp8l9eESAqOpmkpV8S+7ATC0lDLxaH96YbUpw8IfEKyD0gc4NFNe5EJsMw/ahzg6MrroD7GQKsr63g8pcFFZIUx+8CBb0tcucxjgs7d0sYNDxRwr4VHSikixHmIV4omwXqiP8c9C0mjjEBtdrrU YvPWAkUU +eONjd3/6RTpbXMKnSbQvN0I3g+KADF1EQa4jkHdITf0aSLOrMqIHsgIvu3R/4u2vSj1jwbqP9nxy6iqM/1ya3yCoIUfpomoHrsBLn/+CUULRvrD60kex+Yh1D5tnVu6fSfhe3gNdY+3H6k/dX+MQX7KmMQbqcGg9dWXCTuORFWN8sQ34/58OvNB16uiA9doSGJtW6cNuH5P9qywzzRnTlXnR0LbTkmLVVhXh X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Refactor "struct bpf_ringbuf" so that consumer_pos, producer_pos, pending_pos and data are no longer embedded at (static) page offsets within the struct. This can't work for boot-time page size because the page size isn't known at compile-time. Instead, only define the meta data in the struct, along with pointers to those values. At "struct bpf_ringbuf" allocation time, the extra pages are allocated at the end and the pointers are initialized to point to the correct locations. Additionally, only expose the __PAGE_SIZE enum to BTF for compile-time page size builds. We don't know the page size at compile-time for boot-time builds. NOTE: This may need some extra thought; perhaps __PAGE_SIZE should be exposed as 0 in this case? And/or perhaps __PAGE_SIZE_MIN/__PAGE_SIZE_MAX should be exposed? And there would need to be a runtime mechanism for querying the page size (e.g. getpagesize()). Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ kernel/bpf/core.c | 9 ++++++-- kernel/bpf/ringbuf.c | 54 ++++++++++++++++++++++++-------------------- 2 files changed, 37 insertions(+), 26 deletions(-) diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 7ee62e38faf0e..485875aa78e63 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -89,10 +89,15 @@ void *bpf_internal_load_pointer_neg_helper(const struct sk_buff *skb, int k, uns return NULL; } -/* tell bpf programs that include vmlinux.h kernel's PAGE_SIZE */ +/* + * tell bpf programs that include vmlinux.h kernel's PAGE_SIZE. We can only do + * this for compile-time PAGE_SIZE builds. + */ +#if PAGE_SIZE_MIN == PAGE_SIZE_MAX enum page_size_enum { __PAGE_SIZE = PAGE_SIZE }; +#endif struct bpf_prog *bpf_prog_alloc_no_stats(unsigned int size, gfp_t gfp_extra_flags) { @@ -100,7 +105,7 @@ struct bpf_prog *bpf_prog_alloc_no_stats(unsigned int size, gfp_t gfp_extra_flag struct bpf_prog_aux *aux; struct bpf_prog *fp; - size = round_up(size, __PAGE_SIZE); + size = round_up(size, PAGE_SIZE); fp = __vmalloc(size, gfp_flags); if (fp == NULL) return NULL; diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c index e20b90c361316..8e4093ddbc638 100644 --- a/kernel/bpf/ringbuf.c +++ b/kernel/bpf/ringbuf.c @@ -14,9 +14,9 @@ #define RINGBUF_CREATE_FLAG_MASK (BPF_F_NUMA_NODE) -/* non-mmap()'able part of bpf_ringbuf (everything up to consumer page) */ +/* non-mmap()'able part of bpf_ringbuf (everything defined in struct) */ #define RINGBUF_PGOFF \ - (offsetof(struct bpf_ringbuf, consumer_pos) >> PAGE_SHIFT) + (PAGE_ALIGN(sizeof(struct bpf_ringbuf)) >> PAGE_SHIFT) /* consumer page and producer page */ #define RINGBUF_POS_PAGES 2 #define RINGBUF_NR_META_PAGES (RINGBUF_PGOFF + RINGBUF_POS_PAGES) @@ -69,10 +69,10 @@ struct bpf_ringbuf { * validate each sample to ensure that they're correctly formatted, and * fully contained within the ring buffer. */ - unsigned long consumer_pos __aligned(PAGE_SIZE); - unsigned long producer_pos __aligned(PAGE_SIZE); - unsigned long pending_pos; - char data[] __aligned(PAGE_SIZE); + unsigned long *consumer_pos; + unsigned long *producer_pos; + unsigned long *pending_pos; + char *data; }; struct bpf_ringbuf_map { @@ -134,9 +134,15 @@ static struct bpf_ringbuf *bpf_ringbuf_area_alloc(size_t data_sz, int numa_node) rb = vmap(pages, nr_meta_pages + 2 * nr_data_pages, VM_MAP | VM_USERMAP, PAGE_KERNEL); if (rb) { + void *base = rb; + kmemleak_not_leak(pages); rb->pages = pages; rb->nr_pages = nr_pages; + rb->consumer_pos = (unsigned long *)(base + PAGE_SIZE * RINGBUF_PGOFF); + rb->producer_pos = (unsigned long *)(base + PAGE_SIZE * (RINGBUF_PGOFF + 1)); + rb->pending_pos = rb->producer_pos + 1; + rb->data = base + PAGE_SIZE * nr_meta_pages; return rb; } @@ -179,9 +185,9 @@ static struct bpf_ringbuf *bpf_ringbuf_alloc(size_t data_sz, int numa_node) init_irq_work(&rb->work, bpf_ringbuf_notify); rb->mask = data_sz - 1; - rb->consumer_pos = 0; - rb->producer_pos = 0; - rb->pending_pos = 0; + *rb->consumer_pos = 0; + *rb->producer_pos = 0; + *rb->pending_pos = 0; return rb; } @@ -300,8 +306,8 @@ static unsigned long ringbuf_avail_data_sz(struct bpf_ringbuf *rb) { unsigned long cons_pos, prod_pos; - cons_pos = smp_load_acquire(&rb->consumer_pos); - prod_pos = smp_load_acquire(&rb->producer_pos); + cons_pos = smp_load_acquire(rb->consumer_pos); + prod_pos = smp_load_acquire(rb->producer_pos); return prod_pos - cons_pos; } @@ -418,7 +424,7 @@ static void *__bpf_ringbuf_reserve(struct bpf_ringbuf *rb, u64 size) if (len > ringbuf_total_data_sz(rb)) return NULL; - cons_pos = smp_load_acquire(&rb->consumer_pos); + cons_pos = smp_load_acquire(rb->consumer_pos); if (in_nmi()) { if (!spin_trylock_irqsave(&rb->spinlock, flags)) @@ -427,8 +433,8 @@ static void *__bpf_ringbuf_reserve(struct bpf_ringbuf *rb, u64 size) spin_lock_irqsave(&rb->spinlock, flags); } - pend_pos = rb->pending_pos; - prod_pos = rb->producer_pos; + pend_pos = *rb->pending_pos; + prod_pos = *rb->producer_pos; new_prod_pos = prod_pos + len; while (pend_pos < prod_pos) { @@ -440,7 +446,7 @@ static void *__bpf_ringbuf_reserve(struct bpf_ringbuf *rb, u64 size) tmp_size = round_up(tmp_size + BPF_RINGBUF_HDR_SZ, 8); pend_pos += tmp_size; } - rb->pending_pos = pend_pos; + *rb->pending_pos = pend_pos; /* check for out of ringbuf space: * - by ensuring producer position doesn't advance more than @@ -460,7 +466,7 @@ static void *__bpf_ringbuf_reserve(struct bpf_ringbuf *rb, u64 size) hdr->pg_off = pg_off; /* pairs with consumer's smp_load_acquire() */ - smp_store_release(&rb->producer_pos, new_prod_pos); + smp_store_release(rb->producer_pos, new_prod_pos); spin_unlock_irqrestore(&rb->spinlock, flags); @@ -506,7 +512,7 @@ static void bpf_ringbuf_commit(void *sample, u64 flags, bool discard) * new data availability */ rec_pos = (void *)hdr - (void *)rb->data; - cons_pos = smp_load_acquire(&rb->consumer_pos) & rb->mask; + cons_pos = smp_load_acquire(rb->consumer_pos) & rb->mask; if (flags & BPF_RB_FORCE_WAKEUP) irq_work_queue(&rb->work); @@ -580,9 +586,9 @@ BPF_CALL_2(bpf_ringbuf_query, struct bpf_map *, map, u64, flags) case BPF_RB_RING_SIZE: return ringbuf_total_data_sz(rb); case BPF_RB_CONS_POS: - return smp_load_acquire(&rb->consumer_pos); + return smp_load_acquire(rb->consumer_pos); case BPF_RB_PROD_POS: - return smp_load_acquire(&rb->producer_pos); + return smp_load_acquire(rb->producer_pos); default: return 0; } @@ -680,12 +686,12 @@ static int __bpf_user_ringbuf_peek(struct bpf_ringbuf *rb, void **sample, u32 *s u64 cons_pos, prod_pos; /* Synchronizes with smp_store_release() in user-space producer. */ - prod_pos = smp_load_acquire(&rb->producer_pos); + prod_pos = smp_load_acquire(rb->producer_pos); if (prod_pos % 8) return -EINVAL; /* Synchronizes with smp_store_release() in __bpf_user_ringbuf_sample_release() */ - cons_pos = smp_load_acquire(&rb->consumer_pos); + cons_pos = smp_load_acquire(rb->consumer_pos); if (cons_pos >= prod_pos) return -ENODATA; @@ -715,7 +721,7 @@ static int __bpf_user_ringbuf_peek(struct bpf_ringbuf *rb, void **sample, u32 *s * Update the consumer pos, and return -EAGAIN so the caller * knows to skip this sample and try to read the next one. */ - smp_store_release(&rb->consumer_pos, cons_pos + total_len); + smp_store_release(rb->consumer_pos, cons_pos + total_len); return -EAGAIN; } @@ -737,9 +743,9 @@ static void __bpf_user_ringbuf_sample_release(struct bpf_ringbuf *rb, size_t siz * prevents another task from writing to consumer_pos after it was read * by this task with smp_load_acquire() in __bpf_user_ringbuf_peek(). */ - consumer_pos = rb->consumer_pos; + consumer_pos = *rb->consumer_pos; /* Synchronizes with smp_load_acquire() in user-space producer. */ - smp_store_release(&rb->consumer_pos, consumer_pos + rounded_size); + smp_store_release(rb->consumer_pos, consumer_pos + rounded_size); } BPF_CALL_4(bpf_user_ringbuf_drain, struct bpf_map *, map, From patchwork Mon Oct 14 10:58:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834682 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84190D1A43B for ; Mon, 14 Oct 2024 11:00:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1D5696B009C; Mon, 14 Oct 2024 07:00:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 185846B009D; Mon, 14 Oct 2024 07:00:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 026916B009E; Mon, 14 Oct 2024 07:00:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D7F786B009C for ; Mon, 14 Oct 2024 07:00:13 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D3C8240D8B for ; Mon, 14 Oct 2024 11:00:08 +0000 (UTC) X-FDA: 82671913296.03.9982227 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf26.hostedemail.com (Postfix) with ESMTP id 84CEF14001B for ; Mon, 14 Oct 2024 11:00:07 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903470; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S61ImBLZUPZT81QYOovWzbBNwho35ypnuFAVdu44m+A=; b=flg0tX/Kv0/npYSKE/WnMJOvDVCujka34pdWA8UKdh/LACTr9PMDEGDDSRSt771XD0fipf i1nC8bZoH9IkSn5monK2FzQXHN8fFKrLFWblxBqeiR0crLfnMSod7zGWJAX7kxSosMnXvA u5oHCd8SWZn/fFzlGypEIr67qV0abU4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903470; a=rsa-sha256; cv=none; b=gxho8t2V475rnip/Pkf/oYX3XVm8QhhsQS6hvpvg0D02SkU+2ppZ3yLpUYWtHx5BcsWrEv ijbjwD98HCbfj9/4psubj1I4qFu6B8y33cNMwWNzQxO+5z9C3Mm3h8bB43jwA0vkRF4Tod QdMa616kLhMrzLPCZ7/fQppPYHeD79Q= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 15DE8168F; Mon, 14 Oct 2024 04:00:41 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EC2633F51B; Mon, 14 Oct 2024 04:00:08 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-pm@vger.kernel.org Subject: [RFC PATCH v1 14/57] pm/hibernate: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:21 +0100 Message-ID: <20241014105912.3207374-14-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 84CEF14001B X-Stat-Signature: pmpmjr5zpn9r3phrjtxctuppd7cdtowj X-HE-Tag: 1728903607-589818 X-HE-Meta: U2FsdGVkX1/1Ka1uE14V59PtOxr46wczpy93k7jS/xsz72QH+8zMaS9ONMe2MfX1CJkQYXQWhe+jgF7jZyddKBvIdzgTfYWdqXNCOKfjlG1B+grEb8m2a3fcEkgHcw12vxI4iMBZ9fFjA5aheBMbHE79muGe7h+fQjz/9mB1rHYY3R1vdBHEM3Ls8AeB5WvhNE3ySgEPio+mhtUmxd2H6eQ7mIOaOPl6IBmY7Zvz1swl1owq32ce3+XsYWy3D+22ufEXhOiL3vYXJLGevpAwz8U83VcSo/8JeFTMV22metHUEy8uQr4wu+dFf1UMKh3zmy0A03mEdgF43AOSWuYdNW210VmD2cDKIC9fId20SQSzX17pdNe1r16cLRShLOCnNwQNoNBBnrye+9mFaOpyQUcP203wXgT46cp2ti2aixLrJAUrsFWmZOn3ypULqiLTxa4HD6+dUWPyTCpaBxYUChvrr3h2e4GcpUJbigXePnqV6WSpUQ6sc+9ocxmwY2a5ULzzl2LNAiK8gFs2lfiG7gnOcvpuCB1S9k8ALljwhQHYCQnML78d90nqbVYXNRHntDo2josJyyfT8ybrAR2+4j3SGFM/Je5M2FTIMuJMTrqsYOP12bfT0+F64PiRFMeEwjKkFAXOFuiFixtQbwD0w0xSMOONIBzQxcw47G71pBpE7x//aqyKMUqDIEuNBgD8qv+zuHVw8lLfcgqLI5NDxZ/urm1bAe2uEPxEZ09SwjGZk7zOyfMjx+W4aGlUWL/vcIRlCJdDYgTen5UZwjR/sHKE8ZSWyh7C9Q0T8U4GcVbQIQhdlBkPOzrxgaUhLvcKPYGpqm/yhIcgZQQlHIhYZMA9OiXu3lqYiixpL2w15lnX9O3S7CAxUmUz1W+c/MCqc+5tne8a2w3RCtS1kHmUKGPeJmUKnJDVMdlBWywJGeDI02zkQLTBTHcTcW5nvcIdQ78xjmGCbIlZX3IX1z7 XYQPkfPi Nqyud7bQtyzfmAHGY26nUC0GTNnOJKzbCjqhQOuBSg+ez7vTPrCcJOls+lgwtCO6Ih6q2owU2KbwAVTo9sqnqyXe+0Rk/xh7eJktu/6ADGFf9doW/bss8zEgyeSLussGSj7uuoQSI8dUL5pWyd/gKlFH1i17WT+liZCzyHjSjnbqXk4FYdMnPYE0efAb4YSQhJBBaw/qIH16KCs/VPbfMmmZ7vHnH8JnvHkMY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. "struct linked_page", "struct swap_map_page" and "struct swsusp_header" were all previously sized to be exactly PAGE_SIZE. Refactor those structures to remove the padding, then superimpose them on a page at runtime. "struct cmp_data" and "struct dec_data" previously contained embedded "unc" and "cmp" arrays, who's sizes were derived from PAGE_SIZE. We can't use flexible array approach here since there are 2 arrays in the structure, so convert to pointers and define an allocator and deallocator for each struct. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ kernel/power/power.h | 2 +- kernel/power/snapshot.c | 2 +- kernel/power/swap.c | 129 +++++++++++++++++++++++++++++++++------- 3 files changed, 108 insertions(+), 25 deletions(-) diff --git a/kernel/power/power.h b/kernel/power/power.h index de0e6b1077f23..74af2eb8d48a4 100644 --- a/kernel/power/power.h +++ b/kernel/power/power.h @@ -16,7 +16,7 @@ struct swsusp_info { unsigned long image_pages; unsigned long pages; unsigned long size; -} __aligned(PAGE_SIZE); +} __aligned(PAGE_SIZE_MAX); #ifdef CONFIG_HIBERNATION /* kernel/power/snapshot.c */ diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c index 405eddbda4fc5..144e92f786e35 100644 --- a/kernel/power/snapshot.c +++ b/kernel/power/snapshot.c @@ -155,7 +155,7 @@ struct pbe *restore_pblist; struct linked_page { struct linked_page *next; - char data[LINKED_PAGE_DATA_SIZE]; + char data[]; } __packed; /* diff --git a/kernel/power/swap.c b/kernel/power/swap.c index 82b884b67152f..ffd4c864acfa2 100644 --- a/kernel/power/swap.c +++ b/kernel/power/swap.c @@ -59,6 +59,7 @@ static bool clean_pages_on_decompress; */ #define MAP_PAGE_ENTRIES (PAGE_SIZE / sizeof(sector_t) - 1) +#define NEXT_SWAP_INDEX MAP_PAGE_ENTRIES /* * Number of free pages that are not high. @@ -78,8 +79,11 @@ static inline unsigned long reqd_free_pages(void) } struct swap_map_page { - sector_t entries[MAP_PAGE_ENTRIES]; - sector_t next_swap; + /* + * A PAGE_SIZE structure with (PAGE_SIZE / sizeof(sector_t)) entries. + * The last entry, [NEXT_SWAP_INDEX], is `.next_swap`. + */ + sector_t entries[1]; }; struct swap_map_page_list { @@ -103,8 +107,6 @@ struct swap_map_handle { }; struct swsusp_header { - char reserved[PAGE_SIZE - 20 - sizeof(sector_t) - sizeof(int) - - sizeof(u32) - sizeof(u32)]; u32 hw_sig; u32 crc32; sector_t image; @@ -113,6 +115,7 @@ struct swsusp_header { char sig[10]; } __packed; +static char *swsusp_header_pg; static struct swsusp_header *swsusp_header; /* @@ -315,7 +318,7 @@ static int mark_swapfiles(struct swap_map_handle *handle, unsigned int flags) { int error; - hib_submit_io(REQ_OP_READ, swsusp_resume_block, swsusp_header, NULL); + hib_submit_io(REQ_OP_READ, swsusp_resume_block, swsusp_header_pg, NULL); if (!memcmp("SWAP-SPACE",swsusp_header->sig, 10) || !memcmp("SWAPSPACE2",swsusp_header->sig, 10)) { memcpy(swsusp_header->orig_sig,swsusp_header->sig, 10); @@ -329,7 +332,7 @@ static int mark_swapfiles(struct swap_map_handle *handle, unsigned int flags) if (flags & SF_CRC32_MODE) swsusp_header->crc32 = handle->crc32; error = hib_submit_io(REQ_OP_WRITE | REQ_SYNC, - swsusp_resume_block, swsusp_header, NULL); + swsusp_resume_block, swsusp_header_pg, NULL); } else { pr_err("Swap header not found!\n"); error = -ENODEV; @@ -466,7 +469,7 @@ static int swap_write_page(struct swap_map_handle *handle, void *buf, offset = alloc_swapdev_block(root_swap); if (!offset) return -ENOSPC; - handle->cur->next_swap = offset; + handle->cur->entries[NEXT_SWAP_INDEX] = offset; error = write_page(handle->cur, handle->cur_swap, hb); if (error) goto out; @@ -643,8 +646,8 @@ struct cmp_data { wait_queue_head_t done; /* compression done */ size_t unc_len; /* uncompressed length */ size_t cmp_len; /* compressed length */ - unsigned char unc[UNC_SIZE]; /* uncompressed buffer */ - unsigned char cmp[CMP_SIZE]; /* compressed buffer */ + unsigned char *unc; /* uncompressed buffer */ + unsigned char *cmp; /* compressed buffer */ }; /* Indicates the image size after compression */ @@ -683,6 +686,45 @@ static int compress_threadfn(void *data) return 0; } +static void free_cmp_data(struct cmp_data *data, unsigned nr_threads) +{ + int i; + + if (!data) + return; + + for (i = 0; i < nr_threads; i++) { + vfree(data[i].unc); + vfree(data[i].cmp); + } + + vfree(data); +} + +static struct cmp_data *alloc_cmp_data(unsigned nr_threads) +{ + struct cmp_data *data = NULL; + int i = -1; + + data = vzalloc(array_size(nr_threads, sizeof(*data))); + if (!data) + goto fail; + + for (i = 0; i < nr_threads; i++) { + data[i].unc = vzalloc(UNC_SIZE); + if (!data[i].unc) + goto fail; + data[i].cmp = vzalloc(CMP_SIZE); + if (!data[i].cmp) + goto fail; + } + + return data; +fail: + free_cmp_data(data, nr_threads); + return NULL; +} + /** * save_compressed_image - Save the suspend image data after compression. * @handle: Swap map handle to use for saving the image. @@ -724,7 +766,7 @@ static int save_compressed_image(struct swap_map_handle *handle, goto out_clean; } - data = vzalloc(array_size(nr_threads, sizeof(*data))); + data = alloc_cmp_data(nr_threads); if (!data) { pr_err("Failed to allocate %s data\n", hib_comp_algo); ret = -ENOMEM; @@ -902,7 +944,7 @@ static int save_compressed_image(struct swap_map_handle *handle, if (data[thr].cc) crypto_free_comp(data[thr].cc); } - vfree(data); + free_cmp_data(data, nr_threads); } if (page) free_page((unsigned long)page); @@ -1036,7 +1078,7 @@ static int get_swap_reader(struct swap_map_handle *handle, release_swap_reader(handle); return error; } - offset = tmp->map->next_swap; + offset = tmp->map->entries[NEXT_SWAP_INDEX]; } handle->k = 0; handle->cur = handle->maps->map; @@ -1150,8 +1192,8 @@ struct dec_data { wait_queue_head_t done; /* decompression done */ size_t unc_len; /* uncompressed length */ size_t cmp_len; /* compressed length */ - unsigned char unc[UNC_SIZE]; /* uncompressed buffer */ - unsigned char cmp[CMP_SIZE]; /* compressed buffer */ + unsigned char *unc; /* uncompressed buffer */ + unsigned char *cmp; /* compressed buffer */ }; /* @@ -1189,6 +1231,45 @@ static int decompress_threadfn(void *data) return 0; } +static void free_dec_data(struct dec_data *data, unsigned nr_threads) +{ + int i; + + if (!data) + return; + + for (i = 0; i < nr_threads; i++) { + vfree(data[i].unc); + vfree(data[i].cmp); + } + + vfree(data); +} + +static struct dec_data *alloc_dec_data(unsigned nr_threads) +{ + struct dec_data *data = NULL; + int i = -1; + + data = vzalloc(array_size(nr_threads, sizeof(*data))); + if (!data) + goto fail; + + for (i = 0; i < nr_threads; i++) { + data[i].unc = vzalloc(UNC_SIZE); + if (!data[i].unc) + goto fail; + data[i].cmp = vzalloc(CMP_SIZE); + if (!data[i].cmp) + goto fail; + } + + return data; +fail: + free_dec_data(data, nr_threads); + return NULL; +} + /** * load_compressed_image - Load compressed image data and decompress it. * @handle: Swap map handle to use for loading data. @@ -1231,7 +1312,7 @@ static int load_compressed_image(struct swap_map_handle *handle, goto out_clean; } - data = vzalloc(array_size(nr_threads, sizeof(*data))); + data = alloc_dec_data(nr_threads); if (!data) { pr_err("Failed to allocate %s data\n", hib_comp_algo); ret = -ENOMEM; @@ -1510,7 +1591,7 @@ static int load_compressed_image(struct swap_map_handle *handle, if (data[thr].cc) crypto_free_comp(data[thr].cc); } - vfree(data); + free_dec_data(data, nr_threads); } vfree(page); @@ -1569,9 +1650,9 @@ int swsusp_check(bool exclusive) hib_resume_bdev_file = bdev_file_open_by_dev(swsusp_resume_device, BLK_OPEN_READ, holder, NULL); if (!IS_ERR(hib_resume_bdev_file)) { - clear_page(swsusp_header); + clear_page(swsusp_header_pg); error = hib_submit_io(REQ_OP_READ, swsusp_resume_block, - swsusp_header, NULL); + swsusp_header_pg, NULL); if (error) goto put; @@ -1581,7 +1662,7 @@ int swsusp_check(bool exclusive) /* Reset swap signature now */ error = hib_submit_io(REQ_OP_WRITE | REQ_SYNC, swsusp_resume_block, - swsusp_header, NULL); + swsusp_header_pg, NULL); } else { error = -EINVAL; } @@ -1631,12 +1712,12 @@ int swsusp_unmark(void) int error; hib_submit_io(REQ_OP_READ, swsusp_resume_block, - swsusp_header, NULL); + swsusp_header_pg, NULL); if (!memcmp(HIBERNATE_SIG,swsusp_header->sig, 10)) { memcpy(swsusp_header->sig,swsusp_header->orig_sig, 10); error = hib_submit_io(REQ_OP_WRITE | REQ_SYNC, swsusp_resume_block, - swsusp_header, NULL); + swsusp_header_pg, NULL); } else { pr_err("Cannot find swsusp signature!\n"); error = -ENODEV; @@ -1653,9 +1734,11 @@ int swsusp_unmark(void) static int __init swsusp_header_init(void) { - swsusp_header = (struct swsusp_header*) __get_free_page(GFP_KERNEL); - if (!swsusp_header) + swsusp_header_pg = (char *)__get_free_page(GFP_KERNEL); + if (!swsusp_header_pg) panic("Could not allocate memory for swsusp_header\n"); + swsusp_header = (struct swsusp_header *)(swsusp_header_pg + + PAGE_SIZE - sizeof(struct swsusp_header)); return 0; } From patchwork Mon Oct 14 10:58:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96E25D1A45A for ; Mon, 14 Oct 2024 11:00:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2C0316B009E; Mon, 14 Oct 2024 07:00:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 24ABD6B009F; Mon, 14 Oct 2024 07:00:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E7E8E6B00A0; Mon, 14 Oct 2024 07:00:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C28876B009E for ; Mon, 14 Oct 2024 07:00:16 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 63680AAB2C for ; Mon, 14 Oct 2024 11:00:01 +0000 (UTC) X-FDA: 82671913380.01.E22C62D Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf27.hostedemail.com (Postfix) with ESMTP id BDC3640017 for ; Mon, 14 Oct 2024 11:00:08 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf27.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903584; a=rsa-sha256; cv=none; b=o/EuZAdiuwrKjfBOV2PdUP7qBgnmv6NJzqWBSmROnhBcGgG9qIzXyO+BLaFKD8w8ods64U B0qu0hQ57SoJvvin2f0ktjJhnb381Ru1yyNbMugs02G48Jgi68cvmsAc97U/ygvbhy5jV6 ocCizJrw4n/ON1pZOWS4IQgvSKL8oe0= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf27.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903584; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mddGb9Xpz4LiyvZj0cyiEir/DtiqGjM7+BRPU5mZ/qY=; b=eQO+MDv/pih/zVauC5ASlGlB5sXPpYtc2Rc7gr1WTZkaltjVnihOkB++WRvU/ph2iO0PNJ 9dRuG63NaMi2RDc/J17uqO75AXM45sUe1DydPdzXPEBQ0rHotaamIt6Zyw2Ek7ZO4Fk3K0 xlspIMiZrnDqFrJPszzw5M8IUJrCO30= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BA5741691; Mon, 14 Oct 2024 04:00:43 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BB0D53F51B; Mon, 14 Oct 2024 04:00:11 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 15/57] stackdepot: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:22 +0100 Message-ID: <20241014105912.3207374-15-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: hetky5cwf3uqambp4k8jztd86uifmzjs X-Rspamd-Queue-Id: BDC3640017 X-Rspamd-Server: rspam02 X-HE-Tag: 1728903608-267546 X-HE-Meta: U2FsdGVkX1/KBIYTHImmkbJO6vMhYhooP40Ol1JF/RxarUZP0ztc8VwQZekr4oR8ewRSUtjNTJJXhvrjB2ZFNyOQIE8WI63BFQvU/CPIQQAJLVwnYuqM/zvUfe/zySOyQzeHYfrKTmzQ6UeCUoiCCx0QF5r7RACIgnNbWhGsoJ8ztcJ8KDtSKuXzVDVL2AQar3zV9YUqH2xjIhPO1PWxMaN7x4Ij2kEPWlUFUo049OPABNSJ1mIaMJoc0SahP28qyv1k/c6q3peEwpXuDNFy1kAFQJnjvuLoRiG5f10l6ZBvyzaFJSbS312StbWK4DmZSpSMaKr8knUUZHYHefeuEDQetiDbVVIaNtis2s6rabZuP26mFyolYDUv7LTnFgc46qqwsPqNdfTFQxZoWPXVvEHjZuzaV9alI17JMhEyPYd0AwO1sMjLyNAJ4SKIu3AKdYsWz3MjB1nYd+HAqqWjZX5T+oZZUDHpKRia1AuBO4AmzDYF3R/8k6G/DBHSOf4qWsOSVK8i06wqWhkW76kUgT8N13CJQfhpJqEzb27hxbZTo5+FYPN6hltWWR16izwHfFgsa3j1ria4QmkDUF1tj/7ZymaIHUFxKZlVsu5S4crYRnysNa/yc03kv7elsw1nMoReCDM1SSde0o6GSj+Ibe2nF+WwcjJ6BAk4xCelm3ep6jfISS4NUp8tlztvV1OGKNs/hK6GB5hcWi8skErBobs/yisjvIXtLrBxzABKdJefa2jDYO0Ye1QoHdRPYjfR1IO7GKPzOqcki0Onh2uRPiJabVooPv+yqdblGbqMZd0L7qB5PwlbjcbM5lZYVQlHZuoB0dzexjqrmhcB2JQEtmqswf0CU2tGYR2lVlxRj+h4Ei3fs68m35ikwyixj2pcTzoJzagimcbNyCSMwhJVwXi0G86JEWHmNGpwbiwpD65SduI2j+cTTAqc57xcBsGKwKEGLGzYRca3AWcWxa4 wT2FpTzY C+eavLbzJgPRw0voPLw4a7g5lOhxYMQpgm4gVkluCmN3kMU8v/A84/Sdqnf80tvNx1In114pElTfd/RoZcmrdObFMjoM798IZIIUKJazw9AlnW4ioQkL2sJNUhk5YYkUEiCnU1vBhYmOVw/RTKYryqw//ZZ46Sr/rxVQiBP7dTW0YDaruQQbYnmOWfFEvFHXbJ3WA1SGoG8xyFk+m5RcN93YwxJra9CInDI7/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. "union handle_parts" previously calculated the number of bits required for its pool index and offset members based on PAGE_SHIFT. This is problematic for boot-time page size builds because the actual page size isn't known until boot-time. We could use PAGE_SHIFT_MAX in calculating the worst case offset bits, but bits would be wasted that could be used for pool index when PAGE_SIZE is set smaller than MAX, the end result being that stack depot can address less memory than it should. To avoid needing to dynamically define the offset and index bit widths, let's instead fix the pool size and derive the order at runtime based on the PAGE_SIZE. This means that the fields' widths can remain static, with the down side being slightly increased risk of failing to allocate the large folio. This only affects boot-time page size builds. compile-time page size builds will still always allocate order-2 folios. Additionally, wrap global variables that are initialized with PAGE_SIZE derived values using DEFINE_GLOBAL_PAGE_SIZE_VAR() so their initialization can be deferred for boot-time page size builds. Signed-off-by: Ryan Roberts Acked-by: Vlastimil Babka --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ include/linux/stackdepot.h | 6 +++--- lib/stackdepot.c | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index e9ec32fb97d4a..ac877a4e90406 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -32,10 +32,10 @@ typedef u32 depot_stack_handle_t; #define DEPOT_HANDLE_BITS (sizeof(depot_stack_handle_t) * 8) -#define DEPOT_POOL_ORDER 2 /* Pool size order, 4 pages */ -#define DEPOT_POOL_SIZE (1LL << (PAGE_SHIFT + DEPOT_POOL_ORDER)) +#define DEPOT_POOL_ORDER 2 /* Pool size order, 4 pages of PAGE_SIZE_MAX */ +#define DEPOT_POOL_SIZE (1LL << (PAGE_SHIFT_MAX + DEPOT_POOL_ORDER)) #define DEPOT_STACK_ALIGN 4 -#define DEPOT_OFFSET_BITS (DEPOT_POOL_ORDER + PAGE_SHIFT - DEPOT_STACK_ALIGN) +#define DEPOT_OFFSET_BITS (DEPOT_POOL_ORDER + PAGE_SHIFT_MAX - DEPOT_STACK_ALIGN) #define DEPOT_POOL_INDEX_BITS (DEPOT_HANDLE_BITS - DEPOT_OFFSET_BITS - \ STACK_DEPOT_EXTRA_BITS) diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 5ed34cc963fc3..974351f0e9e3c 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -68,7 +68,7 @@ static void *new_pool; /* Number of pools in stack_pools. */ static int pools_num; /* Offset to the unused space in the currently used pool. */ -static size_t pool_offset = DEPOT_POOL_SIZE; +static DEFINE_GLOBAL_PAGE_SIZE_VAR(size_t, pool_offset, DEPOT_POOL_SIZE); /* Freelist of stack records within stack_pools. */ static LIST_HEAD(free_stacks); /* The lock must be held when performing pool or freelist modifications. */ @@ -625,7 +625,7 @@ depot_stack_handle_t stack_depot_save_flags(unsigned long *entries, */ if (unlikely(can_alloc && !READ_ONCE(new_pool))) { page = alloc_pages(gfp_nested_mask(alloc_flags), - DEPOT_POOL_ORDER); + get_order(DEPOT_POOL_SIZE)); if (page) prealloc = page_address(page); } @@ -663,7 +663,7 @@ depot_stack_handle_t stack_depot_save_flags(unsigned long *entries, exit: if (prealloc) { /* Stack depot didn't use this memory, free it. */ - free_pages((unsigned long)prealloc, DEPOT_POOL_ORDER); + free_pages((unsigned long)prealloc, get_order(DEPOT_POOL_SIZE)); } if (found) handle = found->handle.handle; From patchwork Mon Oct 14 10:58:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834684 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B0CFD1A443 for ; Mon, 14 Oct 2024 11:00:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D465C6B009F; Mon, 14 Oct 2024 07:00:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CF77F6B00A0; Mon, 14 Oct 2024 07:00:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B98836B00A1; Mon, 14 Oct 2024 07:00:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9B07F6B009F for ; Mon, 14 Oct 2024 07:00:19 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 67144120D4A for ; Mon, 14 Oct 2024 11:00:12 +0000 (UTC) X-FDA: 82671913632.15.76AB2D6 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf09.hostedemail.com (Postfix) with ESMTP id 00C7914000B for ; Mon, 14 Oct 2024 11:00:12 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903475; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9GjRzzDOoiNCAX5UvCNi+d9DYFyCOtMmFOQ33Q8ONIw=; b=Pa2xIIXFJYi4AHrjF40MPj9TovkRzVaOdpxssesvp1pmoddT5ZHwQeNtshWZfR1tQoR7zi nnX2UphPCOzKhVERMxFxjIKDk8UvmAW+yJQPUQnyi+fRMB2wygNstQyaJc3UdH3tTOyopD fHdteiPJSSWKD4fnH1W6Q3k3zt67iFE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903475; a=rsa-sha256; cv=none; b=1Br+23bv70lgi+enUBiD3EIpA+FwYex/afiyzCAiYyRNyIVCbP/sOzaO1IJSbky2ngVR13 8/DEMsMPGnCA9pE+p/GjhjQirtzBa2NdJYD6Dz9hoV1vToiu4KqzxS40MSlMCa1Tzqmb9m H8Z+YkrvM3S1W2W8gIWr8zvCZR6SGn4= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 89DBA168F; Mon, 14 Oct 2024 04:00:46 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6B5403F51B; Mon, 14 Oct 2024 04:00:14 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-perf-users@vger.kernel.org Subject: [RFC PATCH v1 16/57] perf: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:23 +0100 Message-ID: <20241014105912.3207374-16-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Stat-Signature: 4ih8h5zod1zfg3doi6ogp516afb66ftg X-Rspamd-Queue-Id: 00C7914000B X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1728903612-625756 X-HE-Meta: U2FsdGVkX1+TQNX/5Vt3B2JvK93v2RUzEVoE3XOZaajR5gJybON6DFHpxlRInaagFcU7w5+blFDTTGfXklW9wyX9DbXnuiupZOVuCpgr43JSOI8FaD/S6DaMhCd1OkJ9EGmVAJPmbeEuvNZpdVV0f/Ic8styJ+Mai1kCpnkN1s9GQk6x3tCtnkxv9O4hwbgFyPL458+uQLCRMS1hZfw1VEPnYUbNdhyZwQBq3UxTMXuvnyYlbhi/Ys/wyHjw63oH5fsDJENwwoI8v3A5lZ4K6wixMumprK9lRy025zFI0PYAC5LwfmEDeIa8STEh/meq9T2FOYxaBUml7sL8tooMmkKTCAoDkY0Hw+cf+tEQe+uLZRZx34K8DLv8LvrRJlZI4JTu07wYaYyTv5ZA313IoNc50oFU63ou3FfjH1lup4hNaj1oWa9nCcX6jm6zgotZXD+UTIBBP8lm/XezntWQo1oPlHLT4UPshhZRpx5kNlidFN+Z/75G6GTXMFwY2r4WdZwjN77pl/3ub2eplT06oDdeo9GfC4oiYrLWzCR3YAOEZklfD75doAmxWG6rdScChRhZPlj+7QZpXPDFuk3pCxqt7XbTI1h8vzccSXBn85PADfHcUqvouY104nLbserZTJCZuIDSZd6qN79K2MEBB2OkY3/0xWnWBbBrn7fOg9+3jWdRuGMkDVL8G4S+d8WS3y99RibxxVIZSKG62dQdfhBqKayC7My9RiUaDjx564paHhGTO4vbc5aRt6ij7k02YZ2i/paBwn1ZRsm2xINNZG7GYjl8dEu10gcC04xNnNQBVHNjhF3cmy+BtlUo59KlRj/sKMjernlU32R345SB+J/wmdRetNNaSvfqXBhIUvMg7vuWjokYapZukFjAtMITp/cbjTuhK/s6jLkskEbdJsv0evFz2sJBkUzcU592EucoTw+2fIMEBb+mfazUixLf21QabDnDfbYaCvP7dIz 3mvSR18P hZ4fY2WwMehrQNonQsYUeGWhqaLFeNFmY1PZTVE+EpDlhxibvQRTBKUyUtFU9+KJS8v9NV3usl6FlPGunzr4nqAGdoiTj7mg2DpwzuE4BwbnEDGOe62Jg3XRvIjy2N8dPyXzZ2wuiuZcDct+OUqPBhbJSI3mMWf/MHJ0tfPoy568Hg4gbCTQTEree47d54hiFfcW2shMYjFMcoBCMT1TvbzzJTmS0u0kK1cfQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Refactor a BUILD_BUG_ON() so that we test against the limit; _format is invariant to page size so testing it is no bigger than the minimum supported size is sufficient. Wrap global variables that are initialized with PAGE_SIZE derived values using DEFINE_GLOBAL_PAGE_SIZE_VAR() so their initialization can be deferred for boot-time page size builds. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ include/linux/perf_event.h | 2 +- kernel/events/core.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 1a8942277ddad..b7972155f93eb 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1872,7 +1872,7 @@ _name##_show(struct device *dev, \ struct device_attribute *attr, \ char *page) \ { \ - BUILD_BUG_ON(sizeof(_format) >= PAGE_SIZE); \ + BUILD_BUG_ON(sizeof(_format) >= PAGE_SIZE_MIN); \ return sprintf(page, _format "\n"); \ } \ diff --git a/kernel/events/core.c b/kernel/events/core.c index 8a6c6bbcd658a..81149663ab7d8 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -419,7 +419,7 @@ static struct kmem_cache *perf_event_cache; int sysctl_perf_event_paranoid __read_mostly = 2; /* Minimum for 512 kiB + 1 user control page */ -int sysctl_perf_event_mlock __read_mostly = 512 + (PAGE_SIZE / 1024); /* 'free' kiB per user */ +__DEFINE_GLOBAL_PAGE_SIZE_VAR(int, sysctl_perf_event_mlock, __read_mostly, 512 + (PAGE_SIZE / 1024)); /* 'free' kiB per user */ /* * max perf event sample rate From patchwork Mon Oct 14 10:58:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03171D1A45A for ; Mon, 14 Oct 2024 11:00:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D9F06B00A0; Mon, 14 Oct 2024 07:00:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 88AE26B00A1; Mon, 14 Oct 2024 07:00:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 703806B00A2; Mon, 14 Oct 2024 07:00:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5465D6B00A0 for ; Mon, 14 Oct 2024 07:00:22 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D0A84C0D24 for ; Mon, 14 Oct 2024 11:00:13 +0000 (UTC) X-FDA: 82671913674.25.7135291 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf14.hostedemail.com (Postfix) with ESMTP id 73B47100019 for ; Mon, 14 Oct 2024 11:00:13 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903478; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SCVxAiH0ahzES1Ifh6ZjiVheU6y/DFb6NZERTs5N+W4=; b=5ukpNnwrVMiVVhw+Jd+MvQJkd6Zugjf7CcLx9cUGfggSfRBE4hHpbIdy+0b8jniyocq1aU XyfdKiGh/15YsBJWfJwhTM2/hxzqnP1OAqXDqYxMtk+gGQIuN0cbuviH3r+Gr8fJyTUKkh nTjeWKcYeBFhloL0Io5vOuJdOHoosDQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903478; a=rsa-sha256; cv=none; b=UqnOrdj4AMJDE4k5QqIP8+lPvvM8Jf/HRPuzK+L9q+ekkOFXXh5Jfa8LEiHu1LeO+lIHVb JqPSotHg/M0Oy34ikYmeMIk/yH5Va0Pv9A8CvyaNgVop8JtcPdC0/MNn3nzZSDzqKDxCDf ozQtwcBfU8T27mHVVXoj011dcdXzDrQ= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 58C151684; Mon, 14 Oct 2024 04:00:49 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3A6AC3F51B; Mon, 14 Oct 2024 04:00:17 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 17/57] kvm: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:24 +0100 Message-ID: <20241014105912.3207374-17-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 73B47100019 X-Stat-Signature: rzqs9mickg6kbsqce447g9mqd4ynbwdb X-HE-Tag: 1728903613-878934 X-HE-Meta: U2FsdGVkX18olFc3+WbeRLptbMjXekxiVL9E1Ayv3QCc/gpY+aVxQb5EFPgYXXAp6GGWL7FAKRPaeXk9L4/WLoaIkQRAjjnXq4sQn6YBRQF2jtjbGYEMbr9CYqC6t9K4/tIaJZvmBxv5uS5BgUHK0s+Fox5PKFFZKOe+WE+391nc4Z6X807SkNgFaIUwDyyahTEVE/U2IIHrkyVXPSx7FbuaT5nOCjVKvkRpru4XTZ+yOxqUMB1q+MBETLvs5RUpHpDgHksvQfjZxePA+GzQWIQ3eHFlNqFavqVUv9euI8Uy9YMOs/I6mt+mNjbX5gEfg+p2FYl0u35+5dKB4xfpDGD8rgUQtDOQpPaT/SFPaSOOjm9Qq7GnplCzAq5+7a28u4ne+zZ1IHyoEFQpL8Q4AUpZakw+AuPGoYlRMw/ei5znDrlxEldm9JgtRU0l/0VKPIWWjdkrdFA9depL42mLLKXn+uK6o/qAXj6T2VdV8e4RRCZXPrAoEX1g07DCn/R390GdD7ZIXksM2qYnjrvFKx1INRUR9YZuLmSSMRmz+tlhTQBZVnK3x6ri6kHJF1osze+H8TTnksBqAD7XXgSO0gQJD3gc/O3PEEaBnCa2kidG81Cgt9QLUMXhZedNOEavxeXR1vYqo/VUfFCC49s0n7X3bh5iUtDNfRX/s+MD40VJ4Gxt/l2tR8E8gLNEiONIKkvtEK4nj0UW4Rt0ZfXxVJqlAPaSnVX79NGVD3muMe7xgtZv8jplbgWnWqNT5mD1pr91T2eYxxLyuBhJr3GZImWC2ebCMHKU28WFziHA1a9IUQnR0SOSErT29CGos3e2/37pSkimqJ2dR7/U2RMNdv1IYQn0PDF32MsMje8mLkLj4xjeUrDZ2LVc/6ve7hNomVNME7y9j+voMeh4rRX6EEe4eqggNXAXJfIfFqnCeRS0P0c8IVBBydl5QvhXkSloBINyBgSDYywegrW8/pt JzNWlPOF z2yqA5F2bsPobW7fGywklNaqpc5fGRkPdMfES40btUZxNnkJTA8qDpth2MxUWOQMMXFEXDpWV+qVThKzJ71NHisvQf4g7nmGIUN10o65RGFFtffTUcjiPUticyu7hY5PQ0moD7DMMKpcaA0+r2LGnkl78qNIAim4MyjRsnCThYcW+k4lYFU06sqcaAFD3DcsKnFKw6uI+uyNplZ9lFovurfXTuziGAxWXYt8q X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Modify BUILD_BUG_ON() to compare with page size limit. Signed-off-by: Ryan Roberts Reviewed-by: Sean Christopherson --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ virt/kvm/kvm_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c index cb2b78e92910f..6c862bc41a672 100644 --- a/virt/kvm/kvm_main.c +++ b/virt/kvm/kvm_main.c @@ -4244,7 +4244,7 @@ static int kvm_vm_ioctl_create_vcpu(struct kvm *kvm, unsigned long id) goto vcpu_decrement; } - BUILD_BUG_ON(sizeof(struct kvm_run) > PAGE_SIZE); + BUILD_BUG_ON(sizeof(struct kvm_run) > PAGE_SIZE_MIN); page = alloc_page(GFP_KERNEL_ACCOUNT | __GFP_ZERO); if (!page) { r = -ENOMEM; From patchwork Mon Oct 14 10:58:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834686 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FF84D1A443 for ; Mon, 14 Oct 2024 11:00:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C0E556B00A2; Mon, 14 Oct 2024 07:00:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BBF9A6B00A5; Mon, 14 Oct 2024 07:00:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3A0C6B00A4; Mon, 14 Oct 2024 07:00:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 816B06B00A2 for ; Mon, 14 Oct 2024 07:00:25 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 80BF5140D34 for ; Mon, 14 Oct 2024 11:00:17 +0000 (UTC) X-FDA: 82671913842.21.07A8488 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf03.hostedemail.com (Postfix) with ESMTP id 5AD9E20026 for ; Mon, 14 Oct 2024 11:00:20 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf03.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903508; a=rsa-sha256; cv=none; b=yN+c7LoNLkNlmuKoFgIaBYfNv6fb4bUng6k94MaZcrecg1QnXBCmu7/kC7IK8/STrBhSKY IFJgc7Nrczn6UbdtbNIid8MdrKAB9IFZZ1b9zcQLAbu6bpz46u0E9osZCwisPwp/nzlQVs eHdSV3W9boi6tEhM337lEuhqjBc5OE0= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf03.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903508; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WYYsCtIDe3FfCiqGTFtQtqmkCpDu+2cX7LKMS9zkbAc=; b=7m/MH96p7YIK8JDRRSdycRGzSO3Wf9mC+piS7dQ1nj575tyoRwTvgGpODIDuy1DX0iC8hD xqeVXkX/3pZBxDeh1MAVHqJdnOXKMWdkA40ZA1UbGElCH7mp1/2686aNJp2Ua9mnBWENUh HyQBIzmEcnd+AKG+0ZL+rxNSHloU87k= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 65FA91688; Mon, 14 Oct 2024 04:00:52 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 09E9C3F51B; Mon, 14 Oct 2024 04:00:19 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Masami Hiramatsu , Matthias Brugger , Miroslav Benes , Steven Rostedt , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org Subject: [RFC PATCH v1 18/57] trace: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:25 +0100 Message-ID: <20241014105912.3207374-18-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 5AD9E20026 X-Stat-Signature: uduixfx7w14rcxbffnafrn793cg1bt9e X-Rspam-User: X-HE-Tag: 1728903620-378315 X-HE-Meta: U2FsdGVkX1+sy18YSrwCaZ1f87tW4HzqZB06c8j15JhEk+o0giZ3azeYmWBVRew9p545v1ijmx60pk/eS3969Y7Uh4lUx0nSRaItaWAoIWPylTmANq0/Jh6+4ZNYR7+Hg7FY8njk1/y6Xwsd0p1aZrJjvy0bVKVOk9PaVMYwKwfbYe1nrxvjZzOszUefQzrbtHWxPEfIw9s3nLXXCbETB8G9ti+xGgJmA/3p+sHVoGamLKwkyx/0drAtW0MdeSvUEPVDpcPMaAt+JjoEc0EA6YhOyZ75kX39rUmfNiK9Zn2zQ1G7bsLlZ47gztbYIDYOWtJjG+XkDjz5+j54yoWO7CQUiuAZprtgv0Gufup4MFT31mbQAFYOnm44lKdtZqk/7lGpfoNLcQdMpREnSL8pPO9ffB93gyQMhrW2hP8M4gpqttuwSXRgmzwlHl1nn3siDHuol3rInlc0MxuyLegdU9uVgs5xhgDxEv2s97YSXw4bbkTlr+oXnnhfJkwmRK9kjsXjriKSRSHiQVyEPOhcyMYE/ZOoYN6GYhNLgpUUWAhrTFGz97WKJfKYFO7f9yQDku2DMFu8kvnN9WN0mfVQPBRkLk8ZTVwOS6uzixdLhay0p7jx8KgTjSDy7KSf2tlUgqGbidBiXVbubgGN0k39w6kWXEF6RJGD605IR+DIMfDLvfCcYpXA2UFVD5wVwoVRzfivfM4KGkILMYnUPhHZiuuQezmS9QMsv2H8hOZ7VepF5BEKvxgj78957tU+A2uayZlsrIgba+89Vi3AhTFOWax6hH2baqMy17eYVrBjqZngKef3Q3H3GOK6FWmpW+n+9KNYtGtgElIzomiZim5VeSjYgowZwQJeZ2PzA2LMxb7wBN81SQdHR1a1FbT/l8dbwmE7ep5LsyMdSeczOh7HqdsfpBp3MiF3eIxkuOZswF7/t1hcL3Ms/jtPl/FX75gA2IYpmwAHXWWvszeaqQ0 k7V78uFC Rp4pfug+k/Pb4HIHdacsXV7QIZXNw43Zr//WNrIgA3Ow7XJ2fdq9wCW62kwHqyrtcz4lAamhHqK+oYkNnm9TpcDCgOWh2ObkEb9lEXNOeQFB6dGIurszxaQoE4Z8pCXbUuGx49eZl+EZWz85whUc+oHF0Q6Tnn3DnA51ZTxw+6uCUmvs1TPehmu+ctXqcQSAqqpLT4g0g1AqRQLeQ+hM/aZW9tgwcOgKyRNHo X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Convert BUILD_BUG_ON() BUG_ON() since the argument depends on PAGE_SIZE and its not trivial to test against a page size limit. Redefine FTRACE_KSTACK_ENTRIES so that "struct ftrace_stacks" is always sized at 32K for 64-bit and 16K for 32-bit. It was previously defined in terms of PAGE_SIZE (and worked out at the quoted sizes for a 4K page size). But for 64K pages, the size expanded to 512K. Given the ftrace stacks should be invariant to page size, this seemed like a waste. As a side effect, it removes the PAGE_SIZE compile-time constant assumption from this code. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ kernel/trace/fgraph.c | 2 +- kernel/trace/trace.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/kernel/trace/fgraph.c b/kernel/trace/fgraph.c index d7d4fb403f6f0..47aa5c8d8090e 100644 --- a/kernel/trace/fgraph.c +++ b/kernel/trace/fgraph.c @@ -534,7 +534,7 @@ ftrace_push_return_trace(unsigned long ret, unsigned long func, if (!current->ret_stack) return -EBUSY; - BUILD_BUG_ON(SHADOW_STACK_SIZE % sizeof(long)); + BUG_ON(SHADOW_STACK_SIZE % sizeof(long)); /* Set val to "reserved" with the delta to the new fgraph frame */ val = (FGRAPH_TYPE_RESERVED << FGRAPH_TYPE_SHIFT) | FGRAPH_FRAME_OFFSET; diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index c3b2c7dfadef1..0f2ec3d30579f 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -2887,7 +2887,7 @@ trace_function(struct trace_array *tr, unsigned long ip, unsigned long /* Allow 4 levels of nesting: normal, softirq, irq, NMI */ #define FTRACE_KSTACK_NESTING 4 -#define FTRACE_KSTACK_ENTRIES (PAGE_SIZE / FTRACE_KSTACK_NESTING) +#define FTRACE_KSTACK_ENTRIES (SZ_4K / FTRACE_KSTACK_NESTING) struct ftrace_stack { unsigned long calls[FTRACE_KSTACK_ENTRIES]; From patchwork Mon Oct 14 10:58:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC610D1A443 for ; Mon, 14 Oct 2024 11:00:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6B93B6B00A5; Mon, 14 Oct 2024 07:00:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 640B06B00A6; Mon, 14 Oct 2024 07:00:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50A7E6B00A7; Mon, 14 Oct 2024 07:00:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 29D2F6B00A5 for ; Mon, 14 Oct 2024 07:00:28 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5681680D3C for ; Mon, 14 Oct 2024 11:00:21 +0000 (UTC) X-FDA: 82671913884.01.AC6E779 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf22.hostedemail.com (Postfix) with ESMTP id 39080C001B for ; Mon, 14 Oct 2024 11:00:19 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903484; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=P0Pjm4UAl0+EiWZfuspcZJKa6yxJiEhvXEi1XSb1a9E=; b=f+P+eNTpte7FBcNbMV/9MYU6AIXPjzaMKrGwGIzQtcMBYJqMuuL3MY/ULFa4qwMAQ3DZrx VoKDlrwOP5CzO8/hN22poY/tTivCga4S5lK+540ipd3m3WvH9GTweRT6FuGxh4KtekvzY1 i6fQw0Od81Xc0XApdUdA7IG/3hb6JNM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903484; a=rsa-sha256; cv=none; b=R1firzMuTrgLMLQdwRn9VHzEA//NKsPaTywLlDi8U9mhqEX70Z74atSVd6536owN1xDXA9 lZR1+5Avo3QIgMx8+Gh2XV4F0vEZqClI03vH8EWv/lLKaiZTL7XD3Wwcg41IjHFm6hgMs8 UM+mLDtvWKwp1fh1N/NiDKPo6DqjOrk= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=none; spf=pass (imf22.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5428F1684; Mon, 14 Oct 2024 04:00:55 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 16F953F51B; Mon, 14 Oct 2024 04:00:22 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Baoquan He , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 19/57] crash: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:26 +0100 Message-ID: <20241014105912.3207374-19-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Stat-Signature: e5p3dimteztp8tzkwrdkfce7mhaishbp X-Rspamd-Queue-Id: 39080C001B X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1728903619-703064 X-HE-Meta: U2FsdGVkX1/7I7EIzXv0173+V3YbT3Ii51s0coBb2IesasK92lWn1pYcsg0pX5iMyS4ExOWA0hHeJAYDKS4TZYPhBffVmXUeNpiv3JSsuXmCnlgBtsvtuznoL0g/+CzuyPOJJ8ZgRQbnYphWCYd/yUIHoISerrC2JZrdP77zWlrTRmARM9d/KC8AaCo3cGqC9xzJWxhydcV7IPfSYeuXX6OOOFamZr1AQuHhMjggiQm98kP9pg4vTY3NhBJppmLzc5hswbB6yHTHz5lR3/7ADC4MrBOBm1US5i35I/mciHUmZRpN/BMeK0xxjK6A0jitYp4ZQQKGV/f8FXG/u//htewz9iLBJ3MrFaF8hzgPw8kjKE7wCEJM71oyNwy3d+bfc+5fNc4kyxF3DcZzPcQHNEM7FiErFTlZIR6xqOg6u2h5KpT6VFCIQeahh2cRcjgjoa3d2yn8mpqDeIkz1NapAnJFJVy8f6wzQcRaKnWsR1oPKfj4n9suFxUvc9JHQY5x9cm7ABtNbESyuqPWPaltF2h25ginHOFBJFUGdBI7V9D9A8pwzImZHhWr0N9zrDvek0wtE1SCtsiDFHbXFP3ERNXGqcgDhhP6BrRu/VRIHmhDZD9FWKrXl3hKy9/VpQIezlViry0/DDpvudFIJ2L76RUkl98/FPBy1MLCcfj4fAsKyTViC5+VS/2ppKM9nODll2otFQgR41hLoTYe9bTxcfY8TbR4zuGMNMSTU8+b7mtZ8TBUcd9tEfNmQXlebqiwMRBhNTcrF11sEta95HImYox8JyRYQlsksB8NiA6fkB+mKn4Po7Aq1YHHusG5MbwCMCH2T5azCGS3u7u6TkO1TDniZwX9z6H9eSn+f3zrJW7DRsIpDNR97i7IC3fja+m0R7+wwLB/TQoSae90sur2A/ljY+4PBfZTiYs7Prj98TGZyarMnNkZG1bzhtDovs+QNl+jH/qoYTboQOk3sSh ihqHeJSD yQHkLA8EURflE1HZajmk6D3rAXjfgJuCYMOR7FuT3cOGnLNo91Is8okPzNv0FSNGZ4OrZ4lrCVqzty+3AoK72XSmIeHJPTD/tMwqouPK0Tmr91JfXoxlAVx62bHzvFrLtCOVkFdQaLBBD6jmM6BRdIbF3GPmbAM7Ky4KZ65bpQvv9YMnOB4cRcoQVoIOQi42cZ90HQOPZAI1R/aK86MIPzyDYtbCsJyTZz2EQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Updated BUILD_BUG_ON() to test against limit. Signed-off-by: Ryan Roberts Acked-by: Baoquan He --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ kernel/crash_core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/crash_core.c b/kernel/crash_core.c index 63cf89393c6eb..978c600a47ac8 100644 --- a/kernel/crash_core.c +++ b/kernel/crash_core.c @@ -465,7 +465,7 @@ static int __init crash_notes_memory_init(void) * Break compile if size is bigger than PAGE_SIZE since crash_notes * definitely will be in 2 pages with that. */ - BUILD_BUG_ON(size > PAGE_SIZE); + BUILD_BUG_ON(size > PAGE_SIZE_MIN); crash_notes = __alloc_percpu(size, align); if (!crash_notes) { From patchwork Mon Oct 14 10:58:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834688 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B075D1A43B for ; Mon, 14 Oct 2024 11:00:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C36496B00A6; Mon, 14 Oct 2024 07:00:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BE61B6B00A7; Mon, 14 Oct 2024 07:00:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AAE4A6B00A8; Mon, 14 Oct 2024 07:00:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 7C7206B00A6 for ; Mon, 14 Oct 2024 07:00:31 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 23402A0D74 for ; Mon, 14 Oct 2024 11:00:17 +0000 (UTC) X-FDA: 82671914010.13.B060233 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf03.hostedemail.com (Postfix) with ESMTP id 53B7620028 for ; Mon, 14 Oct 2024 11:00:26 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903557; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zTmw5JCDPKyesvf4SHt2c/Iv1pPV+8Di4FJdKvNluCo=; b=VmaYbp0p3S+/beY9sW7mA3HvlhJXn8kQvrZLw63rHSbacMRfrTuUDB5UYYgayOlnnL6f7f Pv5eOpm4WOlCLG13eL7n2YjWMKy9bcU3b4GbfsF3h5Vd4j7xZZTkqrW4vm1oOp8t475Z/1 TJXty204m2f7mjEEfRUb1eXJt9NKaYg= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903557; a=rsa-sha256; cv=none; b=DgornHToAf5cBZH5rcFdWenrWo7ZgVrKEOzhgkIgprH/+V2ATFIjuTnMTMojDZdiVYTmx5 /2wz2A6dBHSHVaAB/mtbyD1gNrf78Fqnir7ouj9gSIfOnMi7QE3PGddQZ7Y36ce3AyNlsS 9FWKbCCFpH8BIeNWIqJTU0PtQ6JFasU= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 61C921688; Mon, 14 Oct 2024 04:00:58 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0597C3F51B; Mon, 14 Oct 2024 04:00:25 -0700 (PDT) From: Ryan Roberts To: "David S. Miller" , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Herbert Xu , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-crypto@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 20/57] crypto: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:27 +0100 Message-ID: <20241014105912.3207374-20-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 53B7620028 X-Stat-Signature: t48j967cczjbxw5jx51xu8kjea4r8u8p X-HE-Tag: 1728903626-340239 X-HE-Meta: U2FsdGVkX1/QXg8uRvKoB1jlhVsD0wLIfeYxCr283N1NeKS+zM8SNFuFjox2QzHOAEd3/LbiBjR4h5waljEUBURAfHKdpo/A21RqgPrEL41gMUA7R/5v71PawGU4vXayzzWJJre43uZfj94WIPly8HYIKzQwzjm4QecTtWGQf1yaGZ2MpWd4yy7MsOHXi78DrrSLxAQ1Eo1GRJxP+WRTQLn+moyqCXioF1xQfaq+fDCvKHWaMd1+ucsAX+EXVN/5ZYvXbpWFIz7Ev/FGR5/dgas/gFMBK0Bbyp0N0IR4z7HmozT6cnyg1yWZVQyA18jfmKWLmKQFQXQia3O4sWo/PI1JEBtyqrwI19ducqM5dSX+S9xWJwNUTPCytzgBwL28h3YfCSF9DknqMQHM1BB3JYMBNzDpVIbt2GC2F1S9KWaLvam9p7mlXd+Hh6HKweBKXlj6R/qMDGFuWsIGaOD5g2I406HWJ6vGslu68PMdQ5S+q9mo8CnfmUQ1nTDeOk0ryrjga2x7Wc8Rcskw6a4oDmcmYiKfjppD3R8zM4+aEHDLSmrVOj04HbiNqztM246K62FUUkMn73eZIDZc1eJICg0tp8NxRv40GXcKkSbafO/p0hnyjGI4LyKsJonSltvKZf2Khb40/d6lfZwuMibVwQ3oFS/zfYj8rhdkrVBqalrzDb2wstl0wLxfeCddWOM/pt/a9J+cp5SOcbD15emlVGArF3UREPD3XyXEfipJGR9ir+DURXIrl6iTc/S/cCmyK2IZOcZ82j1Z8Ymxvt95mVFfkRPo6oIcpHe6zVjow0YcRJkaG7DSj75SQqK683Add/15sZ+WXm9pqmcYiOBz7xbk14eCDEhYd1kKO+CtKF4BBvFF8TxF9ZkeyomdZiWvjhqJQTYcRTmyRO9rbeBhQSpSccop6EZZYSxn6wI68vflSm2EO9qKL5arCAbKi/cK4Ko2QCXOcjn7vfQ6FI3 ahNpKKGP HLP+KN+jzIcHz/truR2cdBBjOi2zPmJ1gUaIdJrt/89FhbuKyKhFr4ZJMa5b+y2Jrf20i/N2CbWfE5yPYRLxtALoeA2tGxfgx2hxh/mysqqTx15lYFYlkxdIlcWK5Rg6446djvjAq17OuTahhcf/jsAQL5C5fPLw+FKxTDe1WdSL/ELZDNYEU32Nf0UlDSxtOrQ0UDvGeeiGb9glCSM0EF1SA3+G1/MypXmYi X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Updated BUILD_BUG_ON() to test against limit. Signed-off-by: Ryan Roberts Acked-by: Herbert Xu --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ crypto/lskcipher.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/crypto/lskcipher.c b/crypto/lskcipher.c index cdb4897c63e6f..2b84cefba7cd1 100644 --- a/crypto/lskcipher.c +++ b/crypto/lskcipher.c @@ -79,8 +79,8 @@ static int crypto_lskcipher_crypt_unaligned( u8 *tiv; u8 *p; - BUILD_BUG_ON(MAX_CIPHER_BLOCKSIZE > PAGE_SIZE || - MAX_CIPHER_ALIGNMASK >= PAGE_SIZE); + BUILD_BUG_ON(MAX_CIPHER_BLOCKSIZE > PAGE_SIZE_MIN || + MAX_CIPHER_ALIGNMASK >= PAGE_SIZE_MIN); tiv = kmalloc(PAGE_SIZE, GFP_ATOMIC); if (!tiv) From patchwork Mon Oct 14 10:58:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3778FD1A43B for ; Mon, 14 Oct 2024 11:00:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B9D966B0093; Mon, 14 Oct 2024 07:00:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B49D26B00A8; Mon, 14 Oct 2024 07:00:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9EAA66B00A9; Mon, 14 Oct 2024 07:00:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7D9A66B0093 for ; Mon, 14 Oct 2024 07:00:34 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A70471A0D54 for ; Mon, 14 Oct 2024 11:00:20 +0000 (UTC) X-FDA: 82671914220.16.71BA532 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf13.hostedemail.com (Postfix) with ESMTP id 266892001F for ; Mon, 14 Oct 2024 11:00:25 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903560; a=rsa-sha256; cv=none; b=7e0pRIR/TcKOueGYpO+hhqHO8s5MO+egmnOZpIy7SvR25WLxF1ztn7VyLKL5LxgFiKW0U5 5PiCtqSftsQK3eUle7toBa2qNn6OyBHshMIIIIj2ETULshcZALX2/22uBnHpd7BaH7FOAN euJE4me0nG16lkJvCPefCqtIlz7w5EQ= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903560; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WjYxq47rZAOyOAKetOXqZvgmx001+cTD5LxEFEwtYek=; b=FOMvGz1OLfx7PqnkLOE8Uu+DxsWWSGTCvfYWag3asXJzbXW15OSQgYNIBtiTgZyv86qItc j2Vp9p2tnJxHOeCmXKp5tFvZJDGZQYhLjzKGBt+vWyXVF2NPP3OK4FIH1Je6QYuY6qrkGz NaAPKuEnJ9thC2uEpElpiCIWlzOK5pI= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6EB86168F; Mon, 14 Oct 2024 04:01:01 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1323A3F51B; Mon, 14 Oct 2024 04:00:28 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anna Schumaker , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Trond Myklebust , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-nfs@vger.kernel.org Subject: [RFC PATCH v1 21/57] sunrpc: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:28 +0100 Message-ID: <20241014105912.3207374-21-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Stat-Signature: 5ip81b63igciixhg9gnwsfdyeyadpj38 X-Rspamd-Queue-Id: 266892001F X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1728903625-51287 X-HE-Meta: U2FsdGVkX1/1ZO0gnEN7NcRC6mg2g4PEt+IcsdXBSxKF5QDJK1o8/NvXTjbY4W+uQqqL+hRNBS+U6N+6yAoE1IR9DPXqOsWCkwLM8N89GZ0vXMAkPeIHztzol6vnEvbL4de7rbKh0cxN5EMhWRaBYXslVOBcvC0RvPq6dAjZR9COfCmTaI9Q9D0duHQw7c7m/dD8ywBdfcIAfej+H1AGEciIeJ1cLqO0z9B3QUN972GrsRZzXNcmYxm5mGqHl+TkQjaWhaRKGfLnRdxJZJmNXmUqibYEnqGzqCOWzKWxOU3qtgYfUZYpTOEzXotTJLKIzu/496wBYaDXY25sb3brMV70JwF0UYNFlRLA9ivwabJqCezNVjb3FZIjNk2NI1T3ztgXAd/nsUE1JBPEukQha8upbVynrb0PJQU2KtnY39944Gkg4JoXQ5Izaz2zMtIJjQQ8eB450WPrImlkLZRtNIDetcwrrVXo2gpGWdCRYY9dPImrcps+Y1dJ2g4hkfvOTfpAsiFRP0W7Z8M3FgK+Xi5mXjdsIuXAcbtDxs4xBTWDhNrrcTedQsKoCmaN20IZaRVGst3yQvosS6QWizVRt2Xl/nKjkoJdbhTfS7hEpQtUCbQRoQ+rblNjNRNpT+rNf6BNW/bJyEAOvbn0O8MYrsWjEi+1PskOome1Lg0LG8PWN9D85Se7H9s4KYKSUeui7mZuMAAwZ903RMg+HJBMxWCdgRQkedtDyJgWSR54Bd0YNMGYc7MYth6ffckvGBK22kgQ+V5BC81vW2LTH6do0rdOuJz2NzltuZX+wDY44H5GnJIMDQTEu/JSj/MHMjVqiXG//DgW+f4Dco8yjlxJcset6CcLD7K9h923Gu9UV8kv6NenCrfjavhG4MXpLj5kXh++I3TEecJn2KKua2h2er2dZdGElBBAVK4ulkPoAU0IY3B9vc+WjoyD96s5iguhgOl5e76mjCoJOMVM6/k 1DZPFT9B nbt4CUbMFC0x9OSkc1/xeJtsevhy2nrwl8hNKf0/6YVWFCuo3j9TvX7r+KlB9M5J+HV70KM/dnUBiV3tsDbNIbw3tdLxCLfk8qzEhkM/AZeOn7IQS+D9FpIq7IQnRFPG372cCBlXBNDGtyxX+bN8dtcvff49X3AozbSI4GbgZIdSwEDHHvve/oUXKnkpHQf4Z1JKnKfpZ/Yits6MGtesfTCZWrm4wRUQ/Vfxd X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Updated array sizes in various structs to contain enough entries for the smallest supported page size. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ include/linux/sunrpc/svc.h | 8 +++++--- include/linux/sunrpc/svc_rdma.h | 4 ++-- include/linux/sunrpc/svcsock.h | 2 +- 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/include/linux/sunrpc/svc.h b/include/linux/sunrpc/svc.h index a7d0406b9ef59..dda44018b8f36 100644 --- a/include/linux/sunrpc/svc.h +++ b/include/linux/sunrpc/svc.h @@ -160,6 +160,8 @@ extern u32 svc_max_payload(const struct svc_rqst *rqstp); */ #define RPCSVC_MAXPAGES ((RPCSVC_MAXPAYLOAD+PAGE_SIZE-1)/PAGE_SIZE \ + 2 + 1) +#define RPCSVC_MAXPAGES_MAX ((RPCSVC_MAXPAYLOAD+PAGE_SIZE_MIN-1)/PAGE_SIZE_MIN \ + + 2 + 1) /* * The context of a single thread, including the request currently being @@ -190,14 +192,14 @@ struct svc_rqst { struct xdr_stream rq_res_stream; struct page *rq_scratch_page; struct xdr_buf rq_res; - struct page *rq_pages[RPCSVC_MAXPAGES + 1]; + struct page *rq_pages[RPCSVC_MAXPAGES_MAX + 1]; struct page * *rq_respages; /* points into rq_pages */ struct page * *rq_next_page; /* next reply page to use */ struct page * *rq_page_end; /* one past the last page */ struct folio_batch rq_fbatch; - struct kvec rq_vec[RPCSVC_MAXPAGES]; /* generally useful.. */ - struct bio_vec rq_bvec[RPCSVC_MAXPAGES]; + struct kvec rq_vec[RPCSVC_MAXPAGES_MAX]; /* generally useful.. */ + struct bio_vec rq_bvec[RPCSVC_MAXPAGES_MAX]; __be32 rq_xid; /* transmission id */ u32 rq_prog; /* program number */ diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index d33bab33099ab..7c6441e8d6f7a 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -200,7 +200,7 @@ struct svc_rdma_recv_ctxt { struct svc_rdma_pcl rc_reply_pcl; unsigned int rc_page_count; - struct page *rc_pages[RPCSVC_MAXPAGES]; + struct page *rc_pages[RPCSVC_MAXPAGES_MAX]; }; /* @@ -242,7 +242,7 @@ struct svc_rdma_send_ctxt { void *sc_xprt_buf; int sc_page_count; int sc_cur_sge_no; - struct page *sc_pages[RPCSVC_MAXPAGES]; + struct page *sc_pages[RPCSVC_MAXPAGES_MAX]; struct ib_sge sc_sges[]; }; diff --git a/include/linux/sunrpc/svcsock.h b/include/linux/sunrpc/svcsock.h index 7c78ec6356b92..6c6bcc82685a3 100644 --- a/include/linux/sunrpc/svcsock.h +++ b/include/linux/sunrpc/svcsock.h @@ -40,7 +40,7 @@ struct svc_sock { struct completion sk_handshake_done; - struct page * sk_pages[RPCSVC_MAXPAGES]; /* received data */ + struct page * sk_pages[RPCSVC_MAXPAGES_MAX]; /* received data */ }; static inline u32 svc_sock_reclen(struct svc_sock *svsk) From patchwork Mon Oct 14 10:58:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834690 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BE75D1A443 for ; Mon, 14 Oct 2024 11:00:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AFE9F6B0085; Mon, 14 Oct 2024 07:00:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A85FF6B00A9; Mon, 14 Oct 2024 07:00:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 928FE6B00AA; Mon, 14 Oct 2024 07:00:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 695BE6B0085 for ; Mon, 14 Oct 2024 07:00:37 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 0992EABBE3 for ; Mon, 14 Oct 2024 11:00:22 +0000 (UTC) X-FDA: 82671914262.19.FF1FA02 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf24.hostedemail.com (Postfix) with ESMTP id EE7B8180008 for ; Mon, 14 Oct 2024 11:00:33 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; spf=pass (imf24.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903493; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wEHuKb5rtMoKCIErGijWSzrMjkfF9oh6ylp1H4/kTdU=; b=NDQiDkvo3RkvVk2hx4JzrgH8oEcT+1jHgDikhHKfVGfFrK3pQpYeyYGdgCJUVmpSANhC7Y iq2CqsWswBwavhiRNx6jc6UOmZHdSktGuC7jmrF35cjxSLVshQI/bjrC1CBuWnZdfCNEFt WOoXuWl48ryk0oigNmE1jY1bjNHSbwQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903493; a=rsa-sha256; cv=none; b=yJLHpGXPDUqOk0XOLDqe9xl7hd1DmGqBa5zDLUKP5B0amE7c5SeyvfJ+aypcWMLrAo/x3N Y0r9yqvN61YzSaHzsDUUf38RBbH/jPvO4M+CwFg2pihj0nv0zCNjrJR9aULn30o5tGgs8q mNjy8evUzL5fNLX3oCibWpg26FagcWo= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; spf=pass (imf24.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7C133168F; Mon, 14 Oct 2024 04:01:04 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 201A53F51B; Mon, 14 Oct 2024 04:00:32 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Jaroslav Kysela , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Takashi Iwai , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-sound@vger.kernel.org Subject: [RFC PATCH v1 22/57] sound: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:29 +0100 Message-ID: <20241014105912.3207374-22-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Stat-Signature: h9hjrghiggz8exc4syn18ur99ckit8dd X-Rspamd-Queue-Id: EE7B8180008 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1728903633-264471 X-HE-Meta: U2FsdGVkX18OjbFEJyC3Bmz68bXL82/jv+WZzgr3LCVo6J9RNhvfhnGKk3Ht6LnlHFjPqkARq3B8LksR/99siSPuMi73X+WsiCovYaq2R+UgDsca2V+sJTphiu2pNFqC7gOlY2/UobbPy1qPvvABeDT0BwLGKCPfKyd3nalp6VWyDP+a6x71rD5FW8DBjNFFgsDrxNWdafzTIo8c0QbHjNyzYsZ6bBOknJ6tdcuoWkPmfnWs/X+PKTL+s5tSgTv2Q02nBX/pzjLVm49H4X9LLD3h8lDBLBaVqOS9EE2S3Hi4FGTJLgWwa/SwzhR197oJzOp+34+6bcrigmKbuOWJHGwJBl56fOLWDN8N2VFmL/VHI0iN4BkjvphlAbiYOMXL5sqxp10pL2JAZcon2rbVZuRR6oQobrH69/fzzldLoEfXMbnc9w2HduJQBjpuAmEgpk+pphnazY7xmMTUcmPxbD4ojMIn18Q947QWawHvfhz473WHTXisu0EqhjfMK+SCClMXpfaAGzpVNfJmY2RKKgzJyvL/z5wYe6t8DJhoZueoETqLzrLBCG4ZzddkUOCB9wem0BUBokbupn2wTcnzGb7GuOMqCJZ8E0ve+Hl7KS34WWP6t+FyzElGmLDx+5f99FfBSEVsvtNNETI9u3c8l86inzIsG+H2xQvGxmFsEYpd2WjjbPBlpgHVEmJ2YtxPjU2NpoZgxMDQD6Pt1Z+Y4m3UO90vcyotfVpOC7S+xSXgGZ9mrtI6nEqIlcpjl/OVp7dQaVWnzMnCkfXoCJm3mPtW817ErZMGtoh4dQ5zKPFpSB5IBSIdPsDtlDCAIjhRODrfSIWk0tDPf/vC/7lRuHgXANPzi/GxGeXHo1oIhuqNW+d5ryuNm/Z7xweE3o+RArVktbyup/0bkk6Kmsptl4+8suvdV0fAJqIOp2LMMkXrkOlEDX6eIUmL3ehvsAO5846rOh+n6BnvmrRVpdz C05yJMm9 8bxcq1AgfOyrmOB5E73a6U7YdOhowyFfMzsJeb0BaspApnU8L8pa5lYaDzHUiV+trgn8VsX07DSdZyW7TBUDVFC3p6eZMbywpRiu50ddSIMQy3hCbsry/2YOaCfcSCzP034XCIYYoqTNg4o0bPo+lmERIVF1vNWsUbmfJFzXr38fR57YujTyB02NQQxpDXahQT4GgiYbQNFwdKxbcMH4OYOSG1gULh+4+2Rxm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Wrap global variables that are initialized with PAGE_SIZE derived values using DEFINE_GLOBAL_PAGE_SIZE_VAR() so their initialization can be deferred for boot-time page size builds. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ sound/soc/soc-utils.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sound/soc/soc-utils.c b/sound/soc/soc-utils.c index 303823dc45d7a..74e1bc1087de4 100644 --- a/sound/soc/soc-utils.c +++ b/sound/soc/soc-utils.c @@ -98,7 +98,7 @@ int snd_soc_tdm_params_to_bclk(const struct snd_pcm_hw_params *params, } EXPORT_SYMBOL_GPL(snd_soc_tdm_params_to_bclk); -static const struct snd_pcm_hardware dummy_dma_hardware = { +static DEFINE_GLOBAL_PAGE_SIZE_VAR_CONST(struct snd_pcm_hardware, dummy_dma_hardware, { /* Random values to keep userspace happy when checking constraints */ .info = SNDRV_PCM_INFO_INTERLEAVED | SNDRV_PCM_INFO_BLOCK_TRANSFER, @@ -107,7 +107,7 @@ static const struct snd_pcm_hardware dummy_dma_hardware = { .period_bytes_max = PAGE_SIZE*2, .periods_min = 2, .periods_max = 128, -}; +}); static const struct snd_soc_component_driver dummy_platform; From patchwork Mon Oct 14 10:58:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA564D1A43B for ; Mon, 14 Oct 2024 11:00:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 706846B00AA; Mon, 14 Oct 2024 07:00:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6B6476B00AB; Mon, 14 Oct 2024 07:00:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5098D6B00AC; Mon, 14 Oct 2024 07:00:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 30ACC6B00AA for ; Mon, 14 Oct 2024 07:00:41 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6E5BB1C613C for ; Mon, 14 Oct 2024 11:00:32 +0000 (UTC) X-FDA: 82671914598.28.87546CB Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf09.hostedemail.com (Postfix) with ESMTP id 9C11614002A for ; Mon, 14 Oct 2024 11:00:34 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903497; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LsJL+IRqe/3ZGlQHKLYmptDwM4z2+Mr5za/KqOhCeys=; b=pY/jydm3sE0qZ10WsiHnbaBGecV8VOIc2tYABn3G2MualcyUvwxwrGoVpIvOqS/HNGp9ac 1mMip4mWuAAZ+qRPunsl4Z5mtVXKss6n+AT1uIQeIjvHQTanNKHJi9PsiuAoTiLELl9s12 T98iGlJ93HLiwErcFZNdzDGiiQmFwP4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903497; a=rsa-sha256; cv=none; b=gp4JOhyeHeTw8KNkynQC+JOuggInEN6PG9jA1IylzImygO48y9ZtVH/J9lX83g22iNqDNK zndt3Zf3ECNsVucwtf6hFFidBZ6PhzAVi+jId2w3dJvHdWAjUd+q09tv4YQT7cyZSfJ5Fk DpLQcnxY4l0ldfHfJ3fY1PR/Jfwx5l8= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 306CE1691; Mon, 14 Oct 2024 04:01:08 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2DF153F51B; Mon, 14 Oct 2024 04:00:35 -0700 (PDT) From: Ryan Roberts To: "David S. Miller" , Andrew Morton , Anna Schumaker , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Eric Dumazet , Greg Marsden , Ivan Ivanov , Jakub Kicinski , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Paolo Abeni , Trond Myklebust , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-nfs@vger.kernel.org, netdev@vger.kernel.org Subject: [RFC PATCH v1 23/57] net: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:30 +0100 Message-ID: <20241014105912.3207374-23-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Stat-Signature: ctniboq4e53xwz8691cf47nuco95a9xg X-Rspamd-Queue-Id: 9C11614002A X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1728903634-462303 X-HE-Meta: U2FsdGVkX187Ck5esrZ39qDUVM60qSD0R7mQWwrgyNtNjrjTg8pmzhcT37tbDfEoO1KPlES0dvzYaRi5rdpw5nY0KfFdOM/VuqPu8c/ubNwOXxHM6MpH6rfRwg9s+desQy4/ZSfOcc5D0WehUYDpGYrfNPJIJ8zJ8Rj/68eHUqGCihWS0WSzbgft9mwllG2in6S7IDPC4mz1T6LoA3EsqJsORz0KlaeTO+4wr2oNPcOS79HcSnTYvwhCRS2kv+OrrOTsgKVNg8BsATqdxTvuprwCliovqGylZjWN3P4FkQQVSb2YqecKFqLg9LhZS0LmmTJJmugt9r9iiWEZz4CxUOgMTlaXllNBPWETS4beOedndEhLjhjlaxzX/CA0PlX5VmywVGVuEyq7jeBybv8IaeGlvrUqOvvRrmj6Ra+fHYA+pCieGojbphtjaKedq8n0YCOYsvig22WXMguLn2UDvy0maIIdCSapo2M00ycB9S4pudAk7RqjtSwOEQ8LmWzfQyui1xV/SSKr6xMz1qK5Mqhfg46ML+3v+rqshNdoBsq+bgJiu0RRtrStZv34wYQPeTWYqpYhjO6jfsfvp1y9wnTWkYU89kVOgSMczjgdQTbIyGzzcRQbr2U0Ip5ECiqpTqvpOTlhIntzcaxYJTpngc4mp6VuwPo8ChbqiyS5YbKsqhrz51RmJP2nMaTZ8mWz/rbWQd4DVRCQDTkVmJ/7kezU3MXGnwlkfqkBSJQnhG7JaAX2Nlvl2zRJE+MH25kspS3p+3Nf/oGYWM6KD7qBKCISijzANxC/NxG/qIiuhYt85WwncupWaca41KnSS/gfLGZ7guaMqIyR6WEewByvrGtyXAZ/Jc0U2/2pbF8xCAWGA86aiEdNKr4sALNUgiohO2m/YPmYyC3Pz3ZgqY89GdVq0tzOP5xtUC6LrmBiBhtWobIlsy482gaXjRN6xWO7sKwC6KDkzSs+glb/gQo FHvkTCET HsaZkepSxB6ZkabacrxhmcIvo7rek/gLH3YUQLKULEp3Q5yrjlMR+YSWpuIVwcQIhOEpRBobIzu0fz5tIZuMyAu/vu/QagpIRYzmSh1SKxZ96XGOUcACqed3FNz2EHY50L9cS+IC8rPQB2PCzAesasGKVxDwP8S3JUe2k15lRy9m1P8s7rMgPewIWYHhDsA3egocFLW0y+H81nbL7aRyElwyQ+fzusEgUgfNV X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Define NLMSG_GOODSIZE using min() instead of ifdeffery. This will now evaluate to a compile-time constant for compile-time page size, but evaluate at run-time when using boot-time page size. Rework NAPI small page frag infrastructure so that for boot-time page size it is compiled in if 4K page size is in the possible range, but defer deciding to use it to run time when the page size is known. No change for compile-time page size case. Resize cache_defer_hash[] array for PAGE_SIZE_MAX. Convert a complex BUILD_BUG_ON() to runtime BUG_ON(). Wrap global variables that are initialized with PAGE_SIZE derived values using DEFINE_GLOBAL_PAGE_SIZE_VAR() so their initialization can be deferred for boot-time page size builds. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ include/linux/netlink.h | 6 +----- net/core/hotdata.c | 4 ++-- net/core/skbuff.c | 4 ++-- net/core/sysctl_net_core.c | 2 +- net/sunrpc/cache.c | 3 ++- net/unix/af_unix.c | 2 +- 6 files changed, 9 insertions(+), 12 deletions(-) diff --git a/include/linux/netlink.h b/include/linux/netlink.h index b332c2048c755..ffa1e94111f89 100644 --- a/include/linux/netlink.h +++ b/include/linux/netlink.h @@ -267,11 +267,7 @@ netlink_skb_clone(struct sk_buff *skb, gfp_t gfp_mask) * use enormous buffer sizes on recvmsg() calls just to avoid * MSG_TRUNC when PAGE_SIZE is very large. */ -#if PAGE_SIZE < 8192UL -#define NLMSG_GOODSIZE SKB_WITH_OVERHEAD(PAGE_SIZE) -#else -#define NLMSG_GOODSIZE SKB_WITH_OVERHEAD(8192UL) -#endif +#define NLMSG_GOODSIZE SKB_WITH_OVERHEAD(min(PAGE_SIZE, 8192UL)) #define NLMSG_DEFAULT_SIZE (NLMSG_GOODSIZE - NLMSG_HDRLEN) diff --git a/net/core/hotdata.c b/net/core/hotdata.c index d0aaaaa556f22..e1f30e87ba6e9 100644 --- a/net/core/hotdata.c +++ b/net/core/hotdata.c @@ -5,7 +5,7 @@ #include #include -struct net_hotdata net_hotdata __cacheline_aligned = { +__DEFINE_GLOBAL_PAGE_SIZE_VAR(struct net_hotdata, net_hotdata, __cacheline_aligned, { .offload_base = LIST_HEAD_INIT(net_hotdata.offload_base), .ptype_all = LIST_HEAD_INIT(net_hotdata.ptype_all), .gro_normal_batch = 8, @@ -21,5 +21,5 @@ struct net_hotdata net_hotdata __cacheline_aligned = { .sysctl_max_skb_frags = MAX_SKB_FRAGS, .sysctl_skb_defer_max = 64, .sysctl_mem_pcpu_rsv = SK_MEMORY_PCPU_RESERVE -}; +}); EXPORT_SYMBOL(net_hotdata); diff --git a/net/core/skbuff.c b/net/core/skbuff.c index 83f8cd8aa2d16..b6c8eee0cc74b 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -219,9 +219,9 @@ static void skb_under_panic(struct sk_buff *skb, unsigned int sz, void *addr) #define NAPI_SKB_CACHE_BULK 16 #define NAPI_SKB_CACHE_HALF (NAPI_SKB_CACHE_SIZE / 2) -#if PAGE_SIZE == SZ_4K +#if PAGE_SIZE_MIN <= SZ_4K && SZ_4K <= PAGE_SIZE_MAX -#define NAPI_HAS_SMALL_PAGE_FRAG 1 +#define NAPI_HAS_SMALL_PAGE_FRAG (PAGE_SIZE == SZ_4K) #define NAPI_SMALL_PAGE_PFMEMALLOC(nc) ((nc).pfmemalloc) /* specialized page frag allocator using a single order 0 page diff --git a/net/core/sysctl_net_core.c b/net/core/sysctl_net_core.c index 86a2476678c48..a7a2eb7581bd1 100644 --- a/net/core/sysctl_net_core.c +++ b/net/core/sysctl_net_core.c @@ -33,7 +33,7 @@ static int int_3600 = 3600; static int min_sndbuf = SOCK_MIN_SNDBUF; static int min_rcvbuf = SOCK_MIN_RCVBUF; static int max_skb_frags = MAX_SKB_FRAGS; -static int min_mem_pcpu_rsv = SK_MEMORY_PCPU_RESERVE; +static DEFINE_GLOBAL_PAGE_SIZE_VAR(int, min_mem_pcpu_rsv, SK_MEMORY_PCPU_RESERVE); static int net_msg_warn; /* Unused, but still a sysctl */ diff --git a/net/sunrpc/cache.c b/net/sunrpc/cache.c index 95ff747061046..4e682c0cd7586 100644 --- a/net/sunrpc/cache.c +++ b/net/sunrpc/cache.c @@ -573,13 +573,14 @@ EXPORT_SYMBOL_GPL(cache_purge); */ #define DFR_HASHSIZE (PAGE_SIZE/sizeof(struct list_head)) +#define DFR_HASHSIZE_MAX (PAGE_SIZE_MAX/sizeof(struct list_head)) #define DFR_HASH(item) ((((long)item)>>4 ^ (((long)item)>>13)) % DFR_HASHSIZE) #define DFR_MAX 300 /* ??? */ static DEFINE_SPINLOCK(cache_defer_lock); static LIST_HEAD(cache_defer_list); -static struct hlist_head cache_defer_hash[DFR_HASHSIZE]; +static struct hlist_head cache_defer_hash[DFR_HASHSIZE_MAX]; static int cache_defer_cnt; static void __unhash_deferred_req(struct cache_deferred_req *dreq) diff --git a/net/unix/af_unix.c b/net/unix/af_unix.c index 0be0dcb07f7b6..1cf9f583358af 100644 --- a/net/unix/af_unix.c +++ b/net/unix/af_unix.c @@ -2024,7 +2024,7 @@ static int unix_dgram_sendmsg(struct socket *sock, struct msghdr *msg, MAX_SKB_FRAGS * PAGE_SIZE); data_len = PAGE_ALIGN(data_len); - BUILD_BUG_ON(SKB_MAX_ALLOC < PAGE_SIZE); + BUG_ON(SKB_MAX_ALLOC < PAGE_SIZE); } skb = sock_alloc_send_pskb(sk, len - data_len, data_len, From patchwork Mon Oct 14 10:58:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834692 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57B84D1A43B for ; Mon, 14 Oct 2024 11:00:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E16896B0083; Mon, 14 Oct 2024 07:00:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DBEB26B009A; Mon, 14 Oct 2024 07:00:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5B1B6B00AC; Mon, 14 Oct 2024 07:00:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A37BC6B0083 for ; Mon, 14 Oct 2024 07:00:44 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0C631160D34 for ; Mon, 14 Oct 2024 11:00:36 +0000 (UTC) X-FDA: 82671914556.03.7A737A7 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf03.hostedemail.com (Postfix) with ESMTP id A983F20002 for ; Mon, 14 Oct 2024 11:00:39 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903596; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2nn418B1fZ6U0bhmNO3CvxvB05kOVNrCxzyBbdDTXlE=; b=THoQRclw6AsZsqbxqdbzLdi5/xSGON7zkvneum8icjKZk3DZhIpUxz0LA2NdfbZ5PJTvBc wwZl25l0ot6RsLgj7oSp8gO9Fk6HQlSa4sKRAOWAslaIbkpMQvoYFlS9LBIC88hPAybm82 pj9wSIv4dy+4lmclJ7pPezSITfV/b08= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; spf=pass (imf03.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903596; a=rsa-sha256; cv=none; b=2qzCowR2f3KmV49m9hg4IGj051ie4A+cG2ZN6ieyigfPT6AzBLv1wzBMr1RuYIZuCKTv9U 9EyL/qrDvGjy2Y4gfFFJ3QiU2AXgqU1kL4/ydEWJMES/dr7dOLCuk2JREbgkkti6UwZb2q JC5+85pKSZR5viC4kV4OBTkhmklur5o= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B723A168F; Mon, 14 Oct 2024 04:01:11 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D51813F51B; Mon, 14 Oct 2024 04:00:38 -0700 (PDT) From: Ryan Roberts To: "David S. Miller" , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Eric Dumazet , Greg Marsden , Ivan Ivanov , Jakub Kicinski , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Paolo Abeni , Wei Fang , Will Deacon Cc: Ryan Roberts , imx@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org Subject: [RFC PATCH v1 24/57] net: fec: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:31 +0100 Message-ID: <20241014105912.3207374-24-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: ihnqdzgeeq9s97etjg4esg1h9cwtxezo X-Rspamd-Queue-Id: A983F20002 X-Rspamd-Server: rspam11 X-HE-Tag: 1728903639-424607 X-HE-Meta: U2FsdGVkX1/vmxjdSGe3s0jzo3jqnDK0Gu1+mqOfoGjEv/62hCnovCOcvTFxH5EJCO7207krea/S/6gn4r5XorJBRqoy3ErA5PLi+4s9A2YsYfqi4+DwOwAbSqZsywwXdqxhiCKKPmHPhEXA3yulI8hojyO+d5JXdYanedP1AA+2qFFea6nV/6ltQUJs3rRPrVt4cYFq9hJSoUk+SKdBVOlUYogr88oC161Dm3HVVPIrlMeHZTT9oCF2nOBAs9D2e7uerW5KOKwQt9fuM3OmW5kCOXz4ykZa1vXa//9/2w8HHLs4Aill9hqoqp44hw7H2gOguiAe84ENNsMjXh9ks1041mIesKRO/P/sQEsxw7tuyDJigW+MZk94rqUR/CoSEourU6ulWPWQW442JPeiDd/gFqOuExO5D8uE6kBd141R6aTWsnTGwVO9kbNuCjrDuEs2aoPnjEexTAAWfB3cOKaaVOZIcfCzK5MTxoXLh8oAlFfU4/Z8EdX41MIZ3k3Nj+mKxlcM94muFNC8zz5N3X3i4ui9lDrDcH6S3Ig/g3e2SA1P0I56X4bTFlSYatTZCHJB8Ra082N7O7nPDXQhNDNCUEqvU0NaFXY7KfkfDhaQ+MBR9079OCx1FA/PjRwNE8AT0X6JfAPd/Q2OZBFXDlnmBrb3lW581bVwHK0v/miJeq3+VINuVtsCbOFx4mewUaxaZtazPiq60zOtWiYZur15QFBzA6ryaw3iOlg4IH0bQrzZ9v+g8hC2C3K6mMxfQqfJlfQMcyCVaidQBtJT+W5r+cBlLDw85lYlS11HLyRnXsx4cM79tm4ffxpgxa6Y4S3nw+iSOCH+zCohf32jecb02IAVqpnbYFANk3WzFJMMzIZ79l7Jt9cwoyf8HKoyrQD3qb1b9y0BdUxYWvJIravlqb6zqUhf1r+lbmLHPHG4SLSyal0UVIUXsXqUa6ruPdrigRzN/tyXFePV42m uyDzZ1K5 dEovuSuAKeMjymdWuu36mc6HvrLJyD8TKlnf9B2ETr/OEA0B/nuJA74VT+Ta0sw4CKir1g5Lbltyq0khoIoilgTZh86rtYMxVICp6+PYJKVLLJ/0PIrWTMp4qQ/QU+Sa1XCA6gwAc4lnrRlUecgIuD8YU+RAogXGprbCPzjEay+krLS40fg5s+uSglGotI4YJOT41NYTNE3jf5YqSriJ7W9Kr9uzTxwGxG+Dg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Refactored "struct fec_enet_priv_rx_q" to use a flexible array member for "rx_skb_info", since its length depends on PAGE_SIZE. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ drivers/net/ethernet/freescale/fec.h | 3 ++- drivers/net/ethernet/freescale/fec_main.c | 5 +++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/freescale/fec.h b/drivers/net/ethernet/freescale/fec.h index a19cb2a786fd2..afc8b3f360555 100644 --- a/drivers/net/ethernet/freescale/fec.h +++ b/drivers/net/ethernet/freescale/fec.h @@ -571,7 +571,6 @@ struct fec_enet_priv_tx_q { struct fec_enet_priv_rx_q { struct bufdesc_prop bd; - struct fec_enet_priv_txrx_info rx_skb_info[RX_RING_SIZE]; /* page_pool */ struct page_pool *page_pool; @@ -580,6 +579,8 @@ struct fec_enet_priv_rx_q { /* rx queue number, in the range 0-7 */ u8 id; + + struct fec_enet_priv_txrx_info rx_skb_info[]; }; struct fec_stop_mode_gpr { diff --git a/drivers/net/ethernet/freescale/fec_main.c b/drivers/net/ethernet/freescale/fec_main.c index a923cb95cdc62..b9214c12d537e 100644 --- a/drivers/net/ethernet/freescale/fec_main.c +++ b/drivers/net/ethernet/freescale/fec_main.c @@ -3339,6 +3339,8 @@ static int fec_enet_alloc_queue(struct net_device *ndev) int i; int ret = 0; struct fec_enet_priv_tx_q *txq; + size_t rxq_sz = struct_size(fep->rx_queue[0], rx_skb_info, RX_RING_SIZE); + for (i = 0; i < fep->num_tx_queues; i++) { txq = kzalloc(sizeof(*txq), GFP_KERNEL); @@ -3364,8 +3366,7 @@ static int fec_enet_alloc_queue(struct net_device *ndev) } for (i = 0; i < fep->num_rx_queues; i++) { - fep->rx_queue[i] = kzalloc(sizeof(*fep->rx_queue[i]), - GFP_KERNEL); + fep->rx_queue[i] = kzalloc(rxq_sz, GFP_KERNEL); if (!fep->rx_queue[i]) { ret = -ENOMEM; goto alloc_failed; From patchwork Mon Oct 14 10:58:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBD68D1A43B for ; Mon, 14 Oct 2024 11:00:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 841846B00AC; Mon, 14 Oct 2024 07:00:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7F0076B00AD; Mon, 14 Oct 2024 07:00:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68F556B00AE; Mon, 14 Oct 2024 07:00:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 475CC6B00AC for ; Mon, 14 Oct 2024 07:00:48 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id A4098160D33 for ; Mon, 14 Oct 2024 11:00:39 +0000 (UTC) X-FDA: 82671914766.25.725BE1A Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf02.hostedemail.com (Postfix) with ESMTP id B74E380015 for ; Mon, 14 Oct 2024 11:00:34 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903458; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PSUHqjoRyP5Y2BnZxqoD+IML1c1ukTJ/pQox4PTBxLQ=; b=Ai8L1KCAMuZ/F6KeXxqIvI84gUgK2LXGOu6vlCmRy/B/JjkwYQ6CcW3iIqW3xeFqJaMFJ5 Zop1eWu859kUJpcslxRu2DiA9R2gHG3eRjYsXxcntJmuLuvVaSLte2DUFwOE/MzeL20Q40 9ONUJ3VCqpNQpIzRKFUbaaR7zy7fQn4= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903458; a=rsa-sha256; cv=none; b=tZB7f3lnrtAKi9hDq/b3GmWIGJybddk8STGMlNkwwifGemD/xumoIlK1JOfEEIRl/8xotB QBsWQKuRM5+g7q26nEbXbVdqWIzlzCAG7gkkfCVkyYsiCW561mcdySYrW65FCculhQ5eyk TuNqTMl45tGWxXIURx7kremgnjTg/A8= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2C1911692; Mon, 14 Oct 2024 04:01:15 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 68E4F3F51B; Mon, 14 Oct 2024 04:00:42 -0700 (PDT) From: Ryan Roberts To: "David S. Miller" , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Eric Dumazet , Greg Marsden , Ivan Ivanov , Jakub Kicinski , Kalesh Singh , Marc Zyngier , Marcin Wojtas , Mark Rutland , Matthias Brugger , Miroslav Benes , Paolo Abeni , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org Subject: [RFC PATCH v1 25/57] net: marvell: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:32 +0100 Message-ID: <20241014105912.3207374-25-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B74E380015 X-Stat-Signature: p9uri78ccfoxdbeih73fst3fk551xnkx X-Rspam-User: X-HE-Tag: 1728903634-17164 X-HE-Meta: U2FsdGVkX1+N2om6pwe9a78ESpk93emJ5FPHknlzvQ6avPC88sW+QrCb27sXtcPskJ7hQnTEsfUBNYexmhhSnKoA0pmS4CiYSyFmlHCusLWo0SzScvdtU3Qhsp8e4Q7Jj/3Bx4gBcUSrjOO6SJoG9o7fL9HDUbzg+Hea46nPtkAHjgXZI+Y3ROHuczcJbp9zty7W6bwRakK0vmktA5KB2TH3U9noWtHlDC7S2UtCJY6MMGwgGTVD7r5rza1Wx9fHHwDJ8FhckF9F/HbRkh+6pcqIzUrmDEWw+c6Cck9Izgnl9n4WDomoN95TsO1mRzOZis5/Q5D2LxYDYYrWMvBVSC1xJXiuPjcmqW93k+Gxy8MygapFCiyiU+35cDXsWFH8GFy5l0Ap3SCPSENBDoGpN/Aem0M3DyOf6kLa95GZbYz22hHGP2QWBspqnHrBO9qctNEb8gRuPTVabSFEt+22uFhHMuMYgdiE8Z6RPqrIdobSDvVqphGuS8106LFLCmpTwhumqxdWTGb2J1Re0F/zfv/xMCINpopaRp6floDGOmiGRFvV0oobAfkvdg6fQNxPtC1I8qAHUoQPRTyLJoFgDSRPcMHurUFvnrVrpDQts1ONcbvGpZwPdMq3fMwGzEPvTFUbQosQvYpjUQuS0Z6UlfIzUgItk8ZA/NxijzDks/DbE9tkU42iQIXHWbruMNOwHe+kISfuyhvKI/gyzwEqF+fs7EWJ4+7Ky8U3UlRhxFOWyWab5GxbE2DvhZHQOzDCgIfe8d0I8LJwwXdV2UuseWVBQ0N0kFSFzjwVqrrYbFL23oALckYPb7cg74aJYWHGnVb6mG+W9PaOXkmQ5jF5aIx1usfx8ibIuJJJQeDxMP/V2y/9/liTlJdi48CJeq8DYXvdxK8vKJ50LJg/Bhav9dv1ouaBX0cUydxAAAE4KjNvqhpjUozhmwrD2q1m/HXHJmg08lThqU4jAr7QUtk zDjWxmTq 9ZdYEOU09RZOxZEgnV7CQNmplGyWBqFFLpMVd/kQAhMXvUW2THZWLqzcFmYq1Hb9aYkTAnYhN6MxwpOw+DO3l+AV1HNaAFKYKAXQ7a6onZ9uowTMZGl16cIoA02a1CgsUH8eh7Rk9EzLJh3Nd69+JC7pCLbmBrC2rnExuYy8iiYIol4DYCBtnBzPRBYmeiyw0zbiBiPrFdBpZjRINOwq80yHsYsv1SYGX+NscOCb7ULSij98c7Nl0SA6/Wv4wpkSN+Zyt5CthXQP95bnTmxN+ySyl+bIra44HC356pgp0Jt2JRw4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Updated sky2 "struct rx_ring_info" member frag_addr[] to contain enough entries for the smallest supported page size. Updated mvneta "struct mvneta_tx_queue" members tso_hdrs[] and tso_hdrs_phys[] to contain enough entries for the smallest supported page size. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ drivers/net/ethernet/marvell/mvneta.c | 9 ++++++--- drivers/net/ethernet/marvell/sky2.h | 2 +- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c index 41894834fb53c..f3ac371d8f3a7 100644 --- a/drivers/net/ethernet/marvell/mvneta.c +++ b/drivers/net/ethernet/marvell/mvneta.c @@ -346,12 +346,15 @@ /* The size of a TSO header page */ #define MVNETA_TSO_PAGE_SIZE (2 * PAGE_SIZE) +#define MVNETA_TSO_PAGE_SIZE_MIN (2 * PAGE_SIZE_MIN) /* Number of TSO headers per page. This should be a power of 2 */ #define MVNETA_TSO_PER_PAGE (MVNETA_TSO_PAGE_SIZE / TSO_HEADER_SIZE) +#define MVNETA_TSO_PER_PAGE_MIN (MVNETA_TSO_PAGE_SIZE_MIN / TSO_HEADER_SIZE) /* Maximum number of TSO header pages */ #define MVNETA_MAX_TSO_PAGES (MVNETA_MAX_TXD / MVNETA_TSO_PER_PAGE) +#define MVNETA_MAX_TSO_PAGES_MAX (MVNETA_MAX_TXD / MVNETA_TSO_PER_PAGE_MIN) /* descriptor aligned size */ #define MVNETA_DESC_ALIGNED_SIZE 32 @@ -696,10 +699,10 @@ struct mvneta_tx_queue { int next_desc_to_proc; /* DMA buffers for TSO headers */ - char *tso_hdrs[MVNETA_MAX_TSO_PAGES]; + char *tso_hdrs[MVNETA_MAX_TSO_PAGES_MAX]; /* DMA address of TSO headers */ - dma_addr_t tso_hdrs_phys[MVNETA_MAX_TSO_PAGES]; + dma_addr_t tso_hdrs_phys[MVNETA_MAX_TSO_PAGES_MAX]; /* Affinity mask for CPUs*/ cpumask_t affinity_mask; @@ -5895,7 +5898,7 @@ static int __init mvneta_driver_init(void) { int ret; - BUILD_BUG_ON_NOT_POWER_OF_2(MVNETA_TSO_PER_PAGE); + BUILD_BUG_ON_NOT_POWER_OF_2(MVNETA_TSO_PER_PAGE_MIN); ret = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "net/mvneta:online", mvneta_cpu_online, diff --git a/drivers/net/ethernet/marvell/sky2.h b/drivers/net/ethernet/marvell/sky2.h index 8d0bacf4e49cc..8ee73ae087dfc 100644 --- a/drivers/net/ethernet/marvell/sky2.h +++ b/drivers/net/ethernet/marvell/sky2.h @@ -2195,7 +2195,7 @@ struct rx_ring_info { struct sk_buff *skb; dma_addr_t data_addr; DEFINE_DMA_UNMAP_LEN(data_size); - dma_addr_t frag_addr[ETH_JUMBO_MTU >> PAGE_SHIFT ?: 1]; + dma_addr_t frag_addr[ETH_JUMBO_MTU >> PAGE_SHIFT_MIN ?: 1]; }; enum flow_control { From patchwork Mon Oct 14 10:58:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834694 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9B36D1A43B for ; Mon, 14 Oct 2024 11:00:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E5D26B00AD; Mon, 14 Oct 2024 07:00:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4952E6B00AE; Mon, 14 Oct 2024 07:00:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 333AD6B00AF; Mon, 14 Oct 2024 07:00:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1408A6B00AD for ; Mon, 14 Oct 2024 07:00:52 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E48CD140D34 for ; Mon, 14 Oct 2024 11:00:43 +0000 (UTC) X-FDA: 82671915018.11.4A649C4 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf29.hostedemail.com (Postfix) with ESMTP id 0157F120011 for ; Mon, 14 Oct 2024 11:00:41 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903578; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e7pcQwcyCDB+vF+TDiljSktz52PqeG01ydw3eqf01m8=; b=PWLgAJ+u1vrKla4SKRkuw6wq+jU3wg+QIA9mRt30CYhwUUq3bqF8SUS4hELxceymL7jKDf a0msPzsXSY3mOhUE61/QIjbPX1zOfapC/GduuKWIlTDihIEwFuqPaKuxuMVczGBJQ2XbqF jH32eznO3rcpYHn86rpIao4c5b0/iFo= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903578; a=rsa-sha256; cv=none; b=ga75XgV+K6sB2FLlR08KgoGjx/mexXPUfFtUafl2tKiNlANjWHGaWEf14lnuXrpCKoQeHT 6TFP8Dns67hhN79jlvoAaTxvnUJHbTJ6iMBsV4ps3c5abOna1Wu3cQ6dXjpq0PtRaNm9RV ovhs7jsmfEdaqeuPrSYwSxzcbztNrdk= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D37AC169C; Mon, 14 Oct 2024 04:01:18 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D25203F51B; Mon, 14 Oct 2024 04:00:45 -0700 (PDT) From: Ryan Roberts To: "David S. Miller" , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Eric Dumazet , Greg Marsden , Ivan Ivanov , Jakub Kicinski , Jijie Shao , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Paolo Abeni , Salil Mehta , Will Deacon , Yisen Zhuang Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org Subject: [RFC PATCH v1 26/57] net: hns3: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:33 +0100 Message-ID: <20241014105912.3207374-26-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 0157F120011 X-Stat-Signature: 3jakw1u5os8bkncmn5x3zm8ebogbox7t X-HE-Tag: 1728903641-213920 X-HE-Meta: U2FsdGVkX19NC4Psl6FHnSzaUoz1m6sEOjLF6U1LzumdHNvxEoNEjcGlTiSaRVaxalSw+snbZxQ0so4QwCgR/mYglg6rl4H5K0Hh5q2K3/mJ8Af4Y/jWm2QJLmoDiEsvKBiq1vbKV1LCdS6YfKTsiR1avmm7QVC5UsZA3ra99/8seNF7hDSet+t/cJzFuWXHAwT5Yze1lzHzM+qdOrfhphyglyPXNuIWjc7sJ3mZ9asSj+eb3YxofVBq3Bvy6BVejhJIUOWMSD1haJnY/nyDmGLvBd8leK7l7cm1n8UbGnQZTsYg176QvC0N1alI2utZtWpk3JqvHaU2RgBKwdGmgfX7qKmbYFV04qevNoiU2jm4iUt+mYnxnTA9CqASx51z1A+G71qTTCXBRUdX6ndY46Qm6Tqxysn3Xs9HhS2Lqv+5/MBohQI1zal1YoOFdMUBGg2dBCZV5lmnoyoOLcPy4RtRAB4Av8iCueE0AzV6udSJ3W8DACXWOS4DX9y8OYygPvy/agWYMjSBbCdihmPvPZue4M0bC9vN42g95jklfK7Vbg7q/wxjUcIlZdvDz6KjCaiRZyFsl7aGVHNuK+joiEl9qAprhR2nUVXjhisetr9TWof+FWUof0EjuxdLyHGZk+zGBG1UWMgTvcL6ZbblShmP/+hSGR7akdxCnMXSnLKUWdUnzIrOcD6CBxWvYmn3YhDfQf6sRmNxEPjvECRVYYIL1T0JGMOsbnMPAtVnzP+FZ9xpjKza6iam0qxgiXcBeMEh+OyBNi6YGNEFGH+tHu/rFLxgozgxYKhllMenrP6JMf3aYvM68ejN3NlUgjQjLj5CnIEZw1c+cJYT14S11UM92x5L9UfMmBPOj249bwxMm8DsRBW3qq+z5yNadEPIe+oqlopB2kFrcUevV7KBAJsGF3gCsJhuYUw6GIKvBEGeLpKdhN0KMJ0kgIsASYeeTRkWzPLznOmwD5koWKE Ya45cV2m UCHE9zC9AT7EkcmmaZk2XTdUNplZLATarRcW9yROamdmRZrGd/EGOOZJ/jXIfc2z+UdGu9XedyH/wFwcDql+vS7u08NPzcujig2fOU+iiSEG95ONpvWtXH6zYqhUR7mJUbewkZ3BmYxqcZSMzONHoeoiP5oNgcxpxA0D8HkCY+XzM8dNq6+MGZolUJnd78dF8Ul2RlYc9YB37OM3PDBOptWaJ3NjAqY4glJS1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Convert a CPP conditional to a C conditional. The compiler will dead code strip when doing a compile-time page size build, for the same end effect. But this will also work with boot-time page size builds. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ drivers/net/ethernet/hisilicon/hns3/hns3_enet.h | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h index d36c4ed16d8dd..5e675721b7364 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h +++ b/drivers/net/ethernet/hisilicon/hns3/hns3_enet.h @@ -681,10 +681,8 @@ static inline bool hns3_nic_resetting(struct net_device *netdev) static inline unsigned int hns3_page_order(struct hns3_enet_ring *ring) { -#if (PAGE_SIZE < 8192) - if (ring->buf_size > (PAGE_SIZE / 2)) + if (PAGE_SIZE < 8192 && ring->buf_size > (PAGE_SIZE / 2)) return 1; -#endif return 0; } From patchwork Mon Oct 14 10:58:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11060D1A43B for ; Mon, 14 Oct 2024 11:00:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8DD316B00AE; Mon, 14 Oct 2024 07:00:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 864A76B00B0; Mon, 14 Oct 2024 07:00:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6DE136B00B1; Mon, 14 Oct 2024 07:00:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 48A906B00AE for ; Mon, 14 Oct 2024 07:00:55 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 7C8201C6153 for ; Mon, 14 Oct 2024 11:00:46 +0000 (UTC) X-FDA: 82671915060.26.A1F74FE Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf19.hostedemail.com (Postfix) with ESMTP id DE5E81A0028 for ; Mon, 14 Oct 2024 11:00:45 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf19.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903538; a=rsa-sha256; cv=none; b=vkBwMavT9nUK7IQFS1k3zaO2/QuqPXPW8QoJxkIJYBK+6BTob+iCNcTWd3l+r2QOOMqQrf 1SHVFWJ64UU1yVGPdcvAzt5eewwDoOobtemZNIefjHetV4/zGOW4mUs1JUoeomT7gJG2a5 YqvFa4cbXDz81PaWkJB5+tTvxN+CQdY= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf19.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903538; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cjEGbvB+Gk2WgjQzVrxDcv3CTXI9xiLVm+8stz+1Wsc=; b=4A7OSBaNKRQLpvFmm87e7FXQtQzq27Nn0/k4TY14/V6DCKQxVmdNt1v0sYSoza4mt794Rw ZXmS4f8oUjnP20QYfAY04hTcI6cSPsL76Kf/yj6xmAR+xcIK7uBEaw3tUPMr/B2yEyoTv2 fJklWQ0lwqU+UrPl2lNQFR6/mout6Ck= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 47DC9169E; Mon, 14 Oct 2024 04:01:22 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 84C1A3F51B; Mon, 14 Oct 2024 04:00:49 -0700 (PDT) From: Ryan Roberts To: "David S. Miller" , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Eric Dumazet , Greg Marsden , Ivan Ivanov , Jakub Kicinski , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Paolo Abeni , Will Deacon Cc: Ryan Roberts , intel-wired-lan@lists.osuosl.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org Subject: [RFC PATCH v1 27/57] net: e1000: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:34 +0100 Message-ID: <20241014105912.3207374-27-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: DE5E81A0028 X-Stat-Signature: qs47wnnchyt6xyshjd7obphcr517nqd3 X-Rspam-User: X-HE-Tag: 1728903645-457802 X-HE-Meta: U2FsdGVkX1/HGddPOJqXefwqPExwowsgdvd37gncvs8zBi9JPYsMwkoI7xcZ+MfZn5AyxyXafhJ+6h8jBoUtTbiMaQg6JWkWREI37Dao/xIwORz1uIM3+tUAQ2xp4fIMVUhdh9anEneHMZUpn1VuCU2YXjjP05FBobq86qHV3XerPlyIUUGzUr5GTBK4Q66H7vQkUFtnRsMCtyBdHUDvdpcTM2cuCnYFxhytjqkEq0EUttEKdtmod+p1Ypj/TJaSTjfjsGzRy2H3tglBJuIVWzcgEtxw0FfgyIOPbSdocKCGshvGrPYwCBEYfi8Ka55f/u1zmQ0vCHteRSe6OB1Q0tO4Zi8FrDRPaS4Jch39lRzkyjAWUQLKsULKDEiTZXjvdrHb9vepkMFatNcdjdCz8i4Ybm/YCZg+HzA6BuJGT3Z+yyRa2W9J0ZLgxxgscfchRuJTO694vwPViakxnk44kKxlrim2RetzdgMrTWjlN9EcO1HIdMNTxBZkX0mWtVZXkr2k7OET3klFSTiG9xtBAkYH74LfZeYWSTvPd4dA+iiZ72LNQ2x2MdL/QOWrq8s2U75/7dSpgPbGSwYqlELIFAZUrfLDeXNbDRuN57E6t3t1j94W39MEtfqcQq7nKLQQacSuEUAw9YYAeanFdicQUov40zvJWVDjrzhvart/o9f/CntfJ4lX+B+Yi+y061zQXDsawdTId5SB2zhb7NCRuSDvJbBOXK8RZcvf+I9LDDhybehEE+XBwpbtkpGNZ66tHAu4tQYTXCJ7AgIzewamBfIX0U8q0HORaFowtP/sSPbDX75XB290qjGpp3X8mASv+RxuK9Tnhhb45ypjK4qv82au8v28Z+wbPhlbNLbjk6OOm/I1gaHM0JuaE/5ZMfQOzfri3pooFM3r3130i07qyIqcMQbUjgiLGbn/coY2c1U8MjGU/mQ4RuJrcH8+vCKne0Ru+oVzP6XfsTZgZ+r e7VSKWH6 y7eLyOuViWVIpwcgekNVYp0ALyIFkN+cDmo03pIW/z37p9em5lJcSc87hVJz8ylqSqOcW9u+1JvTJHF0CFSziXAFfaoCAU9NCAtd4eRHosxKj2H+RwzPeOfl22FzgPYUDujCrB6jbDdRVVMZX3P62a+eYXtiSM1l81RQadH1YbJnPL2PW0Ch5UVAU5lvAQCTrTQ4OVZ2d9Nxz8d6+V74GuDR1JNLgiWKOjnzl X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Convert CPP conditionals to C conditionals. The compiler will dead code strip when doing a compile-time page size build, for the same end effect. But this will also work with boot-time page size builds. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ drivers/net/ethernet/intel/e1000/e1000_main.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/intel/e1000/e1000_main.c b/drivers/net/ethernet/intel/e1000/e1000_main.c index ab7ae418d2948..cc14788f5bb04 100644 --- a/drivers/net/ethernet/intel/e1000/e1000_main.c +++ b/drivers/net/ethernet/intel/e1000/e1000_main.c @@ -3553,12 +3553,10 @@ static int e1000_change_mtu(struct net_device *netdev, int new_mtu) if (max_frame <= E1000_RXBUFFER_2048) adapter->rx_buffer_len = E1000_RXBUFFER_2048; - else -#if (PAGE_SIZE >= E1000_RXBUFFER_16384) + else if (PAGE_SIZE >= E1000_RXBUFFER_16384) adapter->rx_buffer_len = E1000_RXBUFFER_16384; -#elif (PAGE_SIZE >= E1000_RXBUFFER_4096) + else if (PAGE_SIZE >= E1000_RXBUFFER_4096) adapter->rx_buffer_len = PAGE_SIZE; -#endif /* adjust allocation if LPE protects us, and we aren't using SBP */ if (!hw->tbi_compatibility_on && From patchwork Mon Oct 14 10:58:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834696 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D0AAD1A443 for ; Mon, 14 Oct 2024 11:00:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E9F16B00B1; Mon, 14 Oct 2024 07:00:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2990D6B00B2; Mon, 14 Oct 2024 07:00:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 160066B00B3; Mon, 14 Oct 2024 07:00:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E53F26B00B1 for ; Mon, 14 Oct 2024 07:00:58 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D47D6140D62 for ; Mon, 14 Oct 2024 11:00:50 +0000 (UTC) X-FDA: 82671915144.13.367C055 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf14.hostedemail.com (Postfix) with ESMTP id E33B2100016 for ; Mon, 14 Oct 2024 11:00:49 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf14.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903626; a=rsa-sha256; cv=none; b=Bq9Kyoa2LtJD4/jamzWBaRbbJVF0hrgXSbGodNXjqVxJpk6hvj1z/6zM+O07xfZaJhIQPC RPli1fdRWXVky0Dz1IOcDCQ9t13c/2oLZWOQkJmv2KQx7nu5b/8fylEJlWAcgSiSxw0ztu pHTgeFDDiXtlyhfWEn2xMgJ1kRijcKs= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf14.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903626; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I1DzdKxvBfT38zdagRbaKNLNUfAtWjrd/PCrV+EF/6Q=; b=XzmCQ9aFC5Yd2KczhukOD4OvSwa5KmeQ3kePJ9wPVsV5XOEsy2dmHnfLNAuQt7Qq5G5M4W JlTN2dKD94P8CSHyStHGahsF+oGhJC8HpNEMTrFXXL/YBxW526AQ1c55jhbEC6X/7GN2Fc I6F7MC1k7ndKB/is9zTOeDNvHrMWDtA= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B102016F2; Mon, 14 Oct 2024 04:01:25 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EDCA03F51B; Mon, 14 Oct 2024 04:00:52 -0700 (PDT) From: Ryan Roberts To: "David S. Miller" , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Eric Dumazet , Greg Marsden , Ivan Ivanov , Jakub Kicinski , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Paolo Abeni , Will Deacon Cc: Ryan Roberts , intel-wired-lan@lists.osuosl.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org Subject: [RFC PATCH v1 28/57] net: igbvf: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:35 +0100 Message-ID: <20241014105912.3207374-28-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: jhpu8kk3jfaxb6if45ujz8oa5j5ke7kz X-Rspamd-Queue-Id: E33B2100016 X-Rspamd-Server: rspam02 X-HE-Tag: 1728903649-244503 X-HE-Meta: U2FsdGVkX19qZV1KILMsZJJZ47KD62hvr8BYKvNR3cVOMncLw1SqLzJPhhJq0UaYebt0Le4UcKDGUgKD6XlmH50NnntyIAC4tVbDHSoET3IjCXjWlaJp6ojkS1GTMxo08v7SdxwKFldTsTvfEYXw8Y9UHZTY8B3EuaxKJLrdsTsi/5qN+Ohb8tNgAcTClsY3sXfNk/Th9Y7S1DSQC9en+jMKi2knPBCzE6qvVChCzCEee3HFSTmV+hFAlmvqoeN0vOHOlbZNmYkVYsaw3waZlgGrIECOBihZH91eNGIHuH+Pv3g1u2ngg7lbqgSZiA9gAREoQOM9zn5TYRsTRTHm5wK+2/ZRHEcJr+K2QZjx4vsKGNNYccQw5I3xiVfqw8ActeBltY0LrVv4kM5TPRf+arEvoye/aDPMhHCn5h5+mElGaKaTRDsa/xCcJ8v61zGZtfVxXQfd5A+Jwt+r4KfT8Az5XlMvP20h0U8qzbT9g3RMyKrNVBoGWJh/+Ly5UPgVd82OjBTZ//D2rGJYFfsG82XLvluOo6WnrQSTJA6HOh9bUjG3s3T440jmsvCQ7N0xih9sOgJvfxAKvrG4PQXDUFgFNTsH9SmXc3Df/qOIc58wWPAwqq7KCDSkRhMuB9I4QdmSYjNF2kPPBs79328S/01gVagT5cfj154Q94iSVeubiCUHRQWxh0cB0kkxGFCDsBawj4jidUZFB6nOJXRUVEIlunlDKvo587HFMWRd7vZtSKsaKU3qR7Hv7xeLd60SLBVAphNadTvP0Qq1ykO0BJ18hImSxbN+zxD547S0Zp0lSH1kawr3Ud4eSHZQma7fQvxgRVCR1IGPWu8ghylZfRnJZbH9aCUhj3De0fteoPpfbPtx26hVI2NsPU1a5tH7c8hWciTyrm/F+HmvYUH/P8oWGQM8dQXNvp1xVLWFuji/BPSXVftK47xpQyMSF0h91BsQG3AOgeoc+Xr03cH z6DVOq2J 6Yxr08jUKDj6B9gDM4zZX2rU48CaEZhOTA/Pt7eTwNPck7ACrJIABHjQYPhBzi+GI7PODcaEHoZggYolm7mRZTMh510UZ69C7eyq15P/HHZOY0a21Pb11LtyCRHq0MqEPbxWThvuV+G5o/ix8Ub7mnkpUTRT18+4YfpwdBju9STZm802QWUKAv69Q6BcHcalIDbZq5rRshRImZjQH3qvv+diVcxxB3xqE0HjO X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Convert CPP conditionals to C conditionals. The compiler will dead code strip when doing a compile-time page size build, for the same end effect. But this will also work with boot-time page size builds. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ drivers/net/ethernet/intel/igbvf/netdev.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/intel/igbvf/netdev.c b/drivers/net/ethernet/intel/igbvf/netdev.c index 925d7286a8ee4..2e11d999168de 100644 --- a/drivers/net/ethernet/intel/igbvf/netdev.c +++ b/drivers/net/ethernet/intel/igbvf/netdev.c @@ -2419,12 +2419,10 @@ static int igbvf_change_mtu(struct net_device *netdev, int new_mtu) adapter->rx_buffer_len = 1024; else if (max_frame <= 2048) adapter->rx_buffer_len = 2048; - else -#if (PAGE_SIZE / 2) > 16384 + else if ((PAGE_SIZE / 2) > 16384) adapter->rx_buffer_len = 16384; -#else + else adapter->rx_buffer_len = PAGE_SIZE / 2; -#endif /* adjust allocation if LPE protects us, and we aren't using SBP */ if ((max_frame == ETH_FRAME_LEN + ETH_FCS_LEN) || From patchwork Mon Oct 14 10:58:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834697 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20B4BD1A443 for ; Mon, 14 Oct 2024 11:01:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AD7446B00B6; Mon, 14 Oct 2024 07:01:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A86F36B00B8; Mon, 14 Oct 2024 07:01:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DADA6B00B9; Mon, 14 Oct 2024 07:01:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6DC286B00B6 for ; Mon, 14 Oct 2024 07:01:02 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5744440D9F for ; Mon, 14 Oct 2024 11:00:57 +0000 (UTC) X-FDA: 82671915228.20.D5F7146 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf06.hostedemail.com (Postfix) with ESMTP id EA681180018 for ; Mon, 14 Oct 2024 11:00:55 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903472; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r9lKqnlxiymaFGc73YACDsyodHkCfWFElNljnle06Lc=; b=1U4k0CGkX+7u1KCIm49ThUSt2+MvuBoqpR0fGeiSLr5enMDCpIAWGHNCrgyBiF8z8a+dPa VfoEPinJHgyErYb/Se9ooEtep6lwbdHxcBFj+fj49MTa2uwCRzOzbdwaGVJydc9gJlqm37 QQ3le3fbqdEK3dadPp391Y0opqdO81g= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903472; a=rsa-sha256; cv=none; b=GlPkDoPfFBHbc/gNFgImQYVa7HP8yylxQndBAHqtU1NjjPOmuMk/iJzqh7EhOBe0f/MLk7 9JrhkGYccqyDcq2m2FwmiuVqW5rpf5r14Y6irZyhzb692GLH9ngRCTKWB+boYy9UhVaIIj r3e3Nj6Zdlmn8g4HBGFlP69YEmSoLZ0= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 45F9016A3; Mon, 14 Oct 2024 04:01:29 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 635553F51B; Mon, 14 Oct 2024 04:00:56 -0700 (PDT) From: Ryan Roberts To: "David S. Miller" , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Eric Dumazet , Greg Marsden , Ivan Ivanov , Jakub Kicinski , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Paolo Abeni , Will Deacon Cc: Ryan Roberts , bpf@vger.kernel.org, intel-wired-lan@lists.osuosl.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org Subject: [RFC PATCH v1 29/57] net: igb: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:36 +0100 Message-ID: <20241014105912.3207374-29-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: EA681180018 X-Stat-Signature: dwh6jxhs6ibadxum9w68qun81ftoprt8 X-Rspam-User: X-HE-Tag: 1728903655-544683 X-HE-Meta: U2FsdGVkX1+qyP7Gs9lE42kJpqK21GQfLTk4cer9EMQUOt7fRmVXngYCT78BdpxAbpI/BM7SbifhL7QVk+kWSKJD71KKj/tXfDTy2ly2qpVsQjOIlTzYL8K0bic366k6S6fRt5N6K7QU2QsBYlo5gmRtnBYqd2WFqi1WEdTnr++Ww3jnMuPIrV+2hqdX4iMLlrpy53PvPvsqDsMCvZQrHdX+HCKMH+X0P3IvTIC4c2sjWi/eeDdUC2Zsd3ZgM3qp+IFKBQDzA4wFeI2qjuI2su/1KYsYj6P2NjbpyhaiD4nkEHt4kZR0e+Yr69wYI9QG2X3hLXUAT2txmCkbbEVbfxtHposYAeehpfR5QdSkLOpDPTcNBKJsx8hI8/nXvOzN2ZMrE7HyZw81anRO+6druS7BLfXAF3X/uCyXBmdkiYuof+ZdSQXciJ5c0AAF1QSBicg7qOko+wuWFSc2B0bvEDf1lTQw7uTNtCII1Ft5ybrhe4l4LNwqF4aJpzoNhbRYzltvLDtwSc56SH+OnzagAK0SSGJPzVwiOk0OAeB7iGOH3I8PPrP6wP30ogdU3oVNg6XQuSwRrRWweN2ED5veX2TzjNSnZp6OPU0R3EgOUA59OSWUL14JDKHmcFdO+pClf+n37cwpvUsaPg6U8wKnZ3j0yXXzfQJQyGg2JztEV1rigqX+nx/sdREjSwTMudUmVX++YWF8F3g3YLgml91T99cs/1xehk0lXCqag/1Y3LUUh+GDtaXaCV8CIJMCL/fpb/g9sbf8bdM+M+OrCK4v4tZWP+z74C+c5Y2icO5DaHNHrYkYiSpFf2TxD9qOagQuW2WBerEwzQIm9w0ileBDFLfEp/ISSJlAjANgk5HHAFYbH9BxXdGbgRhOnrD2pw8DCuCBpIwgPlgSqrHJ8I0I6HydIB1iuNFsiC8hreXJ3N8og7twLuVEcjlOr+AfDbluOdmeTz4QXJ44x2RU1il t0RTYZks Wu7J4ARsci1hoh/FOyJOOm/tdF+GxQ4bPPbtrPwUlHdz0jd939lpAB8r4WJ7Cinf9KbMuwL4WmiTS9e3Vs8yMikIhd41sf/irPorLREhGB3eQi9CaIMEOsBdqK9/FlGwll0Ubh9Wr8AAptmJG3O2np8a9+DLw3EiRJ3/aBlIXzEI4Xs5iDarN1mal+1xOrWTn1W2aeDh7+2FmVcNtkiL7Z/rH4EPC58kxr0Jb X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Convert CPP conditionals to C conditionals. The compiler will dead code strip when doing a compile-time page size build, for the same end effect. But this will also work with boot-time page size builds. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ drivers/net/ethernet/intel/igb/igb.h | 25 ++-- drivers/net/ethernet/intel/igb/igb_main.c | 149 +++++++++++----------- 2 files changed, 82 insertions(+), 92 deletions(-) diff --git a/drivers/net/ethernet/intel/igb/igb.h b/drivers/net/ethernet/intel/igb/igb.h index 3c2dc7bdebb50..04aeebcd363b3 100644 --- a/drivers/net/ethernet/intel/igb/igb.h +++ b/drivers/net/ethernet/intel/igb/igb.h @@ -158,7 +158,6 @@ struct vf_mac_filter { * up negative. In these cases we should fall back to the 3K * buffers. */ -#if (PAGE_SIZE < 8192) #define IGB_MAX_FRAME_BUILD_SKB (IGB_RXBUFFER_1536 - NET_IP_ALIGN) #define IGB_2K_TOO_SMALL_WITH_PADDING \ ((NET_SKB_PAD + IGB_TS_HDR_LEN + IGB_RXBUFFER_1536) > SKB_WITH_OVERHEAD(IGB_RXBUFFER_2048)) @@ -177,6 +176,9 @@ static inline int igb_skb_pad(void) { int rx_buf_len; + if (PAGE_SIZE >= 8192) + return NET_SKB_PAD + NET_IP_ALIGN; + /* If a 2K buffer cannot handle a standard Ethernet frame then * optimize padding for a 3K buffer instead of a 1.5K buffer. * @@ -196,9 +198,6 @@ static inline int igb_skb_pad(void) } #define IGB_SKB_PAD igb_skb_pad() -#else -#define IGB_SKB_PAD (NET_SKB_PAD + NET_IP_ALIGN) -#endif /* How many Rx Buffers do we bundle into one write to the hardware ? */ #define IGB_RX_BUFFER_WRITE 16 /* Must be power of 2 */ @@ -280,7 +279,7 @@ struct igb_tx_buffer { struct igb_rx_buffer { dma_addr_t dma; struct page *page; -#if (BITS_PER_LONG > 32) || (PAGE_SIZE >= 65536) +#if (BITS_PER_LONG > 32) || (PAGE_SIZE_MAX >= 65536) __u32 page_offset; #else __u16 page_offset; @@ -403,22 +402,20 @@ enum e1000_ring_flags_t { static inline unsigned int igb_rx_bufsz(struct igb_ring *ring) { -#if (PAGE_SIZE < 8192) - if (ring_uses_large_buffer(ring)) - return IGB_RXBUFFER_3072; + if (PAGE_SIZE < 8192) { + if (ring_uses_large_buffer(ring)) + return IGB_RXBUFFER_3072; - if (ring_uses_build_skb(ring)) - return IGB_MAX_FRAME_BUILD_SKB; -#endif + if (ring_uses_build_skb(ring)) + return IGB_MAX_FRAME_BUILD_SKB; + } return IGB_RXBUFFER_2048; } static inline unsigned int igb_rx_pg_order(struct igb_ring *ring) { -#if (PAGE_SIZE < 8192) - if (ring_uses_large_buffer(ring)) + if (PAGE_SIZE < 8192 && ring_uses_large_buffer(ring)) return 1; -#endif return 0; } diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 1ef4cb871452a..4f2c53dece1a2 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -4797,9 +4797,7 @@ void igb_configure_rx_ring(struct igb_adapter *adapter, static void igb_set_rx_buffer_len(struct igb_adapter *adapter, struct igb_ring *rx_ring) { -#if (PAGE_SIZE < 8192) struct e1000_hw *hw = &adapter->hw; -#endif /* set build_skb and buffer size flags */ clear_ring_build_skb_enabled(rx_ring); @@ -4810,12 +4808,11 @@ static void igb_set_rx_buffer_len(struct igb_adapter *adapter, set_ring_build_skb_enabled(rx_ring); -#if (PAGE_SIZE < 8192) - if (adapter->max_frame_size > IGB_MAX_FRAME_BUILD_SKB || + if (PAGE_SIZE < 8192 && + (adapter->max_frame_size > IGB_MAX_FRAME_BUILD_SKB || IGB_2K_TOO_SMALL_WITH_PADDING || - rd32(E1000_RCTL) & E1000_RCTL_SBP) + rd32(E1000_RCTL) & E1000_RCTL_SBP)) set_ring_uses_large_buffer(rx_ring); -#endif } /** @@ -5314,12 +5311,10 @@ static void igb_set_rx_mode(struct net_device *netdev) E1000_RCTL_VFE); wr32(E1000_RCTL, rctl); -#if (PAGE_SIZE < 8192) - if (!adapter->vfs_allocated_count) { + if (PAGE_SIZE < 8192 && !adapter->vfs_allocated_count) { if (adapter->max_frame_size <= IGB_MAX_FRAME_BUILD_SKB) rlpml = IGB_MAX_FRAME_BUILD_SKB; } -#endif wr32(E1000_RLPML, rlpml); /* In order to support SR-IOV and eventually VMDq it is necessary to set @@ -5338,11 +5333,10 @@ static void igb_set_rx_mode(struct net_device *netdev) /* enable Rx jumbo frames, restrict as needed to support build_skb */ vmolr &= ~E1000_VMOLR_RLPML_MASK; -#if (PAGE_SIZE < 8192) - if (adapter->max_frame_size <= IGB_MAX_FRAME_BUILD_SKB) + if (PAGE_SIZE < 8192 && + adapter->max_frame_size <= IGB_MAX_FRAME_BUILD_SKB) vmolr |= IGB_MAX_FRAME_BUILD_SKB; else -#endif vmolr |= MAX_JUMBO_FRAME_SIZE; vmolr |= E1000_VMOLR_LPE; @@ -8435,17 +8429,17 @@ static bool igb_can_reuse_rx_page(struct igb_rx_buffer *rx_buffer, if (!dev_page_is_reusable(page)) return false; -#if (PAGE_SIZE < 8192) - /* if we are only owner of page we can reuse it */ - if (unlikely((rx_buf_pgcnt - pagecnt_bias) > 1)) - return false; -#else + if (PAGE_SIZE < 8192) { + /* if we are only owner of page we can reuse it */ + if (unlikely((rx_buf_pgcnt - pagecnt_bias) > 1)) + return false; + } else { #define IGB_LAST_OFFSET \ (SKB_WITH_OVERHEAD(PAGE_SIZE) - IGB_RXBUFFER_2048) - if (rx_buffer->page_offset > IGB_LAST_OFFSET) - return false; -#endif + if (rx_buffer->page_offset > IGB_LAST_OFFSET) + return false; + } /* If we have drained the page fragment pool we need to update * the pagecnt_bias and page count so that we fully restock the @@ -8473,20 +8467,22 @@ static void igb_add_rx_frag(struct igb_ring *rx_ring, struct sk_buff *skb, unsigned int size) { -#if (PAGE_SIZE < 8192) - unsigned int truesize = igb_rx_pg_size(rx_ring) / 2; -#else - unsigned int truesize = ring_uses_build_skb(rx_ring) ? + unsigned int truesize; + + if (PAGE_SIZE < 8192) + truesize = igb_rx_pg_size(rx_ring) / 2; + else + truesize = ring_uses_build_skb(rx_ring) ? SKB_DATA_ALIGN(IGB_SKB_PAD + size) : SKB_DATA_ALIGN(size); -#endif + skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, rx_buffer->page, rx_buffer->page_offset, size, truesize); -#if (PAGE_SIZE < 8192) - rx_buffer->page_offset ^= truesize; -#else - rx_buffer->page_offset += truesize; -#endif + + if (PAGE_SIZE < 8192) + rx_buffer->page_offset ^= truesize; + else + rx_buffer->page_offset += truesize; } static struct sk_buff *igb_construct_skb(struct igb_ring *rx_ring, @@ -8494,16 +8490,16 @@ static struct sk_buff *igb_construct_skb(struct igb_ring *rx_ring, struct xdp_buff *xdp, ktime_t timestamp) { -#if (PAGE_SIZE < 8192) - unsigned int truesize = igb_rx_pg_size(rx_ring) / 2; -#else - unsigned int truesize = SKB_DATA_ALIGN(xdp->data_end - - xdp->data_hard_start); -#endif unsigned int size = xdp->data_end - xdp->data; + unsigned int truesize; unsigned int headlen; struct sk_buff *skb; + if (PAGE_SIZE < 8192) + truesize = igb_rx_pg_size(rx_ring) / 2; + else + truesize = SKB_DATA_ALIGN(xdp->data_end - xdp->data_hard_start); + /* prefetch first cache line of first page */ net_prefetch(xdp->data); @@ -8529,11 +8525,10 @@ static struct sk_buff *igb_construct_skb(struct igb_ring *rx_ring, skb_add_rx_frag(skb, 0, rx_buffer->page, (xdp->data + headlen) - page_address(rx_buffer->page), size, truesize); -#if (PAGE_SIZE < 8192) - rx_buffer->page_offset ^= truesize; -#else - rx_buffer->page_offset += truesize; -#endif + if (PAGE_SIZE < 8192) + rx_buffer->page_offset ^= truesize; + else + rx_buffer->page_offset += truesize; } else { rx_buffer->pagecnt_bias++; } @@ -8546,16 +8541,17 @@ static struct sk_buff *igb_build_skb(struct igb_ring *rx_ring, struct xdp_buff *xdp, ktime_t timestamp) { -#if (PAGE_SIZE < 8192) - unsigned int truesize = igb_rx_pg_size(rx_ring) / 2; -#else - unsigned int truesize = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) + - SKB_DATA_ALIGN(xdp->data_end - - xdp->data_hard_start); -#endif unsigned int metasize = xdp->data - xdp->data_meta; + unsigned int truesize; struct sk_buff *skb; + if (PAGE_SIZE < 8192) + truesize = igb_rx_pg_size(rx_ring) / 2; + else + truesize = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) + + SKB_DATA_ALIGN(xdp->data_end - + xdp->data_hard_start); + /* prefetch first cache line of first page */ net_prefetch(xdp->data_meta); @@ -8575,11 +8571,10 @@ static struct sk_buff *igb_build_skb(struct igb_ring *rx_ring, skb_hwtstamps(skb)->hwtstamp = timestamp; /* update buffer offset */ -#if (PAGE_SIZE < 8192) - rx_buffer->page_offset ^= truesize; -#else - rx_buffer->page_offset += truesize; -#endif + if (PAGE_SIZE < 8192) + rx_buffer->page_offset ^= truesize; + else + rx_buffer->page_offset += truesize; return skb; } @@ -8634,14 +8629,14 @@ static unsigned int igb_rx_frame_truesize(struct igb_ring *rx_ring, { unsigned int truesize; -#if (PAGE_SIZE < 8192) - truesize = igb_rx_pg_size(rx_ring) / 2; /* Must be power-of-2 */ -#else - truesize = ring_uses_build_skb(rx_ring) ? - SKB_DATA_ALIGN(IGB_SKB_PAD + size) + - SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) : - SKB_DATA_ALIGN(size); -#endif + if (PAGE_SIZE < 8192) + truesize = igb_rx_pg_size(rx_ring) / 2; /* Must be power-of-2 */ + else + truesize = ring_uses_build_skb(rx_ring) ? + SKB_DATA_ALIGN(IGB_SKB_PAD + size) + + SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) : + SKB_DATA_ALIGN(size); + return truesize; } @@ -8650,11 +8645,11 @@ static void igb_rx_buffer_flip(struct igb_ring *rx_ring, unsigned int size) { unsigned int truesize = igb_rx_frame_truesize(rx_ring, size); -#if (PAGE_SIZE < 8192) - rx_buffer->page_offset ^= truesize; -#else - rx_buffer->page_offset += truesize; -#endif + + if (PAGE_SIZE < 8192) + rx_buffer->page_offset ^= truesize; + else + rx_buffer->page_offset += truesize; } static inline void igb_rx_checksum(struct igb_ring *ring, @@ -8825,12 +8820,12 @@ static struct igb_rx_buffer *igb_get_rx_buffer(struct igb_ring *rx_ring, struct igb_rx_buffer *rx_buffer; rx_buffer = &rx_ring->rx_buffer_info[rx_ring->next_to_clean]; - *rx_buf_pgcnt = -#if (PAGE_SIZE < 8192) - page_count(rx_buffer->page); -#else - 0; -#endif + + if (PAGE_SIZE < 8192) + *rx_buf_pgcnt = page_count(rx_buffer->page); + else + *rx_buf_pgcnt = 0; + prefetchw(rx_buffer->page); /* we are reusing so sync this buffer for CPU use */ @@ -8881,9 +8876,8 @@ static int igb_clean_rx_irq(struct igb_q_vector *q_vector, const int budget) int rx_buf_pgcnt; /* Frame size depend on rx_ring setup when PAGE_SIZE=4K */ -#if (PAGE_SIZE < 8192) - frame_sz = igb_rx_frame_truesize(rx_ring, 0); -#endif + if (PAGE_SIZE < 8192) + frame_sz = igb_rx_frame_truesize(rx_ring, 0); xdp_init_buff(&xdp, frame_sz, &rx_ring->xdp_rxq); while (likely(total_packets < budget)) { @@ -8932,10 +8926,9 @@ static int igb_clean_rx_irq(struct igb_q_vector *q_vector, const int budget) xdp_prepare_buff(&xdp, hard_start, offset, size, true); xdp_buff_clear_frags_flag(&xdp); -#if (PAGE_SIZE > 4096) /* At larger PAGE_SIZE, frame_sz depend on len size */ - xdp.frame_sz = igb_rx_frame_truesize(rx_ring, size); -#endif + if (PAGE_SIZE > 4096) + xdp.frame_sz = igb_rx_frame_truesize(rx_ring, size); skb = igb_run_xdp(adapter, rx_ring, &xdp); } From patchwork Mon Oct 14 10:58:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834698 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDAA2D1A43B for ; Mon, 14 Oct 2024 11:01:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 708C96B00BA; Mon, 14 Oct 2024 07:01:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 691346B00BB; Mon, 14 Oct 2024 07:01:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 50AA96B00BC; Mon, 14 Oct 2024 07:01:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 2F8E96B00BA for ; Mon, 14 Oct 2024 07:01:05 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 4DD851A0D61 for ; Mon, 14 Oct 2024 11:00:51 +0000 (UTC) X-FDA: 82671915564.04.7156D3B Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf21.hostedemail.com (Postfix) with ESMTP id 735561C0020 for ; Mon, 14 Oct 2024 11:00:51 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf21.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903522; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S0gIad1IKQmWm9pceFZyemJEemJ3RuiyqEiPFLVA8hg=; b=WoXMD0xVBf8eP82PKZyNs0wm0a7PNSgTgQnj8dc/b14eAFG5G2+J9F0fHsEm1frEPAjWXm mQtHe7OXBWeLjfhr0QuuwgqtFmwLcHi1ZmGMP7d/zDFJH99oDpePTeUkHp6hWF46p8QU86 w/rqH7PJTCeyWsw5mTl9rh8zqecxTuI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903522; a=rsa-sha256; cv=none; b=Ywq0IG46KJFQsAeX+bFmIS5Sl6dArPrcmwzRakwE8/9oqYBgra4mlkxhRaCRjPFF9FXIRa X7O3uvPgqrwQBn36DtTzXxKhJnza8ze0bl6H3wq4wEMoZ2tV+zAdepZKjG3vNbXhTKxe9y 7pBgPWottvV+DLfcx6h/7/V2R3QBJgg= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf21.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1454116F3; Mon, 14 Oct 2024 04:01:32 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EAD463F51B; Mon, 14 Oct 2024 04:00:59 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon , Yury Norov Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 30/57] drivers/base: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:37 +0100 Message-ID: <20241014105912.3207374-30-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 735561C0020 X-Stat-Signature: m9k3oz4kjoe9xaxekoe9giucdr3mnjro X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1728903651-511570 X-HE-Meta: U2FsdGVkX1/CWF1oVEcjoKRHXv7U5GNFalXPqQu27XmywtrLVSCIN87sUhv5nRfMhyJPUoRpWM5zY6n5oPePWRSNvhocqEpQsKAnWiXRXNknlzpesi2txya/A90yKygVPRSbxfJO7mU5gvK0VNJVQ3CrF3C8DlHfZoUdSAEQDRETWG+k9MLBHfN/FlcM8oL9uVYtbezf5F+ZglHBsJcbPHFGOGLRgJBNY96fGV1TH8j2wjEC+Oh0BCu75S/+qShEj5LadJd4A0W3paDqYkL2tXnH9AJxn1be1WQxpsnkn7oJDUepntM0T5yK8i4D+7VKi0L8SGK0N0OyJ8U4EDAvwDk80HSkEWV/Bouusu/FzGyB+Cji2saK9kcvjwxBVzomqI5xofQlIDqFuPH+cPYv/h/9+qFBYipLtSSHeLzrXji+meL/K9jzegqAVVm0AMJvhycGhAM1IQFvC2MHFWdFLLsR4MsIO2fu0qtPvZSqyj2FKhzsGpctUj37TE6geDmCRhwdpj6dIkQTkCs4oMDUD4SbB3yuYAHkalaJR8jENOY6+cAhtp4sih1hh7ORDWqXl1eMzyzPnkg+DHdvF4f7t8cUe4U5LLnuE3OujA+vBtqBGlfHa7QZpaiOU+Uv5C+qcr208EfuC6n/K9vhgs69ZCzj9Wh+MIlGKYDB874X3ewt9Y7W9sRCyPFR1pPhKcMTWwgnAX64kULnleaJ9IKAak+se8AcRuDJ+BARLktDHhtPlJ3pgNAbN3sE1Mba+yS97fkJduVLHpxn3iWcXVKps5I59Jt5eqS6iPsui0U/dxP2ZMfVKrJOntXzmezRYwGDa3kFgRicN98WzxVq/eE9kHb7cLakyZ8NE66WwN1WT0pSwjAVoFdCOTMQQ13KybmLXdRVSt8qATbc56pwK/l/6AhQaofqcxP51+/aQm7w2R3Yzob361UFsLT8H3qAH0KquAlbUmdns3Dn4gCN5pA 6FK1y0cx wowj+OoJ2DHNheNi15/u5jy/EVx9KNWBHb9mrHI19atc2dC2Q81FUjbPtYvCxu5mmfdU93YFxYl+N7JYXrz6C8hc9/6xhxTNu+eNl2ZYUsj8R/VVT3+n7gvGxFDeQHvVbJKVO8XzzTjJUGhc8yoZouUEgHRA/tUzQvjLgQ5b4b3xaMSaXkcEvLzgqgRaYY2U7gDVkql+MLd1SxDLnvNA0db0k2Z3iHM7RpqCgBGqAtQ/hC3ebHYHhAJ+psGy9qHsZp0uUOqUFKicugLs7MsQdXglLiUHqfYP7u/9hh2wllkylEG0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Update BUILD_BUG_ON() to test against page size limits. CPUMAP_FILE_MAX_BYTES and CPULIST_FILE_MAX_BYTES are both defined relative to PAGE_SIZE, so when these values are assigned to global variables via BIN_ATTR_RO(), let's wrap them with DEFINE_GLOBAL_PAGE_SIZE_VAR() so that their assignment can be deferred until boot-time. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ drivers/base/node.c | 6 +++--- drivers/base/topology.c | 32 ++++++++++++++++---------------- include/linux/cpumask.h | 5 +++++ 3 files changed, 24 insertions(+), 19 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index eb72580288e62..30e6549e4c438 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -45,7 +45,7 @@ static inline ssize_t cpumap_read(struct file *file, struct kobject *kobj, return n; } -static BIN_ATTR_RO(cpumap, CPUMAP_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(cpumap, CPUMAP_FILE_MAX_BYTES); static inline ssize_t cpulist_read(struct file *file, struct kobject *kobj, struct bin_attribute *attr, char *buf, @@ -66,7 +66,7 @@ static inline ssize_t cpulist_read(struct file *file, struct kobject *kobj, return n; } -static BIN_ATTR_RO(cpulist, CPULIST_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(cpulist, CPULIST_FILE_MAX_BYTES); /** * struct node_access_nodes - Access class device to hold user visible @@ -558,7 +558,7 @@ static ssize_t node_read_distance(struct device *dev, * buf is currently PAGE_SIZE in length and each node needs 4 chars * at the most (distance + space or newline). */ - BUILD_BUG_ON(MAX_NUMNODES * 4 > PAGE_SIZE); + BUILD_BUG_ON(MAX_NUMNODES * 4 > PAGE_SIZE_MIN); for_each_online_node(i) { len += sysfs_emit_at(buf, len, "%s%d", diff --git a/drivers/base/topology.c b/drivers/base/topology.c index 89f98be5c5b99..bdbdbefd95b15 100644 --- a/drivers/base/topology.c +++ b/drivers/base/topology.c @@ -62,47 +62,47 @@ define_id_show_func(ppin, "0x%llx"); static DEVICE_ATTR_ADMIN_RO(ppin); define_siblings_read_func(thread_siblings, sibling_cpumask); -static BIN_ATTR_RO(thread_siblings, CPUMAP_FILE_MAX_BYTES); -static BIN_ATTR_RO(thread_siblings_list, CPULIST_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(thread_siblings, CPUMAP_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(thread_siblings_list, CPULIST_FILE_MAX_BYTES); define_siblings_read_func(core_cpus, sibling_cpumask); -static BIN_ATTR_RO(core_cpus, CPUMAP_FILE_MAX_BYTES); -static BIN_ATTR_RO(core_cpus_list, CPULIST_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(core_cpus, CPUMAP_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(core_cpus_list, CPULIST_FILE_MAX_BYTES); define_siblings_read_func(core_siblings, core_cpumask); -static BIN_ATTR_RO(core_siblings, CPUMAP_FILE_MAX_BYTES); -static BIN_ATTR_RO(core_siblings_list, CPULIST_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(core_siblings, CPUMAP_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(core_siblings_list, CPULIST_FILE_MAX_BYTES); #ifdef TOPOLOGY_CLUSTER_SYSFS define_siblings_read_func(cluster_cpus, cluster_cpumask); -static BIN_ATTR_RO(cluster_cpus, CPUMAP_FILE_MAX_BYTES); -static BIN_ATTR_RO(cluster_cpus_list, CPULIST_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(cluster_cpus, CPUMAP_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(cluster_cpus_list, CPULIST_FILE_MAX_BYTES); #endif #ifdef TOPOLOGY_DIE_SYSFS define_siblings_read_func(die_cpus, die_cpumask); -static BIN_ATTR_RO(die_cpus, CPUMAP_FILE_MAX_BYTES); -static BIN_ATTR_RO(die_cpus_list, CPULIST_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(die_cpus, CPUMAP_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(die_cpus_list, CPULIST_FILE_MAX_BYTES); #endif define_siblings_read_func(package_cpus, core_cpumask); -static BIN_ATTR_RO(package_cpus, CPUMAP_FILE_MAX_BYTES); -static BIN_ATTR_RO(package_cpus_list, CPULIST_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(package_cpus, CPUMAP_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(package_cpus_list, CPULIST_FILE_MAX_BYTES); #ifdef TOPOLOGY_BOOK_SYSFS define_id_show_func(book_id, "%d"); static DEVICE_ATTR_RO(book_id); define_siblings_read_func(book_siblings, book_cpumask); -static BIN_ATTR_RO(book_siblings, CPUMAP_FILE_MAX_BYTES); -static BIN_ATTR_RO(book_siblings_list, CPULIST_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(book_siblings, CPUMAP_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(book_siblings_list, CPULIST_FILE_MAX_BYTES); #endif #ifdef TOPOLOGY_DRAWER_SYSFS define_id_show_func(drawer_id, "%d"); static DEVICE_ATTR_RO(drawer_id); define_siblings_read_func(drawer_siblings, drawer_cpumask); -static BIN_ATTR_RO(drawer_siblings, CPUMAP_FILE_MAX_BYTES); -static BIN_ATTR_RO(drawer_siblings_list, CPULIST_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(drawer_siblings, CPUMAP_FILE_MAX_BYTES); +static CPU_FILE_BIN_ATTR_RO(drawer_siblings_list, CPULIST_FILE_MAX_BYTES); #endif static struct bin_attribute *bin_attrs[] = { diff --git a/include/linux/cpumask.h b/include/linux/cpumask.h index 53158de44b837..f654b4198abc2 100644 --- a/include/linux/cpumask.h +++ b/include/linux/cpumask.h @@ -1292,4 +1292,9 @@ cpumap_print_list_to_buf(char *buf, const struct cpumask *mask, ? (NR_CPUS * 9)/32 - 1 : PAGE_SIZE) #define CPULIST_FILE_MAX_BYTES (((NR_CPUS * 7)/2 > PAGE_SIZE) ? (NR_CPUS * 7)/2 : PAGE_SIZE) +#define CPU_FILE_BIN_ATTR_RO(_name, _size) \ + DEFINE_GLOBAL_PAGE_SIZE_VAR(struct bin_attribute, \ + bin_attr_##_name, \ + __BIN_ATTR_RO(_name, _size)) + #endif /* __LINUX_CPUMASK_H */ From patchwork Mon Oct 14 10:58:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 975BBD1A443 for ; Mon, 14 Oct 2024 11:01:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B5DC6B00BC; Mon, 14 Oct 2024 07:01:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0EBCC6B00BD; Mon, 14 Oct 2024 07:01:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ECF076B00BE; Mon, 14 Oct 2024 07:01:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C92A36B00BC for ; Mon, 14 Oct 2024 07:01:07 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 64FB8C0D56 for ; Mon, 14 Oct 2024 11:00:59 +0000 (UTC) X-FDA: 82671915438.20.6479062 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf02.hostedemail.com (Postfix) with ESMTP id 6A69B80012 for ; Mon, 14 Oct 2024 11:00:54 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903509; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xJWf5Towi95SWggcq2L3OEdCfoka1zUWhvozEl9HTkI=; b=uOaQ2L82dd2AstQddlOOn0UqqiswFoh2Co5ZMnC4DWb046uIBBBWbYw7H7BGjr8nzEYHKX x1Hk8NQr/mKRktFTHSAKs/gyolP5SUCnPD+al0+Rw5BwCnMlKIN+akdKtYQCNYwaip3WPL 4wiuuCgunFrrqlfsG/ndYPvsovl1tJg= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903509; a=rsa-sha256; cv=none; b=f9jSBI0Xl9ekkoeupeUkN6OeEhfcsa17IYeOGZ7nf/UtNgz7IrwG2YFGWX1CWpP3cyeQ51 b+PzmClYpnkZbqBVgzdWwgyvvc/tnynTbIj/L8wZWteL1vPpkJ8kv0hdpHFQfFueTWextX H9bFfbKdoPdMXiYHcvBUkLArYkg0StM= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D88E716F8; Mon, 14 Oct 2024 04:01:34 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B95D73F51B; Mon, 14 Oct 2024 04:01:02 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-edac@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 31/57] edac: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:38 +0100 Message-ID: <20241014105912.3207374-31-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 6A69B80012 X-Stat-Signature: ioiq3tdxass1scd8zuwfo1i1mut1nuj4 X-Rspam-User: X-HE-Tag: 1728903654-630468 X-HE-Meta: U2FsdGVkX1/GO8tfMugOAZR3xcrP6wxpd0M5G1kagcKSXOhroHVMFPdSE0kvMUNhW7M9qPz5rBjPJezNlqZMT2WWCWkwbu4ltMZBQpnTk2Zlva+x6oN1emzZtQd0uYuQUvWntSrfEWtvA1qYt7fEdngh7nPHJ0KbWy8nEmnMNdVK+q7SzFcWm9REVrVO+oiCyxVS+DnDbWOz6K4Zie0NmecwC3ZfbfRTrC+DSxtK8HkFifrGM5ztIiUIRgKDxF+vYD/D85BX0Y+pDB09l0kyDLzDt5vHxJt1bzjAYyI2TVPk+wyPaevUop5Um8ms7h/reOmxPji8bRokchHLpy1pYN0GRtPzgYiBVHj76lY0pwNiPal2ZOhi7BAadAVrbZZr8f8EKlGXW7w6dZgaGH37mUrF8wX71przzS70bn122C+Aii234tvP9dtSjKFUNYZseRetodwXN77iYCb5EqxAOIORWpg7xYFYc37Dww+T1xmTztp2xYgEHoMSBiIrzFL29JgOOmCylgErH80CCjqvSYNEznG+IDDl+ICcCfxN2hU7GlJhPFPMLemHBS8lnpS9sAwO96ALpSpXb7dfUTgK+w9nXJ/cC5mZp1mBviPE5OKRIlyQ8FQJekAT69+ffbYLB3QY37f9IPGjmauXRcR8Ro4z3nwCgJPs0Rp8DaMg7RMGd/pzu2xpdVa2zGeG7gcl09/qdGv1aWJY8OkIVE++VdbvrsO3Qw0PNS42OpfPEmO3UUoQIZN8kjYJXuQbP1l9HV5REj1mhXWOuG3igpH4DH2mCHKn5Pm4c8P7blLXnydAKpKaZQdZKlxHXiyDZYVhFEAHZqNKA8YngrLK8i7whgRQW4kxwfPtM2YQpfEcOmu4dAYbXcBOd5KaDPumKmUFf+rrBy50zT9OLbr64HO6rnLSvkt8eFjg5QKFhLvkwge701ZHXzw5towjj543dHkXCaRooeBXqfsNQyG1nds X3LmhoDa qZXIc1mvo23N9YLaYlAfKj5tZuEtgNJ65b861TVNAykx+jjBmOlFIR5mWiQVgwGl9ju4dt+I+CX2BPrPI8V4uBmBe2+QlCwheDdKLvS+FHotg4hw5XZtjv22jKBch/aoIIWfQx+n7Yjo7BCcFigmC35wyMwcKJxla3NP3IV51zerf76jSufQBz+H8vAlDsxtS7OyPqVOyaVKUYL+tZk3nz7GXLQLgCmgNh3iP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Convert PAGES_TO_MiB() and MiB_TO_PAGES() to use the ternary operator so that they continue to work with boot-time page size; Boot-time page size can't be used with CPP because it's value is not known at compile time. For compile-time page size builds, the compiler will dead code strip for the same result. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ drivers/edac/edac_mc.h | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/drivers/edac/edac_mc.h b/drivers/edac/edac_mc.h index 881b00eadf7a5..22132ee86e953 100644 --- a/drivers/edac/edac_mc.h +++ b/drivers/edac/edac_mc.h @@ -37,13 +37,12 @@ #include #include -#if PAGE_SHIFT < 20 -#define PAGES_TO_MiB(pages) ((pages) >> (20 - PAGE_SHIFT)) -#define MiB_TO_PAGES(mb) ((mb) << (20 - PAGE_SHIFT)) -#else /* PAGE_SHIFT > 20 */ -#define PAGES_TO_MiB(pages) ((pages) << (PAGE_SHIFT - 20)) -#define MiB_TO_PAGES(mb) ((mb) >> (PAGE_SHIFT - 20)) -#endif +#define PAGES_TO_MiB(pages) (PAGE_SHIFT < 20 ? \ + ((pages) >> (20 - PAGE_SHIFT)) :\ + ((pages) << (PAGE_SHIFT - 20))) +#define MiB_TO_PAGES(mb) (PAGE_SHIFT < 20 ? \ + ((mb) << (20 - PAGE_SHIFT)) : \ + ((mb) >> (PAGE_SHIFT - 20))) #define edac_printk(level, prefix, fmt, arg...) \ printk(level "EDAC " prefix ": " fmt, ##arg) From patchwork Mon Oct 14 10:58:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834700 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 682A3D1A443 for ; Mon, 14 Oct 2024 11:01:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EA88F6B00BE; Mon, 14 Oct 2024 07:01:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E5A8E6B00BF; Mon, 14 Oct 2024 07:01:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD2E66B00C0; Mon, 14 Oct 2024 07:01:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id AAA336B00BE for ; Mon, 14 Oct 2024 07:01:10 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 49586ABC9B for ; Mon, 14 Oct 2024 11:00:55 +0000 (UTC) X-FDA: 82671915816.29.2751DEC Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf23.hostedemail.com (Postfix) with ESMTP id 0A8CD14002F for ; Mon, 14 Oct 2024 11:01:04 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903480; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NJ0XCsdELuRevj5tw1yfqEQTo4IwlrxkOoKIOsf10K4=; b=6F5fSw5z6NU7OvBKZTcA4ZhY7aX7StClfp3xs7HnnPYeKc6AA9DNFgl64ymAGbB7U4XKvD OrO+AgYFwcwy9Qc2mTLmTb4tKARxJNsE80qfVHlpE/ole6zQoW+7bSHY7B5IJJ50NpkMyW ntQ3raGB7PBP2/vojEOEuSuvgqD7VAg= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903480; a=rsa-sha256; cv=none; b=2U76wJQzgDLHyNA4/u+cfOlr8N4eDon4agLPKBKIw//ub82HNkeVdq9Vu1b2pt6Ema3f2Y 3ptUNsO6+Flw7mgQYr/2te3+AZUDGJRrw+ta8tcYTwU8lfiwEBecRSyIYTmDfdkeE+O4tu rmsU4ijVpemuZEQ4slw2ZE+bE4b03dA= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C7AA116F8; Mon, 14 Oct 2024 04:01:37 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8A33C3F51B; Mon, 14 Oct 2024 04:01:05 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Jens Wiklander , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, op-tee@lists.trustedfirmware.org Subject: [RFC PATCH v1 32/57] optee: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:39 +0100 Message-ID: <20241014105912.3207374-32-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 0A8CD14002F X-Stat-Signature: 9q9rsnk39xfwqgnk8698njt1ksi9ct1d X-Rspam-User: X-HE-Tag: 1728903664-41309 X-HE-Meta: U2FsdGVkX1/RbUwuT54pExggmAa4PivHiuu+0i3BCjN/KnfFZ+WVE3OTjBr0sSB3dVxNqfFi2Pm+8+M7PUZZolCO7EaNtHuzJ6zmPIo1Bpm5lAuUD4PskvkxSUdxLpOxroMR6JA7wy//33e44yd081dOFmCQ0HwJ/KImqyJZx25xAHDAlSCrmcFbix/M0jTBiuBg1pp0+1d8Zea0k8/G4q9ZSh2XalxPSeTa5SkRCSrW+dcl6HuU7N9ovbmrdnN1YmFN6bAVjLPtq6+96JDLynd821XxkLEKyedQI7W0PuZ+EDdBhLp3lILhMHxqPXgcD0CSsbOIDUX/hb1flOz2ov6svRatH2kWlNdjC0JkRd6n4JMKl8yb7PZMwRJcnBYFGcSta1U56/APa4Cz2B34MJrdc3k8z1Fp60QPTnzr06F2EA/pBhbnnNbMjv6aZVjVmcTnwkNiAX7d5cwmHfNQCS/10OXY/Ld9h2S+yyhWW8HYp/UwUpDJiOVFQsEkvbIxiCuk+sOLqvxpG26NuzXyOfBkDxnMvo2xSV7b+EwnMGU52ju0M80BjGWEwB7WVl6Es2ggAUZYQ4R5iti9eIKMNjFAAjadYesoCitAiO3jE1VP6F04KsMIzVGlbO8nwWruPWx80tqU0QetzfZNuoBuLfeMpGDcoLIsxlNR/aQXXBFOH4LHhQNK0W+VH3kTYlSpvMsOvsY3QEz7eOU/Fd14CceokJonYKv7WArGRIzS9EnjrAC1tkEXTMToy4EgT8xu4XKaEcotxjELZAWaAchgiQn/y635uGutuXN+a55slZA7wqWCu0Yv0fUkvhc095YP5GhKO1e1No3uEiqNRVbpGnGlMd898RK3sbJn91NwFe1xRLfVNhqIDuv0ObNyPGpJ94doKXLjiESFiz1BwWCUc7VOhDcgxJ/tZjiQWW144sMRVWDMjvrnSx3bmDnblNyuhT+4L2yasadutglfva4 GoXB5+q1 xPuOhfp/c7juj9AuapxWBRQB0mqfp6yE2phiWnN7mZCv9bCZKfoqwBs8rB5Ud0W66s5ZdZamBYC8xWZOAadnI4z/udIadmnIEFHlMJSw7PHL6jvQhBJuJ2hs8aH8RoFCuwbJSaJ15y3lQ0OZMebR/Vm/S30DF1yDnwzynr4Vr7RYK6F1xFwebO6us4Wr0OTtr1xxRFFBDwXNSVj5VkPJAwOAQw9f1+jxIDQr7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Updated BUILD_BUG_ON() to test against limit. Refactored "struct optee_shm_arg_entry" to use a flexible array member for "map", since its length depends on PAGE_SIZE. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ drivers/tee/optee/call.c | 7 +++++-- drivers/tee/optee/smc_abi.c | 2 +- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/drivers/tee/optee/call.c b/drivers/tee/optee/call.c index 16eb953e14bb6..41bd7ace6606e 100644 --- a/drivers/tee/optee/call.c +++ b/drivers/tee/optee/call.c @@ -36,7 +36,7 @@ struct optee_shm_arg_entry { struct list_head list_node; struct tee_shm *shm; - DECLARE_BITMAP(map, MAX_ARG_COUNT_PER_ENTRY); + unsigned long map[]; }; void optee_cq_init(struct optee_call_queue *cq, int thread_count) @@ -271,6 +271,7 @@ struct optee_msg_arg *optee_get_msg_arg(struct tee_context *ctx, struct optee_shm_arg_entry *entry; struct optee_msg_arg *ma; size_t args_per_entry; + size_t entry_sz; u_long bit; u_int offs; void *res; @@ -293,7 +294,9 @@ struct optee_msg_arg *optee_get_msg_arg(struct tee_context *ctx, /* * No entry was found, let's allocate a new. */ - entry = kzalloc(sizeof(*entry), GFP_KERNEL); + entry_sz = struct_size(entry, map, + BITS_TO_LONGS(MAX_ARG_COUNT_PER_ENTRY)); + entry = kzalloc(entry_sz, GFP_KERNEL); if (!entry) { res = ERR_PTR(-ENOMEM); goto out; diff --git a/drivers/tee/optee/smc_abi.c b/drivers/tee/optee/smc_abi.c index 844285d4f03c1..005689380d848 100644 --- a/drivers/tee/optee/smc_abi.c +++ b/drivers/tee/optee/smc_abi.c @@ -418,7 +418,7 @@ static void optee_fill_pages_list(u64 *dst, struct page **pages, int num_pages, * code heavily relies on this assumption, so it is better be * safe than sorry. */ - BUILD_BUG_ON(PAGE_SIZE < OPTEE_MSG_NONCONTIG_PAGE_SIZE); + BUILD_BUG_ON(PAGE_SIZE_MIN < OPTEE_MSG_NONCONTIG_PAGE_SIZE); pages_data = (void *)dst; /* From patchwork Mon Oct 14 10:58:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834701 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C647D1A443 for ; Mon, 14 Oct 2024 11:01:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1D8016B0096; Mon, 14 Oct 2024 07:01:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 160926B00C0; Mon, 14 Oct 2024 07:01:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 000A36B00C2; Mon, 14 Oct 2024 07:01:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D318F6B00C0 for ; Mon, 14 Oct 2024 07:01:13 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3F083160D2F for ; Mon, 14 Oct 2024 11:01:05 +0000 (UTC) X-FDA: 82671915774.07.E16A06B Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf13.hostedemail.com (Postfix) with ESMTP id 76BE920013 for ; Mon, 14 Oct 2024 11:01:05 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf13.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903567; a=rsa-sha256; cv=none; b=tkWGzMrCvnonNiDz7x6ifBVdhOUWyOwMh4+ha5VKEkK17upa3hxLAVejnOTgfTPQCFR7+A iHchl5+HTb8dFOZZRdWIEXhRuy7BUdSKZixlM9tK+/MjWzvv9kkhQLIhOnmRzp8TkD9+17 rUrwu4wB0IL/I1AF4fA9aFvcmTLXrrg= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf13.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903567; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=X8VEpkJ0okgdpGRrvWSbQdQmtub8XCGEswPb3bOfN9k=; b=nUnThHrsPpyU2HJ3BcaotXtaUgdND9i46uTaM75W2yWQE6DJgKSXgy0OptHGnEpPIeSI6/ Ghakq43QI8wUo1L7nc9V7x7rzjdSpCAw+DqTncdVvAPb+TVGznrocnJhcosr7RhjDjoP0F tT+chi397iulqdvxKYoJNyTAFSjrWcg= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B62B016F8; Mon, 14 Oct 2024 04:01:40 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 78EA33F51B; Mon, 14 Oct 2024 04:01:08 -0700 (PDT) From: Ryan Roberts To: "Jason A. Donenfeld" , "Theodore Ts'o" , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 33/57] random: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:40 +0100 Message-ID: <20241014105912.3207374-33-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 76BE920013 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: y7z8k4r8z9g66c6wxyoeswjxbtn56eej X-HE-Tag: 1728903665-469755 X-HE-Meta: U2FsdGVkX18VSyS6MEQdUR6z9t0eG6MVnVZUjulmLpBE7pf7BlOajLgRa9H/midKrLjWxgmpnzCKTRnGG6C0xJQ8kw/UhOj012TiPTo6vhoShAZxeSEWbE49FFfOx2UZ+MuGfZI5BVLrLNxJh1p9wAKbTVGvtMkTwxVYAP1QMNkOlZnSG2I6pOy6q/RxpOxCUOa2SWK9t7mKGitjZAHoFd1+P317sWOWekCkno/sxweya4ReVTLpW2hjC5s0yBXrajXJtzZjLVSEO37FeyJk6ZOjiXmiu9p+iT6bzF2NI7gKzDuu7dW57j8OIkyI7GZTGhcsNYZHnfsNvN749h7fxlYjcszxS6p2tfzFpsPE5nRMdmJuoZWdglGyg5v2clVrnwuMVSmbP5Ql+xqkqoDX4k+377o9PTda57a80jZzP8zi+/sdmJy1BV60/PtWuZoEeh/7HlBgP/Thv8waXLRrpDu5yS2ZTi7qnikU/fi02xFiHG8jg2SEooaAn/EonoyRb+Iy8BcfmvDDQYp/s4CQyZ9aiZ1mjiuHlOvcCWev1pe4STDeVwuRfsP5f8jcbZgsgOLdOImCqfPL3RGdDKhcjrS8tsgsWf05RQeZltR6oDTL1JQdA4U263NQJAizE/LqDjCNSe6YLUfasZtrijKbKIsDtnw9BO/vS3MpfUkJcaJFa6wNiy9Uq18UVJxHb8+g1oemiEoJve7m2FxHy7+sP+zFmDgGTG25PNVW58mTnN336DAkL2cTzi0B8X7RpCYMfhfh/jPQUQCgryfTZjJNzexdiIbb0ZmfHqu/4fG/PRKh7h5PPibVbnJJm/RPa3RbyU8ZU5yuYhehDD5bRxUhnT+cPsWvOh9OoMK7m21ZkoqcAn8HnEG42XVoQxpun+57We/dw8DmMWTxpxr83W3FjabSyEECORPeTLh9OFUAKRxMiIMOK+Rt1rqaIFW3+ix8ulMhgbkTdp881P1M14c Fo6N8wQl htwXyOZqwBy1JptRVha8KSbY2P69AwAvbvFRRnBmmBG4ftRHqnJexzZtTO6lslAizOnsHF47vcLeqjMHFbGplxiL8ME2ssKlMNolhcqAOCEUBAghO7whRyr/t0ybIE7BlYoe6Lohl094tcu7dPNRNkirv3uJ87c/fl2YuVe9r6Oyr7SdkuWroeoZAyeKxWvMYrWMkZtNAWiV2idDyEgRTIm59q6DfPEiQGpxQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Update BUILD_BUG_ON()s to test against page size limits. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ drivers/char/random.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/char/random.c b/drivers/char/random.c index 87fe61295ea1f..49d6c4ef16df4 100644 --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -466,7 +466,7 @@ static ssize_t get_random_bytes_user(struct iov_iter *iter) if (!iov_iter_count(iter) || copied != sizeof(block)) break; - BUILD_BUG_ON(PAGE_SIZE % sizeof(block) != 0); + BUILD_BUG_ON(PAGE_SIZE_MIN % sizeof(block) != 0); if (ret % PAGE_SIZE == 0) { if (signal_pending(current)) break; @@ -1428,7 +1428,7 @@ static ssize_t write_pool_user(struct iov_iter *iter) if (!iov_iter_count(iter) || copied != sizeof(block)) break; - BUILD_BUG_ON(PAGE_SIZE % sizeof(block) != 0); + BUILD_BUG_ON(PAGE_SIZE_MIN % sizeof(block) != 0); if (ret % PAGE_SIZE == 0) { if (signal_pending(current)) break; From patchwork Mon Oct 14 10:58:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834702 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A1DCD1A43B for ; Mon, 14 Oct 2024 11:01:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0C6116B00C1; Mon, 14 Oct 2024 07:01:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 04ED26B00C2; Mon, 14 Oct 2024 07:01:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DE2B66B00C3; Mon, 14 Oct 2024 07:01:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B89C86B00C1 for ; Mon, 14 Oct 2024 07:01:16 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5754EABCC6 for ; Mon, 14 Oct 2024 11:01:01 +0000 (UTC) X-FDA: 82671916068.29.E55154D Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf24.hostedemail.com (Postfix) with ESMTP id 3F5C9180027 for ; Mon, 14 Oct 2024 11:01:13 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf24.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903644; a=rsa-sha256; cv=none; b=2ozEjjGp8CKspNpIWTroquGflvFkku+HrW1UNIpYPU4n40mMNZQuPUSGwU99lt7anSRSCD AOcmBRGva9019Sj8n02raMK9534dR7nt3JPfrY7+qUqwaLNFdGmZX0NEhOfAxra7cic8NR xvVAuAnCBs28wotnAgKJdl2YiYpVJT4= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf24.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903644; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1NGA+tE4zfya25VBkRDEA5yZy6epmiLwve34sECMDkQ=; b=VCe71I5kJgo8F6+88+z7t1BJI5U1BKtwpKX0+KhmBMT2I9oNn4brSpVAk3gXJ3IV+q9anA nI7QISerbvOuHAqrlYjeD8rX3kxY/DIfAT7G+wKAda87FGezYOChg7iJl+7ailnAlusnxu aVmTSmYZ2hi/aCdWiejNmI5KQPjGNUc= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C49D916F8; Mon, 14 Oct 2024 04:01:43 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 679B23F51B; Mon, 14 Oct 2024 04:01:11 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , Damien Le Moal , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Niklas Cassel , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-ide@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 34/57] sata_sil24: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:41 +0100 Message-ID: <20241014105912.3207374-34-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: wrkgubbm85umrgjswywi8pecq7k9zs1b X-Rspamd-Queue-Id: 3F5C9180027 X-Rspamd-Server: rspam02 X-HE-Tag: 1728903673-645190 X-HE-Meta: U2FsdGVkX197JUlDCJ7Z9V2nTxFQE7SUwmv3IH6oUwO8knMKW/CF0rpZwVHFRCvSaoB1KEj6u0lAlMLPISa+grmsFADvKb87j6zh5SMQqX4KbjW4Deb3r4NQGJPZBGad899l15J/FsOgsaG9yIbWKqLA4w4JfNx6xm3qJl3YYEQGqnQDBT0DzNlI4x3vYOQlkJa4pecKQaC+zM5YoxF54WSdRi+N7oxUw3yC18W7Pdi5S9uLApH+Dzx4c/ypwoWdLGTKAKntPg1l976AbQiVU/gYgi3myn171PN4aHELSaNpAhvE7StuT8dn9E2zr0i4mHlUf8QuviaZJYYKpOBiyORW0ghaFB/035rbAEU3N9PJ6zUY093LbyQuewHFJdYax1Dv8VkaHo/zJcWMQHqqesA1jU53z6+1qx9OAkVggvxXEbi0BnP/K19OTPE9F8bHcs8pHqE5pGFgCHe7NtgbultxtgvIGMpkHaTvqSEzLZYt757opE6np4MbeNzwQJEgQuBjwq3J7FjlmjBOE5S45PGglWZfJFfcXujFLrFW8yi0AqBeOI1k04PDgKTYh5/x5g4tI/BPrzbQ1hGTxqmit4qbpIojKrQoT2XQOVVy3UQA5Pcj4S//DWewFEoRwJMUPaxApVvG8i5d1daM4K9KEEsGBSFSyKemjX6wd9N06434lUBxaoHRpchdQrxATF/Z+igorMt4P3G/kKeDl/aFT+tWQAc4MO+WZEfNUrlI3lm4t1oxBn29fVRukoi+WOjLGCuMEyp7S0Asz4cVtCTAfpfgcd9dx6jdqCUWDYegAYB4QyU166eprC7JzkLiaA6HK8wVzRZJYgU8b+bwS0ShoYmLxWLodeBNZqFgFmEMOSSrlBFcsBnpuGt/me8+1Hf/jgT+qViIxhKKX6D1xk2b/gQ1oKvSMSp5O0h8qtCCDXNAVpzMTyF3PMuH9a4lCNloZl89py8PcTusDjSlZg0 sWHdjY9t qPxQDLZM5kP44h23GqD5tjJLWQyFUW1xg5pQv24M997C4IyloWWs2tJxBk7U8UeHyUHYORsoNtR07j9GzMKGApiV0YT/ITFsUR4y0dVhoW0T/yfkecSfGtdMQQqeOS7PZ6frTgFosX/DwCC4Y1n5MkGoyY9ov/Eu2RMVNRReaMvyhvga2JXz48FJq0exJtZ+VFjZXrgfa0Xwx7wXShJkCjKrzBSYW+KJMHW03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Convert "struct sil24_ata_block" and "struct sil24_atapi_block" to use a flexible array member for their sge[] array. The previous static size of SIL24_MAX_SGE depends on PAGE_SIZE so doesn't work for boot-time page size. Wrap global variables that are initialized with PAGE_SIZE derived values using DEFINE_GLOBAL_PAGE_SIZE_VAR() so their initialization can be deferred for boot-time page size builds. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ drivers/ata/sata_sil24.c | 46 +++++++++++++++++++--------------------- 1 file changed, 22 insertions(+), 24 deletions(-) diff --git a/drivers/ata/sata_sil24.c b/drivers/ata/sata_sil24.c index 72c03cbdaff43..85c6382976626 100644 --- a/drivers/ata/sata_sil24.c +++ b/drivers/ata/sata_sil24.c @@ -42,26 +42,25 @@ struct sil24_sge { __le32 flags; }; +/* + * sil24 fetches in chunks of 64bytes. The first block + * contains the PRB and two SGEs. From the second block, it's + * consisted of four SGEs and called SGT. Calculate the + * number of SGTs that fit into one page. + */ +#define SIL24_PRB_SZ (sizeof(struct sil24_prb) + 2 * sizeof(struct sil24_sge)) +#define SIL24_MAX_SGT ((PAGE_SIZE - SIL24_PRB_SZ) / (4 * sizeof(struct sil24_sge))) + +/* + * This will give us one unused SGEs for ATA. This extra SGE + * will be used to store CDB for ATAPI devices. + */ +#define SIL24_MAX_SGE (4 * SIL24_MAX_SGT + 1) enum { SIL24_HOST_BAR = 0, SIL24_PORT_BAR = 2, - /* sil24 fetches in chunks of 64bytes. The first block - * contains the PRB and two SGEs. From the second block, it's - * consisted of four SGEs and called SGT. Calculate the - * number of SGTs that fit into one page. - */ - SIL24_PRB_SZ = sizeof(struct sil24_prb) - + 2 * sizeof(struct sil24_sge), - SIL24_MAX_SGT = (PAGE_SIZE - SIL24_PRB_SZ) - / (4 * sizeof(struct sil24_sge)), - - /* This will give us one unused SGEs for ATA. This extra SGE - * will be used to store CDB for ATAPI devices. - */ - SIL24_MAX_SGE = 4 * SIL24_MAX_SGT + 1, - /* * Global controller registers (128 bytes @ BAR0) */ @@ -244,13 +243,13 @@ enum { struct sil24_ata_block { struct sil24_prb prb; - struct sil24_sge sge[SIL24_MAX_SGE]; + struct sil24_sge sge[]; }; struct sil24_atapi_block { struct sil24_prb prb; u8 cdb[16]; - struct sil24_sge sge[SIL24_MAX_SGE]; + struct sil24_sge sge[]; }; union sil24_cmd_block { @@ -373,7 +372,7 @@ static struct pci_driver sil24_pci_driver = { #endif }; -static const struct scsi_host_template sil24_sht = { +static DEFINE_GLOBAL_PAGE_SIZE_VAR_CONST(struct scsi_host_template, sil24_sht, { __ATA_BASE_SHT(DRV_NAME), .can_queue = SIL24_MAX_CMDS, .sg_tablesize = SIL24_MAX_SGE, @@ -382,7 +381,7 @@ static const struct scsi_host_template sil24_sht = { .sdev_groups = ata_ncq_sdev_groups, .change_queue_depth = ata_scsi_change_queue_depth, .device_configure = ata_scsi_device_configure -}; +}); static struct ata_port_operations sil24_ops = { .inherits = &sata_pmp_port_ops, @@ -1193,7 +1192,7 @@ static int sil24_port_start(struct ata_port *ap) struct device *dev = ap->host->dev; struct sil24_port_priv *pp; union sil24_cmd_block *cb; - size_t cb_size = sizeof(*cb) * SIL24_MAX_CMDS; + size_t cb_size = PAGE_SIZE * SIL24_MAX_CMDS; dma_addr_t cb_dma; pp = devm_kzalloc(dev, sizeof(*pp), GFP_KERNEL); @@ -1258,7 +1257,6 @@ static void sil24_init_controller(struct ata_host *host) static int sil24_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) { - extern int __MARKER__sil24_cmd_block_is_sized_wrongly; struct ata_port_info pi = sil24_port_info[ent->driver_data]; const struct ata_port_info *ppi[] = { &pi, NULL }; void __iomem * const *iomap; @@ -1266,9 +1264,9 @@ static int sil24_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) int rc; u32 tmp; - /* cause link error if sil24_cmd_block is sized wrongly */ - if (sizeof(union sil24_cmd_block) != PAGE_SIZE) - __MARKER__sil24_cmd_block_is_sized_wrongly = 1; + /* union sil24_cmd_block must be PAGE_SIZE */ + BUG_ON(struct_size_t(struct sil24_atapi_block, sge, SIL24_MAX_SGE) != PAGE_SIZE); + BUG_ON(struct_size_t(struct sil24_ata_block, sge, SIL24_MAX_SGE) > PAGE_SIZE); ata_print_version_once(&pdev->dev, DRV_VERSION); From patchwork Mon Oct 14 10:58:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E531D1A43B for ; Mon, 14 Oct 2024 11:01:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0061A6B0099; Mon, 14 Oct 2024 07:01:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EF43B6B00C4; Mon, 14 Oct 2024 07:01:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D94356B00C5; Mon, 14 Oct 2024 07:01:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B7C9D6B00C3 for ; Mon, 14 Oct 2024 07:01:20 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id DCA9680D98 for ; Mon, 14 Oct 2024 11:01:13 +0000 (UTC) X-FDA: 82671916236.28.B21FF6A Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf03.hostedemail.com (Postfix) with ESMTP id 957BB20007 for ; Mon, 14 Oct 2024 11:01:15 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf03.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903563; a=rsa-sha256; cv=none; b=IYdr0F55T02dUsI+PBAU6EGFJyBh7b3SBHSr25Me664FOdZXkgsKUhuv0vVhQF+ck8B5B4 HSZ/59T3pqoKU589Y5YRRurVlkb+HDTCszXsat7F94aC1Khe+GyHhC/pgxKKgfSwYMS/Iw jrKvdqmSWVMLKQfa5jlitbqm9+pejo8= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf03.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903563; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5o+gDL2mN3xapmdN2m55fCy14ysHc8+6na6uUa6irXM=; b=LeAMsFxqbCQ3RopKwCFmV/Opq6mmXSjbCnh7xYhv/Dk990vR41XNsN6CF3RJLISX2L5QUN 2XiDVngKs1uffhhUihSvFqle1lQ0PsGbvSy5G+/Nwy9EAoy9y+tZO6X59LLY+6IUzJzxtl 9xxLQYzM6VwQ5IMeDtMazjv5lSPEVJ0= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 997AD16F8; Mon, 14 Oct 2024 04:01:47 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 769A53F51B; Mon, 14 Oct 2024 04:01:14 -0700 (PDT) From: Ryan Roberts To: "Michael S. Tsirkin" , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Dominique Martinet , Eric Van Hensbergen , Greg Marsden , Ivan Ivanov , Jason Wang , Jens Axboe , Kalesh Singh , Latchesar Ionkov , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, v9fs@lists.linux.dev, virtualization@lists.linux.dev Subject: [RFC PATCH v1 35/57] virtio: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:42 +0100 Message-ID: <20241014105912.3207374-35-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 957BB20007 X-Stat-Signature: pz1jwfz5ssye973gomzd7daisani34kh X-Rspam-User: X-HE-Tag: 1728903675-263350 X-HE-Meta: U2FsdGVkX1/2BK6jHZAAhJ+ugSaBaGSDBt323BbOgkHfKNXZd9TQOv/jALQjQz7H6o4+MlGeg3SGfdkdM47VVgbCiiY5f/jsJ2gUtZorffru8wAUp33rI1KjRZ8rKRcty0DCWhpmibtgvBQM61xXx7wpClAUmLiWjugRnDztVqJXwjnpmRUYzy7kuWddhi5t1HTFp1ZBnGQmg7jS0dFffGmWcNOh1u/WeEGyCOlBvOLtOpDFWpq/F6D0f2PIhhoKsG7TlYTDE9o3rtECvCSQDGHN6/8e5hrDOiKZdMk5WzhyGDQWfdr9cTwCUlET2Zn+SnPnDEK1PX/vbISnxkdm0tHrHJyPxmriZIDsojs4va27ionY7vokqNCcUsd9OvVH93qxQl7sGGY8TL3cQ+IfaMJyhoq8posFwgMiGxXlCE7/PNjxMhqf7Cz3uWztwN5h5yWrMMywAYrRLbDX1RlKuNwbR5YHtltcEqAeprmqGQxcygnJQ2rfPZfuxGShfot47/S2QU8pr4fgN3/DNF5WTkt2oGeySTThnVYfb88mgDsCT0dKtUoOTssumccEDHcjl9V4d5DR04lAZyCerkdngMjePCWlkVgGezxJePbFBZd4Ei5sbKr2WfAAyKsewTWm6Ljr+YFWNADcfz++vsJZJH4rTBjMRsYth26C+NJc7ak4S2RanqvR3Z8VdIsd6swGP4rsIIet6eO6DOojEH4aq6ll3uiaR0siF0wa4Rv4hQfgnDO94e5ykFQ235XF9vCh1gI9x3uJEBFwPWXNIULDqKqDL1JzDqMMuGHj2AszbL+mcnkjPNjSAA4Df6wovN9dJaFafVFMG0qIoVmNg15235QGib5CJl5YnznCOPfOwiFLS/0FvtLu9Jj9GhJccvLfDV1oenZUCPuVgdcunLxgCo87fvHSRSbDE79Cn6KxXIHMRO4GEQ657UIWvWTeE1G1DF8jgY6lww5MPjp020a yEyXFUQU IEF9DRBGBJFtsjCq+d2zP3mZr9CLuEp2oa7vldXT5QIEjyLCPVytXGjjBWbrs6eLRbtJlRq7Ryu8cD2Ne3RluSZm6IpW7CfnPiQHX8FiQewQ6rr1c0m5Q2q2BAb3LZnDdRiYUVoYaaF6nLrys/yrUBT04dZJfJ7QCPgzj6u6ZxmgBUbj49iLXPqjate4O7XqIMVL0EzcR/WkRNqqiZKYHKpsNeMY2CVTsWfRDBKSUU8pP8yKR7X43ToBN0QmpUMi2rldUET+rH5tJDcSNSYvvrod2Lw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Updated multiple BUILD_BUG_ON() instances to test against page size limits. Wrap global variables that are initialized with PAGE_SIZE derived values using DEFINE_GLOBAL_PAGE_SIZE_VAR() so their initialization can be deferred for boot-time page size builds. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ drivers/block/virtio_blk.c | 2 +- drivers/virtio/virtio_balloon.c | 10 ++++++---- net/9p/trans_virtio.c | 4 ++-- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c index 194417abc1053..8a8960b609bc9 100644 --- a/drivers/block/virtio_blk.c +++ b/drivers/block/virtio_blk.c @@ -899,7 +899,7 @@ static ssize_t serial_show(struct device *dev, int err; /* sysfs gives us a PAGE_SIZE buffer */ - BUILD_BUG_ON(PAGE_SIZE < VIRTIO_BLK_ID_BYTES); + BUILD_BUG_ON(PAGE_SIZE_MIN < VIRTIO_BLK_ID_BYTES); buf[VIRTIO_BLK_ID_BYTES] = '\0'; err = virtblk_get_id(disk, buf); diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c index 54469277ca303..3818d894bd212 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -25,6 +25,7 @@ * page units. */ #define VIRTIO_BALLOON_PAGES_PER_PAGE (unsigned int)(PAGE_SIZE >> VIRTIO_BALLOON_PFN_SHIFT) +#define VIRTIO_BALLOON_PAGES_PER_PAGE_MAX (unsigned int)(PAGE_SIZE_MAX >> VIRTIO_BALLOON_PFN_SHIFT) #define VIRTIO_BALLOON_ARRAY_PFNS_MAX 256 /* Maximum number of (4k) pages to deflate on OOM notifications. */ #define VIRTIO_BALLOON_OOM_NR_PAGES 256 @@ -138,7 +139,7 @@ static u32 page_to_balloon_pfn(struct page *page) { unsigned long pfn = page_to_pfn(page); - BUILD_BUG_ON(PAGE_SHIFT < VIRTIO_BALLOON_PFN_SHIFT); + BUILD_BUG_ON(PAGE_SHIFT_MIN < VIRTIO_BALLOON_PFN_SHIFT); /* Convert pfn from Linux page size to balloon page size. */ return pfn * VIRTIO_BALLOON_PAGES_PER_PAGE; } @@ -228,7 +229,7 @@ static void set_page_pfns(struct virtio_balloon *vb, { unsigned int i; - BUILD_BUG_ON(VIRTIO_BALLOON_PAGES_PER_PAGE > VIRTIO_BALLOON_ARRAY_PFNS_MAX); + BUILD_BUG_ON(VIRTIO_BALLOON_PAGES_PER_PAGE_MAX > VIRTIO_BALLOON_ARRAY_PFNS_MAX); /* * Set balloon pfns pointing at this page. @@ -1042,8 +1043,9 @@ static int virtballoon_probe(struct virtio_device *vdev) * host's base page size. However, it needs more work to report * that value. The hard-coded order would be fine currently. */ -#if defined(CONFIG_ARM64) && defined(CONFIG_ARM64_64K_PAGES) - vb->pr_dev_info.order = 5; +#if defined(CONFIG_ARM64) + if (PAGE_SIZE == SZ_64K) + vb->pr_dev_info.order = 5; #endif err = page_reporting_register(&vb->pr_dev_info); diff --git a/net/9p/trans_virtio.c b/net/9p/trans_virtio.c index 0b8086f58ad55..25b8253011cec 100644 --- a/net/9p/trans_virtio.c +++ b/net/9p/trans_virtio.c @@ -786,7 +786,7 @@ static struct virtio_driver p9_virtio_drv = { .remove = p9_virtio_remove, }; -static struct p9_trans_module p9_virtio_trans = { +static DEFINE_GLOBAL_PAGE_SIZE_VAR(struct p9_trans_module, p9_virtio_trans, { .name = "virtio", .create = p9_virtio_create, .close = p9_virtio_close, @@ -804,7 +804,7 @@ static struct p9_trans_module p9_virtio_trans = { .pooled_rbuffers = false, .def = 1, .owner = THIS_MODULE, -}; +}); /* The standard init function */ static int __init p9_virtio_init(void) From patchwork Mon Oct 14 10:58:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834704 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21061D1A443 for ; Mon, 14 Oct 2024 11:01:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A8ADB6B00C3; Mon, 14 Oct 2024 07:01:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A394E6B00C4; Mon, 14 Oct 2024 07:01:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DDD86B00C5; Mon, 14 Oct 2024 07:01:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 65AF86B00C3 for ; Mon, 14 Oct 2024 07:01:23 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 16393A0D8F for ; Mon, 14 Oct 2024 11:01:09 +0000 (UTC) X-FDA: 82671916362.29.D3F8175 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf29.hostedemail.com (Postfix) with ESMTP id 746BC12001D for ; Mon, 14 Oct 2024 11:01:13 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903493; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SCs5zLvTJKb41Hb5mZyIydGLU4/ZfA/FgaJwPe9tmC8=; b=GCPqk2P3KhNwsKlQRSGeJdB1L1v0VElgm3WAKBlIDZdMRUBAn+GWqfTwIz4bmuHV56t84g 9N3b3LVC6oRWO3KzUIokEJc9Tzjh2U1w7rsf+8EQR75VzGCy6VRvab5uzb5j/e+vpJdX7X UCuv3PS/X7zFYFAq95ugHbDuZXZCkRU= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903493; a=rsa-sha256; cv=none; b=n+QVIqTEnUhNxBlrsZzH+TUKFc7vkvf+CmvM1R9t1tpEZMG2UgH1M+OoEaDTW/TkM1AVHe YYWs6jcrB+PLwz/8/jn5PCrOAOEtzF1CcIwOkBSTyu1QM07szIks0b5BzvBnyflN20eb+5 9q6zU93mTsenpMbcRMbP/Wt/MOTT4u0= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6816B1713; Mon, 14 Oct 2024 04:01:50 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4980A3F51B; Mon, 14 Oct 2024 04:01:18 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, xen-devel@lists.xenproject.org Subject: [RFC PATCH v1 36/57] xen: Remove PAGE_SIZE compile-time constant assumption Date: Mon, 14 Oct 2024 11:58:43 +0100 Message-ID: <20241014105912.3207374-36-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 746BC12001D X-Stat-Signature: rjahyyxh6u7px1qb6hyhzxp68d8n4c9r X-Rspam-User: X-HE-Tag: 1728903673-110878 X-HE-Meta: U2FsdGVkX19reV3713rThi2jb8AijDhx5BwIQg+AnsAqrLo52UZHjSsT1H0tGJ5kE8iS103tVSNROXBpzYus678PUYV/Tig78y2PAGb6gvVmI5wuwLpTb2hf6qOoX1Cm2hX7SL8UwEicWzK9ASfL612R2fDtXBq8m4JKZmrit+0okmXyNTKLwkzAn9UEdBmqzrEPKItMWa28L89s7OOjtszKqWHZd8o3Qc2ngLA7u9yAKxRDcFy57l3H9tG5kzAktXr+KyHXkJi3xH0bYPoK1UMJAgscJ4/7q7U16RPQBpZVDJ0XM39pOHTroNuOW02EaXX2nlhKI1TPjxvB8JP++Doi7wb/Mmcb5l7J8RT4efpr2/Bpj5oz59DrKKlAUn7xyowM+lBdpm2WBFDGes5iT7JO0Z6XN2qpsNdTLiGgPoOXFe1xcXQrd8LlaZZBTzoj+YvHrSNhKWM/5ZAdSxOX6n6vMNpar3EUZ9tXMZxwHpoG+8eP2gn4ERsxJLFMEgh6VlVGowkv8fJhFyhxOvxw5MO3ZFI2/lxKSnIFfkwt736IKPQGuW/2lGO5NUDZBm801o0CXWKWN4UcjzIQGA0nJdqWtQrhUfkLHQRLq9iij0KfDoZKEDvnsk+3HrhdRWTpP9vnDPjZC449kVUNRxCnYLkpkquP9Mse0ECSzAGrf4cHk7F0lmeh+MYdmSVGM0Vux/EFTqB4rqGBgSYYenKYfsJfu/4wYp5xXLEKPg5M0XRaMt1PJlR1H7TsGUtE7v3ZG5in/c5Du7VhVDzGgt+9g9RXihhJTIyDeRUFQyW7k1LAqGbltHdDhoT6/B31C7DVSLdVOgl/diPBxC5B5pLBV64k/pAwuhq7cGzYs3q31BwG33UDIc1ZogBXAJ0qVICLkfAwGoANPQ+WmvESZ9knrg3Yt0zvFKObeG5QYYdJ7QDcP9i15UKKNKg6s3jzuwSxyiyur9FbPuD84Qkj/Bv flf2UknT tBK7M67X7loABWmvQsteTgBSX9viiLcGHAxVIOC6UIv31DnSip9RqncwCG5JwmiInhvewPYt+H5DGLAROtlW4ec4j3dxO1cCEGzx+St/81JIDo/7OmlI/Jlc03dp8ZUUcTa50iB6CaQKbpDXH8c0/bkoksdtFBmf6T1IcG+YFSshXyPY0R0/LZud+3uhu6dQCGIla1TYR6Ve4wcns65M1WL4RHg80Clmojzq2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To prepare for supporting boot-time page size selection, refactor code to remove assumptions about PAGE_SIZE being compile-time constant. Code intended to be equivalent when compile-time page size is active. Allocate enough "frame_list" static storage in the balloon driver for the maximum supported page size. Although continue to use only the first PAGE_SIZE of the buffer at run-time to maintain existing behaviour. Refactor xen_biovec_phys_mergeable() to convert ifdeffery to c if/else. For compile-time page size, the compiler will choose one branch and strip the dead one. For boot-time, it can be evaluated at run time. Refactor a BUILD_BUG_ON to evaluate the limit (when the minimum supported page size is selected at boot-time). Reserve enough storage for max page size in "struct remap_data" and "struct xenbus_map_node". Signed-off-by: Ryan Roberts Reviewed-by: Stefano Stabellini --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ drivers/xen/balloon.c | 11 ++++++----- drivers/xen/biomerge.c | 12 ++++++------ drivers/xen/privcmd.c | 2 +- drivers/xen/xenbus/xenbus_client.c | 5 +++-- drivers/xen/xlate_mmu.c | 6 +++--- include/xen/page.h | 2 ++ 6 files changed, 21 insertions(+), 17 deletions(-) diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c index 528395133b4f8..0ed5f6453af0e 100644 --- a/drivers/xen/balloon.c +++ b/drivers/xen/balloon.c @@ -131,7 +131,8 @@ struct balloon_stats balloon_stats; EXPORT_SYMBOL_GPL(balloon_stats); /* We increase/decrease in batches which fit in a page */ -static xen_pfn_t frame_list[PAGE_SIZE / sizeof(xen_pfn_t)]; +static xen_pfn_t frame_list[PAGE_SIZE_MAX / sizeof(xen_pfn_t)]; +#define FRAME_LIST_NR_ENTRIES (PAGE_SIZE / sizeof(xen_pfn_t)) /* List of ballooned pages, threaded through the mem_map array. */ @@ -389,8 +390,8 @@ static enum bp_state increase_reservation(unsigned long nr_pages) unsigned long i; struct page *page; - if (nr_pages > ARRAY_SIZE(frame_list)) - nr_pages = ARRAY_SIZE(frame_list); + if (nr_pages > FRAME_LIST_NR_ENTRIES) + nr_pages = FRAME_LIST_NR_ENTRIES; page = list_first_entry_or_null(&ballooned_pages, struct page, lru); for (i = 0; i < nr_pages; i++) { @@ -434,8 +435,8 @@ static enum bp_state decrease_reservation(unsigned long nr_pages, gfp_t gfp) int ret; LIST_HEAD(pages); - if (nr_pages > ARRAY_SIZE(frame_list)) - nr_pages = ARRAY_SIZE(frame_list); + if (nr_pages > FRAME_LIST_NR_ENTRIES) + nr_pages = FRAME_LIST_NR_ENTRIES; for (i = 0; i < nr_pages; i++) { page = alloc_page(gfp); diff --git a/drivers/xen/biomerge.c b/drivers/xen/biomerge.c index 05a286d24f148..28f0887e40026 100644 --- a/drivers/xen/biomerge.c +++ b/drivers/xen/biomerge.c @@ -8,16 +8,16 @@ bool xen_biovec_phys_mergeable(const struct bio_vec *vec1, const struct page *page) { -#if XEN_PAGE_SIZE == PAGE_SIZE - unsigned long bfn1 = pfn_to_bfn(page_to_pfn(vec1->bv_page)); - unsigned long bfn2 = pfn_to_bfn(page_to_pfn(page)); + if (XEN_PAGE_SIZE == PAGE_SIZE) { + unsigned long bfn1 = pfn_to_bfn(page_to_pfn(vec1->bv_page)); + unsigned long bfn2 = pfn_to_bfn(page_to_pfn(page)); + + return bfn1 + PFN_DOWN(vec1->bv_offset + vec1->bv_len) == bfn2; + } - return bfn1 + PFN_DOWN(vec1->bv_offset + vec1->bv_len) == bfn2; -#else /* * XXX: Add support for merging bio_vec when using different page * size in Xen and Linux. */ return false; -#endif } diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c index 9563650dfbafc..847f7b806caf7 100644 --- a/drivers/xen/privcmd.c +++ b/drivers/xen/privcmd.c @@ -557,7 +557,7 @@ static long privcmd_ioctl_mmap_batch( state.global_error = 0; state.version = version; - BUILD_BUG_ON(((PAGE_SIZE / sizeof(xen_pfn_t)) % XEN_PFN_PER_PAGE) != 0); + BUILD_BUG_ON(((PAGE_SIZE_MIN / sizeof(xen_pfn_t)) % XEN_PFN_PER_PAGE_MAX) != 0); /* mmap_batch_fn guarantees ret == 0 */ BUG_ON(traverse_pages_block(m.num, sizeof(xen_pfn_t), &pagelist, mmap_batch_fn, &state)); diff --git a/drivers/xen/xenbus/xenbus_client.c b/drivers/xen/xenbus/xenbus_client.c index 51b3124b0d56c..99bde836c10c4 100644 --- a/drivers/xen/xenbus/xenbus_client.c +++ b/drivers/xen/xenbus/xenbus_client.c @@ -49,9 +49,10 @@ #include "xenbus.h" -#define XENBUS_PAGES(_grants) (DIV_ROUND_UP(_grants, XEN_PFN_PER_PAGE)) +#define XENBUS_PAGES(_grants) (DIV_ROUND_UP(_grants, XEN_PFN_PER_PAGE)) +#define XENBUS_PAGES_MAX(_grants) (DIV_ROUND_UP(_grants, XEN_PFN_PER_PAGE_MIN)) -#define XENBUS_MAX_RING_PAGES (XENBUS_PAGES(XENBUS_MAX_RING_GRANTS)) +#define XENBUS_MAX_RING_PAGES (XENBUS_PAGES_MAX(XENBUS_MAX_RING_GRANTS)) struct xenbus_map_node { struct list_head next; diff --git a/drivers/xen/xlate_mmu.c b/drivers/xen/xlate_mmu.c index f17c4c03db30c..a757c801a7542 100644 --- a/drivers/xen/xlate_mmu.c +++ b/drivers/xen/xlate_mmu.c @@ -74,9 +74,9 @@ struct remap_data { int mapped; /* Hypercall parameters */ - int h_errs[XEN_PFN_PER_PAGE]; - xen_ulong_t h_idxs[XEN_PFN_PER_PAGE]; - xen_pfn_t h_gpfns[XEN_PFN_PER_PAGE]; + int h_errs[XEN_PFN_PER_PAGE_MAX]; + xen_ulong_t h_idxs[XEN_PFN_PER_PAGE_MAX]; + xen_pfn_t h_gpfns[XEN_PFN_PER_PAGE_MAX]; int h_iter; /* Iterator */ }; diff --git a/include/xen/page.h b/include/xen/page.h index 285677b42943a..86683a30038a3 100644 --- a/include/xen/page.h +++ b/include/xen/page.h @@ -21,6 +21,8 @@ ((page_to_pfn(page)) << (PAGE_SHIFT - XEN_PAGE_SHIFT)) #define XEN_PFN_PER_PAGE (PAGE_SIZE / XEN_PAGE_SIZE) +#define XEN_PFN_PER_PAGE_MIN (PAGE_SIZE_MIN / XEN_PAGE_SIZE) +#define XEN_PFN_PER_PAGE_MAX (PAGE_SIZE_MAX / XEN_PAGE_SIZE) #define XEN_PFN_DOWN(x) ((x) >> XEN_PAGE_SHIFT) #define XEN_PFN_UP(x) (((x) + XEN_PAGE_SIZE-1) >> XEN_PAGE_SHIFT) From patchwork Mon Oct 14 10:58:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC72ED1A45F for ; Mon, 14 Oct 2024 11:01:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3FAD66B00C4; Mon, 14 Oct 2024 07:01:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A9B66B00C5; Mon, 14 Oct 2024 07:01:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 273186B00C6; Mon, 14 Oct 2024 07:01:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 077086B00C4 for ; Mon, 14 Oct 2024 07:01:26 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id D15D7120D88 for ; Mon, 14 Oct 2024 11:01:18 +0000 (UTC) X-FDA: 82671916278.09.C4EE1D4 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf19.hostedemail.com (Postfix) with ESMTP id 8B3671A0026 for ; Mon, 14 Oct 2024 11:01:16 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903612; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sUJHwqHFRx+14j6h+df8VkAiqppEg2Zp0LAwcqaTc7A=; b=libij//WnbyRHtHM54hBlyh9x8fy5G0z8yMAA/ir+nvaET1juf3GIP6wRnWycIuaD809cS arPGImRw+GXssYsftrRG0mSL4584exsHXIA4zR47Vouf34kIL8wjH4RYWvKpXHfYt5hUH6 VKTbTbIruVaz30kOwecfySmDT68FxA4= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; spf=pass (imf19.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903612; a=rsa-sha256; cv=none; b=6kRG5S3SztztMz1O0yrPPYIO+UsbMMyFMWd9CVrNVdTH2e9GMVfctAMnCHrJgPoURKURej G5luWt4EIvXPVqZklh0TGGeDrUJgE6kbnFTRh/gEEC5/z6sCdU/n/Ht4x4N479SGqHZNcT VoxmS5hsWdgnom1GLMMH4gWcpUAf6S0= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1861F1756; Mon, 14 Oct 2024 04:01:53 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 18CCC3F51B; Mon, 14 Oct 2024 04:01:20 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 37/57] arm64: Fix macros to work in C code in addition to the linker script Date: Mon, 14 Oct 2024 11:58:44 +0100 Message-ID: <20241014105912.3207374-37-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 8B3671A0026 X-Stat-Signature: stsgfwm8ats993zayojq5syo4dp7yg8w X-HE-Tag: 1728903676-94295 X-HE-Meta: U2FsdGVkX18C7P8lEqjZOabvHg4YMxpgiwFaH6Ipz7oPPhyVD074GIuylKQzWGC6A+2H+cCYeNSBvRvo5uGZsFUY+jovLYmZaLcaqscSsnS4tadDNu4Zd41m7LMYEV9hZj+p6Y0lX1X9AWz9cuA6tNvjTOcqCrHFNJAOEsZoLmKthMieCBjbr39KRn/rxkLHfIBUVPtGPc9LpMbELK0X+Q3A077iXdHrniTJmNFSzrX7cmctb1bXUpauzLdunq7gzCYEwlhm8jPXPXSzspjBgLxIrs+gR6hygb8tzTbddcTYz/0i4QPLZjhkd1XYO/ABd0j5B+W5cqlTDkZfO+Utv/9DnxD+WCk8smiFysm/quoxvl+i0nLnaUnZtUB6Himn/GrDvwa3S4BP2noc6tlangCqXvCmjrMz073KJiUdd9F2GiIi3IKkGImTwz6JuMii0kriN8aS1oowpf4LxXREGu+JL4c8wIjEkQe9IBMXVBGEbnQ0q6DiT5WKk1VDN2QuUtz9mxRp/Z2DX/s1fp6JKHUu73D7dbA3+MaZ2CkI/3igjSg48KXaXhroXkQbQwpBRO3w2DtHRgpoNPh5mlR/a9gb/FCQLqp9LDnXixjpmZiKpsvBgUYhGyEKuLOfjFT8e4sCl/2MXEW3ADpLRvyEbIAb6leeHPweSMuzoPIK6yaHy6CWSwoLnu4y6wpFjqetXbm6v7f18PvNnR9XGeYyLDp5xznAAEqEQhD0ufektQDQvP+RewKDpGmWB93SdYjJgdBD7VvnhcZKLeeOP0jmR5Q+okJJWaP5jxZiJ53112Xk+YyRr0V9OAOfvYyys2Ve/PI5EfGPwX2neiwvV0hO0Pzweh8ha/QBBjZcU844yjQofZV53cWb5hi7ohpIewVbgKqaud+68o2zqsEMHe73/BTPBj4Gn8N7nsigtUYIlNwFds3qdu+nLXCUgBYSvQcNDcTWQ/TgTTzmnIK7nPS Dib96NOA CbRCXpJcrJqZYNzeGW0GyxGIjMDxgydxAfHpCnRENkwNVxLWcKbhIgqs3QGMyl4rePFFoeTovx3OOg+AVhkfZfqMmJL3d6Ab4px3CckX15816Fji1IPQfstr028RYEer9GdYvC3gKm68X9Of5dB9U97e6leWnfkwQuXC5L403UHD8tDi8XYY3Eb5VFi4QFV8gGnvX5hbx3oKLyoc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Previously INIT_DIR_SIZE and INIT_IDMAP_DIR_SIZE used _end for the end address of the kernel image. In the linker script context, this resolves to an integer that refers to the link va of the end of the kernel image. But in C code it resolves to a pointer to the end of the image as placed in memory. So there are 2 problems; because its a pointer, we can't do arithmetic on it. And because the image may be in a different location in memory than the one it was linked at, it is not correct to find the image size by subtracting KIMAGE_VADDR. So introduce KIMAGE_VADDR_END, which always represents the link va of the end of the kernel image as an integer. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/kernel-pgtable.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h index bf05a77873a49..1722b9217d47d 100644 --- a/arch/arm64/include/asm/kernel-pgtable.h +++ b/arch/arm64/include/asm/kernel-pgtable.h @@ -35,6 +35,8 @@ #define IDMAP_LEVELS ARM64_HW_PGTABLE_LEVELS(IDMAP_VA_BITS) #define IDMAP_ROOT_LEVEL (4 - IDMAP_LEVELS) +#define KIMAGE_VADDR_END (_AT(u64, _end) - _AT(u64, _text) + KIMAGE_VADDR) + /* * A relocatable kernel may execute from an address that differs from the one at * which it was linked. In the worst case, its runtime placement may intersect @@ -56,10 +58,10 @@ + EARLY_LEVEL(3, (lvls), (vstart), (vend), add) /* each entry needs a next level page table */ \ + EARLY_LEVEL(2, (lvls), (vstart), (vend), add) /* each entry needs a next level page table */ \ + EARLY_LEVEL(1, (lvls), (vstart), (vend), add))/* each entry needs a next level page table */ -#define INIT_DIR_SIZE (PAGE_SIZE * (EARLY_PAGES(SWAPPER_PGTABLE_LEVELS, KIMAGE_VADDR, _end, EXTRA_PAGE) \ +#define INIT_DIR_SIZE (PAGE_SIZE * (EARLY_PAGES(SWAPPER_PGTABLE_LEVELS, KIMAGE_VADDR, KIMAGE_VADDR_END, EXTRA_PAGE) \ + EARLY_SEGMENT_EXTRA_PAGES)) -#define INIT_IDMAP_DIR_PAGES (EARLY_PAGES(INIT_IDMAP_PGTABLE_LEVELS, KIMAGE_VADDR, _end, 1)) +#define INIT_IDMAP_DIR_PAGES (EARLY_PAGES(INIT_IDMAP_PGTABLE_LEVELS, KIMAGE_VADDR, KIMAGE_VADDR_END, 1)) #define INIT_IDMAP_DIR_SIZE ((INIT_IDMAP_DIR_PAGES + EARLY_IDMAP_EXTRA_PAGES) * PAGE_SIZE) #define INIT_IDMAP_FDT_PAGES (EARLY_PAGES(INIT_IDMAP_PGTABLE_LEVELS, 0UL, UL(MAX_FDT_SIZE), 1) - 1) From patchwork Mon Oct 14 10:58:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834706 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A067FD1A43B for ; Mon, 14 Oct 2024 11:01:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 35FAD6B009D; Mon, 14 Oct 2024 07:01:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 30D746B00C5; Mon, 14 Oct 2024 07:01:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1884C6B00C6; Mon, 14 Oct 2024 07:01:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id EBCD96B009D for ; Mon, 14 Oct 2024 07:01:28 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E2D66140D29 for ; Mon, 14 Oct 2024 11:01:20 +0000 (UTC) X-FDA: 82671916572.18.D8394F6 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf29.hostedemail.com (Postfix) with ESMTP id C328A120021 for ; Mon, 14 Oct 2024 11:01:18 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf29.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903571; a=rsa-sha256; cv=none; b=3Qui+Gdx6iIi2/x6qgkVvGlOtq/0vy6E1i4sroySlKDFgHHpwnyR/trjld4NJ2qMgY3CUt hOz2kntc5AC9sAQGpsIssolUMKXGE8OEmCQ1ZMdnSohExuuP4+S+SA7bbgKcNEhWvDDmQQ CT8h15ZRhlG5wluNd6UjFCaX7TSf0LM= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf29.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903571; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Vs2AuMehXjNKtnTB/Gm3C8UdYqhpeS4EkHa/BZQq/AA=; b=a6XqWPwMuFkSensb7dKdHvkGcnyHQVvQBBdmd4jjS8eBFUUD/O6NeUs+hTm9uGCqPApgQA ULhrpunygPPYx3nt/3wmdCwWRslLultYOC/Yu9P8moHsIhPaf+labRFXl3fcZXvivBhudc 5Ms4//uRhv1muvG+aGntJfsNRgbxSTU= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BE0851713; Mon, 14 Oct 2024 04:01:55 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BE5983F51B; Mon, 14 Oct 2024 04:01:23 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 38/57] arm64: Track early pgtable allocation limit Date: Mon, 14 Oct 2024 11:58:45 +0100 Message-ID: <20241014105912.3207374-38-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: C328A120021 X-Stat-Signature: gq35bo6tgdfxu7yfq7ik6y4aroy1i74f X-Rspam-User: X-HE-Tag: 1728903678-439975 X-HE-Meta: U2FsdGVkX18atm0q51992K2VDC+BOEeAhJCXBXhSgwy8VRr0hoa64sOsZDmtE6n1fqvolgcNsanJhoq9lUsQFtLwrcI4gcpe/V5yXNbyLzavS26wh10UPQFgMM9sxHdXiWcIC4YbyW0l0Jcevc6MRDeT9WEvGzyAbH2ZKuxp2n290UfvyUFXQuJi6ObfAkCJTqAmTmncKvHDfhMjtf22zZCD5ARIPtx4DgNBlvQN0W8kFeemEN7RnArHl/K5xtQVMKxPfpTQi/cROA3asFd93FV0FIVyzVHfsJQOUs7Oh1uTwMRss2J+oz/EqLofktNDt1bpe9VtWpVZ9Ed4fvHxRfgbT1vll60Z8mJ0W+DEnscJkCXzGlBoH7nZq+jn+RTXaXzkAZjHpIXu1yFOaBSXp6V0a54Ch/C8K3BPJRz22K06p4jtSd/gea893X8/9ilVh/H4KtQkmoPeLEgYtCih80A0KI0HlqqCKPZUYG85Y1BkgPn99XE6+WWzmihsMDRyu+r8kJX7FB9uiPfnDDdZFybB1NmTC47iEQ9A+NKshxsyjRziqS2/GRgteOyvIHc0pN4sIVxdIKW7qk3dHIQrF+0y+/zJmuCo1TAJLlYoizJ8qlXDlxu8FTMdtIuOAvbegs9jQsBbkQ6xMf3cInUbD1ZvRbxZ6lRW7FKd6g5kI1WB18R9Nty0mO4YhHnyWgcTOa5e4YvFJyFEu2Dp4yUaNnRy/9fy9PM6I/z9iPY+z9dWDzkskAWFzRh3uPvawyTWNENaRbol8OOcnMgH5smuW+zixDat3crHkODnR54wFKFfvq2yV2qo6GRYPyYm191MIKMgbGOK+JnYTcAGlG1pEL7xUC2XE7HFYW5kSOMRQ1KJKwDnHv9w26TQCkiJtrHWyM+iZ0kyvtLIGWMLJR2FWkx9YL5NRY7t6kz9qD0iQwJy/D/9EnogkaiJ/NK/XDxL6bDMk53JQ1qRG2uQlA7 YbeUDgCF AxTnR4tuPNC7MblLMa2M5XBtbECs6beX4UCFdc5ZQxz/3yjDZTkQpI4yKEqVPmCvdOeXqwYsCXu27A0U3LKfdvAPMj3OzEFBmmbO8GtQ0CCnkao8qX9ZL0DqAKZ2ZXf01ShNS2BCfyY2NcN++hQkwFRTqgRyMAxoIpiVqW7xGOaX7pkmhmqKFR/X/4g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Early pgtables (e.g. init_idmap_pg_dir, init_pg_dir, etc) are allocated from statically defined memory blocks within the kernel image that are sized for the calculated worst case requirements. Let's make the allocator aware of the block's limit so that it can detect any overflow. This boils down to passing the limit of the memory block to map_range() so let's add it as a parameter. If an overflow is detected, report the error to __early_cpu_boot_status and park the cpu. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/smp.h | 1 + arch/arm64/kernel/head.S | 3 +- arch/arm64/kernel/image-vars.h | 3 ++ arch/arm64/kernel/pi/map_kernel.c | 35 +++++++++++--------- arch/arm64/kernel/pi/map_range.c | 54 +++++++++++++++++++++++++------ arch/arm64/kernel/pi/pi.h | 4 +-- arch/arm64/mm/mmu.c | 14 +++++--- 7 files changed, 81 insertions(+), 33 deletions(-) diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h index 2510eec026f7e..86edc5f8c9673 100644 --- a/arch/arm64/include/asm/smp.h +++ b/arch/arm64/include/asm/smp.h @@ -22,6 +22,7 @@ #define CPU_STUCK_REASON_52_BIT_VA (UL(1) << CPU_STUCK_REASON_SHIFT) #define CPU_STUCK_REASON_NO_GRAN (UL(2) << CPU_STUCK_REASON_SHIFT) +#define CPU_STUCK_REASON_NO_PGTABLE_MEM (UL(3) << CPU_STUCK_REASON_SHIFT) #ifndef __ASSEMBLY__ diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index cb68adcabe078..7e17a71fd9e4b 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -89,7 +89,8 @@ SYM_CODE_START(primary_entry) mov sp, x1 mov x29, xzr adrp x0, init_idmap_pg_dir - mov x1, xzr + adrp x1, init_idmap_pg_end + mov x2, xzr bl __pi_create_init_idmap /* diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 8f5422ed1b758..a168f3337446f 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -52,6 +52,7 @@ PROVIDE(__pi_cavium_erratum_27456_cpus = cavium_erratum_27456_cpus); #endif PROVIDE(__pi__ctype = _ctype); PROVIDE(__pi_memstart_offset_seed = memstart_offset_seed); +PROVIDE(__pi___early_cpu_boot_status = __early_cpu_boot_status); PROVIDE(__pi_init_idmap_pg_dir = init_idmap_pg_dir); PROVIDE(__pi_init_idmap_pg_end = init_idmap_pg_end); @@ -68,6 +69,8 @@ PROVIDE(__pi___inittext_end = __inittext_end); PROVIDE(__pi___initdata_begin = __initdata_begin); PROVIDE(__pi___initdata_end = __initdata_end); PROVIDE(__pi__data = _data); +PROVIDE(__pi___mmuoff_data_start = __mmuoff_data_start); +PROVIDE(__pi___mmuoff_data_end = __mmuoff_data_end); PROVIDE(__pi___bss_start = __bss_start); PROVIDE(__pi__end = _end); diff --git a/arch/arm64/kernel/pi/map_kernel.c b/arch/arm64/kernel/pi/map_kernel.c index f374a3e5a5fe1..dcf9233ccfff2 100644 --- a/arch/arm64/kernel/pi/map_kernel.c +++ b/arch/arm64/kernel/pi/map_kernel.c @@ -20,11 +20,11 @@ extern const u8 __eh_frame_start[], __eh_frame_end[]; extern void idmap_cpu_replace_ttbr1(void *pgdir); -static void __init map_segment(pgd_t *pg_dir, u64 *pgd, u64 va_offset, - void *start, void *end, pgprot_t prot, - bool may_use_cont, int root_level) +static void __init map_segment(pgd_t *pg_dir, u64 *pgd, u64 limit, + u64 va_offset, void *start, void *end, + pgprot_t prot, bool may_use_cont, int root_level) { - map_range(pgd, ((u64)start + va_offset) & ~PAGE_OFFSET, + map_range(pgd, limit, ((u64)start + va_offset) & ~PAGE_OFFSET, ((u64)end + va_offset) & ~PAGE_OFFSET, (u64)start, prot, root_level, (pte_t *)pg_dir, may_use_cont, 0); } @@ -32,7 +32,7 @@ static void __init map_segment(pgd_t *pg_dir, u64 *pgd, u64 va_offset, static void __init unmap_segment(pgd_t *pg_dir, u64 va_offset, void *start, void *end, int root_level) { - map_segment(pg_dir, NULL, va_offset, start, end, __pgprot(0), + map_segment(pg_dir, NULL, 0, va_offset, start, end, __pgprot(0), false, root_level); } @@ -41,6 +41,7 @@ static void __init map_kernel(u64 kaslr_offset, u64 va_offset, int root_level) bool enable_scs = IS_ENABLED(CONFIG_UNWIND_PATCH_PAC_INTO_SCS); bool twopass = IS_ENABLED(CONFIG_RELOCATABLE); u64 pgdp = (u64)init_pg_dir + PAGE_SIZE; + u64 limit = (u64)init_pg_end; pgprot_t text_prot = PAGE_KERNEL_ROX; pgprot_t data_prot = PAGE_KERNEL; pgprot_t prot; @@ -78,16 +79,16 @@ static void __init map_kernel(u64 kaslr_offset, u64 va_offset, int root_level) twopass |= enable_scs; prot = twopass ? data_prot : text_prot; - map_segment(init_pg_dir, &pgdp, va_offset, _stext, _etext, prot, + map_segment(init_pg_dir, &pgdp, limit, va_offset, _stext, _etext, prot, !twopass, root_level); - map_segment(init_pg_dir, &pgdp, va_offset, __start_rodata, + map_segment(init_pg_dir, &pgdp, limit, va_offset, __start_rodata, __inittext_begin, data_prot, false, root_level); - map_segment(init_pg_dir, &pgdp, va_offset, __inittext_begin, + map_segment(init_pg_dir, &pgdp, limit, va_offset, __inittext_begin, __inittext_end, prot, false, root_level); - map_segment(init_pg_dir, &pgdp, va_offset, __initdata_begin, + map_segment(init_pg_dir, &pgdp, limit, va_offset, __initdata_begin, __initdata_end, data_prot, false, root_level); - map_segment(init_pg_dir, &pgdp, va_offset, _data, _end, data_prot, - true, root_level); + map_segment(init_pg_dir, &pgdp, limit, va_offset, _data, _end, + data_prot, true, root_level); dsb(ishst); idmap_cpu_replace_ttbr1(init_pg_dir); @@ -120,9 +121,9 @@ static void __init map_kernel(u64 kaslr_offset, u64 va_offset, int root_level) * Remap these segments with different permissions * No new page table allocations should be needed */ - map_segment(init_pg_dir, NULL, va_offset, _stext, _etext, + map_segment(init_pg_dir, NULL, 0, va_offset, _stext, _etext, text_prot, true, root_level); - map_segment(init_pg_dir, NULL, va_offset, __inittext_begin, + map_segment(init_pg_dir, NULL, 0, va_offset, __inittext_begin, __inittext_end, text_prot, false, root_level); } @@ -164,7 +165,7 @@ static void __init remap_idmap_for_lpa2(void) * LPA2 compatible fashion, and update the initial ID map while running * from that. */ - create_init_idmap(init_pg_dir, mask); + create_init_idmap(init_pg_dir, init_pg_end, mask); dsb(ishst); set_ttbr0_for_lpa2((u64)init_pg_dir); @@ -175,7 +176,7 @@ static void __init remap_idmap_for_lpa2(void) memset(init_idmap_pg_dir, 0, (u64)init_idmap_pg_end - (u64)init_idmap_pg_dir); - create_init_idmap(init_idmap_pg_dir, mask); + create_init_idmap(init_idmap_pg_dir, init_idmap_pg_end, mask); dsb(ishst); /* switch back to the updated initial ID map */ @@ -188,6 +189,7 @@ static void __init remap_idmap_for_lpa2(void) static void __init map_fdt(u64 fdt) { static u8 ptes[INIT_IDMAP_FDT_SIZE] __initdata __aligned(PAGE_SIZE); + u64 limit = (u64)&ptes[INIT_IDMAP_FDT_SIZE]; u64 efdt = fdt + MAX_FDT_SIZE; u64 ptep = (u64)ptes; @@ -195,7 +197,8 @@ static void __init map_fdt(u64 fdt) * Map up to MAX_FDT_SIZE bytes, but avoid overlap with * the kernel image. */ - map_range(&ptep, fdt, (u64)_text > fdt ? min((u64)_text, efdt) : efdt, + map_range(&ptep, limit, fdt, + (u64)_text > fdt ? min((u64)_text, efdt) : efdt, fdt, PAGE_KERNEL, IDMAP_ROOT_LEVEL, (pte_t *)init_idmap_pg_dir, false, 0); dsb(ishst); diff --git a/arch/arm64/kernel/pi/map_range.c b/arch/arm64/kernel/pi/map_range.c index 5410b2cac5907..f0024d9b1d921 100644 --- a/arch/arm64/kernel/pi/map_range.c +++ b/arch/arm64/kernel/pi/map_range.c @@ -11,11 +11,36 @@ #include "pi.h" +static void __init mmuoff_data_clean(void) +{ + bool cache_ena = !!(read_sysreg(sctlr_el1) & SCTLR_ELx_C); + + if (cache_ena) + dcache_clean_poc((unsigned long)__mmuoff_data_start, + (unsigned long)__mmuoff_data_end); + else + dcache_inval_poc((unsigned long)__mmuoff_data_start, + (unsigned long)__mmuoff_data_end); +} + +static void __init report_cpu_stuck(long val) +{ + val |= CPU_STUCK_IN_KERNEL; + WRITE_ONCE(__early_cpu_boot_status, val); + + /* Ensure the visibility of the status update */ + dsb(ishst); + mmuoff_data_clean(); + + cpu_park_loop(); +} + /** * map_range - Map a contiguous range of physical pages into virtual memory * * @pte: Address of physical pointer to array of pages to * allocate page tables from + * @limit: Physical address of end of page allocation array * @start: Virtual address of the start of the range * @end: Virtual address of the end of the range (exclusive) * @pa: Physical address of the start of the range @@ -26,8 +51,9 @@ * @va_offset: Offset between a physical page and its current mapping * in the VA space */ -void __init map_range(u64 *pte, u64 start, u64 end, u64 pa, pgprot_t prot, - int level, pte_t *tbl, bool may_use_cont, u64 va_offset) +void __init map_range(u64 *pte, u64 limit, u64 start, u64 end, u64 pa, + pgprot_t prot, int level, pte_t *tbl, bool may_use_cont, + u64 va_offset) { u64 cmask = (level == 3) ? CONT_PTE_SIZE - 1 : U64_MAX; u64 protval = pgprot_val(prot) & ~PTE_TYPE_MASK; @@ -56,11 +82,18 @@ void __init map_range(u64 *pte, u64 start, u64 end, u64 pa, pgprot_t prot, * table mapping if necessary and recurse. */ if (pte_none(*tbl)) { + u64 size = PTRS_PER_PTE * sizeof(pte_t); + + if (*pte + size > limit) { + report_cpu_stuck( + CPU_STUCK_REASON_NO_PGTABLE_MEM); + } + *tbl = __pte(__phys_to_pte_val(*pte) | PMD_TYPE_TABLE | PMD_TABLE_UXN); - *pte += PTRS_PER_PTE * sizeof(pte_t); + *pte += size; } - map_range(pte, start, next, pa, prot, level + 1, + map_range(pte, limit, start, next, pa, prot, level + 1, (pte_t *)(__pte_to_phys(*tbl) + va_offset), may_use_cont, va_offset); } else { @@ -87,7 +120,8 @@ void __init map_range(u64 *pte, u64 start, u64 end, u64 pa, pgprot_t prot, } } -asmlinkage u64 __init create_init_idmap(pgd_t *pg_dir, pteval_t clrmask) +asmlinkage u64 __init create_init_idmap(pgd_t *pg_dir, pgd_t *pg_end, + pteval_t clrmask) { u64 ptep = (u64)pg_dir + PAGE_SIZE; pgprot_t text_prot = PAGE_KERNEL_ROX; @@ -96,10 +130,12 @@ asmlinkage u64 __init create_init_idmap(pgd_t *pg_dir, pteval_t clrmask) pgprot_val(text_prot) &= ~clrmask; pgprot_val(data_prot) &= ~clrmask; - map_range(&ptep, (u64)_stext, (u64)__initdata_begin, (u64)_stext, - text_prot, IDMAP_ROOT_LEVEL, (pte_t *)pg_dir, false, 0); - map_range(&ptep, (u64)__initdata_begin, (u64)_end, (u64)__initdata_begin, - data_prot, IDMAP_ROOT_LEVEL, (pte_t *)pg_dir, false, 0); + map_range(&ptep, (u64)pg_end, (u64)_stext, (u64)__initdata_begin, + (u64)_stext, text_prot, IDMAP_ROOT_LEVEL, (pte_t *)pg_dir, + false, 0); + map_range(&ptep, (u64)pg_end, (u64)__initdata_begin, (u64)_end, + (u64)__initdata_begin, data_prot, IDMAP_ROOT_LEVEL, + (pte_t *)pg_dir, false, 0); return ptep; } diff --git a/arch/arm64/kernel/pi/pi.h b/arch/arm64/kernel/pi/pi.h index c91e5e965cd39..20fe0941cb8ee 100644 --- a/arch/arm64/kernel/pi/pi.h +++ b/arch/arm64/kernel/pi/pi.h @@ -28,9 +28,9 @@ u64 kaslr_early_init(void *fdt, int chosen); void relocate_kernel(u64 offset); int scs_patch(const u8 eh_frame[], int size); -void map_range(u64 *pgd, u64 start, u64 end, u64 pa, pgprot_t prot, +void map_range(u64 *pgd, u64 limit, u64 start, u64 end, u64 pa, pgprot_t prot, int level, pte_t *tbl, bool may_use_cont, u64 va_offset); asmlinkage void early_map_kernel(u64 boot_status, void *fdt); -asmlinkage u64 create_init_idmap(pgd_t *pgd, pteval_t clrmask); +asmlinkage u64 create_init_idmap(pgd_t *pgd, pgd_t *pg_end, pteval_t clrmask); diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 353ea5dc32b85..969348a2e93c9 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -773,8 +773,9 @@ static void __init declare_kernel_vmas(void) declare_vma(&vmlinux_seg[4], _data, _end, 0); } -void __pi_map_range(u64 *pgd, u64 start, u64 end, u64 pa, pgprot_t prot, - int level, pte_t *tbl, bool may_use_cont, u64 va_offset); +void __pi_map_range(u64 *pgd, u64 limit, u64 start, u64 end, u64 pa, + pgprot_t prot, int level, pte_t *tbl, bool may_use_cont, + u64 va_offset); static u8 idmap_ptes[IDMAP_LEVELS - 1][PAGE_SIZE] __aligned(PAGE_SIZE) __ro_after_init, kpti_ptes[IDMAP_LEVELS - 1][PAGE_SIZE] __aligned(PAGE_SIZE) __ro_after_init; @@ -784,8 +785,9 @@ static void __init create_idmap(void) u64 start = __pa_symbol(__idmap_text_start); u64 end = __pa_symbol(__idmap_text_end); u64 ptep = __pa_symbol(idmap_ptes); + u64 limit = __pa_symbol(&idmap_ptes[IDMAP_LEVELS - 1][0]); - __pi_map_range(&ptep, start, end, start, PAGE_KERNEL_ROX, + __pi_map_range(&ptep, limit, start, end, start, PAGE_KERNEL_ROX, IDMAP_ROOT_LEVEL, (pte_t *)idmap_pg_dir, false, __phys_to_virt(ptep) - ptep); @@ -798,8 +800,10 @@ static void __init create_idmap(void) * of its synchronization flag in the ID map. */ ptep = __pa_symbol(kpti_ptes); - __pi_map_range(&ptep, pa, pa + sizeof(u32), pa, PAGE_KERNEL, - IDMAP_ROOT_LEVEL, (pte_t *)idmap_pg_dir, false, + limit = __pa_symbol(&kpti_ptes[IDMAP_LEVELS - 1][0]); + __pi_map_range(&ptep, limit, pa, pa + sizeof(u32), pa, + PAGE_KERNEL, IDMAP_ROOT_LEVEL, + (pte_t *)idmap_pg_dir, false, __phys_to_virt(ptep) - ptep); } } From patchwork Mon Oct 14 10:58:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A4C5D1A45F for ; Mon, 14 Oct 2024 11:01:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81B376B00C5; Mon, 14 Oct 2024 07:01:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7CD6C6B00C6; Mon, 14 Oct 2024 07:01:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 669C66B00C7; Mon, 14 Oct 2024 07:01:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 45B756B00C5 for ; Mon, 14 Oct 2024 07:01:31 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 4520940D91 for ; Mon, 14 Oct 2024 11:01:26 +0000 (UTC) X-FDA: 82671916614.10.40A320D Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf19.hostedemail.com (Postfix) with ESMTP id E4E5B1A000B for ; Mon, 14 Oct 2024 11:01:21 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf19.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903658; a=rsa-sha256; cv=none; b=YFgHSFDjb9UXki342PdvWxkMMhYgj0Lx33u/0MzNvrZpURSYcEIjdyH+0yP1asWK0UJJeX pZ5gyzlWA2mZvblkSWJHT4Z2nqTD2J4zaZBFZdx9A9NY6AYyrQ06Zarqke5mYA3v2x0Fwx iJ/47CKFyArPJ3VLVn+LKQypp4avJo8= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf19.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903658; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=11sxvXvLRPO0HsHmpSNAXFOkl+67WniK18hQuyFjNNA=; b=Tw2lOlhO87kh26kZibdqbAB3hDaTci6a5cUmoDlO8WY2CahaRqsE+Y9/54gL97KF1gG+/m mNELjpoVrZbiMStv0mMgK0vq7n9vDDdhRzpbO/6RzzlLEwAo2n4WRmJ2FdwBsmV1isFX4E Jxt+G5cTZ7T9wdvsTUoCFvwD4Epx5Iw= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6F1811756; Mon, 14 Oct 2024 04:01:58 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6E88C3F51B; Mon, 14 Oct 2024 04:01:26 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 39/57] arm64: Introduce macros required for boot-time page selection Date: Mon, 14 Oct 2024 11:58:46 +0100 Message-ID: <20241014105912.3207374-39-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: c5h7bndbqh1jjnwmkp9i7iyijyxu6fiw X-Rspamd-Queue-Id: E4E5B1A000B X-Rspamd-Server: rspam02 X-HE-Tag: 1728903681-920246 X-HE-Meta: U2FsdGVkX19FXE+P51zp0dY0kJwH0gMJyfmgpZYS+wCHG2FCzpTCu/qZkte+iT1i+KmiXUNYXv6jyVgq3E8hRe9q2g2lKOM/h8is562lG/T368JyQs0KTflmX6NtnvbOhsBdMNOPlBc3HgNKRCZl+PpJxB/h5NisQHRh7EXCaq0datnnbkIg4oP85F6ZiRPzaecPFdKv1SaPK7y/foG9joNN8dLiLAz5QtIDO2B3shmip4IbbDaJn5rWJtKwK6zQGtzPuk8riLmqOpUC1RQRjONvL+N46v4tmgYcmA5VkrEdCup7/5TJex4oFb/UcN3R0mEYY2eKYKlspBy30ikrKpQX+VYFRrqYHHzF7V5Ff3wigLbuP52fndYIxt1R9CLCqR2T9xNmfqshhCcAOnn3UPd0u6SnftuC9dmoGypQQ9tnN5VpInyo8VSpiMTv4fq3dw9P8qb/XMhE3ux79lzm7TpjVLq35jmK6ukpc3vrYuBeBHDaUYHRcztfbZYfZQzEjJe0MXcAo29FHj1QNPsyoRUciMPZ0ebQKVuBAMEgunvYFTh3e5pIkjBfUUfczft+T4RbD7RA0F9ob3wuZ4gXZ3gZBY5yZn2/Qf9Nikk8eXEoSIloLZidNuvuQ78dwiCat/Lwab6AiF7lIViRGy9cgVAXD70TPIMYPtTI+j7rUTuUH1DuPteROMxIUjge+OeNbG2un7ymlr8f8hXHflDvhyWR8PuRS7QhIoRvkbZ92agC/eM4Ek46dUJxYqXiq9HowRx3KgE6zR9fJcLhx8dPcQ28kZnMRSycg2XD4IzyMIZz8+GsAXSvXALlJIyC84YlEXCbV0xmjTEAjBqFNr9x5jKk3BeDzznoUb0Id6Km9PwszdLb39AXeUl1z/8sxNr1fxO4o4D7rS/erVhELZmaQ5E7QLt2spU9ZPMnzDk+Ia8GqqbndsaLaiaTzwA3zxwXWLD4cxZuhJmlP8FC8HW SSrZFoyP hThE0ms2HIM0q4+zp5MyQRtzg9mwZZm/sjVgBe02z31ul0Z26hrBgfY0bNNoslf+yER38xzFnYh3Pv1tv7JAmfb2eeWLCoKltJbmRTO3MtwQHG4q3o996eUgnuQdXS1b+dh8xZX6VVVInJ/49ul/Bz0+McnCdklzj2k2TsoQJaz2jYQI3W1TjfJyNEw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This minmal set of macros will allow boot-time page selection support to be added to the arm64 arch code incrementally over the following set of patches. The definitions in pgtable-geometry.h are for compile-time page size currently, but they will be modified in future to support boot-time page size. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/page-def.h | 5 ++-- arch/arm64/include/asm/pgtable-geometry.h | 28 +++++++++++++++++++++++ arch/arm64/include/asm/pgtable-hwdef.h | 16 ++++++++----- 3 files changed, 40 insertions(+), 9 deletions(-) create mode 100644 arch/arm64/include/asm/pgtable-geometry.h diff --git a/arch/arm64/include/asm/page-def.h b/arch/arm64/include/asm/page-def.h index d69971cf49cd2..b99dee0112463 100644 --- a/arch/arm64/include/asm/page-def.h +++ b/arch/arm64/include/asm/page-def.h @@ -9,12 +9,11 @@ #define __ASM_PAGE_DEF_H #include +#include /* PAGE_SHIFT determines the page size */ -#define PAGE_SHIFT CONFIG_PAGE_SHIFT +#define PAGE_SHIFT ptg_page_shift #define PAGE_SIZE (_AC(1, UL) << PAGE_SHIFT) #define PAGE_MASK (~(PAGE_SIZE-1)) -#include - #endif /* __ASM_PAGE_DEF_H */ diff --git a/arch/arm64/include/asm/pgtable-geometry.h b/arch/arm64/include/asm/pgtable-geometry.h new file mode 100644 index 0000000000000..62fe125909c08 --- /dev/null +++ b/arch/arm64/include/asm/pgtable-geometry.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef ASM_PGTABLE_GEOMETRY_H +#define ASM_PGTABLE_GEOMETRY_H + +#define ARM64_PAGE_SHIFT_4K 12 +#define ARM64_PAGE_SHIFT_16K 14 +#define ARM64_PAGE_SHIFT_64K 16 + +#define PAGE_SHIFT_MIN CONFIG_PAGE_SHIFT +#define PAGE_SIZE_MIN (_AC(1, UL) << PAGE_SHIFT_MIN) +#define PAGE_MASK_MIN (~(PAGE_SIZE_MIN-1)) + +#define PAGE_SHIFT_MAX CONFIG_PAGE_SHIFT +#define PAGE_SIZE_MAX (_AC(1, UL) << PAGE_SHIFT_MAX) +#define PAGE_MASK_MAX (~(PAGE_SIZE_MAX-1)) + +#include + +#define ptg_page_shift CONFIG_PAGE_SHIFT +#define ptg_pmd_shift ARM64_HW_PGTABLE_LEVEL_SHIFT(2) +#define ptg_pud_shift ARM64_HW_PGTABLE_LEVEL_SHIFT(1) +#define ptg_p4d_shift ARM64_HW_PGTABLE_LEVEL_SHIFT(0) +#define ptg_pgdir_shift ARM64_HW_PGTABLE_LEVEL_SHIFT(4 - CONFIG_PGTABLE_LEVELS) +#define ptg_cont_pte_shift (CONFIG_ARM64_CONT_PTE_SHIFT + PAGE_SHIFT) +#define ptg_cont_pmd_shift (CONFIG_ARM64_CONT_PMD_SHIFT + PMD_SHIFT) +#define ptg_pgtable_levels CONFIG_PGTABLE_LEVELS + +#endif /* ASM_PGTABLE_GEOMETRY_H */ diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index 1f60aa1bc750c..54a9153f56bc5 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -41,39 +41,43 @@ #define ARM64_HW_PGTABLE_LEVEL_SHIFT(n) ((PAGE_SHIFT - 3) * (4 - (n)) + 3) #define PTRS_PER_PTE (1 << (PAGE_SHIFT - 3)) +#define MAX_PTRS_PER_PTE (1 << (PAGE_SHIFT_MAX - 3)) /* * PMD_SHIFT determines the size a level 2 page table entry can map. */ #if CONFIG_PGTABLE_LEVELS > 2 -#define PMD_SHIFT ARM64_HW_PGTABLE_LEVEL_SHIFT(2) +#define PMD_SHIFT ptg_pmd_shift #define PMD_SIZE (_AC(1, UL) << PMD_SHIFT) #define PMD_MASK (~(PMD_SIZE-1)) #define PTRS_PER_PMD (1 << (PAGE_SHIFT - 3)) +#define MAX_PTRS_PER_PMD (1 << (PAGE_SHIFT_MAX - 3)) #endif /* * PUD_SHIFT determines the size a level 1 page table entry can map. */ #if CONFIG_PGTABLE_LEVELS > 3 -#define PUD_SHIFT ARM64_HW_PGTABLE_LEVEL_SHIFT(1) +#define PUD_SHIFT ptg_pud_shift #define PUD_SIZE (_AC(1, UL) << PUD_SHIFT) #define PUD_MASK (~(PUD_SIZE-1)) #define PTRS_PER_PUD (1 << (PAGE_SHIFT - 3)) +#define MAX_PTRS_PER_PUD (1 << (PAGE_SHIFT_MAX - 3)) #endif #if CONFIG_PGTABLE_LEVELS > 4 -#define P4D_SHIFT ARM64_HW_PGTABLE_LEVEL_SHIFT(0) +#define P4D_SHIFT ptg_p4d_shift #define P4D_SIZE (_AC(1, UL) << P4D_SHIFT) #define P4D_MASK (~(P4D_SIZE-1)) #define PTRS_PER_P4D (1 << (PAGE_SHIFT - 3)) +#define MAX_PTRS_PER_P4D (1 << (PAGE_SHIFT_MAX - 3)) #endif /* * PGDIR_SHIFT determines the size a top-level page table entry can map * (depending on the configuration, this level can be -1, 0, 1 or 2). */ -#define PGDIR_SHIFT ARM64_HW_PGTABLE_LEVEL_SHIFT(4 - CONFIG_PGTABLE_LEVELS) +#define PGDIR_SHIFT ptg_pgdir_shift #define PGDIR_SIZE (_AC(1, UL) << PGDIR_SHIFT) #define PGDIR_MASK (~(PGDIR_SIZE-1)) #define PTRS_PER_PGD (1 << (VA_BITS - PGDIR_SHIFT)) @@ -81,12 +85,12 @@ /* * Contiguous page definitions. */ -#define CONT_PTE_SHIFT (CONFIG_ARM64_CONT_PTE_SHIFT + PAGE_SHIFT) +#define CONT_PTE_SHIFT ptg_cont_pte_shift #define CONT_PTES (1 << (CONT_PTE_SHIFT - PAGE_SHIFT)) #define CONT_PTE_SIZE (CONT_PTES * PAGE_SIZE) #define CONT_PTE_MASK (~(CONT_PTE_SIZE - 1)) -#define CONT_PMD_SHIFT (CONFIG_ARM64_CONT_PMD_SHIFT + PMD_SHIFT) +#define CONT_PMD_SHIFT ptg_cont_pmd_shift #define CONT_PMDS (1 << (CONT_PMD_SHIFT - PMD_SHIFT)) #define CONT_PMD_SIZE (CONT_PMDS * PMD_SIZE) #define CONT_PMD_MASK (~(CONT_PMD_SIZE - 1)) From patchwork Mon Oct 14 10:58:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834708 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0A45D1A43B for ; Mon, 14 Oct 2024 11:01:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7566F6B00C6; Mon, 14 Oct 2024 07:01:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 704A46B00C7; Mon, 14 Oct 2024 07:01:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 53F176B00C8; Mon, 14 Oct 2024 07:01:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 252FA6B00C6 for ; Mon, 14 Oct 2024 07:01:34 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 82F98160D48 for ; Mon, 14 Oct 2024 11:01:25 +0000 (UTC) X-FDA: 82671916698.25.88B1F17 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf18.hostedemail.com (Postfix) with ESMTP id A65E81C0023 for ; Mon, 14 Oct 2024 11:01:29 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf18.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903577; a=rsa-sha256; cv=none; b=D//9nPhLYARCiNBAWAeapiMT/XAjzg02SDTdNT/4KcdZWf9I9rnhCRdSn6uGHkoDtIX7V8 6rSSQmOoR41vnvxvJNQ83QhczyDLtdvF3xFkNDaJmKw6q7WeiC790mQivaJT0+arqeMBWk KRUmyl+hrVI6SzCp1kfMWWbHvUuX0Pw= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf18.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903577; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=v2NcdhGv+sWZMGvU1TlMDdXozrRt5nJhcZPuO+Evnng=; b=HnebGT8d8kZ5V7To3BhogkWIXIVmBPhpC4KU/txmBYnXoTOsp3YmBFF1KHgVKyQ1Zn40qO 64HfzBy0VEoTY0mlTz7Mu2ub0osJmBZtd6MtVATfBEaNmMneP4VFVZk+oCQ0r3PaItn9R+ 1v9HyjGuJ3TeSeSPThGUD8ku8N9orFE= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1FFE01713; Mon, 14 Oct 2024 04:02:01 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 204203F51B; Mon, 14 Oct 2024 04:01:29 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 40/57] arm64: Refactor early pgtable size calculation macros Date: Mon, 14 Oct 2024 11:58:47 +0100 Message-ID: <20241014105912.3207374-40-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: A65E81C0023 X-Stat-Signature: h8zg1ckyrutyixrombij9urqm5jtzyym X-Rspam-User: X-HE-Tag: 1728903689-859212 X-HE-Meta: U2FsdGVkX19akHCeMk8BhfmbVN5weHuqCxP26FnMDATtRunz1XpBSj7/LtVq6DpKiHy1SXpdX1g4xFcIWy9LYF/Ra2xvrlnzjraYvhg3uO+ANsJxgLbvNy6N8aP7vgJfwreNKroaVcqLsJOA6P8QYUA/+tF57BA8AhHgj7MgJHi3S6+hV7VTIEbVn1mRAkK/oymI52RRZXI6MWIz8yuk1dm+qp7vtiVZX5bKH3asOfZMU8EsUTvu/+nUtBPA7tEgFmdXbWsRl3FK0ABo/lgTtnrnt1NNnZHMm9MXk1gKulTZQoi5pG8ZQzVij4JLB+VNF0H4WGbNDN3ML5sQKpXssMlY5+wh1gfDZ17qjBphhbyGxaJC7OXqOvcDkWKLT3E/hBx8LhoN8VbWfTJ0kp4dd4rMYtG0ZiOf+EbPji9euJC+ucilre9GayWjlCGEF5SIi9wuPf9Ch02TodwOULUnxQIGsV15aHBguz8dIJlkjGEFgsGzSZf2CfnewFywJKp8jejhK0Rtu/zK8pJ3BaqvAweFtYMrpwHyCZASksW9JHE8kpCEm709ifWUpviJgcQUhZ087TngxH0uOuf4p8vibjUKp9GHyTmj/+/N9B1z4F2MoIpNkZTu8rmqB+BVNAhbBDhVzYOnYnwHFHeQHtdYZR7QW8vKXKVI1nYTUX4FPCoy3TBbC1SRvhCaDqi1WiptD2vRrakzPLCwGysWq0JkPiueySl1u3z7krinQ1nppoEe+4w1JCD5ihRc923hQgeIl9T8pLOxG8BnyK9xEOAGSWwiPPtVcCt2ruxKLKgvHLPia//2vW6t4dklYBHe5wbP7zdLbruwzFNkWIz8TaSsrCsxZAnz+dD+tGqTn9LUpdahLBoeOOdM/15Ev56L2GRy7bE/5ui5HJUGDKStzPRcFOpY2m8MmEkl2Usw9tw4XIe5LL1A+7SWWYTZzxSan0NJ2636NWw14ZbFlOK9vXb cZmWqkrI 5ipUqVHGZF9jc9k6WfTjWrVlNqzseiRlOPagtQAqfIvN2IElBK6CChRwgz6nHrVkwUfQ7Or8+vapVRnqQeD3M0zCTOzDhQoi6nG57gDqqxAAaxSycN4jW2UiXswopFo1VKHZjVU51f6l30yuFPIx2wbeiPYteCXc52vD1dbNfMhp8KavA4XDTR4FLZQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The various early idmaps and init/swapper pgtables are constructed using static storage, the size of which is obviously calculated at compile-time based on the selected page size. But in the near future, boot-time page size builds will need to statically allocate enough storage for the worst case, depending on which page size is selected. Therefore, refactor the macros that determine the storage requirement to take a page_shift parameter, then perform the calculation for each page size we are compiling with support for and take the max. For compile-time page size builds, the end result is exactly the same because there is only 1 page size we support. For boot-time page size builds we end up with the worst case required size. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/kernel-pgtable.h | 148 +++++++++++++++++------- arch/arm64/include/asm/pgtable-hwdef.h | 6 +- arch/arm64/kernel/pi/map_kernel.c | 6 +- arch/arm64/kernel/pi/map_range.c | 8 +- arch/arm64/kernel/vmlinux.lds.S | 4 +- arch/arm64/mm/mmu.c | 13 +-- 6 files changed, 124 insertions(+), 61 deletions(-) diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h index 1722b9217d47d..facdf273d4cda 100644 --- a/arch/arm64/include/asm/kernel-pgtable.h +++ b/arch/arm64/include/asm/kernel-pgtable.h @@ -12,28 +12,38 @@ #include #include +#define PGTABLE_LEVELS(page_shift, va_bits) \ + __ARM64_HW_PGTABLE_LEVELS(page_shift, va_bits) +#define PGTABLE_LEVEL_SHIFT(page_shift, n) \ + __ARM64_HW_PGTABLE_LEVEL_SHIFT(page_shift, n) +#define PGTABLE_LEVEL_SIZE(page_shift, n) \ + (UL(1) << PGTABLE_LEVEL_SHIFT(page_shift, n)) + /* * The physical and virtual addresses of the start of the kernel image are * equal modulo 2 MiB (per the arm64 booting.txt requirements). Hence we can * use section mapping with 4K (section size = 2M) but not with 16K (section * size = 32M) or 64K (section size = 512M). */ -#if defined(PMD_SIZE) && PMD_SIZE <= MIN_KIMG_ALIGN -#define SWAPPER_BLOCK_SHIFT PMD_SHIFT -#define SWAPPER_SKIP_LEVEL 1 -#else -#define SWAPPER_BLOCK_SHIFT PAGE_SHIFT -#define SWAPPER_SKIP_LEVEL 0 -#endif -#define SWAPPER_BLOCK_SIZE (UL(1) << SWAPPER_BLOCK_SHIFT) -#define SWAPPER_TABLE_SHIFT (SWAPPER_BLOCK_SHIFT + PAGE_SHIFT - 3) - -#define SWAPPER_PGTABLE_LEVELS (CONFIG_PGTABLE_LEVELS - SWAPPER_SKIP_LEVEL) -#define INIT_IDMAP_PGTABLE_LEVELS (IDMAP_LEVELS - SWAPPER_SKIP_LEVEL) - -#define IDMAP_VA_BITS 48 -#define IDMAP_LEVELS ARM64_HW_PGTABLE_LEVELS(IDMAP_VA_BITS) -#define IDMAP_ROOT_LEVEL (4 - IDMAP_LEVELS) +#define SWAPPER_BLOCK_SHIFT(page_shift) \ + ((PGTABLE_LEVEL_SIZE(page_shift, 2) <= MIN_KIMG_ALIGN) ? \ + PGTABLE_LEVEL_SHIFT(page_shift, 2) : (page_shift)) + +#define SWAPPER_SKIP_LEVEL(page_shift) \ + ((PGTABLE_LEVEL_SIZE(page_shift, 2) <= MIN_KIMG_ALIGN) ? 1 : 0) + +#define SWAPPER_BLOCK_SIZE(page_shift) \ + (UL(1) << SWAPPER_BLOCK_SHIFT(page_shift)) + +#define SWAPPER_PGTABLE_LEVELS(page_shift) \ + (PGTABLE_LEVELS(page_shift, VA_BITS) - SWAPPER_SKIP_LEVEL(page_shift)) + +#define INIT_IDMAP_PGTABLE_LEVELS(page_shift) \ + (IDMAP_LEVELS(page_shift) - SWAPPER_SKIP_LEVEL(page_shift)) + +#define IDMAP_VA_BITS 48 +#define IDMAP_LEVELS(page_shift) PGTABLE_LEVELS(page_shift, IDMAP_VA_BITS) +#define IDMAP_ROOT_LEVEL(page_shift) (4 - IDMAP_LEVELS(page_shift)) #define KIMAGE_VADDR_END (_AT(u64, _end) - _AT(u64, _text) + KIMAGE_VADDR) @@ -43,47 +53,99 @@ * with two adjacent PGDIR entries, which means that an additional page table * may be needed at each subordinate level. */ -#define EXTRA_PAGE __is_defined(CONFIG_RELOCATABLE) +#define EXTRA_PAGE __is_defined(CONFIG_RELOCATABLE) -#define SPAN_NR_ENTRIES(vstart, vend, shift) \ +#define SPAN_NR_ENTRIES(vstart, vend, shift) \ ((((vend) - 1) >> (shift)) - ((vstart) >> (shift)) + 1) -#define EARLY_ENTRIES(vstart, vend, shift, add) \ +#define EARLY_ENTRIES(vstart, vend, shift, add) \ (SPAN_NR_ENTRIES(vstart, vend, shift) + (add)) -#define EARLY_LEVEL(lvl, lvls, vstart, vend, add) \ - (lvls > lvl ? EARLY_ENTRIES(vstart, vend, SWAPPER_BLOCK_SHIFT + lvl * (PAGE_SHIFT - 3), add) : 0) - -#define EARLY_PAGES(lvls, vstart, vend, add) (1 /* PGDIR page */ \ - + EARLY_LEVEL(3, (lvls), (vstart), (vend), add) /* each entry needs a next level page table */ \ - + EARLY_LEVEL(2, (lvls), (vstart), (vend), add) /* each entry needs a next level page table */ \ - + EARLY_LEVEL(1, (lvls), (vstart), (vend), add))/* each entry needs a next level page table */ -#define INIT_DIR_SIZE (PAGE_SIZE * (EARLY_PAGES(SWAPPER_PGTABLE_LEVELS, KIMAGE_VADDR, KIMAGE_VADDR_END, EXTRA_PAGE) \ - + EARLY_SEGMENT_EXTRA_PAGES)) - -#define INIT_IDMAP_DIR_PAGES (EARLY_PAGES(INIT_IDMAP_PGTABLE_LEVELS, KIMAGE_VADDR, KIMAGE_VADDR_END, 1)) -#define INIT_IDMAP_DIR_SIZE ((INIT_IDMAP_DIR_PAGES + EARLY_IDMAP_EXTRA_PAGES) * PAGE_SIZE) +#define EARLY_LEVEL(page_shift, lvl, lvls, vstart, vend, add) \ + (lvls > lvl ? EARLY_ENTRIES(vstart, vend, \ + SWAPPER_BLOCK_SHIFT(page_shift) + lvl * ((page_shift) - 3), \ + add) : 0) -#define INIT_IDMAP_FDT_PAGES (EARLY_PAGES(INIT_IDMAP_PGTABLE_LEVELS, 0UL, UL(MAX_FDT_SIZE), 1) - 1) -#define INIT_IDMAP_FDT_SIZE ((INIT_IDMAP_FDT_PAGES + EARLY_IDMAP_EXTRA_FDT_PAGES) * PAGE_SIZE) +#define EARLY_PAGES(page_shift, lvls, vstart, vend, add) (1 /* PGDIR */ \ + + EARLY_LEVEL((page_shift), 3, (lvls), (vstart), (vend), add) \ + + EARLY_LEVEL((page_shift), 2, (lvls), (vstart), (vend), add) \ + + EARLY_LEVEL((page_shift), 1, (lvls), (vstart), (vend), add)) /* The number of segments in the kernel image (text, rodata, inittext, initdata, data+bss) */ -#define KERNEL_SEGMENT_COUNT 5 +#define KERNEL_SEGMENT_COUNT 5 -#if SWAPPER_BLOCK_SIZE > SEGMENT_ALIGN -#define EARLY_SEGMENT_EXTRA_PAGES (KERNEL_SEGMENT_COUNT + 1) /* * The initial ID map consists of the kernel image, mapped as two separate * segments, and may appear misaligned wrt the swapper block size. This means * we need 3 additional pages. The DT could straddle a swapper block boundary, * so it may need 2. */ -#define EARLY_IDMAP_EXTRA_PAGES 3 -#define EARLY_IDMAP_EXTRA_FDT_PAGES 2 -#else -#define EARLY_SEGMENT_EXTRA_PAGES 0 -#define EARLY_IDMAP_EXTRA_PAGES 0 -#define EARLY_IDMAP_EXTRA_FDT_PAGES 0 -#endif +#define EARLY_SEGMENT_EXTRA_PAGES(page_shift) \ + ((SWAPPER_BLOCK_SIZE(page_shift) > SEGMENT_ALIGN) ? \ + (KERNEL_SEGMENT_COUNT + 1) : 0) + +#define EARLY_IDMAP_EXTRA_PAGES(page_shift) \ + ((SWAPPER_BLOCK_SIZE(page_shift) > SEGMENT_ALIGN) ? 3 : 0) + +#define EARLY_IDMAP_EXTRA_FDT_PAGES(page_shift) \ + ((SWAPPER_BLOCK_SIZE(page_shift) > SEGMENT_ALIGN) ? 2 : 0) + +#define INIT_DIR_PAGES(page_shift) \ + (EARLY_PAGES((page_shift), SWAPPER_PGTABLE_LEVELS(page_shift), \ + KIMAGE_VADDR, KIMAGE_VADDR_END, EXTRA_PAGE)) + +#define INIT_DIR_SIZE(page_shift) \ + ((INIT_DIR_PAGES(page_shift) + \ + EARLY_SEGMENT_EXTRA_PAGES(page_shift)) * (UL(1) << (page_shift))) + +#define INIT_IDMAP_DIR_PAGES(page_shift) \ + (EARLY_PAGES((page_shift), \ + INIT_IDMAP_PGTABLE_LEVELS(page_shift), \ + KIMAGE_VADDR, KIMAGE_VADDR_END, 1)) + +#define INIT_IDMAP_DIR_SIZE(page_shift) \ + ((INIT_IDMAP_DIR_PAGES(page_shift) + \ + EARLY_IDMAP_EXTRA_PAGES(page_shift)) * (UL(1) << (page_shift))) + +#define INIT_IDMAP_FDT_PAGES(page_shift) \ + (EARLY_PAGES((page_shift), \ + INIT_IDMAP_PGTABLE_LEVELS(page_shift), \ + UL(0), UL(MAX_FDT_SIZE), 1) - 1) + +#define INIT_IDMAP_FDT_SIZE(page_shift) \ + ((INIT_IDMAP_FDT_PAGES(page_shift) + \ + EARLY_IDMAP_EXTRA_FDT_PAGES(page_shift)) * (UL(1) << (page_shift))) + +#define VAL_IF_HAVE_PGSZ(val, page_shift) \ + ((page_shift) >= PAGE_SHIFT_MIN && \ + (page_shift) <= PAGE_SHIFT_MAX ? (val) : 0) + +#define MAX_IF_HAVE_PGSZ(val4k, val16k, val64k) \ + MAX(VAL_IF_HAVE_PGSZ((val4k), ARM64_PAGE_SHIFT_4K), MAX( \ + VAL_IF_HAVE_PGSZ((val16k), ARM64_PAGE_SHIFT_16K), \ + VAL_IF_HAVE_PGSZ((val64k), ARM64_PAGE_SHIFT_64K))) + +#define IDMAP_LEVELS_MAX \ + MAX_IF_HAVE_PGSZ(IDMAP_LEVELS(ARM64_PAGE_SHIFT_4K), \ + IDMAP_LEVELS(ARM64_PAGE_SHIFT_16K), \ + IDMAP_LEVELS(ARM64_PAGE_SHIFT_64K)) + +#define __INIT_DIR_SIZE_MAX \ + MAX_IF_HAVE_PGSZ(INIT_DIR_SIZE(ARM64_PAGE_SHIFT_4K), \ + INIT_DIR_SIZE(ARM64_PAGE_SHIFT_16K), \ + INIT_DIR_SIZE(ARM64_PAGE_SHIFT_64K)) + +#define INIT_DIR_SIZE_MAX \ + MAX(__INIT_DIR_SIZE_MAX, INIT_IDMAP_DIR_SIZE_MAX) + +#define INIT_IDMAP_DIR_SIZE_MAX \ + MAX_IF_HAVE_PGSZ(INIT_IDMAP_DIR_SIZE(ARM64_PAGE_SHIFT_4K), \ + INIT_IDMAP_DIR_SIZE(ARM64_PAGE_SHIFT_16K), \ + INIT_IDMAP_DIR_SIZE(ARM64_PAGE_SHIFT_64K)) + +#define INIT_IDMAP_FDT_SIZE_MAX \ + MAX_IF_HAVE_PGSZ(INIT_IDMAP_FDT_SIZE(ARM64_PAGE_SHIFT_4K), \ + INIT_IDMAP_FDT_SIZE(ARM64_PAGE_SHIFT_16K), \ + INIT_IDMAP_FDT_SIZE(ARM64_PAGE_SHIFT_64K)) #endif /* __ASM_KERNEL_PGTABLE_H */ diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index 54a9153f56bc5..ca8bcbc1fe220 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -23,7 +23,8 @@ * * which gets simplified as : */ -#define ARM64_HW_PGTABLE_LEVELS(va_bits) (((va_bits) - 4) / (PAGE_SHIFT - 3)) +#define __ARM64_HW_PGTABLE_LEVELS(page_shift, va_bits) (((va_bits) - 4) / ((page_shift) - 3)) +#define ARM64_HW_PGTABLE_LEVELS(va_bits) __ARM64_HW_PGTABLE_LEVELS(PAGE_SHIFT, va_bits) /* * Size mapped by an entry at level n ( -1 <= n <= 3) @@ -38,7 +39,8 @@ * Rearranging it a bit we get : * (4 - n) * (PAGE_SHIFT - 3) + 3 */ -#define ARM64_HW_PGTABLE_LEVEL_SHIFT(n) ((PAGE_SHIFT - 3) * (4 - (n)) + 3) +#define __ARM64_HW_PGTABLE_LEVEL_SHIFT(page_shift, n) (((page_shift) - 3) * (4 - (n)) + 3) +#define ARM64_HW_PGTABLE_LEVEL_SHIFT(n) __ARM64_HW_PGTABLE_LEVEL_SHIFT(PAGE_SHIFT, n) #define PTRS_PER_PTE (1 << (PAGE_SHIFT - 3)) #define MAX_PTRS_PER_PTE (1 << (PAGE_SHIFT_MAX - 3)) diff --git a/arch/arm64/kernel/pi/map_kernel.c b/arch/arm64/kernel/pi/map_kernel.c index dcf9233ccfff2..a53fc225d2d0d 100644 --- a/arch/arm64/kernel/pi/map_kernel.c +++ b/arch/arm64/kernel/pi/map_kernel.c @@ -188,8 +188,8 @@ static void __init remap_idmap_for_lpa2(void) static void __init map_fdt(u64 fdt) { - static u8 ptes[INIT_IDMAP_FDT_SIZE] __initdata __aligned(PAGE_SIZE); - u64 limit = (u64)&ptes[INIT_IDMAP_FDT_SIZE]; + static u8 ptes[INIT_IDMAP_FDT_SIZE_MAX] __initdata __aligned(PAGE_SIZE); + u64 limit = (u64)&ptes[INIT_IDMAP_FDT_SIZE_MAX]; u64 efdt = fdt + MAX_FDT_SIZE; u64 ptep = (u64)ptes; @@ -199,7 +199,7 @@ static void __init map_fdt(u64 fdt) */ map_range(&ptep, limit, fdt, (u64)_text > fdt ? min((u64)_text, efdt) : efdt, - fdt, PAGE_KERNEL, IDMAP_ROOT_LEVEL, + fdt, PAGE_KERNEL, IDMAP_ROOT_LEVEL(PAGE_SHIFT), (pte_t *)init_idmap_pg_dir, false, 0); dsb(ishst); } diff --git a/arch/arm64/kernel/pi/map_range.c b/arch/arm64/kernel/pi/map_range.c index f0024d9b1d921..b62d2e3135f81 100644 --- a/arch/arm64/kernel/pi/map_range.c +++ b/arch/arm64/kernel/pi/map_range.c @@ -131,11 +131,11 @@ asmlinkage u64 __init create_init_idmap(pgd_t *pg_dir, pgd_t *pg_end, pgprot_val(data_prot) &= ~clrmask; map_range(&ptep, (u64)pg_end, (u64)_stext, (u64)__initdata_begin, - (u64)_stext, text_prot, IDMAP_ROOT_LEVEL, (pte_t *)pg_dir, - false, 0); + (u64)_stext, text_prot, + IDMAP_ROOT_LEVEL(PAGE_SHIFT), (pte_t *)pg_dir, false, 0); map_range(&ptep, (u64)pg_end, (u64)__initdata_begin, (u64)_end, - (u64)__initdata_begin, data_prot, IDMAP_ROOT_LEVEL, - (pte_t *)pg_dir, false, 0); + (u64)__initdata_begin, data_prot, + IDMAP_ROOT_LEVEL(PAGE_SHIFT), (pte_t *)pg_dir, false, 0); return ptep; } diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 55a8e310ea12c..7f3f6d709ae73 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -249,7 +249,7 @@ SECTIONS __initdata_begin = .; init_idmap_pg_dir = .; - . += INIT_IDMAP_DIR_SIZE; + . += INIT_IDMAP_DIR_SIZE_MAX; init_idmap_pg_end = .; .init.data : { @@ -319,7 +319,7 @@ SECTIONS . = ALIGN(PAGE_SIZE); init_pg_dir = .; - . += INIT_DIR_SIZE; + . += INIT_DIR_SIZE_MAX; init_pg_end = .; /* end of zero-init region */ diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 969348a2e93c9..d4d30eaefb4cd 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -777,19 +777,19 @@ void __pi_map_range(u64 *pgd, u64 limit, u64 start, u64 end, u64 pa, pgprot_t prot, int level, pte_t *tbl, bool may_use_cont, u64 va_offset); -static u8 idmap_ptes[IDMAP_LEVELS - 1][PAGE_SIZE] __aligned(PAGE_SIZE) __ro_after_init, - kpti_ptes[IDMAP_LEVELS - 1][PAGE_SIZE] __aligned(PAGE_SIZE) __ro_after_init; +static u8 idmap_ptes[IDMAP_LEVELS_MAX - 1][PAGE_SIZE] __aligned(PAGE_SIZE) __ro_after_init, + kpti_ptes[IDMAP_LEVELS_MAX - 1][PAGE_SIZE] __aligned(PAGE_SIZE) __ro_after_init; static void __init create_idmap(void) { u64 start = __pa_symbol(__idmap_text_start); u64 end = __pa_symbol(__idmap_text_end); u64 ptep = __pa_symbol(idmap_ptes); - u64 limit = __pa_symbol(&idmap_ptes[IDMAP_LEVELS - 1][0]); + u64 limit = __pa_symbol(&idmap_ptes[IDMAP_LEVELS_MAX - 1][0]); __pi_map_range(&ptep, limit, start, end, start, PAGE_KERNEL_ROX, - IDMAP_ROOT_LEVEL, (pte_t *)idmap_pg_dir, false, - __phys_to_virt(ptep) - ptep); + IDMAP_ROOT_LEVEL(PAGE_SHIFT), (pte_t *)idmap_pg_dir, + false, __phys_to_virt(ptep) - ptep); if (IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0) && !arm64_use_ng_mappings) { extern u32 __idmap_kpti_flag; @@ -800,9 +800,8 @@ static void __init create_idmap(void) * of its synchronization flag in the ID map. */ ptep = __pa_symbol(kpti_ptes); - limit = __pa_symbol(&kpti_ptes[IDMAP_LEVELS - 1][0]); __pi_map_range(&ptep, limit, pa, pa + sizeof(u32), pa, - PAGE_KERNEL, IDMAP_ROOT_LEVEL, + PAGE_KERNEL, IDMAP_ROOT_LEVEL(PAGE_SHIFT), (pte_t *)idmap_pg_dir, false, __phys_to_virt(ptep) - ptep); } From patchwork Mon Oct 14 10:58:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD19BD1A45A for ; Mon, 14 Oct 2024 11:01:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B0846B00D1; Mon, 14 Oct 2024 07:01:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0F12E6B00D2; Mon, 14 Oct 2024 07:01:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E33BD6B00D3; Mon, 14 Oct 2024 07:01:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id BEC3D6B00D1 for ; Mon, 14 Oct 2024 07:01:36 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 02ED51C6168 for ; Mon, 14 Oct 2024 11:01:27 +0000 (UTC) X-FDA: 82671916740.07.4593A44 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf01.hostedemail.com (Postfix) with ESMTP id 92DA94000C for ; Mon, 14 Oct 2024 11:01:29 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf01.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903554; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=q5VF7i0WQs1N+NdAn0MqfjtRBJLE8glwn7/XULIeBsQ=; b=5TiNgSMV+2qMq1j/92OYJYszZqRme8OV2omRluU7d0A1Cgb6oLhw7VdKVZM1qTrm61Rik2 JLUrS0EbP1pMdGQTWZvB8ZuUz/LSgcZT/R8//75tnx3aOmES562JjO0nZO5V5gSAaFSaYg 3UDtPM/xRkzJ03KI4NbUPBy0LlACVuk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903554; a=rsa-sha256; cv=none; b=M196qU8NBMISWNNPeUrGWoi9nB0brw4ZhRikfK8RRCn0Idbd6TEolO2sdmbLOoY3t7sXKh 5osBegzbcFL9Qnhub//L6QDtDx1id1XCIhFS/EvknpDpIfs09nsis6pnMS3fR7cXYRVNH2 K0GLkQiTy5Vn5xfvUasFxNBj3Wyr5gs= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf01.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C45931758; Mon, 14 Oct 2024 04:02:03 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C54893F51B; Mon, 14 Oct 2024 04:01:31 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 41/57] arm64: Pass desired page size on command line Date: Mon, 14 Oct 2024 11:58:48 +0100 Message-ID: <20241014105912.3207374-41-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 92DA94000C X-Stat-Signature: cbo3i4es3yx9eawsb5u7akqcuyuxds8b X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1728903689-450985 X-HE-Meta: U2FsdGVkX19hV+HzbV6qWVtBJY2QrMR/9T7sEH0yYOt8ewE7oLFJ+v4PBAW8UHeOmvn3kAaInmOBgSQ5vUcQMHgKzHPCigirju4xYnYTy0U3umFrKe59lnNBdmWsIoeZU0l3Rlg/eGvRVuh4yqP+fK4YTDYpmwbu5rNvzDN4eLDKEPsPTLr91aCwuKmkg4ue4SvMHe2QfeAbaWxAyokvIwsyV9ISxEUiFokm4UxenrpGF52851SsOoUHckhp1T2e74K+DE6m9AEbgoWalsdGQ+STQfZzvL0+Ye82AkN2i43A+EVXqnVdqaELzc/J5/TVzDLfRRk2G4tPqZaJewME8T1NLmAYY9+cxQYkL1s+vtFf5Qs11qEHFgC9whHxqO57oZqyRJX+x65ZBLN8fNvAJ9P/vcz/jKKXUAhz3iWtegJzofpbrD9FQVehWuSeVRrjaT87h1MyEOTgctDaTfPucNF85MdPzkU5+jvAkkvTK8e8VwJC0x5ZN2xQPfO44VzL5Iu/JT7NizUppMfMyPK7EBnfmuUo5bPbo82IsSMvvKhZ8zJvpJasvN4Poc6g7nPA5eO/TIm49sn96W8O4rBeeotPHIODd1d1CCt8dhJ+1Ygb8IkPWe8Knbxc1gCEDAvy7VrqlUV4g2iSMqtslFRsLn6GT1ese3I5TZGp3DJNw+nwOl3CTZkTBA/Dsayzl9vE1K4EcGBYlFRJdOoNDzk3cNo+ie5Joj/ubb7oBgipORzhKecKLFrlmfhcLkpJdi5o6VuFZ0SVSS8//Vuvxgz2ZXmPjripQUn2xliOTVQKKXTCS2PmeFVD8nmc/x6l5qlaHh3vCH8fq1rwfwfrC0HUVzS6QHNaBXeAJaz3WhUQBfWQTtiKzd2bshNpz0srs/VXehPz4rnciAAQ4EVf8pHXV97tahS60j9qG6qKVyutJD5SPjDWvjlbFFwUIN+DrTzYaOoGUmSfLKQ/JCHtJmV Gk6NO4XB qpLKr213niU6NUvasL40jl1ltSQeJs0UT4h+ZWHcvgMwTBGrfO1zsdRDS9TUjnRLxY+PlhNGIcBODgbPHlyST0fN1/ks7hxMmn/K+qDbqHxM6UGFwCbCgWlSt0IGM6zhNkR7gWwEQlxjCT5bLvYHt/zJiKf1BoXmxnIo0Wx4fbLUCOHhYYmr6J4TnlA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Allow user to pass desired page size via command line as either "arm64.pagesize=4k", "arm64.pagesize=16k", or "arm64.pagesize=64k". The specified value is stored in the SW_FEATURE register as an encoded page shift in a 4 bit field. We only allow setting the page size override if the requested size is supported by the HW and is within the compile-time [PAGE_SIZE_MIN, PAGE_SIZE_MAX] range. This second condition means that overrides get ignored when we have a compile-time page size (because PAGE_SIZE_MIN == PAGE_SIZE_MAX). Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/cpufeature.h | 11 ++++++++ arch/arm64/kernel/pi/idreg-override.c | 36 +++++++++++++++++++++++++++ 2 files changed, 47 insertions(+) diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index 5584342672715..4edbb586810d7 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -18,6 +18,7 @@ #define ARM64_SW_FEATURE_OVERRIDE_NOKASLR 0 #define ARM64_SW_FEATURE_OVERRIDE_HVHE 4 #define ARM64_SW_FEATURE_OVERRIDE_RODATA_OFF 8 +#define ARM64_SW_FEATURE_OVERRIDE_PAGESHIFT 12 #ifndef __ASSEMBLY__ @@ -963,6 +964,16 @@ static inline bool arm64_test_sw_feature_override(int feat) &arm64_sw_feature_override); } +static inline int arm64_pageshift_cmdline(void) +{ + int val; + + val = arm64_apply_feature_override(0, + ARM64_SW_FEATURE_OVERRIDE_PAGESHIFT, + 4, &arm64_sw_feature_override); + return val ? val * 2 + 10 : 0; +} + static inline bool kaslr_disabled_cmdline(void) { return arm64_test_sw_feature_override(ARM64_SW_FEATURE_OVERRIDE_NOKASLR); diff --git a/arch/arm64/kernel/pi/idreg-override.c b/arch/arm64/kernel/pi/idreg-override.c index 29d4b6244a6f6..5a38bdb231bc8 100644 --- a/arch/arm64/kernel/pi/idreg-override.c +++ b/arch/arm64/kernel/pi/idreg-override.c @@ -183,6 +183,38 @@ static bool __init hvhe_filter(u64 val) ID_AA64MMFR1_EL1_VH_SHIFT)); } +static bool __init pageshift_filter(u64 val) +{ + u64 mmfr0 = read_sysreg_s(SYS_ID_AA64MMFR0_EL1); + u32 tgran64 = SYS_FIELD_GET(ID_AA64MMFR0_EL1, TGRAN64, mmfr0); + u32 tgran16 = SYS_FIELD_GET(ID_AA64MMFR0_EL1, TGRAN16, mmfr0); + u32 tgran4 = SYS_FIELD_GET(ID_AA64MMFR0_EL1, TGRAN4, mmfr0); + + /* pageshift is stored compressed in 4 bit field. */ + if (val) + val = val * 2 + 10; + + if (val < PAGE_SHIFT_MIN || val > PAGE_SHIFT_MAX) + return false; + + if (val == ARM64_PAGE_SHIFT_64K && + tgran64 >= ID_AA64MMFR0_EL1_TGRAN64_SUPPORTED_MIN && + tgran64 <= ID_AA64MMFR0_EL1_TGRAN64_SUPPORTED_MAX) + return true; + + if (val == ARM64_PAGE_SHIFT_16K && + tgran16 >= ID_AA64MMFR0_EL1_TGRAN16_SUPPORTED_MIN && + tgran16 <= ID_AA64MMFR0_EL1_TGRAN16_SUPPORTED_MAX) + return true; + + if (val == ARM64_PAGE_SHIFT_4K && + tgran4 >= ID_AA64MMFR0_EL1_TGRAN4_SUPPORTED_MIN && + tgran4 <= ID_AA64MMFR0_EL1_TGRAN4_SUPPORTED_MAX) + return true; + + return false; +} + static const struct ftr_set_desc sw_features __prel64_initconst = { .name = "arm64_sw", .override = &arm64_sw_feature_override, @@ -190,6 +222,7 @@ static const struct ftr_set_desc sw_features __prel64_initconst = { FIELD("nokaslr", ARM64_SW_FEATURE_OVERRIDE_NOKASLR, NULL), FIELD("hvhe", ARM64_SW_FEATURE_OVERRIDE_HVHE, hvhe_filter), FIELD("rodataoff", ARM64_SW_FEATURE_OVERRIDE_RODATA_OFF, NULL), + FIELD("pageshift", ARM64_SW_FEATURE_OVERRIDE_PAGESHIFT, pageshift_filter), {} }, }; @@ -225,6 +258,9 @@ static const struct { { "rodata=off", "arm64_sw.rodataoff=1" }, { "arm64.nolva", "id_aa64mmfr2.varange=0" }, { "arm64.no32bit_el0", "id_aa64pfr0.el0=1" }, + { "arm64.pagesize=4k", "arm64_sw.pageshift=1" }, + { "arm64.pagesize=16k", "arm64_sw.pageshift=2" }, + { "arm64.pagesize=64k", "arm64_sw.pageshift=3" }, }; static int __init parse_hexdigit(const char *p, u64 *v) From patchwork Mon Oct 14 10:58:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834710 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92913D1A443 for ; Mon, 14 Oct 2024 11:01:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4EB66B00D2; Mon, 14 Oct 2024 07:01:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D03D76B00D4; Mon, 14 Oct 2024 07:01:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B74A66B00D5; Mon, 14 Oct 2024 07:01:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8E3366B00D2 for ; Mon, 14 Oct 2024 07:01:39 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 6FB28120D47 for ; Mon, 14 Oct 2024 11:01:32 +0000 (UTC) X-FDA: 82671917034.14.53CD70F Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP id 9A6F840011 for ; Mon, 14 Oct 2024 11:01:32 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903582; a=rsa-sha256; cv=none; b=uFn7grq3dJZ8qI7vYIvJtIkS4NfeDYhGaUWG23kWWEJ9w4jgPZx+i/1kUEqPgaPoveUakg DKX4/XGjKAJ497nZiB85BkGJMtJgahA0W5U/Fw/qwSdPg5wVQGfEubiIDrz3Zh0zg00Nu3 lnOs2JRnSUyrJu3FWWMGiqM/vaf9PZs= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903582; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MTB1pReV6clp7TUE7p0HuOwj9vCbr05wShDzVX3O4iE=; b=5XwQCpltwT01/XHlOHayFJyWRv5gO9OAzvVT3Kt6L3og6rJ93OaSjsbHoX/xJNLuxCLfBK z5KnO7/a3O6Ke08YVECIkBPOX27K+lbDKbtGBtdUV6OTw8qY4/US+4hfLi5yBydNiDFN2e hadlCLHiGVplJO7YrKlJQ0ZNE9ALaIM= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7674D175A; Mon, 14 Oct 2024 04:02:06 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 760F73F51B; Mon, 14 Oct 2024 04:01:34 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 42/57] arm64: Divorce early init from PAGE_SIZE Date: Mon, 14 Oct 2024 11:58:49 +0100 Message-ID: <20241014105912.3207374-42-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 9A6F840011 X-Stat-Signature: e19aadeydf1sdmd88j7ths8tqsoda44a X-Rspam-User: X-HE-Tag: 1728903692-93812 X-HE-Meta: U2FsdGVkX19eAhplImz9noOkP64qD8s0TIjbeCKGtzxOHDZ0k0WWcvs0guL1CoijBGIVOe2h/JOhuCDeJ5zn8ZbDAqQQWiRiQ9JISqHRFy6FYErAo0gGnoOXCGESY76fzRvrFMqGvNlADdqLwFTAEJwtBO2btZhH/XyZD6bf5K+LMcUy9b7C8krhaiSWziZ23lr/OpyX0SRREWGcFmZh6XeFi9TlolPoxIWS2r4dZYTTU4VzonUuxNGpmtvPRgVuu3cCGxd2dbBmduxQ4c+mGXoLxUDEwLx2odRAlG2drnoAsZIEFd6BBJWWKbnLxE8eLEbSTJlWTQfzKjpZyR+7fxrGyC+0PZbfRiYRtzqKFHdZ1Jut99psQ4FZBBfbfhQykQYhE4JAbogbBEXxrOItC1wMpHVztkBWmLoCJkk1GiwSNZ4aAXm+COI9lXeUYtYOnZRKdCCwKCC3sgwnZPDbYNEKvD2mvz3tksOxXtEVI/8VQrHNEtULklpp58Ix+26MQHAib8MQObUzMetpBvZ6WPsRculnFnkddW0Zn9WXGimeAxv3Zg+X29nM6C6yMkZJuZz8JK04j17lWxpSM6piKMWlwnLJ55zgv4X0oCNhQCXy0YwSIXt+Ty0vqS6ZtnMUVY9UQqXd4ekfJAFagf4/2lA3qAuOMQ6bJhWKiiEr7cnKKrGnygLdXRFuAPByhAEFYa999HGN/FYl1XS9GQngAp2zOStvqGO6tBB8ISrukmIjwW9+tnf6r8IuXcaMyn+JiSKiKaakSC78aRWB1bbsxiGCYefGBWisCp7Q72PKpqpP/AiXHPR+sf580lCRado2pr6DmzUsa4JXRYQ0kj4UOBBVgSqorzD2QWKZcwCFKiOjwHCMTIBuYxC/EgyohV/W0X2tRfN15LO0uHKXwnv7JVeEUvpbU5RmfaFEAvMCRArthlVfuDlLM+hU+tR6XE/567YI38zHic38DOWN84x KwnOspyY cVAIJ7E6TrDLutETYrzt7sHdjDeGNxqiwR3i1aX/fyQeARFwdT2fkxFTTZE8oS76fiucDAxxxXCsSDwHCMKKID51vHADH882fNjZcUUAUQdIZiIbzUhcjLGtcbsfhhgHaIn3V6VQDU8f9IA0R2SEj/ddhAbMS3II3FATZnuITHEd448920nkRn1R4P1BGU/n3COl9O75DtXaaA4QTn7e+Jd0JQmSx7Aaybgo5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Refactor all the early code between entrypoint and start_kernel() so that it does not rely on compile-time configuration of page size. For now, that code chooses to use the compile-time page size as its boot-time selected page size, but this will change in the near future to allow the page size to be specified on the command line. An initial page size is selected by probe_init_idmap_page_shift(), which is used for the init_idmap_pg_dir. This selects the largest page size that the HW supports and which is also in the range of page sizes allowed by the compile-time config. For now, the allowed range only covers the compile-time selected page size, but in future boot-time page size builds, the range will be expanded. Once the mmu is enabled, we access the command line in the device tree as before, which allows us to determine the page size requested by the user, filtered by the same allowed compile-time range of page sizes - still just the compile-time selected page size for now. If no acceptable page size was specified on the command line, we fall back to the page size selected in probe_init_idmap_page_shift(). We then do a dance to repaint init_idmap_pg_dir for the final page size and for LPA2, if in use. Finally with that installed, we can continue booting the kernel. For all of this to work, we must replace previous compile-time decisions with run-time decisions. For the most part, we can do this by looking at tcr_el1.tg0 to determine the installed page size. These run-time decisions are not in hot code paths so I don't anticipate any performance regressions as a result and therefore I prefer the simplicity of using the run-time approach even for builds that specify a compile-time page size. Of course to be able to actually change the early page size, the static storage for the page tables need to be sized for the maximum requirement. Currently they are still sized for the compile-time page size. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/assembler.h | 50 +++++++- arch/arm64/include/asm/cpufeature.h | 33 +++-- arch/arm64/include/asm/pgtable-prot.h | 2 +- arch/arm64/kernel/head.S | 40 ++++-- arch/arm64/kernel/pi/idreg-override.c | 32 +++-- arch/arm64/kernel/pi/map_kernel.c | 87 +++++++++---- arch/arm64/kernel/pi/map_range.c | 171 ++++++++++++++++++++++---- arch/arm64/kernel/pi/pi.h | 61 ++++++++- arch/arm64/mm/proc.S | 21 ++-- 9 files changed, 405 insertions(+), 92 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index bc0b0d75acef7..77c2d707adb1a 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -568,6 +568,14 @@ alternative_endif mrs \rd, sp_el0 .endm +/* + * Retrieve and return tcr_el1.tg0 in \tg0. + */ + .macro get_tg0, tg0 + mrs \tg0, tcr_el1 + and \tg0, \tg0, #TCR_TG0_MASK + .endm + /* * If the kernel is built for 52-bit virtual addressing but the hardware only * supports 48 bits, we cannot program the pgdir address into TTBR1 directly, @@ -584,12 +592,16 @@ alternative_endif * ttbr: Value of ttbr to set, modified. */ .macro offset_ttbr1, ttbr, tmp -#if defined(CONFIG_ARM64_VA_BITS_52) && !defined(CONFIG_ARM64_LPA2) +#if defined(CONFIG_ARM64_VA_BITS_52) + get_tg0 \tmp + cmp \tmp, #TCR_TG0_64K + b.ne .Ldone\@ mrs \tmp, tcr_el1 and \tmp, \tmp, #TCR_T1SZ_MASK cmp \tmp, #TCR_T1SZ(VA_BITS_MIN) orr \tmp, \ttbr, #TTBR1_BADDR_4852_OFFSET csel \ttbr, \tmp, \ttbr, eq +.Ldone\@: #endif .endm @@ -863,4 +875,40 @@ alternative_cb ARM64_ALWAYS_SYSTEM, spectre_bhb_patch_clearbhb alternative_cb_end #endif /* CONFIG_MITIGATE_SPECTRE_BRANCH_HISTORY */ .endm + +/* + * Given \tg0, populates \val with one of the 3 passed in values, corresponding + * to the page size advertised by \tg0. + */ + .macro value_for_page_size, val, tg0, val4k, val16k, val64k +.Lsz_64k\@: + cmp \tg0, #TCR_TG0_64K + b.ne .Lsz_16k\@ + mov \val, #\val64k + b .Ldone\@ +.Lsz_16k\@: + cmp \tg0, #TCR_TG0_16K + b.ne .Lsz_4k\@ + mov \val, #\val16k + b .Ldone\@ +.Lsz_4k\@: + mov \val, #\val4k +.Ldone\@: + .endm + + .macro tgran_shift, val, tg0 + value_for_page_size \val, \tg0, ID_AA64MMFR0_EL1_TGRAN4_SHIFT, ID_AA64MMFR0_EL1_TGRAN16_SHIFT, ID_AA64MMFR0_EL1_TGRAN64_SHIFT + .endm + + .macro tgran_min, val, tg0 + value_for_page_size \val, \tg0, ID_AA64MMFR0_EL1_TGRAN4_SUPPORTED_MIN, ID_AA64MMFR0_EL1_TGRAN16_SUPPORTED_MIN, ID_AA64MMFR0_EL1_TGRAN64_SUPPORTED_MIN + .endm + + .macro tgran_max, val, tg0 + value_for_page_size \val, \tg0, ID_AA64MMFR0_EL1_TGRAN4_SUPPORTED_MAX, ID_AA64MMFR0_EL1_TGRAN16_SUPPORTED_MAX, ID_AA64MMFR0_EL1_TGRAN64_SUPPORTED_MAX + .endm + + .macro tgran_lpa2, val, tg0 + value_for_page_size \val, \tg0, ID_AA64MMFR0_EL1_TGRAN4_52_BIT, ID_AA64MMFR0_EL1_TGRAN16_52_BIT, -1 + .endm #endif /* __ASM_ASSEMBLER_H */ diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index 4edbb586810d7..2c22cfdc04bc7 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -26,6 +26,7 @@ #include #include #include +#include /* * CPU feature register tracking @@ -1014,10 +1015,13 @@ static inline bool cpu_has_pac(void) &id_aa64isar2_override); } -static inline bool cpu_has_lva(void) +static inline bool cpu_has_lva(u64 page_size) { u64 mmfr2; + if (page_size != SZ_64K) + return false; + mmfr2 = read_sysreg_s(SYS_ID_AA64MMFR2_EL1); mmfr2 &= ~id_aa64mmfr2_override.mask; mmfr2 |= id_aa64mmfr2_override.val; @@ -1025,22 +1029,31 @@ static inline bool cpu_has_lva(void) ID_AA64MMFR2_EL1_VARange_SHIFT); } -static inline bool cpu_has_lpa2(void) +static inline bool cpu_has_lpa2(u64 page_size) { -#ifdef CONFIG_ARM64_LPA2 u64 mmfr0; int feat; + int shift; + int minval; + + switch (page_size) { + case SZ_4K: + shift = ID_AA64MMFR0_EL1_TGRAN4_SHIFT; + minval = ID_AA64MMFR0_EL1_TGRAN4_52_BIT; + break; + case SZ_16K: + shift = ID_AA64MMFR0_EL1_TGRAN16_SHIFT; + minval = ID_AA64MMFR0_EL1_TGRAN16_52_BIT; + break; + default: + return false; + } mmfr0 = read_sysreg(id_aa64mmfr0_el1); mmfr0 &= ~id_aa64mmfr0_override.mask; mmfr0 |= id_aa64mmfr0_override.val; - feat = cpuid_feature_extract_signed_field(mmfr0, - ID_AA64MMFR0_EL1_TGRAN_SHIFT); - - return feat >= ID_AA64MMFR0_EL1_TGRAN_LPA2; -#else - return false; -#endif + feat = cpuid_feature_extract_signed_field(mmfr0, shift); + return feat >= minval; } #endif /* __ASSEMBLY__ */ diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h index b11cfb9fdd379..f8ebf424ca016 100644 --- a/arch/arm64/include/asm/pgtable-prot.h +++ b/arch/arm64/include/asm/pgtable-prot.h @@ -74,7 +74,7 @@ extern bool arm64_use_ng_mappings; #define PTE_MAYBE_NG (arm64_use_ng_mappings ? PTE_NG : 0) #define PMD_MAYBE_NG (arm64_use_ng_mappings ? PMD_SECT_NG : 0) -#ifndef CONFIG_ARM64_LPA2 +#ifndef CONFIG_ARM64_VA_BITS_52 #define lpa2_is_enabled() false #define PTE_MAYBE_SHARED PTE_SHARED #define PMD_MAYBE_SHARED PMD_SECT_S diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 7e17a71fd9e4b..761b7f5633e15 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -88,6 +88,8 @@ SYM_CODE_START(primary_entry) adrp x1, early_init_stack mov sp, x1 mov x29, xzr + bl __pi_probe_init_idmap_page_shift + mov x3, x0 adrp x0, init_idmap_pg_dir adrp x1, init_idmap_pg_end mov x2, xzr @@ -471,11 +473,16 @@ SYM_FUNC_END(set_cpu_boot_mode_flag) */ .section ".idmap.text","a" SYM_FUNC_START(__enable_mmu) + get_tg0 x3 + tgran_shift x4, x3 + tgran_min x5, x3 + tgran_max x6, x3 mrs x3, ID_AA64MMFR0_EL1 - ubfx x3, x3, #ID_AA64MMFR0_EL1_TGRAN_SHIFT, 4 - cmp x3, #ID_AA64MMFR0_EL1_TGRAN_SUPPORTED_MIN + lsr x3, x3, x4 + ubfx x3, x3, #0, 4 + cmp x3, x5 b.lt __no_granule_support - cmp x3, #ID_AA64MMFR0_EL1_TGRAN_SUPPORTED_MAX + cmp x3, x6 b.gt __no_granule_support phys_to_ttbr x2, x2 msr ttbr0_el1, x2 // load TTBR0 @@ -488,17 +495,32 @@ SYM_FUNC_END(__enable_mmu) #ifdef CONFIG_ARM64_VA_BITS_52 SYM_FUNC_START(__cpu_secondary_check52bitva) -#ifndef CONFIG_ARM64_LPA2 + /* + * tcr_el1 is not yet loaded (there is a chicken-and-egg problem) so we + * can't figure out LPA2 vs LVA from that. But this is only called for + * secondary cpus so tcr_boot_fields has already been populated by the + * primary cpu. So grab that and rely on the DS bit to tell us if we are + * LPA2 or LVA. + */ + adr_l x1, __pi_tcr_boot_fields + ldr x0, [x1] + and x1, x0, #TCR_DS + cbnz x1, .Llpa2 +.Llva: mrs_s x0, SYS_ID_AA64MMFR2_EL1 and x0, x0, ID_AA64MMFR2_EL1_VARange_MASK cbnz x0, 2f -#else + b .Lfail +.Llpa2: + and x0, x0, #TCR_TG0_MASK + tgran_shift x1, x0 + tgran_lpa2 x2, x0 mrs x0, id_aa64mmfr0_el1 - sbfx x0, x0, #ID_AA64MMFR0_EL1_TGRAN_SHIFT, 4 - cmp x0, #ID_AA64MMFR0_EL1_TGRAN_LPA2 + lsr x0, x0, x1 + sbfx x0, x0, #0, 4 + cmp x0, x2 b.ge 2f -#endif - +.Lfail: update_early_cpu_boot_status \ CPU_STUCK_IN_KERNEL | CPU_STUCK_REASON_52_BIT_VA, x0, x1 1: wfe diff --git a/arch/arm64/kernel/pi/idreg-override.c b/arch/arm64/kernel/pi/idreg-override.c index 5a38bdb231bc8..0685c0a3255e2 100644 --- a/arch/arm64/kernel/pi/idreg-override.c +++ b/arch/arm64/kernel/pi/idreg-override.c @@ -62,20 +62,34 @@ static const struct ftr_set_desc mmfr1 __prel64_initconst = { static bool __init mmfr2_varange_filter(u64 val) { - int __maybe_unused feat; + u64 mmfr0; + int feat; + int shift; + int minval; if (val) return false; -#ifdef CONFIG_ARM64_LPA2 - feat = cpuid_feature_extract_signed_field(read_sysreg(id_aa64mmfr0_el1), - ID_AA64MMFR0_EL1_TGRAN_SHIFT); - if (feat >= ID_AA64MMFR0_EL1_TGRAN_LPA2) { - id_aa64mmfr0_override.val |= - (ID_AA64MMFR0_EL1_TGRAN_LPA2 - 1) << ID_AA64MMFR0_EL1_TGRAN_SHIFT; - id_aa64mmfr0_override.mask |= 0xfU << ID_AA64MMFR0_EL1_TGRAN_SHIFT; + mmfr0 = read_sysreg(id_aa64mmfr0_el1); + + /* Remove LPA2 support for 4K granule. */ + shift = ID_AA64MMFR0_EL1_TGRAN4_SHIFT; + minval = ID_AA64MMFR0_EL1_TGRAN4_52_BIT; + feat = cpuid_feature_extract_signed_field(mmfr0, shift); + if (feat >= minval) { + id_aa64mmfr0_override.val |= (minval - 1) << shift; + id_aa64mmfr0_override.mask |= 0xfU << shift; + } + + /* Remove LPA2 support for 16K granule. */ + shift = ID_AA64MMFR0_EL1_TGRAN16_SHIFT; + minval = ID_AA64MMFR0_EL1_TGRAN16_52_BIT; + feat = cpuid_feature_extract_signed_field(mmfr0, shift); + if (feat >= minval) { + id_aa64mmfr0_override.val |= (minval - 1) << shift; + id_aa64mmfr0_override.mask |= 0xfU << shift; } -#endif + return true; } diff --git a/arch/arm64/kernel/pi/map_kernel.c b/arch/arm64/kernel/pi/map_kernel.c index a53fc225d2d0d..7a62d4238449d 100644 --- a/arch/arm64/kernel/pi/map_kernel.c +++ b/arch/arm64/kernel/pi/map_kernel.c @@ -133,10 +133,18 @@ static void __init map_kernel(u64 kaslr_offset, u64 va_offset, int root_level) idmap_cpu_replace_ttbr1(swapper_pg_dir); } -static void noinline __section(".idmap.text") set_ttbr0_for_lpa2(u64 ttbr) +static void noinline __section(".idmap.text") set_ttbr0(u64 ttbr, bool use_lpa2, + int page_shift) { u64 sctlr = read_sysreg(sctlr_el1); - u64 tcr = read_sysreg(tcr_el1) | TCR_DS; + u64 tcr = read_sysreg(tcr_el1); + u64 boot_fields = early_page_shift_to_tcr_tgx(page_shift); + + if (use_lpa2) + boot_fields |= TCR_DS; + + tcr &= ~(TCR_TG0_MASK | TCR_TG1_MASK | TCR_DS); + tcr |= boot_fields; asm(" msr sctlr_el1, %0 ;" " isb ;" @@ -149,57 +157,66 @@ static void noinline __section(".idmap.text") set_ttbr0_for_lpa2(u64 ttbr) " msr sctlr_el1, %3 ;" " isb ;" :: "r"(sctlr & ~SCTLR_ELx_M), "r"(ttbr), "r"(tcr), "r"(sctlr)); + + /* Stash this for __cpu_setup to configure secondary cpus' tcr_el1. */ + set_tcr_boot_fields(boot_fields); } -static void __init remap_idmap_for_lpa2(void) +static void __init remap_idmap(bool use_lpa2, int page_shift) { /* clear the bits that change meaning once LPA2 is turned on */ - pteval_t mask = PTE_SHARED; + pteval_t mask = use_lpa2 ? PTE_SHARED : 0; /* - * We have to clear bits [9:8] in all block or page descriptors in the - * initial ID map, as otherwise they will be (mis)interpreted as + * For LPA2, We have to clear bits [9:8] in all block or page descriptors + * in the initial ID map, as otherwise they will be (mis)interpreted as * physical address bits once we flick the LPA2 switch (TCR.DS). Since * we cannot manipulate live descriptors in that way without creating * potential TLB conflicts, let's create another temporary ID map in a * LPA2 compatible fashion, and update the initial ID map while running * from that. */ - create_init_idmap(init_pg_dir, init_pg_end, mask); + create_init_idmap(init_pg_dir, init_pg_end, mask, page_shift); dsb(ishst); - set_ttbr0_for_lpa2((u64)init_pg_dir); + set_ttbr0((u64)init_pg_dir, use_lpa2, page_shift); /* - * Recreate the initial ID map with the same granularity as before. - * Don't bother with the FDT, we no longer need it after this. + * Recreate the initial ID map with new page size and, if LPA2 is in + * use, bits [9:8] cleared. */ memset(init_idmap_pg_dir, 0, (u64)init_idmap_pg_end - (u64)init_idmap_pg_dir); - create_init_idmap(init_idmap_pg_dir, init_idmap_pg_end, mask); + create_init_idmap(init_idmap_pg_dir, init_idmap_pg_end, mask, page_shift); dsb(ishst); /* switch back to the updated initial ID map */ - set_ttbr0_for_lpa2((u64)init_idmap_pg_dir); + set_ttbr0((u64)init_idmap_pg_dir, use_lpa2, page_shift); /* wipe the temporary ID map from memory */ memset(init_pg_dir, 0, (u64)init_pg_end - (u64)init_pg_dir); } -static void __init map_fdt(u64 fdt) +static void __init map_fdt(u64 fdt, int page_shift) { static u8 ptes[INIT_IDMAP_FDT_SIZE_MAX] __initdata __aligned(PAGE_SIZE); + static bool first_time __initdata = true; u64 limit = (u64)&ptes[INIT_IDMAP_FDT_SIZE_MAX]; u64 efdt = fdt + MAX_FDT_SIZE; u64 ptep = (u64)ptes; + if (!first_time) { + memset(ptes, 0, sizeof(ptes)); + first_time = false; + } + /* * Map up to MAX_FDT_SIZE bytes, but avoid overlap with * the kernel image. */ map_range(&ptep, limit, fdt, (u64)_text > fdt ? min((u64)_text, efdt) : efdt, - fdt, PAGE_KERNEL, IDMAP_ROOT_LEVEL(PAGE_SHIFT), + fdt, PAGE_KERNEL, IDMAP_ROOT_LEVEL(page_shift), (pte_t *)init_idmap_pg_dir, false, 0); dsb(ishst); } @@ -207,13 +224,16 @@ static void __init map_fdt(u64 fdt) asmlinkage void __init early_map_kernel(u64 boot_status, void *fdt) { static char const chosen_str[] __initconst = "/chosen"; + u64 early_page_shift = early_tcr_tg0_to_page_shift(); u64 va_base, pa_base = (u64)&_text; u64 kaslr_offset = pa_base % MIN_KIMG_ALIGN; - int root_level = 4 - CONFIG_PGTABLE_LEVELS; int va_bits = VA_BITS; + bool use_lpa2 = false; + int root_level; + int page_shift; int chosen; - map_fdt((u64)fdt); + map_fdt((u64)fdt, early_page_shift); /* Clear BSS and the initial page tables */ memset(__bss_start, 0, (u64)init_pg_end - (u64)__bss_start); @@ -222,16 +242,37 @@ asmlinkage void __init early_map_kernel(u64 boot_status, void *fdt) chosen = fdt_path_offset(fdt, chosen_str); init_feature_override(boot_status, fdt, chosen); - if (IS_ENABLED(CONFIG_ARM64_64K_PAGES) && !cpu_has_lva()) { - va_bits = VA_BITS_MIN; - } else if (IS_ENABLED(CONFIG_ARM64_LPA2) && !cpu_has_lpa2()) { - va_bits = VA_BITS_MIN; - root_level++; + /* Get page_shift from cmdline, falling back to early_page_shift. */ + page_shift = arm64_pageshift_cmdline(); + if (!page_shift) + page_shift = early_page_shift; + + if (va_bits > 48) { + u64 page_size = early_page_size(page_shift); + + if (page_size == SZ_64K) { + if (!cpu_has_lva(page_size)) + va_bits = VA_BITS_MIN; + } else { + use_lpa2 = cpu_has_lpa2(page_size); + if (!use_lpa2) + va_bits = VA_BITS_MIN; + } } if (va_bits > VA_BITS_MIN) sysreg_clear_set(tcr_el1, TCR_T1SZ_MASK, TCR_T1SZ(va_bits)); + /* + * This will update tg0/tg1 in tcr for the final page size. After this, + * PAGE_SIZE and friends can be used safely. kaslr_early_init(), below, + * is the first such user. + */ + if (use_lpa2 || page_shift != early_page_shift) { + remap_idmap(use_lpa2, page_shift); + map_fdt((u64)fdt, page_shift); + } + /* * The virtual KASLR displacement modulo 2MiB is decided by the * physical placement of the image, as otherwise, we might not be able @@ -248,9 +289,7 @@ asmlinkage void __init early_map_kernel(u64 boot_status, void *fdt) kaslr_offset |= kaslr_seed & ~(MIN_KIMG_ALIGN - 1); } - if (IS_ENABLED(CONFIG_ARM64_LPA2) && va_bits > VA_BITS_MIN) - remap_idmap_for_lpa2(); - va_base = KIMAGE_VADDR + kaslr_offset; + root_level = 4 - PGTABLE_LEVELS(page_shift, va_bits); map_kernel(kaslr_offset, va_base - pa_base, root_level); } diff --git a/arch/arm64/kernel/pi/map_range.c b/arch/arm64/kernel/pi/map_range.c index b62d2e3135f81..be5470a969a47 100644 --- a/arch/arm64/kernel/pi/map_range.c +++ b/arch/arm64/kernel/pi/map_range.c @@ -11,6 +11,34 @@ #include "pi.h" +static inline u64 __init pte_get_oa(pte_t pte, int page_shift, bool oa52bit) +{ + pteval_t pv = pte_val(pte); + +#ifdef CONFIG_ARM64_PA_BITS_52 + if (oa52bit) { + if (early_page_size(page_shift) == SZ_64K) + return (pv & GENMASK(47, 16)) | + ((pv & GENMASK(15, 12)) << 36); + return (pv & GENMASK(49, 12)) | ((pv & GENMASK(9, 8)) << 42); + } +#endif + return pv & GENMASK(47, 12); +} + +static inline u64 __init pte_prep_oa(u64 oa, int page_shift, bool oa52bit) +{ +#ifdef CONFIG_ARM64_PA_BITS_52 + if (oa52bit) { + if (early_page_size(page_shift) == SZ_64K) + return (oa & GENMASK(47, 16)) | + ((oa >> 36) & GENMASK(15, 12)); + return (oa & GENMASK(49, 12)) | ((oa >> 42) & GENMASK(9, 8)); + } +#endif + return oa; +} + static void __init mmuoff_data_clean(void) { bool cache_ena = !!(read_sysreg(sctlr_el1) & SCTLR_ELx_C); @@ -35,8 +63,19 @@ static void __init report_cpu_stuck(long val) cpu_park_loop(); } +u64 __section(".mmuoff.data.read") tcr_boot_fields; + +void __init set_tcr_boot_fields(u64 val) +{ + WRITE_ONCE(tcr_boot_fields, val); + + /* Ensure the visibility of the new value */ + dsb(ishst); + mmuoff_data_clean(); +} + /** - * map_range - Map a contiguous range of physical pages into virtual memory + * __map_range - Map a contiguous range of physical pages into virtual memory * * @pte: Address of physical pointer to array of pages to * allocate page tables from @@ -50,21 +89,28 @@ static void __init report_cpu_stuck(long val) * @may_use_cont: Whether the use of the contiguous attribute is allowed * @va_offset: Offset between a physical page and its current mapping * in the VA space + * @page_shift: Page size (as a shift) to create page table for + * @oa52bit: Whether to store output addresses in 52-bit format */ -void __init map_range(u64 *pte, u64 limit, u64 start, u64 end, u64 pa, - pgprot_t prot, int level, pte_t *tbl, bool may_use_cont, - u64 va_offset) +static void __init __map_range(u64 *pte, u64 limit, u64 start, u64 end, u64 pa, + pgprot_t prot, int level, pte_t *tbl, + bool may_use_cont, u64 va_offset, int page_shift, + bool oa52bit) { - u64 cmask = (level == 3) ? CONT_PTE_SIZE - 1 : U64_MAX; + const u64 page_size = early_page_size(page_shift); + const u64 page_mask = early_page_mask(page_shift); + const u64 cont_pte_size = early_cont_pte_size(page_shift); + const u64 ptrs_per_pte = early_ptrs_per_pte(page_shift); + u64 cmask = (level == 3) ? cont_pte_size - 1 : U64_MAX; u64 protval = pgprot_val(prot) & ~PTE_TYPE_MASK; - int lshift = (3 - level) * (PAGE_SHIFT - 3); - u64 lmask = (PAGE_SIZE << lshift) - 1; + int lshift = (3 - level) * (page_shift - 3); + u64 lmask = (page_size << lshift) - 1; - start &= PAGE_MASK; - pa &= PAGE_MASK; + start &= page_mask; + pa &= page_mask; /* Advance tbl to the entry that covers start */ - tbl += (start >> (lshift + PAGE_SHIFT)) % PTRS_PER_PTE; + tbl += (start >> (lshift + page_shift)) % ptrs_per_pte; /* * Set the right block/page bits for this level unless we are @@ -74,7 +120,7 @@ void __init map_range(u64 *pte, u64 limit, u64 start, u64 end, u64 pa, protval |= (level < 3) ? PMD_TYPE_SECT : PTE_TYPE_PAGE; while (start < end) { - u64 next = min((start | lmask) + 1, PAGE_ALIGN(end)); + u64 next = min((start | lmask) + 1, ALIGN(end, page_size)); if (level < 3 && (start | next | pa) & lmask) { /* @@ -82,20 +128,20 @@ void __init map_range(u64 *pte, u64 limit, u64 start, u64 end, u64 pa, * table mapping if necessary and recurse. */ if (pte_none(*tbl)) { - u64 size = PTRS_PER_PTE * sizeof(pte_t); + u64 size = ptrs_per_pte * sizeof(pte_t); if (*pte + size > limit) { report_cpu_stuck( CPU_STUCK_REASON_NO_PGTABLE_MEM); } - *tbl = __pte(__phys_to_pte_val(*pte) | + *tbl = __pte(pte_prep_oa(*pte, page_shift, oa52bit) | PMD_TYPE_TABLE | PMD_TABLE_UXN); *pte += size; } - map_range(pte, limit, start, next, pa, prot, level + 1, - (pte_t *)(__pte_to_phys(*tbl) + va_offset), - may_use_cont, va_offset); + __map_range(pte, limit, start, next, pa, prot, level + 1, + (pte_t *)(pte_get_oa(*tbl, page_shift, oa52bit) + va_offset), + may_use_cont, va_offset, page_shift, oa52bit); } else { /* * Start a contiguous range if start and pa are @@ -112,7 +158,8 @@ void __init map_range(u64 *pte, u64 limit, u64 start, u64 end, u64 pa, protval &= ~PTE_CONT; /* Put down a block or page mapping */ - *tbl = __pte(__phys_to_pte_val(pa) | protval); + *tbl = __pte(pte_prep_oa(pa, page_shift, oa52bit) | + protval); } pa += next - start; start = next; @@ -120,22 +167,96 @@ void __init map_range(u64 *pte, u64 limit, u64 start, u64 end, u64 pa, } } +/** + * map_range - Map a contiguous range of physical pages into virtual memory + * + * As per __map_range(), except it uses the page_shift and oa52bit of the + * currently tcr-installed granule size instead of passing explicitly. + */ +void __init map_range(u64 *pte, u64 limit, u64 start, u64 end, u64 pa, + pgprot_t prot, int level, pte_t *tbl, bool may_use_cont, + u64 va_offset) +{ + int page_shift = early_tcr_tg0_to_page_shift(); + bool oa52bit = false; + +#ifdef CONFIG_ARM64_PA_BITS_52 + /* + * We can safely assume 52bit for 64K pages because if it turns out to + * be 48bit, its still safe to treat [51:48] as address bits because + * they are 0. + */ + if (early_page_size(page_shift) == SZ_64K) + oa52bit = true; + /* + * For 4K and 16K, on the other hand, those bits are used for something + * else when LPA2 is not explicitly enabled. Deliberately not using + * read_tcr() since it is marked pure, and at this point, the tcr is not + * yet stable. + */ + else if (read_sysreg(tcr_el1) & TCR_DS) + oa52bit = true; +#endif + + __map_range(pte, limit, start, end, pa, prot, level, tbl, may_use_cont, + va_offset, page_shift, oa52bit); +} + +asmlinkage u64 __init probe_init_idmap_page_shift(void) +{ + u64 mmfr0 = read_sysreg_s(SYS_ID_AA64MMFR0_EL1); + u32 tgran64 = SYS_FIELD_GET(ID_AA64MMFR0_EL1, TGRAN64, mmfr0); + u32 tgran16 = SYS_FIELD_GET(ID_AA64MMFR0_EL1, TGRAN16, mmfr0); + u32 tgran4 = SYS_FIELD_GET(ID_AA64MMFR0_EL1, TGRAN4, mmfr0); + u64 page_shift; + + /* + * Select the largest page size supported by the HW, which is also + * allowed by the compilation config. + */ + if (ARM64_PAGE_SHIFT_64K >= PAGE_SHIFT_MIN && + ARM64_PAGE_SHIFT_64K <= PAGE_SHIFT_MAX && + tgran64 >= ID_AA64MMFR0_EL1_TGRAN64_SUPPORTED_MIN && + tgran64 <= ID_AA64MMFR0_EL1_TGRAN64_SUPPORTED_MAX) + page_shift = ARM64_PAGE_SHIFT_64K; + else if (ARM64_PAGE_SHIFT_16K >= PAGE_SHIFT_MIN && + ARM64_PAGE_SHIFT_16K <= PAGE_SHIFT_MAX && + tgran16 >= ID_AA64MMFR0_EL1_TGRAN16_SUPPORTED_MIN && + tgran16 <= ID_AA64MMFR0_EL1_TGRAN16_SUPPORTED_MAX) + page_shift = ARM64_PAGE_SHIFT_16K; + else if (ARM64_PAGE_SHIFT_4K >= PAGE_SHIFT_MIN && + ARM64_PAGE_SHIFT_4K <= PAGE_SHIFT_MAX && + tgran4 >= ID_AA64MMFR0_EL1_TGRAN4_SUPPORTED_MIN && + tgran4 <= ID_AA64MMFR0_EL1_TGRAN4_SUPPORTED_MAX) + page_shift = ARM64_PAGE_SHIFT_4K; + else + report_cpu_stuck(CPU_STUCK_REASON_NO_GRAN); + + /* Stash this for __cpu_setup to configure primary cpu's tcr_el1. */ + set_tcr_boot_fields(early_page_shift_to_tcr_tgx(page_shift)); + + return page_shift; +} + asmlinkage u64 __init create_init_idmap(pgd_t *pg_dir, pgd_t *pg_end, - pteval_t clrmask) + pteval_t clrmask, int page_shift) { - u64 ptep = (u64)pg_dir + PAGE_SIZE; + const u64 page_size = early_page_size(page_shift); + u64 ptep = (u64)pg_dir + page_size; pgprot_t text_prot = PAGE_KERNEL_ROX; pgprot_t data_prot = PAGE_KERNEL; pgprot_val(text_prot) &= ~clrmask; pgprot_val(data_prot) &= ~clrmask; - map_range(&ptep, (u64)pg_end, (u64)_stext, (u64)__initdata_begin, - (u64)_stext, text_prot, - IDMAP_ROOT_LEVEL(PAGE_SHIFT), (pte_t *)pg_dir, false, 0); - map_range(&ptep, (u64)pg_end, (u64)__initdata_begin, (u64)_end, - (u64)__initdata_begin, data_prot, - IDMAP_ROOT_LEVEL(PAGE_SHIFT), (pte_t *)pg_dir, false, 0); + __map_range(&ptep, (u64)pg_end, (u64)_stext, (u64)__initdata_begin, + (u64)_stext, text_prot, + IDMAP_ROOT_LEVEL(page_shift), (pte_t *)pg_dir, false, 0, + page_shift, false); + __map_range(&ptep, (u64)pg_end, (u64)__initdata_begin, (u64)_end, + (u64)__initdata_begin, data_prot, + IDMAP_ROOT_LEVEL(page_shift), (pte_t *)pg_dir, false, 0, + page_shift, false); return ptep; } diff --git a/arch/arm64/kernel/pi/pi.h b/arch/arm64/kernel/pi/pi.h index 20fe0941cb8ee..15c14d0aa6c63 100644 --- a/arch/arm64/kernel/pi/pi.h +++ b/arch/arm64/kernel/pi/pi.h @@ -23,6 +23,62 @@ extern bool dynamic_scs_is_enabled; extern pgd_t init_idmap_pg_dir[], init_idmap_pg_end[]; +static inline u64 early_page_size(int page_shift) +{ + return 1UL << page_shift; +} + +static inline u64 early_page_mask(int page_shift) +{ + return ~(early_page_size(page_shift) - 1); +} + +static inline u64 early_cont_pte_size(int page_shift) +{ + switch (page_shift) { + case 16: /* 64K */ + case 14: /* 16K */ + return SZ_2M; + default: /* 12 4K */ + return SZ_64K; + } +} + +static inline u64 early_ptrs_per_pte(int page_shift) +{ + return 1UL << (page_shift - 3); +} + +static inline int early_tcr_tg0_to_page_shift(void) +{ + /* + * Deliberately not using read_tcr() since it is marked pure, and at + * this point, the tcr is not yet stable. + */ + u64 tg0 = read_sysreg(tcr_el1) & TCR_TG0_MASK; + + switch (tg0) { + case TCR_TG0_64K: + return 16; + case TCR_TG0_16K: + return 14; + default: /* TCR_TG0_4K */ + return 12; + } +} + +static inline u64 early_page_shift_to_tcr_tgx(int page_shift) +{ + switch (early_page_size(page_shift)) { + case SZ_64K: + return TCR_TG0_64K | TCR_TG1_64K; + case SZ_16K: + return TCR_TG0_16K | TCR_TG1_16K; + default: + return TCR_TG0_4K | TCR_TG1_4K; + } +} + void init_feature_override(u64 boot_status, const void *fdt, int chosen); u64 kaslr_early_init(void *fdt, int chosen); void relocate_kernel(u64 offset); @@ -33,4 +89,7 @@ void map_range(u64 *pgd, u64 limit, u64 start, u64 end, u64 pa, pgprot_t prot, asmlinkage void early_map_kernel(u64 boot_status, void *fdt); -asmlinkage u64 create_init_idmap(pgd_t *pgd, pgd_t *pg_end, pteval_t clrmask); +void set_tcr_boot_fields(u64 val); +asmlinkage u64 probe_init_idmap_page_shift(void); +asmlinkage u64 create_init_idmap(pgd_t *pgd, pgd_t *pg_end, pteval_t clrmask, + int page_shift); diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index f4bc6c5bac062..ab5aa84923524 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -22,14 +22,6 @@ #include #include -#ifdef CONFIG_ARM64_64K_PAGES -#define TCR_TG_FLAGS TCR_TG0_64K | TCR_TG1_64K -#elif defined(CONFIG_ARM64_16K_PAGES) -#define TCR_TG_FLAGS TCR_TG0_16K | TCR_TG1_16K -#else /* CONFIG_ARM64_4K_PAGES */ -#define TCR_TG_FLAGS TCR_TG0_4K | TCR_TG1_4K -#endif - #ifdef CONFIG_RANDOMIZE_BASE #define TCR_KASLR_FLAGS TCR_NFD1 #else @@ -469,18 +461,23 @@ SYM_FUNC_START(__cpu_setup) tcr .req x16 mov_q mair, MAIR_EL1_SET mov_q tcr, TCR_T0SZ(IDMAP_VA_BITS) | TCR_T1SZ(VA_BITS_MIN) | TCR_CACHE_FLAGS | \ - TCR_SMP_FLAGS | TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ + TCR_SMP_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ TCR_TBI0 | TCR_A1 | TCR_KASAN_SW_FLAGS | TCR_MTE_FLAGS + /* + * Insert the boot-time determined fields (TG0, TG1 and DS), which are + * cached in the tcr_boot_fields variable. + */ + adr_l x2, __pi_tcr_boot_fields + ldr x3, [x2] + orr tcr, tcr, x3 + tcr_clear_errata_bits tcr, x9, x5 #ifdef CONFIG_ARM64_VA_BITS_52 mov x9, #64 - VA_BITS alternative_if ARM64_HAS_VA52 tcr_set_t1sz tcr, x9 -#ifdef CONFIG_ARM64_LPA2 - orr tcr, tcr, #TCR_DS -#endif alternative_else_nop_endif #endif From patchwork Mon Oct 14 10:58:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6A45D1A43B for ; Mon, 14 Oct 2024 11:01:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 05B1D6B00D4; Mon, 14 Oct 2024 07:01:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 004A86B00D6; Mon, 14 Oct 2024 07:01:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CFBC36B00D7; Mon, 14 Oct 2024 07:01:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id AB4E46B00D4 for ; Mon, 14 Oct 2024 07:01:42 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4D401ABCB3 for ; Mon, 14 Oct 2024 11:01:27 +0000 (UTC) X-FDA: 82671916950.09.5A76472 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf21.hostedemail.com (Postfix) with ESMTP id 234661C0004 for ; Mon, 14 Oct 2024 11:01:28 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903629; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r9v3S3hMjuiCiaBUIv6jmo6gDiOvZ9WOws2YRMMwzI8=; b=hQoh4qdYjOdaa0R8xZoBrHDpbMqxlShL3YfnoqMm1PvchMAiaZc7iYpsQu3NBUQMiYsS9l 1x+dFapv3RqLQ95n1XG1UWICPPkqGJDPOfRuFWzE7htqKadyk3nJ2862or3/a3xUEiY5ne OB5fJg+McgsXNP8zy/KJXazXu3iXXhQ= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903629; a=rsa-sha256; cv=none; b=xiwN4+AS3zAe61jXvjn3p7O69T5Ynrf56uFsgVTkGQVd6Ozz16rU+88wtqrLjm3l+VRN4l rcWz9mGFtJaIrzpT5IbqorzpNYHHV4z13jIuxWMAjkZCy6fXaHbmCmg1t9b30/PG0r5Vqr 3s8rtHP/Nw6oG0Dkjwuvq58ahKzj9rM= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C1414175D; Mon, 14 Oct 2024 04:02:09 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2811C3F51B; Mon, 14 Oct 2024 04:01:37 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Andrey Ryabinin , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Oliver Upton , Thomas Gleixner , Will Deacon Cc: Ryan Roberts , kasan-dev@googlegroups.com, kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 43/57] arm64: Clean up simple cases of CONFIG_ARM64_*K_PAGES Date: Mon, 14 Oct 2024 11:58:50 +0100 Message-ID: <20241014105912.3207374-43-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 234661C0004 X-Stat-Signature: 53bxsknt8ysrxocm8nrq8h4ztg6ycowe X-HE-Tag: 1728903688-569153 X-HE-Meta: U2FsdGVkX19t5ZlZ1XrYYsRtbHJ1DACzK/8gmDwPdgOxnkjQNmaBT9HyyI7tVRnBZZBa9QK4Fgc2MxS/FcYE+/+qggPRbbcnHND4mjAY+68G+lqTgFGwWv8QHCKd/8O2ERZIyAiRLHmN/sJOkuywfSD4fnODZOLfxw26/u95pn24ZecSPSK+WHS1nG/n0IV3CyJS3QSSkn5E3t3WXRWRCzJEaapQY8A3eSMDxBH45rt4mu0knbhVpMGNiswqfVZSldZ9oiXs4GDIoxpPx2EhOwrpc0koOzGY7LarqVdwhPu9RrCdgvjBlC+Nvdpyt2cLRcOwkwGuxDmimdngPaXKLLlzM9cxQle7A2vk/GujF1F6RKfWO7CzO9P3SeR5GK15hxdhrt+aMJ8J5VsJ4TJ4R8iPFiyBY7502ii9HaRtBc8UwCXTNQFnO9Lpr5pk8ma+u6cSRlNpk8o3cTqHjaSDf0OFPLybwxTghFbjS9Q0pcq4phGPE9AhXTwfs1rIKk93J+n5q3FEDDQdDOfldtEP0BHWsExIFH7ZqtU9Y88K5J6iOKkMOaTlKxqN4nNwjw4VhavqTvX94u5x/GFNA0FK5V4gfjIvJHnw/JC6EyWKaEKNczTGRth/KJkYLlVEF0CXX9Cp2ei7XmyUlJdfEWojCm7STyM2mhCOUF1Q7QZrHDogcKeQlYUmvjfcEYvsyB8rEueypEpiErBmgKh6GQLiYg/ZDOcYNIDVY/pHJUTb0UvQO34iAGwwUK/DuakDhBo2hNjUBrJMe/Mfnnb7+my7AXObP9L1Tg8pWNhNxh5AWe07zXHQ4aKjCxhqpm6qlRRouGK8WVO9hC9kEBIQK6VsEWE0RCYYCQ198nHmQl9kYiPMQ6RJxwcb9CEZ+zO1CKm28zz53IfV8wWloct+G4GSpIlatowjmCzUFNPQX9HzAr6bCwlZwTJAbH1/PvT033+VUHUsmDc6S5XAwvmuziU 3SKODi1i 4LDFNCExJW6B5vYi63CwSP3BDOgO6eAACxznZ1KdrGm1rsJc+Ej3rcr15VJ0hLLRUDE4Iumv1Y3C77KoRIgRFD16zDivd+p/z1+Xz+/hZhWZq5fKR50KARbbRguyIg1qusqxB/Z4ae8IVjvET35wrb3x8KBiTH9MB5sctW8R0ld0VNH+1//ViscrY/UvQii6pX/HwQvwVgtsdqlchzmX938xQcSRWpwg9uMS+xylIV0M0Dtvu16S7xbLUcuY0cRk3YyTVma+3eD39rjOQHEsLyxbgt9T971KS4el8Us+9iJH4/cE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There are a number of places that define macros conditionally depending on which of the CONFIG_ARM64_*K_PAGES macros are defined. But in preparation for supporting boot-time page size selection, we will no longer be able to make these decisions at compile time. So let's refactor the code to check the size of PAGE_SIZE using the ternary operator. This approach will still resolve to compile-time constants when configured for a compile-time page size, but it will also work when we turn PAGE_SIZE into a run-time value. Additionally, IS_ENABLED(CONFIG_ARM64_*K_PAGES) instances are also converted to test the size of PAGE_SIZE. Additionally modify ARM64_HAS_VA52 capability detection to use a custom match function, which chooses which feature register and field to check based on PAGE_SIZE. The compiler will eliminate the other page sizes when selecting a compile time page size, but will also now cope with seting page size at boot time. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/kvm_arm.h | 21 ++++------- arch/arm64/include/asm/kvm_pgtable.h | 6 +--- arch/arm64/include/asm/memory.h | 7 ++-- arch/arm64/include/asm/processor.h | 10 +++--- arch/arm64/include/asm/sparsemem.h | 11 ++---- arch/arm64/include/asm/sysreg.h | 54 ++++++++++++++++++---------- arch/arm64/kernel/cpufeature.c | 43 +++++++++++++--------- arch/arm64/mm/fixmap.c | 2 +- arch/arm64/mm/init.c | 20 +++++------ arch/arm64/mm/kasan_init.c | 8 ++--- arch/arm64/mm/mmu.c | 2 +- drivers/irqchip/irq-gic-v3-its.c | 2 +- 12 files changed, 94 insertions(+), 92 deletions(-) diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index d81cc746e0ebd..08155dc17ad17 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -189,22 +189,13 @@ * Entry_Level = 4 - Number_of_levels. * */ -#ifdef CONFIG_ARM64_64K_PAGES +#define VTCR_EL2_TGRAN \ + (PAGE_SIZE == SZ_64K ? \ + VTCR_EL2_TG0_64K : \ + (PAGE_SIZE == SZ_16K ? VTCR_EL2_TG0_16K : VTCR_EL2_TG0_4K)) -#define VTCR_EL2_TGRAN VTCR_EL2_TG0_64K -#define VTCR_EL2_TGRAN_SL0_BASE 3UL - -#elif defined(CONFIG_ARM64_16K_PAGES) - -#define VTCR_EL2_TGRAN VTCR_EL2_TG0_16K -#define VTCR_EL2_TGRAN_SL0_BASE 3UL - -#else /* 4K */ - -#define VTCR_EL2_TGRAN VTCR_EL2_TG0_4K -#define VTCR_EL2_TGRAN_SL0_BASE 2UL - -#endif +#define VTCR_EL2_TGRAN_SL0_BASE \ + (PAGE_SIZE == SZ_64K ? 3UL : (PAGE_SIZE == SZ_16K ? 3UL : 2UL)) #define VTCR_EL2_LVLS_TO_SL0(levels) \ ((VTCR_EL2_TGRAN_SL0_BASE - (4 - (levels))) << VTCR_EL2_SL0_SHIFT) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/kvm_pgtable.h index 19278dfe79782..796614bf59e78 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -20,11 +20,7 @@ * - 16K (level 2): 32MB * - 64K (level 2): 512MB */ -#ifdef CONFIG_ARM64_4K_PAGES -#define KVM_PGTABLE_MIN_BLOCK_LEVEL 1 -#else -#define KVM_PGTABLE_MIN_BLOCK_LEVEL 2 -#endif +#define KVM_PGTABLE_MIN_BLOCK_LEVEL (PAGE_SIZE == SZ_4K ? 1 : 2) #define kvm_lpa2_is_enabled() system_supports_lpa2() diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 54fb014eba058..6aa97fa22dc30 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -188,11 +188,8 @@ #define MT_S2_FWB_NORMAL_NC 5 #define MT_S2_FWB_DEVICE_nGnRE 1 -#ifdef CONFIG_ARM64_4K_PAGES -#define IOREMAP_MAX_ORDER (PUD_SHIFT) -#else -#define IOREMAP_MAX_ORDER (PMD_SHIFT) -#endif +#define IOREMAP_MAX_ORDER \ + (PAGE_SIZE == SZ_4K ? PUD_SHIFT : PMD_SHIFT) /* * Open-coded (swapper_pg_dir - reserved_pg_dir) as this cannot be calculated diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index f77371232d8c6..444694a4e6733 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -55,15 +55,15 @@ #define TASK_SIZE_MAX (UL(1) << VA_BITS) #ifdef CONFIG_COMPAT -#if defined(CONFIG_ARM64_64K_PAGES) && defined(CONFIG_KUSER_HELPERS) +#if defined(CONFIG_KUSER_HELPERS) /* - * With CONFIG_ARM64_64K_PAGES enabled, the last page is occupied - * by the compat vectors page. + * With 64K pages in use, the last page is occupied by the compat vectors page. */ -#define TASK_SIZE_32 UL(0x100000000) +#define TASK_SIZE_32 \ + (PAGE_SIZE == SZ_64K ? UL(0x100000000) : (UL(0x100000000) - PAGE_SIZE)) #else #define TASK_SIZE_32 (UL(0x100000000) - PAGE_SIZE) -#endif /* CONFIG_ARM64_64K_PAGES */ +#endif /* CONFIG_KUSER_HELPERS */ #define TASK_SIZE (test_thread_flag(TIF_32BIT) ? \ TASK_SIZE_32 : TASK_SIZE_64) #define TASK_SIZE_OF(tsk) (test_tsk_thread_flag(tsk, TIF_32BIT) ? \ diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h index 8a8acc220371c..a05fdd54014f7 100644 --- a/arch/arm64/include/asm/sparsemem.h +++ b/arch/arm64/include/asm/sparsemem.h @@ -11,19 +11,12 @@ * Section size must be at least 512MB for 64K base * page size config. Otherwise it will be less than * MAX_PAGE_ORDER and the build process will fail. - */ -#ifdef CONFIG_ARM64_64K_PAGES -#define SECTION_SIZE_BITS 29 - -#else - -/* + * * Section size must be at least 128MB for 4K base * page size config. Otherwise PMD based huge page * entries could not be created for vmemmap mappings. * 16K follows 4K for simplicity. */ -#define SECTION_SIZE_BITS 27 -#endif /* CONFIG_ARM64_64K_PAGES */ +#define SECTION_SIZE_BITS (PAGE_SIZE == SZ_64K ? 29 : 27) #endif diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 4a9ea103817e8..cbcf861bbf2a6 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -10,10 +10,12 @@ #define __ASM_SYSREG_H #include +#include #include #include #include +#include /* * ARMv8 ARM reserves the following encoding for system registers: @@ -913,24 +915,40 @@ #define ID_AA64MMFR0_EL1_PARANGE_MAX ID_AA64MMFR0_EL1_PARANGE_48 #endif -#if defined(CONFIG_ARM64_4K_PAGES) -#define ID_AA64MMFR0_EL1_TGRAN_SHIFT ID_AA64MMFR0_EL1_TGRAN4_SHIFT -#define ID_AA64MMFR0_EL1_TGRAN_LPA2 ID_AA64MMFR0_EL1_TGRAN4_52_BIT -#define ID_AA64MMFR0_EL1_TGRAN_SUPPORTED_MIN ID_AA64MMFR0_EL1_TGRAN4_SUPPORTED_MIN -#define ID_AA64MMFR0_EL1_TGRAN_SUPPORTED_MAX ID_AA64MMFR0_EL1_TGRAN4_SUPPORTED_MAX -#define ID_AA64MMFR0_EL1_TGRAN_2_SHIFT ID_AA64MMFR0_EL1_TGRAN4_2_SHIFT -#elif defined(CONFIG_ARM64_16K_PAGES) -#define ID_AA64MMFR0_EL1_TGRAN_SHIFT ID_AA64MMFR0_EL1_TGRAN16_SHIFT -#define ID_AA64MMFR0_EL1_TGRAN_LPA2 ID_AA64MMFR0_EL1_TGRAN16_52_BIT -#define ID_AA64MMFR0_EL1_TGRAN_SUPPORTED_MIN ID_AA64MMFR0_EL1_TGRAN16_SUPPORTED_MIN -#define ID_AA64MMFR0_EL1_TGRAN_SUPPORTED_MAX ID_AA64MMFR0_EL1_TGRAN16_SUPPORTED_MAX -#define ID_AA64MMFR0_EL1_TGRAN_2_SHIFT ID_AA64MMFR0_EL1_TGRAN16_2_SHIFT -#elif defined(CONFIG_ARM64_64K_PAGES) -#define ID_AA64MMFR0_EL1_TGRAN_SHIFT ID_AA64MMFR0_EL1_TGRAN64_SHIFT -#define ID_AA64MMFR0_EL1_TGRAN_SUPPORTED_MIN ID_AA64MMFR0_EL1_TGRAN64_SUPPORTED_MIN -#define ID_AA64MMFR0_EL1_TGRAN_SUPPORTED_MAX ID_AA64MMFR0_EL1_TGRAN64_SUPPORTED_MAX -#define ID_AA64MMFR0_EL1_TGRAN_2_SHIFT ID_AA64MMFR0_EL1_TGRAN64_2_SHIFT -#endif +#define ID_AA64MMFR0_EL1_TGRAN_SHIFT \ + (PAGE_SIZE == SZ_4K ? \ + ID_AA64MMFR0_EL1_TGRAN4_SHIFT : \ + (PAGE_SIZE == SZ_16K ? \ + ID_AA64MMFR0_EL1_TGRAN16_SHIFT : \ + ID_AA64MMFR0_EL1_TGRAN64_SHIFT)) + +#define ID_AA64MMFR0_EL1_TGRAN_LPA2 \ + (PAGE_SIZE == SZ_4K ? \ + ID_AA64MMFR0_EL1_TGRAN4_52_BIT : \ + (PAGE_SIZE == SZ_16K ? \ + ID_AA64MMFR0_EL1_TGRAN16_52_BIT : \ + -1)) + +#define ID_AA64MMFR0_EL1_TGRAN_SUPPORTED_MIN \ + (PAGE_SIZE == SZ_4K ? \ + ID_AA64MMFR0_EL1_TGRAN4_SUPPORTED_MIN : \ + (PAGE_SIZE == SZ_16K ? \ + ID_AA64MMFR0_EL1_TGRAN16_SUPPORTED_MIN : \ + ID_AA64MMFR0_EL1_TGRAN64_SUPPORTED_MIN)) + +#define ID_AA64MMFR0_EL1_TGRAN_SUPPORTED_MAX \ + (PAGE_SIZE == SZ_4K ? \ + ID_AA64MMFR0_EL1_TGRAN4_SUPPORTED_MAX : \ + (PAGE_SIZE == SZ_16K ? \ + ID_AA64MMFR0_EL1_TGRAN16_SUPPORTED_MAX : \ + ID_AA64MMFR0_EL1_TGRAN64_SUPPORTED_MAX)) + +#define ID_AA64MMFR0_EL1_TGRAN_2_SHIFT \ + (PAGE_SIZE == SZ_4K ? \ + ID_AA64MMFR0_EL1_TGRAN4_2_SHIFT : \ + (PAGE_SIZE == SZ_16K ? \ + ID_AA64MMFR0_EL1_TGRAN16_2_SHIFT : \ + ID_AA64MMFR0_EL1_TGRAN64_2_SHIFT)) #define CPACR_EL1_FPEN_EL1EN (BIT(20)) /* enable EL1 access */ #define CPACR_EL1_FPEN_EL0EN (BIT(21)) /* enable EL0 access, if EL1EN set */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 646ecd3069fdd..7705c9c0e7142 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1831,11 +1831,13 @@ static bool has_nv1(const struct arm64_cpu_capabilities *entry, int scope) is_midr_in_range_list(read_cpuid_id(), nv1_ni_list))); } -#if defined(ID_AA64MMFR0_EL1_TGRAN_LPA2) && defined(ID_AA64MMFR0_EL1_TGRAN_2_SUPPORTED_LPA2) static bool has_lpa2_at_stage1(u64 mmfr0) { unsigned int tgran; + if (PAGE_SIZE == SZ_64K) + return false; + tgran = cpuid_feature_extract_unsigned_field(mmfr0, ID_AA64MMFR0_EL1_TGRAN_SHIFT); return tgran == ID_AA64MMFR0_EL1_TGRAN_LPA2; @@ -1845,6 +1847,9 @@ static bool has_lpa2_at_stage2(u64 mmfr0) { unsigned int tgran; + if (PAGE_SIZE == SZ_64K) + return false; + tgran = cpuid_feature_extract_unsigned_field(mmfr0, ID_AA64MMFR0_EL1_TGRAN_2_SHIFT); return tgran == ID_AA64MMFR0_EL1_TGRAN_2_SUPPORTED_LPA2; @@ -1857,10 +1862,26 @@ static bool has_lpa2(const struct arm64_cpu_capabilities *entry, int scope) mmfr0 = read_sanitised_ftr_reg(SYS_ID_AA64MMFR0_EL1); return has_lpa2_at_stage1(mmfr0) && has_lpa2_at_stage2(mmfr0); } -#else -static bool has_lpa2(const struct arm64_cpu_capabilities *entry, int scope) + +#ifdef CONFIG_ARM64_VA_BITS_52 +static bool has_va52(const struct arm64_cpu_capabilities *entry, int scope) { - return false; + const struct arm64_cpu_capabilities entry_64k = { + ARM64_CPUID_FIELDS(ID_AA64MMFR2_EL1, VARange, 52) + }; + const struct arm64_cpu_capabilities entry_16k = { + ARM64_CPUID_FIELDS(ID_AA64MMFR0_EL1, TGRAN16, 52_BIT) + }; + const struct arm64_cpu_capabilities entry_4k = { + ARM64_CPUID_FIELDS(ID_AA64MMFR0_EL1, TGRAN4, 52_BIT) + }; + + if (PAGE_SIZE == SZ_64K) + return has_cpuid_feature(&entry_64k, scope); + else if (PAGE_SIZE == SZ_16K) + return has_cpuid_feature(&entry_16k, scope); + else + return has_cpuid_feature(&entry_4k, scope); } #endif @@ -2847,20 +2868,10 @@ static const struct arm64_cpu_capabilities arm64_features[] = { }, #ifdef CONFIG_ARM64_VA_BITS_52 { + .desc = "52-bit Virtual Addressing", .capability = ARM64_HAS_VA52, .type = ARM64_CPUCAP_BOOT_CPU_FEATURE, - .matches = has_cpuid_feature, -#ifdef CONFIG_ARM64_64K_PAGES - .desc = "52-bit Virtual Addressing (LVA)", - ARM64_CPUID_FIELDS(ID_AA64MMFR2_EL1, VARange, 52) -#else - .desc = "52-bit Virtual Addressing (LPA2)", -#ifdef CONFIG_ARM64_4K_PAGES - ARM64_CPUID_FIELDS(ID_AA64MMFR0_EL1, TGRAN4, 52_BIT) -#else - ARM64_CPUID_FIELDS(ID_AA64MMFR0_EL1, TGRAN16, 52_BIT) -#endif -#endif + .matches = has_va52, }, #endif { diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index de1e09d986ad2..15ce3253ad359 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -82,7 +82,7 @@ static void __init early_fixmap_init_pud(p4d_t *p4dp, unsigned long addr, * share the top level pgd entry, which should only happen on * 16k/4 levels configurations. */ - BUG_ON(!IS_ENABLED(CONFIG_ARM64_16K_PAGES)); + BUG_ON(PAGE_SIZE != SZ_16K); } if (p4d_none(p4d)) diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 9b5ab6818f7f3..42eb246949072 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -73,13 +73,10 @@ phys_addr_t __ro_after_init arm64_dma_phys_limit; * (64k granule), or a multiple that can be mapped using contiguous bits * in the page tables: 32 * PMD_SIZE (16k granule) */ -#if defined(CONFIG_ARM64_4K_PAGES) -#define ARM64_MEMSTART_SHIFT PUD_SHIFT -#elif defined(CONFIG_ARM64_16K_PAGES) -#define ARM64_MEMSTART_SHIFT CONT_PMD_SHIFT -#else -#define ARM64_MEMSTART_SHIFT PMD_SHIFT -#endif +#define ARM64_MEMSTART_SHIFT \ + (PAGE_SIZE == SZ_4K ? \ + PUD_SHIFT : \ + (PAGE_SIZE == SZ_16K ? CONT_PMD_SHIFT : PMD_SHIFT)) /* * sparsemem vmemmap imposes an additional requirement on the alignment of @@ -87,11 +84,10 @@ phys_addr_t __ro_after_init arm64_dma_phys_limit; * has a direct correspondence, and needs to appear sufficiently aligned * in the virtual address space. */ -#if ARM64_MEMSTART_SHIFT < SECTION_SIZE_BITS -#define ARM64_MEMSTART_ALIGN (1UL << SECTION_SIZE_BITS) -#else -#define ARM64_MEMSTART_ALIGN (1UL << ARM64_MEMSTART_SHIFT) -#endif +#define ARM64_MEMSTART_ALIGN \ + (ARM64_MEMSTART_SHIFT < SECTION_SIZE_BITS ? \ + (1UL << SECTION_SIZE_BITS) : \ + (1UL << ARM64_MEMSTART_SHIFT)) static void __init arch_reserve_crashkernel(void) { diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index b65a29440a0c9..9af897fb3c432 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -178,10 +178,10 @@ static void __init kasan_pgd_populate(unsigned long addr, unsigned long end, } while (pgdp++, addr = next, addr != end); } -#if defined(CONFIG_ARM64_64K_PAGES) || CONFIG_PGTABLE_LEVELS > 4 +#if CONFIG_PGTABLE_LEVELS > 4 #define SHADOW_ALIGN P4D_SIZE #else -#define SHADOW_ALIGN PUD_SIZE +#define SHADOW_ALIGN (PAGE_SIZE == SZ_64K ? P4D_SIZE : PUD_SIZE) #endif /* @@ -243,8 +243,8 @@ static int __init root_level_idx(u64 addr) * not implemented. This means we need to index the table as usual, * instead of masking off bits based on vabits_actual. */ - u64 vabits = IS_ENABLED(CONFIG_ARM64_64K_PAGES) ? VA_BITS - : vabits_actual; + u64 vabits = PAGE_SIZE == SZ_64K ? VA_BITS + : vabits_actual; int shift = (ARM64_HW_PGTABLE_LEVELS(vabits) - 1) * (PAGE_SHIFT - 3); return (addr & ~_PAGE_OFFSET(vabits)) >> (shift + PAGE_SHIFT); diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index d4d30eaefb4cd..a528787c1e550 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1179,7 +1179,7 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, { WARN_ON((start < VMEMMAP_START) || (end > VMEMMAP_END)); - if (!IS_ENABLED(CONFIG_ARM64_4K_PAGES)) + if (PAGE_SIZE != SZ_4K) return vmemmap_populate_basepages(start, end, node, altmap); else return vmemmap_populate_hugepages(start, end, node, altmap); diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c index fdec478ba5e70..b745579b4b9f3 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -2323,7 +2323,7 @@ static int its_setup_baser(struct its_node *its, struct its_baser *baser, baser_phys = virt_to_phys(base); /* Check if the physical address of the memory is above 48bits */ - if (IS_ENABLED(CONFIG_ARM64_64K_PAGES) && (baser_phys >> 48)) { + if (PAGE_SIZE == SZ_64K && (baser_phys >> 48)) { /* 52bit PA is supported only when PageSize=64K */ if (psz != SZ_64K) { From patchwork Mon Oct 14 10:58:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834712 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D95C3D1A43B for ; Mon, 14 Oct 2024 11:01:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E350D6B00D6; Mon, 14 Oct 2024 07:01:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DBC876B00D8; Mon, 14 Oct 2024 07:01:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5F536B00D9; Mon, 14 Oct 2024 07:01:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A3FA36B00D6 for ; Mon, 14 Oct 2024 07:01:45 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 35178ABCB3 for ; Mon, 14 Oct 2024 11:01:30 +0000 (UTC) X-FDA: 82671917118.08.3B0F186 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf28.hostedemail.com (Postfix) with ESMTP id C8244C001A for ; Mon, 14 Oct 2024 11:01:37 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf28.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903672; a=rsa-sha256; cv=none; b=1tDy2cchji7/WcIi242EbFSx/H1jnpUcglz5aDABNSFNtwrf3MdMqZ4oNPl+nV4y2LS5Kf fVCdBQhBdrMjn1PCCwVyfX5MGymgwhd9oIyCinGiWet8Vs3pj5xWvn0Z1smMdwv1BLb6Xm 0VSoa8MOCEU1kQB04WB6Ii1tAy0zsU0= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf28.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903672; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XkvwSRpxq/ReVb2KZsrYsZWT5AXEytyt77PllAZm++A=; b=3kxTJ0jBzOR0GuppQX6yteezCcrUYEKbGYOK017lHoeGGOxmqYyFNrPTrjhDYTz4sn/iPQ T5iJfndppGul9/cpSdwt8tkO/w7kVKzgD+TkSZwCYXTtjcaW4v/H700sah9EfJC0cixUsy xT0SxnbO2kiEF6Igy/loIQaPqO26Uy8= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AEC241763; Mon, 14 Oct 2024 04:02:12 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 722EB3F51B; Mon, 14 Oct 2024 04:01:40 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Oliver Upton , Will Deacon Cc: Ryan Roberts , kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 44/57] arm64: Align sections to PAGE_SIZE_MAX Date: Mon, 14 Oct 2024 11:58:51 +0100 Message-ID: <20241014105912.3207374-44-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: su7jrfy91smegafs4izhzrbor9coigfd X-Rspamd-Queue-Id: C8244C001A X-Rspamd-Server: rspam02 X-HE-Tag: 1728903697-563734 X-HE-Meta: U2FsdGVkX18UMajl+fb/ZgqPEO/YCVsI866Y5UB5nWZc9WClPhaSXOWNziBRBo9WOBRwPv4XRIshhy5vPAKLy9A3/PH3MH/fl2H0Aoh12lTyQy2k+Dcf6AgEvsNcyAoJE/TsEM8fJ7oc/a3t6ZSmlEZo8l/PJWOhvkwRRm4XX/BuwhPinMqzlEqXVrmG9sILo4nrR1ul2dpbggz06VYK/YT06Vm8XbxZZNE/+mBnqeWFQFSeQGERizWfZXk5V5OBCpxPs2bvs+jf+n5etBEUoIOelNlndFD2krh1U7il5fmN6aNbP7tGINh4O/4kz2WIieksvmJ3LwsS+L+GF09gUOEtL7fT4uX4xFNtl0sDretB8e+tGutzdgbbp1rkL7YzaPSXilgUMNMv9vj0/9e8Gy4GAyHIYa4aU7XRKR7oYKDONWDvrgD4k78+mvtS3/RDIeNLjdgviFl9wKHXDrpxKaidx5Tjk3bqe8xfSk3lZaE5L2aSxAuwAQx0ePmXUCDRA6ojzARfeT3woPBZKQRrXtv+eIXtfJz8j9GIbpE3Ip2IHj9SuJlSaFhpcifyiB8JrfwWWjzlrAqHlcltJNzkKa6gG1Q+wPnZg9ievRtfGb3Xnl43hn1j91oQwmXg2eI2UhJdg2WhYaDMAgxlR/tk0BTXwBoMizwlW3nu/cQDdeHPEYnNq6kB16oYVM9QzZvgbDizAurySV6kbQRw2ybG/3Ly8npekAxZr0Li73XmSNQfAklcu1m8GAXPRAYAycZMJbJ/k6L48CG8RGrSLT5nf/x5PIxNh+g5AsKLDpL6OAJ/9LVGsO+GsAjNJItzwGzkvJRg16DftXjplHkU9qgIOBTV04if1sEdtifURT9Q4vXz1VnY70m3lhYua2JrU1EEd5VKMWFRr8WpkKOKNbACEqtgfjTJA8hXI+qcWnr5BiK83TF4gJXHULUxxHu1m0IJt6I7MxhWER97gNjKed3 TE4hnUoy 767sFTOUH3fKxmx98z0g/HcZJTmToN4ZkpohrEy8tL4K7wbrmYGOkwuoRhZ4uKFEnZhbJEIp6dWIfQYYkrDFJetHKLp17mp2YR1hjFHBDf6M16pn9I5Zrl1FQyjNgWNMi927vSKe9Y5RY/VM4PPojNXZIkFqg7TWHbqsmWrG9aUb468hxdw2BzkFAXhJhvNs5efZAN7CcSSzht2dsng0ndQND6RId1TyFehsQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Increase alignment of sections in nvhe hyp, vdso and final vmlinux image from PAGE_SIZE to PAGE_SIZE_MAX. For compile-time PAGE_SIZE, PAGE_SIZE_MAX == PAGE_SIZE so there is no change. For boot-time PAGE_SIZE, PAGE_SIZE_MAX is the largest selectable page size. For a boot-time page size build, image size is comparable to a 64K page size compile-time build. In future, it may be desirable to optimize run-time memory consumption by freeing unused padding pages when the boot-time selected page size is less than PAGE_SIZE_MAX. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/memory.h | 4 +-- arch/arm64/kernel/vdso-wrap.S | 4 +-- arch/arm64/kernel/vdso.c | 7 +++--- arch/arm64/kernel/vdso/vdso.lds.S | 4 +-- arch/arm64/kernel/vdso32-wrap.S | 4 +-- arch/arm64/kernel/vdso32/vdso.lds.S | 4 +-- arch/arm64/kernel/vmlinux.lds.S | 38 ++++++++++++++--------------- arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 2 +- 8 files changed, 34 insertions(+), 33 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 6aa97fa22dc30..5393a859183f7 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -195,13 +195,13 @@ * Open-coded (swapper_pg_dir - reserved_pg_dir) as this cannot be calculated * until link time. */ -#define RESERVED_SWAPPER_OFFSET (PAGE_SIZE) +#define RESERVED_SWAPPER_OFFSET (PAGE_SIZE_MAX) /* * Open-coded (swapper_pg_dir - tramp_pg_dir) as this cannot be calculated * until link time. */ -#define TRAMP_SWAPPER_OFFSET (2 * PAGE_SIZE) +#define TRAMP_SWAPPER_OFFSET (2 * PAGE_SIZE_MAX) #ifndef __ASSEMBLY__ diff --git a/arch/arm64/kernel/vdso-wrap.S b/arch/arm64/kernel/vdso-wrap.S index c4b1990bf2be0..79fa77628199b 100644 --- a/arch/arm64/kernel/vdso-wrap.S +++ b/arch/arm64/kernel/vdso-wrap.S @@ -13,10 +13,10 @@ .globl vdso_start, vdso_end .section .rodata - .balign PAGE_SIZE + .balign PAGE_SIZE_MAX vdso_start: .incbin "arch/arm64/kernel/vdso/vdso.so" - .balign PAGE_SIZE + .balign PAGE_SIZE_MAX vdso_end: .previous diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c index 89b6e78400023..1efe98909a2e0 100644 --- a/arch/arm64/kernel/vdso.c +++ b/arch/arm64/kernel/vdso.c @@ -195,7 +195,7 @@ static int __setup_additional_pages(enum vdso_abi abi, vdso_text_len = vdso_info[abi].vdso_pages << PAGE_SHIFT; /* Be sure to map the data page */ - vdso_mapping_len = vdso_text_len + VVAR_NR_PAGES * PAGE_SIZE; + vdso_mapping_len = vdso_text_len + VVAR_NR_PAGES * PAGE_SIZE_MAX; vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0); if (IS_ERR_VALUE(vdso_base)) { @@ -203,7 +203,8 @@ static int __setup_additional_pages(enum vdso_abi abi, goto up_fail; } - ret = _install_special_mapping(mm, vdso_base, VVAR_NR_PAGES * PAGE_SIZE, + ret = _install_special_mapping(mm, vdso_base, + VVAR_NR_PAGES * PAGE_SIZE_MAX, VM_READ|VM_MAYREAD|VM_PFNMAP, vdso_info[abi].dm); if (IS_ERR(ret)) @@ -212,7 +213,7 @@ static int __setup_additional_pages(enum vdso_abi abi, if (system_supports_bti_kernel()) gp_flags = VM_ARM64_BTI; - vdso_base += VVAR_NR_PAGES * PAGE_SIZE; + vdso_base += VVAR_NR_PAGES * PAGE_SIZE_MAX; mm->context.vdso = (void *)vdso_base; ret = _install_special_mapping(mm, vdso_base, vdso_text_len, VM_READ|VM_EXEC|gp_flags| diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S index 45354f2ddf706..f7d1537a689e8 100644 --- a/arch/arm64/kernel/vdso/vdso.lds.S +++ b/arch/arm64/kernel/vdso/vdso.lds.S @@ -18,9 +18,9 @@ OUTPUT_ARCH(aarch64) SECTIONS { - PROVIDE(_vdso_data = . - __VVAR_PAGES * PAGE_SIZE); + PROVIDE(_vdso_data = . - __VVAR_PAGES * PAGE_SIZE_MAX); #ifdef CONFIG_TIME_NS - PROVIDE(_timens_data = _vdso_data + PAGE_SIZE); + PROVIDE(_timens_data = _vdso_data + PAGE_SIZE_MAX); #endif . = VDSO_LBASE + SIZEOF_HEADERS; diff --git a/arch/arm64/kernel/vdso32-wrap.S b/arch/arm64/kernel/vdso32-wrap.S index e72ac7bc4c04f..1c6069d6c457e 100644 --- a/arch/arm64/kernel/vdso32-wrap.S +++ b/arch/arm64/kernel/vdso32-wrap.S @@ -10,10 +10,10 @@ .globl vdso32_start, vdso32_end .section .rodata - .balign PAGE_SIZE + .balign PAGE_SIZE_MAX vdso32_start: .incbin "arch/arm64/kernel/vdso32/vdso.so" - .balign PAGE_SIZE + .balign PAGE_SIZE_MAX vdso32_end: .previous diff --git a/arch/arm64/kernel/vdso32/vdso.lds.S b/arch/arm64/kernel/vdso32/vdso.lds.S index 8d95d7d35057d..c46d18a69d1ce 100644 --- a/arch/arm64/kernel/vdso32/vdso.lds.S +++ b/arch/arm64/kernel/vdso32/vdso.lds.S @@ -18,9 +18,9 @@ OUTPUT_ARCH(arm) SECTIONS { - PROVIDE_HIDDEN(_vdso_data = . - __VVAR_PAGES * PAGE_SIZE); + PROVIDE_HIDDEN(_vdso_data = . - __VVAR_PAGES * PAGE_SIZE_MAX); #ifdef CONFIG_TIME_NS - PROVIDE_HIDDEN(_timens_data = _vdso_data + PAGE_SIZE); + PROVIDE_HIDDEN(_timens_data = _vdso_data + PAGE_SIZE_MAX); #endif . = VDSO_LBASE + SIZEOF_HEADERS; diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 7f3f6d709ae73..1ef6dea13b57c 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -15,16 +15,16 @@ #define HYPERVISOR_DATA_SECTIONS \ HYP_SECTION_NAME(.rodata) : { \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ __hyp_rodata_start = .; \ *(HYP_SECTION_NAME(.data..ro_after_init)) \ *(HYP_SECTION_NAME(.rodata)) \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ __hyp_rodata_end = .; \ } #define HYPERVISOR_PERCPU_SECTION \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ HYP_SECTION_NAME(.data..percpu) : { \ *(HYP_SECTION_NAME(.data..percpu)) \ } @@ -39,7 +39,7 @@ #define BSS_FIRST_SECTIONS \ __hyp_bss_start = .; \ *(HYP_SECTION_NAME(.bss)) \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ __hyp_bss_end = .; /* @@ -48,7 +48,7 @@ * between them, which can in some cases cause the linker to misalign them. To * work around the issue, force a page alignment for __bss_start. */ -#define SBSS_ALIGN PAGE_SIZE +#define SBSS_ALIGN PAGE_SIZE_MAX #else /* CONFIG_KVM */ #define HYPERVISOR_EXTABLE #define HYPERVISOR_DATA_SECTIONS @@ -75,14 +75,14 @@ ENTRY(_text) jiffies = jiffies_64; #define HYPERVISOR_TEXT \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ __hyp_idmap_text_start = .; \ *(.hyp.idmap.text) \ __hyp_idmap_text_end = .; \ __hyp_text_start = .; \ *(.hyp.text) \ HYPERVISOR_EXTABLE \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ __hyp_text_end = .; #define IDMAP_TEXT \ @@ -113,11 +113,11 @@ jiffies = jiffies_64; #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 #define TRAMP_TEXT \ - . = ALIGN(PAGE_SIZE); \ + . = ALIGN(PAGE_SIZE_MAX); \ __entry_tramp_text_start = .; \ *(.entry.tramp.text) \ - . = ALIGN(PAGE_SIZE); \ __entry_tramp_text_end = .; \ + . = ALIGN(PAGE_SIZE_MAX); \ *(.entry.tramp.rodata) #else #define TRAMP_TEXT @@ -187,7 +187,7 @@ SECTIONS _etext = .; /* End of text section */ /* everything from this point to __init_begin will be marked RO NX */ - RO_DATA(PAGE_SIZE) + RO_DATA(PAGE_SIZE_MAX) HYPERVISOR_DATA_SECTIONS @@ -206,22 +206,22 @@ SECTIONS HIBERNATE_TEXT KEXEC_TEXT IDMAP_TEXT - . = ALIGN(PAGE_SIZE); + . = ALIGN(PAGE_SIZE_MAX); } idmap_pg_dir = .; - . += PAGE_SIZE; + . += PAGE_SIZE_MAX; #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 tramp_pg_dir = .; - . += PAGE_SIZE; + . += PAGE_SIZE_MAX; #endif reserved_pg_dir = .; - . += PAGE_SIZE; + . += PAGE_SIZE_MAX; swapper_pg_dir = .; - . += PAGE_SIZE; + . += PAGE_SIZE_MAX; . = ALIGN(SEGMENT_ALIGN); __init_begin = .; @@ -290,7 +290,7 @@ SECTIONS _data = .; _sdata = .; - RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_ALIGN) + RW_DATA(L1_CACHE_BYTES, PAGE_SIZE_MAX, THREAD_ALIGN) /* * Data written with the MMU off but read with the MMU on requires @@ -317,7 +317,7 @@ SECTIONS /* start of zero-init region */ BSS_SECTION(SBSS_ALIGN, 0, 0) - . = ALIGN(PAGE_SIZE); + . = ALIGN(PAGE_SIZE_MAX); init_pg_dir = .; . += INIT_DIR_SIZE_MAX; init_pg_end = .; @@ -356,7 +356,7 @@ SECTIONS * former is page-aligned, but the latter may not be with 16K or 64K pages, so * it should also not cross a page boundary. */ -ASSERT(__hyp_idmap_text_end - __hyp_idmap_text_start <= PAGE_SIZE, +ASSERT(__hyp_idmap_text_end - __hyp_idmap_text_start <= SZ_4K, "HYP init code too big") ASSERT(__idmap_text_end - (__idmap_text_start & ~(SZ_4K - 1)) <= SZ_4K, "ID map text too big or misaligned") @@ -367,7 +367,7 @@ ASSERT(__hibernate_exit_text_start == swsusp_arch_suspend_exit, "Hibernate exit text does not start with swsusp_arch_suspend_exit") #endif #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 -ASSERT((__entry_tramp_text_end - __entry_tramp_text_start) <= 3*PAGE_SIZE, +ASSERT((__entry_tramp_text_end - __entry_tramp_text_start) <= 3 * SZ_4K, "Entry trampoline text too big") #endif #ifdef CONFIG_KVM diff --git a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S index f4562f417d3fc..74c7c21626270 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S @@ -21,7 +21,7 @@ SECTIONS { * .hyp..data..percpu needs to be page aligned to maintain the same * alignment for when linking into vmlinux. */ - . = ALIGN(PAGE_SIZE); + . = ALIGN(PAGE_SIZE_MAX); BEGIN_HYP_SECTION(.data..percpu) PERCPU_INPUT(L1_CACHE_BYTES) END_HYP_SECTION From patchwork Mon Oct 14 10:58:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16139D1A45A for ; Mon, 14 Oct 2024 11:01:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 953CF6B00D8; Mon, 14 Oct 2024 07:01:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 902F76B00DA; Mon, 14 Oct 2024 07:01:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A20D6B00DB; Mon, 14 Oct 2024 07:01:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5951C6B00D8 for ; Mon, 14 Oct 2024 07:01:48 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 820D780D96 for ; Mon, 14 Oct 2024 11:01:41 +0000 (UTC) X-FDA: 82671917328.12.7078B93 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf07.hostedemail.com (Postfix) with ESMTP id B3DFA4001F for ; Mon, 14 Oct 2024 11:01:37 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903564; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OACUuu7EkK/bVx0oPSFpoAc1oPx/wt8iZOtMNIFSzjQ=; b=tRLhfdfIBPsahxvB08TXwQc95xbWz7bzem0/LOx+idZ8LRRNe3vn7fpYZgeAv4oytrxgp1 EgQR2+fljVUHVWrDiFp3C/3PLH/ctPkyWw7974+zLbISjrM6xbFtfVdjABteTS5nuQJA3q 0hGVmD2ARl2xzPWcfn9Qqf5Qcixjd0U= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903564; a=rsa-sha256; cv=none; b=jmUwi65Lqdkhs02Zru0GcLRQRKSF94IA+mc7xVsCmS9WCvToh/sAu6x8J6jWu6hR6qam+S Ypjw1J9IoYpkCgCWkADWPq0CWqeJCM95bOpYgvKKcpmzNqSPFA2z6JSNfENJnecgs/HOUq 3msDQKCgq7P9pKbUakyQ7Zh0eqRzR+I= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5E6BA1764; Mon, 14 Oct 2024 04:02:15 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5FBE13F51B; Mon, 14 Oct 2024 04:01:43 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 45/57] arm64: Rework trampoline rodata mapping Date: Mon, 14 Oct 2024 11:58:52 +0100 Message-ID: <20241014105912.3207374-45-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B3DFA4001F X-Stat-Signature: xsy6gzjuyr6at81qqdtqa7n9tdkyik7x X-HE-Tag: 1728903697-529767 X-HE-Meta: U2FsdGVkX1+8KHz8oIVN+rM1CspMuHHvnLANQup23QWEpxMK+zTzs6StcOSqi1TCfE2jbBgEz0k01dltOSw6ISlvaeSumE+K/ICPjOYENHO0nHYzO4S5LC5S6i8xBxJcup6ackRIHmpURIUxEh/UQA4gSdwQRYTvsNNBZ/iJWtH9zEG2eX4slUhF63KFR4x4ZXbx4m57sN8jc4VVBz4nlebrtIPqjmZwobd4DkhW55Eys3UmzyOhzqqBtLKYUhtB17BfVTNYyy+PNg+w5/0pRkmSMLvu1qTrWLrJk2YhHJsimhRFsUsTVDvaMtmPlbtF40AtjeHDDnb1xXYkj8p7nVcIYYCvzMBI2JQnPVwSYQa+WO5Nm2KgmqT0AOwoncPpCQEG05VRvd5OWa+ZumEatSih47I7EVMX80Ww+tqOmsFrIZ1VJXJJvPWxjwLMdrWuXFJs6roSvye2RC2m4gRV81aj5JEDzUmpiraz1ZQea3jO1z1koxfcgwELtU33sWVGH3OvpG2bUNhZqzoBwDyxOnEfYvDlt4/b3SgCq991Cs4K6yXgkVnC403e4IFMnGztYo3m5N+a3Nlkndhj0UC8hbe+ucIhQGihQM40m7q8u9HBRBzwq6JhZWXDRAQGCkpbnvPXapW9HElTAaPNRO7PeaDDxN4513sTSqzIfpzG3wxQFCUfn40xUaq7x/5PXAxXXR7XSBMLN1kb/34B4lswepK8GW0lp+ZrQ0MRvS7MppPiWOnJHnfv8YDL4i/oDa9UC7hzJknMc1xKKGMyYV80vEzoe7scH/ZrS/FjTL+zfzsown0vo/9Egbg3TXoiLlPuJbx3M7y6ukfYLjUFGufMs15mCJI1hw8RQfH0u7T+sxJTwBOf6yqcc71Fj95iD/eAw98i2/4YExMB0KVAP2NcYV+A2QUPAOEJc4t3BAmX+z6dUCavRruxnNiEYnWhs2/wGh7KzxFJB4l8OBWtTiB ltgE9Wio Vsqy6n5X1U33cHsKBwvQCQJgok2hF8ItvDAtV92BdU7JpIcAZmTur7kqIfpLViIYhMUziSsDJmbe3S8nO2ABKoiRfqox3L3xLDZnj3Ze/kUbOjXoGMHDTVWWOvXSKzquW3R4JwL5+6CxdzVTNJ7Sjm0TCuRefzQ9bS2gAPlaCGny4cz3GtF6RzEJHQA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Now that the trampoline rodata is aligned to the next PAGE_SIZE_MAX boundary after the end of the trampoline text, the code that maps it in the fixmap is incorrect, because it still assumes the rodata is in the next page immediately after the text. Of course it still works for now with compile-time page size but for boot-time page size when selecting a page size less than PAGE_SIZE_MAX, it will fail. So let's fix that by allocating sufficient fixmap slots to cover the extra alignment padding in the worst case (PAGE_SIZE == PAGE_SIZE_MIN) and explicitly mapping the rodata to the slot offset correctly from the text. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/fixmap.h | 16 +++++++++++----- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/vmlinux.lds.S | 4 +++- arch/arm64/mm/mmu.c | 22 ++++++++++++++-------- 4 files changed, 29 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixmap.h index 87e307804b99c..9a496d54dfe6e 100644 --- a/arch/arm64/include/asm/fixmap.h +++ b/arch/arm64/include/asm/fixmap.h @@ -59,13 +59,19 @@ enum fixed_addresses { #endif /* CONFIG_ACPI_APEI_GHES */ #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +#define TRAMP_TEXT_SIZE (PAGE_SIZE_MIN * 3) #ifdef CONFIG_RELOCATABLE - FIX_ENTRY_TRAMP_TEXT4, /* one extra slot for the data page */ +#define TRAMP_DATA_SIZE PAGE_SIZE_MIN +#define TRAMP_PAD_SIZE (PAGE_SIZE_MAX - PAGE_SIZE_MIN) +#else +#define TRAMP_DATA_SIZE 0 +#define TRAMP_PAD_SIZE 0 #endif - FIX_ENTRY_TRAMP_TEXT3, - FIX_ENTRY_TRAMP_TEXT2, - FIX_ENTRY_TRAMP_TEXT1, -#define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_TEXT1)) +#define TRAMP_SIZE (TRAMP_TEXT_SIZE + TRAMP_DATA_SIZE + TRAMP_PAD_SIZE) + FIX_ENTRY_TRAMP_END, + FIX_ENTRY_TRAMP_BEGIN = FIX_ENTRY_TRAMP_END + + DIV_ROUND_UP(TRAMP_SIZE, PAGE_SIZE_MIN) - 1, +#define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_BEGIN)) #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ __end_of_permanent_fixed_addresses, diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 40971ac1303f9..252ec58963093 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -21,6 +21,7 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __entry_tramp_rodata_start[], __entry_tramp_rodata_end[]; extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; static inline size_t entry_tramp_text_size(void) diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 1ef6dea13b57c..09fcc234c0f77 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -118,7 +118,9 @@ jiffies = jiffies_64; *(.entry.tramp.text) \ __entry_tramp_text_end = .; \ . = ALIGN(PAGE_SIZE_MAX); \ - *(.entry.tramp.rodata) + __entry_tramp_rodata_start = .; \ + *(.entry.tramp.rodata) \ + __entry_tramp_rodata_end = .; #else #define TRAMP_TEXT #endif diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index a528787c1e550..84df9f278d24d 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -734,25 +734,31 @@ static int __init map_entry_trampoline(void) return 0; pgprot_t prot = kernel_exec_prot(); - phys_addr_t pa_start = __pa_symbol(__entry_tramp_text_start); + phys_addr_t pa_text = __pa_symbol(__entry_tramp_text_start); + phys_addr_t pa_data = __pa_symbol(__entry_tramp_rodata_start); + int slot = FIX_ENTRY_TRAMP_BEGIN; /* The trampoline is always mapped and can therefore be global */ pgprot_val(prot) &= ~PTE_NG; /* Map only the text into the trampoline page table */ memset(tramp_pg_dir, 0, PGD_SIZE); - __create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS, + __create_pgd_mapping(tramp_pg_dir, pa_text, TRAMP_VALIAS, entry_tramp_text_size(), prot, __pgd_pgtable_alloc, NO_BLOCK_MAPPINGS); /* Map both the text and data into the kernel page table */ - for (i = 0; i < DIV_ROUND_UP(entry_tramp_text_size(), PAGE_SIZE); i++) - __set_fixmap(FIX_ENTRY_TRAMP_TEXT1 - i, - pa_start + i * PAGE_SIZE, prot); + for (i = 0; i < DIV_ROUND_UP(entry_tramp_text_size(), PAGE_SIZE); i++) { + __set_fixmap(slot, pa_text, prot); + pa_text += PAGE_SIZE; + slot--; + } - if (IS_ENABLED(CONFIG_RELOCATABLE)) - __set_fixmap(FIX_ENTRY_TRAMP_TEXT1 - i, - pa_start + i * PAGE_SIZE, PAGE_KERNEL_RO); + if (IS_ENABLED(CONFIG_RELOCATABLE)) { + slot -= (pa_data - pa_text) / PAGE_SIZE; + VM_BUG_ON(slot < FIX_ENTRY_TRAMP_END); + __set_fixmap(slot, pa_data, PAGE_KERNEL_RO); + } return 0; } From patchwork Mon Oct 14 10:58:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834714 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 277BED1A43B for ; Mon, 14 Oct 2024 11:01:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4252B6B00DA; Mon, 14 Oct 2024 07:01:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 30D736B00DC; Mon, 14 Oct 2024 07:01:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1AF996B00DD; Mon, 14 Oct 2024 07:01:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id EC1766B00DA for ; Mon, 14 Oct 2024 07:01:50 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5A92D160D40 for ; Mon, 14 Oct 2024 11:01:42 +0000 (UTC) X-FDA: 82671917370.13.D3565FF Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf13.hostedemail.com (Postfix) with ESMTP id A51D320003 for ; Mon, 14 Oct 2024 11:01:42 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903637; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Pem7/e2IY8K92Dd4GqzgmCKk29IF+hVKC5zPWZhX1eg=; b=XFVPCjj/4CrU6zui5wfQ5CaGdXwKF6NdPljkq/zi7KmUmSnjkG33kWm0AwuVTH6uVBaRNy brSOzCGz2Nptut77BgKXqazVKU8zNpKadwVBWOFxyRjTJ8LR/xjbXORzJG13DJ78RgVTUj b+HPGfiBaIVeUCWvhdrJ2vvG7Ct8XlI= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903637; a=rsa-sha256; cv=none; b=hnU/PYU8xPbW8RLU5SfRXeG/ZN4UPWkguM33xAod7EVMiom4TI57mYfVpnrLorp6lz0vav 5p+gaQt3voL9eNZcBgQ52TWqe1dXwi76zwVARf/ufTMb2TXNjWshOL4QSvbCYbW/CEW7su mPWeaDCgAJFmCHJ85cTQobjkpaOYAuY= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0E2C6176A; Mon, 14 Oct 2024 04:02:18 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0FACB3F51B; Mon, 14 Oct 2024 04:01:45 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 46/57] arm64: Generalize fixmap for boot-time page size Date: Mon, 14 Oct 2024 11:58:53 +0100 Message-ID: <20241014105912.3207374-46-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: A51D320003 X-Stat-Signature: oqehapmzwqq9hdgqec5tp6yp3ez6cxfh X-HE-Tag: 1728903702-38123 X-HE-Meta: U2FsdGVkX19i7S2Fz0g0n2OvXaxdCm4hSRbKaBqz8SxJuvxs6ZW0KDs0AihbaVzn2d5gPYU4FLpERz1o//Vn2pfR/KZRXri4wMkuwI7ViyKClQXA+RpFSH2MQNoWYrROf3HGfq4skGCFqqDgs+kCgUPjoHzm1hftq7SxJ4L15Yd1uj+lEsCrGsPbdfhmS7fwsKrVfACBfct4+03JlQ7YcxRK6y8rERgCesqs3SE0bWdQih6K5thpQ/XT79LeJVchKhX/qUAjhL/hV4pkD8oVuVSmH+EPoVbkudVHAJrRA/MvNiyBEy49cblk7nu3I5V00ngwhJckws2q0HEan6n2748Y6qIAwC/a77enXE+8eYGGSav9AJFjJBceUHoTKo66kn6aCUv/Tr0syoxpYN7uK7RtALaYpar/B1mzgE52HdqYTj1HaHw5hf8T/jmM3z85sZqmM8uNPzu6y4MnsoFWRy4XFkwNPBe2w2gcEdRfIereJQWZWfMCwQTnesVuf3DFXr8LxbB3sHyJxmB61YnSuWFTi9P8aii50qAwHl2MInrSLDeBp2a5A+iUqjk4xUZTK2t6vxDRHs/HU1yKLBSx9DwKFjG0agMGlBcZT5yWm4HOeV0dJfaGmJfvCc+3Pwy8rzaqibCg77GmYGBi7Wq08QvmWQnjlVFTawxWisCe0GoPCJhjXB/7D3/6vVcPHdWZOjdL4S6ytVjLMbwYc/Q3dYtOBTZfQKrX3hmXLHGpr+ILdbyXQRVvIE3EhfWQnexIuzR5nYSQi2PIktiTSXbZfWraGXeFR2HB63vrWJ/wf8l5A5C1v3sWm9uQcjZNnXIWXL387ZFM7/mIupRxmoh2ddYzGekfmcK4gc4Iut8LxtX2tBsPp9/vuEey3/eTpkqX7q67+GweiRmwJJ2z95MqnS8vwkjFASSsL0iJslWx8GEMMlNT9bnIl2i1SrrLqYj7FEh3WJugTPvbYY/kMgb 0XFCq+rt 8J/NPMmer5aKMLF8JtDS4lsXAPuYGOxlhLkkyAMiXTwdNrmmznYno7UtJapi/QmeWmTuRSZbc0lTA3BFmf8/tV7mZ4uTn6k82IFCdBX3QJdxiLRP9TEzlseg2rJssUrtxGVerLk/HTLBXUleNLsxMvexOeC9gIpPkQRK2FdWM5eIqq9hZGmCsg5Grng== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Some fixmap fixed address slots previously depended on PAGE_SIZE (i.e. to determine how many slots were required to cover a given size). Since we require the fixed address slots to be compile-time constant, let's work out the worst case number of required slots when page size is PAGE_SIZE_MIN instead. Additionally, let's determine the worst-case number of PTE tables we require and statically allocate enough memory. For compile-time page size builds, the end result is the same as it was previously. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/fixmap.h | 12 ++++++++---- arch/arm64/mm/fixmap.c | 34 ++++++++++++++++++++++----------- 2 files changed, 31 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixmap.h index 9a496d54dfe6e..c73fd3c1334ff 100644 --- a/arch/arm64/include/asm/fixmap.h +++ b/arch/arm64/include/asm/fixmap.h @@ -43,7 +43,7 @@ enum fixed_addresses { * whether it crosses any page boundary. */ FIX_FDT_END, - FIX_FDT = FIX_FDT_END + DIV_ROUND_UP(MAX_FDT_SIZE, PAGE_SIZE) + 1, + FIX_FDT = FIX_FDT_END + DIV_ROUND_UP(MAX_FDT_SIZE, PAGE_SIZE_MIN) + 1, FIX_EARLYCON_MEM_BASE, FIX_TEXT_POKE0, @@ -79,7 +79,7 @@ enum fixed_addresses { * Temporary boot-time mappings, used by early_ioremap(), * before ioremap() is functional. */ -#define NR_FIX_BTMAPS (SZ_256K / PAGE_SIZE) +#define NR_FIX_BTMAPS (SZ_256K / PAGE_SIZE_MIN) #define FIX_BTMAPS_SLOTS 7 #define TOTAL_FIX_BTMAPS (NR_FIX_BTMAPS * FIX_BTMAPS_SLOTS) @@ -101,8 +101,12 @@ enum fixed_addresses { #define FIXADDR_SIZE (__end_of_permanent_fixed_addresses << PAGE_SHIFT) #define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE) -#define FIXADDR_TOT_SIZE (__end_of_fixed_addresses << PAGE_SHIFT) -#define FIXADDR_TOT_START (FIXADDR_TOP - FIXADDR_TOT_SIZE) +#define __FIXADDR_TOT_SIZE(page_shift) \ + (__end_of_fixed_addresses << (page_shift)) +#define __FIXADDR_TOT_START(page_shift) \ + (FIXADDR_TOP - __FIXADDR_TOT_SIZE(page_shift)) +#define FIXADDR_TOT_SIZE __FIXADDR_TOT_SIZE(PAGE_SHIFT) +#define FIXADDR_TOT_START __FIXADDR_TOT_START(PAGE_SHIFT) #define FIXMAP_PAGE_IO __pgprot(PROT_DEVICE_nGnRE) diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index 15ce3253ad359..a0dcf2375ccb4 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -17,27 +17,39 @@ #include /* ensure that the fixmap region does not grow down into the PCI I/O region */ -static_assert(FIXADDR_TOT_START > PCI_IO_END); +static_assert(__FIXADDR_TOT_START(PAGE_SHIFT_MAX) > PCI_IO_END); -#define NR_BM_PTE_TABLES \ - SPAN_NR_ENTRIES(FIXADDR_TOT_START, FIXADDR_TOP, PMD_SHIFT) -#define NR_BM_PMD_TABLES \ - SPAN_NR_ENTRIES(FIXADDR_TOT_START, FIXADDR_TOP, PUD_SHIFT) +#define FIXMAP_LEVEL(page_shift, lvl, vstart, vend) \ + SPAN_NR_ENTRIES(vstart, vend, PGTABLE_LEVEL_SHIFT(page_shift, lvl)) -static_assert(NR_BM_PMD_TABLES == 1); +#define FIXMAP_PAGES(page_shift, level) \ + FIXMAP_LEVEL(page_shift, level, \ + __FIXADDR_TOT_START(page_shift), FIXADDR_TOP) + +#define FIXMAP_SIZE(page_shift, level) \ + (FIXMAP_PAGES(page_shift, level) * (UL(1) << (page_shift))) + +#define FIXMAP_PTE_SIZE_MAX \ + MAX_IF_HAVE_PGSZ(FIXMAP_SIZE(ARM64_PAGE_SHIFT_4K, 2), \ + FIXMAP_SIZE(ARM64_PAGE_SHIFT_16K, 2), \ + FIXMAP_SIZE(ARM64_PAGE_SHIFT_64K, 2)) + +static_assert(FIXMAP_PAGES(ARM64_PAGE_SHIFT_4K, 1) == 1); +static_assert(FIXMAP_PAGES(ARM64_PAGE_SHIFT_16K, 1) == 1); +static_assert(FIXMAP_PAGES(ARM64_PAGE_SHIFT_64K, 1) == 1); #define __BM_TABLE_IDX(addr, shift) \ (((addr) >> (shift)) - (FIXADDR_TOT_START >> (shift))) #define BM_PTE_TABLE_IDX(addr) __BM_TABLE_IDX(addr, PMD_SHIFT) -static pte_t bm_pte[NR_BM_PTE_TABLES][PTRS_PER_PTE] __page_aligned_bss; -static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss __maybe_unused; -static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss __maybe_unused; +static pte_t bm_pte[FIXMAP_PTE_SIZE_MAX / sizeof(pte_t)] __page_aligned_bss; +static pmd_t bm_pmd[MAX_PTRS_PER_PMD] __page_aligned_bss __maybe_unused; +static pud_t bm_pud[MAX_PTRS_PER_PUD] __page_aligned_bss __maybe_unused; static inline pte_t *fixmap_pte(unsigned long addr) { - return &bm_pte[BM_PTE_TABLE_IDX(addr)][pte_index(addr)]; + return &bm_pte[BM_PTE_TABLE_IDX(addr) * PTRS_PER_PTE + pte_index(addr)]; } static void __init early_fixmap_init_pte(pmd_t *pmdp, unsigned long addr) @@ -46,7 +58,7 @@ static void __init early_fixmap_init_pte(pmd_t *pmdp, unsigned long addr) pte_t *ptep; if (pmd_none(pmd)) { - ptep = bm_pte[BM_PTE_TABLE_IDX(addr)]; + ptep = &bm_pte[BM_PTE_TABLE_IDX(addr) * PTRS_PER_PTE]; __pmd_populate(pmdp, __pa_symbol(ptep), PMD_TYPE_TABLE); } } From patchwork Mon Oct 14 10:58:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49F6CD1A443 for ; Mon, 14 Oct 2024 11:01:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 07D3E6B00DC; Mon, 14 Oct 2024 07:01:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E7F456B00DE; Mon, 14 Oct 2024 07:01:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D1FBF6B00DF; Mon, 14 Oct 2024 07:01:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B0A5B6B00DC for ; Mon, 14 Oct 2024 07:01:53 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 9A4AC40D90 for ; Mon, 14 Oct 2024 11:01:48 +0000 (UTC) X-FDA: 82671917370.17.6171290 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP id D60C14001B for ; Mon, 14 Oct 2024 11:01:46 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903680; a=rsa-sha256; cv=none; b=NvTw7MQTeeFdstKfPtIwyWBiNtxp54VE57zQ/SlVNrJNNPpbrDo4yUvt2T2gaVTUTuFCUs JNPKMhU7bOQ7Ia//emYVxPwLkBVSOKg3rxSZfTo5WYwRzzdJvhTN44PQub4tHO6xwC63EY WUTkJyWG5fMYDrOautJpvBqwlKlQpOE= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf17.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903680; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YPpXHP2mLKgaz5VTCYQnV2Bzp3U6nRyN8VK8UrnJsws=; b=kYN4p4Z5DqvJ7j7Q7zqcd51SML1KIO361fUsUjPYSWM095ufzKMdjr1keaKaSxzgRBZmPZ BwxcT6P3lPzD1kxID1jPkzRrxoodIfDg22BQfEgIivPqeNi+TExsOjU687owdj+GXz+w3L twVfEHtu+dg1bN39TsKS3+0i4fxJTS0= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B1DC6176B; Mon, 14 Oct 2024 04:02:20 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B31C53F51B; Mon, 14 Oct 2024 04:01:48 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 47/57] arm64: Statically allocate and align for worst-case page size Date: Mon, 14 Oct 2024 11:58:54 +0100 Message-ID: <20241014105912.3207374-47-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: joqenu748e6bw8gjf6mgh554p7cbnh1a X-Rspamd-Queue-Id: D60C14001B X-Rspamd-Server: rspam02 X-HE-Tag: 1728903706-777284 X-HE-Meta: U2FsdGVkX1/siQjSLk/TcW34l2s1KC3s3zUfmhvN/2XN0MWFJpvtuD+ayEdpnEAnqP3mpSa0SctWWy3/Y2bYWY01u1sEtgRRkts4nLJgjz1U6xNC3M63FSUVrx78sZIDFbmJhZAuz6ozp5X5NHBNuMjOeA4fMzXKLdUnrhSF0LW2ahJpfi4KLtjDgTFoTOVFEX4xcNEiepnjIILCOhwLjvQ/QQUaixR2i4BA09UzPQKfkMZXI4+mqELK6yJXp+qOufMOGuOuHb7irWTpo8j/e9B48T34NAlJQN9ZSsVwb35M73lumRYpMZynaPQTdb/05EY90DDaSQaRTI8Fy4w2XltwRjmUp5tPyUuCLoFmNVXY7BSLBO7ZRnQWu24y+iiEOYXdTUrw6eTcU/ID0NWvt+TGx87kguPwvJx4wYXZcVH8YsV6wTMV/Zpp5UZ8mk8wqBjJUu+bpRtT5iekedwzQFnt1R0dgdqh+GdjzAig8yebkTc+DL8MokzE2mD0i22QaL9sxmsPC0D49bM/tTgH+FdP/rFHNL+mFl/RjonMrbTi0M/kNN5j5bhBMkUoaloNBPX+6TiPvxywcuNViUzJzrYlPAkVnaxC8uJ5kDrrHwb8MzUp1mGlfDWAd0SOy3we7qFkco2UyPdlGZumYbQSeK8sO8+1wko4VpjArj0o/FHOKxxsGfRSWym4e74R3nJyVlHSRs5bWj2uejdMoNSrMy1wuWH8x0Jzf1UQpD4YpIkeFA+JkO8CmkvlqDYjQ72M8dg/3a2/UvBMZlzSODZyqjN5osn6vrpEA07dJnJ/bf3W3M8dTblHIfxHEFJj/uiUmvtOt/WczBxC60SY/RPO915ko/A32p2r8NMKUEl52PzkKA+VzHbmNjjiP7+ELYeZomzFjlsmw9bteYCa+o6syJeprdOs/b5dm27lktLXjqo7JhwjTTgize42JEGpZaRhxadFr7i013i+JIvI0b7 M1phto1x YQ6iWqOMrLC33P5k+PhnZHY86vibeivUJrhmdLKWFIGjvNL1WpbHEnHmLfovXqEv5mDmg7RqMzodr8/T4gGcyayLPGu+37JxQ9gC1+vUy3LF6y+2uv5MOGpcGdjNxoQUUOgeP4dgONtBlCj8Sx2cDSU8Rnw76xQOiyZ+FCl3zbtI4mZtjSBNiF9WTKg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Increase the size and alignment of the zero page and various static buffers used for page tables to PAGE_SIZE_MAX. This resolves to the same thing for compile-time page size builds. For boot-time builds, we may in future consider freeing unused pages at runtime when the selected page size is less than MAX. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/pgtable.h | 2 +- arch/arm64/kernel/pi/map_kernel.c | 2 +- arch/arm64/mm/mmu.c | 6 +++--- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 7a4f5604be3f7..fd47f70a42396 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -61,7 +61,7 @@ * ZERO_PAGE is a global shared page that is always zero: used * for zero-mapped memory areas etc.. */ -extern unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)]; +extern unsigned long empty_zero_page[PAGE_SIZE_MAX / sizeof(unsigned long)]; #define ZERO_PAGE(vaddr) phys_to_page(__pa_symbol(empty_zero_page)) #define pte_ERROR(e) \ diff --git a/arch/arm64/kernel/pi/map_kernel.c b/arch/arm64/kernel/pi/map_kernel.c index 7a62d4238449d..deb8cd50b0b0c 100644 --- a/arch/arm64/kernel/pi/map_kernel.c +++ b/arch/arm64/kernel/pi/map_kernel.c @@ -199,7 +199,7 @@ static void __init remap_idmap(bool use_lpa2, int page_shift) static void __init map_fdt(u64 fdt, int page_shift) { - static u8 ptes[INIT_IDMAP_FDT_SIZE_MAX] __initdata __aligned(PAGE_SIZE); + static u8 ptes[INIT_IDMAP_FDT_SIZE_MAX] __initdata __aligned(PAGE_SIZE_MAX); static bool first_time __initdata = true; u64 limit = (u64)&ptes[INIT_IDMAP_FDT_SIZE_MAX]; u64 efdt = fdt + MAX_FDT_SIZE; diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 84df9f278d24d..b4cd3b6a73c22 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -62,7 +62,7 @@ long __section(".mmuoff.data.write") __early_cpu_boot_status; * Empty_zero_page is a special page that is used for zero-initialized data * and COW. */ -unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)] __page_aligned_bss; +unsigned long empty_zero_page[PAGE_SIZE_MAX / sizeof(unsigned long)] __page_aligned_bss; EXPORT_SYMBOL(empty_zero_page); static DEFINE_SPINLOCK(swapper_pgdir_lock); @@ -783,8 +783,8 @@ void __pi_map_range(u64 *pgd, u64 limit, u64 start, u64 end, u64 pa, pgprot_t prot, int level, pte_t *tbl, bool may_use_cont, u64 va_offset); -static u8 idmap_ptes[IDMAP_LEVELS_MAX - 1][PAGE_SIZE] __aligned(PAGE_SIZE) __ro_after_init, - kpti_ptes[IDMAP_LEVELS_MAX - 1][PAGE_SIZE] __aligned(PAGE_SIZE) __ro_after_init; +static u8 idmap_ptes[IDMAP_LEVELS_MAX - 1][PAGE_SIZE_MAX] __aligned(PAGE_SIZE_MAX) __ro_after_init, + kpti_ptes[IDMAP_LEVELS_MAX - 1][PAGE_SIZE_MAX] __aligned(PAGE_SIZE_MAX) __ro_after_init; static void __init create_idmap(void) { From patchwork Mon Oct 14 10:58:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834716 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C7AED1A43B for ; Mon, 14 Oct 2024 11:01:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CD4CC6B00DE; Mon, 14 Oct 2024 07:01:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C5C476B00E0; Mon, 14 Oct 2024 07:01:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AD6706B00E1; Mon, 14 Oct 2024 07:01:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 8D8E06B00DE for ; Mon, 14 Oct 2024 07:01:56 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 615A7120D7C for ; Mon, 14 Oct 2024 11:01:49 +0000 (UTC) X-FDA: 82671917706.18.966F78A Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf02.hostedemail.com (Postfix) with ESMTP id 2A4A68001B for ; Mon, 14 Oct 2024 11:01:42 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903642; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1Pv7dW4G5U3czwTGJgTczogiNPWDVKQoCd/L9alqD4s=; b=zIIZEUPu/UghhH5hucKYfyW/ezQ/12o6x2q/AECDEqVTf2hCOCqnF3t/7Rh3FK1K/J2EIS GlAG91W4c/nlSCDfTPJ9xSwjAQsuL87UTUhESBjwFN5oyPgCqASZulQ+JAKrDbkWP7en/F JZT7lBnsc3dtIy0E85Kqkhh8GXK/dQc= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903642; a=rsa-sha256; cv=none; b=ymswIXlRFCAnmomwsao3GNc3asoGh+EsNLvOeQy8fVPyqlETJBEJP/vfWbDmeFAdKzbTri zsZsnnZ8Vh7ToUIomAMQXQ7bkKvN8q63v3I+MjHFoLe4hrfPIgZrlwTRxlhNoyllD3VnBV clqvso+WkBg4PMhZAU6UA++eM4ddHzg= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A0064176C; Mon, 14 Oct 2024 04:02:23 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 62FEC3F51B; Mon, 14 Oct 2024 04:01:51 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Oliver Upton , Will Deacon Cc: Ryan Roberts , kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 48/57] arm64: Convert switch to if for non-const comparison values Date: Mon, 14 Oct 2024 11:58:55 +0100 Message-ID: <20241014105912.3207374-48-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 2A4A68001B X-Stat-Signature: yhdnfw79jb3fbrntgqwyyu1jkwuhc6ff X-HE-Tag: 1728903702-123624 X-HE-Meta: U2FsdGVkX18xomW7F3aCYxHyGdeXRyrl5hhiaAdHDAHikBJMHvvZhCFFZPtljj9yIdrSkKrv/Kj4EuwRZ6m7i3Ci9AkwWhpMYQFhHEU1KraTC5kitLNkyUr66yzHQ94xOCku3JCwia40lsKp/Qz2ALulDpAaYIw3tCnp7cELvHaUnfNZqKNJKj718LMPEPW1LzHAxz7/qMzdTFYFLqTsFZ04XL92/FU3MuaMywga069Ug8ogllRp2sHnNWLErdFI394Q12qAMy1boJLwXJLPKHfbdWCEMZBJVXGLRebVR0p7t4zfELm7llCXJY4Yert3vwm7rv6swkDADydq/qjY1vSzUFBIvujXtXdbVQbv5QGJNyUrjJy239JzayGxHEEajeMCx/E6+pMTcZ34sqPAe0ZvOYU0K9/BYRHt7EHMkOluk/b7Yco4ug7Z3V6XOMeW+R6OvUpKQBwepaJo5TETDrT7IvaCWWLIOLGhWyCMIWMZ4UI8NbNdzm9g5cau0Qd6Ap1yhL8m4gGBGSpk0cUXR/tRTiWrWMt0Nizku9agW44zlCqdR/qpFKzmqA1x7eddZIfRdNb8Ah5Xkw96IjwdAnoDmsxtFo+GrjerUf51PC+cmSS8ICz7xGtLdnReJWcloaYjyLyiL5k6hdfkbNMfPV+XaajJ4hYq2TwYsw8yiTOlgZCss+6Pk/k9YTie8qPmA9r/VPvo3H8nqy+W+FBLcfob4zgBtev3XUSUBYrHeCC91VA5X3kRvOqfBzqu6nJPOhfV6kNnG6KlCFVSpKn0rGuqFhuIIMBWZNC/2D3qo/Ry5ZdGaeNP/jCMVaoMeRu1bNj+jIIfMmYZ6FLXAklfjTaI5NFBFsAvz8q3SNIbhjkthfT18xEVWoVaMX7i6sK8fB3pXPFfsTV03TqJTmRRo5Cl0gUlDC/zOgbl2ANbQ9O1uva9/ARJNSl81vlhDw9T17Huh1DuFYeVZq/37Ax x6vm9NOa d+NZop+CWiMTFcnd5wxJQPbG5zDnq9Fd8+FU/GUGBeOVbB1DZuIECwc4RyeUGOkH+eqDB4dxtdqwatIHfwsIDaTpmtam+Y5oLFMlrEGBcuOmfegGmFIhE1UCd2oh7ppWCFJcuL3C0r0jWMKaiHyOUzoaBZCwj9SboE6NodFqXStoAZH/p1MtC2ul01Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When we enable boot-time page size, some macros are no longer compile-time constants. Where these macros are used as cases in switch statements, the switch statements no longer compile. Let's convert these to if/else blocks, which can handle the runtime values. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/kvm/mmu.c | 32 +++++++++++++++----------------- arch/arm64/mm/hugetlbpage.c | 34 +++++++++++----------------------- 2 files changed, 26 insertions(+), 40 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index a509b63bd4dd5..248a2d7ad6dbb 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1487,29 +1487,27 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, vma_shift = get_vma_page_shift(vma, hva); } - switch (vma_shift) { #ifndef __PAGETABLE_PMD_FOLDED - case PUD_SHIFT: - if (fault_supports_stage2_huge_mapping(memslot, hva, PUD_SIZE)) - break; - fallthrough; + if (vma_shift == PUD_SHIFT) { + if (!fault_supports_stage2_huge_mapping(memslot, hva, PUD_SIZE)) + vma_shift = PMD_SHIFT; + } #endif - case CONT_PMD_SHIFT: + if (vma_shift == CONT_PMD_SHIFT) { vma_shift = PMD_SHIFT; - fallthrough; - case PMD_SHIFT: - if (fault_supports_stage2_huge_mapping(memslot, hva, PMD_SIZE)) - break; - fallthrough; - case CONT_PTE_SHIFT: + } + if (vma_shift == PMD_SHIFT) { + if (!fault_supports_stage2_huge_mapping(memslot, hva, PMD_SIZE)) + vma_shift = PAGE_SHIFT; + } + if (vma_shift == CONT_PTE_SHIFT) { vma_shift = PAGE_SHIFT; force_pte = true; - fallthrough; - case PAGE_SHIFT: - break; - default: - WARN_ONCE(1, "Unknown vma_shift %d", vma_shift); } + if (vma_shift != PUD_SHIFT && + vma_shift != PMD_SHIFT && + vma_shift != PAGE_SHIFT) + WARN_ONCE(1, "Unknown vma_shift %d", vma_shift); vma_pagesize = 1UL << vma_shift; diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 5f1e2103888b7..bc98c20655bba 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -51,16 +51,12 @@ void __init arm64_hugetlb_cma_reserve(void) static bool __hugetlb_valid_size(unsigned long size) { - switch (size) { #ifndef __PAGETABLE_PMD_FOLDED - case PUD_SIZE: + if (size == PUD_SIZE) return pud_sect_supported(); #endif - case CONT_PMD_SIZE: - case PMD_SIZE: - case CONT_PTE_SIZE: + if (size == CONT_PMD_SIZE || size == PMD_SIZE || size == CONT_PTE_SIZE) return true; - } return false; } @@ -104,24 +100,20 @@ static inline int num_contig_ptes(unsigned long size, size_t *pgsize) *pgsize = size; - switch (size) { #ifndef __PAGETABLE_PMD_FOLDED - case PUD_SIZE: + if (size == PUD_SIZE) { if (pud_sect_supported()) contig_ptes = 1; - break; + } else #endif - case PMD_SIZE: + if (size == PMD_SIZE) { contig_ptes = 1; - break; - case CONT_PMD_SIZE: + } else if (size == CONT_PMD_SIZE) { *pgsize = PMD_SIZE; contig_ptes = CONT_PMDS; - break; - case CONT_PTE_SIZE: + } else if (size == CONT_PTE_SIZE) { *pgsize = PAGE_SIZE; contig_ptes = CONT_PTES; - break; } return contig_ptes; @@ -339,20 +331,16 @@ unsigned long hugetlb_mask_last_page(struct hstate *h) { unsigned long hp_size = huge_page_size(h); - switch (hp_size) { #ifndef __PAGETABLE_PMD_FOLDED - case PUD_SIZE: + if (hp_size == PUD_SIZE) return PGDIR_SIZE - PUD_SIZE; #endif - case CONT_PMD_SIZE: + if (hp_size == CONT_PMD_SIZE) return PUD_SIZE - CONT_PMD_SIZE; - case PMD_SIZE: + if (hp_size == PMD_SIZE) return PUD_SIZE - PMD_SIZE; - case CONT_PTE_SIZE: + if (hp_size == CONT_PTE_SIZE) return PMD_SIZE - CONT_PTE_SIZE; - default: - break; - } return 0UL; } From patchwork Mon Oct 14 10:58:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E93AD1A43B for ; Mon, 14 Oct 2024 11:02:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB27F6B00E0; Mon, 14 Oct 2024 07:01:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A3A456B00E2; Mon, 14 Oct 2024 07:01:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 88E456B00E3; Mon, 14 Oct 2024 07:01:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6A4006B00E0 for ; Mon, 14 Oct 2024 07:01:59 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 9169280D96 for ; Mon, 14 Oct 2024 11:01:52 +0000 (UTC) X-FDA: 82671917748.26.03F7C58 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf01.hostedemail.com (Postfix) with ESMTP id 12D364001B for ; Mon, 14 Oct 2024 11:01:51 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903671; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=97p+R0dspaRUKJqMA/Y3wP7vnE/E8FcpBV0RwE1dMZY=; b=7J5Bt4T8LII6d4nO6FLOvCKFSwHcm4cd/c29lBq7jNTmltHkdFK+T26j+28A+ACDs3OgfX wYFSnMWCrjEr6zdiTljZ0+6ajuRR6oVcWKWg0Na/QqQ9rQRuEMJjLD4+Q3RXWjSGiSYVwt /lr5op+e04XONj+1qTF+Ebyvkm3fAV4= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903671; a=rsa-sha256; cv=none; b=M42SW2xQQHuAVsJFqE6xCmO+taRk/dTa9nrtZnxS78s36FlFTkYO/Vf6yS8pCA5AZWix6Q 4thio+lG3k6olDhnrekJe3gPbyWVc5bZ/C75gfFIKuAcp5/TvUFzrPSQrEu/TV43iaTrCK k6l8G/fqS1ceDMJpUjI/A9+bxg7xFfA= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4FFED19F0; Mon, 14 Oct 2024 04:02:26 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 512493F51B; Mon, 14 Oct 2024 04:01:54 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 49/57] arm64: Convert BUILD_BUG_ON to VM_BUG_ON Date: Mon, 14 Oct 2024 11:58:56 +0100 Message-ID: <20241014105912.3207374-49-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 67cfct18wbdofytk7kxshbytk4dhq7iq X-Rspamd-Queue-Id: 12D364001B X-Rspamd-Server: rspam11 X-HE-Tag: 1728903711-262718 X-HE-Meta: U2FsdGVkX19fVbl1RtZtvaZ7+7pMbI6w1CMrfdvf/kqtXZBZWgAUUxJA10iVmlF9/zuxTC3FIRP7PUdDqtyt8J7a1A2zAt6R/fmfy2RntHmp896u6nnZN3iPC81JTTLoqWScXkJrmGenZwjxqr2wOaPzAqcXWeWsjac3UZjmoG+PulfWzVkD25Ao1ozjZn7lFNhtW9A2fhOLi7tuPi3H1I1Ubhg9IkEHmQ9JaygMfGCqWs76yuMw7O4DRsEIWyBn/PupxsSdrgmvdvaBRwBMMnj1zw/h2JPL+vLTai1xBrx1jHf5Nx8xgVKpSBvkpl1l1Cpj0YDHzwPG38lWjaRw/7uUUhcDFPLhQj51G7PaoPsmPsgzdjf74XE+I5QpE01st09HJGXe3h3khYmjLz24M5+VeRWbIEQkqbPMK9mZPpaVcDXR5TFVVV/yBeN13dAIPLcps9ouzFEb+TmDihhbfrFuMLzyrM4CnbVs9FAiQzbsQhtOO6icepiy3SzLJiTjtJ1LLRV4tsr3OZ/+R3sn4/mwZsdO0NgZToLGDQg23rAyatzRIVVE70zyoCqxEEKoMzr6CQrQLD4FU/JuaClnLmsBLLyXpPuqnmiMCT8j/Htp0UpholJdQy0lYpbcXtdYN5R0lYSDOt98hPsfY90o0/gH6sxVSZGFIWPWzk0070ihY1lRym65I4z+PGTCRKmODCSq2ouLxOAuULhlYBVL8j7IwcTlVZcJOSj/pi1gBByBik7ZzJGkrgfaR8hrSa6dGJaWrLbUrONYqV7MHJTxtQvlWpNWRkhi+VmsTd1huLwe3cnZxUQ2dHg8K8qZCF9a8E/CCtBjiUPi4Rn9kFEpXQ1zCkZ0rg9LRm17pldgm6vuO9cFz0K15bzfnozw6BbYfOXbFDhHSUV1t5wdqrwzqKYh7AGXgD4UiqIaaZSptAUwWyAmjeuWzcI01x/F+7WeCOebgKVat2aIlzVK4Fe v64awk07 v/SVo6F412/sWSconcVUrgWqO9wKe1VgyX5LGOsJ/r0n6N5hCJX4zgEw5hMDnli/P0tlfeakUGM3yKXufrIfZYu3J3UPaGi8dwvsaZxDsg9bUVLXaXNnY1ugObvzX0xRVgeiUi/DnsS82fbic3vEJ+1spP90RcwlfakQduTwDr0h/ts6oN+WITFdJw0Zy/k0o+TJkcGNRwAIM741FP811OWT3aycHpAsKrhud X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There are some build bug checks that will no longer compile for boot-time page size because the values they are testing are no longer compile-time constants. Resolve these by converting them to VM_BUG_ON, which will perform a runtime check. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/mm/init.c | 6 +++--- arch/arm64/mm/mmu.c | 4 ++-- arch/arm64/mm/pgd.c | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 42eb246949072..4d24034418b39 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -388,15 +388,15 @@ void __init mem_init(void) * detected at build time already. */ #ifdef CONFIG_COMPAT - BUILD_BUG_ON(TASK_SIZE_32 > DEFAULT_MAP_WINDOW_64); + VM_BUG_ON(TASK_SIZE_32 > DEFAULT_MAP_WINDOW_64); #endif /* * Selected page table levels should match when derived from * scratch using the virtual address range and page size. */ - BUILD_BUG_ON(ARM64_HW_PGTABLE_LEVELS(CONFIG_ARM64_VA_BITS) != - CONFIG_PGTABLE_LEVELS); + VM_BUG_ON(ARM64_HW_PGTABLE_LEVELS(CONFIG_ARM64_VA_BITS) != + CONFIG_PGTABLE_LEVELS); if (PAGE_SIZE >= 16384 && get_num_physpages() <= 128) { extern int sysctl_overcommit_memory; diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index b4cd3b6a73c22..ad7fd3fda705a 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -639,8 +639,8 @@ static void __init map_mem(pgd_t *pgdp) * entire reduced VA space is covered by a single pgd_t which will have * been populated without the PXNTable attribute by the time we get here.) */ - BUILD_BUG_ON(pgd_index(direct_map_end - 1) == pgd_index(direct_map_end) && - pgd_index(_PAGE_OFFSET(VA_BITS_MIN)) != PTRS_PER_PGD - 1); + VM_BUG_ON(pgd_index(direct_map_end - 1) == pgd_index(direct_map_end) && + pgd_index(_PAGE_OFFSET(VA_BITS_MIN)) != PTRS_PER_PGD - 1); early_kfence_pool = arm64_kfence_alloc_pool(); diff --git a/arch/arm64/mm/pgd.c b/arch/arm64/mm/pgd.c index 0c501cabc2384..4b106510358b1 100644 --- a/arch/arm64/mm/pgd.c +++ b/arch/arm64/mm/pgd.c @@ -56,7 +56,7 @@ void __init pgtable_cache_init(void) * With 52-bit physical addresses, the architecture requires the * top-level table to be aligned to at least 64 bytes. */ - BUILD_BUG_ON(PGD_SIZE < 64); + VM_BUG_ON(PGD_SIZE < 64); #endif /* From patchwork Mon Oct 14 10:58:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834718 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6DB2D1A43B for ; Mon, 14 Oct 2024 11:02:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E5A36B00E2; Mon, 14 Oct 2024 07:02:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 46DC16B00E4; Mon, 14 Oct 2024 07:02:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C0946B00E5; Mon, 14 Oct 2024 07:02:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 0C1886B00E2 for ; Mon, 14 Oct 2024 07:02:02 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 28C591A0D6B for ; Mon, 14 Oct 2024 11:01:48 +0000 (UTC) X-FDA: 82671917832.13.01DBC17 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf27.hostedemail.com (Postfix) with ESMTP id 0AB6140026 for ; Mon, 14 Oct 2024 11:01:53 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf27.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903604; a=rsa-sha256; cv=none; b=mQ8RIRiEmtBcMkqVVebIDJ+U+QG//rX5tfuwSUPzwWR/NPZiZuVWMazCtAUAqWlQObNQ9o WngpvrokqgC/6G15IBiVsnc6ORVTD0yYethLgMr/l7CsmjteCm+H2bj0dbVt6w87YSIMWL FF4WOffb6MpPcjxYveb288UIBW9Ou6U= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf27.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903604; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1XPkM6wVUNtvQ6sRxRh3jqob0EiliLUCc+oDCpziCPI=; b=b93oYE5zfPBI/vhdrauu0eFlegyv6nHnWHQ1D7GlTmJWP1QatIb3+2WRhEf4ZUOD0vcWgi RqbVM/lWUaEird91qfV1jpY4dl9YZnCUZkSnjNsSCx0nzee0fdwSk1IgX87ZUGskPs3owb D/hsEC6eZxkzMrfoid21Pn8cLyVb+to= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0495C1A00; Mon, 14 Oct 2024 04:02:29 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0154B3F51B; Mon, 14 Oct 2024 04:01:56 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 50/57] arm64: Remove PAGE_SZ asm-offset Date: Mon, 14 Oct 2024 11:58:57 +0100 Message-ID: <20241014105912.3207374-50-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 0AB6140026 X-Stat-Signature: efehr6bhfrdb9p5x1ojznta98g46u65y X-Rspam-User: X-HE-Tag: 1728903713-911181 X-HE-Meta: U2FsdGVkX19BTQDh2qgMhCn60MqcBrSvQW2AIqEnEW4b3hQF2lT1AFIiRxZTN+I/+af/QDcsXze2iMnpWtECv5xeaJA3woxUfK5etjBxucTVnqB4qqznjswlocbBsXJmPPckPsLQM5ZCsBnZvkpCrSkl4yK1OEa10A7SfGtZjrtBn04Y8jEwkul2dmZP3ZcomyadVME23Xzt9twwD7cdEwMcoNSX2unm/Yrklvi8q2eLRmL7I1kOuM3THsTqg6MK8NfX/4jLggNTqCnUrOOVasUf93deL50EUm6Wb7w2F6Zk0aQ05F1Mm6vY/Ygvmc79Sctsu/D59MnF5aAoNes9FyStXCQ1zm8BS/WJYbu7/q/VghBhxawBNGE+O7dVXA5LEzTWGMlJ8dkEE22lrPuXKqH8QtxqbP5VWRGZKS66dY1cqciDQxnut0z9WyoX2+K6rDG1O3VjgbE0h3SDvac6/7fjsEiSHwSjnm5aGaK+jU412gRq0hw966HnEifZAouRyIjXzaQORBrcljOCb1Twv84UnjTuXUSlK0JlRUsZYh0dq1kNg0+DdWXhXWOtHMirW5mo5130WyoVt04BZz/CPbKffLx/zbtmpcUST4I0OCFRgmJxWPhUXh5WmXGDN016HsUj2ru1GCapfTnRgwmyqrhGlZxiLZ4oZHAs6pq+E6lTitQzN85IZpoitKjJPD3HulTBLGUX76GoMbCzXgzeHDENNV0Oxjsr3ONAtHYbnKtkd54SLVrq3P1cz+1w7QOYWdiqv0R+RV7EDR7FfNby8S4WC3wfCY8R0RrkXqDDJ1bSr1IKEpZzik1KVRAW9Hiqq1LZDYrG8LrcNZOWIglcbDY63xCvPp3hE1BX80YRPKRibF8oxKAdmqBhKTNtD2JYV6tf+tfyr8xuWnyPKsWTT4zr3JsMZYtI6XB8X1hjoaLGAmaTEDGlUXr6Kx4dLNnWC563oo8vrskW75bpoXy AngAp8w6 1/W14wkrZ3YCBFt6iEuMvpZdPeyCT84l0aXQAbTRLOd6UK26b1ENugBZWpX5HXtIPpNzSvuKAADKhffHvvCaLWMd0hG+09clVpzs7IAxNZ3tCmA3a8KzVVaioHMdtfgoRnlAlUyTRBCMVRdRTWe2MtpcieuOSpkRKq2YongYVWgatM44YO0J5XwWSbCA0c5juzz486NeTG4G+C8BPdCBENomV14yJsXAPqUox X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: PAGE_SZ is not used anywhere and for boot-time builds, where it is not a compile-time constant, we don't know the value anyway. So let's remove it. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/kernel/asm-offsets.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 27de1dddb0abe..f32b8d7f00b2a 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -111,8 +111,6 @@ int main(void) BLANK(); DEFINE(VM_EXEC, VM_EXEC); BLANK(); - DEFINE(PAGE_SZ, PAGE_SIZE); - BLANK(); DEFINE(DMA_TO_DEVICE, DMA_TO_DEVICE); DEFINE(DMA_FROM_DEVICE, DMA_FROM_DEVICE); BLANK(); From patchwork Mon Oct 14 10:58:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59280D1A443 for ; Mon, 14 Oct 2024 11:02:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D718D6B00E4; Mon, 14 Oct 2024 07:02:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D21176B00E6; Mon, 14 Oct 2024 07:02:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4AE96B00E7; Mon, 14 Oct 2024 07:02:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 962356B00E4 for ; Mon, 14 Oct 2024 07:02:04 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 914D4140D64 for ; Mon, 14 Oct 2024 11:01:56 +0000 (UTC) X-FDA: 82671918042.02.616BFC7 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf20.hostedemail.com (Postfix) with ESMTP id 6C6EB1C0026 for ; Mon, 14 Oct 2024 11:01:55 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf20.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903581; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VfAcTUi91ptQaK/CGe1CeXA9aaPE8rozFIbYUTlYxXU=; b=205FlPVQe6qhG42HhW00t9BVcd+5xFylrNhg0da+GPP3hC6yy896Nnc/+ZD+ALosHOaCc4 fHieCfDNLPY8KEIGVF/DXqMxX8v5toAT12z/HZQ35sVLYiiz67d4iOjn1LmU7g9GGq/Nt/ jlCw9E/1NAADyI/V2q/lhJv7vlix3SA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903581; a=rsa-sha256; cv=none; b=gppj76eNZAI1EaSo20qcKWliEGpsG9uQMBYh2uX72LpwoNng4s+mP0OnX1H6nHGSLESsBW AUtxDKhd8z0heNnoRfYYUo4LCt5kDD3dU/wRZ83HLje8nFpTTfR66vcm0UJ1/DOhuh+6Di ivsaehrzPTEyaFPGKMrFwB/PNXCIWfc= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf20.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A49A51A25; Mon, 14 Oct 2024 04:02:31 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A5B033F51B; Mon, 14 Oct 2024 04:01:59 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 51/57] arm64: Introduce cpu features for page sizes Date: Mon, 14 Oct 2024 11:58:58 +0100 Message-ID: <20241014105912.3207374-51-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6C6EB1C0026 X-Stat-Signature: gpjxw9y38b136cq1g7yhg75rdmuxj3n5 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1728903715-532256 X-HE-Meta: U2FsdGVkX18SiMqX1pPWtypPAKhRRrNtFl9CAg+MxkMlP/KLMBr9+EUCCRqGmQXcbW5NsuVxhENc2HYQwlAp2bwDDht+5XdEiVX5hsR5zXGSqZ+yNSZ4c3xcfzggKM1Gw3Zbg8HbxCG6K2b8mXfALBDqornPs1ttsWaoxJ3YcPeCh4g4Y1PtusY9n0KqZJEq2mvLU9TaAhijZ/ueLPAdILo9eRz0Ref506UCfLoZFxIq4lo1nS3OwW0NUBX/D1hjLrnBBF+7aO+MPHxkswViGpY56ey2HHjZOeXNBmu5LNiJAlTzeoW1edoWXTXrmjkrR9EkaQfXZ9pqkyqc20lgDiJ2dThr+EIhHreT4R9Q6JS4MHjBGudPxFaVo3GnkZBZ/NEcpkU3tCay4DaGEekNcHbF46gs61FaPGQ/dHxUpru9vzWGsZV/hIPH8jalgQ+YgNe0g0HHafCKQ/+xJlDMoIaVmKLPkEOZXOorGhNdPM9/RrFtKxet3ZT1cz8MtcBx6oYZEPmWRwlh5ZzL0jWe1weE0OIvKWdkBXNBylHQeJKtx4pdLMvzy2VFo1fN/W/oR4yNfOJV1YXFb41RF2Ey/cYja2oGKbaJVMkk72SKVC/H9+3XQyAfkgY4ieX/enLx9jP2wqwGyhwNH3V5afMq2ij97uzbA4V6VK21ClAMX3traAgu3RjC6l7aWAlZvQAli+tCBubCuwStT9YTp3v6RMP3W0ugcTg/dQ0y6NHMlpLaFPw49v/bIRNeJmCQUQRLgLlJ7Eyyxagi45DYN8+ic4/kyy8TeW8v4X6qvnJpcEo90pOMCZl5p4kdzSK1NrPaVH7n9W3S0/3Wh8Zo5iqflhqx0rEtst/3r6NYEO+z6A6Ji31OZpHiQtuA4sEDngNtmTx0MvwY6u5V/c1X+H3rRpy5+5ddTXc7nQVfw21aPOSrPT+FLPTh64BxCcv2vlF9vj/4fYgYY4bicpX6r/J AzS8bCoU 4MYnMgob/Hizg1TH70qDj62LoVEQcjvINnx2DEdxMPcQI7Cyh00wrVYOrdtF7dXVp502FT6gUQj9ExVpLwp+tqutyppsJUSkxnIlChmZPTRBjz2xBpdVGD0cwOz/QVHqjGc8PN8h385Q6ty2oEqPLNILt76r9PCCd1Y3r1a+/Wb32s+bZtmKZi1rL5A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduce one boot cpu feature per page size, ARM64_USE_PAGE_SIZE_*K. These will enable use of the alternatives framework to patch code based on the page size selected at boot. Additionally, this provides a neat way to figure out what page size is in use during boot from the kernel log. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/kernel/cpufeature.c | 25 +++++++++++++++++++++++++ arch/arm64/tools/cpucaps | 3 +++ 2 files changed, 28 insertions(+) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 7705c9c0e7142..e5618423bb99d 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1831,6 +1831,13 @@ static bool has_nv1(const struct arm64_cpu_capabilities *entry, int scope) is_midr_in_range_list(read_cpuid_id(), nv1_ni_list))); } +static bool use_page_size(const struct arm64_cpu_capabilities *entry, int scope) +{ + return (entry->capability == ARM64_USE_PAGE_SIZE_4K && PAGE_SIZE == SZ_4K) || + (entry->capability == ARM64_USE_PAGE_SIZE_16K && PAGE_SIZE == SZ_16K) || + (entry->capability == ARM64_USE_PAGE_SIZE_64K && PAGE_SIZE == SZ_64K); +} + static bool has_lpa2_at_stage1(u64 mmfr0) { unsigned int tgran; @@ -2881,6 +2888,24 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = has_nv1, ARM64_CPUID_FIELDS_NEG(ID_AA64MMFR4_EL1, E2H0, NI_NV1) }, + { + .desc = "4K page size", + .capability = ARM64_USE_PAGE_SIZE_4K, + .type = ARM64_CPUCAP_BOOT_CPU_FEATURE, + .matches = use_page_size, + }, + { + .desc = "16K page size", + .capability = ARM64_USE_PAGE_SIZE_16K, + .type = ARM64_CPUCAP_BOOT_CPU_FEATURE, + .matches = use_page_size, + }, + { + .desc = "64K page size", + .capability = ARM64_USE_PAGE_SIZE_64K, + .type = ARM64_CPUCAP_BOOT_CPU_FEATURE, + .matches = use_page_size, + }, {}, }; diff --git a/arch/arm64/tools/cpucaps b/arch/arm64/tools/cpucaps index ac3429d892b9a..5cb75675303c6 100644 --- a/arch/arm64/tools/cpucaps +++ b/arch/arm64/tools/cpucaps @@ -71,6 +71,9 @@ SPECTRE_BHB SSBS SVE UNMAP_KERNEL_AT_EL0 +USE_PAGE_SIZE_4K +USE_PAGE_SIZE_16K +USE_PAGE_SIZE_64K WORKAROUND_834220 WORKAROUND_843419 WORKAROUND_845719 From patchwork Mon Oct 14 10:58:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834720 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13451D1A43B for ; Mon, 14 Oct 2024 11:02:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D985E6B00E6; Mon, 14 Oct 2024 07:02:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D491E6B00E8; Mon, 14 Oct 2024 07:02:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4C0E6B00E9; Mon, 14 Oct 2024 07:02:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 954616B00E6 for ; Mon, 14 Oct 2024 07:02:07 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7BC7640D98 for ; Mon, 14 Oct 2024 11:02:02 +0000 (UTC) X-FDA: 82671918126.24.15D0613 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf17.hostedemail.com (Postfix) with ESMTP id A6F0540010 for ; Mon, 14 Oct 2024 11:02:00 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903537; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CYEBM3jqbad+bofavVbJlOnhYtw41610magjm74TJ1s=; b=Opw5YVl71UDtLMxX8B/5P08iV7nGPnrEI22+IgaqPa3WUrk7ck1hac10jsWD0JOQJnRE+Y 5jCm8RqSzhgK2uW1enxOcNFSbssMmIgXD+dV1AKURXy11hsw07DLVGjqwUqSby1Yybc6pz iuIbIgV3yAiY7qcFyeXt/a09op7ejwQ= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; spf=pass (imf17.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903537; a=rsa-sha256; cv=none; b=K96luwJ6DttQxOonIrivk5S9poJAiqMOP2mJk4eNURQyKZZGlyDIFZdQ2kJXmdMybvEhN9 SyuR9+0Q2HNe/lbfxvonaA44kFSSzrK79U6PHNidlr7LI+D65+z4YQ+hgtmSc7Dd4ufhJA p5/41duBCAOBKlmDYsR4KH2MCikWtqg= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 926581A2D; Mon, 14 Oct 2024 04:02:34 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 55EA53F51B; Mon, 14 Oct 2024 04:02:02 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Oliver Upton , Will Deacon Cc: Ryan Roberts , kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 52/57] arm64: Remove PAGE_SIZE from assembly code Date: Mon, 14 Oct 2024 11:58:59 +0100 Message-ID: <20241014105912.3207374-52-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: A6F0540010 X-Stat-Signature: 6s19sbyz884p17wqb6zbod5gz79j1k47 X-Rspam-User: X-HE-Tag: 1728903720-97948 X-HE-Meta: U2FsdGVkX1/9JSsM7B8b7BDoDjtwlnj0mFf/qvKOJ5O1RGTnPgXLqeCdi9ShOb23ISQmWMMn9ruRn2b97D6BUiS0IfJEmantcFxIFxwplKRCLH/VHR0FEaNY1npO0WSXXkVRq0ZAMpEUH8pHVGLIV48Z4wf5hSwBYGvqFdQj9aFBg/3jNg1aAOtN0E8Buq5txBQKrNsIt8GGtOWqFcx96xtoVJBBV6h9VVuQ0rM4o2wsMGVNVzwK+lxbmCBps6vYs2evk5Xp7t8nDdS7ISLRZOXXKH0eJprATcRus1OvvNhvF8XAyYi+kFB1bXyu1FJVl0xcx0xVv5Q8gb7fqApHQop6Ko3iUKMK7ngVsM6HeAaeaJVWc0JjJCj31SfabfdcGrx/HNheWuzFmPaOl+nGuP217SS6LtzTfM9wn/mB7MYvrI3wIgo1YyZjv3kM3D0joqmQ9vg++FyCid8VxbIcEHYzA4XXrjAntqplkTw4bfrtMOjZm3yWYhG4CMR1rqLakGjUcRHg5eCUuzgevkoYqHSBXkLbaOT2+995NqbFma6gNdpaSjcFQL/eHnlw08GtEdTXg1W4jEpAjtz0ap5MBMtArZxjUcXfBaz4I3dN0KLTIx3wXpfJaXx/2veHsiFk/bU0x5bYW9Hx1+gCZHPgudjdRl0OCsNUBkujRzFdh3DIkZcOhPMz9QQGiAf5iu8noGQktj0b1cZJyLEqGK/HABRbRB+E13J0/9wLGz49VZkeqeX8X80ckZn9xrDbjKjTidaTaw07pXlU51/Ggi8Am3M/qhh/ncSDBkNNwPg1VXJgsHQ9S9ZJxUzC6D5glMORUUh7UiEwBtYiXx/BdqyCmyWXYlE8QetYBXTtdUX8/RW+hWWuiVrDw5ddkEywVB3+vU8BcPyjcwC1kwX+lzy/5zm0Vj0+v5YsBFM2ok40vZ9LJUNfvdXzDBxcSs2Ar/HYbi0KerIGelE9MsauTPx q2LxmgyK Ba311+I2zRILMvnljYg3dGiCpDINSzaBrkLYMN1GUnzcNfB+FpZsoJaaipXtsoAa2BZRxBArdVXPATlQKGV9/Q3rK+ocxmt0mQi7/wJ4IHx45LjJo4jz6PPA7X/SbuXsajidq7OupyMq7ECJMaIhQxOQS/dc53JTxwoYIkIMeJCIzjgSvjRBJc1G9+3OLLBixBf5Z86oBEp6kAUF9c583r1J/JsXRGlOBSS+v X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Remove usage of PAGE_SHIFT, PAGE_SIZE and PAGE_MASK macros from assembly code since these are no longer compile-time constants when boot-time page size is in use. For the most part, they are replaced with run-time lookups based on the value of TG0. This is done outside of loops so while there is a cost of a few extra instructions, performance should not be impacted. However, invalid_host_el2_vect requires that the page shift be an immediate since it has no registers to spare. So for this, let's use alternatives patching. This code is guarranteed not to run until after patching is complete. __pi_copy_page has no registers to spare to hold the page size, and we want to avoid having to reload it on every iteration of the loop. Since I couldn't provably conclude that the function is not called prior to alternatives patching, I opted to make a copy of the function for each page size and branch to the right one at the start. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/assembler.h | 18 +++++++++++++--- arch/arm64/kernel/hibernate-asm.S | 6 ++++-- arch/arm64/kernel/relocate_kernel.S | 10 ++++++--- arch/arm64/kvm/hyp/nvhe/host.S | 10 ++++++++- arch/arm64/lib/clear_page.S | 7 ++++-- arch/arm64/lib/copy_page.S | 33 +++++++++++++++++++++-------- arch/arm64/lib/mte.S | 27 +++++++++++++++++------ 7 files changed, 85 insertions(+), 26 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 77c2d707adb1a..6424fd6be1cbe 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -495,9 +495,11 @@ alternative_endif .Lskip_\@: .endm /* - * copy_page - copy src to dest using temp registers t1-t8 + * copy_page - copy src to dest using temp registers t1-t9 */ - .macro copy_page dest:req src:req t1:req t2:req t3:req t4:req t5:req t6:req t7:req t8:req + .macro copy_page dest:req src:req t1:req t2:req t3:req t4:req t5:req t6:req t7:req t8:req t9:req + get_page_size \t9 + sub \t9, \t9, #1 // (PAGE_SIZE - 1) in \t9 9998: ldp \t1, \t2, [\src] ldp \t3, \t4, [\src, #16] ldp \t5, \t6, [\src, #32] @@ -508,7 +510,7 @@ alternative_endif stnp \t5, \t6, [\dest, #32] stnp \t7, \t8, [\dest, #48] add \dest, \dest, #64 - tst \src, #(PAGE_SIZE - 1) + tst \src, \t9 b.ne 9998b .endm @@ -911,4 +913,14 @@ alternative_cb_end .macro tgran_lpa2, val, tg0 value_for_page_size \val, \tg0, ID_AA64MMFR0_EL1_TGRAN4_52_BIT, ID_AA64MMFR0_EL1_TGRAN16_52_BIT, -1 .endm + + .macro get_page_size, val + get_tg0 \val + value_for_page_size \val, \val, SZ_4K, SZ_16K, SZ_64K + .endm + + .macro get_page_mask, val + get_tg0 \val + value_for_page_size \val, \val, (~(SZ_4K-1)), (~(SZ_16K-1)), (~(SZ_64K-1)) + .endm #endif /* __ASM_ASSEMBLER_H */ diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index 0e1d9c3c6a933..375b2fcf82e84 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -57,6 +57,8 @@ SYM_CODE_START(swsusp_arch_suspend_exit) mov x24, x4 mov x25, x5 + get_page_size x12 + /* walk the restore_pblist and use copy_page() to over-write memory */ mov x19, x3 @@ -64,9 +66,9 @@ SYM_CODE_START(swsusp_arch_suspend_exit) mov x0, x10 ldr x1, [x19, #HIBERN_PBE_ADDR] - copy_page x0, x1, x2, x3, x4, x5, x6, x7, x8, x9 + copy_page x0, x1, x2, x3, x4, x5, x6, x7, x8, x9, x11 - add x1, x10, #PAGE_SIZE + add x1, x10, x12 /* Clean the copied page to PoU - based on caches_clean_inval_pou() */ raw_dcache_line_size x2, x3 sub x3, x2, #1 diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 413f899e4ac63..bc4f37fba6c74 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -46,6 +46,10 @@ SYM_CODE_START(arm64_relocate_new_kernel) ldr x27, [x0, #KIMAGE_ARCH_EL2_VECTORS] ldr x26, [x0, #KIMAGE_ARCH_DTB_MEM] + /* Grab page size values. */ + get_page_size x10 /* x10 = PAGE_SIZE */ + get_page_mask x11 /* x11 = PAGE_MASK */ + /* Setup the list loop variables. */ ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ @@ -54,7 +58,7 @@ SYM_CODE_START(arm64_relocate_new_kernel) raw_dcache_line_size x15, x1 /* x15 = dcache line size */ break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: - and x12, x16, PAGE_MASK /* x12 = addr */ + and x12, x16, x11 /* x12 = addr */ sub x12, x12, x22 /* Convert x12 to virt */ /* Test the entry flags. */ .Ltest_source: @@ -62,8 +66,8 @@ SYM_CODE_START(arm64_relocate_new_kernel) /* Invalidate dest page to PoC. */ mov x19, x13 - copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 - add x1, x19, #PAGE_SIZE + copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8, x9 + add x1, x19, x10 dcache_by_myline_op civac, sy, x19, x1, x15, x20 b .Lnext .Ltest_indirection: diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 3d610fc51f4d3..2b0d583fcf1af 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -193,7 +193,15 @@ SYM_FUNC_END(__host_hvc) */ add sp, sp, x0 // sp' = sp + x0 sub x0, sp, x0 // x0' = sp' - x0 = (sp + x0) - x0 = sp - tbz x0, #PAGE_SHIFT, .L__hyp_sp_overflow\@ +alternative_if ARM64_USE_PAGE_SIZE_4K + tbz x0, #ARM64_PAGE_SHIFT_4K, .L__hyp_sp_overflow\@ +alternative_else_nop_endif +alternative_if ARM64_USE_PAGE_SIZE_16K + tbz x0, #ARM64_PAGE_SHIFT_16K, .L__hyp_sp_overflow\@ +alternative_else_nop_endif +alternative_if ARM64_USE_PAGE_SIZE_64K + tbz x0, #ARM64_PAGE_SHIFT_64K, .L__hyp_sp_overflow\@ +alternative_else_nop_endif sub x0, sp, x0 // x0'' = sp' - x0' = (sp + x0) - sp = x0 sub sp, sp, x0 // sp'' = sp' - x0 = (sp + x0) - x0 = sp diff --git a/arch/arm64/lib/clear_page.S b/arch/arm64/lib/clear_page.S index ebde40e7fa2b2..b6f2cb8d704cc 100644 --- a/arch/arm64/lib/clear_page.S +++ b/arch/arm64/lib/clear_page.S @@ -15,6 +15,9 @@ * x0 - dest */ SYM_FUNC_START(__pi_clear_page) + get_page_size x3 + sub x3, x3, #1 /* (PAGE_SIZE - 1) in x3 */ + mrs x1, dczid_el0 tbnz x1, #4, 2f /* Branch if DC ZVA is prohibited */ and w1, w1, #0xf @@ -23,7 +26,7 @@ SYM_FUNC_START(__pi_clear_page) 1: dc zva, x0 add x0, x0, x1 - tst x0, #(PAGE_SIZE - 1) + tst x0, x3 b.ne 1b ret @@ -32,7 +35,7 @@ SYM_FUNC_START(__pi_clear_page) stnp xzr, xzr, [x0, #32] stnp xzr, xzr, [x0, #48] add x0, x0, #64 - tst x0, #(PAGE_SIZE - 1) + tst x0, x3 b.ne 2b ret SYM_FUNC_END(__pi_clear_page) diff --git a/arch/arm64/lib/copy_page.S b/arch/arm64/lib/copy_page.S index 6a56d7cf309da..6c19b03ab4d69 100644 --- a/arch/arm64/lib/copy_page.S +++ b/arch/arm64/lib/copy_page.S @@ -10,14 +10,7 @@ #include #include -/* - * Copy a page from src to dest (both are page aligned) - * - * Parameters: - * x0 - dest - * x1 - src - */ -SYM_FUNC_START(__pi_copy_page) + .macro copy_page_body, page_size ldp x2, x3, [x1] ldp x4, x5, [x1, #16] ldp x6, x7, [x1, #32] @@ -30,7 +23,7 @@ SYM_FUNC_START(__pi_copy_page) add x0, x0, #256 add x1, x1, #128 1: - tst x0, #(PAGE_SIZE - 1) + tst x0, #(\page_size - 1) stnp x2, x3, [x0, #-256] ldp x2, x3, [x1] @@ -62,7 +55,29 @@ SYM_FUNC_START(__pi_copy_page) stnp x12, x13, [x0, #80 - 256] stnp x14, x15, [x0, #96 - 256] stnp x16, x17, [x0, #112 - 256] + .endm +/* + * Copy a page from src to dest (both are page aligned) + * + * Parameters: + * x0 - dest + * x1 - src + */ +SYM_FUNC_START(__pi_copy_page) + get_tg0 x2 +.Lsz_64k: + cmp x2, #TCR_TG0_64K + b.ne .Lsz_16k + copy_page_body SZ_64K + ret +.Lsz_16k: + cmp x2, #TCR_TG0_16K + b.ne .Lsz_4k + copy_page_body SZ_16K + ret +.Lsz_4k: + copy_page_body SZ_4K ret SYM_FUNC_END(__pi_copy_page) SYM_FUNC_ALIAS(copy_page, __pi_copy_page) diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S index 5018ac03b6bf3..b4f6f5be0ec79 100644 --- a/arch/arm64/lib/mte.S +++ b/arch/arm64/lib/mte.S @@ -28,10 +28,13 @@ * x0 - address of the page to be cleared */ SYM_FUNC_START(mte_clear_page_tags) + get_page_size x3 + sub x3, x3, #1 // (PAGE_SIZE - 1) in x3 + multitag_transfer_size x1, x2 1: stgm xzr, [x0] add x0, x0, x1 - tst x0, #(PAGE_SIZE - 1) + tst x0, x3 b.ne 1b ret SYM_FUNC_END(mte_clear_page_tags) @@ -43,6 +46,9 @@ SYM_FUNC_END(mte_clear_page_tags) * x0 - address to the beginning of the page */ SYM_FUNC_START(mte_zero_clear_page_tags) + get_page_size x3 + sub x3, x3, #1 // (PAGE_SIZE - 1) in x3 + and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag mrs x1, dczid_el0 tbnz x1, #4, 2f // Branch if DC GZVA is prohibited @@ -52,12 +58,12 @@ SYM_FUNC_START(mte_zero_clear_page_tags) 1: dc gzva, x0 add x0, x0, x1 - tst x0, #(PAGE_SIZE - 1) + tst x0, x3 b.ne 1b ret 2: stz2g x0, [x0], #(MTE_GRANULE_SIZE * 2) - tst x0, #(PAGE_SIZE - 1) + tst x0, x3 b.ne 2b ret SYM_FUNC_END(mte_zero_clear_page_tags) @@ -68,6 +74,9 @@ SYM_FUNC_END(mte_zero_clear_page_tags) * x1 - address of the source page */ SYM_FUNC_START(mte_copy_page_tags) + get_page_size x7 + sub x7, x7, #1 // (PAGE_SIZE - 1) in x7 + mov x2, x0 mov x3, x1 multitag_transfer_size x5, x6 @@ -75,7 +84,7 @@ SYM_FUNC_START(mte_copy_page_tags) stgm x4, [x2] add x2, x2, x5 add x3, x3, x5 - tst x2, #(PAGE_SIZE - 1) + tst x2, x7 b.ne 1b ret SYM_FUNC_END(mte_copy_page_tags) @@ -137,6 +146,9 @@ SYM_FUNC_END(mte_copy_tags_to_user) * x1 - tag storage, MTE_PAGE_TAG_STORAGE bytes */ SYM_FUNC_START(mte_save_page_tags) + get_page_size x3 + sub x3, x3, #1 // (PAGE_SIZE - 1) in x3 + multitag_transfer_size x7, x5 1: mov x2, #0 @@ -149,7 +161,7 @@ SYM_FUNC_START(mte_save_page_tags) str x2, [x1], #8 - tst x0, #(PAGE_SIZE - 1) + tst x0, x3 b.ne 1b ret @@ -161,6 +173,9 @@ SYM_FUNC_END(mte_save_page_tags) * x1 - tag storage, MTE_PAGE_TAG_STORAGE bytes */ SYM_FUNC_START(mte_restore_page_tags) + get_page_size x3 + sub x3, x3, #1 // (PAGE_SIZE - 1) in x3 + multitag_transfer_size x7, x5 1: ldr x2, [x1], #8 @@ -170,7 +185,7 @@ SYM_FUNC_START(mte_restore_page_tags) tst x0, #0xFF b.ne 2b - tst x0, #(PAGE_SIZE - 1) + tst x0, x3 b.ne 1b ret From patchwork Mon Oct 14 10:59:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 027E5D1A443 for ; Mon, 14 Oct 2024 11:02:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE8E06B00B3; Mon, 14 Oct 2024 07:02:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D6D686B00B9; Mon, 14 Oct 2024 07:02:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC0926B00EA; Mon, 14 Oct 2024 07:02:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 91F8B6B00B3 for ; Mon, 14 Oct 2024 07:02:11 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id CE78BA0D9E for ; Mon, 14 Oct 2024 11:01:56 +0000 (UTC) X-FDA: 82671918252.21.32BE6F3 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf24.hostedemail.com (Postfix) with ESMTP id 74D15180009 for ; Mon, 14 Oct 2024 11:02:07 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf24.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903682; a=rsa-sha256; cv=none; b=Br2vZvO32NmEczUnvoMmzZG8meLJoi3LF0HY9OMYqw/BhpU4/Am7PBRpYs6lZBUwkzcYt9 KTmJylyTzq8TUa3XiOyXRr0nrB1tbmhRj8iZ6XSWqoaISSf0DvnK8kifWjBFtViH0tJ4r0 5nXtJ+Mmw72rwvS8KdyABuLfel4vQZg= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf24.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903682; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/eQV3kabbUGp6wsnBtNJdYTniQk07KbfuLXN2MYutAQ=; b=5kAV7zzLq3XVz6ox+vZ9ZBLNsf4SN07q2qS/Pp3R2Zb6OaB0uU78G8NZbLjck08wGFKc90 8Tb/4z2r/AciBCID0ec/PnzwlCi25cpRyqGA1RhLmBkgBwxlkASXOb4wgdELz22evq+MLg jLshlku/Coy5O/mtD6Ied+EoiN9f1nA= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 08F6A1713; Mon, 14 Oct 2024 04:02:38 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 44CBF3F51B; Mon, 14 Oct 2024 04:02:05 -0700 (PDT) From: Ryan Roberts To: "Aneesh Kumar K.V" , Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Nick Piggin , Oliver Upton , Peter Zijlstra , Will Deacon Cc: Ryan Roberts , kvmarm@lists.linux.dev, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 53/57] arm64: Runtime-fold pmd level Date: Mon, 14 Oct 2024 11:59:00 +0100 Message-ID: <20241014105912.3207374-53-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 74D15180009 X-Rspamd-Server: rspam01 X-Stat-Signature: asmbqjutnnjsm6i1rfu9apddq64p9m9e X-HE-Tag: 1728903727-523361 X-HE-Meta: U2FsdGVkX1+/NBW2Kk2coesliVjP2TXN9RQ0wHoEmc35Y1ilIHMk/3r+lr6XSevXIwAYCilc9kNvaPXo9VnWbL0/YWI/ik+rxOH9A4a8yw/FhSAL/21gxib1ZATGiQNWIHJuxksMBsmaJ4lVBct0ib6B9HXyLetnqc/4ZYeGg5dA1YgjtTqkTi/32377fOxnJMto09TAJyApsDj+e8l9tSPSgOlq4X5r7ulHS6ytLCdKhnh5/G/+IYzdsavx/Z/rc5FvXoIL+tYzqHIdA0xOd6gdof8cqX270YZgYwImMbQPNSOJKtEWJ17HrR4c7bmN+z3gE85lxuZd2WM48dXREjmydje36nYEy0hOUNbRSU5J+p8y4Dto4QyEio1XE/hv6EC48TSGv4Ff2WnmuxYX3SQ6PJr6WdiM6F3BuBpnDq8GMxeTdptFW52phEFgogrinPcUN5Ny4i+TBJwzCFMV6+e+w7GoKYE/YLB2fcmgcXvbQbt8fBrkTAVLDj6mjWqjXw7fpIGyMVFRTzdfDWhsd7IuTj7Yzd68oewfbb8YA4ARbPDsdmjXlpVKlX3CllSOFqtacKLNH2TqpP1DtSIbuHjwtgX3xSxMueB3qm2L9AhY6qwEQvXlzdXj4zEWGE8Ht2W6qQ1RYH8tokVukGiDbcf9KNxc7tPyFhSui/Z3fz3NzmNg5/EyhlMFUH4GMj0JPKjxQdMja47hUf7wmNnDj6ArfCxnEVe6pLfKihfzeJzCjDnxhDetwfsF40jOc75TK2jYpeEttLY2WTHvtGsJtol/jrXsGdIS7HtbMy7fVkWgAup/ercu6y5Kls33vp++tYmoy9gK91iOQ1EThnyJmlEKcQavQZgE8hCja9G81S9GO66SOs9qd5oWXVtJ6wA1RD3dpUtMK97kUnHhfRR/KkctN2N1WoZ5+nmr0Za1cMCh/SJRzVFqKAT2e703BfAnBDupeyWw4W9WilVK76p D/38PuGr rQQyZ6mAiKRBPM6+bqdvXtWUUFOg71vxZwCcnQp1fYfFeZd3fUk9SIS23uBPZIjXYAFhiayE0/wHXEbcVAuywlS/jc9Lidx0twVUFTKnvIoD92aov37aWH/11EPM14ZgG6EJ6bXxU/8b6epXzDUdxHI0enh5UroY8umlnfELuUkKABIR/8jJ/7e+ltg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For a given VA size, the number of levels of lookup depends on the page size. With boot-time page size selection, we therefore don't know how many levels of lookup we require until boot time. So we need to runtime-fold some levels of lookup. We already have code to runtime-fold p4d and pud levels; that exists for LPA2 fallback paths and can be repurposed for our needs. But pmd level also needs to support runtime folding; for example, 16K/36-bit and 64K/42-bit configs require only 2 levels. So let's add the required code. However, note that until we actually add the boot-time page size config, pgtable_l3_enabled() simply returns the compile-time determined answer. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/pgalloc.h | 16 +++- arch/arm64/include/asm/pgtable.h | 123 +++++++++++++++++++++++-------- arch/arm64/include/asm/tlb.h | 3 + arch/arm64/kernel/cpufeature.c | 4 +- arch/arm64/kvm/mmu.c | 9 +-- arch/arm64/mm/fixmap.c | 2 +- arch/arm64/mm/hugetlbpage.c | 16 ++-- arch/arm64/mm/init.c | 2 +- arch/arm64/mm/mmu.c | 2 +- arch/arm64/mm/ptdump.c | 3 +- 10 files changed, 126 insertions(+), 54 deletions(-) diff --git a/arch/arm64/include/asm/pgalloc.h b/arch/arm64/include/asm/pgalloc.h index 8ff5f2a2579e4..51cc2f32931d2 100644 --- a/arch/arm64/include/asm/pgalloc.h +++ b/arch/arm64/include/asm/pgalloc.h @@ -15,6 +15,7 @@ #define __HAVE_ARCH_PGD_FREE #define __HAVE_ARCH_PUD_FREE +#define __HAVE_ARCH_PMD_FREE #include #define PGD_SIZE (PTRS_PER_PGD * sizeof(pgd_t)) @@ -23,7 +24,8 @@ static inline void __pud_populate(pud_t *pudp, phys_addr_t pmdp, pudval_t prot) { - set_pud(pudp, __pud(__phys_to_pud_val(pmdp) | prot)); + if (pgtable_l3_enabled()) + set_pud(pudp, __pud(__phys_to_pud_val(pmdp) | prot)); } static inline void pud_populate(struct mm_struct *mm, pud_t *pudp, pmd_t *pmdp) @@ -33,6 +35,18 @@ static inline void pud_populate(struct mm_struct *mm, pud_t *pudp, pmd_t *pmdp) pudval |= (mm == &init_mm) ? PUD_TABLE_UXN : PUD_TABLE_PXN; __pud_populate(pudp, __pa(pmdp), pudval); } + +static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd) +{ + struct ptdesc *ptdesc = virt_to_ptdesc(pmd); + + if (!pgtable_l3_enabled()) + return; + + BUG_ON((unsigned long)pmd & (PAGE_SIZE-1)); + pagetable_pmd_dtor(ptdesc); + pagetable_free(ptdesc); +} #else static inline void __pud_populate(pud_t *pudp, phys_addr_t pmdp, pudval_t prot) { diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index fd47f70a42396..8ead41da715b0 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -672,15 +672,21 @@ extern pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, #define pmd_leaf_size(pmd) (pmd_cont(pmd) ? CONT_PMD_SIZE : PMD_SIZE) #define pte_leaf_size(pte) (pte_cont(pte) ? CONT_PTE_SIZE : PAGE_SIZE) -#if defined(CONFIG_ARM64_64K_PAGES) || CONFIG_PGTABLE_LEVELS < 3 -static inline bool pud_sect(pud_t pud) { return false; } -static inline bool pud_table(pud_t pud) { return true; } -#else -#define pud_sect(pud) ((pud_val(pud) & PUD_TYPE_MASK) == \ - PUD_TYPE_SECT) -#define pud_table(pud) ((pud_val(pud) & PUD_TYPE_MASK) == \ - PUD_TYPE_TABLE) -#endif +static inline bool pgtable_l3_enabled(void); + +static inline bool pud_sect(pud_t pud) +{ + if (PAGE_SIZE == SZ_64K || !pgtable_l3_enabled()) + return false; + return (pud_val(pud) & PUD_TYPE_MASK) == PUD_TYPE_SECT; +} + +static inline bool pud_table(pud_t pud) +{ + if (PAGE_SIZE == SZ_64K || !pgtable_l3_enabled()) + return true; + return (pud_val(pud) & PUD_TYPE_MASK) == PUD_TYPE_TABLE; +} extern pgd_t init_pg_dir[]; extern pgd_t init_pg_end[]; @@ -699,12 +705,10 @@ static inline bool in_swapper_pgdir(void *addr) static inline void set_pmd(pmd_t *pmdp, pmd_t pmd) { -#ifdef __PAGETABLE_PMD_FOLDED - if (in_swapper_pgdir(pmdp)) { + if (!pgtable_l3_enabled() && in_swapper_pgdir(pmdp)) { set_swapper_pgd((pgd_t *)pmdp, __pgd(pmd_val(pmd))); return; } -#endif /* __PAGETABLE_PMD_FOLDED */ WRITE_ONCE(*pmdp, pmd); @@ -749,20 +753,27 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd) #if CONFIG_PGTABLE_LEVELS > 2 +static __always_inline bool pgtable_l3_enabled(void) +{ + return true; +} + +static inline bool mm_pmd_folded(const struct mm_struct *mm) +{ + return !pgtable_l3_enabled(); +} +#define mm_pmd_folded mm_pmd_folded + #define pmd_ERROR(e) \ pr_err("%s:%d: bad pmd %016llx.\n", __FILE__, __LINE__, pmd_val(e)) -#define pud_none(pud) (!pud_val(pud)) -#define pud_bad(pud) (!pud_table(pud)) -#define pud_present(pud) pte_present(pud_pte(pud)) -#ifndef __PAGETABLE_PMD_FOLDED -#define pud_leaf(pud) (pud_present(pud) && !pud_table(pud)) -#else -#define pud_leaf(pud) false -#endif -#define pud_valid(pud) pte_valid(pud_pte(pud)) -#define pud_user(pud) pte_user(pud_pte(pud)) -#define pud_user_exec(pud) pte_user_exec(pud_pte(pud)) +#define pud_none(pud) (pgtable_l3_enabled() && !pud_val(pud)) +#define pud_bad(pud) (pgtable_l3_enabled() && !pud_table(pud)) +#define pud_present(pud) (!pgtable_l3_enabled() || pte_present(pud_pte(pud))) +#define pud_leaf(pud) (pgtable_l3_enabled() && pte_present(pud_pte(pud)) && !pud_table(pud)) +#define pud_valid(pud) (pgtable_l3_enabled() && pte_valid(pud_pte(pud))) +#define pud_user(pud) (pgtable_l3_enabled() && pte_user(pud_pte(pud))) +#define pud_user_exec(pud) (pgtable_l3_enabled() && pte_user_exec(pud_pte(pud))) static inline bool pgtable_l4_enabled(void); @@ -783,7 +794,8 @@ static inline void set_pud(pud_t *pudp, pud_t pud) static inline void pud_clear(pud_t *pudp) { - set_pud(pudp, __pud(0)); + if (pgtable_l3_enabled()) + set_pud(pudp, __pud(0)); } static inline phys_addr_t pud_page_paddr(pud_t pud) @@ -791,25 +803,74 @@ static inline phys_addr_t pud_page_paddr(pud_t pud) return __pud_to_phys(pud); } +#define pmd_index(addr) (((addr) >> PMD_SHIFT) & (PTRS_PER_PMD - 1)) + +static inline pmd_t *pud_to_folded_pmd(pud_t *pudp, unsigned long addr) +{ + return (pmd_t *)pudp; +} + static inline pmd_t *pud_pgtable(pud_t pud) { return (pmd_t *)__va(pud_page_paddr(pud)); } -/* Find an entry in the second-level page table. */ -#define pmd_offset_phys(dir, addr) (pud_page_paddr(READ_ONCE(*(dir))) + pmd_index(addr) * sizeof(pmd_t)) +static inline phys_addr_t pmd_offset_phys(pud_t *pudp, unsigned long addr) +{ + BUG_ON(!pgtable_l3_enabled()); + + return pud_page_paddr(READ_ONCE(*pudp)) + pmd_index(addr) * sizeof(pmd_t); +} + +static inline pmd_t *pmd_offset_lockless(pud_t *pudp, pud_t pud, + unsigned long addr) +{ + if (!pgtable_l3_enabled()) + return pud_to_folded_pmd(pudp, addr); + return (pmd_t *)__va(pud_page_paddr(pud)) + pmd_index(addr); +} +#define pmd_offset_lockless pmd_offset_lockless -#define pmd_set_fixmap(addr) ((pmd_t *)set_fixmap_offset(FIX_PMD, addr)) -#define pmd_set_fixmap_offset(pud, addr) pmd_set_fixmap(pmd_offset_phys(pud, addr)) -#define pmd_clear_fixmap() clear_fixmap(FIX_PMD) +static inline pmd_t *pmd_offset(pud_t *pudp, unsigned long addr) +{ + return pmd_offset_lockless(pudp, READ_ONCE(*pudp), addr); +} +#define pmd_offset pmd_offset -#define pud_page(pud) phys_to_page(__pud_to_phys(pud)) +static inline pmd_t *pmd_set_fixmap(unsigned long addr) +{ + if (!pgtable_l3_enabled()) + return NULL; + return (pmd_t *)set_fixmap_offset(FIX_PMD, addr); +} + +static inline pmd_t *pmd_set_fixmap_offset(pud_t *pudp, unsigned long addr) +{ + if (!pgtable_l3_enabled()) + return pud_to_folded_pmd(pudp, addr); + return pmd_set_fixmap(pmd_offset_phys(pudp, addr)); +} + +static inline void pmd_clear_fixmap(void) +{ + if (pgtable_l3_enabled()) + clear_fixmap(FIX_PMD); +} /* use ONLY for statically allocated translation tables */ -#define pmd_offset_kimg(dir,addr) ((pmd_t *)__phys_to_kimg(pmd_offset_phys((dir), (addr)))) +static inline pmd_t *pmd_offset_kimg(pud_t *pudp, u64 addr) +{ + if (!pgtable_l3_enabled()) + return pud_to_folded_pmd(pudp, addr); + return (pmd_t *)__phys_to_kimg(pmd_offset_phys(pudp, addr)); +} + +#define pud_page(pud) phys_to_page(__pud_to_phys(pud)) #else +static inline bool pgtable_l3_enabled(void) { return false; } + #define pud_valid(pud) false #define pud_page_paddr(pud) ({ BUILD_BUG(); 0; }) #define pud_user_exec(pud) pud_user(pud) /* Always 0 with folding */ diff --git a/arch/arm64/include/asm/tlb.h b/arch/arm64/include/asm/tlb.h index a947c6e784ed2..527630f0803c6 100644 --- a/arch/arm64/include/asm/tlb.h +++ b/arch/arm64/include/asm/tlb.h @@ -92,6 +92,9 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmdp, { struct ptdesc *ptdesc = virt_to_ptdesc(pmdp); + if (!pgtable_l3_enabled()) + return; + pagetable_pmd_dtor(ptdesc); tlb_remove_ptdesc(tlb, ptdesc); } diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index e5618423bb99d..663cc76569a27 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1923,8 +1923,10 @@ static int __init __kpti_install_ng_mappings(void *__unused) if (levels == 5 && !pgtable_l5_enabled()) levels = 4; - else if (levels == 4 && !pgtable_l4_enabled()) + if (levels == 4 && !pgtable_l4_enabled()) levels = 3; + if (levels == 3 && !pgtable_l3_enabled()) + levels = 2; remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings); diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 248a2d7ad6dbb..146ecdaaaf647 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1370,12 +1370,11 @@ static int get_vma_page_shift(struct vm_area_struct *vma, unsigned long hva) pa = (vma->vm_pgoff << PAGE_SHIFT) + (hva - vma->vm_start); -#ifndef __PAGETABLE_PMD_FOLDED - if ((hva & (PUD_SIZE - 1)) == (pa & (PUD_SIZE - 1)) && + if (pgtable_l3_enabled() && + (hva & (PUD_SIZE - 1)) == (pa & (PUD_SIZE - 1)) && ALIGN_DOWN(hva, PUD_SIZE) >= vma->vm_start && ALIGN(hva, PUD_SIZE) <= vma->vm_end) return PUD_SHIFT; -#endif if ((hva & (PMD_SIZE - 1)) == (pa & (PMD_SIZE - 1)) && ALIGN_DOWN(hva, PMD_SIZE) >= vma->vm_start && @@ -1487,12 +1486,10 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, vma_shift = get_vma_page_shift(vma, hva); } -#ifndef __PAGETABLE_PMD_FOLDED - if (vma_shift == PUD_SHIFT) { + if (pgtable_l3_enabled() && vma_shift == PUD_SHIFT) { if (!fault_supports_stage2_huge_mapping(memslot, hva, PUD_SIZE)) vma_shift = PMD_SHIFT; } -#endif if (vma_shift == CONT_PMD_SHIFT) { vma_shift = PMD_SHIFT; } diff --git a/arch/arm64/mm/fixmap.c b/arch/arm64/mm/fixmap.c index a0dcf2375ccb4..f2c6678046a96 100644 --- a/arch/arm64/mm/fixmap.c +++ b/arch/arm64/mm/fixmap.c @@ -87,7 +87,7 @@ static void __init early_fixmap_init_pud(p4d_t *p4dp, unsigned long addr, p4d_t p4d = READ_ONCE(*p4dp); pud_t *pudp; - if (CONFIG_PGTABLE_LEVELS > 3 && !p4d_none(p4d) && + if (ptg_pgtable_levels > 3 && !p4d_none(p4d) && p4d_page_paddr(p4d) != __pa_symbol(bm_pud)) { /* * We only end up here if the kernel mapping and the fixmap diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index bc98c20655bba..2add0839179e3 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -51,10 +51,9 @@ void __init arm64_hugetlb_cma_reserve(void) static bool __hugetlb_valid_size(unsigned long size) { -#ifndef __PAGETABLE_PMD_FOLDED - if (size == PUD_SIZE) + if (pgtable_l3_enabled() && size == PUD_SIZE) return pud_sect_supported(); -#endif + if (size == CONT_PMD_SIZE || size == PMD_SIZE || size == CONT_PTE_SIZE) return true; @@ -100,13 +99,10 @@ static inline int num_contig_ptes(unsigned long size, size_t *pgsize) *pgsize = size; -#ifndef __PAGETABLE_PMD_FOLDED - if (size == PUD_SIZE) { + if (pgtable_l3_enabled() && size == PUD_SIZE) { if (pud_sect_supported()) contig_ptes = 1; - } else -#endif - if (size == PMD_SIZE) { + } else if (size == PMD_SIZE) { contig_ptes = 1; } else if (size == CONT_PMD_SIZE) { *pgsize = PMD_SIZE; @@ -331,10 +327,8 @@ unsigned long hugetlb_mask_last_page(struct hstate *h) { unsigned long hp_size = huge_page_size(h); -#ifndef __PAGETABLE_PMD_FOLDED - if (hp_size == PUD_SIZE) + if (pgtable_l3_enabled() && hp_size == PUD_SIZE) return PGDIR_SIZE - PUD_SIZE; -#endif if (hp_size == CONT_PMD_SIZE) return PUD_SIZE - CONT_PMD_SIZE; if (hp_size == PMD_SIZE) diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 4d24034418b39..62587104f30d8 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -396,7 +396,7 @@ void __init mem_init(void) * scratch using the virtual address range and page size. */ VM_BUG_ON(ARM64_HW_PGTABLE_LEVELS(CONFIG_ARM64_VA_BITS) != - CONFIG_PGTABLE_LEVELS); + ptg_pgtable_levels); if (PAGE_SIZE >= 16384 && get_num_physpages() <= 128) { extern int sysctl_overcommit_memory; diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index ad7fd3fda705a..b78a341cd9e70 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1046,7 +1046,7 @@ static void free_empty_pmd_table(pud_t *pudp, unsigned long addr, free_empty_pte_table(pmdp, addr, next, floor, ceiling); } while (addr = next, addr < end); - if (CONFIG_PGTABLE_LEVELS <= 2) + if (!pgtable_l3_enabled()) return; if (!pgtable_range_aligned(start, end, floor, ceiling, PUD_MASK)) diff --git a/arch/arm64/mm/ptdump.c b/arch/arm64/mm/ptdump.c index 6986827e0d645..045a4188afc10 100644 --- a/arch/arm64/mm/ptdump.c +++ b/arch/arm64/mm/ptdump.c @@ -230,7 +230,8 @@ static void note_page(struct ptdump_state *pt_st, unsigned long addr, int level, /* check if the current level has been folded dynamically */ if ((level == 1 && mm_p4d_folded(st->mm)) || - (level == 2 && mm_pud_folded(st->mm))) + (level == 2 && mm_pud_folded(st->mm)) || + (level == 3 && mm_pmd_folded(st->mm))) level = 0; if (level >= 0) From patchwork Mon Oct 14 10:59:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834722 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 234FFD1A45A for ; Mon, 14 Oct 2024 11:02:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EF7A46B00B9; Mon, 14 Oct 2024 07:02:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EA66C6B00EB; Mon, 14 Oct 2024 07:02:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CFA6C6B00EC; Mon, 14 Oct 2024 07:02:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B07726B00B9 for ; Mon, 14 Oct 2024 07:02:13 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0DEE3160D36 for ; Mon, 14 Oct 2024 11:02:05 +0000 (UTC) X-FDA: 82671918378.12.77457C8 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf27.hostedemail.com (Postfix) with ESMTP id B746E40023 for ; Mon, 14 Oct 2024 11:02:05 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf27.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903616; a=rsa-sha256; cv=none; b=HXFJyjw6cdZkX7f/iFE0ReyhXJQFfBwBhtYHtIwu1Lw8dNg8qCu90tufZKBkmQpcX7wRxH ftkRhDla4+m8dPgquGHhhKeSIfMMarJfzf7cCIuin0NZpxVy/J2unbkC1I+51xJh4hXHTD mZrPbSSv6e+YH746Ac0MccmE61WaH7s= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf27.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903616; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=24J2s1s+JtIBsBU/tXNcd36ly+a41o9WkxedPu4yJ84=; b=MedPfjvcbTdTc3LemoFxL7KxULdfYQuHZi4pnoNU7gvjCI4+ap6vwqrTSI0HI69cgvjCsk dxx9c+rZrA0Fc1kr8xs4VvvN69bHiuL5tGU3U9TniK8IzQKLX+ISk3P/aqXM1XAx4mSR+6 dI5u/jB5SiDpiAwu78zb34NMcuiuegg= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AD6AF1763; Mon, 14 Oct 2024 04:02:40 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id AE3BE3F51B; Mon, 14 Oct 2024 04:02:08 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 54/57] arm64: Support runtime folding in idmap_kpti_install_ng_mappings Date: Mon, 14 Oct 2024 11:59:01 +0100 Message-ID: <20241014105912.3207374-54-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: B746E40023 X-Stat-Signature: 6buarbkndns9b6hprb9fh8rg355kk4fe X-Rspam-User: X-HE-Tag: 1728903725-331710 X-HE-Meta: U2FsdGVkX1+wEFsunBVRkGhUWz1U06y2u2eKfpuMPPvHTiwl1IaP7w27IKV9Z1mhPrDrC7IrKtgo4eyhqqhSUp/eRCWUnEvxLULvoax49NJLNGZdo/8pH2wJGpdt5S1tfE9e4bTLUkwEfROA4mJH7ATGmx4ixmQODJZTqn/tjcIX+Ya9Cl44xvQKGQH2VO/ZXdTNpEEcahA8ro/LS+jEdEcJ/YNeGE9pVkDT5ZfhK7j9NVMwyAqq4H8Bxke90vMP+BnN680eOP6Pwak2CFsGiaZC/xni7wyRqSGcg2sb2axwaiqokhKS/x/347OcvOmMOn3dqlTauCRlGZfd6WR2TIE/1gr5/znIEwKlUVs/9Pq3XVKlwlj+e/H/T7u1vOCzrAEAtqH3UxJ8xGtG3XnSoVot10/yxpOLZxs0WlDLEmok+C/x/IZWMYoqfqOH87VY5tD9qpfTm44bMhUtqDVOcOOkM0E8Kgx9wPs2SeSmeIVB2YxOxE5SLF/JicEvtVPPHnvGkNAhC9GNY5YMR63O2XMwFcxlgokbEkscpfm2Fy1KIsqcMAaRaZUzVwMsQDQDmoK1cKDosoe509wFacsXYNKnMd6Ql3l5UT+9Eq33zrtIys3scjA7L8N2pOIhGULBxclYQ8YHL9Sgj1Tqluk6DPOn8EhsHjn++/A1rmTFZmPXDug4H3XRVS2jbl4JQZDo+hC/WrRXRr/4IqaxCuEobfOff5ZQfrgPvTl4O9tVW5zbkK1sdNC6Xzqpzu6YT3NOs0I/laJ+RbPb52v3/Y2zJzN4pIbt7c5bKHRStklIHVY7VdA6HL+d5n4yRNefn9t5cF/xyPfuJutsfJYKUT/MgIMS9wk0lBw8qmvXIOmuDUEe3RhI0xJ+nSw43UwshYEjztd94AM5mQEE4jBvRjFZ0BojsnQ4GeFKvhtxabjZoFDL0+TN6VcH06nxuiyExxmt1lnBUzgkhiRHKAzi8/R 133mltlZ W8AT4gNKVJTYXKnh8u8WP0o0PUvr9g4DnupSumMEZisr7KU0YkZqtycCatLFVS3f2R521F6gLclkFaLD4sBdwbICcQVA++x64kiC7tAF6n18KREbxf8a99KjZ69jdhT8QuzWAArxkxkWUxNL4+A+9vvNjq77pU81rM/wNuor7orbHX9n1RRlJknTQC6bHzsR/UvIv5afLM8zgMV1yMwWAnXkZZIh43pAjwqE/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: TODO: Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/assembler.h | 5 ++ arch/arm64/kernel/cpufeature.c | 21 +++++- arch/arm64/mm/proc.S | 107 ++++++++++++++++++++++------- 3 files changed, 108 insertions(+), 25 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 6424fd6be1cbe..0cfa7c3efd214 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -919,6 +919,11 @@ alternative_cb_end value_for_page_size \val, \val, SZ_4K, SZ_16K, SZ_64K .endm + .macro get_page_shift, val + get_tg0 \val + value_for_page_size \val, \val, ARM64_PAGE_SHIFT_4K, ARM64_PAGE_SHIFT_16K, ARM64_PAGE_SHIFT_64K + .endm + .macro get_page_mask, val get_tg0 \val value_for_page_size \val, \val, (~(SZ_4K-1)), (~(SZ_16K-1)), (~(SZ_64K-1)) diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 663cc76569a27..ee94de556d3f0 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1908,11 +1908,27 @@ static phys_addr_t __init kpti_ng_pgd_alloc(int shift) return kpti_ng_temp_alloc; } +struct install_ng_pgtable_geometry { + unsigned long ptrs_per_pte; + unsigned long ptrs_per_pmd; + unsigned long ptrs_per_pud; + unsigned long ptrs_per_p4d; + unsigned long ptrs_per_pgd; +}; + static int __init __kpti_install_ng_mappings(void *__unused) { - typedef void (kpti_remap_fn)(int, int, phys_addr_t, unsigned long); + typedef void (kpti_remap_fn)(int, int, phys_addr_t, unsigned long, + struct install_ng_pgtable_geometry *); extern kpti_remap_fn idmap_kpti_install_ng_mappings; kpti_remap_fn *remap_fn; + struct install_ng_pgtable_geometry geometry = { + .ptrs_per_pte = PTRS_PER_PTE, + .ptrs_per_pmd = PTRS_PER_PMD, + .ptrs_per_pud = PTRS_PER_PUD, + .ptrs_per_p4d = PTRS_PER_P4D, + .ptrs_per_pgd = PTRS_PER_PGD, + }; int cpu = smp_processor_id(); int levels = CONFIG_PGTABLE_LEVELS; @@ -1957,7 +1973,8 @@ static int __init __kpti_install_ng_mappings(void *__unused) } cpu_install_idmap(); - remap_fn(cpu, num_online_cpus(), kpti_ng_temp_pgd_pa, KPTI_NG_TEMP_VA); + remap_fn(cpu, num_online_cpus(), kpti_ng_temp_pgd_pa, KPTI_NG_TEMP_VA, + &geometry); cpu_uninstall_idmap(); if (!cpu) { diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index ab5aa84923524..11bf6ba6dac33 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -190,7 +190,7 @@ SYM_FUNC_ALIAS(__pi_idmap_cpu_replace_ttbr1, idmap_cpu_replace_ttbr1) .pushsection ".idmap.text", "a" .macro pte_to_phys, phys, pte - and \phys, \pte, #PTE_ADDR_LOW + and \phys, \pte, pte_addr_low #ifdef CONFIG_ARM64_PA_BITS_52 and \pte, \pte, #PTE_ADDR_HIGH orr \phys, \phys, \pte, lsl #PTE_ADDR_HIGH_SHIFT @@ -198,7 +198,8 @@ SYM_FUNC_ALIAS(__pi_idmap_cpu_replace_ttbr1, idmap_cpu_replace_ttbr1) .endm .macro kpti_mk_tbl_ng, type, num_entries - add end_\type\()p, cur_\type\()p, #\num_entries * 8 + lsl scratch, \num_entries, #3 + add end_\type\()p, cur_\type\()p, scratch .Ldo_\type: ldr \type, [cur_\type\()p], #8 // Load the entry and advance tbz \type, #0, .Lnext_\type // Skip invalid and @@ -220,14 +221,18 @@ SYM_FUNC_ALIAS(__pi_idmap_cpu_replace_ttbr1, idmap_cpu_replace_ttbr1) .macro kpti_map_pgtbl, type, level str xzr, [temp_pte, #8 * (\level + 2)] // break before make dsb nshst - add pte, temp_pte, #PAGE_SIZE * (\level + 2) + mov scratch, #(\level + 2) + mul scratch, scratch, page_size + add pte, temp_pte, scratch lsr pte, pte, #12 tlbi vaae1, pte dsb nsh isb phys_to_pte pte, cur_\type\()p - add cur_\type\()p, temp_pte, #PAGE_SIZE * (\level + 2) + mov scratch, #(\level + 2) + mul scratch, scratch, page_size + add cur_\type\()p, temp_pte, scratch orr pte, pte, pte_flags str pte, [temp_pte, #8 * (\level + 2)] dsb nshst @@ -235,7 +240,8 @@ SYM_FUNC_ALIAS(__pi_idmap_cpu_replace_ttbr1, idmap_cpu_replace_ttbr1) /* * void __kpti_install_ng_mappings(int cpu, int num_secondaries, phys_addr_t temp_pgd, - * unsigned long temp_pte_va) + * unsigned long temp_pte_va, + * struct install_ng_pgtable_geometry *geometry) * * Called exactly once from stop_machine context by each CPU found during boot. */ @@ -251,6 +257,8 @@ SYM_TYPED_FUNC_START(idmap_kpti_install_ng_mappings) temp_pgd_phys .req x2 swapper_ttb .req x3 flag_ptr .req x4 + geometry .req x4 + scratch .req x4 cur_pgdp .req x5 end_pgdp .req x6 pgd .req x7 @@ -264,18 +272,45 @@ SYM_TYPED_FUNC_START(idmap_kpti_install_ng_mappings) valid .req x17 cur_p4dp .req x19 end_p4dp .req x20 - - mov x5, x3 // preserve temp_pte arg - mrs swapper_ttb, ttbr1_el1 - adr_l flag_ptr, __idmap_kpti_flag + page_size .req x21 + ptrs_per_pte .req x22 + ptrs_per_pmd .req x23 + ptrs_per_pud .req x24 + ptrs_per_p4d .req x25 + ptrs_per_pgd .req x26 + pte_addr_low .req x27 cbnz cpu, __idmap_kpti_secondary -#if CONFIG_PGTABLE_LEVELS > 4 - stp x29, x30, [sp, #-32]! + /* Preserve callee-saved registers */ + stp x19, x20, [sp, #-96]! + stp x21, x22, [sp, #80] + stp x23, x24, [sp, #64] + stp x25, x26, [sp, #48] + stp x27, x28, [sp, #32] + stp x29, x30, [sp, #16] mov x29, sp - stp x19, x20, [sp, #16] -#endif + + /* Load pgtable geometry parameters */ + get_page_size page_size + ldr ptrs_per_pte, [geometry, #0] + ldr ptrs_per_pmd, [geometry, #8] + ldr ptrs_per_pud, [geometry, #16] + ldr ptrs_per_p4d, [geometry, #24] + ldr ptrs_per_pgd, [geometry, #32] + + /* Precalculate pte_addr_low mask */ + get_page_shift x0 + mov pte_addr_low, #50 + sub pte_addr_low, pte_addr_low, x0 + mov scratch, #1 + lsl pte_addr_low, scratch, pte_addr_low + sub pte_addr_low, pte_addr_low, #1 + lsl pte_addr_low, pte_addr_low, x0 + + mov temp_pte, x3 + mrs swapper_ttb, ttbr1_el1 + adr_l flag_ptr, __idmap_kpti_flag /* We're the boot CPU. Wait for the others to catch up */ sevl @@ -290,7 +325,6 @@ SYM_TYPED_FUNC_START(idmap_kpti_install_ng_mappings) msr ttbr1_el1, temp_pgd_phys isb - mov temp_pte, x5 mov_q pte_flags, KPTI_NG_PTE_FLAGS /* Everybody is enjoying the idmap, so we can rewrite swapper. */ @@ -320,7 +354,7 @@ alternative_else_nop_endif /* PGD */ adrp cur_pgdp, swapper_pg_dir kpti_map_pgtbl pgd, -1 - kpti_mk_tbl_ng pgd, PTRS_PER_PGD + kpti_mk_tbl_ng pgd, ptrs_per_pgd /* Ensure all the updated entries are visible to secondary CPUs */ dsb ishst @@ -331,21 +365,33 @@ alternative_else_nop_endif isb /* Set the flag to zero to indicate that we're all done */ + adr_l flag_ptr, __idmap_kpti_flag str wzr, [flag_ptr] -#if CONFIG_PGTABLE_LEVELS > 4 - ldp x19, x20, [sp, #16] - ldp x29, x30, [sp], #32 -#endif + + /* Restore callee-saved registers */ + ldp x29, x30, [sp, #16] + ldp x27, x28, [sp, #32] + ldp x25, x26, [sp, #48] + ldp x23, x24, [sp, #64] + ldp x21, x22, [sp, #80] + ldp x19, x20, [sp], #96 + ret .Lderef_pgd: /* P4D */ .if CONFIG_PGTABLE_LEVELS > 4 p4d .req x30 + cmp ptrs_per_p4d, #1 + b.eq .Lfold_p4d pte_to_phys cur_p4dp, pgd kpti_map_pgtbl p4d, 0 - kpti_mk_tbl_ng p4d, PTRS_PER_P4D + kpti_mk_tbl_ng p4d, ptrs_per_p4d b .Lnext_pgd +.Lfold_p4d: + mov p4d, pgd // fold to next level + mov cur_p4dp, end_p4dp // must be equal to terminate loop + b .Lderef_p4d .else /* CONFIG_PGTABLE_LEVELS <= 4 */ p4d .req pgd .set .Lnext_p4d, .Lnext_pgd @@ -355,10 +401,16 @@ alternative_else_nop_endif /* PUD */ .if CONFIG_PGTABLE_LEVELS > 3 pud .req x10 + cmp ptrs_per_pud, #1 + b.eq .Lfold_pud pte_to_phys cur_pudp, p4d kpti_map_pgtbl pud, 1 - kpti_mk_tbl_ng pud, PTRS_PER_PUD + kpti_mk_tbl_ng pud, ptrs_per_pud b .Lnext_p4d +.Lfold_pud: + mov pud, p4d // fold to next level + mov cur_pudp, end_pudp // must be equal to terminate loop + b .Lderef_pud .else /* CONFIG_PGTABLE_LEVELS <= 3 */ pud .req pgd .set .Lnext_pud, .Lnext_pgd @@ -368,10 +420,16 @@ alternative_else_nop_endif /* PMD */ .if CONFIG_PGTABLE_LEVELS > 2 pmd .req x13 + cmp ptrs_per_pmd, #1 + b.eq .Lfold_pmd pte_to_phys cur_pmdp, pud kpti_map_pgtbl pmd, 2 - kpti_mk_tbl_ng pmd, PTRS_PER_PMD + kpti_mk_tbl_ng pmd, ptrs_per_pmd b .Lnext_pud +.Lfold_pmd: + mov pmd, pud // fold to next level + mov cur_pmdp, end_pmdp // must be equal to terminate loop + b .Lderef_pmd .else /* CONFIG_PGTABLE_LEVELS <= 2 */ pmd .req pgd .set .Lnext_pmd, .Lnext_pgd @@ -381,7 +439,7 @@ alternative_else_nop_endif /* PTE */ pte_to_phys cur_ptep, pmd kpti_map_pgtbl pte, 3 - kpti_mk_tbl_ng pte, PTRS_PER_PTE + kpti_mk_tbl_ng pte, ptrs_per_pte b .Lnext_pmd .unreq cpu @@ -408,6 +466,9 @@ alternative_else_nop_endif /* Secondary CPUs end up here */ __idmap_kpti_secondary: + mrs swapper_ttb, ttbr1_el1 + adr_l flag_ptr, __idmap_kpti_flag + /* Uninstall swapper before surgery begins */ __idmap_cpu_set_reserved_ttbr1 x16, x17 From patchwork Mon Oct 14 10:59:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF800D1A443 for ; Mon, 14 Oct 2024 11:02:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97B076B00EB; Mon, 14 Oct 2024 07:02:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 903CA6B00ED; Mon, 14 Oct 2024 07:02:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A2B16B00EE; Mon, 14 Oct 2024 07:02:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5A6DC6B00EB for ; Mon, 14 Oct 2024 07:02:16 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id EBB96ABCCA for ; Mon, 14 Oct 2024 11:02:00 +0000 (UTC) X-FDA: 82671918504.21.CD867FF Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf04.hostedemail.com (Postfix) with ESMTP id 4BAF240014 for ; Mon, 14 Oct 2024 11:02:06 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf04.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903619; a=rsa-sha256; cv=none; b=TUWZxyovyFQ34bEznMN+Hsgev1lsiAAf98q5pQ6esY8bxBEt+w6217eVwG/6P+oD84MFuM p1io+r0moDVcTRZB6SuFD0u5UE22/gnwmbwzsq9Q6gYkUkGSQX6GsE2u8B5BsJHc2Ww8j5 2rXMk+zJwuwG9j7OGy5ivaPXWbOj1Is= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf04.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903619; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BkgcTQ3nJpwwleh27xwrJdMVBeha9+gRlJOOjRb7rV4=; b=xKD3B3/xGJVAwQlG0nalqNs/1M88gbwRuXXOsKNv+uKck+01dUrWG4S2xHrA+WN+Vqvphs DgC84iwWDC7+CxmfHiqIPOZAIFSqt5jDRCss2QFv+wGywQWPwTAEbYYXxWQjaa8UeKKlZE ahJmQtBUNFjtLshjpmQfLpwUNOHh+qM= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5E32F1756; Mon, 14 Oct 2024 04:02:43 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 5E9263F51B; Mon, 14 Oct 2024 04:02:11 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Will Deacon Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 55/57] arm64: TRAMP_VALIAS is no longer compile-time constant Date: Mon, 14 Oct 2024 11:59:02 +0100 Message-ID: <20241014105912.3207374-55-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 4BAF240014 X-Stat-Signature: 38k7zu73khbband9uod66p77th4s5bwt X-Rspam-User: X-HE-Tag: 1728903726-328534 X-HE-Meta: U2FsdGVkX19N8OycGUm1/+1Z4qsIJefY3N7yGym7HKBiNuBTXyQ5UJk+C/v2GscuFTx4egGe5dc88BJobT5nKOWW2uDbQy6Uo7frWm3oiOi1rEjttW0ghFck84dpaDoWHFEMttQK01D1cSoMmLKnqBGV7mPh1OPzTjCM42JG3H3YsqqRD6P119LQxv2DfhmrveCm2HAruWcfZhjSGrZi9LjUnCBBSbQTm6JyygNugKwGmL2wuVMua6uBhUSX9RUwP9dw/gGvS+3QAaQ5dE7hyhTdtLmCSYFubU5ovMTLTiX2T5dnrqjhipOswVAfPNBqY8TyCiIWulQrn7YFTTeahf7nfbCfLNqijOtBo4lMmqJKZq2rKFaLIthzhLHIc+xx0Z7M3UG4Eo0ELQ10GkjC/5BxAS7MqDxnG5eX7vW16pkrusZu73+UbRp963ZfcthQks4ocKKiviN9dSHsBzMHoDvncxz5gClpfdyfYvuZCAFvb4z0HrcuUkbPkYY4dDNRznyQhTq+igbGQfB7dFYbnooilHW7m7ckzwySjUoQVm3JvDfrCrF6xb65iiCByERB40YdTBGDs6iq2HlDIztmbtDYJQ4tWdilYtK/NYvoxl78gPQDXa2ll5q1TcMiAgSDnTzUVQw2b9qOuUS0y77R0Vlkoo3um1udcdf5cLCJ2epRaaNAD6k4fodwdE2PhJwtdOsIn3P/17SZsNmeC1iST3VpXGg56xh+bF1hxufSxxfQ1u4iXjWr1xVGTeC4qYwGPQgHbd55afDO5AO21zPoOGZd+yMXZThUaG5MPq+nlLZDx8g58l4W9LJyY5Y3rJ3Zl3jEVj3eHp1vhm42AlviKnzwcvQzwMKe58FFs/sih4WdCiSSbgU3yZiDlKzREVmypXQ1P2m+E0nHOerd3ke2ju1hkasjQiDj/K5e8muPw6ssKRQFq9opv5rXb/94b6/UwNPPOtlaAeb/6C0GrYO 4QFHwS1D 79NGLGg3pj5cuTSfTQGn/i/s+4e4CXU3//jpOATLkn0LWW25Q1U1GNdGY78TlJYUBJ5AlRqqqeo62mJrs9bstcBRcmLm48RJCOIHYE/5ZS6KrqCRdjaf3fQZfKaveZKAKI10EpBRtHsTb0w6HLbQ9SoETK/Ht0ssbIxPTVvQoB8znV7L56dZkSlme/yWIZseZr+za13dDlPRxU240+XooaK0p29pLVjbFLU9X X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When boot-time page size is in operation, TRAMP_VALIAS is no longer a compile-time constant, because the VA of a fixmap slot depends upon PAGE_SIZE. Let's handle this by instead exporting the slot index, FIX_ENTRY_TRAMP_BEGIN,to assembly, then do the TRAMP_VALIAS calculation per page size and use alternatives to decide which variant to activate. Note that for the tramp_map_kernel case, we are one instruction short of space in the vector to have NOPs for all 3 page size variants. So we do if/else for 16K/64K and branch around it for the 4K case. This saves 2 instructions. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/kernel/asm-offsets.c | 2 +- arch/arm64/kernel/entry.S | 50 ++++++++++++++++++++++++++------- 2 files changed, 41 insertions(+), 11 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index f32b8d7f00b2a..c45fa3e281884 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -172,7 +172,7 @@ int main(void) DEFINE(ARM64_FTR_SYSVAL, offsetof(struct arm64_ftr_reg, sys_val)); BLANK(); #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 - DEFINE(TRAMP_VALIAS, TRAMP_VALIAS); + DEFINE(FIX_ENTRY_TRAMP_BEGIN, FIX_ENTRY_TRAMP_BEGIN); #endif #ifdef CONFIG_ARM_SDE_INTERFACE DEFINE(SDEI_EVENT_INTREGS, offsetof(struct sdei_registered_event, interrupted_regs)); diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 7ef0e127b149f..ba47dc8672c04 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -101,11 +101,27 @@ .org .Lventry_start\@ + 128 // Did we overflow the ventry slot? .endm +#define TRAMP_VALIAS(page_shift) (FIXADDR_TOP - (FIX_ENTRY_TRAMP_BEGIN << (page_shift))) + .macro tramp_alias, dst, sym - .set .Lalias\@, TRAMP_VALIAS + \sym - .entry.tramp.text - movz \dst, :abs_g2_s:.Lalias\@ - movk \dst, :abs_g1_nc:.Lalias\@ - movk \dst, :abs_g0_nc:.Lalias\@ +alternative_if ARM64_USE_PAGE_SIZE_4K + .set .Lalias4k\@, TRAMP_VALIAS(ARM64_PAGE_SHIFT_4K) + \sym - .entry.tramp.text + movz \dst, :abs_g2_s:.Lalias4k\@ + movk \dst, :abs_g1_nc:.Lalias4k\@ + movk \dst, :abs_g0_nc:.Lalias4k\@ +alternative_else_nop_endif +alternative_if ARM64_USE_PAGE_SIZE_16K + .set .Lalias16k\@, TRAMP_VALIAS(ARM64_PAGE_SHIFT_16K) + \sym - .entry.tramp.text + movz \dst, :abs_g2_s:.Lalias16k\@ + movk \dst, :abs_g1_nc:.Lalias16k\@ + movk \dst, :abs_g0_nc:.Lalias16k\@ +alternative_else_nop_endif +alternative_if ARM64_USE_PAGE_SIZE_64K + .set .Lalias64k\@, TRAMP_VALIAS(ARM64_PAGE_SHIFT_64K) + \sym - .entry.tramp.text + movz \dst, :abs_g2_s:.Lalias64k\@ + movk \dst, :abs_g1_nc:.Lalias64k\@ + movk \dst, :abs_g0_nc:.Lalias64k\@ +alternative_else_nop_endif .endm /* @@ -627,16 +643,30 @@ SYM_CODE_END(ret_to_user) bic \tmp, \tmp, #USER_ASID_FLAG msr ttbr1_el1, \tmp #ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003 -alternative_if ARM64_WORKAROUND_QCOM_FALKOR_E1003 +alternative_if_not ARM64_WORKAROUND_QCOM_FALKOR_E1003 + b .Lskip_falkor_e1003\@ +alternative_else_nop_endif /* ASID already in \tmp[63:48] */ - movk \tmp, #:abs_g2_nc:(TRAMP_VALIAS >> 12) - movk \tmp, #:abs_g1_nc:(TRAMP_VALIAS >> 12) - /* 2MB boundary containing the vectors, so we nobble the walk cache */ - movk \tmp, #:abs_g0_nc:((TRAMP_VALIAS & ~(SZ_2M - 1)) >> 12) +alternative_if ARM64_USE_PAGE_SIZE_4K + movk \tmp, #:abs_g2_nc:(TRAMP_VALIAS(ARM64_PAGE_SHIFT_4K) >> 12) + movk \tmp, #:abs_g1_nc:(TRAMP_VALIAS(ARM64_PAGE_SHIFT_4K) >> 12) + movk \tmp, #:abs_g0_nc:((TRAMP_VALIAS(ARM64_PAGE_SHIFT_4K) & ~(SZ_2M - 1)) >> 12) + b .Lfinish_falkor_e1003\@ +alternative_else_nop_endif +alternative_if ARM64_USE_PAGE_SIZE_16K + movk \tmp, #:abs_g2_nc:(TRAMP_VALIAS(ARM64_PAGE_SHIFT_16K) >> 12) + movk \tmp, #:abs_g1_nc:(TRAMP_VALIAS(ARM64_PAGE_SHIFT_16K) >> 12) + movk \tmp, #:abs_g0_nc:((TRAMP_VALIAS(ARM64_PAGE_SHIFT_16K) & ~(SZ_2M - 1)) >> 12) +alternative_else /* ARM64_USE_PAGE_SIZE_64K */ + movk \tmp, #:abs_g2_nc:(TRAMP_VALIAS(ARM64_PAGE_SHIFT_64K) >> 12) + movk \tmp, #:abs_g1_nc:(TRAMP_VALIAS(ARM64_PAGE_SHIFT_64K) >> 12) + movk \tmp, #:abs_g0_nc:((TRAMP_VALIAS(ARM64_PAGE_SHIFT_64K) & ~(SZ_2M - 1)) >> 12) +alternative_endif +.Lfinish_falkor_e1003\@: isb tlbi vae1, \tmp dsb nsh -alternative_else_nop_endif +.Lskip_falkor_e1003\@: #endif /* CONFIG_QCOM_FALKOR_ERRATUM_1003 */ .endm From patchwork Mon Oct 14 10:59:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834724 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2C73D1A45A for ; Mon, 14 Oct 2024 11:02:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB1AA6B00ED; Mon, 14 Oct 2024 07:02:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C6B76B00EF; Mon, 14 Oct 2024 07:02:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 864B46B00F0; Mon, 14 Oct 2024 07:02:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 66E066B00ED for ; Mon, 14 Oct 2024 07:02:19 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 97E7F80DB5 for ; Mon, 14 Oct 2024 11:02:12 +0000 (UTC) X-FDA: 82671918714.29.D281452 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf06.hostedemail.com (Postfix) with ESMTP id 070D8180012 for ; Mon, 14 Oct 2024 11:02:12 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf06.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903691; a=rsa-sha256; cv=none; b=ptULGsFQWPVcJXpxFZl17huKO5cmX7UuOz2DH9vfkudtdJsfrG7I2TLdduzJibUQHs7ziN Wq8yJIuxET7omRSmfxbscqeVmKhAeMH9UVYFIMsGVWVbqYozSsZ7JtU5sd6QggakF/842X BRHBQJf2YG8wzCca6GFgIyi9ZjX+n88= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf06.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903691; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cPtmn6bEPN9e8rdSq6k9AO9DyT36CNVCz9b2AR0mIuI=; b=Of91oKTfsa1uzFtbYRWasH6yNVbkaAQuKbH41W2J412cEwDS74FkOReMjM/Flbx07rjgcS k6ioPbzk0nbObDZe5ikhQdVG4Idn5XvZL5aR0B01z/5jOqIrGDkPIGKxUcXz1yMT1oDCfS XqtSszoj7CuGcPuKvktXG94mq5ao+t8= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6C0D71756; Mon, 14 Oct 2024 04:02:46 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0FB423F51B; Mon, 14 Oct 2024 04:02:13 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Oliver Upton , Will Deacon Cc: Ryan Roberts , kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 56/57] arm64: Determine THREAD_SIZE at boot-time Date: Mon, 14 Oct 2024 11:59:03 +0100 Message-ID: <20241014105912.3207374-56-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 070D8180012 X-Rspamd-Server: rspam01 X-Stat-Signature: 8s8egkzz6y7dc4kx6m9o7mxt79uoc4d4 X-HE-Tag: 1728903732-707447 X-HE-Meta: U2FsdGVkX1+/PBKnUuOgNe3s5RGIVqn8E5M/bvqBhNO4fx/j0PR7C9y3UWrOKmKu2J2HKxCHv3ARN1ydEiONiEk1WT5rozNiaJFlgpLeXnSXEiJYNtaPQNwC3mMFRqxHFdFJ4fSxobH6jBDwGkVO2sYzCZ/rip/BA8N022PoD7rPj7YstK/1Mt0SO0uqXf3ib2b1S/wFL25mYZ4oMxFRX3EpKgWBdR4SMdIYHmvYHar/QJ/6Z2owug15aRhs1aCPoyDVRKECFPtWy9xJYKK2kbBt4f+xZWLZJezU8cazYN7wPLkq7cU08s4Oh5fCl3dTOHo9s9e8xCTqmkAVb6aYeW1FOxXs0mImd7AOTvgfTeuKOSjf1xs/NfiJoi9kOh1zCxU4rH8Fj5Y/tAxCr5w1LrZPVTFfuKA55VF7sgrkKH5O7mUEf/bdcvpobaDZT47+InJKoUR36CuCbTC8WZeESKrSSNHrj5Z0wCX+1a/txF/jeY4C7S2j1xZJ49TQ5N3+KX9fRzIYYLHSpcbbDQwd/Fuyj1qbwze9+nMs/8Y/HFecBIDRWChHrqn0cJb2Cu0XOdXVfHcPk5p0hmgXRxWbfAK6I1I7NILDOqbqoThOqCKVCQJWLpKS8O9UwR7eolIuXaY3t5zNuUPCoF86gYHcsorl0VpttWSdAYHDfX1EaceAcan9eBuqLqQZZUtOo3X8LJkOFe3q8QdkNcTN7sSnuVFyh0M3eFhlEOXKQU+aHI9x5Wv3AHqCCXsvEyK35l+TL3pMmkyv7dOTSKOu9A1fu58WCajTBPiZDTH8oDmcp4Y7eyD8FNEihmnwzg6+nsUZns8DiHdRu1ZQfqJDa0NGqoKoQcX0Nb5bFWjLLKbXF5NlPP5peIEIYbotxcygyIvXGOphkL1S6AWmp8M5vYm3UzdPG18N/WYtu+CxhVr93ObRap6eNimf8/Ae9wFAVfosT/H1HbQUEho8FSft0va XJ0nDr1E SQUqkc0nZmS0A+4bZR2LDxOBKbJyDj9lelNaW6DWTJmu/prklf2EAlACurkUf0byypBK/icVS1TxHM5vav+M9kTdCCO7DVYi/nO0DSlfgtVmt8kAVG9U8pgmpEkxwKi6mtc7/AnGcqE/O5XNIkA8jXmoL/3t9Zbyp+Ned9dAmOw4CWIhT+3OAFBHyHgmYkskpvDs+VwwgbdaMGIv1+YcOEg+C9vVIbzIc3KCS X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since THREAD_SIZE depends on PAGE_SIZE when stacks are vmapped, we must defer the decision on THREAD_SIZE until we have selected PAGE_SIZE at boot. The one wrinkle is entry.S's requirement to have THREAD_SHIFT as an immediate in order to check that the stack has not overflowed without clobbering any registers, early in the exception handler. Solve this by patching alternatives. During early boot, all 3 options are NOPs until the alternative is patched in. So we forgo overflow checking until boot-cpu patching is complete. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/include/asm/assembler.h | 5 +++ arch/arm64/include/asm/efi.h | 2 +- arch/arm64/include/asm/memory.h | 51 +++++++++++++++++++++++++----- arch/arm64/kernel/efi.c | 2 +- arch/arm64/kernel/entry.S | 10 +++++- arch/arm64/kernel/head.S | 3 +- arch/arm64/kernel/vmlinux.lds.S | 4 +-- arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 2 +- 8 files changed, 64 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 0cfa7c3efd214..745328e7768b7 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -928,4 +928,9 @@ alternative_cb_end get_tg0 \val value_for_page_size \val, \val, (~(SZ_4K-1)), (~(SZ_16K-1)), (~(SZ_64K-1)) .endm + + .macro get_task_size, val + get_tg0 \val + value_for_page_size \val, \val, (1 << THREAD_SHIFT_4K), (1 << THREAD_SHIFT_16K), (1 << THREAD_SHIFT_64K) + .endm #endif /* __ASM_ASSEMBLER_H */ diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h index bcd5622aa0968..913f599c14e40 100644 --- a/arch/arm64/include/asm/efi.h +++ b/arch/arm64/include/asm/efi.h @@ -68,7 +68,7 @@ void arch_efi_call_virt_teardown(void); * kernel need greater alignment than we require the segments to be padded to. */ #define EFI_KIMG_ALIGN \ - (SEGMENT_ALIGN > THREAD_ALIGN ? SEGMENT_ALIGN : THREAD_ALIGN) + (SEGMENT_ALIGN > THREAD_ALIGN_MAX ? SEGMENT_ALIGN : THREAD_ALIGN_MAX) /* * On arm64, we have to ensure that the initrd ends up in the linear region, diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 5393a859183f7..e28f5700ef022 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -110,23 +110,56 @@ #define PAGE_END (_PAGE_END(VA_BITS_MIN)) #endif /* CONFIG_KASAN */ -#define MIN_THREAD_SHIFT (14 + KASAN_THREAD_SHIFT) +#define IDEAL_THREAD_SHIFT (14 + KASAN_THREAD_SHIFT) /* * VMAP'd stacks are allocated at page granularity, so we must ensure that such * stacks are a multiple of page size. */ -#if defined(CONFIG_VMAP_STACK) && (MIN_THREAD_SHIFT < PAGE_SHIFT) -#define THREAD_SHIFT PAGE_SHIFT + +#if defined(CONFIG_VMAP_STACK) +#define THREAD_SHIFT \ + (IDEAL_THREAD_SHIFT < PAGE_SHIFT ? PAGE_SHIFT : IDEAL_THREAD_SHIFT) +#if (IDEAL_THREAD_SHIFT < PAGE_SHIFT_MIN) +#define THREAD_SHIFT_MIN PAGE_SHIFT_MIN #else -#define THREAD_SHIFT MIN_THREAD_SHIFT +#define THREAD_SHIFT_MIN IDEAL_THREAD_SHIFT #endif - -#if THREAD_SHIFT >= PAGE_SHIFT -#define THREAD_SIZE_ORDER (THREAD_SHIFT - PAGE_SHIFT) +#if (IDEAL_THREAD_SHIFT < PAGE_SHIFT_MAX) +#define THREAD_SHIFT_MAX PAGE_SHIFT_MAX +#else +#define THREAD_SHIFT_MAX IDEAL_THREAD_SHIFT +#endif +#if (IDEAL_THREAD_SHIFT < ARM64_PAGE_SHIFT_4K) +#define THREAD_SHIFT_4K ARM64_PAGE_SHIFT_4K +#else +#define THREAD_SHIFT_4K IDEAL_THREAD_SHIFT +#endif +#if (IDEAL_THREAD_SHIFT < ARM64_PAGE_SHIFT_16K) +#define THREAD_SHIFT_16K ARM64_PAGE_SHIFT_16K +#else +#define THREAD_SHIFT_16K IDEAL_THREAD_SHIFT +#endif +#if (IDEAL_THREAD_SHIFT < ARM64_PAGE_SHIFT_64K) +#define THREAD_SHIFT_64K ARM64_PAGE_SHIFT_64K +#else +#define THREAD_SHIFT_64K IDEAL_THREAD_SHIFT #endif +#else +#define THREAD_SHIFT IDEAL_THREAD_SHIFT +#define THREAD_SHIFT_MIN IDEAL_THREAD_SHIFT +#define THREAD_SHIFT_MAX IDEAL_THREAD_SHIFT +#define THREAD_SHIFT_4K IDEAL_THREAD_SHIFT +#define THREAD_SHIFT_16K IDEAL_THREAD_SHIFT +#define THREAD_SHIFT_64K IDEAL_THREAD_SHIFT +#endif + +#define THREAD_SIZE_ORDER \ + (PAGE_SHIFT < THREAD_SHIFT ? THREAD_SHIFT - PAGE_SHIFT : 0) #define THREAD_SIZE (UL(1) << THREAD_SHIFT) +#define THREAD_SIZE_MIN (UL(1) << THREAD_SHIFT_MIN) +#define THREAD_SIZE_MAX (UL(1) << THREAD_SHIFT_MAX) /* * By aligning VMAP'd stacks to 2 * THREAD_SIZE, we can detect overflow by @@ -135,11 +168,13 @@ */ #ifdef CONFIG_VMAP_STACK #define THREAD_ALIGN (2 * THREAD_SIZE) +#define THREAD_ALIGN_MAX (2 * THREAD_SIZE_MAX) #else #define THREAD_ALIGN THREAD_SIZE +#define THREAD_ALIGN_MAX THREAD_SIZE_MAX #endif -#define IRQ_STACK_SIZE THREAD_SIZE +#define IRQ_STACK_SIZE THREAD_SIZE_MIN #define OVERFLOW_STACK_SIZE SZ_4K diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c index 712718aed5dd9..ebc44b7e83199 100644 --- a/arch/arm64/kernel/efi.c +++ b/arch/arm64/kernel/efi.c @@ -197,7 +197,7 @@ bool efi_runtime_fixup_exception(struct pt_regs *regs, const char *msg) } /* EFI requires 8 KiB of stack space for runtime services */ -static_assert(THREAD_SIZE >= SZ_8K); +static_assert(THREAD_SIZE_MIN >= SZ_8K); static int __init arm64_efi_rt_init(void) { diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index ba47dc8672c04..1ab65e406b62e 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -62,7 +62,15 @@ */ add sp, sp, x0 // sp' = sp + x0 sub x0, sp, x0 // x0' = sp' - x0 = (sp + x0) - x0 = sp - tbnz x0, #THREAD_SHIFT, 0f +alternative_if ARM64_USE_PAGE_SIZE_4K + tbnz x0, #THREAD_SHIFT_4K, 0f +alternative_else_nop_endif +alternative_if ARM64_USE_PAGE_SIZE_16K + tbnz x0, #THREAD_SHIFT_16K, 0f +alternative_else_nop_endif +alternative_if ARM64_USE_PAGE_SIZE_64K + tbnz x0, #THREAD_SHIFT_64K, 0f +alternative_else_nop_endif sub x0, sp, x0 // x0'' = sp' - x0' = (sp + x0) - sp = x0 sub sp, sp, x0 // sp'' = sp' - x0 = (sp + x0) - x0 = sp b el\el\ht\()_\regsize\()_\label diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 761b7f5633e15..2530ee5cee548 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -198,7 +198,8 @@ SYM_CODE_END(preserve_boot_args) msr sp_el0, \tsk ldr \tmp1, [\tsk, #TSK_STACK] - add sp, \tmp1, #THREAD_SIZE + get_task_size \tmp2 + add sp, \tmp1, \tmp2 sub sp, sp, #PT_REGS_SIZE stp xzr, xzr, [sp, #S_STACKFRAME] diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 09fcc234c0f77..937900a458a89 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -60,11 +60,11 @@ #define RO_EXCEPTION_TABLE_ALIGN 4 #define RUNTIME_DISCARD_EXIT +#include #include #include #include #include -#include #include #include "image.h" @@ -292,7 +292,7 @@ SECTIONS _data = .; _sdata = .; - RW_DATA(L1_CACHE_BYTES, PAGE_SIZE_MAX, THREAD_ALIGN) + RW_DATA(L1_CACHE_BYTES, PAGE_SIZE_MAX, THREAD_ALIGN_MAX) /* * Data written with the MMU off but read with the MMU on requires diff --git a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S index 74c7c21626270..fe1fbfa8f8f05 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S @@ -7,9 +7,9 @@ */ #include +#include #include #include -#include SECTIONS { HYP_SECTION(.idmap.text) From patchwork Mon Oct 14 10:59:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13834725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5F61D1A443 for ; Mon, 14 Oct 2024 11:02:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B75466B00EF; Mon, 14 Oct 2024 07:02:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AFD016B00F1; Mon, 14 Oct 2024 07:02:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 976976B00F2; Mon, 14 Oct 2024 07:02:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6E4A16B00EF for ; Mon, 14 Oct 2024 07:02:22 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 19F25A0D73 for ; Mon, 14 Oct 2024 11:02:08 +0000 (UTC) X-FDA: 82671918714.06.A8FCA8A Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf02.hostedemail.com (Postfix) with ESMTP id 0996F8001B for ; Mon, 14 Oct 2024 11:02:08 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728903694; a=rsa-sha256; cv=none; b=udr4bRyWSo5y7dWzWuLwx9YKqvYeic3oy4175XGPEVW05MXk1CkOznd5xHEU5L77nsGO/a AgMLfEFpe6ejjcktJzWRhzerzllUTmLLKEbu8QsqIBR3/QYAAwLa+y4Fl5X8kTkJ/pX3t4 da0JAqys+axmEoJG/lRb4tIvOV8K48Y= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728903694; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xQxbzyYZeDib8MIhoxkIiml0NfyISTXv5vXI0QGhods=; b=7BM3wmVBzVlcbZhRFi4ORV/ip0Ritzd4vapc8StrYWPNgfW9sglYitne5pHzff3cD0/Lf3 iGoat3VzZx2qFilXsHp445NtksUu3KEEYY22E00O0jSuZliu8RIfBerZTbXTO8Ucbrg3eq SQY0qrWGr9kcpgQv3MH9CCxgQIVjMSg= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7ACB21758; Mon, 14 Oct 2024 04:02:49 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1E2F93F51B; Mon, 14 Oct 2024 04:02:16 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Greg Marsden , Ivan Ivanov , Kalesh Singh , Marc Zyngier , Mark Rutland , Matthias Brugger , Miroslav Benes , Oliver Upton , Will Deacon Cc: Ryan Roberts , kvmarm@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH v1 57/57] arm64: Enable boot-time page size selection Date: Mon, 14 Oct 2024 11:59:04 +0100 Message-ID: <20241014105912.3207374-57-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241014105912.3207374-1-ryan.roberts@arm.com> References: <20241014105514.3206191-1-ryan.roberts@arm.com> <20241014105912.3207374-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 0996F8001B X-Rspamd-Server: rspam01 X-Stat-Signature: uhzityhs6qpo5jwnoouhyxd8bhe6zboq X-HE-Tag: 1728903728-801390 X-HE-Meta: U2FsdGVkX1/aytAmGaq8HgTRaQV04NJssZtxhubo97YMVwoZ5l6W9CQSjLn72vxSITt31yROdHikTv3P2zmDNy1iVqgpYCFKbhdZHbR0SLsV/JAEZc97/1kO4rlBxXmk5SCO2SaqPonMgEFEMAe3yTgFOsijWkA3ZzHM1t5TNZCc/RKpPBUF821kTJcldemWydJezQ8s7PpDL9dVESTS1/ciwvxMrg4Py20EgFIjhSUUp3oBqWp0daGBEjM46VYN6z4oON9kTIvLZWN+Lnj5kVZu0qfgMgDxUgDN6AzYyN165cVEN9KyroY8h8G1uVANY+WVfTHV8z6cnh8491QiWS4Zrvhiz2gwpcz93k5faWKmXMzKaO+QRMRGzFEj68AgXyloCfe7krq2zosnUwEMHoAaICtLeVlfcqSzC3eSPVyJNzlPaHgY/cwy7ICjjOtrodAgwJlE4k3cPrB+2Wg5un8woOGY2biWjrsD7v9f6oLuN3iRBcpIOO8chirJAX2osnqqUqAoWxqU7Es9E5r1G/WkjtUmdOySP8PaFx/Y+Rb0HfbyU7EE96z8+VZxBmd/Ma7TGxKwku5dIz+ny8d3rdog63Y9JjHGPhMMYLG8356elxFASp4ISCz0HciztfMm8Z0bdItPlDjR7zEpr1SF7ii1OsiLvVaX9V1Y62KOFeVtAchH7C8QWTDDfWZAs7in6YbP91wpr/iDLM/ghV+xcLPNzZsfWJGQiJ8KrGl8AQUlJGcsOpmHXh6l8rFdamriiFs2/UBpeNSimxpz8CDnzczwqApB8+PW6fLLsjglVCqxFwOlDzU7gP8AQvupKvfTKjGh9cyIY6P7fXRDTm6JNapXrNT+nAVt4ZeRcZ7hhhbZWEokDSl2RT9iBVL0WUbC/2RD3Po9qaDi87rU/iR15lBMyIgEq1S3Iwfj+j9RzEIwzK1by/t0dX7nGc3ONzNQKbAJSwo1NS+7LThN5Fb ctiklpKm 7MMRAUcyK1y0tx3HxocyBDbpgfGviqC4JDuZyqnzBNyocY0wbGTqtROTubfX9YC2rnaij0XqNnM2JcL4BgYpfq2Zl0tbag2C7LcpYTnGxqGZ9vJWw5vi0oEqhYRKC/HbWP9qE9TsgQBeA5Ed2+fhBabO3BNslk6Mf/mWPpab3F9CUkDVZbw4/JE189g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduce a new Kconfig, ARM64_BOOT_TIME_PAGE_SIZE, which can be selected instead of a page size. When selected, the resulting kernel's page size can be configured at boot via the command line. For now, boot-time page size kernels are limited to 48-bit VA, since more work is required to support LPA2. Additionally MMAP_RND_BITS and SECTION_SIZE_BITS are configured for the worst case (64K pages). Future work could be implemented to be able to configure these at boot time for optimial page size-specific values. Signed-off-by: Ryan Roberts --- ***NOTE*** Any confused maintainers may want to read the cover note here for context: https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/ arch/arm64/Kconfig | 26 ++++++++++--- arch/arm64/include/asm/kvm_hyp.h | 11 ++++++ arch/arm64/include/asm/pgtable-geometry.h | 22 ++++++++++- arch/arm64/include/asm/pgtable-hwdef.h | 6 +-- arch/arm64/include/asm/pgtable.h | 10 ++++- arch/arm64/include/asm/sparsemem.h | 4 ++ arch/arm64/kernel/image-vars.h | 11 ++++++ arch/arm64/kernel/image.h | 4 ++ arch/arm64/kernel/pi/map_kernel.c | 45 ++++++++++++++++++++++ arch/arm64/kvm/arm.c | 10 +++++ arch/arm64/kvm/hyp/nvhe/Makefile | 1 + arch/arm64/kvm/hyp/nvhe/pgtable-geometry.c | 16 ++++++++ arch/arm64/mm/Makefile | 1 + arch/arm64/mm/pgd.c | 10 +++-- arch/arm64/mm/pgtable-geometry.c | 24 ++++++++++++ drivers/firmware/efi/libstub/arm64.c | 3 +- 16 files changed, 187 insertions(+), 17 deletions(-) create mode 100644 arch/arm64/kvm/hyp/nvhe/pgtable-geometry.c create mode 100644 arch/arm64/mm/pgtable-geometry.c diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a2f8ff354ca67..573d308741169 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -121,6 +121,7 @@ config ARM64 select BUILDTIME_TABLE_SORT select CLONE_BACKWARDS select COMMON_CLK + select CONSTRUCTORS if ARM64_BOOT_TIME_PAGE_SIZE select CPU_PM if (SUSPEND || CPU_IDLE) select CPUMASK_OFFSTACK if NR_CPUS > 256 select CRC32 @@ -284,18 +285,20 @@ config MMU config ARM64_CONT_PTE_SHIFT int + depends on !ARM64_BOOT_TIME_PAGE_SIZE default 5 if PAGE_SIZE_64KB default 7 if PAGE_SIZE_16KB default 4 config ARM64_CONT_PMD_SHIFT int + depends on !ARM64_BOOT_TIME_PAGE_SIZE default 5 if PAGE_SIZE_64KB default 5 if PAGE_SIZE_16KB default 4 config ARCH_MMAP_RND_BITS_MIN - default 14 if PAGE_SIZE_64KB + default 14 if ARM64_64K_PAGES || ARM64_BOOT_TIME_PAGE_SIZE default 16 if PAGE_SIZE_16KB default 18 @@ -306,15 +309,15 @@ config ARCH_MMAP_RND_BITS_MAX default 24 if ARM64_VA_BITS=39 default 27 if ARM64_VA_BITS=42 default 30 if ARM64_VA_BITS=47 - default 29 if ARM64_VA_BITS=48 && ARM64_64K_PAGES + default 29 if ARM64_VA_BITS=48 && (ARM64_64K_PAGES || ARM64_BOOT_TIME_PAGE_SIZE) default 31 if ARM64_VA_BITS=48 && ARM64_16K_PAGES default 33 if ARM64_VA_BITS=48 - default 14 if ARM64_64K_PAGES + default 14 if ARM64_64K_PAGES || ARM64_BOOT_TIME_PAGE_SIZE default 16 if ARM64_16K_PAGES default 18 config ARCH_MMAP_RND_COMPAT_BITS_MIN - default 7 if ARM64_64K_PAGES + default 7 if ARM64_64K_PAGES || ARM64_BOOT_TIME_PAGE_SIZE default 9 if ARM64_16K_PAGES default 11 @@ -362,6 +365,7 @@ config FIX_EARLYCON_MEM config PGTABLE_LEVELS int + default 4 if ARM64_BOOT_TIME_PAGE_SIZE # Advertise max supported levels default 2 if ARM64_16K_PAGES && ARM64_VA_BITS_36 default 2 if ARM64_64K_PAGES && ARM64_VA_BITS_42 default 3 if ARM64_64K_PAGES && (ARM64_VA_BITS_48 || ARM64_VA_BITS_52) @@ -1316,6 +1320,14 @@ config ARM64_64K_PAGES look-up. AArch32 emulation requires applications compiled with 64K aligned segments. +config ARM64_BOOT_TIME_PAGE_SIZE + bool "Boot-time selection" + select HAVE_PAGE_SIZE_64KB # Advertise largest page size to core + help + Select desired page size (4KB, 16KB or 64KB) at boot-time via the + kernel command line option "arm64.pagesize=4k", "arm64.pagesize=16k" + or "arm64.pagesize=64k". + endchoice choice @@ -1348,6 +1360,7 @@ config ARM64_VA_BITS_48 config ARM64_VA_BITS_52 bool "52-bit" depends on ARM64_PAN || !ARM64_SW_TTBR0_PAN + depends on !ARM64_BOOT_TIME_PAGE_SIZE help Enable 52-bit virtual addressing for userspace when explicitly requested via a hint to mmap(). The kernel will also use 52-bit @@ -1588,9 +1601,10 @@ config XEN # 4K | 27 | 12 | 15 | 10 | # 16K | 27 | 14 | 13 | 11 | # 64K | 29 | 16 | 13 | 13 | +# BOOT| 29 | 16 (max) | 13 | 13 | config ARCH_FORCE_MAX_ORDER int - default "13" if ARM64_64K_PAGES + default "13" if ARM64_64K_PAGES || ARM64_BOOT_TIME_PAGE_SIZE default "11" if ARM64_16K_PAGES default "10" help @@ -1663,7 +1677,7 @@ config ARM64_TAGGED_ADDR_ABI menuconfig COMPAT bool "Kernel support for 32-bit EL0" - depends on ARM64_4K_PAGES || EXPERT + depends on ARM64_4K_PAGES || ARM64_BOOT_TIME_PAGE_SIZE || EXPERT select HAVE_UID16 select OLD_SIGSUSPEND3 select COMPAT_OLD_SIGACTION diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index c838309e4ec47..9397a14642afa 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -145,4 +145,15 @@ extern unsigned long kvm_nvhe_sym(__icache_flags); extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits); extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl); +#ifdef CONFIG_ARM64_BOOT_TIME_PAGE_SIZE +extern int kvm_nvhe_sym(ptg_page_shift); +extern int kvm_nvhe_sym(ptg_pmd_shift); +extern int kvm_nvhe_sym(ptg_pud_shift); +extern int kvm_nvhe_sym(ptg_p4d_shift); +extern int kvm_nvhe_sym(ptg_pgdir_shift); +extern int kvm_nvhe_sym(ptg_cont_pte_shift); +extern int kvm_nvhe_sym(ptg_cont_pmd_shift); +extern int kvm_nvhe_sym(ptg_pgtable_levels); +#endif /* CONFIG_ARM64_BOOT_TIME_PAGE_SIZE */ + #endif /* __ARM64_KVM_HYP_H__ */ diff --git a/arch/arm64/include/asm/pgtable-geometry.h b/arch/arm64/include/asm/pgtable-geometry.h index 62fe125909c08..18a8c8d499ecc 100644 --- a/arch/arm64/include/asm/pgtable-geometry.h +++ b/arch/arm64/include/asm/pgtable-geometry.h @@ -6,16 +6,33 @@ #define ARM64_PAGE_SHIFT_16K 14 #define ARM64_PAGE_SHIFT_64K 16 +#ifdef CONFIG_ARM64_BOOT_TIME_PAGE_SIZE +#define PAGE_SHIFT_MIN ARM64_PAGE_SHIFT_4K +#define PAGE_SHIFT_MAX ARM64_PAGE_SHIFT_64K +#else /* CONFIG_ARM64_BOOT_TIME_PAGE_SIZE */ #define PAGE_SHIFT_MIN CONFIG_PAGE_SHIFT +#define PAGE_SHIFT_MAX CONFIG_PAGE_SHIFT +#endif /* CONFIG_ARM64_BOOT_TIME_PAGE_SIZE */ + #define PAGE_SIZE_MIN (_AC(1, UL) << PAGE_SHIFT_MIN) #define PAGE_MASK_MIN (~(PAGE_SIZE_MIN-1)) - -#define PAGE_SHIFT_MAX CONFIG_PAGE_SHIFT #define PAGE_SIZE_MAX (_AC(1, UL) << PAGE_SHIFT_MAX) #define PAGE_MASK_MAX (~(PAGE_SIZE_MAX-1)) #include +#ifdef CONFIG_ARM64_BOOT_TIME_PAGE_SIZE +#ifndef __ASSEMBLY__ +extern int ptg_page_shift; +extern int ptg_pmd_shift; +extern int ptg_pud_shift; +extern int ptg_p4d_shift; +extern int ptg_pgdir_shift; +extern int ptg_cont_pte_shift; +extern int ptg_cont_pmd_shift; +extern int ptg_pgtable_levels; +#endif /* __ASSEMBLY__ */ +#else /* CONFIG_ARM64_BOOT_TIME_PAGE_SIZE */ #define ptg_page_shift CONFIG_PAGE_SHIFT #define ptg_pmd_shift ARM64_HW_PGTABLE_LEVEL_SHIFT(2) #define ptg_pud_shift ARM64_HW_PGTABLE_LEVEL_SHIFT(1) @@ -24,5 +41,6 @@ #define ptg_cont_pte_shift (CONFIG_ARM64_CONT_PTE_SHIFT + PAGE_SHIFT) #define ptg_cont_pmd_shift (CONFIG_ARM64_CONT_PMD_SHIFT + PMD_SHIFT) #define ptg_pgtable_levels CONFIG_PGTABLE_LEVELS +#endif /* CONFIG_ARM64_BOOT_TIME_PAGE_SIZE */ #endif /* ASM_PGTABLE_GEOMETRY_H */ diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index ca8bcbc1fe220..da5404617acbf 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -52,7 +52,7 @@ #define PMD_SHIFT ptg_pmd_shift #define PMD_SIZE (_AC(1, UL) << PMD_SHIFT) #define PMD_MASK (~(PMD_SIZE-1)) -#define PTRS_PER_PMD (1 << (PAGE_SHIFT - 3)) +#define PTRS_PER_PMD (ptg_pgtable_levels > 2 ? (1 << (PAGE_SHIFT - 3)) : 1) #define MAX_PTRS_PER_PMD (1 << (PAGE_SHIFT_MAX - 3)) #endif @@ -63,7 +63,7 @@ #define PUD_SHIFT ptg_pud_shift #define PUD_SIZE (_AC(1, UL) << PUD_SHIFT) #define PUD_MASK (~(PUD_SIZE-1)) -#define PTRS_PER_PUD (1 << (PAGE_SHIFT - 3)) +#define PTRS_PER_PUD (ptg_pgtable_levels > 3 ? (1 << (PAGE_SHIFT - 3)) : 1) #define MAX_PTRS_PER_PUD (1 << (PAGE_SHIFT_MAX - 3)) #endif @@ -71,7 +71,7 @@ #define P4D_SHIFT ptg_p4d_shift #define P4D_SIZE (_AC(1, UL) << P4D_SHIFT) #define P4D_MASK (~(P4D_SIZE-1)) -#define PTRS_PER_P4D (1 << (PAGE_SHIFT - 3)) +#define PTRS_PER_P4D (ptg_pgtable_levels > 4 ? (1 << (PAGE_SHIFT - 3)) : 1) #define MAX_PTRS_PER_P4D (1 << (PAGE_SHIFT_MAX - 3)) #endif diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 8ead41da715b0..ad9f75f5cc29a 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -755,7 +755,7 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd) static __always_inline bool pgtable_l3_enabled(void) { - return true; + return ptg_pgtable_levels > 2; } static inline bool mm_pmd_folded(const struct mm_struct *mm) @@ -888,6 +888,8 @@ static inline bool pgtable_l3_enabled(void) { return false; } static __always_inline bool pgtable_l4_enabled(void) { + if (IS_ENABLED(CONFIG_ARM64_BOOT_TIME_PAGE_SIZE)) + return ptg_pgtable_levels > 3; if (CONFIG_PGTABLE_LEVELS > 4 || !IS_ENABLED(CONFIG_ARM64_LPA2)) return true; if (!alternative_has_cap_likely(ARM64_ALWAYS_BOOT)) @@ -935,6 +937,8 @@ static inline phys_addr_t p4d_page_paddr(p4d_t p4d) static inline pud_t *p4d_to_folded_pud(p4d_t *p4dp, unsigned long addr) { + if (IS_ENABLED(CONFIG_ARM64_BOOT_TIME_PAGE_SIZE)) + return (pud_t *)p4dp; return (pud_t *)PTR_ALIGN_DOWN(p4dp, PAGE_SIZE) + pud_index(addr); } @@ -1014,6 +1018,8 @@ static inline bool pgtable_l4_enabled(void) { return false; } static __always_inline bool pgtable_l5_enabled(void) { + if (IS_ENABLED(CONFIG_ARM64_BOOT_TIME_PAGE_SIZE)) + return ptg_pgtable_levels > 4; if (!alternative_has_cap_likely(ARM64_ALWAYS_BOOT)) return vabits_actual == VA_BITS; return alternative_has_cap_unlikely(ARM64_HAS_VA52); @@ -1059,6 +1065,8 @@ static inline phys_addr_t pgd_page_paddr(pgd_t pgd) static inline p4d_t *pgd_to_folded_p4d(pgd_t *pgdp, unsigned long addr) { + if (IS_ENABLED(CONFIG_ARM64_BOOT_TIME_PAGE_SIZE)) + return (p4d_t *)pgdp; return (p4d_t *)PTR_ALIGN_DOWN(pgdp, PAGE_SIZE) + p4d_index(addr); } diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h index a05fdd54014f7..2daf1263ba638 100644 --- a/arch/arm64/include/asm/sparsemem.h +++ b/arch/arm64/include/asm/sparsemem.h @@ -17,6 +17,10 @@ * entries could not be created for vmemmap mappings. * 16K follows 4K for simplicity. */ +#ifdef CONFIG_ARM64_BOOT_TIME_PAGE_SIZE +#define SECTION_SIZE_BITS 29 +#else #define SECTION_SIZE_BITS (PAGE_SIZE == SZ_64K ? 29 : 27) +#endif #endif diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index a168f3337446f..9968320f83bc4 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -36,6 +36,17 @@ PROVIDE(__pi___memcpy = __pi_memcpy); PROVIDE(__pi___memmove = __pi_memmove); PROVIDE(__pi___memset = __pi_memset); +#ifdef CONFIG_ARM64_BOOT_TIME_PAGE_SIZE +PROVIDE(__pi_ptg_page_shift = ptg_page_shift); +PROVIDE(__pi_ptg_pmd_shift = ptg_pmd_shift); +PROVIDE(__pi_ptg_pud_shift = ptg_pud_shift); +PROVIDE(__pi_ptg_p4d_shift = ptg_p4d_shift); +PROVIDE(__pi_ptg_pgdir_shift = ptg_pgdir_shift); +PROVIDE(__pi_ptg_cont_pte_shift = ptg_cont_pte_shift); +PROVIDE(__pi_ptg_cont_pmd_shift = ptg_cont_pmd_shift); +PROVIDE(__pi_ptg_pgtable_levels = ptg_pgtable_levels); +#endif /* CONFIG_ARM64_BOOT_TIME_PAGE_SIZE */ + PROVIDE(__pi_id_aa64isar1_override = id_aa64isar1_override); PROVIDE(__pi_id_aa64isar2_override = id_aa64isar2_override); PROVIDE(__pi_id_aa64mmfr0_override = id_aa64mmfr0_override); diff --git a/arch/arm64/kernel/image.h b/arch/arm64/kernel/image.h index 7bc3ba8979019..01502fc3b891b 100644 --- a/arch/arm64/kernel/image.h +++ b/arch/arm64/kernel/image.h @@ -47,7 +47,11 @@ #define __HEAD_FLAG_BE ARM64_IMAGE_FLAG_LE #endif +#ifdef CONFIG_ARM64_BOOT_TIME_PAGE_SIZE +#define __HEAD_FLAG_PAGE_SIZE 0 +#else #define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2) +#endif #define __HEAD_FLAG_PHYS_BASE 1 diff --git a/arch/arm64/kernel/pi/map_kernel.c b/arch/arm64/kernel/pi/map_kernel.c index deb8cd50b0b0c..22b3c70e04f9c 100644 --- a/arch/arm64/kernel/pi/map_kernel.c +++ b/arch/arm64/kernel/pi/map_kernel.c @@ -221,6 +221,49 @@ static void __init map_fdt(u64 fdt, int page_shift) dsb(ishst); } +#ifdef CONFIG_ARM64_BOOT_TIME_PAGE_SIZE +static void __init ptg_init(int page_shift) +{ + ptg_pgtable_levels = + __ARM64_HW_PGTABLE_LEVELS(page_shift, CONFIG_ARM64_VA_BITS); + + ptg_pgdir_shift = __ARM64_HW_PGTABLE_LEVEL_SHIFT(page_shift, + 4 - ptg_pgtable_levels); + + ptg_p4d_shift = ptg_pgtable_levels >= 5 ? + __ARM64_HW_PGTABLE_LEVEL_SHIFT(page_shift, 0) : + ptg_pgdir_shift; + + ptg_pud_shift = ptg_pgtable_levels >= 4 ? + __ARM64_HW_PGTABLE_LEVEL_SHIFT(page_shift, 1) : + ptg_pgdir_shift; + + ptg_pmd_shift = ptg_pgtable_levels >= 3 ? + __ARM64_HW_PGTABLE_LEVEL_SHIFT(page_shift, 2) : + ptg_pgdir_shift; + + ptg_page_shift = page_shift; + + switch (page_shift) { + case ARM64_PAGE_SHIFT_64K: + ptg_cont_pte_shift = ptg_page_shift + 5; + ptg_cont_pmd_shift = ptg_pmd_shift + 5; + break; + case ARM64_PAGE_SHIFT_16K: + ptg_cont_pte_shift = ptg_page_shift + 7; + ptg_cont_pmd_shift = ptg_pmd_shift + 5; + break; + default: /* ARM64_PAGE_SHIFT_4K */ + ptg_cont_pte_shift = ptg_page_shift + 4; + ptg_cont_pmd_shift = ptg_pmd_shift + 4; + } +} +#else /* CONFIG_ARM64_BOOT_TIME_PAGE_SIZE */ +static inline void ptg_init(int page_shift) +{ +} +#endif /* CONFIG_ARM64_BOOT_TIME_PAGE_SIZE */ + asmlinkage void __init early_map_kernel(u64 boot_status, void *fdt) { static char const chosen_str[] __initconst = "/chosen"; @@ -247,6 +290,8 @@ asmlinkage void __init early_map_kernel(u64 boot_status, void *fdt) if (!page_shift) page_shift = early_page_shift; + ptg_init(page_shift); + if (va_bits > 48) { u64 page_size = early_page_size(page_shift); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 9bef7638342ef..c835a50b8b768 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -2424,6 +2424,16 @@ static void kvm_hyp_init_symbols(void) kvm_nvhe_sym(id_aa64smfr0_el1_sys_val) = read_sanitised_ftr_reg(SYS_ID_AA64SMFR0_EL1); kvm_nvhe_sym(__icache_flags) = __icache_flags; kvm_nvhe_sym(kvm_arm_vmid_bits) = kvm_arm_vmid_bits; +#ifdef CONFIG_ARM64_BOOT_TIME_PAGE_SIZE + kvm_nvhe_sym(ptg_page_shift) = ptg_page_shift; + kvm_nvhe_sym(ptg_pmd_shift) = ptg_pmd_shift; + kvm_nvhe_sym(ptg_pud_shift) = ptg_pud_shift; + kvm_nvhe_sym(ptg_p4d_shift) = ptg_p4d_shift; + kvm_nvhe_sym(ptg_pgdir_shift) = ptg_pgdir_shift; + kvm_nvhe_sym(ptg_cont_pte_shift) = ptg_cont_pte_shift; + kvm_nvhe_sym(ptg_cont_pmd_shift) = ptg_cont_pmd_shift; + kvm_nvhe_sym(ptg_pgtable_levels) = ptg_pgtable_levels; +#endif /* CONFIG_ARM64_BOOT_TIME_PAGE_SIZE */ } static int __init kvm_hyp_init_protection(u32 hyp_va_bits) diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index b43426a493df5..a8fcbb84c7996 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -27,6 +27,7 @@ hyp-obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o cache.o setup.o mm.o mem_protect.o sys_regs.o pkvm.o stacktrace.o ffa.o hyp-obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o ../pgtable.o +hyp-obj-$(CONFIG_ARM64_BOOT_TIME_PAGE_SIZE) += pgtable-geometry.o hyp-obj-$(CONFIG_LIST_HARDENED) += list_debug.o hyp-obj-y += $(lib-objs) diff --git a/arch/arm64/kvm/hyp/nvhe/pgtable-geometry.c b/arch/arm64/kvm/hyp/nvhe/pgtable-geometry.c new file mode 100644 index 0000000000000..17f807450a31a --- /dev/null +++ b/arch/arm64/kvm/hyp/nvhe/pgtable-geometry.c @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 ARM Ltd. + */ + +#include +#include + +int ptg_page_shift __ro_after_init; +int ptg_pmd_shift __ro_after_init; +int ptg_pud_shift __ro_after_init; +int ptg_p4d_shift __ro_after_init; +int ptg_pgdir_shift __ro_after_init; +int ptg_cont_pte_shift __ro_after_init; +int ptg_cont_pmd_shift __ro_after_init; +int ptg_pgtable_levels __ro_after_init; diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile index 60454256945b8..2ba30d06b35fe 100644 --- a/arch/arm64/mm/Makefile +++ b/arch/arm64/mm/Makefile @@ -3,6 +3,7 @@ obj-y := dma-mapping.o extable.o fault.o init.o \ cache.o copypage.o flush.o \ ioremap.o mmap.o pgd.o mmu.o \ context.o proc.o pageattr.o fixmap.o +obj-$(CONFIG_ARM64_BOOT_TIME_PAGE_SIZE) += pgtable-geometry.o obj-$(CONFIG_ARM64_CONTPTE) += contpte.o obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o diff --git a/arch/arm64/mm/pgd.c b/arch/arm64/mm/pgd.c index 4b106510358b1..c052d0dcb0c69 100644 --- a/arch/arm64/mm/pgd.c +++ b/arch/arm64/mm/pgd.c @@ -21,10 +21,12 @@ static bool pgdir_is_page_size(void) { if (PGD_SIZE == PAGE_SIZE) return true; - if (CONFIG_PGTABLE_LEVELS == 4) - return !pgtable_l4_enabled(); - if (CONFIG_PGTABLE_LEVELS == 5) - return !pgtable_l5_enabled(); + if (!IS_ENABLED(CONFIG_ARM64_BOOT_TIME_PAGE_SIZE)) { + if (CONFIG_PGTABLE_LEVELS == 4) + return !pgtable_l4_enabled(); + if (CONFIG_PGTABLE_LEVELS == 5) + return !pgtable_l5_enabled(); + } return false; } diff --git a/arch/arm64/mm/pgtable-geometry.c b/arch/arm64/mm/pgtable-geometry.c new file mode 100644 index 0000000000000..ba50637f1e9d0 --- /dev/null +++ b/arch/arm64/mm/pgtable-geometry.c @@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2024 ARM Ltd. + */ + +#include +#include + +/* + * TODO: These should be __ro_after_init, but we need to write to them from the + * pi code where they are mapped in the early page table as read-only. + * __ro_after_init doesn't become writable until later when the swapper pgtable + * is fully set up. We should update the early page table to map __ro_after_init + * as read-write. + */ + +int ptg_page_shift __read_mostly; +int ptg_pmd_shift __read_mostly; +int ptg_pud_shift __read_mostly; +int ptg_p4d_shift __read_mostly; +int ptg_pgdir_shift __read_mostly; +int ptg_cont_pte_shift __read_mostly; +int ptg_cont_pmd_shift __read_mostly; +int ptg_pgtable_levels __read_mostly; diff --git a/drivers/firmware/efi/libstub/arm64.c b/drivers/firmware/efi/libstub/arm64.c index e57cd3de0a00f..8db9dba7d5423 100644 --- a/drivers/firmware/efi/libstub/arm64.c +++ b/drivers/firmware/efi/libstub/arm64.c @@ -68,7 +68,8 @@ efi_status_t check_platform_features(void) efi_novamap = true; /* UEFI mandates support for 4 KB granularity, no need to check */ - if (IS_ENABLED(CONFIG_ARM64_4K_PAGES)) + if (IS_ENABLED(CONFIG_ARM64_4K_PAGES) || + IS_ENABLED(CONFIG_ARM64_BOOT_TIME_PAGE_SIZE)) return EFI_SUCCESS; tg = (read_cpuid(ID_AA64MMFR0_EL1) >> ID_AA64MMFR0_EL1_TGRAN_SHIFT) & 0xf;