From patchwork Fri Jul 1 11:10:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12903177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AAC92C433EF for ; Fri, 1 Jul 2022 11:12:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=ov5FGWiF3j9FSIDl7THZL1kclPnSJnd76tkCgFF3HnM=; b=Ctfr3/gX71qUCi fvRYVXqTxhMUxFS8eu1NlEHTegCKVqtObX72xCjKl5a142HL40w+/FMacmaPLv+WPD30nRUDKP9WI jlFtgJllVOKf6J8M4l3mK8dx5ePB1K3nR0jfXiABI32LIPy53ELAlWB1kWPNtSHQHm71WGstCAH7j XhbSY0Su5Guc/w8yCRL5my8yi+6b/apDbjWJC2eBUDKAm1x5WG1ess1V35s/gokcOSjbOizUiqSC6 SV8K4yD70FiDzuVAy+nidCK2ryM4ez7ejHhnUiYLYhnuzJPCxKJy8M1nER2h7fNi6TUy+fMthtaCQ QBpbpYzKf0gR0DpJcSxg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o7EYB-004GyV-44; Fri, 01 Jul 2022 11:11:03 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o7EY7-004Gy5-Ln for linux-arm-kernel@lists.infradead.org; Fri, 01 Jul 2022 11:11:01 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 18D90B80D20; Fri, 1 Jul 2022 11:10:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 480D5C3411E; Fri, 1 Jul 2022 11:10:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656673856; bh=Dct2GreszwXm3xbLSPDkUB6G2joAgssnxRYOl4XGRJU=; h=From:To:Cc:Subject:Date:From; b=ULBO+f5zUZDx9PqQi9TldKFmiPxxBbNGBoeuYr9FLYyWW9kVX/rXjSI5h+OuIMVCd fzRLj76VMts4rtXc+6cxpYFvJOYtPdaeC1RPSTQAM8ENDgXmoGK2914kTbhqnQ4R52 ZZVjix6/5UehQF/ZLIAVHMuULwedSXHpdD36Rhk5dwhEiQKc8ti1v5I/FanllqhKBg +VJ4nezkiH+v1Jl5BiITQDAysb12usMSkWpLm/Jr4+S29QMcHOajfL8113Q2a9dW2K Mx0SC6MpjZ5c1AwqgDsWgGVYHnuwfk/0i19y/T2gTkY5j2tqohug2rTe+mnOewXEZl IM1J0zKIIozhA== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Will Deacon , Anshuman Khandual , Mark Rutland , Joey Gouly Subject: [PATCH v3] arm64: mm: fix booting with 52-bit address space Date: Fri, 1 Jul 2022 13:10:45 +0200 Message-Id: <20220701111045.2944309-1-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5221; h=from:subject; bh=Dct2GreszwXm3xbLSPDkUB6G2joAgssnxRYOl4XGRJU=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBivtY0JXUS6E5FzYcTyks/6o3sQte3Fbp4PEwZzKWm rDNNyxqJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYr7WNAAKCRDDTyI5ktmPJFJrC/ 9SgYSETdvp48Ublf+s8wV1fWRwEdKMtf/7S6P7kXEIps5zFJajFYOguMDS0FPJ3obZC0tjS8KuOMVj akdZVcEo5P4oje82aQYtp6rb5b23yaHONeDPXWnBLQKdCYJY7hpZ3TjAs+5r+cvBOYeuDOr093ih48 eryW8vdn1zlKoM18QRVxd6awYt5fEki6nMA9wItYTJuTWiO5xW+BKeyRL/kxQ7c3xpyRxv3b2Q7nZz Z9lRtYirr9dNj0jHLG1UG2z29pdDB+JiEtxVU4rObZrNAst8t6ar629MZ3XCmOVN6Uk0TOd/W79XyY 4VC86OZ/TD28aFDyYC1/Chrz+XWFveJjkyQmg4lB9MctdC5wZNV6eVsGZiuEX1XCbwGsri898L2mbm qKiXFvXlVqrXHPe2vY4gVODKxoHZVxI55llNc7aQdeKLltbJCXvfO6aheChuc7BkJIA9zYhwznKUxV OIhRH0IIp2xyuYQm115jeQzeNEJkEMVQv+sK1gnLRBBjs= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220701_041100_061227_48DE3FC1 X-CRM114-Status: GOOD ( 19.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Joey reports that booting 52-bit VA capable builds on 52-bit VA capable CPUs is broken since commit 0d9b1ffefabe ("arm64: mm: make vabits_actual a build time constant if possible"). This is due to the fact that the primary CPU reads the vabits_actual variable before it has been assigned. The reason for deferring the assignment of vabits_actual was that we try to perform as few stores to memory as we can with the MMU and caches off, due to the cache coherency issues it creates. Since __cpu_setup() [which is where the read of vabits_actual occurs] is also called on the secondary boot path, we cannot just read the CPU ID registers directly, given that the size of the VA space is decided by the capabilities of the primary CPU. So let's read vabits_actual only on the secondary boot path, and read the CPU ID registers directly on the primary boot path, by making it a function parameter of __cpu_setup(). To ensure that all users of vabits_actual (including kasan_early_init()) observe the correct value, move the assignment of vabits_actual back into asm code, but still defer it to after the MMU and caches have been enabled. Cc: Will Deacon Cc: Anshuman Khandual Cc: Mark Rutland Fixes: 0d9b1ffefabe ("arm64: mm: make vabits_actual a build time constant if possible") Reported-by: Joey Gouly Co-developed-by: Joey Gouly Signed-off-by: Joey Gouly Signed-off-by: Ard Biesheuvel --- v3: fix KASAN too arch/arm64/kernel/head.S | 18 ++++++++++++++++++ arch/arm64/mm/init.c | 15 +-------------- arch/arm64/mm/proc.S | 5 +++-- 3 files changed, 22 insertions(+), 16 deletions(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index c300b43659dc..ae0a9e44ca19 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -82,6 +82,7 @@ * x22 create_idmap() .. start_kernel() ID map VA of the DT blob * x23 primary_entry() .. start_kernel() physical misalignment/KASLR offset * x24 __primary_switch() linear map KASLR seed + * x25 primary_entry() .. start_kernel() supported VA size * x28 create_idmap() callee preserved temp register */ SYM_CODE_START(primary_entry) @@ -96,6 +97,14 @@ SYM_CODE_START(primary_entry) * On return, the CPU will be ready for the MMU to be turned on and * the TCR will have been set. */ +#if VA_BITS > 48 + mrs_s x0, SYS_ID_AA64MMFR2_EL1 + tst x0, #0xf << ID_AA64MMFR2_LVA_SHIFT + mov x0, #VA_BITS + mov x25, #VA_BITS_MIN + csel x25, x25, x0, eq + mov x0, x25 +#endif bl __cpu_setup // initialise processor b __primary_switch SYM_CODE_END(primary_entry) @@ -434,6 +443,12 @@ SYM_FUNC_START_LOCAL(__primary_switched) bl __pi_memset dsb ishst // Make zero page visible to PTW +#if VA_BITS > 48 + adr_l x8, vabits_actual // Set this early so KASAN early init + str x25, [x8] // ... observes the correct value + dc civac, x8 // Make visible to booting secondaries +#endif + #ifdef CONFIG_RANDOMIZE_BASE adrp x5, memstart_offset_seed // Save KASLR linear map seed strh w24, [x5, :lo12:memstart_offset_seed] @@ -579,6 +594,9 @@ SYM_FUNC_START_LOCAL(secondary_startup) mov x20, x0 // preserve boot mode bl switch_to_vhe bl __cpu_secondary_check52bitva +#if VA_BITS > 48 + ldr_l x0, vabits_actual +#endif bl __cpu_setup // initialise processor adrp x1, swapper_pg_dir adrp x2, idmap_pg_dir diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c index 1faa6760895e..339ee84e5a61 100644 --- a/arch/arm64/mm/init.c +++ b/arch/arm64/mm/init.c @@ -265,20 +265,7 @@ early_param("mem", early_mem); void __init arm64_memblock_init(void) { - s64 linear_region_size; - -#if VA_BITS > 48 - if (cpuid_feature_extract_unsigned_field( - read_sysreg_s(SYS_ID_AA64MMFR2_EL1), - ID_AA64MMFR2_LVA_SHIFT)) - vabits_actual = VA_BITS; - - /* make the variable visible to secondaries with the MMU off */ - dcache_clean_inval_poc((u64)&vabits_actual, - (u64)&vabits_actual + sizeof(vabits_actual)); -#endif - - linear_region_size = PAGE_END - _PAGE_OFFSET(vabits_actual); + s64 linear_region_size = PAGE_END - _PAGE_OFFSET(vabits_actual); /* * Corner case: 52-bit VA capable systems running KVM in nVHE mode may diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 493b8ffc9be5..7837a69524c5 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -398,6 +398,8 @@ SYM_FUNC_END(idmap_kpti_install_ng_mappings) * * Initialise the processor for turning the MMU on. * + * Input: + * x0 - actual number of VA bits (ignored unless VA_BITS > 48) * Output: * Return in x0 the value of the SCTLR_EL1 register. */ @@ -467,8 +469,7 @@ SYM_FUNC_START(__cpu_setup) tcr_clear_errata_bits tcr, x9, x5 #ifdef CONFIG_ARM64_VA_BITS_52 - ldr_l x9, vabits_actual - sub x9, xzr, x9 + sub x9, xzr, x0 add x9, x9, #64 tcr_set_t1sz tcr, x9 #else