From patchwork Wed Jun 22 16:10:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12891146 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2A1EC43334 for ; Wed, 22 Jun 2022 16:26:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=y3kEoivo5Copf5pT/u0awHW6k8G64237d1FA/vE6iuM=; b=0SK5DAez8vdi+Q SgwkBUlWaRWFBOCznQHmwz2n5vmXp8LkVht7OVm46V1lhxf/AZkBG4AZ/bN4BdGxI+JALj7SNuRbq SQ8lTN7UJhyw93Sbn6SZ4G3bmmJATryUCmK5mBRytbAVLChHjOyU2pkkQdKAb158nVXYRSG0kcG5W +d6+dgOKOc+eBWKnIu/hlR636xxrZpyZE/Ey8cM7WWu/rVkE0FW9ziBW1yGSHQ64KpiFWQ2F54tfY jbyHeMeHmjJldClyP5slJwTsEVPKhTCzXyrWPx47l0856iiYSqRI6sA9lnmVmRtp3NrZII7RUI7bF yfKDw4dR+JIuecYCXuTw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o439z-00BV6Z-7v; Wed, 22 Jun 2022 16:24:55 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o42vt-00BOsi-I3 for linux-arm-kernel@lists.infradead.org; Wed, 22 Jun 2022 16:10:23 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id DF8B5619D7; Wed, 22 Jun 2022 16:10:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1D45CC34114; Wed, 22 Jun 2022 16:10:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655914220; bh=qkvbnlGCwVKpXjb3+gCMI55oA3ZtTlwTKki2/hd9FxI=; h=From:To:Cc:Subject:Date:From; b=FPeYRKLqzLTmZrjoq5p4gSRiOu3W6juTDS5mi3jiPVHY2qC+b8rGb5B8lDV6FY7ME Hx9P6fyxZjoSLWfFMc3w+wOyVvcj7m/v6ZRcbr7YNaQF5uRsttUWet8OMXqs9xBxvu 5xdMzO9zBicPImf8O6rQJVPTKuEvWCigQoatB4KPHbO91RbBFTHbi2rDtqHftmRfQv i7fPe3z9mOzQ8tGzIjr76zu5weDcF7voi3npz/+LkI34fUB71OFkqr5y3hbashhRGt Y4CmBqvdnQJ3m1/ccY8cytiH3QWvh2LTnhNWz+Cn8tcIyZHCIpFtPiJiNSwMtIcLHH Cdfeb+1W5PThQ== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: mark.rutland@arm.com, will@kernel.org, catalin.marinas@arm.com, Ard Biesheuvel Subject: [PATCH v2] arm64: entry: simplify trampoline data page Date: Wed, 22 Jun 2022 18:10:10 +0200 Message-Id: <20220622161010.3845775-1-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5521; h=from:subject; bh=qkvbnlGCwVKpXjb3+gCMI55oA3ZtTlwTKki2/hd9FxI=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBisz7hGh2oP7Qhbb7v82DrqFZwvP84nVAurkIY2jMn +4p9ZHuJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYrM+4QAKCRDDTyI5ktmPJLPUC/ 9Ju9XkHyKO4YCcSBRa1G/FmGhq9bgcZyPkL1aH3sUaDjrHZe51noV8ZPfdQTc9ZJJ7WXX6wTECaA6e DHCAUlHojea7V7x3AHPX5rkj+6/C6p+v84PFq97Tzdd7UGU6+SFCGbpzeDQNSjV9yKXiyzXeBk2UmP IEP9pnc45GUU6pzy0aD17uMLGY6I4m6PUjsAgzc1M7VJ0e32pQMsgshkpq3AiRN6UMNPY9JzGvH6zu lc8H0BPcvpAraMyXjo0RiRDqZQo/0wq7zjXF3+Nl2DC+Fw576OIkBMMH15lCDX20Iy6JJl19Urdy4F UvjdNrVq1LmEBXWB/IrIK9suUHKxIKLXGCyMLWbh7CyEOLn2kHHCIfegd43D6BTd7PCO0+K6fgLN6m aGSE77lH8jyUweMMUWoeu+wVGA9c8LMaai0Mx7BWVBUxXx5pNakDu8Mcgy1yU2RDX8Tn0pn+oyvWv2 CGzFooBbwvkI2pxZQORVn85PrSexRg7A52dnotlgR4uO4= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220622_091021_706869_DC726334 X-CRM114-Status: GOOD ( 17.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Get rid of some clunky open coded arithmetic on section addresses, by emitting the trampoline data variables into a separate, dedicated r/o data section, and putting it at the next page boundary. This way, we can access the literals via single LDR instruction. While at it, get rid of other, implicit literals, and use ADRP/ADD or MOVZ/MOVK sequences, as appropriate. Note that the latter are only supported for CONFIG_RELOCATABLE=n (which is usually the case if CONFIG_RANDOMIZE_BASE=n), so update the CPP conditionals to reflect this. Acked-by: Mark Rutland Signed-off-by: Ard Biesheuvel --- v2: incorporate suggested comment by Mark and add his ack arch/arm64/include/asm/fixmap.h | 4 +- arch/arm64/kernel/entry.S | 53 ++++++++------------ arch/arm64/kernel/vmlinux.lds.S | 3 +- arch/arm64/mm/mmu.c | 10 ++-- 4 files changed, 30 insertions(+), 40 deletions(-) diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixmap.h index daff882883f9..71ed5fdf718b 100644 --- a/arch/arm64/include/asm/fixmap.h +++ b/arch/arm64/include/asm/fixmap.h @@ -62,10 +62,12 @@ enum fixed_addresses { #endif /* CONFIG_ACPI_APEI_GHES */ #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +#ifdef CONFIG_RELOCATABLE + FIX_ENTRY_TRAMP_TEXT4, /* one extra slot for the data page */ +#endif FIX_ENTRY_TRAMP_TEXT3, FIX_ENTRY_TRAMP_TEXT2, FIX_ENTRY_TRAMP_TEXT1, - FIX_ENTRY_TRAMP_DATA, #define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_TEXT1)) #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ __end_of_permanent_fixed_addresses, diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 5b82b9292400..254fe31c03a0 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -636,18 +636,28 @@ alternative_else_nop_endif */ .endm - .macro tramp_data_page dst - adr_l \dst, .entry.tramp.text - sub \dst, \dst, PAGE_SIZE - .endm - - .macro tramp_data_read_var dst, var -#ifdef CONFIG_RANDOMIZE_BASE - tramp_data_page \dst - add \dst, \dst, #:lo12:__entry_tramp_data_\var - ldr \dst, [\dst] + .macro tramp_data_read_var dst, var +#ifdef CONFIG_RELOCATABLE + ldr \dst, .L__tramp_data_\var + .ifndef .L__tramp_data_\var + .pushsection ".entry.tramp.rodata", "a", %progbits + .align 3 +.L__tramp_data_\var: + .quad \var + .popsection + .endif #else - ldr \dst, =\var + /* + * As !RELOCATABLE implies !RANDOMIZE_BASE the address is always a + * compile time constant (and hence not secret and not worth hiding). + * + * As statically allocated kernel code and data always live in the top + * 47 bits of the address space we can sign-extend bit 47 and avoid an + * instruction to load the upper 16 bits (which must be 0xFFFF). + */ + movz \dst, :abs_g2_s:\var + movk \dst, :abs_g1_nc:\var + movk \dst, :abs_g0_nc:\var #endif .endm @@ -695,7 +705,7 @@ alternative_else_nop_endif msr vbar_el1, x30 isb .else - ldr x30, =vectors + adr_l x30, vectors .endif // \kpti == 1 .if \bhb == BHB_MITIGATION_FW @@ -764,24 +774,7 @@ SYM_CODE_END(tramp_exit_native) SYM_CODE_START(tramp_exit_compat) tramp_exit 32 SYM_CODE_END(tramp_exit_compat) - - .ltorg .popsection // .entry.tramp.text -#ifdef CONFIG_RANDOMIZE_BASE - .pushsection ".rodata", "a" - .align PAGE_SHIFT -SYM_DATA_START(__entry_tramp_data_start) -__entry_tramp_data_vectors: - .quad vectors -#ifdef CONFIG_ARM_SDE_INTERFACE -__entry_tramp_data___sdei_asm_handler: - .quad __sdei_asm_handler -#endif /* CONFIG_ARM_SDE_INTERFACE */ -__entry_tramp_data_this_cpu_vector: - .quad this_cpu_vector -SYM_DATA_END(__entry_tramp_data_start) - .popsection // .rodata -#endif /* CONFIG_RANDOMIZE_BASE */ #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ /* @@ -932,7 +925,6 @@ NOKPROBE(call_on_irq_stack) * This clobbers x4, __sdei_handler() will restore this from firmware's * copy. */ -.ltorg .pushsection ".entry.tramp.text", "ax" SYM_CODE_START(__sdei_asm_entry_trampoline) mrs x4, ttbr1_el1 @@ -967,7 +959,6 @@ SYM_CODE_START(__sdei_asm_exit_trampoline) 1: sdei_handler_exit exit_mode=x2 SYM_CODE_END(__sdei_asm_exit_trampoline) NOKPROBE(__sdei_asm_exit_trampoline) - .ltorg .popsection // .entry.tramp.text #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 2d4a8f995175..8a078c0ee140 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -115,7 +115,8 @@ jiffies = jiffies_64; __entry_tramp_text_start = .; \ *(.entry.tramp.text) \ . = ALIGN(PAGE_SIZE); \ - __entry_tramp_text_end = .; + __entry_tramp_text_end = .; \ + *(.entry.tramp.rodata) #else #define TRAMP_TEXT #endif diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 626ec32873c6..be4d6c3f5692 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -665,13 +665,9 @@ static int __init map_entry_trampoline(void) __set_fixmap(FIX_ENTRY_TRAMP_TEXT1 - i, pa_start + i * PAGE_SIZE, prot); - if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) { - extern char __entry_tramp_data_start[]; - - __set_fixmap(FIX_ENTRY_TRAMP_DATA, - __pa_symbol(__entry_tramp_data_start), - PAGE_KERNEL_RO); - } + if (IS_ENABLED(CONFIG_RELOCATABLE)) + __set_fixmap(FIX_ENTRY_TRAMP_TEXT1 - i, + pa_start + i * PAGE_SIZE, PAGE_KERNEL_RO); return 0; }