From patchwork Wed Apr 27 10:22:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12828573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2A8F3C433EF for ; Wed, 27 Apr 2022 10:24:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=1SmUQkGrlGWxXVvxk0CK+wIuUwhZqH4XZrHtJNef9BQ=; b=wIf0grPRahIkXy oUHoLQ9mDvp3YiM+8vYm57FipvZ1czjmNaPryBC38qgAMDj24xmxf5IO9ixLLP4YS0aYFVfUnoGpN fe5U5DJQ8ccUt1q7nf14ETSN8O4/CrYchuW9i2DmBTfjnobtM4cFNdrY/ZQLVU71Cq9R2AfVFCMXA huXOBLWW5E8GYFxJqI27QSOjerjKM++S05RU3/9ikvrNWvh7PBwuvZDduDKJ0HCZXCI24GwjLa2bX I5bISqMcJ2dkRh5Y6nHGejHD8h3ZGG1bYqK3E0sYI9Eo5rDfsJsr3IOsZ0IpX/c8DkEMMH0jk1MWl JK+7nrsFvNXXv9lcxI0Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njepF-000v9V-FB; Wed, 27 Apr 2022 10:23:13 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njepC-000v80-37 for linux-arm-kernel@lists.infradead.org; Wed, 27 Apr 2022 10:23:11 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 0073CB825FA; Wed, 27 Apr 2022 10:23:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 36CABC385AE; Wed, 27 Apr 2022 10:23:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1651054984; bh=M+szED1BsHeoGvMAgSMQaHW46ADaPw+SBNrZuHCIJqA=; h=From:To:Cc:Subject:Date:From; b=OjeXOFgJkZBudSEEGuABBqP0W74Z+UzoUEetSB/NBvwLeQxQrN8BKkW1xVYRMTay7 hXXEo7gy/Y0WBI/w7vMDWjs0LhnPAZqMzNqfUby/tYUuV0XLjUmv9KEOhqDB8I2168 kkNekjhXzjwSwW/7E6z0fpchAa8xiWPGusfKNggQX4dcM3hDTQDPmG0yeIMcXr8jtk GWWgxNl33UqOQBlSDOtEjnQIN+DfVuObcJidIP7IOPsiI1WH6VXAE/bZHrq+CVNvQt Sl2v05UkcL37jK5uS9cyawkzQNRCWpCwzFEBVJTKhp/nvyyxJ4epEd4w55wMBucDrv X4Iyyao6tzXwA== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, Ard Biesheuvel , James Morse Subject: [PATCH] arm64: entry: simplify trampoline data page Date: Wed, 27 Apr 2022 12:22:55 +0200 Message-Id: <20220427102255.1336386-1-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5075; h=from:subject; bh=M+szED1BsHeoGvMAgSMQaHW46ADaPw+SBNrZuHCIJqA=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBiaRl+TlIGHFsSLNk+OcFJacgMhL/nHcmQzS+ydM/T gz3jPAGJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYmkZfgAKCRDDTyI5ktmPJAj0C/ 9pfMEgjIhyCLUOBljwr7O30xDm+BXfK7xlyeuQMMKLy7PRPZXWX41kqmeIwY0M7OzOmzCiJNEb7P/t q4KmUUOO0ULvyXbFOOey9CXhV49Ixagk3NCYi4Kjam57OcRhm2+WUlRjC5vvyPc6n8sjXxKRNvPlT3 AYH1ZHa96fNvtgjjH66T0Ahh1hoCWUyOLyGX9kZeXjdbyPxrqfVKXvR6O3tMZoXbJvcTPMtU6dX552 K3uF2scc33zoFBzzs6DSDQ5QEA1d2FJz9xdTceC6/9b9mDzxfC2kNTUxlZO6IP3/by3LyVT11ZDuql QxeXSclDIODaBhEIRmczrNXVih/S642FDfd3FuWOAnBUVdsZJhf7eZvvNf45mR20kWtIDm8sQ1qjft I9CtxI8JBcOBPGhTb0FIRQ/CJ1WX0AkSNYvT59qa4lmKYNRL9gJwui7ZYt8jHfIp10Gns3V+CtjN6z 7Gne2vKnF3a7GOwQFoCDTLDWuwp6KAnEAyGGtGg6zMRfE= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220427_032310_471452_0316C045 X-CRM114-Status: GOOD ( 14.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Get rid of some clunky open coded arithmetic on section addresses, by emitting the trampoline data variables into a separate, dedicated r/o data section, and putting it at the next page boundary. This way, we can access the literals via single LDR instruction. While at it, get rid of other, implicit literals, and use ADRP/ADD or MOVZ/MOVK sequences, as appropriate. Note that the latter are only supported for CONFIG_RELOCATABLE=n (which is usually the case if CONFIG_RANDOMIZE_BASE=n), so update the CPP conditionals to reflect this. Cc: James Morse Signed-off-by: Ard Biesheuvel Acked-by: Mark Rutland --- arch/arm64/include/asm/fixmap.h | 4 +- arch/arm64/kernel/entry.S | 45 ++++++-------------- arch/arm64/kernel/vmlinux.lds.S | 3 +- arch/arm64/mm/mmu.c | 10 ++--- 4 files changed, 22 insertions(+), 40 deletions(-) diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixmap.h index daff882883f9..71ed5fdf718b 100644 --- a/arch/arm64/include/asm/fixmap.h +++ b/arch/arm64/include/asm/fixmap.h @@ -62,10 +62,12 @@ enum fixed_addresses { #endif /* CONFIG_ACPI_APEI_GHES */ #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +#ifdef CONFIG_RELOCATABLE + FIX_ENTRY_TRAMP_TEXT4, /* one extra slot for the data page */ +#endif FIX_ENTRY_TRAMP_TEXT3, FIX_ENTRY_TRAMP_TEXT2, FIX_ENTRY_TRAMP_TEXT1, - FIX_ENTRY_TRAMP_DATA, #define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_TEXT1)) #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ __end_of_permanent_fixed_addresses, diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index ede028dee81b..aed2b41e05aa 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -636,18 +636,20 @@ alternative_else_nop_endif */ .endm - .macro tramp_data_page dst - adr_l \dst, .entry.tramp.text - sub \dst, \dst, PAGE_SIZE - .endm - - .macro tramp_data_read_var dst, var -#ifdef CONFIG_RANDOMIZE_BASE - tramp_data_page \dst - add \dst, \dst, #:lo12:__entry_tramp_data_\var - ldr \dst, [\dst] + .macro tramp_data_read_var dst, var +#ifdef CONFIG_RELOCATABLE + ldr \dst, .L__tramp_data_\var + .ifndef .L__tramp_data_\var + .pushsection ".entry.tramp.rodata", "a", %progbits + .align 3 +.L__tramp_data_\var: + .quad \var + .popsection + .endif #else - ldr \dst, =\var + movz \dst, :abs_g2_s:\var + movk \dst, :abs_g1_nc:\var + movk \dst, :abs_g0_nc:\var #endif .endm @@ -695,7 +697,7 @@ alternative_else_nop_endif msr vbar_el1, x30 isb .else - ldr x30, =vectors + adr_l x30, vectors .endif // \kpti == 1 .if \bhb == BHB_MITIGATION_FW @@ -764,24 +766,7 @@ SYM_CODE_END(tramp_exit_native) SYM_CODE_START(tramp_exit_compat) tramp_exit 32 SYM_CODE_END(tramp_exit_compat) - - .ltorg .popsection // .entry.tramp.text -#ifdef CONFIG_RANDOMIZE_BASE - .pushsection ".rodata", "a" - .align PAGE_SHIFT -SYM_DATA_START(__entry_tramp_data_start) -__entry_tramp_data_vectors: - .quad vectors -#ifdef CONFIG_ARM_SDE_INTERFACE -__entry_tramp_data___sdei_asm_handler: - .quad __sdei_asm_handler -#endif /* CONFIG_ARM_SDE_INTERFACE */ -__entry_tramp_data_this_cpu_vector: - .quad this_cpu_vector -SYM_DATA_END(__entry_tramp_data_start) - .popsection // .rodata -#endif /* CONFIG_RANDOMIZE_BASE */ #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ /* @@ -932,7 +917,6 @@ NOKPROBE(call_on_irq_stack) * This clobbers x4, __sdei_handler() will restore this from firmware's * copy. */ -.ltorg .pushsection ".entry.tramp.text", "ax" SYM_CODE_START(__sdei_asm_entry_trampoline) mrs x4, ttbr1_el1 @@ -967,7 +951,6 @@ SYM_CODE_START(__sdei_asm_exit_trampoline) 1: sdei_handler_exit exit_mode=x2 SYM_CODE_END(__sdei_asm_exit_trampoline) NOKPROBE(__sdei_asm_exit_trampoline) - .ltorg .popsection // .entry.tramp.text #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index edaf0faf766f..17e554be9198 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -117,7 +117,8 @@ jiffies = jiffies_64; __entry_tramp_text_start = .; \ *(.entry.tramp.text) \ . = ALIGN(PAGE_SIZE); \ - __entry_tramp_text_end = .; + __entry_tramp_text_end = .; \ + *(.entry.tramp.rodata) #else #define TRAMP_TEXT #endif diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 626ec32873c6..be4d6c3f5692 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -665,13 +665,9 @@ static int __init map_entry_trampoline(void) __set_fixmap(FIX_ENTRY_TRAMP_TEXT1 - i, pa_start + i * PAGE_SIZE, prot); - if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) { - extern char __entry_tramp_data_start[]; - - __set_fixmap(FIX_ENTRY_TRAMP_DATA, - __pa_symbol(__entry_tramp_data_start), - PAGE_KERNEL_RO); - } + if (IS_ENABLED(CONFIG_RELOCATABLE)) + __set_fixmap(FIX_ENTRY_TRAMP_TEXT1 - i, + pa_start + i * PAGE_SIZE, PAGE_KERNEL_RO); return 0; }