From patchwork Tue Nov 29 14:18:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13058630 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 97859C4321E for ; Tue, 29 Nov 2022 14:20:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JeXXrNIhl7ry5+ZT3CKUbBHpdLTpUIhrXveDZuZ84Ts=; b=aRMZnvo18nurtX tTcArgecPWBUOUeG6oelZ2ktoRiVXCqD7bvyUu21tyJhuJ9uuS6xQ0jAwjlh3VK/SU8SyQs6klBeW JzE+mk1OsuyvHE6QoE68yCRQrfIfk9XJYSw00R1+lsernJy3Pc/cmedJmuxbBAcAOnLq0uCt5dac6 B9GOGcl03q79HwEJsH/eFlHmNllgxzJsyjYkxTvlm2mGfSYKS+x7rbow9Boylb+MQ6VHLWcgim/c/ 0gqRDjBRFr7biwKcBheoRvMNmWnF0nPWvZFFPbtIdd9rM69tmJu/Y3YIE/MZM+aC4NEOA4nG36QqP vzCMbd5PGxEoySpWagUg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p01S6-009AeR-BD; Tue, 29 Nov 2022 14:19:14 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p01RC-009AHS-0L for linux-arm-kernel@lists.infradead.org; Tue, 29 Nov 2022 14:18:19 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 4FF0EB81699; Tue, 29 Nov 2022 14:18:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 439C3C433D7; Tue, 29 Nov 2022 14:18:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669731495; bh=x72zQXffLuuNmZdX8TIc7uq8kqTybuuDkPbyv5eookQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sJhPhFVPnCmByJeGeuohM03FEPbv39+8eC0tg84NlAZaeUes+MvZKDOCfnVEeQt78 HO1diBrLhnG/cfN2wh3aQDE2kzNyWk93L7O4jM8w990676EWlfWgu0pIZIgrEtEWJu nghE14OLqjrCRhpl+PjcwIwpXmuc2HbeFeZSvMLDaKx0IFSDl1lfNXflD99r3wef2+ rcUXWckSHgXMURFpPcyzqMEtxGudsih7zuiVkJF010z7qeQ6KzlWYL9W8/hB6ttcMI lpc/xd74nI99dCbDqNkOtzuIYzTmsNb5F7J+tpbLTaBC/Hmy37Xz/zLvbKL2YiRSmm eTRUGr41GiDtA== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Marc Zyngier , Will Deacon , Mark Rutland , Kees Cook , Catalin Marinas , Mark Brown Subject: [PATCH 2/4] arm64: assembler: Add macros for return address protection Date: Tue, 29 Nov 2022 15:18:01 +0100 Message-Id: <20221129141803.1746898-3-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221129141803.1746898-1-ardb@kernel.org> References: <20221129141803.1746898-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4030; i=ardb@kernel.org; h=from:subject; bh=x72zQXffLuuNmZdX8TIc7uq8kqTybuuDkPbyv5eookQ=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBjhhSXma7NJVNOopbTlw5mJriiM0A2nnzX9EadvFiU pvnMBMSJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCY4YUlwAKCRDDTyI5ktmPJGKRC/ 9ETD3hxxKER96v8rcZlZJZttk16rhdzRiKYTSeJxnhAiWid0eLhq1fDLDCmv54f0fP6Twxnf03ZDim MVmiatQA1f4rV+V2sY7fCv4LMXjwZsoz3HG6sWMCcoNi9Wsd/kkC+GbGeGQaZ+9y08geRwYDIfswAF +w4J5BAtjv8+YIUaL2kC4g95D2IyH3kbjung9Vz4kumISOzjAEV3fPKjM8crV4gAxgJmKVcKzqYMoa AKPSBSUTklY/2peSJfaW/z4Qh0gfSnC7hco6vUu9xBsk1yOagVMrzQW+GslVj1Vtf1OmJMmLK+v277 S2V1Ks/RFcC88Y9UZtyCt1WIwZLauApHb7XofDpMEtiZzYlcPTfs7xSYE5aDtKuIxcw8FLdxIVkjKX wRFDLoZSKpSrMcZMrYZFrTK/nfPTV56xpj2Q/9HB1hcx/dtr7AKieW7bzDzDW34xmWdwPU6BkVsv/+ eDztgRSWLTS1IRIQ/zuv0Ko482EhHV1wiYehN5gH1thsk= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221129_061818_414643_803B168F X-CRM114-Status: GOOD ( 18.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When in-kernel pointer authentication is configured, emit PACIASP and AUTIASP instructions as well as shadow call stack pushes and pops, depending on the configuration. Note that dynamic shadow call stack makes this slightly tricky, as it depends on in-kernel BTI as well. The resulting code will never contain both PAC and shadow call stack operations, even if shadow call stack support is not configured as dynamic. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/assembler.h | 81 ++++++++++++++++++++ 1 file changed, 81 insertions(+) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 3d1714a7eb6411ba..99d74c29ab3cbe05 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -692,6 +692,85 @@ alternative_endif #endif .endm + /* + * protect_return_address - protect the return address value in + * register @reg, either by signing it using PAC and/or by storing it + * on the shadow call stack. + * + * The sequence below emits a shadow call stack push if the feature is + * enabled, and if in-kernel PAC is enabled as well, the instruction + * will be patched into a PACIA instruction involving the same register + * address (and SP as the modifier) if PAC is detected at runtime. + * + * If in-kernel BTI and dynamic shadow call stacks are also configured, + * it becomes a bit more tricky, because then, shadow call stacks will + * only be enabled on non-BTI hardware, regardless of the PAUTH state. + * In that case, we emit one of the following sequences. + * + * PAC+BTI enabled No PAC or BTI BTI without PAC PAC without BTI + * + * B 0f NOP B 0f NOP + * NOP SCS push SCS push NOP + * 0: PACIA NOP NOP PACIA + * + * Note that, due to the code patching occuring at function entry and + * exit, these macros must not be used in code that may execute before + * the boot CPU feature based code patching has completed. + */ + .macro protect_return_address, reg=x30 +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL +#if defined(CONFIG_DYNAMIC_SCS) && defined(CONFIG_ARM64_BTI_KERNEL) +alternative_if ARM64_BTI + b .L0_\@ +alternative_else_nop_endif +#endif +alternative_if_not ARM64_HAS_ADDRESS_AUTH +#endif +#ifdef CONFIG_SHADOW_CALL_STACK + str \reg, [x18], #8 +#endif +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL +#if !defined(CONFIG_SHADOW_CALL_STACK) || \ + (defined(CONFIG_DYNAMIC_SCS) && defined(CONFIG_ARM64_BTI_KERNEL)) +.L0_\@: nop +#endif +alternative_else +#if defined(CONFIG_DYNAMIC_SCS) && defined(CONFIG_ARM64_BTI_KERNEL) + nop +#endif + .arch_extension pauth + pacia \reg, sp +alternative_endif +#endif + .endm + + /* + * restore_return_address - restore the return address value in + * register @reg, either by authenticating it using PAC and/or + * reloading it from the shadow call stack. + */ + .macro restore_return_address, reg=x30 +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL +alternative_if ARM64_HAS_ADDRESS_AUTH + .arch_extension pauth + autia \reg, sp +alternative_else_nop_endif +#if defined(CONFIG_DYNAMIC_SCS) && defined(CONFIG_ARM64_BTI_KERNEL) +alternative_if ARM64_BTI + b .L0_\@ +alternative_else_nop_endif +#endif +alternative_if_not ARM64_HAS_ADDRESS_AUTH +#endif +#ifdef CONFIG_SHADOW_CALL_STACK + ldr \reg, [x18, #-8]! +#endif +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL +alternative_else_nop_endif +.L0_\@: +#endif + .endm + /* * frame_push - Push @regcount callee saved registers to the stack, * starting at x19, as well as x29/x30, and set x29 to @@ -699,6 +778,7 @@ alternative_endif * for locals. */ .macro frame_push, regcount:req, extra + protect_return_address __frame st, \regcount, \extra .endm @@ -710,6 +790,7 @@ alternative_endif */ .macro frame_pop __frame ld + restore_return_address .endm .macro __frame_regs, reg1, reg2, op, num