From patchwork Fri Apr 16 12:50:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Kiss X-Patchwork-Id: 12207617 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AD7CC43462 for ; Fri, 16 Apr 2021 12:52:46 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B1DF4610F7 for ; Fri, 16 Apr 2021 12:52:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B1DF4610F7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=I1PC1PH7Ikw2uCMWV+fW7iXpOQXrMtQFfnWb3ZAft9Q=; b=BHFX1XmVvwR+mK0kfjBIh7jFL V2sShzO5N+P32k4MV3bkGFznfxfRz3aTieIEKKVmx8NuL4NCETVOt8tueCLh24MWRrDzCENdjrqjj Am+wsd+9KbI0HopuQk0EhJIEgKuinGysfyfKx01hf/svfvnAdPUGHFlqgo3HHM3afjGUlzRehA0Jl 8pebD0+O2ZfFCH9zBjops6EOO2ivEYggnZBXssqPFUD7jJK44rAaEyo2SPYm/Fn79kUCyDDoA1hZC SegWlT0p0vDOC2ZGSSo29RqRiSXRmqL76QMJZiDp/iyiRbh7Yua2bzFMGot7Wg/yPL7Lzr3UnW/UW /UwCfpNYQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lXNw8-0027MW-TT; Fri, 16 Apr 2021 12:51:05 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lXNvv-0027Ko-Ej for linux-arm-kernel@desiato.infradead.org; Fri, 16 Apr 2021 12:50:51 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=jDMnabmURYBK39sHjREThl6k0r37qbvrRU/5PqH59sA=; b=H3r3VprRiEJY63ZvkvRKKu5SoP GscOuhAo6cYz+JHIMwp9FoPB9DWsIkh1M3vRKZ+QTkfqEv2+sCTsAKsFO+FA6pADjOO0h6kYQ6Vhp 8qcd9ZIyuaTI965+iK48NBp3CQFN5s9VrqV4AVTlpLc+lsDtU3IukopZKVN3dj/rtp9MKVqfCpfQf zLbMafWVJ111Qr+QWhoHgv3JU6k/MtF0kZKqPhnqHFCm0vpa5qJ2aRYLI5vUyMAS6XiOTqUgOMidM dvXFjpkSUNG858merQxAHLU1yCITHknXIfF836x1pP5Ez8LkP7WSme9VsDW3mIiYxeNihIWC0DGH5 bLdvGIgQ==; Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lXNvs-009N5r-7b for linux-arm-kernel@lists.infradead.org; Fri, 16 Apr 2021 12:50:50 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 39E9C12FC; Fri, 16 Apr 2021 05:50:41 -0700 (PDT) Received: from e120529.arm.com (unknown [10.37.12.5]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 05CF63F85F; Fri, 16 Apr 2021 05:50:39 -0700 (PDT) From: Daniel Kiss To: will@kernel.org Cc: Ard.Biesheuvel@arm.com, Catalin.Marinas@arm.com, linux-arm-kernel@lists.infradead.org, pcc@google.com, Daniel Kiss Subject: [PATCH v4 1/2] arm64: Add ARM64_PTR_AUTH_KERNEL config option Date: Fri, 16 Apr 2021 14:50:14 +0200 Message-Id: <20210416125015.2651652-2-daniel.kiss@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416125015.2651652-1-daniel.kiss@arm.com> References: <20210416125015.2651652-1-daniel.kiss@arm.com> MIME-Version: 1.0 X-ARM-No-Footer: FoSSMail X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210416_055048_391074_AF0FB0F8 X-CRM114-Status: GOOD ( 21.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This patch add the ARM64_PTR_AUTH_KERNEL config and deals with the build aspect of it. Userspace support has no dependency on the toolchain therefore all toolchain checks and build flags are controlled the new config option. The default config behavior will not be changed. Signed-off-by: Daniel Kiss Acked-by: Will Deacon --- arch/arm64/Kconfig | 33 +++++++++++++++++++-------------- arch/arm64/Makefile | 2 +- arch/arm64/kernel/asm-offsets.c | 2 ++ drivers/misc/lkdtm/bugs.c | 6 +++--- 4 files changed, 25 insertions(+), 18 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index e4e1b6550115..ebee6d396d58 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1492,12 +1492,6 @@ menu "ARMv8.3 architectural features" config ARM64_PTR_AUTH bool "Enable support for pointer authentication" default y - depends on (CC_HAS_SIGN_RETURN_ADDRESS || CC_HAS_BRANCH_PROT_PAC_RET) && AS_HAS_PAC - # Modern compilers insert a .note.gnu.property section note for PAC - # which is only understood by binutils starting with version 2.33.1. - depends on LD_IS_LLD || LD_VERSION >= 23301 || (CC_IS_GCC && GCC_VERSION < 90100) - depends on !CC_IS_CLANG || AS_HAS_CFI_NEGATE_RA_STATE - depends on (!FUNCTION_GRAPH_TRACER || DYNAMIC_FTRACE_WITH_REGS) help Pointer authentication (part of the ARMv8.3 Extensions) provides instructions for signing and authenticating pointers against secret @@ -1509,13 +1503,6 @@ config ARM64_PTR_AUTH for each process at exec() time, with these keys being context-switched along with the process. - If the compiler supports the -mbranch-protection or - -msign-return-address flag (e.g. GCC 7 or later), then this option - will also cause the kernel itself to be compiled with return address - protection. In this case, and if the target hardware is known to - support pointer authentication, then CONFIG_STACKPROTECTOR can be - disabled with minimal loss of protection. - The feature is detected at runtime. If the feature is not present in hardware it will not be advertised to userspace/KVM guest nor will it be enabled. @@ -1526,6 +1513,24 @@ config ARM64_PTR_AUTH but with the feature disabled. On such a system, this option should not be selected. +config ARM64_PTR_AUTH_KERNEL + bool "Use pointer authentication for kernel" + default y + depends on ARM64_PTR_AUTH + depends on (CC_HAS_SIGN_RETURN_ADDRESS || CC_HAS_BRANCH_PROT_PAC_RET) && AS_HAS_PAC + # Modern compilers insert a .note.gnu.property section note for PAC + # which is only understood by binutils starting with version 2.33.1. + depends on LD_IS_LLD || LD_VERSION >= 233010000 || (CC_IS_GCC && GCC_VERSION < 90100) + depends on !CC_IS_CLANG || AS_HAS_CFI_NEGATE_RA_STATE + depends on (!FUNCTION_GRAPH_TRACER || DYNAMIC_FTRACE_WITH_REGS) + help + If the compiler supports the -mbranch-protection or + -msign-return-address flag (e.g. GCC 7 or later), then this option + will cause the kernel itself to be compiled with return address + protection. In this case, and if the target hardware is known to + support pointer authentication, then CONFIG_STACKPROTECTOR can be + disabled with minimal loss of protection. + This feature works with FUNCTION_GRAPH_TRACER option only if DYNAMIC_FTRACE_WITH_REGS is enabled. @@ -1617,7 +1622,7 @@ config ARM64_BTI_KERNEL bool "Use Branch Target Identification for kernel" default y depends on ARM64_BTI - depends on ARM64_PTR_AUTH + depends on ARM64_PTR_AUTH_KERNEL depends on CC_HAS_BRANCH_PROT_PAC_RET_BTI # https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94697 depends on !CC_IS_GCC || GCC_VERSION >= 100100 diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 5b84aec31ed3..3b9d156b3e5e 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -70,7 +70,7 @@ endif # off, this will be overridden if we are using branch protection. branch-prot-flags-y += $(call cc-option,-mbranch-protection=none) -ifeq ($(CONFIG_ARM64_PTR_AUTH),y) +ifeq ($(CONFIG_ARM64_PTR_AUTH_KERNEL),y) branch-prot-flags-$(CONFIG_CC_HAS_SIGN_RETURN_ADDRESS) := -msign-return-address=all # We enable additional protection for leaf functions as there is some # narrow potential for ROP protection benefits and no substantial diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index a36e2fc330d4..e4c6585909ed 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -45,6 +45,8 @@ int main(void) DEFINE(THREAD_CPU_CONTEXT, offsetof(struct task_struct, thread.cpu_context)); #ifdef CONFIG_ARM64_PTR_AUTH DEFINE(THREAD_KEYS_USER, offsetof(struct task_struct, thread.keys_user)); +#endif +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL DEFINE(THREAD_KEYS_KERNEL, offsetof(struct task_struct, thread.keys_kernel)); #endif #ifdef CONFIG_ARM64_MTE diff --git a/drivers/misc/lkdtm/bugs.c b/drivers/misc/lkdtm/bugs.c index 110f5a8538e9..b5b809b3e1ef 100644 --- a/drivers/misc/lkdtm/bugs.c +++ b/drivers/misc/lkdtm/bugs.c @@ -446,7 +446,7 @@ void lkdtm_DOUBLE_FAULT(void) #ifdef CONFIG_ARM64 static noinline void change_pac_parameters(void) { - if (IS_ENABLED(CONFIG_ARM64_PTR_AUTH)) { + if (IS_ENABLED(CONFIG_ARM64_PTR_AUTH_KERNEL)) { /* Reset the keys of current task */ ptrauth_thread_init_kernel(current); ptrauth_thread_switch_kernel(current); @@ -460,8 +460,8 @@ noinline void lkdtm_CORRUPT_PAC(void) #define CORRUPT_PAC_ITERATE 10 int i; - if (!IS_ENABLED(CONFIG_ARM64_PTR_AUTH)) - pr_err("FAIL: kernel not built with CONFIG_ARM64_PTR_AUTH\n"); + if (!IS_ENABLED(CONFIG_ARM64_PTR_AUTH_KERNEL)) + pr_err("FAIL: kernel not built with CONFIG_ARM64_PTR_AUTH_KERNEL\n"); if (!system_supports_address_auth()) { pr_err("FAIL: CPU lacks pointer authentication feature\n"); From patchwork Fri Apr 16 12:50:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Kiss X-Patchwork-Id: 12207615 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A2D4C433B4 for ; Fri, 16 Apr 2021 12:52:45 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 95AEF61107 for ; Fri, 16 Apr 2021 12:52:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 95AEF61107 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=HHVDMB/VRYAQb4HhJP9CzratrkRiYXLUZvcn15LOPQk=; b=XbtZitvLd6sS0BuIiIweKUnLI Kmpd4SCQ1tssZWK+PgLKApg3bcDo/3TOpgJRC0T//S9wV3EZqlFia+ZfNOGCU9AixdPfAwKPb9WmE f473CTQlPp7YVULPO5TziWgOzhal6bEleHD1B8ns+ldbEwWjmziWq6SXbKdkoS5/N4mIqo5ApogIW zaSELbg6kTsYomLdhGtDpDXWe/jZnJL/04FIK3muQqFK0N1T7I644BP9Z+3YLteg/vF98AbdpyTbc VR+3bO3flmwhTkC1l79w7FfDMp5JNEJRl/QrN66SBbE8tSj7H91UP7LFDPku32rR4N9QrJPnmRMal phhgnLlIQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lXNw0-0027Lo-0B; Fri, 16 Apr 2021 12:50:56 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lXNvv-0027Ki-Ej for linux-arm-kernel@desiato.infradead.org; Fri, 16 Apr 2021 12:50:51 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender :Reply-To:Content-Type:Content-ID:Content-Description; bh=Dixm/wuHnKn2BFS0iLSPkSpGOjgNlCyX8dMBNAPP3rg=; b=FTHc84rpz3VFGzYnx2Lq5eewfg jaNbcIjj4kvWvCcdIx4gkClYMPOx2or/wTw1ks9JmeZOjHHf17bK5hH4SaD2kTVNAkspkPlfikhdN 3xAFaRkM+0TKqAQV9apxaJhmP+K5Ue0YAiqkzHr650JULIpBDmGsHCCVliI0wYyz7UdRzYdGnpR4m ZsNkuJHPZ3sM20PHWsoKse9Pg2yuSp8tDebSxN0YpzeRbZtuLU0nRqLd3xZXGLuHIZjKwp28L1O6k WK+g3etTzqfOjY4blgZNb5zdx2aVFi5C5D3xPXm71+cWa/o3ljL7xbmrBwXWWEYHOY2QXU1xqfiOn 5bKaaHmA==; Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lXNvr-009N5q-Ay for linux-arm-kernel@lists.infradead.org; Fri, 16 Apr 2021 12:50:49 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B3DD11396; Fri, 16 Apr 2021 05:50:42 -0700 (PDT) Received: from e120529.arm.com (unknown [10.37.12.5]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 80DD13F85F; Fri, 16 Apr 2021 05:50:41 -0700 (PDT) From: Daniel Kiss To: will@kernel.org Cc: Ard.Biesheuvel@arm.com, Catalin.Marinas@arm.com, linux-arm-kernel@lists.infradead.org, pcc@google.com, Daniel Kiss Subject: [PATCH v4 2/2] arm64: Conditionally configure PTR_AUTH key of the kernel. Date: Fri, 16 Apr 2021 14:50:15 +0200 Message-Id: <20210416125015.2651652-3-daniel.kiss@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210416125015.2651652-1-daniel.kiss@arm.com> References: <20210416125015.2651652-1-daniel.kiss@arm.com> MIME-Version: 1.0 X-ARM-No-Footer: FoSSMail X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210416_055047_518733_51582E58 X-CRM114-Status: GOOD ( 15.05 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If the kernel is not compiled with CONFIG_ARM64_PTR_AUTH_KERNEL=y, then no PACI/AUTI instructions are expected while the kernel is running so the kernel's key will not be used. Write of a system regiters is expensive therefore avoid it not required. Signed-off-by: Daniel Kiss --- arch/arm64/include/asm/asm_pointer_auth.h | 63 +++++++++++++---------- arch/arm64/include/asm/pointer_auth.h | 44 +++++++++------- arch/arm64/include/asm/processor.h | 2 + arch/arm64/kernel/asm-offsets.c | 2 + 4 files changed, 65 insertions(+), 46 deletions(-) diff --git a/arch/arm64/include/asm/asm_pointer_auth.h b/arch/arm64/include/asm/asm_pointer_auth.h index 52dead2a8640..413cc36fdd5c 100644 --- a/arch/arm64/include/asm/asm_pointer_auth.h +++ b/arch/arm64/include/asm/asm_pointer_auth.h @@ -7,6 +7,38 @@ #include #include +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL + .macro __ptrauth_keys_install_kernel_nosync tsk, tmp1, tmp2, tmp3 + mov \tmp1, #THREAD_KEYS_KERNEL + add \tmp1, \tsk, \tmp1 + ldp \tmp2, \tmp3, [\tmp1, #PTRAUTH_KERNEL_KEY_APIA] + msr_s SYS_APIAKEYLO_EL1, \tmp2 + msr_s SYS_APIAKEYHI_EL1, \tmp3 + .endm + + .macro ptrauth_keys_install_kernel_nosync tsk, tmp1, tmp2, tmp3 +alternative_if ARM64_HAS_ADDRESS_AUTH + __ptrauth_keys_install_kernel_nosync \tsk, \tmp1, \tmp2, \tmp3 +alternative_else_nop_endif + .endm + + .macro ptrauth_keys_install_kernel tsk, tmp1, tmp2, tmp3 +alternative_if ARM64_HAS_ADDRESS_AUTH + __ptrauth_keys_install_kernel_nosync \tsk, \tmp1, \tmp2, \tmp3 + isb +alternative_else_nop_endif + .endm + +#else /* CONFIG_ARM64_PTR_AUTH_KERNEL */ + + .macro ptrauth_keys_install_kernel_nosync tsk, tmp1, tmp2, tmp3 + .endm + + .macro ptrauth_keys_install_kernel tsk, tmp1, tmp2, tmp3 + .endm + +#endif /* CONFIG_ARM64_PTR_AUTH_KERNEL */ + #ifdef CONFIG_ARM64_PTR_AUTH /* * thread.keys_user.ap* as offset exceeds the #imm offset range @@ -39,27 +71,6 @@ alternative_if ARM64_HAS_GENERIC_AUTH alternative_else_nop_endif .endm - .macro __ptrauth_keys_install_kernel_nosync tsk, tmp1, tmp2, tmp3 - mov \tmp1, #THREAD_KEYS_KERNEL - add \tmp1, \tsk, \tmp1 - ldp \tmp2, \tmp3, [\tmp1, #PTRAUTH_KERNEL_KEY_APIA] - msr_s SYS_APIAKEYLO_EL1, \tmp2 - msr_s SYS_APIAKEYHI_EL1, \tmp3 - .endm - - .macro ptrauth_keys_install_kernel_nosync tsk, tmp1, tmp2, tmp3 -alternative_if ARM64_HAS_ADDRESS_AUTH - __ptrauth_keys_install_kernel_nosync \tsk, \tmp1, \tmp2, \tmp3 -alternative_else_nop_endif - .endm - - .macro ptrauth_keys_install_kernel tsk, tmp1, tmp2, tmp3 -alternative_if ARM64_HAS_ADDRESS_AUTH - __ptrauth_keys_install_kernel_nosync \tsk, \tmp1, \tmp2, \tmp3 - isb -alternative_else_nop_endif - .endm - .macro __ptrauth_keys_init_cpu tsk, tmp1, tmp2, tmp3 mrs \tmp1, id_aa64isar1_el1 ubfx \tmp1, \tmp1, #ID_AA64ISAR1_APA_SHIFT, #8 @@ -69,7 +80,9 @@ alternative_else_nop_endif mrs \tmp2, sctlr_el1 orr \tmp2, \tmp2, \tmp1 msr sctlr_el1, \tmp2 +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL __ptrauth_keys_install_kernel_nosync \tsk, \tmp1, \tmp2, \tmp3 +#endif isb .Lno_addr_auth\@: .endm @@ -82,17 +95,11 @@ alternative_else_nop_endif .Lno_addr_auth\@: .endm -#else /* CONFIG_ARM64_PTR_AUTH */ +#else /* !CONFIG_ARM64_PTR_AUTH */ .macro ptrauth_keys_install_user tsk, tmp1, tmp2, tmp3 .endm - .macro ptrauth_keys_install_kernel_nosync tsk, tmp1, tmp2, tmp3 - .endm - - .macro ptrauth_keys_install_kernel tsk, tmp1, tmp2, tmp3 - .endm - #endif /* CONFIG_ARM64_PTR_AUTH */ #endif /* __ASM_ASM_POINTER_AUTH_H */ diff --git a/arch/arm64/include/asm/pointer_auth.h b/arch/arm64/include/asm/pointer_auth.h index b112a11e9302..32e540e3042f 100644 --- a/arch/arm64/include/asm/pointer_auth.h +++ b/arch/arm64/include/asm/pointer_auth.h @@ -30,23 +30,12 @@ struct ptrauth_keys_user { struct ptrauth_key apga; }; +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL + struct ptrauth_keys_kernel { struct ptrauth_key apia; }; -static inline void ptrauth_keys_init_user(struct ptrauth_keys_user *keys) -{ - if (system_supports_address_auth()) { - get_random_bytes(&keys->apia, sizeof(keys->apia)); - get_random_bytes(&keys->apib, sizeof(keys->apib)); - get_random_bytes(&keys->apda, sizeof(keys->apda)); - get_random_bytes(&keys->apdb, sizeof(keys->apdb)); - } - - if (system_supports_generic_auth()) - get_random_bytes(&keys->apga, sizeof(keys->apga)); -} - #define __ptrauth_key_install_nosync(k, v) \ do { \ struct ptrauth_key __pki_v = (v); \ @@ -69,6 +58,21 @@ static __always_inline void ptrauth_keys_switch_kernel(struct ptrauth_keys_kerne isb(); } +#endif /* CONFIG_ARM64_PTR_AUTH_KERNEL */ + +static inline void ptrauth_keys_init_user(struct ptrauth_keys_user *keys) +{ + if (system_supports_address_auth()) { + get_random_bytes(&keys->apia, sizeof(keys->apia)); + get_random_bytes(&keys->apib, sizeof(keys->apib)); + get_random_bytes(&keys->apda, sizeof(keys->apda)); + get_random_bytes(&keys->apdb, sizeof(keys->apdb)); + } + + if (system_supports_generic_auth()) + get_random_bytes(&keys->apga, sizeof(keys->apga)); +} + extern int ptrauth_prctl_reset_keys(struct task_struct *tsk, unsigned long arg); static inline unsigned long ptrauth_strip_insn_pac(unsigned long ptr) @@ -87,18 +91,22 @@ static __always_inline void ptrauth_enable(void) #define ptrauth_thread_init_user(tsk) \ ptrauth_keys_init_user(&(tsk)->thread.keys_user) -#define ptrauth_thread_init_kernel(tsk) \ - ptrauth_keys_init_kernel(&(tsk)->thread.keys_kernel) -#define ptrauth_thread_switch_kernel(tsk) \ - ptrauth_keys_switch_kernel(&(tsk)->thread.keys_kernel) #else /* CONFIG_ARM64_PTR_AUTH */ #define ptrauth_enable() #define ptrauth_prctl_reset_keys(tsk, arg) (-EINVAL) #define ptrauth_strip_insn_pac(lr) (lr) #define ptrauth_thread_init_user(tsk) +#endif /* CONFIG_ARM64_PTR_AUTH */ + +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL +#define ptrauth_thread_init_kernel(tsk) \ + ptrauth_keys_init_kernel(&(tsk)->thread.keys_kernel) +#define ptrauth_thread_switch_kernel(tsk) \ + ptrauth_keys_switch_kernel(&(tsk)->thread.keys_kernel) +#else #define ptrauth_thread_init_kernel(tsk) #define ptrauth_thread_switch_kernel(tsk) -#endif /* CONFIG_ARM64_PTR_AUTH */ +#endif /* CONFIG_ARM64_PTR_AUTH_KERNEL */ #endif /* __ASM_POINTER_AUTH_H */ diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index efc10e9041a0..9392c0dc2eeb 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -148,8 +148,10 @@ struct thread_struct { struct debug_info debug; /* debugging */ #ifdef CONFIG_ARM64_PTR_AUTH struct ptrauth_keys_user keys_user; +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL struct ptrauth_keys_kernel keys_kernel; #endif +#endif #ifdef CONFIG_ARM64_MTE u64 sctlr_tcf0; u64 gcr_user_excl; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index e4c6585909ed..443ec8b7622b 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -153,7 +153,9 @@ int main(void) DEFINE(PTRAUTH_USER_KEY_APDA, offsetof(struct ptrauth_keys_user, apda)); DEFINE(PTRAUTH_USER_KEY_APDB, offsetof(struct ptrauth_keys_user, apdb)); DEFINE(PTRAUTH_USER_KEY_APGA, offsetof(struct ptrauth_keys_user, apga)); +#ifdef CONFIG_ARM64_PTR_AUTH_KERNEL DEFINE(PTRAUTH_KERNEL_KEY_APIA, offsetof(struct ptrauth_keys_kernel, apia)); +#endif BLANK(); #endif return 0;