From patchwork Fri Oct 5 08:47:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Kristina_Mart=C5=A1enko?= X-Patchwork-Id: 10627745 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F08AD1515 for ; Fri, 5 Oct 2018 09:47:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EFAD029243 for ; Fri, 5 Oct 2018 09:47:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E317C29245; Fri, 5 Oct 2018 09:47:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 3EAB429243 for ; Fri, 5 Oct 2018 09:47:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=PRRoxRayksNyAwOkc3ZBPl0ODVzNhgWhnZNDTY9CE4o=; b=BV2ehmbURKm5AYthkeYdC6MITx slwi2yyQlu7nSbGQJ3rNk+ehiiHmRbwS1CI3wJAOKm2Cg3RvHG+J/amFnodvDSlJgS8rRFjcw3+8d s4+VM4SN06WOd4jH0iCwD1+VAvN8djyogUsO6BieCzzAJBQPzzda2hzhZ+naJ+5Ca4IOa/5Mvdo66 N11//nyhomQiMIcPLeX1I9Td428IaLNU0WIu42VL+tNp748XveeSU9WGqPJsJGw3QLblzl0Z6k7IJ JDjVqRYu+oaUBuewgbUEN0RK6mJDSeiX8FAEIdhim1ODdFl8HSibkJBe7bchtLYO9ZhwLWWBx+v8a mOM6t0Xw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g8Mhe-0002P2-7d; Fri, 05 Oct 2018 09:47:22 +0000 Received: from merlin.infradead.org ([2001:8b0:10b:1231::1]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1g8MhD-0002Cw-Qc for linux-arm-kernel@bombadil.infradead.org; Fri, 05 Oct 2018 09:46:56 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=merlin.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=C6x/TQUm9ZtEAE+LdWNDWRJqOYjlki5n7JYQdaR+hvY=; b=ZXdHYnXGYjf2Eh0byxd5NUuKz y1Y+gBGdZXZrjETrls4zvV6b7ZNX07dim5Zm+pG8cRUPJ2hAS41IXsuJxq1QmvsYUBJLUBhSaeSKx nblcwaMfw4cBf/D5+8Tk7o7jNSJ50Nra7jEHvUgautGpVYa3O756uacbR1nM63rG16HIMYxMn6DXy XHInCqX+pyz3OExPTiFCGBHYcu6XwMUKwhLVU1nlh/VtvbmJUsBfxdpiSBC8Px0Fw9Dm8iH2TpMd4 zvR5YpI7RmisqPpxdR5/6B7l0xXqQKGlIKwKJ9hREJLogbj/yOxQBhcrPSqtzPz+HFj2sCKfBlxwH leOgrhDNA==; Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70] helo=foss.arm.com) by merlin.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1g8LpF-0002Ai-1E for linux-arm-kernel@lists.infradead.org; Fri, 05 Oct 2018 08:51:10 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 54F6A1650; Fri, 5 Oct 2018 01:50:58 -0700 (PDT) Received: from moonbear.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 201433F5B3; Fri, 5 Oct 2018 01:50:54 -0700 (PDT) From: Kristina Martsenko To: linux-arm-kernel@lists.infradead.org Subject: [RFC 16/17] arm64: initialize and switch ptrauth kernel keys Date: Fri, 5 Oct 2018 09:47:53 +0100 Message-Id: <20181005084754.20950-17-kristina.martsenko@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181005084754.20950-1-kristina.martsenko@arm.com> References: <20181005084754.20950-1-kristina.martsenko@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181005_045109_251449_AE72DECC X-CRM114-Status: GOOD ( 22.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , linux-arch@vger.kernel.org, Andrew Jones , Jacob Bramley , Arnd Bergmann , Ard Biesheuvel , Marc Zyngier , Catalin Marinas , Adam Wallis , "Suzuki K . Poulose" , Will Deacon , Christoffer Dall , kvmarm@lists.cs.columbia.edu, Ramana Radhakrishnan , Amit Kachhap , Dave P Martin , linux-kernel@vger.kernel.org, Kees Cook MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Set up keys to use pointer auth in the kernel. Each task has its own APIAKey, which is initialized during fork. The key is changed during context switch and on kernel entry from EL0. A function that changes the key cannot return, so inline such functions. Signed-off-by: Mark Rutland Signed-off-by: Kristina Martsenko Reviewed-by: Amit Daniel Kachhap --- arch/arm64/include/asm/pointer_auth.h | 9 ++++++++- arch/arm64/include/asm/ptrauth-asm.h | 13 +++++++++++++ arch/arm64/include/asm/thread_info.h | 1 + arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/entry.S | 4 ++++ arch/arm64/kernel/process.c | 3 +++ arch/arm64/kernel/smp.c | 3 +++ 7 files changed, 33 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/pointer_auth.h b/arch/arm64/include/asm/pointer_auth.h index 0634f06c3af2..e94ca7df8dab 100644 --- a/arch/arm64/include/asm/pointer_auth.h +++ b/arch/arm64/include/asm/pointer_auth.h @@ -50,12 +50,13 @@ do { \ write_sysreg_s(__pki_v.hi, SYS_ ## k ## KEYHI_EL1); \ } while (0) -static inline void ptrauth_keys_switch(struct ptrauth_keys *keys) +static __always_inline void ptrauth_keys_switch(struct ptrauth_keys *keys) { if (!cpus_have_const_cap(ARM64_HAS_ADDRESS_AUTH)) return; __ptrauth_key_install(APIA, keys->apia); + isb(); } static __always_inline void ptrauth_cpu_enable(void) @@ -85,11 +86,17 @@ static inline unsigned long ptrauth_strip_insn_pac(unsigned long ptr) #define ptrauth_task_init_user(tsk) \ ptrauth_keys_init(&(tsk)->thread_info.keys_user) +#define ptrauth_task_init_kernel(tsk) \ + ptrauth_keys_init(&(tsk)->thread_info.keys_kernel) +#define ptrauth_task_switch(tsk) \ + ptrauth_keys_switch(&(tsk)->thread_info.keys_kernel) #else /* CONFIG_ARM64_PTR_AUTH */ #define __no_ptrauth #define ptrauth_strip_insn_pac(lr) (lr) #define ptrauth_task_init_user(tsk) +#define ptrauth_task_init_kernel(tsk) +#define ptrauth_task_switch(tsk) #define ptrauth_cpu_enable(tsk) #endif /* CONFIG_ARM64_PTR_AUTH */ diff --git a/arch/arm64/include/asm/ptrauth-asm.h b/arch/arm64/include/asm/ptrauth-asm.h index f50bdfc4046c..3ef1cc8903d5 100644 --- a/arch/arm64/include/asm/ptrauth-asm.h +++ b/arch/arm64/include/asm/ptrauth-asm.h @@ -16,11 +16,24 @@ alternative_if ARM64_HAS_ADDRESS_AUTH alternative_else_nop_endif .endm + .macro ptrauth_keys_install_kernel tsk, tmp +alternative_if ARM64_HAS_ADDRESS_AUTH + ldr \tmp, [\tsk, #(TSK_TI_KEYS_KERNEL + PTRAUTH_KEY_APIALO)] + msr_s SYS_APIAKEYLO_EL1, \tmp + ldr \tmp, [\tsk, #(TSK_TI_KEYS_KERNEL + PTRAUTH_KEY_APIAHI)] + msr_s SYS_APIAKEYHI_EL1, \tmp + isb +alternative_else_nop_endif + .endm + #else /* CONFIG_ARM64_PTR_AUTH */ .macro ptrauth_keys_install_user tsk, tmp .endm + .macro ptrauth_keys_install_kernel tsk, tmp + .endm + #endif /* CONFIG_ARM64_PTR_AUTH */ #endif /* __ASM_PTRAUTH_ASM_H */ diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index ea9272fb52d4..e3ec5345addc 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -46,6 +46,7 @@ struct thread_info { int preempt_count; /* 0 => preemptable, <0 => bug */ #ifdef CONFIG_ARM64_PTR_AUTH struct ptrauth_keys keys_user; + struct ptrauth_keys keys_kernel; #endif }; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index b6be0dd037fd..6c61c9722b47 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -47,6 +47,7 @@ int main(void) #endif #ifdef CONFIG_ARM64_PTR_AUTH DEFINE(TSK_TI_KEYS_USER, offsetof(struct task_struct, thread_info.keys_user)); + DEFINE(TSK_TI_KEYS_KERNEL, offsetof(struct task_struct, thread_info.keys_kernel)); #endif DEFINE(TSK_STACK, offsetof(struct task_struct, stack)); BLANK(); diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 1e925f6d2978..a4503da445f7 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -250,6 +250,10 @@ alternative_else_nop_endif msr sp_el0, tsk .endif + .if \el == 0 + ptrauth_keys_install_kernel tsk, x20 + .endif + /* * Registers that may be useful after this macro is invoked: * diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 857ae05cd04c..a866996610de 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -330,6 +330,8 @@ int copy_thread(unsigned long clone_flags, unsigned long stack_start, */ fpsimd_flush_task_state(p); + ptrauth_task_init_kernel(p); + if (likely(!(p->flags & PF_KTHREAD))) { *childregs = *current_pt_regs(); childregs->regs[0] = 0; @@ -426,6 +428,7 @@ __notrace_funcgraph struct task_struct *__switch_to(struct task_struct *prev, contextidr_thread_switch(next); entry_task_switch(next); uao_thread_switch(next); + ptrauth_task_switch(next); /* * Complete any pending TLB or cache maintenance on this CPU in case diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 09690024dce8..d952dd62c780 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -212,6 +212,7 @@ asmlinkage notrace void secondary_start_kernel(void) */ check_local_cpu_capabilities(); + ptrauth_task_switch(current); ptrauth_cpu_enable(); if (cpu_ops[cpu]->cpu_postboot) @@ -418,6 +419,8 @@ void __init __no_ptrauth smp_prepare_boot_cpu(void) jump_label_init(); cpuinfo_store_boot_cpu(); + ptrauth_task_init_kernel(current); + ptrauth_task_switch(current); ptrauth_cpu_enable(); }