From patchwork Wed May 17 21:28:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13245734 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3D64AC77B7A for ; Wed, 17 May 2023 21:29:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=X1Z/snakEKDGcygWYLMtvPjp/7kJjZaHLGBVA7oi2Jg=; b=WhJcg8ze38oSuM x4fKpULF89RN4+p+zYZ8cXb7SwV/n1X3pZXTQGn9DnvWt3t4Q0q+D87/54lqufA9gN+k8xT0FH/mW HV81B+8j96aYwYaS/5dZQZ4GyqSeRuZIeyDTVVw2ybpdlKLSfPDe7VzPwj5q1F20Yg8OV6RwayB41 EPDXBULg0XLve4xBQebGXSMOSmxtk5YDu0DqiaxQCyZKocY/XnzmzPisefxeN6DBM4SslVpT3dX1W /iiB68w17301FAq4ytxOZLAgLNnpg5T55z5h0nbe6upGatZjdylUg5QApQeIumQVeVn7XA+GxJyay E7vcf4j9pxhQVbW0lm/A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pzOhT-00B74M-2a; Wed, 17 May 2023 21:28:47 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pzOh8-00B6sd-0K for linux-arm-kernel@lists.infradead.org; Wed, 17 May 2023 21:28:28 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 9E2B1639A3; Wed, 17 May 2023 21:28:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AADE9C4339C; Wed, 17 May 2023 21:28:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684358905; bh=4sHvPOqcHnYLKjQVAZLTltW9HQFEXAcIyKrAmkLS2fU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OuOpaAbf9duJOm7KRqTH3fcdFXUtZwnXq3Imwz5EXHcILS5ivWLaSvQEDRxwExk1V Gicrc3DQZxFdH/biTtPX7fiKlmwlHS9uDlLGrZ03EO+fBiGYnuwibdudRwIyJhPfKW WksvZPmD9d4E3GKOrWydUpWZ5U5qw9y9i9BKHOeGUQoYf5BsLh2BvPTgcuR6kM7nQZ vgUjpQeoS5s6uuEYHfnJWMuMSvY6rWmFD+1eFC+QiwHTEnUlZMu+YYbzPHUFVRlHaw Xi9l/WWdVtmFZcWaHqmj7TcXTLaI3mNPW33seiP5mf5FnIyHamCi1gHrMGoaRem05O +O2krFVUGrf1Q== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Linus Walleij , Arnd Bergmann , Russell King , Nicolas Pitre Subject: [PATCH v2 5/8] ARM: vfp: Use undef hook for handling VFP exceptions Date: Wed, 17 May 2023 23:28:05 +0200 Message-Id: <20230517212808.3895190-6-ardb@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230517212808.3895190-1-ardb@kernel.org> References: <20230517212808.3895190-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=10857; i=ardb@kernel.org; h=from:subject; bh=4sHvPOqcHnYLKjQVAZLTltW9HQFEXAcIyKrAmkLS2fU=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JISXV7YniG9lrRToulXm8kS/LjzzbrrW9M6X1VnHGEVOp3 2vuHOfqKGVhEONgkBVTZBGY/ffdztMTpWqdZ8nCzGFlAhnCwMUpABP5qcHwh8unSeXvW7OwlWzX OE6JvuDuYGl5k8KiutpinXvfWaXMZYwMM1Tvh1QX6am9do+aLbBz108d7bJGkwz1FSuTF+YqWRY zAwA= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230517_142826_222236_4BC7E0CD X-CRM114-Status: GOOD ( 33.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that the VFP support code has been reimplemented as a C function that takes a struct pt_regs pointer and an opcode, we can use the existing undef_hook framework to deal with undef exceptions triggered by VFP instructions instead of having special handling in assembler. Reviewed-by: Linus Walleij Signed-off-by: Ard Biesheuvel --- arch/arm/kernel/entry-armv.S | 53 ---------- arch/arm/vfp/Makefile | 2 +- arch/arm/vfp/entry.S | 31 ------ arch/arm/vfp/vfpmodule.c | 109 ++++++++++---------- 4 files changed, 54 insertions(+), 141 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index ba47f6aac5ff8692..0e40b2566f598b83 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -557,13 +557,6 @@ ENDPROC(__und_usr) * co-processor instructions. However, we have to watch out * for the ARM6/ARM7 SWI bug. * - * NEON is a special case that has to be handled here. Not all - * NEON instructions are co-processor instructions, so we have - * to make a special case of checking for them. Plus, there's - * five groups of them, so we have a table of mask/opcode pairs - * to check against, and if any match then we branch off into the - * NEON handler code. - * * Emulators may wish to make use of the following registers: * r0 = instruction opcode (32-bit ARM or two 16-bit Thumb) * r2 = PC value to resume execution after successful emulation @@ -575,25 +568,8 @@ ENDPROC(__und_usr) @ @ Fall-through from Thumb-2 __und_usr @ -#ifdef CONFIG_NEON - get_thread_info r10 @ get current thread - adr r6, .LCneon_thumb_opcodes - b 2f -#endif call_fpe: get_thread_info r10 @ get current thread -#ifdef CONFIG_NEON - adr r6, .LCneon_arm_opcodes -2: ldr r5, [r6], #4 @ mask value - ldr r7, [r6], #4 @ opcode bits matching in mask - cmp r5, #0 @ end mask? - beq 1f - and r8, r0, r5 - cmp r8, r7 @ NEON instruction? - bne 2b - b do_vfp @ let VFP handler handle this -1: -#endif tst r0, #0x08000000 @ only CDP/CPRT/LDC/STC have bit 27 tstne r0, #0x04000000 @ bit 26 set on both ARM and Thumb-2 reteq lr @@ -620,42 +596,13 @@ call_fpe: ret.w lr @ CP#7 ret.w lr @ CP#8 ret.w lr @ CP#9 -#ifdef CONFIG_VFP - W(b) do_vfp @ CP#10 (VFP) - W(b) do_vfp @ CP#11 (VFP) -#else ret.w lr @ CP#10 (VFP) ret.w lr @ CP#11 (VFP) -#endif ret.w lr @ CP#12 ret.w lr @ CP#13 ret.w lr @ CP#14 (Debug) ret.w lr @ CP#15 (Control) -#ifdef CONFIG_NEON - .align 6 - -.LCneon_arm_opcodes: - .word 0xfe000000 @ mask - .word 0xf2000000 @ opcode - - .word 0xff100000 @ mask - .word 0xf4000000 @ opcode - - .word 0x00000000 @ mask - .word 0x00000000 @ opcode - -.LCneon_thumb_opcodes: - .word 0xef000000 @ mask - .word 0xef000000 @ opcode - - .word 0xff100000 @ mask - .word 0xf9000000 @ opcode - - .word 0x00000000 @ mask - .word 0x00000000 @ opcode -#endif - do_fpe: add r10, r10, #TI_FPSTATE @ r10 = workspace ldr_va pc, fp_enter, tmp=r4 @ Call FP module USR entry point diff --git a/arch/arm/vfp/Makefile b/arch/arm/vfp/Makefile index 749901a72d6dc6c4..dfd64bc2b2fbdd06 100644 --- a/arch/arm/vfp/Makefile +++ b/arch/arm/vfp/Makefile @@ -8,4 +8,4 @@ # ccflags-y := -DDEBUG # asflags-y := -DDEBUG -obj-y += vfpmodule.o entry.o vfphw.o vfpsingle.o vfpdouble.o +obj-y += vfpmodule.o vfphw.o vfpsingle.o vfpdouble.o diff --git a/arch/arm/vfp/entry.S b/arch/arm/vfp/entry.S deleted file mode 100644 index 547c94c62cd3a66a..0000000000000000 --- a/arch/arm/vfp/entry.S +++ /dev/null @@ -1,31 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * linux/arch/arm/vfp/entry.S - * - * Copyright (C) 2004 ARM Limited. - * Written by Deep Blue Solutions Limited. - */ -#include -#include -#include -#include -#include -#include - -@ VFP entry point. -@ -@ r0 = instruction opcode (32-bit ARM or two 16-bit Thumb) -@ r2 = PC value to resume execution after successful emulation -@ r9 = normal "successful" return address -@ r10 = this threads thread_info structure -@ lr = unrecognised instruction return address -@ IRQs enabled. -@ -ENTRY(do_vfp) - mov r1, r0 @ pass trigger opcode via R1 - mov r0, sp @ pass struct pt_regs via R0 - bl vfp_support_entry @ dispatch the VFP exception - cmp r0, #0 @ handled successfully? - reteq r9 @ then use R9 as return address - ret lr @ pass to undef handler -ENDPROC(do_vfp) diff --git a/arch/arm/vfp/vfpmodule.c b/arch/arm/vfp/vfpmodule.c index 7572cb5b28a2eb08..58a9442add24b227 100644 --- a/arch/arm/vfp/vfpmodule.c +++ b/arch/arm/vfp/vfpmodule.c @@ -641,15 +641,37 @@ static int vfp_starting_cpu(unsigned int unused) return 0; } +static int vfp_kmode_exception(struct pt_regs *regs, unsigned int instr) +{ + /* + * If we reach this point, a floating point exception has been raised + * while running in kernel mode. If the NEON/VFP unit was enabled at the + * time, it means a VFP instruction has been issued that requires + * software assistance to complete, something which is not currently + * supported in kernel mode. + * If the NEON/VFP unit was disabled, and the location pointed to below + * is properly preceded by a call to kernel_neon_begin(), something has + * caused the task to be scheduled out and back in again. In this case, + * rebuilding and running with CONFIG_DEBUG_ATOMIC_SLEEP enabled should + * be helpful in localizing the problem. + */ + if (fmrx(FPEXC) & FPEXC_EN) + pr_crit("BUG: unsupported FP instruction in kernel mode\n"); + else + pr_crit("BUG: FP instruction issued in kernel mode with FP unit disabled\n"); + pr_crit("FPEXC == 0x%08x\n", fmrx(FPEXC)); + return 1; +} + /* - * vfp_support_entry - Handle VFP exception from user mode + * vfp_support_entry - Handle VFP exception * * @regs: pt_regs structure holding the register state at exception entry * @trigger: The opcode of the instruction that triggered the exception * * Returns 0 if the exception was handled, or an error code otherwise. */ -asmlinkage int vfp_support_entry(struct pt_regs *regs, u32 trigger) +static int vfp_support_entry(struct pt_regs *regs, u32 trigger) { struct thread_info *ti = current_thread_info(); u32 fpexc; @@ -657,6 +679,9 @@ asmlinkage int vfp_support_entry(struct pt_regs *regs, u32 trigger) if (unlikely(!have_vfp)) return -ENODEV; + if (!user_mode(regs)) + return vfp_kmode_exception(regs, trigger); + local_bh_disable(); fpexc = fmrx(FPEXC); @@ -722,7 +747,6 @@ asmlinkage int vfp_support_entry(struct pt_regs *regs, u32 trigger) * replay the instruction that trapped. */ fmxr(FPEXC, fpexc); - regs->ARM_pc -= 4; } else { /* Check for synchronous or asynchronous exceptions */ if (!(fpexc & (FPEXC_EX | FPEXC_DEX))) { @@ -743,78 +767,47 @@ asmlinkage int vfp_support_entry(struct pt_regs *regs, u32 trigger) fpexc |= FPEXC_DEX; } } -bounce: VFP_bounce(trigger, fpexc, regs); +bounce: regs->ARM_pc += 4; + VFP_bounce(trigger, fpexc, regs); } local_bh_enable(); return 0; } -#ifdef CONFIG_KERNEL_MODE_NEON - -static int vfp_kmode_exception(struct pt_regs *regs, unsigned int instr) -{ - /* - * If we reach this point, a floating point exception has been raised - * while running in kernel mode. If the NEON/VFP unit was enabled at the - * time, it means a VFP instruction has been issued that requires - * software assistance to complete, something which is not currently - * supported in kernel mode. - * If the NEON/VFP unit was disabled, and the location pointed to below - * is properly preceded by a call to kernel_neon_begin(), something has - * caused the task to be scheduled out and back in again. In this case, - * rebuilding and running with CONFIG_DEBUG_ATOMIC_SLEEP enabled should - * be helpful in localizing the problem. - */ - if (fmrx(FPEXC) & FPEXC_EN) - pr_crit("BUG: unsupported FP instruction in kernel mode\n"); - else - pr_crit("BUG: FP instruction issued in kernel mode with FP unit disabled\n"); - pr_crit("FPEXC == 0x%08x\n", fmrx(FPEXC)); - return 1; -} - -static struct undef_hook vfp_kmode_exception_hook[] = {{ +static struct undef_hook neon_support_hook[] = {{ .instr_mask = 0xfe000000, .instr_val = 0xf2000000, - .cpsr_mask = MODE_MASK | PSR_T_BIT, - .cpsr_val = SVC_MODE, - .fn = vfp_kmode_exception, + .cpsr_mask = PSR_T_BIT, + .cpsr_val = 0, + .fn = vfp_support_entry, }, { .instr_mask = 0xff100000, .instr_val = 0xf4000000, - .cpsr_mask = MODE_MASK | PSR_T_BIT, - .cpsr_val = SVC_MODE, - .fn = vfp_kmode_exception, + .cpsr_mask = PSR_T_BIT, + .cpsr_val = 0, + .fn = vfp_support_entry, }, { .instr_mask = 0xef000000, .instr_val = 0xef000000, - .cpsr_mask = MODE_MASK | PSR_T_BIT, - .cpsr_val = SVC_MODE | PSR_T_BIT, - .fn = vfp_kmode_exception, + .cpsr_mask = PSR_T_BIT, + .cpsr_val = PSR_T_BIT, + .fn = vfp_support_entry, }, { .instr_mask = 0xff100000, .instr_val = 0xf9000000, - .cpsr_mask = MODE_MASK | PSR_T_BIT, - .cpsr_val = SVC_MODE | PSR_T_BIT, - .fn = vfp_kmode_exception, -}, { - .instr_mask = 0x0c000e00, - .instr_val = 0x0c000a00, - .cpsr_mask = MODE_MASK, - .cpsr_val = SVC_MODE, - .fn = vfp_kmode_exception, + .cpsr_mask = PSR_T_BIT, + .cpsr_val = PSR_T_BIT, + .fn = vfp_support_entry, }}; -static int __init vfp_kmode_exception_hook_init(void) -{ - int i; +static struct undef_hook vfp_support_hook = { + .instr_mask = 0x0c000e00, + .instr_val = 0x0c000a00, + .fn = vfp_support_entry, +}; - for (i = 0; i < ARRAY_SIZE(vfp_kmode_exception_hook); i++) - register_undef_hook(&vfp_kmode_exception_hook[i]); - return 0; -} -subsys_initcall(vfp_kmode_exception_hook_init); +#ifdef CONFIG_KERNEL_MODE_NEON /* * Kernel-side NEON support functions @@ -919,8 +912,11 @@ static int __init vfp_init(void) * for NEON if the hardware has the MVFR registers. */ if (IS_ENABLED(CONFIG_NEON) && - (fmrx(MVFR1) & 0x000fff00) == 0x00011100) + (fmrx(MVFR1) & 0x000fff00) == 0x00011100) { elf_hwcap |= HWCAP_NEON; + for (int i = 0; i < ARRAY_SIZE(neon_support_hook); i++) + register_undef_hook(&neon_support_hook[i]); + } if (IS_ENABLED(CONFIG_VFPv3)) { u32 mvfr0 = fmrx(MVFR0); @@ -989,6 +985,7 @@ static int __init vfp_init(void) have_vfp = true; + register_undef_hook(&vfp_support_hook); thread_register_notifier(&vfp_notifier_block); vfp_pm_init();