From patchwork Thu Oct 10 11:33:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13829994 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3B97ECF11D2 for ; Thu, 10 Oct 2024 11:44:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=MQAt5SKeBz56FAHiHxrYIQX+VEnQ+LorFKx32LG+SI8=; b=0/junopiBQBNj4FkFQ2cU/Sncu nE7qFPNifOwsKT3xmWF2vno3gw52a11AWEsGAZJd3l/0O6gx4exrhVBEMwjJ4XjoWIIwwqGJ8eatF i3PpaXXe5XUT88940XjU3iT6Qrle4U3JRV0HUFl80WXwGxpPjjbU4w/aukVd0iwWFG/kZCcwlW7og NrBypXr4+H6ix0A7c3ZZX8zenQu1XOL3S18ZrH2oT6gxUXcvwGYFPZe8Bk+GJ5N7I3WTDbFiQJxMh icUFICa9dzZPq1H/uwwY/gFyx+YdZiv8pwmQQj3xn24rT2yUGK5pcgM7QosfMzIETYCXp+zFd9IXW nc/yeoDQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1syraV-0000000Cahf-0sYo; Thu, 10 Oct 2024 11:44:11 +0000 Received: from mail-ej1-x62c.google.com ([2a00:1450:4864:20::62c]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1syrQY-0000000CYBp-3X6I for linux-arm-kernel@lists.infradead.org; Thu, 10 Oct 2024 11:34:00 +0000 Received: by mail-ej1-x62c.google.com with SMTP id a640c23a62f3a-a9982159d98so132436566b.1 for ; Thu, 10 Oct 2024 04:33:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1728560033; x=1729164833; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=MQAt5SKeBz56FAHiHxrYIQX+VEnQ+LorFKx32LG+SI8=; b=FDqOGqgDQJl5n88mpoc1tD+Tn6LNVr/aHAkWFbu4bqobTrgXq3PxoUHCADSPLJK5BT 3b5oFqA94zwIDHTSib8+6aCaG/NYObrSbER4N8Eay4CDRRaL9ndvddsnvnpzratZQlwJ /fANtG4Mp9YNfpJnfcCQOPPcLXXQ9lZ9mXzOvqadjM/ZZWqlnkf/+HZqTNdl+AphNJk1 pkYJWFdqMzd4dtAqsEiVEe0rEyt/OQj6Rbcm/yxfip03PTbfPVlGu5QHwVPs3AyvSeRw 0Y1ZozJIdr06XYHx7HkvROHzqSfRauJNepTdsNoOx2u0qIg4bWqD2Ffsse0GQSRN1cZV W1uQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728560033; x=1729164833; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MQAt5SKeBz56FAHiHxrYIQX+VEnQ+LorFKx32LG+SI8=; b=jjdFv3X4voWntKBSwjen+uOc9cspCRW5jx3EQ353nBn43DTyGgaYeL38G1f9Sqb3Fq s/h95HPjtosrruE1YL2YysOTpDQbiU8E0/dR1Ewwjvne6P9IZVQaoaHjjMN7u8jP8ztk 8SvJpwykiLkkOo5saOOiEuqdNSdP+2r0JThdknKKIZpphK0NO2Ut85mE61DMV4xxutiw XhZZspNuaAc694R74G7u4ajOdgTh9l3pU3s1zGsNijAdAqy3+VKNTTRDrYm8pDpANnqU bn2W7tSVckimI+ili8xxhe02MOryBX4YBHnylEiZ4AsllyGNhY/wjEo/M2M0QP39YB8B xAoA== X-Gm-Message-State: AOJu0YxsvnRPEkKnCTs1FRbAv3QMQUbAxU/JwRebuWAFF+x2b7kIR3Xg yU2lBH79Xd/zBnA2uQG4hmKSUt0Bir0P6wUnah1VFg+GJIQ6ygMKlZhUhVqDc7s= X-Google-Smtp-Source: AGHT+IECf+nIoR64vGLK0N0tRyy3m6kGQh9nhQM+SGfR2yrRMxhPLMUSOA5Znclos1a6qtcyfJhm9w== X-Received: by 2002:a17:907:3e23:b0:a8a:78bb:1e2 with SMTP id a640c23a62f3a-a998d117c0bmr654465266b.6.1728560033384; Thu, 10 Oct 2024 04:33:53 -0700 (PDT) Received: from lino.lan ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a99a7f25406sm77619066b.69.2024.10.10.04.33.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Oct 2024 04:33:52 -0700 (PDT) From: Linus Walleij Date: Thu, 10 Oct 2024 13:33:44 +0200 Subject: [PATCH RFC 06/28] ARM: entry: Invoke syscalls using C MIME-Version: 1.0 Message-Id: <20241010-arm-generic-entry-v1-6-b94f451d087b@linaro.org> References: <20241010-arm-generic-entry-v1-0-b94f451d087b@linaro.org> In-Reply-To: <20241010-arm-generic-entry-v1-0-b94f451d087b@linaro.org> To: Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241010_043354_948138_3942F28F X-CRM114-Status: GOOD ( 27.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This makes the assembly invoking syscalls switch over to doing this from C with a small assembly stub to actually jump into the syscall. Split the syscall invocation into two paths: one for plain invocation and one for tracing (also known as "reload" as it was reloading the registers from regs). We rename this path with the infix "trace" as that code will be trace-specific as we move code over to C. Some registers such as r1 and lr get cobbled during the C calls and need to be restored when we return. Right now the part in C doesn't do much more than check the syscall number to be valid (a test previously done with a cmp r #NR_syscalls inside the invoke_syscall macro) but we will gradually factor over more assembly to C that can then be switched to the generic entry code so the exercise gets a point. Tested with a full system boot and by issuing some command line tools with strace to make sure the tracing path still works. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 3 ++ arch/arm/kernel/Makefile | 3 +- arch/arm/kernel/entry-common.S | 65 ++++++++++++++++++++++++++++++++++++------ arch/arm/kernel/entry-header.S | 25 ---------------- arch/arm/kernel/syscall.c | 26 +++++++++++++++++ 5 files changed, 88 insertions(+), 34 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index fe4326d938c1..1d21f26ecf51 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -19,6 +19,9 @@ extern const unsigned long sys_call_table[]; +int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp); +int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp); + static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) { diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile index aaae31b8c4a5..d5a128a4228d 100644 --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -20,7 +20,8 @@ CFLAGS_REMOVE_return_address.o = -pg obj-y := elf.o entry-common.o irq.o opcodes.o \ process.o ptrace.o reboot.o io.o \ setup.o signal.o sigreturn_codes.o \ - stacktrace.o sys_arm.o time.o traps.o + stacktrace.o sys_arm.o time.o traps.o \ + syscall.o KASAN_SANITIZE_stacktrace.o := n KASAN_SANITIZE_traps.o := n diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index da5c2d4b62e5..57aa1084a047 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -254,21 +254,23 @@ ENTRY(vector_swi) #else str scno, [tsk, #TI_ABI_SYSCALL] #endif - /* - * Reload the registers that may have been corrupted on entry to - * the syscall assembly (by tracing or context tracking.) - */ - TRACE( ldmia sp, {r0 - r3} ) - + mov r1, sp @ put regs into r1 ldr r10, [tsk, #TI_FLAGS] @ check for syscall tracing stmdb sp!, {r4, r5} @ push fifth and sixth args tst r10, #_TIF_SYSCALL_WORK @ are we tracing syscalls? bne __sys_trace - invoke_syscall tbl, scno, r10, __ret_fast_syscall + mov r0, tbl + /* r1 already contains regs */ + mov r2, scno @ syscall number from r7 + badr r3, __ret_fast_syscall + bl invoke_syscall + /* Restore regs into r1 and lr after C call */ + badr lr, __ret_fast_syscall add r1, sp, #S_OFF + 2: cmp scno, #(__ARM_NR_BASE - __NR_SYSCALL_BASE) eor r0, scno, #__NR_SYSCALL_BASE @ put OS number back bcs arm_syscall @@ -301,7 +303,16 @@ __sys_trace: add r0, sp, #S_OFF bl syscall_trace_enter mov scno, r0 - invoke_syscall tbl, scno, r10, __sys_trace_return, reload=1 + mov r2, r0 @ scno into r2 + add r1, sp, #S_R0 + S_OFF @ pointer to regs + mov r0, tbl + badr r3, __sys_trace_return + bl invoke_syscall_trace + + /* Restore regs into r1 and lr after C call */ + add r1, sp, #S_R0 + S_OFF @ pointer to regs + badr lr, __sys_trace_return + cmp scno, #-1 @ skip the syscall? bne 2b add sp, sp, #S_OFF @ restore stack @@ -415,6 +426,44 @@ sys_mmap2: b sys_mmap_pgoff ENDPROC(sys_mmap2) +/* + * This call wants: + * r0: syscall table + * r1: regs + * r2: syscall number + * r3: pointer to return function + */ +SYM_TYPED_FUNC_START(invoke_syscall_asm) +#ifdef CONFIG_CPU_SPECTRE + csdb +#endif + mov tbl, r0 + mov scno, r2 + mov lr, r3 @ return address + ldmia r1, {r0 - r3} @ reload r0-r3 + /* Arguments 5 and 6 are (hopefully) on the stack */ + ldr pc, [tbl, scno, lsl #2] @ call sys_* routine +SYM_FUNC_END(invoke_syscall_asm) + +/* + * This call wants: + * r0: syscall table + * r1: regs + * r2: syscall number + * r3: pointer to return function + */ +SYM_TYPED_FUNC_START(invoke_syscall_trace_asm) +#ifdef CONFIG_CPU_SPECTRE + csdb +#endif + mov tbl, r0 + mov scno, r2 + mov lr, r3 @ return address + ldmia r1, {r0 - r6} @ reload r0-r6 + stmia sp, {r4, r5} @ update stack arguments + ldr pc, [tbl, scno, lsl #2] @ call sys_* routine +SYM_FUNC_END(invoke_syscall_trace_asm) + #ifdef CONFIG_OABI_COMPAT /* diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 99411fa91350..52b4fa97226d 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -389,31 +389,6 @@ ALT_UP_B(.L1_\@) #endif .endm - .macro invoke_syscall, table, nr, tmp, ret, reload=0 -#ifdef CONFIG_CPU_SPECTRE - mov \tmp, \nr - cmp \tmp, #NR_syscalls @ check upper syscall limit - movcs \tmp, #0 - csdb - badr lr, \ret @ return address - .if \reload - add r1, sp, #S_R0 + S_OFF @ pointer to regs - ldmiacc r1, {r0 - r6} @ reload r0-r6 - stmiacc sp, {r4, r5} @ update stack arguments - .endif - ldrcc pc, [\table, \tmp, lsl #2] @ call sys_* routine -#else - cmp \nr, #NR_syscalls @ check upper syscall limit - badr lr, \ret @ return address - .if \reload - add r1, sp, #S_R0 + S_OFF @ pointer to regs - ldmiacc r1, {r0 - r6} @ reload r0-r6 - stmiacc sp, {r4, r5} @ update stack arguments - .endif - ldrcc pc, [\table, \nr, lsl #2] @ call sys_* routine -#endif - .endm - /* * These are the registers used in the syscall handler, and allow us to * have in theory up to 7 arguments to a function - r0 to r6. diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c new file mode 100644 index 000000000000..be193266299f --- /dev/null +++ b/arch/arm/kernel/syscall.c @@ -0,0 +1,26 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include + +int invoke_syscall_asm(void *table, struct pt_regs *regs, int scno, void *retp); + +__visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp) +{ + if (scno < NR_syscalls) + /* Doing this with return makes sure the stack gets pop:ed */ + return invoke_syscall_asm(table, regs, scno, retp); + + return 0; +} + +int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno, void *retp); + +__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp) +{ + if (scno < NR_syscalls) + /* Doing this with return makes sure the stack gets pop:ed */ + return invoke_syscall_trace_asm(table, regs, scno, retp); + + return 0; +}