From patchwork Wed Feb 12 11:22:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C1BFCC021A0 for ; Wed, 12 Feb 2025 11:25:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=2IJUdaO+5dTLVXPMwfcKz8isZKL74F9HfpoDdIYxpPA=; b=4u/FqDB+zR1O8mZo5zYA+Shlua 3JOrXralEJOJTqJySeSt3gEtJPPnBbflCrNfBg0EWp+pCEmAGFXuySzQ+ucCa6JreP/M1/iLEItzg 4Noo1sSArZKsGmgxKjBQiKjZf2yroupwvMKZzG1eqOS5O7g6OyHHPDsqdhECY8ygC/LmPTsg0ZqkX 9YhjoZV/CkfPUxWH+UMIsGILjLcniLva4IPRY7239i33teGPSsUvWLG7BpyHckoujFi83elzTiPWr sFBvU+N5dmXeKJFoAVsCePky42hWSpHqNAeK75NT1YcibvJpnuqvz3A5WvU1dMQnbKMq3kDuuUG5z ejRAN6ew==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiArh-00000007A8k-2a8Y; Wed, 12 Feb 2025 11:25:13 +0000 Received: from mail-lf1-x136.google.com ([2a00:1450:4864:20::136]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApg-000000079am-33nU for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:09 +0000 Received: by mail-lf1-x136.google.com with SMTP id 2adb3069b0e04-5450475df18so4526475e87.2 for ; Wed, 12 Feb 2025 03:23:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359387; x=1739964187; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=2IJUdaO+5dTLVXPMwfcKz8isZKL74F9HfpoDdIYxpPA=; b=ntSEU8/rxrSe/YTfU/u5jJa7OxVG0biyY190Yw2lPHq6Jx2IVH/PGt7Q/A/p66Nvau 1dupkbR0O2ZFFsyfrMUJ2GbJ/S8ZPHW/3ocMFvjR7E2S3nHMgazJByt5pQjRsjFHTfqO zsUa4w1ZyQjN9fRhjXTqcl8rEmgQIW6/EOkeMNCk/4RXamfixywasuJujvcVqEojorXk OyJxzSFF9Qu1dQT0El40/FkKrem0aDMwMnjkK5p2vS/jTL/lSwqz8UROy6saCmS9kvqd DdbpaA3/sZWl6aj+uDYgotwWT/4sYQKETgaMjSdnO7oI6fMUpfMRtdgO+ifwpIJBXrRt +gGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359387; x=1739964187; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2IJUdaO+5dTLVXPMwfcKz8isZKL74F9HfpoDdIYxpPA=; b=cKjCNsTbz3A/TsBHHeaSq7KJCXfdfVGw91sxWJhvbte8naJcEe0czrnkeksQJmgPyN JaR/EkD4tQWtRyORqULYvaIo4A92Tb5J8naYuInu0DBdNFaoimfvT6HCUjy8NmBKQxwP uVeCF6Ikls2Qr8F1x8I63xiIozQiyNPjBfM5cI4VyTLlj+a3vGVVn99KWmW+KResWf/H fCO/e1tZxFanxwFZSR9ZyBjZ/eaQwmknrqzkzGXjTafqHPPaTB6sZgMZaCEyOo11aUgY ZqeeADoz+PpTA1HabmTXQPx38uyaiKlJs/UqR6Zl9GSoTzfV5D/bj3h82rhTozm+o9fx Wb8A== X-Gm-Message-State: AOJu0YzczvbJpYq/5+gSAZbUtZmGrPOQzCIKa6Dz0sn2WdGlOHQEe3Ix kIvw62cPuFbnXPryn694sKe6xrDCA0W0NB1vY5dclfEXQqCskcp52bl0XRYw74U= X-Gm-Gg: ASbGncuj7IVKB6Jwm9pHaO5Ie7IrlvGcbXeVZUGgba8tGWv+DON2sgN76qtr0tCjKSO kxmp7SwQQFr0hln+dAACYP7aEtBSqrgQJNZyxVdz3FeG/xs6tQr2sifSrOwxHHj8pOG9mrlx+uM K/UJSwlHuDU76w6ClUqbH1yLAWI7UA7N3tJ4+j0bbKPZ0Nk6MmZWZ9/Bn5Bwa/HxUbpzBXfaBa6 mRL327o5YeGJsrcIkEBkjNdtLKxhGy6l+b7l21WgKR2tY0XNATeQIr8XSjxlSkeZfIjAx53goDv yWMYdwBpM3g9Nm7d/YAsO8o2/Q== X-Google-Smtp-Source: AGHT+IGpPfdcv9OC9/zIBoroOvzjRg04ZxGmAWsM7JxX7mbT/ANhbd0eNJDQrR2b6LwjPhGzC4w9WA== X-Received: by 2002:a05:6512:2809:b0:545:6c8:94de with SMTP id 2adb3069b0e04-54518181238mr925712e87.47.1739359387058; Wed, 12 Feb 2025 03:23:07 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:06 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:22:55 +0100 Subject: [PATCH v4 01/31] ARM: Prepare includes for generic entry MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-1-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032308_803304_DF8D34AC X-CRM114-Status: GOOD ( 13.99 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The generic entry code needs a static inline function for regs_irq_disabled() so add this, in further requires the signature of on_thread_stack() to be __always_inline so add this as well. Signed-off-by: Linus Walleij --- arch/arm/include/asm/ptrace.h | 5 +++++ arch/arm/include/asm/stacktrace.h | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/arm/include/asm/ptrace.h b/arch/arm/include/asm/ptrace.h index 6eb311fb2da06fa393f8be7caec8d997637a88b2..a01c66f0ad907882af2a383e53de8aca68c551e5 100644 --- a/arch/arm/include/asm/ptrace.h +++ b/arch/arm/include/asm/ptrace.h @@ -52,6 +52,11 @@ struct svc_pt_regs { #define fast_interrupts_enabled(regs) \ (!((regs)->ARM_cpsr & PSR_F_BIT)) +static inline int regs_irqs_disabled(struct pt_regs *regs) +{ + return !interrupts_enabled(regs); +} + /* Are the current registers suitable for user mode? * (used to maintain security in signal handlers) */ diff --git a/arch/arm/include/asm/stacktrace.h b/arch/arm/include/asm/stacktrace.h index f80a85b091d6c4ff365e15ae7100af1c3aed597f..815b5f256af164daa053f3bfd6cb2f54042a8d2b 100644 --- a/arch/arm/include/asm/stacktrace.h +++ b/arch/arm/include/asm/stacktrace.h @@ -26,7 +26,7 @@ struct stackframe { #endif }; -static inline bool on_thread_stack(void) +static __always_inline bool on_thread_stack(void) { unsigned long delta = current_stack_pointer ^ (unsigned long)current->stack; From patchwork Wed Feb 12 11:22:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 24C94C02198 for ; Wed, 12 Feb 2025 11:31:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=LeNYI4Yc1OrceUVt/GJDCyO7YwVHZ9lpFFe5ekDh0x0=; b=QoanVifvSAUUrzprJbp/Nu9RPX Yi1A1WgFbBdLCWK+h4wkCuhhLWzZL5c6cnlBCy/YrZ+B9S1LAP0ve6C8qv+CTGwBK/eouvGEmC1sX u3N3dOCVgSMdVCfVS+D9SuKwlU79YxG0GCFEszeyeYya+9D8g4DEPvPj5Sac7JqjOGO2zx1ds1ALR DftU1Bo7GaDJ0R6LFaCpe0Mo8Qu3cnqYVfnd6wLyVUJEKm6007JSwyJQRinDhXwL6W/Bfe8ugSySV tAax5/7GAe5JPbU80Raxf+OZF4o+zKSlh1A9kNLSHhRi/gH3ESIrsNi8ieOcgb8t0SAvb07H8yZXg 7CE7F9CQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiAxL-00000007Bf3-3MpY; Wed, 12 Feb 2025 11:31:03 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApm-000000079cm-3b40 for linux-arm-kernel@bombadil.infradead.org; Wed, 12 Feb 2025 11:23:15 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Sender:Reply-To:Content-ID:Content-Description; bh=LeNYI4Yc1OrceUVt/GJDCyO7YwVHZ9lpFFe5ekDh0x0=; b=im2KWtmsv9MLpYJVp+ScoE1Ser IEqVnSMa+j4bBTPq5jEG2ZX+XUKFDAdZxHtBpxeI2XmCRBkNwZb0V72vJVgJOAHd4ibwXnOmppJPd Bfw7/dK0NpKoTao9zn0PS9HPjssiZhifqfZ/98tFTVKhEgl0Wf1Eac0PidzEpcM6xC/d3y+cPmseE ocyapWG4Fs4xkRvZnq3Al9+UhIzhu6ipNjsF1YWA/AmfcbiOQBCsV89jeqcPBzJ5U3c+pPo4T7Yi8 OPiGqIvtE/szp2qazSBXCkRwq9UKrZ3tJeQxZ+EKC1C9MbGrPvLpQp/BBUCprTr/FD6LRzZSCE0jc N22LPLLw==; Received: from mail-lj1-x22c.google.com ([2a00:1450:4864:20::22c]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApj-00000000nmn-0dmO for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:13 +0000 Received: by mail-lj1-x22c.google.com with SMTP id 38308e7fff4ca-30227c56b11so68278681fa.3 for ; Wed, 12 Feb 2025 03:23:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359388; x=1739964188; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=LeNYI4Yc1OrceUVt/GJDCyO7YwVHZ9lpFFe5ekDh0x0=; b=MU+xc6xCewrnvBty1cIJcRN7T2Z7gNgaYGmpkwE+UMbLnICTHPa/oZajEpcUuCcTex jT0LVi2jPH3v6IWhVeXV/5I/GVk7A39sYPy/3lOdPqSo5KRqlL4n/1WeAqsf2Qci9i+6 lmNyjHUOv1BJ3YWpM2njeVp7H5zgyiqo/khrXDiVZxprd+PiQo2I9wID5FfGKqh2sSbY iWQNUFhaKxiaSmMEaGv9GxRBmjqO4qt3sEmDEQhDLjJor21vgulXF8Di1MBfpiDqfsUE N9RMSUg1xlJLve4J18QKDVnN7qgdF3pM0Y3AsqRbO1o+38gvGyXcCYOFyh8qAI4xwuWL WQEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359388; x=1739964188; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LeNYI4Yc1OrceUVt/GJDCyO7YwVHZ9lpFFe5ekDh0x0=; b=lkhUlW+BV64xpCr8EDpTIzw5DHwCJrE2rEIOEqwM78JDrbiavWGTkdEyUub7Dw/Jx1 VaWJ6bZ0KqAHiF3jsympSmQBdVzs9qew+JYJIFWiC3cABvU+uXqI15gXXkzLLuiXz3eO 3E8FDomsLJM3QtUOt6ZCHAQ8kjye9+kK6QDxKpApkmQyz1zQOj+d0LRi3noM7VX9TumC 1aOX05AAHyTmm21BDOL3ao1rpRSAvJ4qlCgK4QuW72tfi1vp8LlTmvYXPr/lO9tigbDM ZHwEEmznykSmyRvDVAkIEWdXpWceAtaW7Kwo0dOQqaMCVKUQifpvubv/+xfKtHwRU/Yx 1ivA== X-Gm-Message-State: AOJu0Yw1iCL5hzCd9AufUUXcGkYWG9QbOxQDDCnmZALK2K+9PC8Nupiz gYQf91uz6Kfq0deEnM1VhsTjSRzmoxFD5kfyzQnwEZAKDGhtHAlW0p2byGd2wfY= X-Gm-Gg: ASbGncsp1RHkfXRVzrsosWcCTF0CXG6OK0Et6zH3zlCUHmgr9oLF0A8eZSVPVQQFcud 8wCO4a7qNiYhn86uxXvev9ExhM2YSRhWcJm6n7UNJEsxq9tvLk4LwZYakWHa5k/aj4tkWrsBYqH tM7QroYJI2woWITE6IbWsuKZ+laty3t1u585a8qqXMmhFBUzRGhBztT7XWhn7YhOnYV/W3G8hbR xy9eKX0SV8TdD77lFyOsEj7nWQ0qFQdfTxgTe8tBvXLbH90ubTt/INjtHtbR12oaAln5OhbCmoi 7Mnqk9IIDux5OMKG/UzoMfoR5A== X-Google-Smtp-Source: AGHT+IF0bN8IrtjGLfeKlbBP9UpoS79dKnT/CxxK+D4rlHCjY6PG5wH4SvZpBeywkQxHf+e/L4Kf6g== X-Received: by 2002:a05:651c:50a:b0:308:eabd:2991 with SMTP id 38308e7fff4ca-30904fc99ccmr6630861fa.1.1739359387948; Wed, 12 Feb 2025 03:23:07 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:07 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:22:56 +0100 Subject: [PATCH v4 02/31] ARM: ptrace: Split report_syscall() MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-2-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_112311_454370_8E1286B1 X-CRM114-Status: GOOD ( 14.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The generic entry code requires that report_syscall() be provided in two parts: report_syscall_enter() and report_syscall_exit() so split the combined function in two. Signed-off-by: Linus Walleij --- arch/arm/kernel/ptrace.c | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/arch/arm/kernel/ptrace.c b/arch/arm/kernel/ptrace.c index c421a899fc84c476a672cbfe1c8ece8355d8512d..07b0daf47441f1f76a8af416acc74fa5ed770403 100644 --- a/arch/arm/kernel/ptrace.c +++ b/arch/arm/kernel/ptrace.c @@ -827,31 +827,45 @@ enum ptrace_syscall_dir { PTRACE_SYSCALL_EXIT, }; -static void report_syscall(struct pt_regs *regs, enum ptrace_syscall_dir dir) +static void report_syscall_enter(struct pt_regs *regs) { unsigned long ip; /* * IP is used to denote syscall entry/exit: - * IP = 0 -> entry, =1 -> exit + * IP = 0 -> entry */ ip = regs->ARM_ip; - regs->ARM_ip = dir; + regs->ARM_ip = PTRACE_SYSCALL_ENTER; - if (dir == PTRACE_SYSCALL_EXIT) - ptrace_report_syscall_exit(regs, 0); - else if (ptrace_report_syscall_entry(regs)) + if (ptrace_report_syscall_entry(regs)) current_thread_info()->abi_syscall = -1; regs->ARM_ip = ip; } +static void report_syscall_exit(struct pt_regs *regs) +{ + unsigned long ip; + + /* + * IP is used to denote syscall entry/exit: + * IP = 1 -> exit + */ + ip = regs->ARM_ip; + regs->ARM_ip = PTRACE_SYSCALL_EXIT; + + ptrace_report_syscall_exit(regs, 0); + + regs->ARM_ip = ip; +} + asmlinkage int syscall_trace_enter(struct pt_regs *regs) { int scno; if (test_thread_flag(TIF_SYSCALL_TRACE)) - report_syscall(regs, PTRACE_SYSCALL_ENTER); + report_syscall_enter(regs); /* Do seccomp after ptrace; syscall may have changed. */ #ifdef CONFIG_HAVE_ARCH_SECCOMP_FILTER @@ -892,5 +906,5 @@ asmlinkage void syscall_trace_exit(struct pt_regs *regs) trace_sys_exit(regs, regs_return_value(regs)); if (test_thread_flag(TIF_SYSCALL_TRACE)) - report_syscall(regs, PTRACE_SYSCALL_EXIT); + report_syscall_exit(regs); } From patchwork Wed Feb 12 11:22:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 16886C02198 for ; Wed, 12 Feb 2025 11:28:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=e5mhd6DszF79mgTOhWslIzEoODnT/4x94Ab5tm+ZGKw=; b=fE2gG47A1+QlIQ6ItDfsdSqU1R bFfzoxyuz7zR6iVLQGjb3gbKpUl5wazBOJQURyGliSlgMbfg+gamdFwsp7cie12xarQ3OeOAUGJcS 0v9UFGVJ+DJQOY04bI3EAsfwwCmi80RzwGqY659R/2wCoxP8ojBzJaGes0rWu4lZYufqWAKXkf/UW ndRF6iOjKkI6trYoi9d+ffJYl1okl2rywtQzhbhJ1SbDSidNi9ineMiCzrH/XTtVF3Jbugwvj8XrY MvHo5Jqzzi3VDnUn+MmNDp+2ySxs9kKGe4iAZJhVrHPsV5jsvOfo+8LKG0i3sOiC+HKe4VQJBWPDo 2DteZeaA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiAuU-00000007AvU-01r7; Wed, 12 Feb 2025 11:28:06 +0000 Received: from mail-lj1-f173.google.com ([209.85.208.173]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApj-000000079bT-1Zlj for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:12 +0000 Received: by mail-lj1-f173.google.com with SMTP id 38308e7fff4ca-308ec50eb3dso33312341fa.1 for ; Wed, 12 Feb 2025 03:23:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359389; x=1739964189; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=e5mhd6DszF79mgTOhWslIzEoODnT/4x94Ab5tm+ZGKw=; b=d8Th23ZBKl9TkYOW3Yd23Q/YJaJne8iXzauoDJlrXqSwpFxp0L5gGZHOga6hx0Vkdo YxMAx1mcJaXb/D8JV3uakKV4AAsy8rZbp8M+JHYCLOWKa3qiDvEkpdDTRgfHdKjtGL+w mFeSlGaCTZtkkrEkNKvyR0ODVcKMXdbLpxLA9jol4zCaCGrh2ToWEfG9O7m7jL9hibz1 XZ1aITNSpuoqqmSVFE5CcojAjRnSPcYV6pAUTJvu/+ceibKepUMo03LMn2NFi+NZAVDy LaIJoYaQ/4rC6MvmEY0hg6kkoaXlm9AXY4CbOseXIYyXkjMq5JZn9gdzd8fcyK4FGdso rpPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359389; x=1739964189; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=e5mhd6DszF79mgTOhWslIzEoODnT/4x94Ab5tm+ZGKw=; b=v4Wd4x01J6VUp6uylPKNAjtbtnX9CHGCyxaojPLPOnGBfY4v/+HQkn9TgeP5RFuXad 2hXsJZRAloM3kWjcWfSSV4vaep3a4A5Wu4prklY6/Y6hrCOfn7m3g4vGAmZHco31q0x2 TSV55OMsqAmWCA58PHcv7zSBANqEETpdD0ti230mqbmUmQW0k6MKk4a/TRvZizJHgWX+ ysptvRiIO2GVYRD/dTwfecxbJw5wDhkWmw+glyLS4oqM0zGOZXr2P4I10GEhUcWdyBMl /BOZDwnSYyxaPoNRxyRwJpCY3VAf4wBz1+lqKGb23eKBCrH8VsWuULzzMQGBjIMxkSRR +yiQ== X-Gm-Message-State: AOJu0Yy+hfsKKjkS2ch4vYGQYRtWyJY9Z5dMTRmr8F/6a9FzDdhFCtsu eLiJRkDhwr/7UxrAMmnBK3QtEpYDXjuzez0rOLM9iKgIQ7q2prwMZNKu3tuAaFo= X-Gm-Gg: ASbGncv/Ptwg9TX5mEDRFkMVZoHY550bFWp5MeJhnNezWYP3lvy6/lDJun7NFCaxoIV C6qAEKA2Hv/V0kuNnNY8gt66Zzona769nOd341539sywsgOH+M/7NGpB7dMHd8KrxP3AWtYwrt+ KXHiUR9GA35+j1fkxRCf2iZWKD3T4o9GIp4HpIM7eNdKVek0KcX7peXyumYPbndmwTknQegWx2R TNnRfH6gpAT+tH4xxKLMv+t+3TnlYJM6tZ/8Ofo7W4Ptu5icVQq7QDe5Um0YukCsGSkRrsCVmC7 4n15QGMV4Dxtlyv+8lU3sgJUeA== X-Google-Smtp-Source: AGHT+IHTMqYXojqkTP6hLJE6s2yoQfyjlXS0zVkP4hZmvDjU6HV8U27+O3ADPsPzDGK4vKbkksnjiQ== X-Received: by 2002:a05:651c:19aa:b0:308:ffa1:88f8 with SMTP id 38308e7fff4ca-309036d733cmr10944651fa.23.1739359388991; Wed, 12 Feb 2025 03:23:08 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:08 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:22:57 +0100 Subject: [PATCH v4 03/31] ARM: entry: Skip ret_slow_syscall label MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-3-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032311_416132_82C8DC48 X-CRM114-Status: GOOD ( 11.03 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Don't introduce this extra label anymore now that we have a clearly defined entry point for ret_to_user. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index f379c852dcb72fa5e960bba5621cc2152f0df0d3..2f93c2f6d5b8fdf6aaf3dda0192bde144fcf238d 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -102,7 +102,6 @@ ENDPROC(ret_fast_syscall) * do_work_pending() will update this state if necessary. */ ENTRY(ret_to_user) -ret_slow_syscall: #if IS_ENABLED(CONFIG_DEBUG_RSEQ) /* do_rseq_syscall needs interrupts enabled. */ enable_irq_notrace @ enable interrupts @@ -136,7 +135,7 @@ ENTRY(ret_from_fork) badrne lr, 1f retne r5 1: get_thread_info tsk - b ret_slow_syscall + b ret_to_user ENDPROC(ret_from_fork) /*============================================================================= @@ -310,13 +309,13 @@ __sys_trace_return_nosave: enable_irq_notrace mov r0, sp bl syscall_trace_exit - b ret_slow_syscall + b ret_to_user __sys_trace_return: str r0, [sp, #S_R0 + S_OFF]! @ save returned r0 mov r0, sp bl syscall_trace_exit - b ret_slow_syscall + b ret_to_user .macro syscall_table_start, sym .equ __sys_nr, 0 From patchwork Wed Feb 12 11:22:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971804 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EC31BC02198 for ; Wed, 12 Feb 2025 11:35:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=7LU1Mx0M1JJLQ6X7nOndn1GNJ7UBsOKiHS90jS7YqHQ=; b=BjJicqwK9nyr+ha4s4+jPv63iJ /wU4c10wUMlYH/lI3p4rY4B2vGErQ7dBqXMgn+nWureSOfJPZxQAnw5+5E8of3mFQ7IwxWhdktGJh bh9APoUJk73eNJyDwT5NUmrEcIGoEg/T5DDjo3u0sDJXLsps1oPnSYO3e+vjxQwZX3e3CD/Dj3nu/ smuPnBltmtFLT4kMneH+QDSDpGLBs/vNvQ2f4iSYNUI1hkljGWK1bdrGDnUgnCWEJvwzXXOYZQA0A 4FC5GZNA+vEdcqzuJzNivKEOsYmlsX58NXfnrww7IzyWUPIRT94pGSXQS9IPQQoZjwsvwxYPd+Mjs GIL3X7Wg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiB1X-00000007CjE-3uJk; Wed, 12 Feb 2025 11:35:23 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApp-000000079eA-3Fan for linux-arm-kernel@bombadil.infradead.org; Wed, 12 Feb 2025 11:23:17 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Sender:Reply-To:Content-ID:Content-Description; bh=7LU1Mx0M1JJLQ6X7nOndn1GNJ7UBsOKiHS90jS7YqHQ=; b=fTe/hhd2ZlvB0CdJY9F1jwjJrU /XttdT5aSu+KljBQQajgYIkcTkzFCC8svsDhWMAEvEN5k4FJ0g8vOynFpHYIUA7qu6UBWxPM9TpTy 9WVjJXvu352MP5r4LaoJGY0TdIOqge1D50/Fk4PohpUDoa3AmRwdTe92CAraUIj9cQSsshVPjvyhF xaFd0L3v2oXHrhU0PYs+HxT1UWRWRavCCMfg0oxupzBfBhEYIyFviwVAE9xMawWyB8slndmyjjXgx LyCnud8SoVgiAJ5McCNtbboXXu0ZoG0tuVnHH1ShJ9YhUEXs+1E5S9Udpz0X5c5c36lx81WnXxVH4 fNbbfnzA==; Received: from mail-lj1-x22c.google.com ([2a00:1450:4864:20::22c]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApm-00000000nn7-2EF1 for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:16 +0000 Received: by mail-lj1-x22c.google.com with SMTP id 38308e7fff4ca-308f04beb24so24630581fa.2 for ; Wed, 12 Feb 2025 03:23:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359393; x=1739964193; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=7LU1Mx0M1JJLQ6X7nOndn1GNJ7UBsOKiHS90jS7YqHQ=; b=SBQCUcG0XfiaOlx2vuYrtoiVcXhP51ovIVtMsZMGlFaRxtc8iWG2SMMeNq5j3tUUM1 GFmzfWPW5Xt6OY+J4eZKmOwrDHEa5AU75Tk4L+22JkfUhrkuFmRb+emgccGzO5nq/99d vW4yEY/bzP5mLoKHMhu8YsV2jVHrpUBZ0qAxFWxwCFZvcJ1mueJJwozVDeV6UC5akAGQ pRvXk3f4Evl6OsyDC3qlzp9932CjOfxuPbW+v6++uyLErjbW4bk0uHzvxBK5wWV6BHoW 48VpP0LdvetA8DthkAB7LsA6766QhR282DXfE5R8tc7+NZ4Wcnt2ivjQfPIhekvviUAw WVog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359393; x=1739964193; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7LU1Mx0M1JJLQ6X7nOndn1GNJ7UBsOKiHS90jS7YqHQ=; b=euwZYsD5OZ9darOOj7EYzo4y8mcTVvFP/0m7TjNEWEvAEMvH+MO8aVLEULYL9uKn6g ZbFKCJQ/TNhkhnYoNqUfyUiKq9iTMw4LKoeN0PpSeTdvtr7Kf3ecB9u4NjbPJHkhAZln z0J1CGj6uFVC28K9ZeVz5GKhHrM2ZsrdM8KDj0U9wUy074MG0FZ3nNU07dbuzQVPafdl giUPsI6LYmkCAjU3qgT4OLDwfSUWp34vx/KD5wTCxPtq+j7m55701aoR+1ljm1KcARCq /vW4LjqcGRv+U8Us9Bpunfx5RLlXz6DND1vSCjkDOrnq+i17nVKGMG6J9GFLV2Z4IRak xumA== X-Gm-Message-State: AOJu0Yw+R0YP24IoWOVyosKnR6aBkvY91qIQi517vI3rdNxAztSwwFqt ldIcEjpPTEfOGjh0VQ+cBiAYHkKbNH2XasAdEDMCDLPil7ETRTLyJiqTOyMXfZQ= X-Gm-Gg: ASbGncs8EzdEHcnyfAbpvQiSfFsatgG71poymcCQpMHO6aa/9PlQUlH3CRUVfkjpsPZ 355AEBDQ7UwZgb9VKLPI5Mf3pbZXijd0yefs8YsAymWW5VPc82QK2PTdT5lPnfroInRBeFPvu/x Zm3KKnyUraHmzPcVK9kOBoHJZux9CU3FgkA2xuVuJmlahspaTPJn54hdpjfRMBe9nBredwdmdx6 4FFobrRnAhX3YDgzG99gnDsHFyFnJJGUdDfAbINzhWSNg0Zc2GcLd3V3XE73TJUGclLAm/2rfvB Ow3vzejKcYOjjsHc8/M5dxlgJg== X-Google-Smtp-Source: AGHT+IFszoZ8SBA/9FJsuY6Uu27qPFJKW5beQGmqgaj/FjRsBLL1NiJEufHRTdvsslQRkUBHSfB1ow== X-Received: by 2002:a05:651c:198d:b0:308:f01f:182c with SMTP id 38308e7fff4ca-309036b453emr11165271fa.12.1739359389988; Wed, 12 Feb 2025 03:23:09 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:09 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:22:58 +0100 Subject: [PATCH v4 04/31] ARM: process: Rewrite ret_from_fork i C MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-4-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_112314_979822_AA27A4A8 X-CRM114-Status: GOOD ( 19.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move the direct assembly call to schedule_tail() and the call starting a kernel thread into a C version of ret_from_fork() in process.c and rename the old ret_from_fork to ret_from_fork_asm following the pattern in x86. Leave a comment on what we will do when we get to switching to generic entry with this as a base. Signed-off-by: Linus Walleij --- arch/arm/include/asm/switch_to.h | 4 ++++ arch/arm/kernel/entry-common.S | 22 +++++++++++++++------- arch/arm/kernel/process.c | 24 ++++++++++++++++++++++-- 3 files changed, 41 insertions(+), 9 deletions(-) diff --git a/arch/arm/include/asm/switch_to.h b/arch/arm/include/asm/switch_to.h index 9372348516ce9b80fa713966943d0bc622e86066..18d084aff5a4faa7cc5e40d48fbc4aa97a89b0c6 100644 --- a/arch/arm/include/asm/switch_to.h +++ b/arch/arm/include/asm/switch_to.h @@ -17,6 +17,10 @@ #define __complete_pending_tlbi() #endif +asmlinkage void ret_from_fork_asm(void); +__visible void ret_from_fork(struct task_struct *prev, struct pt_regs *regs, + int (*fn)(void *), void *fn_arg); + /* * switch_to(prev, next) should switch from task `prev' to `next' * `prev' will never be the same as `next'. schedule() itself diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 2f93c2f6d5b8fdf6aaf3dda0192bde144fcf238d..df6961a1006b7a0f3522728812bf52ccb849e511 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -5,6 +5,7 @@ * Copyright (C) 2000 Russell King */ +#include #include #include #include @@ -128,15 +129,22 @@ ENDPROC(ret_to_user) /* * This is how we return from a fork. */ -ENTRY(ret_from_fork) - bl schedule_tail - cmp r5, #0 - movne r0, r4 - badrne lr, 1f - retne r5 +SYM_TYPED_FUNC_START(ret_from_fork_asm) + /* + * ret_from_fork() expects: + * r0: prev + * r1: regs + * r2: fn() for kernel process or NULL + * r3: fn_arg for kernel process or N/A + */ + mov r1, sp + mov r2, r5 + mov r3, r4 + bl ret_from_fork + mov r0, #0 1: get_thread_info tsk b ret_to_user -ENDPROC(ret_from_fork) +SYM_FUNC_END(ret_from_fork_asm) /*============================================================================= * SWI handler diff --git a/arch/arm/kernel/process.c b/arch/arm/kernel/process.c index e16ed102960cb01d625e2ccc484f238b824b03b3..81c1c3f988344185917cf4e53d0d8ee47ae912f4 100644 --- a/arch/arm/kernel/process.c +++ b/arch/arm/kernel/process.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -230,7 +231,26 @@ void flush_thread(void) thread_notify(THREAD_NOTIFY_FLUSH, thread); } -asmlinkage void ret_from_fork(void) __asm__("ret_from_fork"); +/* The assembly stub sets up the parameters and calls ret_from_fork() */ +asmlinkage void ret_from_fork_asm(void) __asm__("ret_from_fork_asm"); + +__visible void ret_from_fork(struct task_struct *prev, struct pt_regs *regs, + int (*fn)(void *), void *fn_arg) +{ + schedule_tail(prev); + + /* Is this a kernel thread? */ + if (unlikely(fn)) { + fn(fn_arg); + /* + * A kernel thread is allowed to return here after successfully + * calling kernel_execve(). Exit to userspace to complete the + * execve() syscall. + */ + } + + /* syscall_exit_to_user_mode(regs); here once we switch to generic entry */ +} int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) { @@ -263,7 +283,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) thread->cpu_context.r5 = (unsigned long)args->fn; childregs->ARM_cpsr = SVC_MODE; } - thread->cpu_context.pc = (unsigned long)ret_from_fork; + thread->cpu_context.pc = (unsigned long)ret_from_fork_asm; thread->cpu_context.sp = (unsigned long)childregs; clear_ptrace_hw_breakpoint(p); From patchwork Wed Feb 12 11:22:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971800 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 56B27C02198 for ; Wed, 12 Feb 2025 11:29:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=aCFlnePKkNndn1RG+PlEu//ug35hNzDbDVRdXX0Jf/g=; b=MdoGUQtorrh38vdV+6MFAHk0Jn /KkiQrhmlA/SGN6OJpVmTo9x7onqOxsfWvciD6aWGfQy1wjLijy37RoNjC/mIiu1JOaT0T5pe9pOv +nRcoIJO0b5XB+3STbrsFoN2Phn3POtOGmEtyyXDufgMP2KzA+If4Y6U/7mqvuUQnQQhEX8hg3LNl l3o12LSR2tv2zpNgSsuSVFe4k2Xt8iE7Tk7kTkjV04M58+klAcSPgJPfb5egk5O0sJCRBNQndawj4 J12vHYqIbK5/7QSv5XA0Qm4CC9HoEgmfp4H8e+UxGZ31++qWlBItrnFOq2Gst8j1VFZiK4ftLNquo pObSke7g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiAvv-00000007BJP-3hEZ; Wed, 12 Feb 2025 11:29:35 +0000 Received: from mail-lj1-x236.google.com ([2a00:1450:4864:20::236]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApk-000000079bp-3QsL for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:13 +0000 Received: by mail-lj1-x236.google.com with SMTP id 38308e7fff4ca-307325f2436so61492251fa.0 for ; Wed, 12 Feb 2025 03:23:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359391; x=1739964191; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=aCFlnePKkNndn1RG+PlEu//ug35hNzDbDVRdXX0Jf/g=; b=WFLD34OylWj0ia9bJYZc0tX5Osi/e2DyJaGMQVwukCrSNJ+R7uOEAn0+5d3Us61faA 70PRlGQGXfevVqEDKia7ntVfiniddhCR2yJ2FmLnKU7Qr/wNBBJkt5N/WqYbKGiIf0Ta VprgdESfcP0hmMeEmHfLi+gVvgx0Hdb/9Oea0fmREhcTY0ckxu1765GGS/IK5TFtXldc 9mBJqT/wwtirFHDjo5JUR37lSo8Pgv7HeHtZ0B4dXogCLLauNrR/SZAzbbWPPA/QGf1p I55Mp5KXQM7d6LQqGpHampu59ITBx6hF81AQUaSHLAsV+MLRA7T4ZiiBGP4egLDZfkff 3KnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359391; x=1739964191; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aCFlnePKkNndn1RG+PlEu//ug35hNzDbDVRdXX0Jf/g=; b=bqL+lTzhXOUKSD4PKxipGXMivV4yt8O1lSGNwySUREXgZAhZvwRMhZbbnyrmjQyeiq PciEtKfkykLqrwLB4GRF8P7DUdvVA+eHH/1fD6K+ywrVk7LYMTrNvmlNm7hv5ms/4t/T 6Bq6/ecCpJ9y3DepC0Qc6/LEsWHdN1L0jceulm2VnKHHeL+Evt5BJGk8MUqwaySoBJNf aOk7SAXbXFYzx/tbtwS8RGC01uzpo31ZeOfiJbD01d2I6AKwLyxlSbP3N/hfGO9vzXVm tc0GRRvBAVMEmIcPo8A7FbI2+Xl/TsUOFCZzmn1oUM35ql1iNrN3geWtIBY9amLzch9g j+CQ== X-Gm-Message-State: AOJu0YyY6TdtiyNsYEc3Yv89aGXw/fGr/K1eHWgj25hB2GMzt/1mPa7B 3Vts9lr7spfPPsX9yY+t044oG8338I30C1YCr+YD6I4N7w0mk+pSJSTRw+jRQkw= X-Gm-Gg: ASbGnctsHo0mkQqP4IbR1poNs2B8hSc72CzYaHt0mFtjMTSRPqsHcLbKZDjqBct80W0 8H6imZRVqVHbeoGY1IpR2LdORW+JTIzAZcD3xnGmxf2jUK7NFY6zqGgDU7mSJKckU/ILiBSDTH9 3ITaVze582HrScI0lekKJIxwwLKebiDe2Dynml3AxkBWRbhtw2stT9ml4pvC+v9Kd2C0oICRXVn bYnnq7c18HaepuZTu6qyvCesgfGvfMjJpnDVPx4SjiEtNBwH6NXCSF8W99l4b+etOHFdEIowGq3 Ya+h5whydF4jqSXR78mtdm4PNg== X-Google-Smtp-Source: AGHT+IEYodYzadqUx2Im4nLmPVhkj0QvmLDghRs5R4XGCmQddXyPfDz0d1vVwCk+T4sDxY8MmfEg4A== X-Received: by 2002:a2e:a588:0:b0:300:2d54:c2c8 with SMTP id 38308e7fff4ca-3090369c207mr7437881fa.8.1739359390812; Wed, 12 Feb 2025 03:23:10 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:10 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:22:59 +0100 Subject: [PATCH v4 05/31] ARM: process: Remove local restart MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-5-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032312_864219_4F05E0F6 X-CRM114-Status: GOOD ( 22.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The ARM kernel contains a quirk to handle syscall restarts inside the kernel without exiting to userspace. The generic entry cannot handle this. Rename do_signal() to arch_do_signal_or_restart() to fit with the upcoming generic entry conversion. This is essentially a revert of commit 81783786d5cf "ARM: 7473/1: deal with handlerless restarts without leaving the kernel" from 2012. Other solutions may be possible, such as checking the PC after do_work_pending and assume it is a local restart if that address is inside the kernel, or modifying the generic entry code to track local restarts like ARM does and pass that information back. Signed-off-by: Linus Walleij --- arch/arm/include/asm/signal.h | 3 +-- arch/arm/kernel/entry-common.S | 9 +-------- arch/arm/kernel/signal.c | 28 +++++++++------------------- 3 files changed, 11 insertions(+), 29 deletions(-) diff --git a/arch/arm/include/asm/signal.h b/arch/arm/include/asm/signal.h index 8b84092d151800cb4076672d1c31f0a83777d0d8..7acccc96840c8a17744cc3f2894f19d142aba4fd 100644 --- a/arch/arm/include/asm/signal.h +++ b/arch/arm/include/asm/signal.h @@ -24,7 +24,6 @@ typedef struct { #include void do_rseq_syscall(struct pt_regs *regs); -int do_work_pending(struct pt_regs *regs, unsigned int thread_flags, - int syscall); +void do_work_pending(struct pt_regs *regs, unsigned int thread_flags); #endif diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index df6961a1006b7a0f3522728812bf52ccb849e511..da5c2d4b62e56cce644b15e329ab4149d69c5b79 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -86,14 +86,8 @@ ENDPROC(ret_fast_syscall) bne __sys_trace_return_nosave slow_work_pending: mov r0, sp @ 'regs' - mov r2, why @ 'syscall' bl do_work_pending - cmp r0, #0 - beq no_work_pending - movlt scno, #(__NR_restart_syscall - __NR_SYSCALL_BASE) - str scno, [tsk, #TI_ABI_SYSCALL] @ make sure tracers see update - ldmia sp, {r0 - r6} @ have to reload r0 - r6 - b local_restart @ ... and off we go + b no_work_pending ENDPROC(ret_fast_syscall) /* @@ -266,7 +260,6 @@ ENTRY(vector_swi) */ TRACE( ldmia sp, {r0 - r3} ) -local_restart: ldr r10, [tsk, #TI_FLAGS] @ check for syscall tracing stmdb sp!, {r4, r5} @ push fifth and sixth args diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index 79a6730fa0eb7d6581343bdbbeb6834dcaabb9f8..7b1a16e86b236575efdc29cda9b751e8e2a3f64e 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include "signal.h" @@ -534,9 +535,10 @@ static void handle_signal(struct ksignal *ksig, struct pt_regs *regs) * the kernel can handle, and then we build all the user-level signal handling * stack-frames in one go after that. */ -static int do_signal(struct pt_regs *regs, int syscall) +static void arch_do_signal_or_restart(struct pt_regs *regs) { unsigned int retval = 0, continue_addr = 0, restart_addr = 0; + bool syscall = (syscall_get_nr(current, regs) != -1); struct ksignal ksig; int restart = 0; @@ -590,16 +592,14 @@ static int do_signal(struct pt_regs *regs, int syscall) } else { /* no handler */ restore_saved_sigmask(); - if (unlikely(restart) && regs->ARM_pc == restart_addr) { + if (unlikely(restart) && regs->ARM_pc == restart_addr) regs->ARM_pc = continue_addr; - return restart; - } } - return 0; + return; } -asmlinkage int -do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall) +asmlinkage void +do_work_pending(struct pt_regs *regs, unsigned int thread_flags) { /* * The assembly code enters us with IRQs off, but it hasn't @@ -612,19 +612,10 @@ do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall) schedule(); } else { if (unlikely(!user_mode(regs))) - return 0; + return; local_irq_enable(); if (thread_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL)) { - int restart = do_signal(regs, syscall); - if (unlikely(restart)) { - /* - * Restart without handlers. - * Deal with it without leaving - * the kernel space. - */ - return restart; - } - syscall = 0; + arch_do_signal_or_restart(regs); } else if (thread_flags & _TIF_UPROBE) { uprobe_notify_resume(regs); } else { @@ -634,7 +625,6 @@ do_work_pending(struct pt_regs *regs, unsigned int thread_flags, int syscall) local_irq_disable(); thread_flags = read_thread_flags(); } while (thread_flags & _TIF_WORK_MASK); - return 0; } struct page *get_signal_page(void) From patchwork Wed Feb 12 11:23:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971802 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A6E80C02198 for ; Wed, 12 Feb 2025 11:32:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=H2oUY/WKIEq8Evp3ZyFIu+cgZC04yoqr/o6eiYlPpGs=; b=My9lIvfvJqT77szVCmMENBTrR6 OuTbVTHQKwTWVceRzHgMTovA+yvr+USGHrvLve+u3Vj0eBtlEAb6UmvUQqHypPTrdY2W+sjaQgYGv WBGvOT/EB7ADzI78hgI32xEz4hrdoF1gmsUqedZO+mHgtzqbx+TA8LNBTrrbYPFeu3ehham0lmii6 JWlzsWMDR85I0G9511swrNjg1mxQuw4FUVQFMdKBVBkjvbFHUnIcJKzzRzkWfEO3HF7XgpTxeD/DB exEc6OsrakGxWmueZ4J9QtOPz/80QCqJR7iQskTRVl3qQIk/H4JhKrupiWK0JN+YiA0MMfRnRsh+a UxXp+2Gw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiAyk-00000007By7-1yi3; Wed, 12 Feb 2025 11:32:30 +0000 Received: from mail-lj1-x22b.google.com ([2a00:1450:4864:20::22b]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApm-000000079cP-0pRp for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:16 +0000 Received: by mail-lj1-x22b.google.com with SMTP id 38308e7fff4ca-309025ec288so10966541fa.1 for ; Wed, 12 Feb 2025 03:23:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359392; x=1739964192; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=H2oUY/WKIEq8Evp3ZyFIu+cgZC04yoqr/o6eiYlPpGs=; b=osScllEnjIPfXvxOKKBqYsHRjTXyKeFR5RdwhOzAOzz8oVISMwhn1n6ZWe5NO+9d3Y L895ysUmPO2BeLAM3ijWiDsJxAupgFAurIRkrGdPeW6GNlMlaeiBUxWdnjp4hh7brGEe Zn+4DnQG0GTp3bvIPXdXN5P1kMr4nVg3LaAT7Ramuyj0S8PJXnkbfHg7hs9zM+tmvRs+ IWjiz/WdMbFkVDvplcn/FVLbAPFIpipzxee8grWXWWL94Z3UpoDQIX3YVwXdHJb7xo8C s6GbT37RtjwzKOq0rSrhg6xO1X24uQ2T/Fp726GAatF6aQY+t4T8xo4gjVOl4JIT3xLA 1c6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359392; x=1739964192; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=H2oUY/WKIEq8Evp3ZyFIu+cgZC04yoqr/o6eiYlPpGs=; b=xKt5/0T0hz7B6WNkA3ZrhA0hIvgi3bINhRG/agXd3usPBJJZ9KOw4fIqNTVZ4q/nuG jQL5TmJnpqPvSrU7ERRkj4KATjkZfReyBS9faMVj0AwAEbAy2wg6ygXVOhSUUma05OZM f0vn2pDpgh62GCZOYgFLlnt5AmOOzaecZEnhkhhyJUsr2c2cxlUasEZdf0cQ3Ku1Xltd ZIkD+hmNQNxyAQM/S29VdyOFZStWqEa1qwhXfb5ov0qqCCJhxoQMsZ57urqSLS6lejWG c8KSbQS7/Fevs8WbtuM5TE2vQ/K8ExbJH7J7jBSDx8Iq0Md1wUoh/1qOBf4d5keuJ9wy +/MA== X-Gm-Message-State: AOJu0YwzNyjBk3EvMEyMuVBIP1TY/6epoIVOU/ELocWcTLR2sCgYk3jl 9bxHfcMsHVmRTKrjisyvYBYEUB1nds4xgI1se43G78m4cOdBoPRVa/wVJjZcIPk= X-Gm-Gg: ASbGncv8E5+Wmg+I00uVOOMe8BY+4CuuoqdBGLaG1w90wtzFvd5JXQtR8c9w97alqOe 8ULkM1u6TdLkoZlXC/e0wnoyBSxmgXSS20yBsoixPhP8INnNYNxQfDxYEhTQkr3nRTcFye9PDiV zoaZB2ZNCKM5NBGdPUBhyhX7pFhMz/bsnOyEjS+KfekbOnChEQK3UDV9a110F96c5WrO8Gqnj8i 5qTV2P7E9r5JirLQuJaYDssLvvLx5SreCAKoOknrrkHDhnK89GiqdT1sHu6eFb6Py3ar3IJltWz oGRYLA8m2jSWcwKonO/I2cxj1Q== X-Google-Smtp-Source: AGHT+IH2Z8XUiCSIZ4pm4fNVY3XO/dOsfnhbTpqThsZgocm3jA3puP8wyJuyUu+ACjQA/BPKj/bIlw== X-Received: by 2002:a2e:be1e:0:b0:308:eabd:299b with SMTP id 38308e7fff4ca-3090379c74amr11879891fa.7.1739359391922; Wed, 12 Feb 2025 03:23:11 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:11 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:00 +0100 Subject: [PATCH v4 06/31] ARM: entry: Invoke syscalls using C MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-6-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032314_264779_E6CCB841 X-CRM114-Status: GOOD ( 28.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This makes the assembly invoking syscalls switch over to doing this from C with a small assembly stub to actually jump into the syscall. Split the syscall invocation into two paths: one for plain invocation and one for tracing (also known as "reload" as it was reloading the registers from regs). We rename this path with the infix "trace" as that code will be trace-specific as we move code over to C. Some registers such as r1 and lr get cobbled during the C calls and need to be restored when we return. Right now the part in C doesn't do much more than check the syscall number to be valid (a test previously done with a cmp r #NR_syscalls inside the invoke_syscall macro) but we will gradually factor over more assembly to C that can then be switched to the generic entry code so the exercise gets a point. Since both functions are called from a C file that is in turn only called from assembly, the functions have to be tagged with __ADDRESSABLE() in order not to upset CFI. Tested with a full system boot and by issuing some command line tools with strace to make sure the tracing path still works. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 3 ++ arch/arm/kernel/Makefile | 3 +- arch/arm/kernel/entry-common.S | 65 ++++++++++++++++++++++++++++++++++++------ arch/arm/kernel/entry-header.S | 25 ---------------- arch/arm/kernel/syscall.c | 28 ++++++++++++++++++ 5 files changed, 90 insertions(+), 34 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index fe4326d938c18efb54c96930e6b52ab7da9b2942..1d21f26ecf510ac00a878b4d51a5753ee49475f5 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -19,6 +19,9 @@ extern const unsigned long sys_call_table[]; +int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp); +int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp); + static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) { diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile index b3333d070390a8ff366a306931614b1b260647bd..69ddf51081f4791982518d37df60c11211b8955a 100644 --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -20,7 +20,8 @@ CFLAGS_REMOVE_return_address.o = -pg obj-y := elf.o entry-common.o irq.o opcodes.o \ process.o ptrace.o reboot.o io.o \ setup.o signal.o sigreturn_codes.o \ - stacktrace.o sys_arm.o time.o traps.o + stacktrace.o sys_arm.o time.o traps.o \ + syscall.o KASAN_SANITIZE_stacktrace.o := n KASAN_SANITIZE_traps.o := n diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index da5c2d4b62e56cce644b15e329ab4149d69c5b79..57aa1084a0476e9c218cd100ce4fdf4aaa748234 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -254,21 +254,23 @@ ENTRY(vector_swi) #else str scno, [tsk, #TI_ABI_SYSCALL] #endif - /* - * Reload the registers that may have been corrupted on entry to - * the syscall assembly (by tracing or context tracking.) - */ - TRACE( ldmia sp, {r0 - r3} ) - + mov r1, sp @ put regs into r1 ldr r10, [tsk, #TI_FLAGS] @ check for syscall tracing stmdb sp!, {r4, r5} @ push fifth and sixth args tst r10, #_TIF_SYSCALL_WORK @ are we tracing syscalls? bne __sys_trace - invoke_syscall tbl, scno, r10, __ret_fast_syscall + mov r0, tbl + /* r1 already contains regs */ + mov r2, scno @ syscall number from r7 + badr r3, __ret_fast_syscall + bl invoke_syscall + /* Restore regs into r1 and lr after C call */ + badr lr, __ret_fast_syscall add r1, sp, #S_OFF + 2: cmp scno, #(__ARM_NR_BASE - __NR_SYSCALL_BASE) eor r0, scno, #__NR_SYSCALL_BASE @ put OS number back bcs arm_syscall @@ -301,7 +303,16 @@ __sys_trace: add r0, sp, #S_OFF bl syscall_trace_enter mov scno, r0 - invoke_syscall tbl, scno, r10, __sys_trace_return, reload=1 + mov r2, r0 @ scno into r2 + add r1, sp, #S_R0 + S_OFF @ pointer to regs + mov r0, tbl + badr r3, __sys_trace_return + bl invoke_syscall_trace + + /* Restore regs into r1 and lr after C call */ + add r1, sp, #S_R0 + S_OFF @ pointer to regs + badr lr, __sys_trace_return + cmp scno, #-1 @ skip the syscall? bne 2b add sp, sp, #S_OFF @ restore stack @@ -415,6 +426,44 @@ sys_mmap2: b sys_mmap_pgoff ENDPROC(sys_mmap2) +/* + * This call wants: + * r0: syscall table + * r1: regs + * r2: syscall number + * r3: pointer to return function + */ +SYM_TYPED_FUNC_START(invoke_syscall_asm) +#ifdef CONFIG_CPU_SPECTRE + csdb +#endif + mov tbl, r0 + mov scno, r2 + mov lr, r3 @ return address + ldmia r1, {r0 - r3} @ reload r0-r3 + /* Arguments 5 and 6 are (hopefully) on the stack */ + ldr pc, [tbl, scno, lsl #2] @ call sys_* routine +SYM_FUNC_END(invoke_syscall_asm) + +/* + * This call wants: + * r0: syscall table + * r1: regs + * r2: syscall number + * r3: pointer to return function + */ +SYM_TYPED_FUNC_START(invoke_syscall_trace_asm) +#ifdef CONFIG_CPU_SPECTRE + csdb +#endif + mov tbl, r0 + mov scno, r2 + mov lr, r3 @ return address + ldmia r1, {r0 - r6} @ reload r0-r6 + stmia sp, {r4, r5} @ update stack arguments + ldr pc, [tbl, scno, lsl #2] @ call sys_* routine +SYM_FUNC_END(invoke_syscall_trace_asm) + #ifdef CONFIG_OABI_COMPAT /* diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 99411fa913501cad8b55ef04a2b8ab3d44f3e39e..52b4fa97226dbfa4b55aca8eaf74ae24e1e220f8 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -389,31 +389,6 @@ ALT_UP_B(.L1_\@) #endif .endm - .macro invoke_syscall, table, nr, tmp, ret, reload=0 -#ifdef CONFIG_CPU_SPECTRE - mov \tmp, \nr - cmp \tmp, #NR_syscalls @ check upper syscall limit - movcs \tmp, #0 - csdb - badr lr, \ret @ return address - .if \reload - add r1, sp, #S_R0 + S_OFF @ pointer to regs - ldmiacc r1, {r0 - r6} @ reload r0-r6 - stmiacc sp, {r4, r5} @ update stack arguments - .endif - ldrcc pc, [\table, \tmp, lsl #2] @ call sys_* routine -#else - cmp \nr, #NR_syscalls @ check upper syscall limit - badr lr, \ret @ return address - .if \reload - add r1, sp, #S_R0 + S_OFF @ pointer to regs - ldmiacc r1, {r0 - r6} @ reload r0-r6 - stmiacc sp, {r4, r5} @ update stack arguments - .endif - ldrcc pc, [\table, \nr, lsl #2] @ call sys_* routine -#endif - .endm - /* * These are the registers used in the syscall handler, and allow us to * have in theory up to 7 arguments to a function - r0 to r6. diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c new file mode 100644 index 0000000000000000000000000000000000000000..edc0ac88ec60ce3f23149a526de7dc1205906552 --- /dev/null +++ b/arch/arm/kernel/syscall.c @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include + +int invoke_syscall_asm(void *table, struct pt_regs *regs, int scno, void *retp); +__ADDRESSABLE(invoke_syscall_asm); + +__visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp) +{ + if (scno < NR_syscalls) + /* Doing this with return makes sure the stack gets pop:ed */ + return invoke_syscall_asm(table, regs, scno, retp); + + return 0; +} + +int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno, void *retp); +__ADDRESSABLE(invoke_syscall_trace_asm); + +__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp) +{ + if (scno < NR_syscalls) + /* Doing this with return makes sure the stack gets pop:ed */ + return invoke_syscall_trace_asm(table, regs, scno, retp); + + return 0; +} From patchwork Wed Feb 12 11:23:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ED300C02198 for ; Wed, 12 Feb 2025 11:34:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=yplTGeg8q5Jb1IHX+lZzUxdHnSw+Ki6MseFOXEsgMb4=; b=3O0yweDf6FvaTqiLi856UShoOv nF7wNjOUHBD33Q9JaPiyLQzoWuADG9qbJ6fliKKp+nfP3tI8k7bhQN9vSnSEpfb023py6+yGHWXiF 4j/1E5TJDkZhpx9pRjFKN7+QvUHr/634Av6WIE5i9VB9oyrg5x7EkwdxMfUyaYXxl+FcGjzh65Bkl jIxD1rKbMC1HA85mHkCH9+8Y8snQtzjugYA10EIZTL8bOwWXVFViZG3VhzyUyMv2YsenJuEqDMXDG p7CFVnc9+XAIL3gPtMFaWpqXUcuCewBGTWWbK+OE8IopEPWqd7EfU4MiqH5w/TKbZal768Eudqc5v yaEuRi/Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiB08-00000007CEc-0v99; Wed, 12 Feb 2025 11:33:56 +0000 Received: from mail-lj1-x22f.google.com ([2a00:1450:4864:20::22f]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApn-000000079d2-2R0z for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:16 +0000 Received: by mail-lj1-x22f.google.com with SMTP id 38308e7fff4ca-30613802a04so65148611fa.2 for ; Wed, 12 Feb 2025 03:23:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359394; x=1739964194; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=yplTGeg8q5Jb1IHX+lZzUxdHnSw+Ki6MseFOXEsgMb4=; b=DsLE7A+oxsTHm7FLfaFXMh96Wa31OZ013bwaUypL6rze8BpT1OOfIQtrTb9ZKqDrYX i6pbUd2JDRZNaTIVaUUJBQqezZjBb40DBMu0eHNHCe48ivvoh5gC4WRea+4IHhU1Otpw y+Dc7k0I5HO+tyiLBNqKSmtsO1PtHLEIf4jz1F2yIKFF5hxsHuX9YAZkhxeKUB7N58XO YngZ8cJFFM6ZxGTcGI9evMvbOGD05u3oc7eCb/GJIOShE3KVPNO+/pgGhz4Q/VnLmazB MUmTxg4U55qPLDqPh22xC8298QNUBLoYeKB+MiEGfSfZ3wNN1enPYke+D4c/ByHqiDF6 6S9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359394; x=1739964194; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yplTGeg8q5Jb1IHX+lZzUxdHnSw+Ki6MseFOXEsgMb4=; b=S3/OWNWVisx1EULnlFbIcIoVxRtNKtJs9vaSsYYUIh+PX9EkoDLBLd9I00HCQVAtsK ZUydO0s8xZK6BONLJr53nIwLWLWiTcenJ1dEn9XskbiFT4MtRrynxxMi/xICg0RBekGO Oxc58kOIeuEuvyuuCV0y65JMOG0sfHJ+SxBcH30Ipc6f+Yi3xLFLcbPp8NLd4ZtIl2Fb SpMC5TI7d1lNly9Fh+8wUr6eZXzVzVfYsoQPdB3rl57eUgSKuQS7XF3Xdil1dM8Pkk25 LHK13deL6Yt7Q5D1cC0DUT6aPj9VzIBbfkH9eGAc2hLDFrtGLYoc0oIv+9Js+aXjQlrd vBuA== X-Gm-Message-State: AOJu0YyEKgRL0NsL5RT3PeLTH6wYVt1gv8IvycbNNVXc1bhWQD1/ZM0r nEUEno1PZdfdAh/2w3flPbQGoAqiurjJNtgAPArGiccJgPWeebmJZamVQ/BzxP8= X-Gm-Gg: ASbGncv4lyK55XW9qLpmn1uizH6v53zqmL9TT10Vh7N7Hl1QTa1pmIcZgM6YvATWzmd 7utRseORfWkdHiARigHHmER5/nZLI7ZQZCWHm/sAMXJH3OVpOzJJ8gydN3kxF0B5osLpBumgFQq yqwW4z/RWEOek5sD175ZAkVT6neLfRg98NcxsEOUXsVwVeH1TU6iMuiktxofbz3PZFCpj08ELps iXbImLw299ZKwp3ouZ4wbSj/DmyngzEeSgPF/Rty1qKZulw6QFbie3k8kkPI1D94h4noJgcX/hL bavwfvddUFVRfDfAfFcuQElXOw== X-Google-Smtp-Source: AGHT+IEaYca91uWTU7cyiSHpPYPrfctGa1tgN1r2xzyJA7Zpo1TE5CUGZf48fi1/xqnUPpwUa4Slpw== X-Received: by 2002:a2e:2206:0:b0:308:eb34:101e with SMTP id 38308e7fff4ca-30903911e32mr7352541fa.29.1739359393267; Wed, 12 Feb 2025 03:23:13 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:12 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:01 +0100 Subject: [PATCH v4 07/31] ARM: entry: Rewrite two asm calls in C MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-7-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032315_640891_672F5457 X-CRM114-Status: GOOD ( 17.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The normal and trace entry code calls out to arm_syscall() and sys_ni_syscall() from assembly, but these calls can be moved over to the new C implementation. Signed-off-by: Linus Walleij --- arch/arm/include/asm/traps.h | 2 +- arch/arm/kernel/entry-common.S | 23 +++++------------------ arch/arm/kernel/syscall.c | 13 +++++++++++-- arch/arm/kernel/traps.c | 2 +- 4 files changed, 18 insertions(+), 22 deletions(-) diff --git a/arch/arm/include/asm/traps.h b/arch/arm/include/asm/traps.h index 2621b9fb9b19b064aa40a8ab4a3a89067b98a010..b888912c2450e0e8eec8139166c7292f283545c7 100644 --- a/arch/arm/include/asm/traps.h +++ b/arch/arm/include/asm/traps.h @@ -40,7 +40,7 @@ asmlinkage void dump_backtrace_stm(u32 *stack, u32 instruction, const char *logl asmlinkage void do_undefinstr(struct pt_regs *regs); asmlinkage void handle_fiq_as_nmi(struct pt_regs *regs); asmlinkage void bad_mode(struct pt_regs *regs, int reason); -asmlinkage int arm_syscall(int no, struct pt_regs *regs); +int arm_syscall(int no, struct pt_regs *regs); asmlinkage void baddataabort(int code, unsigned long instr, struct pt_regs *regs); asmlinkage void __div0(void); asmlinkage void handle_bad_stack(struct pt_regs *regs); diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 57aa1084a0476e9c218cd100ce4fdf4aaa748234..77801d039b5f6562b231a6fd4979e318c7e736a6 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -264,18 +264,10 @@ ENTRY(vector_swi) mov r0, tbl /* r1 already contains regs */ mov r2, scno @ syscall number from r7 - badr r3, __ret_fast_syscall - bl invoke_syscall - - /* Restore regs into r1 and lr after C call */ + /* We return here no matter what, also pass this as an argument */ badr lr, __ret_fast_syscall - add r1, sp, #S_OFF - -2: cmp scno, #(__ARM_NR_BASE - __NR_SYSCALL_BASE) - eor r0, scno, #__NR_SYSCALL_BASE @ put OS number back - bcs arm_syscall - mov why, #0 @ no longer a real syscall - b sys_ni_syscall @ not private func + mov r3, lr + b invoke_syscall #if defined(CONFIG_OABI_COMPAT) || !defined(CONFIG_AEABI) /* @@ -308,13 +300,8 @@ __sys_trace: mov r0, tbl badr r3, __sys_trace_return bl invoke_syscall_trace - - /* Restore regs into r1 and lr after C call */ - add r1, sp, #S_R0 + S_OFF @ pointer to regs - badr lr, __sys_trace_return - - cmp scno, #-1 @ skip the syscall? - bne 2b + cmp r0, #-1 + bne __sys_trace_return add sp, sp, #S_OFF @ restore stack __sys_trace_return_nosave: diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index edc0ac88ec60ce3f23149a526de7dc1205906552..d637dc74b1c70415933898fbcadbedc71df5f654 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -12,7 +12,10 @@ __visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void * /* Doing this with return makes sure the stack gets pop:ed */ return invoke_syscall_asm(table, regs, scno, retp); - return 0; + if (scno >= __ARM_NR_BASE) + return arm_syscall(scno, regs); + + return sys_ni_syscall(); } int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno, void *retp); @@ -20,9 +23,15 @@ __ADDRESSABLE(invoke_syscall_trace_asm); __visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp) { + if (scno == -1) + return -1; + if (scno < NR_syscalls) /* Doing this with return makes sure the stack gets pop:ed */ return invoke_syscall_trace_asm(table, regs, scno, retp); - return 0; + if (scno >= __ARM_NR_BASE) + return arm_syscall(scno, regs); + + return sys_ni_syscall(); } diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index 6ea645939573fb65ed36f5435a9c94e98828f45b..942c77767919466d5b89d83d8eb77032b119473e 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -609,7 +609,7 @@ do_cache_op(unsigned long start, unsigned long end, int flags) * 0x9f0000 - 0x9fffff are some more esoteric system calls */ #define NR(x) ((__ARM_NR_##x) - __ARM_NR_BASE) -asmlinkage int arm_syscall(int no, struct pt_regs *regs) +int arm_syscall(int no, struct pt_regs *regs) { if ((no >> 16) != (__ARM_NR_BASE>> 16)) return bad_syscall(no, regs); From patchwork Wed Feb 12 11:23:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 95876C02198 for ; Wed, 12 Feb 2025 11:36:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=gCv9qyhC7Wn0x41/PmaNXNh/X9H9ffqS+9z2MGX0UwA=; b=e5qH0sgYHuDYpdRvce6NE6sZ+j loQ1ZuXpgqah0YfixhHvh1rRhCYc/mchxqA0EIh49vZJPVLJQ+EqZt4UfilakYrfcOY6zU7yASjn2 CeH9d+3O/fMUXGyF2+sYX/f/OP/XT5gRD8ubyAjlcOGMRkkRV4Oxq+bafh7vMwq5qRmcnS0AN6clb Xp/F5suLRMU3vlnkRG0wccoOn/kiT9uNBKteUF8yKfFh3sh8UIYjrIEpHNXhstwbkjr/hETIrSdHQ +Mszuk9Hi2i36Enn3NgtPj29JdDCfv+JvQx67BkOmvJBCUz2kpeOZOA5YDQLxH28dkMrsAppOBDhF siDdyQzA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiB2v-00000007Czt-2kgn; Wed, 12 Feb 2025 11:36:49 +0000 Received: from mail-lj1-f177.google.com ([209.85.208.177]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApo-000000079dE-3NJb for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:17 +0000 Received: by mail-lj1-f177.google.com with SMTP id 38308e7fff4ca-30737db1ab1so61503211fa.1 for ; Wed, 12 Feb 2025 03:23:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359394; x=1739964194; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=gCv9qyhC7Wn0x41/PmaNXNh/X9H9ffqS+9z2MGX0UwA=; b=se0Q7+u4cVQiBRIPHrb5UQ7EfvU7IMz0eLYvWAHJCxqzPtsXUwllvzOMloJimeXT5A i71oGDUet26z/YWDkvJkgd5WWTuz8wHovDPrc4Vrj//6UdefUh5TeNMXxG8Mztcfr1FD pSvpp4n2M1rdNSOLH7+MFUmJkLRRIZpIAthI3TrOmuloNhrzO5bkUFyHvW8I2eL/Vd32 Iz3GEAVePIlaDG9Hp2J4uInKIE8Y7Dn7sd2+QoFBghPxnnFsyCVVWyZz2HsTc/RGiXL8 8VHbRhhyF6XtmdlhBrKEuyOG+khSgyE4kYvsxVVLbLMSfI+N3n6lC4etd4WV8OzD6xwd yf7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359394; x=1739964194; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gCv9qyhC7Wn0x41/PmaNXNh/X9H9ffqS+9z2MGX0UwA=; b=DLnickrrHEt13r0Ei6Z1b0Xiyb/yAXJNqMmclZPjCELoHfFM7ay9xR2Q0pyAagBlYo WtfzKhaYTVxQWny5HD7QxV1NwjMJSaWml2zvPYcXrWGduwOo/m+US8PFBgFQNhyIDhxU +nxULRaSoMgiC2CDon/TdEQ4iPZDbNnvG2TM0PZKJF9827dFQ+belCMo/Ip7LS3TXp50 84Pg9T2NQHTPi5vaNWVa5dLg8kXNu1kjQeRBKtQjG6hLYHJd2cRrM7hkNf7RpGyGUYXn 1RdbsaJAtK/TLGUUgpfPrzd5frwgWBJAC1tvFSDa/ti8UIPuZu4f5+DylDY337/tpOgP rsIA== X-Gm-Message-State: AOJu0YyXcR+C+9ZUprhjiQ1wnZ0kEVdvzU2clqlUy/pb8TGHVTyQfaXf TEPBh4KG1WsG+2Tt3cEcNFQRuGkXnPt5teKxz39ULVeRHefVpJaJGE9GUqS1xlkY5NUhzMOQeYH z X-Gm-Gg: ASbGncve2/02cnmaxsOrObkkxmKfwiVV2OnSuAproo19WXGYtTDmzkEDmS3PmrblAnH sTp1R8JXKbYuYhjKNC5/tmw2oQLAJxW9DmaXhFlbdR/LXYoXBYcdqfCPloO0dQ3fX2jm2GITjhK 95Qf/KRwE8ZSzDWsNZFZtdOnAxzBI5IoHPuJTKftId7eii7d6Lw8zvspR5HwDMEnNXZceyrYnBN 6YUv72Y4aPSD44KW/fJoen6Z3rUbZThEZSrK2lpRvKt7EKzymmWtNtMdZA29iabAUDiaL54K37T wxXqK96MYQvsNPC/2FxIceDkSg== X-Google-Smtp-Source: AGHT+IFfP7os8MQ1u1YZSNkqYKfZSgb/vW3ksTMdEyemTkz16qCj36YkFgGny3jWjjj/tutrgWjYQw== X-Received: by 2002:a05:651c:19aa:b0:302:3534:1d68 with SMTP id 38308e7fff4ca-309037abe6fmr8480831fa.4.1739359394471; Wed, 12 Feb 2025 03:23:14 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:13 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:02 +0100 Subject: [PATCH v4 08/31] ARM: entry: Move trace entry to C function MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-8-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032316_852210_1748B6F7 X-CRM114-Status: GOOD ( 15.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This moves over the code entering into tracing into the C syscall invocation code. As syscall_trace_enter() will provide the current syscall number from the regs we need not provide the syscall number to the invocation function any more. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 2 +- arch/arm/kernel/entry-common.S | 6 +----- arch/arm/kernel/syscall.c | 5 ++++- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index 1d21f26ecf510ac00a878b4d51a5753ee49475f5..66067d165ba3fbbe0f840a89ae396eb1a311bacc 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -20,7 +20,7 @@ extern const unsigned long sys_call_table[]; int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp); -int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp); +int invoke_syscall_trace(void *table, struct pt_regs *regs, void *retp); static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 77801d039b5f6562b231a6fd4979e318c7e736a6..3cfc6d952ff99be9c4c1be4481ac3039260e3e57 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -292,13 +292,9 @@ ENDPROC(vector_swi) * context switches, and waiting for our parent to respond. */ __sys_trace: - add r0, sp, #S_OFF - bl syscall_trace_enter - mov scno, r0 - mov r2, r0 @ scno into r2 add r1, sp, #S_R0 + S_OFF @ pointer to regs mov r0, tbl - badr r3, __sys_trace_return + badr r2, __sys_trace_return bl invoke_syscall_trace cmp r0, #-1 bne __sys_trace_return diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index d637dc74b1c70415933898fbcadbedc71df5f654..377e16c8c53c8e809e3f8b157a889ef115d5c043 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -21,8 +21,11 @@ __visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void * int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno, void *retp); __ADDRESSABLE(invoke_syscall_trace_asm); -__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno, void *retp) +__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, void *retp) { + int scno; + + scno = syscall_trace_enter(regs); if (scno == -1) return -1; From patchwork Wed Feb 12 11:23:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971806 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DF0F3C02198 for ; Wed, 12 Feb 2025 11:38:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=aUy2L+59XSB2dO0W6jnqgQpUaRtZ0LvR6wu2Q9qABCw=; b=rM2Owigs0uqH8xTs8NWqP3yFNq rDd7zmAKzso/lok426zfhb2DAw8LOmKU3UpShbnQjNAwx4pLz+RnVAD0TdpqnDfTjYW/Egwk7VkyQ crmIk0HouryFpiUvYPkg3lVXn8oyNk7YWc0gylNTT1AJDMdwnvjvDEdYQ+fRIj/uD1uWOmX8v0Fp4 UV59BSvVz/oiBB1FO1oAwEE8WhqIsTjZPAglbAoFFzT+jGqig1AFN6A0BPePPyr9IOGmuYD11qmof 3nTHDsTV7eppI67CRuqUW8jKwrnTqsBc7LXjLfZY3nDZ7jsm0/o2mf1Z1/FGbA2jAcJgqUyxMKQ0M f9WYrx0Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiB4K-00000007DBb-23L1; Wed, 12 Feb 2025 11:38:16 +0000 Received: from mail-lj1-f175.google.com ([209.85.208.175]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApp-000000079dn-30d2 for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:18 +0000 Received: by mail-lj1-f175.google.com with SMTP id 38308e7fff4ca-308edbc368cso32848351fa.1 for ; Wed, 12 Feb 2025 03:23:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359395; x=1739964195; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=aUy2L+59XSB2dO0W6jnqgQpUaRtZ0LvR6wu2Q9qABCw=; b=PfLleF3FJZhAg4fNGZykvQMXqrapXK7/oGYhltGHJ20nkr4VuN2gW0WOGItoBWivMU ZApB9rTmnIWwkODIKBFAztInmqygz9uBbbSlrX/4I5r4fFLlcAw8VWItPcaZ/fjcChz4 e60uQg1M9RwiQ9EaURPkSF4qE2DwL2NHW+1LY9etMLZtsR9Lz5opltDld7OHWOnh5A9a fHiAZkVjcxuSCVz2vFRTvgkO+XWne1Q/wvQapCLSQinsievMypz+WCyXUHnYKj3EfC3X HsmFEy0y3QvhUYvsr9Eza0fw41uN01f+a0LSb1DZA1lRYJHRqxmIeFfiXklsT4du0y4U h80w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359395; x=1739964195; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aUy2L+59XSB2dO0W6jnqgQpUaRtZ0LvR6wu2Q9qABCw=; b=mFRs6NcCl6/46TieOw40ViXMYGehJVMbVyDLKw/CUgULWtCKQGQdK5yVY630TOlRdS 9kDDHmDo0rDk8rYXY/te9j5/LJ4MI9RsrDL3UMG+Xf1eEZPmrp+e7l+krAa+7BSV1VOz gcPp2exGR4EvuZBW25wMXU7ZJJTD9a+GW0vKv6y3p3FAOBhTf4G+hpQTmNWZvRymKp9m lH7rWbFTiMb8b3b2ttm0govOY6t2RI36PtzeQWYS6uRkw/p9pGLeGo2BfZjK//wl4QJX i1kQY9fq+3qUXHXBHPL3uF5WqUEt9JXP7AvuCX600CJZKrYJJWeb66tMxYKcdNmEsygj Bd1A== X-Gm-Message-State: AOJu0YyPerKAnaugQhJA/TSoub3hKuCd9JBVlFtcftwRNK4dbIsQFE11 5dF2Z1plRp/JaoUkigSz2xS08nVSTp6Hk8RPmNhO+EbgnyU+a1CCOfeY5b/P3ic= X-Gm-Gg: ASbGncseHYhWRFaBlkgfIhGVRd5z5gp3stxja5Ef6Td8Gk4zJT2Rp9curxMHTEOM4e9 m9S/Pvdaa9JMucMgVrobPemQxBtISipSFjMxqb0PA8ZLd1z/oVMPvihbWTGiSQKEkbNwBBHkFZ0 meonWTYsU8Dd4KORV/JZAw86MCN1wbqVtIoFeQmIc46vEFRVyS4CpcKurtfFuc1Y8YgAK1GWVQM IjAxvOIKiuOZF/OJH52gnv4w1s/auRkrNQbYcujl8Z9mlZ3Ck931QPKa4ADAnGz9GPSqAASnzWj UVpZKmS0+dWZn7XufaLJatrGDA== X-Google-Smtp-Source: AGHT+IHZJ/tBIMRwnLsYcu+g9QAXHEyBK/NodAzDXyRtpuVQmFBcjHxeJrEL5OicySIl+6hJ7MT1qQ== X-Received: by 2002:a05:651c:895:b0:308:efa4:d27f with SMTP id 38308e7fff4ca-30904fc98f2mr9827531fa.2.1739359395534; Wed, 12 Feb 2025 03:23:15 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:15 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:03 +0100 Subject: [PATCH v4 09/31] ARM: entry: save the syscall sp in thread_info MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-9-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032317_811704_3791056D X-CRM114-Status: GOOD ( 18.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We are going to rewrite the syscall handling in C, which means that the stack used by the call code is no longer predicatably 8 bytes (for syscall arguments r4 and r5) but a varying number of bytes depending on how nested the C code is. However the current code is just assuming it can rewind the stack by adding 8 to sp if a syscall is interrupted by a sigreturn call. Solve this by storing the entry sp in the per-task struct thread_info and use that in the sigreturn wrapper instead. We already have the thread info available in the SWI entry and sigreturn is probably not so common that retrieveing a pointer to thread_info should affect anything very much. Storing this per-task in thread_info makes the solution SMP robust. Signed-off-by: Linus Walleij --- arch/arm/include/asm/thread_info.h | 1 + arch/arm/kernel/asm-offsets.c | 1 + arch/arm/kernel/entry-common.S | 8 ++++++-- 3 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/arm/include/asm/thread_info.h b/arch/arm/include/asm/thread_info.h index 943ffcf069d29cf4a035964d20d56f7ebdd6d602..d8a45c5a10496aaf806bfeaa0353d5e8985bd6f5 100644 --- a/arch/arm/include/asm/thread_info.h +++ b/arch/arm/include/asm/thread_info.h @@ -67,6 +67,7 @@ struct thread_info { __u32 cpu_domain; /* cpu domain */ struct cpu_context_save cpu_context; /* cpu context */ __u32 abi_syscall; /* ABI type and syscall nr */ + __u32 sp_syscall; /* SP when entering syscall */ unsigned long tp_value[2]; /* TLS registers */ union fp_state fpstate __attribute__((aligned(8))); union vfp_state vfpstate; diff --git a/arch/arm/kernel/asm-offsets.c b/arch/arm/kernel/asm-offsets.c index 4853875740d0fe61c6bbc32ddd9a16fa8d1fb530..c9525cbb26b73827821aa746030e56b037f49556 100644 --- a/arch/arm/kernel/asm-offsets.c +++ b/arch/arm/kernel/asm-offsets.c @@ -49,6 +49,7 @@ int main(void) DEFINE(TI_CPU_DOMAIN, offsetof(struct thread_info, cpu_domain)); DEFINE(TI_CPU_SAVE, offsetof(struct thread_info, cpu_context)); DEFINE(TI_ABI_SYSCALL, offsetof(struct thread_info, abi_syscall)); + DEFINE(TI_SP_SYSCALL, offsetof(struct thread_info, sp_syscall)); DEFINE(TI_TP_VALUE, offsetof(struct thread_info, tp_value)); DEFINE(TI_FPSTATE, offsetof(struct thread_info, fpstate)); #ifdef CONFIG_VFP diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 3cfc6d952ff99be9c4c1be4481ac3039260e3e57..8baab7f97f59c434396f30b08ddd3029c5f9c0e5 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -232,6 +232,8 @@ ENTRY(vector_swi) uaccess_disable tbl get_thread_info tsk + /* Save a per-task copy of SP for sigreturn */ + str sp, [tsk, #TI_SP_SYSCALL] adr tbl, sys_call_table @ load syscall table pointer @@ -377,13 +379,15 @@ sys_syscall: ENDPROC(sys_syscall) sys_sigreturn_wrapper: - add r0, sp, #S_OFF + get_thread_info tsk + ldr r0, [tsk, #TI_SP_SYSCALL] @ read back SP mov why, #0 @ prevent syscall restart handling b sys_sigreturn ENDPROC(sys_sigreturn_wrapper) sys_rt_sigreturn_wrapper: - add r0, sp, #S_OFF + get_thread_info tsk + ldr r0, [tsk, #TI_SP_SYSCALL] @ read back SP mov why, #0 @ prevent syscall restart handling b sys_rt_sigreturn ENDPROC(sys_rt_sigreturn_wrapper) From patchwork Wed Feb 12 11:23:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971808 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BF133C021A0 for ; Wed, 12 Feb 2025 11:41:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=QZ+1V9qTPsDzOzqvLQMw/vPgzhrwZ5TSLeEMjw/aHNM=; b=rbwjcCMNvJyMBbg4bMIkOykps8 6A6DAsH8/5oDYYNJbguln2Tu3mbQATMbmG0Jm3qWfd/r7UCvJLMYca/Kwlzt9J8VGaxLT0OV03exj l4QEB1GuX4zrZ0ntpbDOhG8ds7GUHGQSfBl0y2Ce/A4u0Ng10Y5+hYrGVUOrShjpL/VGKT3qkOXTh W3uHYK0q6S5jxbatNXXD6O9W9EmTba1bOabDeK9yxhR6G/fLYAjhFwltCqsymNutwtXXqaD3r3zGj cK54XvqlsLb+rnILz0H8df8ib1sN0qR1AhSiWS/Cp4kTYNWh0mpZhE2fq6er2Dfid0sqvHYIwOill Jrn8pHIA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiB76-00000007DcT-1HS3; Wed, 12 Feb 2025 11:41:08 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApt-000000079fN-0owH for linux-arm-kernel@bombadil.infradead.org; Wed, 12 Feb 2025 11:23:21 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Sender:Reply-To:Content-ID:Content-Description; bh=QZ+1V9qTPsDzOzqvLQMw/vPgzhrwZ5TSLeEMjw/aHNM=; b=H0ZLai/+qLxjS5HVrPBxlkUS0E hkfvjr6psznY/ksSpVuTFhCcd6B/w6+E6hUp7xhnAnWZILAFlyC1m2CgNh4oO5Vs2twRuHUv3B1gP uwPEF3olaz5PROVbwU2+S0L1ZbWmLIqaw3+yMw6QVsQSrKLOaGK9oD1JCzSUUvLuOcrhEMkw4SRK+ HmOP4csQoBGSFPiTxb4BLZWgZpO1b9B4wdHVRQbAun0Y86s3SoNUq3oMFyQUD0l8HMpBKnXF+L+NJ EPcRKrQnqGEX49IRN/TxVLpYiUY0Hi0CEWRxGZjO4FAwnX228N7ufBfQq6B5d06atXCUckVpDUfM8 gWY5QfUg==; Received: from mail-lj1-x22c.google.com ([2a00:1450:4864:20::22c]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApq-00000000nnT-1DHb for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:20 +0000 Received: by mail-lj1-x22c.google.com with SMTP id 38308e7fff4ca-30761be8fa7so60301071fa.2 for ; Wed, 12 Feb 2025 03:23:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359396; x=1739964196; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=QZ+1V9qTPsDzOzqvLQMw/vPgzhrwZ5TSLeEMjw/aHNM=; b=Z/6iJ5T9IIUIKQt7G1ydUIJa9LK/tUd9jUIoQzW1zc/90UI6LJkh5rlI4gtBvWWMxU 3rjneteLjaIHRdLaiI2SnJ5W8cEmsWwS4RexfhOOT/W2xbVg9+TaVY4aZ3wo6eWfsTnm Y49n7j2H0q9go+qVD4iRMmgYvD3U9RjDDScMr22ZiIDGnLboll1k9VwQCRXcyRJU1/N1 tVxLDqCTVOE/RXpOOnBwUinciEM1au0gCwE4dhokzzQlm5cA1V24cyKNFrBGBWsdYVVM gWWoNTgyOu1Ncts5Y4Ng2X/1iri5ZDbY07X5hM2bj8CPEI7JxOtJzjzv6HreYbU3WOqD XgJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359396; x=1739964196; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QZ+1V9qTPsDzOzqvLQMw/vPgzhrwZ5TSLeEMjw/aHNM=; b=Hu3leoyiIcYtwlLOkffRWrnEjCSlFBAlM+JF4l4zAVH/07ACht9WF+feBbvdbONoD3 CUZl0umsZgJbitlhBgv8yUyZWImtYGVfx0QgCfMaVDzFTNvVjt8wjKzzIWs0ZqTKtpUP cP7q+MlE2aqSW6RKK5sDb53kFV28KMQwan3EM1IgRPLUpvD5K9dXaiks8Vhyo3aKtf/D /D0E5YDtsmKdUJAOZpYUAy7tX7sfVrb/XCcKoo+HPJ25J4+DprEzebuvtLufc/uOxr1v rvK58jlbTlIjWMuufyGH7YXbYJbKRSe/WlgDaM6MdfwGAXZfMGzX32jrOhWfjiBo9vVm 2GvA== X-Gm-Message-State: AOJu0YzTE68HoGCyqG2hq4OkizkOTwPIdOBjsminFbudwELShtVyTFnq J4SYXac8nHw63hH0wAha2lo582+HJ075iRDGI5C8lITK3d22StyPJHVNjFvvQME= X-Gm-Gg: ASbGnct4f9W0sZHVuDkWWZ9u9VEJ+GLNxG0LNRWhqTeKbZa35eu3MQMCHHWH2h+KQE5 oMbDuWY5RYBCo+ErFlPJ1jRtBQCgX9B04cZsrTwF80rE5wG6eE9chHJClmAgLfkOQ78eOQsYIR5 xAqkF+lE2VinTc7mdLSP6jDb7KJ/qEynmUKcheZCGq2Eg7ouCuZNFa3YJ3c1O9X7+CrHMQuAS9t RQhi9sPU+pjflmoD7CIANyv+FPUYTvxyRmlkYyB8F7BR5cdeZZtpTCuSUqd1hTroN0gMzKN04EC I66Cc+i+ybmgo0oeN85DwC+4vA== X-Google-Smtp-Source: AGHT+IEy2zhtjzr1xUwkwwU7OEnGwL574LYD2AIUrLmh4kmR8+NHxnHLbPnyZ0TB2gEDQ1t6v/qifw== X-Received: by 2002:a2e:be22:0:b0:302:3003:97e with SMTP id 38308e7fff4ca-309036dbf25mr11761381fa.30.1739359396492; Wed, 12 Feb 2025 03:23:16 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:16 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:04 +0100 Subject: [PATCH v4 10/31] ARM: entry: move all tracing invocation to C MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-10-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_112318_561609_4A17FEF6 X-CRM114-Status: GOOD ( 21.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The traced invocation of syscall is rewritten in C, moving over also the call to syscall_trace_exit() to C. To do this we cannot have invoke_syscall_trace_asm() be a leaf call, which is the mechanism we hithereto relied on to make sure the stack is pulled back to the state we were at before we called out to the code written in C. Push the registers potentially used by C on the stack, and make a copy of the two stack-pushed syscall arguments on the top of the stack before invoking the syscall and then drop these copied arguments and pop back the registers before returning from invoke_syscall_trace_asm(). As we are now calling from and returning to C, we no longer need to pass the regs pointer around so drop it. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 2 +- arch/arm/kernel/entry-common.S | 32 +++++++++++++++----------------- arch/arm/kernel/ptrace.c | 2 +- arch/arm/kernel/syscall.c | 30 +++++++++++++++++++++--------- 4 files changed, 38 insertions(+), 28 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index 66067d165ba3fbbe0f840a89ae396eb1a311bacc..cb0073c4151bf70a82d15e17a95b6b6f48b245d6 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -20,7 +20,7 @@ extern const unsigned long sys_call_table[]; int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp); -int invoke_syscall_trace(void *table, struct pt_regs *regs, void *retp); +void invoke_syscall_trace(void *table, struct pt_regs *regs); static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 8baab7f97f59c434396f30b08ddd3029c5f9c0e5..dbc947d301ec11bb13007cb4bb161c035ede5c10 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -296,22 +296,11 @@ ENDPROC(vector_swi) __sys_trace: add r1, sp, #S_R0 + S_OFF @ pointer to regs mov r0, tbl - badr r2, __sys_trace_return bl invoke_syscall_trace - cmp r0, #-1 - bne __sys_trace_return - add sp, sp, #S_OFF @ restore stack - -__sys_trace_return_nosave: - enable_irq_notrace - mov r0, sp - bl syscall_trace_exit + add sp, sp, #S_OFF @ restore stack pointer b ret_to_user -__sys_trace_return: - str r0, [sp, #S_R0 + S_OFF]! @ save returned r0 - mov r0, sp - bl syscall_trace_exit +__sys_trace_return_nosave: b ret_to_user .macro syscall_table_start, sym @@ -437,18 +426,27 @@ SYM_FUNC_END(invoke_syscall_asm) * r0: syscall table * r1: regs * r2: syscall number - * r3: pointer to return function */ SYM_TYPED_FUNC_START(invoke_syscall_trace_asm) #ifdef CONFIG_CPU_SPECTRE csdb #endif + /* Save registers because we are being called from C */ + push {r4 - r10, lr} mov tbl, r0 + /* Make space to copy the two syscall stack arguments */ + sub sp, sp, #S_OFF mov scno, r2 - mov lr, r3 @ return address - ldmia r1, {r0 - r6} @ reload r0-r6 - stmia sp, {r4, r5} @ update stack arguments + badr lr, __invoke_syscall_trace_ret @ return right here + ldmia r1, {r0 - r6} @ reload r0-r6 from regs + stmia sp, {r4, r5} @ copy stack arguments ldr pc, [tbl, scno, lsl #2] @ call sys_* routine +__invoke_syscall_trace_ret: + /* Drop the copied stack arguments */ + add sp, sp, #S_OFF + pop {r4 - r10, lr} + ARM( mov pc, lr ) + THUMB( bx lr ) SYM_FUNC_END(invoke_syscall_trace_asm) #ifdef CONFIG_OABI_COMPAT diff --git a/arch/arm/kernel/ptrace.c b/arch/arm/kernel/ptrace.c index 07b0daf47441f1f76a8af416acc74fa5ed770403..ac7b98ae47249b4a00fc1cb871bb2d309bfd8c88 100644 --- a/arch/arm/kernel/ptrace.c +++ b/arch/arm/kernel/ptrace.c @@ -888,7 +888,7 @@ asmlinkage int syscall_trace_enter(struct pt_regs *regs) return scno; } -asmlinkage void syscall_trace_exit(struct pt_regs *regs) +void syscall_trace_exit(struct pt_regs *regs) { /* * Audit the syscall before anything else, as a debugger may diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index 377e16c8c53c8e809e3f8b157a889ef115d5c043..e90f04f35b3485968a1e9046cf0889f56cca92e5 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -18,23 +18,35 @@ __visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void * return sys_ni_syscall(); } -int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno, void *retp); +int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno); __ADDRESSABLE(invoke_syscall_trace_asm); -__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, void *retp) +__visible void invoke_syscall_trace(void *table, struct pt_regs *regs) { int scno; + int ret; scno = syscall_trace_enter(regs); if (scno == -1) - return -1; + goto trace_exit_nosave; - if (scno < NR_syscalls) - /* Doing this with return makes sure the stack gets pop:ed */ - return invoke_syscall_trace_asm(table, regs, scno, retp); + if (scno < NR_syscalls) { + ret = invoke_syscall_trace_asm(table, regs, scno); + goto trace_exit_save; + } - if (scno >= __ARM_NR_BASE) - return arm_syscall(scno, regs); + if (scno >= __ARM_NR_BASE) { + ret = arm_syscall(scno, regs); + goto trace_exit_save; + } - return sys_ni_syscall(); + ret = sys_ni_syscall(); + +trace_exit_save: + /* Save return value from syscall */ + regs->ARM_r0 = ret; + +trace_exit_nosave: + local_irq_enable(); + syscall_trace_exit(regs); } From patchwork Wed Feb 12 11:23:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971807 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7106CC021A1 for ; Wed, 12 Feb 2025 11:39:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=fjCfgKpzfePVa0TyVy/JvnFpB7AUCXqKC35hzfV3hLA=; b=KipDdDBW+/oZ4Md0tUjgSPc4qW Z5y0IkGFNd57TuG19zCtiib8m8Fsnk92rJtwSp1PzuIkh45X3uacRaZEZ2P3jkSxS6DxmhpXcIZ70 vSefBjUFfI12+ddYGPO28WD0Cu0JUoS/+LE1rgs+IprTyK8/NzA8hv3BpuxmtcDCoet4bI7AKDP0C P+O+BFvA8A6qPDbiCnzdAuc92yAmeBqoxQCh77FqPAjgvDF7BN5qvJRqrRQZpBnqE1KOxXBL5CqvI dkdhPoIl75Hf+FNR65l6LnYme6Qfr9mETwsNnWSaTvOIyCRbikcq3UL+gArD+DivHU4jeZeRa5kZX bRLuaOIw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiB5i-00000007DOD-1crF; Wed, 12 Feb 2025 11:39:42 +0000 Received: from mail-lj1-x22e.google.com ([2a00:1450:4864:20::22e]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApr-000000079eg-2HuX for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:21 +0000 Received: by mail-lj1-x22e.google.com with SMTP id 38308e7fff4ca-30902641fc2so9490781fa.2 for ; Wed, 12 Feb 2025 03:23:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359398; x=1739964198; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=fjCfgKpzfePVa0TyVy/JvnFpB7AUCXqKC35hzfV3hLA=; b=kVpmhNrlJRss6uhxvp4nv1wgY7/ur9UmMEuROaveWCyEDKlu2MLrjpiUOaCnxoqtkH CFZpk1KCbhrdufmhRByTL3KaNqeXtzlCshxPwO2EN4JyUrD4wZb4+JwnEZ/V/q2CChHE jgBhkcIXI6f2kxP6B84JkOxgkHhK51YKfWSoO7p2xqQLXxeoNkUx/jE/wdyDW2q4xv9B JgkBjYr0FGswOWDS7+FwO1kjVjAJ+j5CJNg8hPMhoRnaoD/4a8BI8uXEGbiFURoRxQNv qAeUksNh6bCUum6HGQ6BLvvYw9S9UXiNPwmiaaGbUJMYGdoUpy2uFCGuL+99F5ad4lfo gY5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359398; x=1739964198; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fjCfgKpzfePVa0TyVy/JvnFpB7AUCXqKC35hzfV3hLA=; b=U9NQcWlwUp7tPARXqZPzbqQxkNtcDHqDmnn5caDlp3cYi93L3qFWe1OILfTiKm42HX 3DCqZ7FhCY/AAbEBF5s2ifvmHN+nYC2q1GPxDrE+b6Kb0uxVJB7eb++7ByQffZiJkTGI h3e3keRgqmBXgx0gCyMyFvW7ZE+dvGmImoD1f2gPbVINQOy9+eoGm7Xnx258PCGhf9/h /iGzxb6j5EjPqgwMgpNr+wHgUsvgZ5mHE0Yybcok4W9g/XHDPydb0CR9LbwED8aoX8Xk moV9MzDDYX0jt1gUsPk04DRzaVDbaIMBVEzb5p4saL39xcidetcpgHXj49Lzyb894c+A E3zg== X-Gm-Message-State: AOJu0Yzl/jlQRqZhNP8bTL3eVZnJ3JdjtP6Z1I+fFn5aWpwPC9UsPwKi CjHdqwT7UnJo2HEy+17SichO4gaQadLAWzYvr1VV/2QIXRO8+H9TFgy2Tr3tgaw= X-Gm-Gg: ASbGncuIPmawMgz0PlSQFIIKacoToAW2RkWrGNzvyFo/BeeaZS78ylgrkySFNqvf/KM atBQEZ+FHiXMGvqnK00sGOrOIpje/tcJuwvN6/cY6lozkVR9/t3oDFEYbJ6pWAM2MPoaqri3Adw az2d9YCYckn3IIUHXdbU8hBfZLdDcaOJKBpbp1ajBfPFB14HEErFJlqLfqr/iuHLEK49BFNUtH2 PJ1bfZIbD9/w09cpsQyoyF05t0CzhmqIuWGp2ogkEN9qh2OGe9SErMJzavR9kqi31d0gPtf5cJq 0oIC+mhNaZB7esGmkiNUwgzQzg== X-Google-Smtp-Source: AGHT+IH751jL04M7McMnVl+Lf+gieXyjpD8AD36LFvaCxdH/f/sQl+jWkhu6bmO5y/ajdqZRtC2dng== X-Received: by 2002:a05:651c:211e:b0:304:68e5:eabd with SMTP id 38308e7fff4ca-30903699ffdmr9620161fa.3.1739359397554; Wed, 12 Feb 2025 03:23:17 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:17 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:05 +0100 Subject: [PATCH v4 11/31] ARM: entry: Merge the common and trace entry code MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-11-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032319_635913_31735BBC X-CRM114-Status: GOOD ( 24.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The trace entry code now can handle recursive and complex calls in C using stack. Move the common code over to using that approach. We now use the ret_fast_syscall return path also when tracing, which appears to work just fine. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 3 +-- arch/arm/kernel/entry-common.S | 61 +++++++++--------------------------------- arch/arm/kernel/syscall.c | 35 +++++++++++------------- 3 files changed, 28 insertions(+), 71 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index cb0073c4151bf70a82d15e17a95b6b6f48b245d6..9c664d8c5718f58034f442c20a030b59faf65dd9 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -19,8 +19,7 @@ extern const unsigned long sys_call_table[]; -int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp); -void invoke_syscall_trace(void *table, struct pt_regs *regs); +int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno); static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index dbc947d301ec11bb13007cb4bb161c035ede5c10..f0f1f8723965313b287158fb2a238908db72fcd2 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -39,7 +39,6 @@ saved_pc .req lr * from those features make this path too inefficient. */ ret_fast_syscall: -__ret_fast_syscall: UNWIND(.fnstart ) UNWIND(.cantunwind ) disable_irq_notrace @ disable interrupts @@ -47,13 +46,13 @@ __ret_fast_syscall: movs r1, r1, lsl #16 bne fast_work_pending - restore_user_regs fast = 1, offset = S_OFF + restore_user_regs fast = 0, offset = S_OFF UNWIND(.fnend ) ENDPROC(ret_fast_syscall) /* Ok, we need to do extra processing, enter the slow path. */ fast_work_pending: - str r0, [sp, #S_R0+S_OFF]! @ returned r0 + add sp, sp, #(S_R0 + S_OFF) /* fall through to work_pending */ #else /* @@ -63,10 +62,9 @@ fast_work_pending: * call. */ ret_fast_syscall: -__ret_fast_syscall: UNWIND(.fnstart ) UNWIND(.cantunwind ) - str r0, [sp, #S_R0 + S_OFF]! @ save returned r0 + add sp, sp, #(S_R0 + S_OFF) #if IS_ENABLED(CONFIG_DEBUG_RSEQ) /* do_rseq_syscall needs interrupts enabled. */ mov r0, sp @ 'regs' @@ -83,7 +81,9 @@ ENDPROC(ret_fast_syscall) #endif tst r1, #_TIF_SYSCALL_WORK - bne __sys_trace_return_nosave + beq slow_work_pending + b ret_to_user + slow_work_pending: mov r0, sp @ 'regs' bl do_work_pending @@ -257,19 +257,15 @@ ENTRY(vector_swi) str scno, [tsk, #TI_ABI_SYSCALL] #endif mov r1, sp @ put regs into r1 - ldr r10, [tsk, #TI_FLAGS] @ check for syscall tracing stmdb sp!, {r4, r5} @ push fifth and sixth args - - tst r10, #_TIF_SYSCALL_WORK @ are we tracing syscalls? - bne __sys_trace - mov r0, tbl - /* r1 already contains regs */ mov r2, scno @ syscall number from r7 - /* We return here no matter what, also pass this as an argument */ - badr lr, __ret_fast_syscall - mov r3, lr - b invoke_syscall + bl invoke_syscall_trace + cmp r0, #0 + beq ret_fast_syscall + /* This path taken when tracing */ + add sp, sp, #(S_R0 + S_OFF) + b ret_to_user #if defined(CONFIG_OABI_COMPAT) || !defined(CONFIG_AEABI) /* @@ -289,20 +285,6 @@ ENTRY(vector_swi) ENDPROC(vector_swi) .ltorg - /* - * This is the really slow path. We're going to be doing - * context switches, and waiting for our parent to respond. - */ -__sys_trace: - add r1, sp, #S_R0 + S_OFF @ pointer to regs - mov r0, tbl - bl invoke_syscall_trace - add sp, sp, #S_OFF @ restore stack pointer - b ret_to_user - -__sys_trace_return_nosave: - b ret_to_user - .macro syscall_table_start, sym .equ __sys_nr, 0 .type \sym, #object @@ -402,25 +384,6 @@ sys_mmap2: b sys_mmap_pgoff ENDPROC(sys_mmap2) -/* - * This call wants: - * r0: syscall table - * r1: regs - * r2: syscall number - * r3: pointer to return function - */ -SYM_TYPED_FUNC_START(invoke_syscall_asm) -#ifdef CONFIG_CPU_SPECTRE - csdb -#endif - mov tbl, r0 - mov scno, r2 - mov lr, r3 @ return address - ldmia r1, {r0 - r3} @ reload r0-r3 - /* Arguments 5 and 6 are (hopefully) on the stack */ - ldr pc, [tbl, scno, lsl #2] @ call sys_* routine -SYM_FUNC_END(invoke_syscall_asm) - /* * This call wants: * r0: syscall table diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index e90f04f35b3485968a1e9046cf0889f56cca92e5..36330023b5484399a4ca2fdb5727b410e2c74b6f 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -3,50 +3,45 @@ #include #include -int invoke_syscall_asm(void *table, struct pt_regs *regs, int scno, void *retp); -__ADDRESSABLE(invoke_syscall_asm); - -__visible int invoke_syscall(void *table, struct pt_regs *regs, int scno, void *retp) +static inline bool has_syscall_work(unsigned long flags) { - if (scno < NR_syscalls) - /* Doing this with return makes sure the stack gets pop:ed */ - return invoke_syscall_asm(table, regs, scno, retp); - - if (scno >= __ARM_NR_BASE) - return arm_syscall(scno, regs); - - return sys_ni_syscall(); + return unlikely(flags & _TIF_SYSCALL_WORK); } int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno); __ADDRESSABLE(invoke_syscall_trace_asm); -__visible void invoke_syscall_trace(void *table, struct pt_regs *regs) +__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno) { - int scno; + unsigned long flags = read_thread_flags(); int ret; - scno = syscall_trace_enter(regs); - if (scno == -1) - goto trace_exit_nosave; + if (has_syscall_work(flags)) { + scno = syscall_trace_enter(regs); + if (scno == -1) + goto trace_exit_nosave; + } if (scno < NR_syscalls) { ret = invoke_syscall_trace_asm(table, regs, scno); - goto trace_exit_save; + goto exit_save; } if (scno >= __ARM_NR_BASE) { ret = arm_syscall(scno, regs); - goto trace_exit_save; + goto exit_save; } ret = sys_ni_syscall(); -trace_exit_save: +exit_save: /* Save return value from syscall */ regs->ARM_r0 = ret; + if (!has_syscall_work(flags)) + return 0; trace_exit_nosave: local_irq_enable(); syscall_trace_exit(regs); + return 1; } From patchwork Wed Feb 12 11:23:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971809 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9FDC3C02198 for ; Wed, 12 Feb 2025 11:42:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=pr2oSZFQs/khvnnGeJohJHSvChXi9NfDOygi+WZQm/M=; b=P73bYsA1FcwbCuq9YjJ0DjiB9F M3aQ2ZMYC4+i/QQ8xqIn0Sc/0PVg2wsBUvXaFBnuLYWiRyetFfPLjTgyg86Oh48GwkpmXhLsSkMUS YI/Q/vX97bac6Ts86Uf/HRrYkEVcDQKI2QMafT2buBOgCqGxwWQeFC+M1ii5fzNz8gC6VbBw4epiS 1Y0ZqX6gfNeeuuLxnu6MPMjjQnF0ciZVwj1XWpsbSRy6ke9EKzg5uVtsnPW2c7Ynti/w3X79a1gpR iX4n/9MXwIvL0bTEGwYqKv5tTen0rE1xR91Y0qDQiEMkmtSOA+98ZO9L8aBswOGOTQHMWmcxWLrHn aWlJudlw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiB8T-00000007DpX-0D8Y; Wed, 12 Feb 2025 11:42:33 +0000 Received: from mail-lf1-x136.google.com ([2a00:1450:4864:20::136]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAps-000000079eq-0cYv for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:21 +0000 Received: by mail-lf1-x136.google.com with SMTP id 2adb3069b0e04-5450b91da41so2982325e87.3 for ; Wed, 12 Feb 2025 03:23:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359399; x=1739964199; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=pr2oSZFQs/khvnnGeJohJHSvChXi9NfDOygi+WZQm/M=; b=bB3REIlQH6EeaIzGGs5zMfUVah8nRWR6uP7AC0wIFfTq4/xBhvupo0DHOt5heS8FOg SRgj0m1D353Y+U6pjq+OuSWKV6UD4JHdfE0jE6kgTOSKXSDDMIcBBLabfAvRv5gvX/OT 8BK6w46tsq4DB5IsASb1VNliebdOtTF205yzmgH8228DUeHkOYuJfySCqUx3FjEALvok r6ABUdKTGs25ZmdVHmS/iRm/BH+5wc4Sxh57MgLisGej2kL8PYH851SG80tD/RnfuFC2 wqGud1RdsYqDIoj1Yzh0hd15+jNkPYP/B/wOMbI6CO9wXioOF3SqxWUtK1LB1NTI+F8H C0aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359399; x=1739964199; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pr2oSZFQs/khvnnGeJohJHSvChXi9NfDOygi+WZQm/M=; b=QvCjRl58T+hk5+16Z6rUAIYqmB9FLCSBWcPpEEzQwt2vXySaaheNwA9ker3GP8pV9d YKYBTMOyLp5V7h+Jds4YzZKOPH18nZ8avbSqACrn0qS/LlwTXEO76ko5VpwW3EDaT023 eRaOUaW6t/29JCJgBUAwFReO38SZchqlxmrXG50agjsvw0CxrK+KBZ7tA+LLfuwSnp/J II7ocTKBsdOatJUpbbtI1EvQqBNKfmPP8NKGk7dTJTI6JBV9WRQxxyVySYiYII05/DBb GwdLyH2kVTukEb7WaTgyT68S3Rk49TAqCY7Nns8ee3WeR1SCaJmqf1arhxLxIQUI6bGj ERsg== X-Gm-Message-State: AOJu0YwCQXqD0Yl8xojPwoExbXR2GaxRaNuM1ZBm0ZHBCWIvPh7oi9aQ SkxrtjyR9brUV6xu2hkqssvMjbtPwoEo8jVz2ejJDrhjqlp+CrrJLce79hnB4I8= X-Gm-Gg: ASbGncsj0co93r1Iz5RfSEgc7Siblf0iW7UnH4NH8j77F/phq7tEULZ/N0RGVDD8jgL ma9Tjspf/MJUPetRecVxERTiLXEXdI2X0rniX675wZ6CqjX1ITucL/ybbzE3AOqlXVBAjhTAL6e ThOAJCBtePkWfec8/gctkvgnSOIcgYMv/aUNavHC/zel3FRtW54wDxfjtomSS8+YvBhx9qPc4eH bHsiqQPB+VVHaBdy2dU6PhQxyVCET2tFB8EK+ZIHxDukf/W1bGvVmLvS0aHQEU6vJUsa8EdFZjF clwn3sQCc9LiexDhVc7etHZQPw== X-Google-Smtp-Source: AGHT+IH+IdKt2FT6tNPNoUo8a9kRuj3wtpUX5pKBXA/WPHTi8mdjHVXpF8+ZHzJO2eM+nciMsumGwg== X-Received: by 2002:a05:6512:3b99:b0:545:cd5:84ce with SMTP id 2adb3069b0e04-545180de56dmr819224e87.8.1739359398590; Wed, 12 Feb 2025 03:23:18 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:18 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:06 +0100 Subject: [PATCH v4 12/31] ARM: entry: Rename syscall invocation MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-12-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032320_191064_8D0FC1A5 X-CRM114-Status: GOOD ( 15.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that we do not have a separate trace syscall invocation path, rename the invocation functions to be generic. This can be squashed into the previous patch, it is just done here for the previous patch to be easier to get a cleaner diff and be easier to read. Signed-off-by: Linus Walleij --- arch/arm/include/asm/syscall.h | 2 +- arch/arm/kernel/entry-common.S | 10 +++++----- arch/arm/kernel/syscall.c | 8 ++++---- 3 files changed, 10 insertions(+), 10 deletions(-) diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index 9c664d8c5718f58034f442c20a030b59faf65dd9..b94fd7ce17f82f0baf06b74800245112e1f13002 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -19,7 +19,7 @@ extern const unsigned long sys_call_table[]; -int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno); +int invoke_syscall(void *table, struct pt_regs *regs, int scno); static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index f0f1f8723965313b287158fb2a238908db72fcd2..f1e48002bd3075ea67b5883178583127fa0055c6 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -260,7 +260,7 @@ ENTRY(vector_swi) stmdb sp!, {r4, r5} @ push fifth and sixth args mov r0, tbl mov r2, scno @ syscall number from r7 - bl invoke_syscall_trace + bl invoke_syscall cmp r0, #0 beq ret_fast_syscall /* This path taken when tracing */ @@ -390,7 +390,7 @@ ENDPROC(sys_mmap2) * r1: regs * r2: syscall number */ -SYM_TYPED_FUNC_START(invoke_syscall_trace_asm) +SYM_TYPED_FUNC_START(invoke_syscall_asm) #ifdef CONFIG_CPU_SPECTRE csdb #endif @@ -400,17 +400,17 @@ SYM_TYPED_FUNC_START(invoke_syscall_trace_asm) /* Make space to copy the two syscall stack arguments */ sub sp, sp, #S_OFF mov scno, r2 - badr lr, __invoke_syscall_trace_ret @ return right here + badr lr, __invoke_syscall_ret @ return right here ldmia r1, {r0 - r6} @ reload r0-r6 from regs stmia sp, {r4, r5} @ copy stack arguments ldr pc, [tbl, scno, lsl #2] @ call sys_* routine -__invoke_syscall_trace_ret: +__invoke_syscall_ret: /* Drop the copied stack arguments */ add sp, sp, #S_OFF pop {r4 - r10, lr} ARM( mov pc, lr ) THUMB( bx lr ) -SYM_FUNC_END(invoke_syscall_trace_asm) +SYM_FUNC_END(invoke_syscall_asm) #ifdef CONFIG_OABI_COMPAT diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index 36330023b5484399a4ca2fdb5727b410e2c74b6f..6f518ec3870fb36a64d1b671b20f96d63a52bf78 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -8,10 +8,10 @@ static inline bool has_syscall_work(unsigned long flags) return unlikely(flags & _TIF_SYSCALL_WORK); } -int invoke_syscall_trace_asm(void *table, struct pt_regs *regs, int scno); -__ADDRESSABLE(invoke_syscall_trace_asm); +int invoke_syscall_asm(void *table, struct pt_regs *regs, int scno); +__ADDRESSABLE(invoke_syscall_asm); -__visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno) +__visible int invoke_syscall(void *table, struct pt_regs *regs, int scno) { unsigned long flags = read_thread_flags(); int ret; @@ -23,7 +23,7 @@ __visible int invoke_syscall_trace(void *table, struct pt_regs *regs, int scno) } if (scno < NR_syscalls) { - ret = invoke_syscall_trace_asm(table, regs, scno); + ret = invoke_syscall_asm(table, regs, scno); goto exit_save; } From patchwork Wed Feb 12 11:23:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971810 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B475FC02198 for ; Wed, 12 Feb 2025 11:44:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Ugjh5wDawxRKPKlBJt/tVpltkHg0eR7Bu8LyZ1i/cWo=; b=V9AYk8sV6ZMIzS/Jducv4OMmii vWkBf3rNcFOpb/31LEMW+G0zth7mVzzUL56i0woa3UL7PqLdRBANIZqm+/MVVOa2lfVyxMwmSy/P0 oT0R+JBHrirxeTythZ1GDSHv0xc4E/3cpeT7up6c1JqQPlEWoxqhro3cQAhQeoaBhX0/lVN3Mz/M7 oZZVNEAN57YVDHb3SCVBvx6Btk3zyP3txTA9dCONrIbtWotvsJYKj3CHsXe/Z1cV34ps7vY29ESW3 f7Ben4aH1I3xiMAFrXfFgkm1wvpOoWyPdWeHB57R/4OPSPjFG3qky1kbVAekJapeYWqpB02jfzJ5h nTRwNBUQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiB9r-00000007E2H-2yUL; Wed, 12 Feb 2025 11:43:59 +0000 Received: from mail-lj1-x235.google.com ([2a00:1450:4864:20::235]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApt-000000079fM-2ARW for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:22 +0000 Received: by mail-lj1-x235.google.com with SMTP id 38308e7fff4ca-307c13298eeso7334581fa.0 for ; Wed, 12 Feb 2025 03:23:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359400; x=1739964200; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Ugjh5wDawxRKPKlBJt/tVpltkHg0eR7Bu8LyZ1i/cWo=; b=KWx6VOflL0mgMCxV23pGqsBxc9czo3eBmIpLyNuiRuWL9QNn6CwkSI3139FtL0sq47 EUqvZaZwhTc3KnShnYPCXl8HKe7D6pXviNf0vnA0ng7JjKzgx7h3/MbQzcA/FCyVGxxq Yres+VkEdr6b9VTStU507jTxqxt7LVnpfry3xLzFvZXE3GNqWBnLiu21Pwxex2IKR2f6 NlIVBH2r7bEEOJBTuY3NsFwsskC0yAtvO2QwH5bR7DSXD0OCHyN+CDs31Vr15OqA8NQE uMP/bGLvYi/LsTyYMtMUAMWa5i0CR4Tnbn8p0aYbsG2CBQn2+CMRhUxX3La4N99SObOx hQGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359400; x=1739964200; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ugjh5wDawxRKPKlBJt/tVpltkHg0eR7Bu8LyZ1i/cWo=; b=Zlsv2Hi/2nMCTWfBkqI2++DXZtdpz4sXnFtFRwz9T5pEBQxvJpOM9lTOVEONILzZ/f A3HnnZ/4OE0xI9CoIisdQa4zsApWjD+JkcyYzZNks/WFMbMfaAklTWhpeCmSJxiDYzRx vWjHYuBEs/SR5JWzSvU6t5XiYj3KsVFHTwgSHSAxuEMYHHLtdDN7o4WhyMCoC0QUW8to NbHPNDndhHZxzbpO+i6nJwTkYsYXF/NrepPo2UEIwUr9mVxpahoxTneZ+/xxshc0Kuhg HyLNIKcDKsiIanSNIQLYttI8uBsjKjKZ+ezGU4z3p15B3rV1IEl1vUsui08Ikb0AxWOA iSog== X-Gm-Message-State: AOJu0YxYa1fEgcZDO3ik6q89IKON0nYt5+JXEWoiLU7KcmxCQItxBWAx 6QtrOBjurY/OlNtTJba45J7323ZpGYanEWyJnUlvuqKKP7QkKOk28/4Br6Wnwlw= X-Gm-Gg: ASbGncuWaJcSn84HQwssO0mt/nMEGyIQs1D5foyo43poVkMacp+S7dfLDb71uvqqHPr qyAcF53L+dfvaaIPPzhrL0EL9hJoKeM1anNT+cOawrG98gGNig4RM+l1vcUn0u9scECWckxd/zC CPkOUT57wKcJByUGK/sfDGgwmFCiMYKlL8jOp/d7fkDSEPYiPxiZlVtUNNDoslu29qGZb0Jzsh3 l8TsmVb+0aRqwDIs8Kr3H15BGgOTFc7fBncGKp/NuEq3FWfbSxjsr11qa2jyAVnUOCOBQt8yAws eXEXqLNGe8SfMMvqjeeyGSgrzA== X-Google-Smtp-Source: AGHT+IH8i+hiWxB2GgOn8G+iygoMLu98Ofz4FfSp+m0uwMKygIL9Wg48i31mDB147IwXjZ52Svqzaw== X-Received: by 2002:a05:651c:c8:b0:308:f52c:7698 with SMTP id 38308e7fff4ca-308f91ce77emr25765241fa.13.1739359399603; Wed, 12 Feb 2025 03:23:19 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:19 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:07 +0100 Subject: [PATCH v4 13/31] ARM: entry: Create user_mode_enter/exit MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-13-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032321_567988_1E971AA6 X-CRM114-Status: GOOD ( 19.14 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The callbacks to the context tracking will be repurposed for several uses that are needed on the IRQ transition to/from userspace. Rename the macro, establish call sites in C calling into the context tracking following the corresponding generic entry function prototypes, despite the assembly macro names become a bit long this makes it clear to readers exactly what is going on and where this call will go. Drop the ifdefs pertaining to context tracking from the macro. The C calls we will use have stubs that will compile these out anyway. The inversion of the signature of the context tracking calls are especially confusing since the generic entry uses the reverse semantics: *enter from* user mode (to kernel mode) and *exit to* user mode (from kernel mode) instead of the other way around as the old context tracker code user_exit_callable() and user_enter_callable() which have inverted semantics. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 14 ++++++++++++++ arch/arm/kernel/Makefile | 2 +- arch/arm/kernel/entry-armv.S | 2 +- arch/arm/kernel/entry-common.S | 4 ++-- arch/arm/kernel/entry-header.S | 24 ++++++++++++------------ arch/arm/kernel/entry.c | 15 +++++++++++++++ 6 files changed, 45 insertions(+), 16 deletions(-) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h new file mode 100644 index 0000000000000000000000000000000000000000..e26f369375ca3cf762f92fb499657a666b223ca2 --- /dev/null +++ b/arch/arm/include/asm/entry.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __ASM_ENTRY_H__ +#define __ASM_ENTRY_H__ + +struct pt_regs; + +/* + * These are copies of generic entry headers so we can transition + * to generic entry once they are semantically equivalent. + */ +void irqentry_enter_from_user_mode(struct pt_regs *regs); +void irqentry_exit_to_user_mode(struct pt_regs *regs); + +#endif /* __ASM_ENTRY_H__ */ diff --git a/arch/arm/kernel/Makefile b/arch/arm/kernel/Makefile index 69ddf51081f4791982518d37df60c11211b8955a..dd45c6eb8cbdd2595edc4f26026c12b25dd49717 100644 --- a/arch/arm/kernel/Makefile +++ b/arch/arm/kernel/Makefile @@ -17,7 +17,7 @@ CFLAGS_REMOVE_return_address.o = -pg # Object file lists. -obj-y := elf.o entry-common.o irq.o opcodes.o \ +obj-y := elf.o entry.o entry-common.o irq.o opcodes.o \ process.o ptrace.o reboot.o io.o \ setup.o signal.o sigreturn_codes.o \ stacktrace.o sys_arm.o time.o traps.o \ diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index ef6a657c8d130663e68acd07226815ce8a3fff7d..93a37437a9520118eff411f009e8f6bd729412fc 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -406,7 +406,7 @@ ENDPROC(__fiq_abt) #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_off #endif - ct_user_exit save = 0 + asm_irqentry_enter_from_user_mode save = 0 .endif .endm diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index f1e48002bd3075ea67b5883178583127fa0055c6..ff1dd3169346f3770cad6b7e218f5d74ffc646fe 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -111,7 +111,7 @@ ENTRY(ret_to_user_from_irq) no_work_pending: asm_trace_hardirqs_on save = 0 - ct_user_enter save = 0 + asm_irqentry_exit_to_user_mode save = 0 #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack @@ -191,7 +191,7 @@ ENTRY(vector_swi) alignment_trap r10, ip, cr_alignment asm_trace_hardirqs_on save=0 enable_irq_notrace - ct_user_exit save=0 + asm_irqentry_enter_from_user_mode save = 0 /* * Get the system call number. diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 52b4fa97226dbfa4b55aca8eaf74ae24e1e220f8..fb5bb019199b2871e29e306a29bea8fdf47dd7f3 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -362,31 +362,31 @@ ALT_UP_B(.L1_\@) .endm /* - * Context tracking subsystem. Used to instrument transitions + * Context tracking and other mode transitions. Used to instrument transitions * between user and kernel mode. - */ - .macro ct_user_exit, save = 1 -#ifdef CONFIG_CONTEXT_TRACKING_USER +*/ + .macro asm_irqentry_enter_from_user_mode, save = 1 .if \save stmdb sp!, {r0-r3, ip, lr} - bl user_exit_callable + mov r0, sp @ regs + bl irqentry_enter_from_user_mode ldmia sp!, {r0-r3, ip, lr} .else - bl user_exit_callable + mov r0, sp @ regs + bl irqentry_enter_from_user_mode .endif -#endif .endm - .macro ct_user_enter, save = 1 -#ifdef CONFIG_CONTEXT_TRACKING_USER + .macro asm_irqentry_exit_to_user_mode, save = 1 .if \save stmdb sp!, {r0-r3, ip, lr} - bl user_enter_callable + mov r0, sp @ regs + bl irqentry_exit_to_user_mode ldmia sp!, {r0-r3, ip, lr} .else - bl user_enter_callable + mov r0, sp @ regs + bl irqentry_exit_to_user_mode .endif -#endif .endm /* diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c new file mode 100644 index 0000000000000000000000000000000000000000..8b2e8ea66c1376759d6c0c14aad8728895b3ff1e --- /dev/null +++ b/arch/arm/kernel/entry.c @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include + +noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) +{ + /* This context tracking call has inverse naming */ + user_exit_callable(); +} + +noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) +{ + /* This context tracking call has inverse naming */ + user_enter_callable(); +} From patchwork Wed Feb 12 11:23:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971811 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E38D2C021A0 for ; Wed, 12 Feb 2025 11:45:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=MgvrpQqwFQ0R0umdPRsAplUpCRMZEe+n8Q2dyj0aMbc=; b=TzVME0153Ln1liY6to+ko9QM0W 1nrNc3QZPGloGettzdenES5nhTRwOxpYuM/uGgePVCCIshwWaA/R2Msj2tJeT3uAQqdLDdSzonmA2 OwtXThgoMqBryhctCBFye25JtznTiHcTYZFSYBmfRFsKpTzKJCmXt3Em75gSJd12E4XnXCNZmP2SN KLWbNbeyIN4fcbfUP7/z0pLbZtJrzWCG/Jiz9PIxf3bMEiJD1HwcgM/iXstjIeuObeNKmYMok7Fh9 /Ktgn94BF/srtrZbjvZJvlNHQaQmG6R+HkvWLbRi4I7WUOpzeydfHX1bRvtK/C0thdZRp4O5UsObj 9GXkarbw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBBF-00000007EK6-1n7U; Wed, 12 Feb 2025 11:45:25 +0000 Received: from mail-lj1-f170.google.com ([209.85.208.170]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApu-000000079fo-3Gi8 for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:23 +0000 Received: by mail-lj1-f170.google.com with SMTP id 38308e7fff4ca-307bc125e2eso62128231fa.3 for ; Wed, 12 Feb 2025 03:23:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359401; x=1739964201; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=MgvrpQqwFQ0R0umdPRsAplUpCRMZEe+n8Q2dyj0aMbc=; b=aqPFOedqVlrfQFvuOOO1iLlgHgkTtSvgl1Dm77ytZ1YJLp/nMErK8RxU4A1F58bBdS r1IWO+jB8+ezvQuVchvoMUwg/rLyPpx13Jj/IIEL/qqsSlZwuDxMgHiTSlT1stj31JuI RT+e/YmWYSloAwtrH3N8ikHuI9rWk3dyYCbY6TavG/41mOmPhXlh8RgZ6x2AUCCNwv4a Xmirp8y1Z0J0b+cnQhpwdkB5rbDjuz7uHFYDitaB5/hkvSnLLaihFOBQ727GKSyU/OA6 do4oecNPdxizjTgLq/gy7LpWOE3ktUPCFPgn09D9sIeqi3y7Suws1aJZEy7yjaKiCgtW 0p5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359401; x=1739964201; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MgvrpQqwFQ0R0umdPRsAplUpCRMZEe+n8Q2dyj0aMbc=; b=lak1sKwEDflXXIRDfikeOb5pQgh78uKqauxs5oapH/8WflQ89emTojRz+zjnWAgAK9 H4fLCu8+vfHWp2wkMbLy8nOPpal5zqGkWS/AaTZu4ldEta4RTDXEqPrvqpwBqYUp6Scj Fgc1sdlmLXwDUbJXwnvKnD+gFro2Im5FiKN/PGmEGlXVO6I6Ne2y2pKcryAzb4MQIEzc fUzg0L8qLNYebk2iF9nYQ2cNog0RxvPVCTmWobFJve0iZcxPJtCFddm83DBfXEkgJ+8n qt4LKcO/sQunepxNZZi5fWY5roYdf2tVOmUiY3/iARFmKVqPqQ5T8iXVsntSoIGG4UYb 42VQ== X-Gm-Message-State: AOJu0YxJt6oyp0CCphnylM6BZeQ1coJVQ4YFhtuoR+O6VJR6QIX7BOsf j6hutMc+OQ9RX7AKZOsz2UDsbFn7ZB+ukcby1OwxL7YfLZK0RbTLXKXsQEeN4S0= X-Gm-Gg: ASbGnctxbwZ7N9fR1QRnR5aKPSnsVfGoKlw7Id2ChINetwH+1m1nRntHqAujle2RWfp JhudA4Uv5STxSOxFSBYg8GiZ2vQwyDVAIPA8cqNEWBIyD7O06omeCrIi0gkEpgKtbgMfYl6dM5h 4FJqMUJ7tKZO3vMQw1Hiwk05UW4ba+qPEWpl4aBAyOBQNKg8pFlaxSwuBttp0Ao+Epy5hzCSfLE 77ZbYs1lpdS+0Fcgcr1Lt08nCOZJcnUBKzX1Ptkt/UTTrm6X1AjyenkqeCyte0idGYIGcEeBItf jhuWAm5zq9Ol6sJtW6JWBo9aiQ== X-Google-Smtp-Source: AGHT+IFJnUYDjF8FB8NajpoFIXNop4QIT5L6sFMa3fWK7tI1+RrzzH8NZHHj8Q+W0l9RcxU2y5/akA== X-Received: by 2002:a2e:9fc2:0:b0:307:9648:65c3 with SMTP id 38308e7fff4ca-30903665fe5mr11295941fa.15.1739359400622; Wed, 12 Feb 2025 03:23:20 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:20 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:08 +0100 Subject: [PATCH v4 14/31] ARM: entry: Drop trace argument from usr_entry macro MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-14-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032322_820284_CDCCE422 X-CRM114-Status: GOOD ( 12.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The trace argument select whether to look into tracing of the hardirqs for the exceptions from userspace, but also selects whether to call further down to establish context for RCU. Split off a second macro that is called on everything except the FIQ since this is the only outlier. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 93a37437a9520118eff411f009e8f6bd729412fc..c71110126fc105fc6ac2d6cb0f5f399b4c8b1548 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -357,7 +357,7 @@ ENDPROC(__fiq_abt) #error "sizeof(struct pt_regs) must be a multiple of 8" #endif - .macro usr_entry, trace=1, uaccess=1 + .macro usr_entry, uaccess=1 UNWIND(.fnstart ) UNWIND(.cantunwind ) @ don't unwind the user space sub sp, sp, #PT_REGS_SIZE @@ -401,13 +401,14 @@ ENDPROC(__fiq_abt) @ Clear FP to mark the first stack frame @ zero_fp + .endm - .if \trace + /* Called after usr_entry for everything except FIQ */ + .macro usr_entry_enter #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_off #endif asm_irqentry_enter_from_user_mode save = 0 - .endif .endm .macro kuser_cmpxchg_check @@ -429,6 +430,7 @@ ENDPROC(__fiq_abt) .align 5 __dabt_usr: usr_entry uaccess=0 + usr_entry_enter kuser_cmpxchg_check mov r2, sp dabt_helper @@ -439,6 +441,7 @@ ENDPROC(__dabt_usr) .align 5 __irq_usr: usr_entry + usr_entry_enter kuser_cmpxchg_check irq_handler from_user=1 get_thread_info tsk @@ -452,6 +455,7 @@ ENDPROC(__irq_usr) .align 5 __und_usr: usr_entry uaccess=0 + usr_entry_enter @ IRQs must be enabled before attempting to read the instruction from @ user space since that could cause a page/translation fault if the @@ -476,6 +480,7 @@ ENDPROC(__und_usr) .align 5 __pabt_usr: usr_entry + usr_entry_enter mov r2, sp @ regs pabt_helper UNWIND(.fnend ) @@ -495,7 +500,7 @@ ENDPROC(ret_from_exception) .align 5 __fiq_usr: - usr_entry trace=0 + usr_entry kuser_cmpxchg_check mov r0, sp @ struct pt_regs *regs bl handle_fiq_as_nmi From patchwork Wed Feb 12 11:23:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971812 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D97C5C02198 for ; Wed, 12 Feb 2025 11:47:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=0l4jcdIT0+K6QMNd0VK8RPhMVALn2TkHG1rRCOsPTks=; b=T/l4ezRBextsHxXz46fUEevT/H ZV/thqSLI7UiSCCQQ1riaP4niHUCscMJOEz+VVM5JpuHiuCFyMjC2P2eKgSMZYaEzbimvtqMbv6z5 yX4Pc+AWtO54SSp97+wjqhIyxbbFeEdzTZCOZLykSjy+HyuZa5lyBUvb+VBWb5JN2Rp5JfoATlWY8 4fgUJdaCfquzgjg92thm++v38buAZwQJnc1iwlJUbJk0Cze5+Wyh2CBvSUrpFEFxeoYVUv7BVW0wG 9RVCCteWaLMhO57iyB0NXAk1Lz+9VGijiG0Y4782MQATYMRkV/vMtYgG4dudfBcz/BjF+zbkC3L1U l9SVdVxA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBCe-00000007EXA-0vbK; Wed, 12 Feb 2025 11:46:52 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApz-000000079hF-0mTt for linux-arm-kernel@bombadil.infradead.org; Wed, 12 Feb 2025 11:23:27 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Sender:Reply-To:Content-ID:Content-Description; bh=0l4jcdIT0+K6QMNd0VK8RPhMVALn2TkHG1rRCOsPTks=; b=UVYLqkT0WCW/Xiadm/RDt0xgTU 75oam2RAbjEnnfOrYjYMDeYlKFkDM9/wAr0t4mz5ood4YZWNgEXLWQ3PhZu7LSMhFLORVSBLFFnRI O2HSeqg4ojn+UJQ8TMsN1WedIs/YMuuev5X7IECTqXeor6y96mvVdrwO/wrzJ0nLzd88IQGWcwzzH YwdS2Q6Zu8nQc3aJcNw9exrU6jAOfTeLuU6kADCk4cCsQaRsKYLaQFSqa2Ubd7XNQ7JYFYnUMxdmb 3+pXFn6babDEx27KYhs3tFkF6Lj/7Xlft27SWqEnYYqOSCi7lThfRfoTPVirqieqbefFhS5bZbwU0 BY8v1PPg==; Received: from mail-lj1-x229.google.com ([2a00:1450:4864:20::229]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApw-00000000no4-0sJy for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:26 +0000 Received: by mail-lj1-x229.google.com with SMTP id 38308e7fff4ca-30738a717ffso58894451fa.0 for ; Wed, 12 Feb 2025 03:23:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359402; x=1739964202; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=0l4jcdIT0+K6QMNd0VK8RPhMVALn2TkHG1rRCOsPTks=; b=nJXEZmEF5fpp1JThWtd+EpFmxpX6H4y1yoCMsSkYV9BeKyzrKIL9NJp4cTRLe5VaTd 4sE06IjttFwhmjX8ab34UlLBDmhPpk2V21f0Yce8fm/RQRB4pic+iNx1gfPY2joxprJs dXb9A2KAzu8Fu3X2CTpppAmZn1tqClQJSBa528/NynKgKaHUY8ObQxnipzv8fKOCxMdh 1RIoWq/JmD6tAMIl6lTFE+2M7V/JeOTNDF7FqjYQmY2cS9Mc83kDa0SnWVy/afN7/8As Ups4ClBPvQlxCwxLJmGlbV0jBfUvvuVQ1l8+kOz0lIZ9O5KDzUaBvRlrMIJ7gabRkOir /18A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359402; x=1739964202; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0l4jcdIT0+K6QMNd0VK8RPhMVALn2TkHG1rRCOsPTks=; b=mpzazbZ0Wf6Y6O7ZsI6OTsqkHH3I6Rah24QwP/s6PlW9F8XBl0h2Y3R5AfAFOM853G EcMMIBVZk3V6JYGM07jUvUaYKsZeJGm72JxrMYn1E+LsNHJgpDola0DkFLYmXH9nxSwq ht7A+cZasMwcT053bvzdgVv3j9LpYUhCa3ErzL0uWgrdiUzr6B7XSYSH+k6riHIQjQMM MkbGBAW6v75tTcpMLBy6PCIiUV4jh5LxVLh5qj9Zx1u71ZZpGJwTzkQFfkEkl72DQ9gV 4g3h7b5rRZ+FBIP3uhs1H/Y3Z12y/8DYSC+f3VhPKjqPzdvlWYLxtHcs7XIIq5TzW7a1 fdng== X-Gm-Message-State: AOJu0YxoUmzMgtqIkXeFtjN95tj4fFXqBrYeQCY8qArwuoyLNopE3o1o +3Yrh+5jCAk1m8nipXEJ4vIYQbg1N0w0yyue7DsmjlIxCkTKNBCXPn/JQqwbC+c= X-Gm-Gg: ASbGncv7ro9dIr3nexabEpZt/MqjUWUYJMEx8QIcfQx93TlFypWqIVIHmltc6QyNiFU FYARlE+K/T2zgjNdYvckrHs2U/IODtEvjPDj5bKU+nQT5xx2UPAHawJ+Jc3ehw9rM1vqw+VwATi gv1i3gpQipsq1qVPVKgQarlhd+B4DKL4hBoEc0CWzvsFiMjcjP/TqnakP0if97zlLBCYNhFLTHC 2LzWKFF0wT096M3oUrnUBD/8DPkXvHyRB1LTAcoPvPnELAyR/2rG6bWBlsa/ROIX8ppTadvtY9I VYQQR9Ot9WIvFv4eG+JLDcmcmQ== X-Google-Smtp-Source: AGHT+IE0MOK5A4Hgs6x7f54TlSM9Gy9GRZ5jq57qtCi8sQldGDST9iVi2gYFwCJT4+05NjX+kZxcDQ== X-Received: by 2002:a2e:a909:0:b0:307:e302:a34 with SMTP id 38308e7fff4ca-3090500af87mr9903901fa.20.1739359401620; Wed, 12 Feb 2025 03:23:21 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:21 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:09 +0100 Subject: [PATCH v4 15/31] ARM: entry: Separate call path for syscall SWI entry MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-15-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_112324_553542_434D4DC6 X-CRM114-Status: GOOD ( 18.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The syscalls (SWIs, software interrupts) are deviating from how any other interrupts are handled as they enable the IRQs again while processing the syscall, while "hard" IRQs disable all interrupts until they are handled. Break out syscall_enter_from_user_mode() into its own function and call it instead of irqentry_enter_from_user_mode(). As we are moving toward generic entry, we use the signature from the generic function. As the generic function requires the syscall number to be determined, we move the call down below the code that figures out the syscall number, the only practical effect should be that interrupts are re-enabled a few instructions later. As we move the trace_hardirqs_on/off() calls into C, we can just get rid of the helper macro usr_entry_enter again and call asm_irqentry_enter_from_user_mode directly. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 1 + arch/arm/kernel/entry-armv.S | 16 ++++------------ arch/arm/kernel/entry-common.S | 18 +++++++++++++----- arch/arm/kernel/entry.c | 14 ++++++++++++++ 4 files changed, 32 insertions(+), 17 deletions(-) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index e26f369375ca3cf762f92fb499657a666b223ca2..e259b074caef75c7f777b18199623f07bebee5b4 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -8,6 +8,7 @@ struct pt_regs; * These are copies of generic entry headers so we can transition * to generic entry once they are semantically equivalent. */ +long syscall_enter_from_user_mode(struct pt_regs *regs, long); void irqentry_enter_from_user_mode(struct pt_regs *regs); void irqentry_exit_to_user_mode(struct pt_regs *regs); diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index c71110126fc105fc6ac2d6cb0f5f399b4c8b1548..6edf362ab1e1035dafebf6fb7c55db71462c1eae 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -403,14 +403,6 @@ ENDPROC(__fiq_abt) zero_fp .endm - /* Called after usr_entry for everything except FIQ */ - .macro usr_entry_enter -#ifdef CONFIG_TRACE_IRQFLAGS - bl trace_hardirqs_off -#endif - asm_irqentry_enter_from_user_mode save = 0 - .endm - .macro kuser_cmpxchg_check #if !defined(CONFIG_CPU_32v6K) && defined(CONFIG_KUSER_HELPERS) #ifndef CONFIG_MMU @@ -430,7 +422,7 @@ ENDPROC(__fiq_abt) .align 5 __dabt_usr: usr_entry uaccess=0 - usr_entry_enter + asm_irqentry_enter_from_user_mode save = 0 kuser_cmpxchg_check mov r2, sp dabt_helper @@ -441,7 +433,7 @@ ENDPROC(__dabt_usr) .align 5 __irq_usr: usr_entry - usr_entry_enter + asm_irqentry_enter_from_user_mode save = 0 kuser_cmpxchg_check irq_handler from_user=1 get_thread_info tsk @@ -455,7 +447,7 @@ ENDPROC(__irq_usr) .align 5 __und_usr: usr_entry uaccess=0 - usr_entry_enter + asm_irqentry_enter_from_user_mode save = 0 @ IRQs must be enabled before attempting to read the instruction from @ user space since that could cause a page/translation fault if the @@ -480,7 +472,7 @@ ENDPROC(__und_usr) .align 5 __pabt_usr: usr_entry - usr_entry_enter + asm_irqentry_enter_from_user_mode save = 0 mov r2, sp @ regs pabt_helper UNWIND(.fnend ) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index ff1dd3169346f3770cad6b7e218f5d74ffc646fe..14b2495cae3c2f95b0dfecd849b4e16ec143dbe9 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -109,8 +109,6 @@ ENTRY(ret_to_user_from_irq) movs r1, r1, lsl #16 bne slow_work_pending no_work_pending: - asm_trace_hardirqs_on save = 0 - asm_irqentry_exit_to_user_mode save = 0 #ifdef CONFIG_GCC_PLUGIN_STACKLEAK @@ -189,9 +187,6 @@ ENTRY(vector_swi) reload_current r10, ip zero_fp alignment_trap r10, ip, cr_alignment - asm_trace_hardirqs_on save=0 - enable_irq_notrace - asm_irqentry_enter_from_user_mode save = 0 /* * Get the system call number. @@ -256,6 +251,19 @@ ENTRY(vector_swi) #else str scno, [tsk, #TI_ABI_SYSCALL] #endif + + /* + * Calling out to C to be careful to save and restore registers. + * This call could modify the syscall number. scno is r7 so we + * do not save and restore r7. + */ + mov r0, sp @ regs + mov r1, scno + push {r4 - r6, r8 - r10, lr} + bl syscall_enter_from_user_mode + pop {r4 - r6, r8 - r10, lr} + mov scno, r0 + mov r1, sp @ put regs into r1 stmdb sp!, {r4, r5} @ push fifth and sixth args mov r0, tbl diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 8b2e8ea66c1376759d6c0c14aad8728895b3ff1e..1973947c7ad753fccd694b3ef334fba1326f58b6 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -1,15 +1,29 @@ // SPDX-License-Identifier: GPL-2.0 #include #include +#include + +long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) +{ + trace_hardirqs_on(); + local_irq_enable(); + /* This context tracking call has inverse naming */ + user_exit_callable(); + + /* This will optionally be modified later */ + return syscall; +} noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) { + trace_hardirqs_off(); /* This context tracking call has inverse naming */ user_exit_callable(); } noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) { + trace_hardirqs_on(); /* This context tracking call has inverse naming */ user_enter_callable(); } From patchwork Wed Feb 12 11:23:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971814 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5DEC2C021A1 for ; Wed, 12 Feb 2025 11:49:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ggLp2N6nIh1N25l9qqVFGy2pp6+E0t6/i4rH9zHeAXo=; b=Yqfs7EyOcBBlxno5zXvqRyXKUc JQHwagMBDYOywhFAHix7AlzOovmKctsH58ce9GTv8phWhlvnDJ5e62OGaGmjzduJyZmf4xRZ6nYk1 oBa1V2O63D2Gd86PA3tKdM4ZkuxQRy2uV59SZ5bpaJdmCutc0cDMYJJXEdozIScAfHeDLMBtcoG+h j8QXtV6OlAyI1zaSHocbtLNxNvYxLYoCkSBPNTk9saEgNhJx6TX/UK2H6AoriRsxiATQpvTWomgW6 BauLiAuo0Crwo5sdy6HMfIyVeXpE5ogqcI9ny8gHdb33PdYyNqQl3coYNiv5SMNO8djqAGvz8fSeX EObltE6Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBFP-00000007F2M-2lTl; Wed, 12 Feb 2025 11:49:43 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq0-000000079iG-3jDA for linux-arm-kernel@bombadil.infradead.org; Wed, 12 Feb 2025 11:23:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Sender:Reply-To:Content-ID:Content-Description; bh=ggLp2N6nIh1N25l9qqVFGy2pp6+E0t6/i4rH9zHeAXo=; b=LYAryLDxz9ag/+zTcPSuPxECiw ogblM7bFiaMCOyLQ+cwfP0duHGM57k5jt5+x8vIHdhUOnVNGPdBiPrEi4L+qte4I2ic3741bNiwPM 95CuLmuCgykS4LA1sF4sH2OVlL3LJaIg4TIfIQqHAuNL+y3p9lo52qB3ltL7dbr0yWrdMZ0JZsiwi ujAevgh9ILF8S9wG5nynbvd8klSkb2Dsmhwp5vFxc7O0aXLEwFRLzwIFUinAZt19ISfnb+7MINZXC t248XY+m+h/3eYY0gV68ChcIpi7LyezpL4adGac0ZuJkOApVqJ5WsfRHyqwOnp64qxX6Nh/it+jtt XothSpAg==; Received: from mail-lj1-x236.google.com ([2a00:1450:4864:20::236]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApx-00000000no7-1Fk4 for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:27 +0000 Received: by mail-lj1-x236.google.com with SMTP id 38308e7fff4ca-30797730cbdso65111621fa.3 for ; Wed, 12 Feb 2025 03:23:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359403; x=1739964203; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=ggLp2N6nIh1N25l9qqVFGy2pp6+E0t6/i4rH9zHeAXo=; b=mmDn3NhDayuNm3Q5B3sSrvv/yS5ePPoTCEhoR6DDlrhs5fPNLXVvTid/t53J/ScFh3 V037b3K1yaqx5IOLCpHkdGi8+T8kMwVRxbV1ltrm91xPqp75XKOgO1irBGnhaGfayzTY /LNcS4t35wZfG8LSdJXFpebVKtWusYCjzLHS7iMPUx8WHqMrQLsyOXh/g+hfLUeQAAJO 5AMWIGBSjTdxfhCXxeLj8g5m3LGC7bm6TjJDZttoon4LssSp7bkv8Td4dWe2OOP+7QBQ KZzlzbMM51wVIYdtciaapq5HBMp+wXXyz3fwS3JblX5Xzk6d8cmvoaIbaulaaJMMKxvt XQxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359403; x=1739964203; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ggLp2N6nIh1N25l9qqVFGy2pp6+E0t6/i4rH9zHeAXo=; b=u2tBxGukKy0npamlsJN87mBBG57R1T/SfeEiqRWEkZulS8PRIaAGdEu2/Lbw3xMIxU sTBGMxtGXqgNTzSpTa0rmctukfJ+1+NaqCnvzuvYpcBGWtDWHyqFyFIEpJrOiKD5fyoH kCfU2W/MqQ6XFJqr+lqBi/FaLgpnqwiaaSgU3/wWBI/51b9akoEOUlu34/b/mU70Wu2o j8edNUcJ8nC823PyA0UMapk+NjnWJVRYfJdpvEVykBC1996vL+5CZU3VhCXGceOiLEpZ OiqxUwIgb//t/Ek/mP090rXcbm9tNpNJx7Bd3cpRWRRVSl4FRRgVWHfoZMfjxFpaFfOe rhNQ== X-Gm-Message-State: AOJu0YwHZi653YP4kpzEswOKTYlBkHyiLqU3XcCx3gdVrR3oRczv1L8h yELyZRZtZQMjmZWu4Xj/c3CzHD/2br6bJRUpRu1sBEu7nDGeZ+ax6GTuQ1Auwak= X-Gm-Gg: ASbGncs//uvQrRgl/aOu0eHBL3GO7UXLM/Kxt4c1eWBjl0ILXytdzqhbKOeqw2me+lC RIyRkIvoWCDseY6bEXdVlZgKC2bxRZai//SmKxfJrOf5wYz/2TVE/TvWMeaQw+Xhhpa4cA17Emu eHNB1ohX5h/DuheHVhWT2COk9i9KZ6EZSZoApNuBiQ6DWPgeXXsQPWq6HuWTyDxortd1fGZE6DV RF9DzZYH9VPoTauX8Y8/gH2l2rhVo/lGdpjmDXh6AeCinrUFWdrOjZNle4zASp3aeTcKobIK4yF iEED1GmOMIdGUAWu9RqGw5gOWA== X-Google-Smtp-Source: AGHT+IE++2ckcXtWyXX7JU0MwX+37DcqZEVj69OKEYC5Hko3s34PTk0ej3SG2in6uu2FQAzgMWO6uA== X-Received: by 2002:a05:651c:503:b0:308:f4cc:952e with SMTP id 38308e7fff4ca-309036b51c0mr12255951fa.11.1739359402652; Wed, 12 Feb 2025 03:23:22 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:22 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:10 +0100 Subject: [PATCH v4 16/31] ARM: entry: Drop argument to asm_irqentry macros MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-16-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_112325_528668_D0E362E5 X-CRM114-Status: GOOD ( 13.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org asm_irqentry_enter_from_user_mode and asm_irqentry_exit_to_user_mode have a "save" argument that will save and restore registers before the call to the C function. Now all invocations set this argument to 0 so drop the surplus code. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 8 ++++---- arch/arm/kernel/entry-common.S | 2 +- arch/arm/kernel/entry-header.S | 18 ++---------------- 3 files changed, 7 insertions(+), 21 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 6edf362ab1e1035dafebf6fb7c55db71462c1eae..460aa92c3c1f50de905550acf363c58f509bfe0a 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -422,7 +422,7 @@ ENDPROC(__fiq_abt) .align 5 __dabt_usr: usr_entry uaccess=0 - asm_irqentry_enter_from_user_mode save = 0 + asm_irqentry_enter_from_user_mode kuser_cmpxchg_check mov r2, sp dabt_helper @@ -433,7 +433,7 @@ ENDPROC(__dabt_usr) .align 5 __irq_usr: usr_entry - asm_irqentry_enter_from_user_mode save = 0 + asm_irqentry_enter_from_user_mode kuser_cmpxchg_check irq_handler from_user=1 get_thread_info tsk @@ -447,7 +447,7 @@ ENDPROC(__irq_usr) .align 5 __und_usr: usr_entry uaccess=0 - asm_irqentry_enter_from_user_mode save = 0 + asm_irqentry_enter_from_user_mode @ IRQs must be enabled before attempting to read the instruction from @ user space since that could cause a page/translation fault if the @@ -472,7 +472,7 @@ ENDPROC(__und_usr) .align 5 __pabt_usr: usr_entry - asm_irqentry_enter_from_user_mode save = 0 + asm_irqentry_enter_from_user_mode mov r2, sp @ regs pabt_helper UNWIND(.fnend ) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 14b2495cae3c2f95b0dfecd849b4e16ec143dbe9..df564388905ee019cd5553f8b37e678da59e3222 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -109,7 +109,7 @@ ENTRY(ret_to_user_from_irq) movs r1, r1, lsl #16 bne slow_work_pending no_work_pending: - asm_irqentry_exit_to_user_mode save = 0 + asm_irqentry_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index fb5bb019199b2871e29e306a29bea8fdf47dd7f3..50c0b55adc7421e7be123c9d00f94b1ebb93ff9e 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -365,28 +365,14 @@ ALT_UP_B(.L1_\@) * Context tracking and other mode transitions. Used to instrument transitions * between user and kernel mode. */ - .macro asm_irqentry_enter_from_user_mode, save = 1 - .if \save - stmdb sp!, {r0-r3, ip, lr} + .macro asm_irqentry_enter_from_user_mode mov r0, sp @ regs bl irqentry_enter_from_user_mode - ldmia sp!, {r0-r3, ip, lr} - .else - mov r0, sp @ regs - bl irqentry_enter_from_user_mode - .endif .endm - .macro asm_irqentry_exit_to_user_mode, save = 1 - .if \save - stmdb sp!, {r0-r3, ip, lr} + .macro asm_irqentry_exit_to_user_mode mov r0, sp @ regs bl irqentry_exit_to_user_mode - ldmia sp!, {r0-r3, ip, lr} - .else - mov r0, sp @ regs - bl irqentry_exit_to_user_mode - .endif .endm /* From patchwork Wed Feb 12 11:23:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 78248C02198 for ; Wed, 12 Feb 2025 11:51:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=0KMIzffvEsBK43v+gCyDeKdrTwv9xJqaffLfxp2Y9ec=; b=zO4fm8IFpzt43bjnAomkz2nchP k96EHkWdmn7vtHDalqBb1Zh2tXSXuEVkYOJbexKvTdakqajI9ywFNzIEpiiQAmBuT5628NuLrmA1W VWNde0AxukwcjC9w7am7YI/9FofJB4E0/x1O948K/SUy2XuHHPiiHh+NghpLjcuPeaw64Yy1MCW8Y IBzTClhSLYl8XU6CyWIl6OMU0jRTPWl7p+8p1duJDWE1a3aOe2y9X9Sqi8fuWgxu49zYAOcOTgF7g k0GHJTyuJBUQ6La37z2ky4Ohzsx86BsORPcNwWVJ1oJSy9L8ZIrjrqouMExYr/Jn0BPFGSTNXCyLb YAyXu9Zg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBGo-00000007Fao-1rq7; Wed, 12 Feb 2025 11:51:10 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq1-000000079iJ-30xh for linux-arm-kernel@bombadil.infradead.org; Wed, 12 Feb 2025 11:23:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Sender:Reply-To:Content-ID:Content-Description; bh=0KMIzffvEsBK43v+gCyDeKdrTwv9xJqaffLfxp2Y9ec=; b=AWZa61c2pCkElSejbPf4x4E0yy P+G4ZFEYhvfQy5PWknf6laJB1zMjsvFZPHvome/vi++p0XtV23RoA5E7V0uVIM/vDjBWaT3DzTrf/ qtdnGpdse3c9x2+om2o6b8ZUCNTmFA/Dv4pP3AKvhmQ6+3efaG1eKRRb+vibydAqfABaXi+c4KOcl 0NBnlJ5uuDD+mCb8DXM45YbrlwkqRDPe3jvlu++8FCg2pHoj4SafADpryRUhl4Q2i3p75Wcr+aiXR vhecqcMQ1xGt4VSCPwdR0O11Xb5JizAIFyVtfsYTzXuXs/FQPQL+ODqQQa1TwgjfnwxFskWCJyHTs cx2npBXg==; Received: from mail-lj1-x232.google.com ([2a00:1450:4864:20::232]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApy-00000000noe-1EQV for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:28 +0000 Received: by mail-lj1-x232.google.com with SMTP id 38308e7fff4ca-307d1ab59c6so61919961fa.1 for ; Wed, 12 Feb 2025 03:23:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359404; x=1739964204; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=0KMIzffvEsBK43v+gCyDeKdrTwv9xJqaffLfxp2Y9ec=; b=Dhi5Ao3mfjGVcVPewoI0Oj61ssGo/ZBQFdlA5Fh7n7nymJi4qpp9+yyoddtfu89dm4 vePXTYxiCB15iDILTqa9CaXfu8X02f8k3EUfiFirnyZTsh8xGpuXgcDrnCDTETiG47PA 9u7C8qtA3VdTp6VG9/fGBRCASEuAntNwJ2IjZMt/s0poJbi/m2oiR4S4bPuAhce14YB7 +V39TpLhJHsN+OK83QDsekzuiiA6CtinS/ABTJs9VfvwH8a0L5lMHlvcqWK+khXi8xt9 TfFG65tPwkm1QdYNV7gro83AM/VwydIT2xYywHDVjuOwVlsus42bsVXQskTHUFKhnSns E94A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359404; x=1739964204; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0KMIzffvEsBK43v+gCyDeKdrTwv9xJqaffLfxp2Y9ec=; b=sE2GZfhl4Rd/U0wx37nPu/6JSQ2d+KU84P7PaiIGQhA9U1Xia7eMrMNE9Y7ETvxJOD gpjJjTjEXS/vtIEj5ohwVDystRs51RvkesgKKsnmn9WyCeGR0GLMWXs3wcbAy+Kt46he 1ZltdRvJWBDPsVMujf9eufX4fqRmgs1KmMeMjw41CSSmWtA+kDG1XsaalQoQfTtDQ/kA KHVs3xf4lKgyS0BS63t9tnSxMTCS6a/TwoEAdc27BOqTsaiXVp0mcxZWuGJFQtaaEMeR GwXBOCsWs+kaD8dRDvsyazy38Y+BJOXwzs/wQixz652Jv4BW7V4HQc18wE6bBoGlxKKx kFPQ== X-Gm-Message-State: AOJu0YyN+ff4kpZrrQ93S6/Z/L+0GfrPq1HT6RCf9cMZ8dxOnXTisTM9 cRMG/EbujYWQdJBazlhuibMiMg2t4LBSHYxvwf+Yq/sEn49slVidsgqtbbz2nwI= X-Gm-Gg: ASbGncs39A7yxTJeI/f5RhbIEy3eErnVRUP+IqVpSSCSjL6iHj+NRgc/LqMWmjfB+iy Z8mCtXPPcth+l/plNoqBiTQFCWd7yTh3PDcV0LVMdnllcdRpnamiY/B5VLmgGs0DznZgvIip4S2 k4G+jnQOt+7/UOfNktORtcyw2Hyq3W/G08b3IIjrVVE3iWwUeisFk2+TNGfa0l8i+efCqbXEimZ hfONt5GfzcTAIGnwYUHw6awZH7+UI4BOjIeU+EbSxaYPcYjKxIvQhBRWv77Bm+yDQjwJ2AvSCt5 yz3ckuD7MqyrgtfmEyVmiYgQdQ== X-Google-Smtp-Source: AGHT+IF3FbVuyAVSXE6FSKtFUxtJ6fX5M5a69TIwSs1mwLT4jbjl8bopzEgPtqY5QadYENn8IcrBjg== X-Received: by 2002:a2e:a916:0:b0:308:eb05:9d3e with SMTP id 38308e7fff4ca-309036dc200mr10746881fa.23.1739359403613; Wed, 12 Feb 2025 03:23:23 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:23 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:11 +0100 Subject: [PATCH v4 17/31] ARM: entry: Implement syscall_exit_to_user_mode() MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-17-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_112326_536666_503E0288 X-CRM114-Status: GOOD ( 15.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Mirroring syscall_enter_from_user_mode() we create the sibling call syscall_exit_to_user_mode() and start to move some stuff over from assembly, first the rseq call and the interrupt disablement. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 1 + arch/arm/include/asm/signal.h | 1 - arch/arm/kernel/entry-common.S | 10 ++-------- arch/arm/kernel/entry.c | 8 ++++++++ arch/arm/kernel/signal.c | 7 ------- 5 files changed, 11 insertions(+), 16 deletions(-) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index e259b074caef75c7f777b18199623f07bebee5b4..8b8cc9a1eefbd00dd75d4e344576fc184a413768 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -9,6 +9,7 @@ struct pt_regs; * to generic entry once they are semantically equivalent. */ long syscall_enter_from_user_mode(struct pt_regs *regs, long); +void syscall_exit_to_user_mode(struct pt_regs *regs); void irqentry_enter_from_user_mode(struct pt_regs *regs); void irqentry_exit_to_user_mode(struct pt_regs *regs); diff --git a/arch/arm/include/asm/signal.h b/arch/arm/include/asm/signal.h index 7acccc96840c8a17744cc3f2894f19d142aba4fd..bb753cd3e29fc54186543c48b2aee7b221301a37 100644 --- a/arch/arm/include/asm/signal.h +++ b/arch/arm/include/asm/signal.h @@ -23,7 +23,6 @@ typedef struct { #include -void do_rseq_syscall(struct pt_regs *regs); void do_work_pending(struct pt_regs *regs, unsigned int thread_flags); #endif diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index df564388905ee019cd5553f8b37e678da59e3222..3559e0a9f5e079f1af91392bf000d39f9b45906c 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -65,12 +65,9 @@ ret_fast_syscall: UNWIND(.fnstart ) UNWIND(.cantunwind ) add sp, sp, #(S_R0 + S_OFF) -#if IS_ENABLED(CONFIG_DEBUG_RSEQ) /* do_rseq_syscall needs interrupts enabled. */ mov r0, sp @ 'regs' - bl do_rseq_syscall -#endif - disable_irq_notrace @ disable interrupts + bl syscall_exit_to_user_mode ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing movs r1, r1, lsl #16 beq no_work_pending @@ -97,13 +94,10 @@ ENDPROC(ret_fast_syscall) * do_work_pending() will update this state if necessary. */ ENTRY(ret_to_user) -#if IS_ENABLED(CONFIG_DEBUG_RSEQ) /* do_rseq_syscall needs interrupts enabled. */ enable_irq_notrace @ enable interrupts mov r0, sp @ 'regs' - bl do_rseq_syscall -#endif - disable_irq_notrace @ disable interrupts + bl syscall_exit_to_user_mode ENTRY(ret_to_user_from_irq) ldr r1, [tsk, #TI_FLAGS] movs r1, r1, lsl #16 diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 1973947c7ad753fccd694b3ef334fba1326f58b6..47233804733752aef0f2fec6f54badeab18e98d0 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -1,7 +1,9 @@ // SPDX-License-Identifier: GPL-2.0 #include +#include #include #include +#include long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) { @@ -14,6 +16,12 @@ long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) return syscall; } +void syscall_exit_to_user_mode(struct pt_regs *regs) +{ + rseq_syscall(regs); + local_irq_disable(); +} + noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) { trace_hardirqs_off(); diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index 7b1a16e86b236575efdc29cda9b751e8e2a3f64e..03b20637a2e1ca6f327223c9132915a97a9c4697 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -659,13 +659,6 @@ struct page *get_signal_page(void) return page; } -#ifdef CONFIG_DEBUG_RSEQ -asmlinkage void do_rseq_syscall(struct pt_regs *regs) -{ - rseq_syscall(regs); -} -#endif - /* * Compile-time assertions for siginfo_t offsets. Check NSIG* as well, as * changes likely come with new fields that should be added below. From patchwork Wed Feb 12 11:23:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971813 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AC456C02198 for ; Wed, 12 Feb 2025 11:48:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=kElQgdyrfkPgzICs9KEd5Is5KMXf1IbQ9ph46r/iv/o=; b=fyhZFRP4sTR/SU43p2eiq9vrO1 qWVga170Ydpe2YViz8EpIS7viTg4Qvo+IBv8KdV1I7rkAkTJfMdTbxVTI4z23cq9PkW7pVp3ql8xW auWq2RnWiMQkGUlkMwgLMXNBVkyBLR9gU+une1I6FV+BLFOW+cIMQUhqgjgBlxtHmEEJKiFSUyuxr USoIGDKEfuu9uehiq5QT7vjqjigKnLotsuYaWy++sqr3frZxlnV+kr20Gl0yju2QzzsRy/fqMvcPs 61BJWV8rSqDjNHaxzfqB72BUSKji7nwnXsph8SzJTyWUzgPDYjlsHu4UIOVNw+Hl6irWzzbI0BHGY yDYi1+qg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBE1-00000007Elr-46QO; Wed, 12 Feb 2025 11:48:17 +0000 Received: from mail-lj1-f179.google.com ([209.85.208.179]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApy-000000079hC-3UTR for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:28 +0000 Received: by mail-lj1-f179.google.com with SMTP id 38308e7fff4ca-308ee953553so33249531fa.0 for ; Wed, 12 Feb 2025 03:23:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359405; x=1739964205; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=kElQgdyrfkPgzICs9KEd5Is5KMXf1IbQ9ph46r/iv/o=; b=HHIrAnVsGi2u6oJVUhesbLtoq0RGsrt7Wz4uxIHb4qhQ4a4O0RUTbEidyIN6mzlIT6 So3zuHKPJ2BhL6peA59AqHAIbr6Uf5u/IhSP18WCxYztnpxinDXgO44lH/ArkiBOKi4i MoO9MpiXlb3paJNeTu4k8ad3PqGsSyib0OWKGLgiw5DkB1JR1VvAYBLXQ53cUeuSU4vs MkfaaNisTSZvuJc700OCqWLCgAZaIJjw6TqCdPkVpwIftwht/eDXnGM7gUeiTVzlB3te lCd75ymemfvN8PyapHOwEPNkJdqr8xtviO6EYcAO4F1JtaxRkRMirxE0U6H/P81tPMfR bjfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359405; x=1739964205; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kElQgdyrfkPgzICs9KEd5Is5KMXf1IbQ9ph46r/iv/o=; b=ZYGPx/WQT6QewcD5muJcvdXQYsoY/wWaoQWIV4oM0C2ysutMwwg6vRspNajHffdqOB wlIJur0lIfj66aBTSf1Fj9LZ2qMAv+npUATMeJS+7+3DZk0BJRM+5bXUXzHOckxw1Sg/ OUomn7R0DeO2H+yqId7nZVI0vkIy4AfJW7BH/lHcNijZlZmlbq+2TIjUh0lQw3w2zlhu L6GKHcquK/z7hJ+Ik7eeursnx965O3j01eeEITKh6OOK9d5RLwGczVu6sHaQCHBMAcPI 7/7p/PMipJw+YHy4Q3E2fMwHI3a3gJxKrUEI15zDu79SCigA+k4r3sXbPrbl6HUjuKFT NIoA== X-Gm-Message-State: AOJu0Yx5XZv5gJKDdve1MKGtphawxIKk7sUE+wcCzfzcb+hR8IFecF8f Cs13Z5XVMhiIerT+HcNZgdONKd190Rw6bEKk1aVCe7NB0dHIGAPPGGC76eQ0doI= X-Gm-Gg: ASbGnctsRTYOm10O/zRpthazuaNKTTFCG4s0BmNliXIZN429dHu9ozJmMPST4KLDhbM Cc0FKhmvxH7t1mnevN3Na4fJBhXOKRLBBG+xGE6SVMqHtDMeqL+7rwed2hIAkws7On68fihnaWy 1DgKab/kitYSCBZv1BkvQa0hXC9RnJb7364QJBBrsNyAq0kIyYViuZim/YAVdGH9WdtpiIbyyBG 8W89Lnh9jrAta7rNqIj+DlWRHHRcK9kSnfv0etnfOZ3ZEmhmf6gpPhNma8ZDTZmnRLnCMDyPO7w 1YmzpAloc1e4NJB9pam4A5APTg== X-Google-Smtp-Source: AGHT+IE/lLojbQa5ylZyujHSAywjLKA0KHvd1gHdqbGhs1YJlJx+LEqW8/3kRM9xKQQzC/5ig+Neww== X-Received: by 2002:a05:651c:1a0a:b0:308:e587:ca79 with SMTP id 38308e7fff4ca-309037c1998mr10011751fa.11.1739359404686; Wed, 12 Feb 2025 03:23:24 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:24 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:12 +0100 Subject: [PATCH v4 18/31] ARM: entry: Drop the superfast ret_fast_syscall MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-18-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032326_876107_1B7CFBFA X-CRM114-Status: GOOD ( 14.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When not using any irqflags tracing, context tracking or rseq debugging, the ARM kernel uses a "superfast" ret_fast_syscall with a very few assembly instructions. With generic entry, such local optimizations are not really possible, we need to call C unconditionally and take the (hopefully small) overhead. With generic entry in C, most of the tracing (etc) calls are replaced by stubs and optimized out by the compiler. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 29 +---------------------------- 1 file changed, 1 insertion(+), 28 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 3559e0a9f5e079f1af91392bf000d39f9b45906c..dddbd6ba1eb9fcbc2c0d81f24e1a9b3231558d7f 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -30,34 +30,8 @@ saved_pc .req lr .section .entry.text,"ax",%progbits .align 5 -#if !(IS_ENABLED(CONFIG_TRACE_IRQFLAGS) || IS_ENABLED(CONFIG_CONTEXT_TRACKING_USER) || \ - IS_ENABLED(CONFIG_DEBUG_RSEQ)) /* - * This is the fast syscall return path. We do as little as possible here, - * such as avoiding writing r0 to the stack. We only use this path if we - * have tracing, context tracking and rseq debug disabled - the overheads - * from those features make this path too inefficient. - */ -ret_fast_syscall: - UNWIND(.fnstart ) - UNWIND(.cantunwind ) - disable_irq_notrace @ disable interrupts - ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing - movs r1, r1, lsl #16 - bne fast_work_pending - - restore_user_regs fast = 0, offset = S_OFF - UNWIND(.fnend ) -ENDPROC(ret_fast_syscall) - - /* Ok, we need to do extra processing, enter the slow path. */ -fast_work_pending: - add sp, sp, #(S_R0 + S_OFF) - /* fall through to work_pending */ -#else -/* - * The "replacement" ret_fast_syscall for when tracing, context tracking, - * or rseq debug is enabled. As we will need to call out to some C functions, + * As we will need to call out to some C functions, * we save r0 first to avoid needing to save registers around each C function * call. */ @@ -75,7 +49,6 @@ ret_fast_syscall: ENDPROC(ret_fast_syscall) /* Slower path - fall through to work_pending */ -#endif tst r1, #_TIF_SYSCALL_WORK beq slow_work_pending From patchwork Wed Feb 12 11:23:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971817 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 253C5C02198 for ; Wed, 12 Feb 2025 11:54:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=SK4XvykyJ6Zn+vIxSK9vSx/zpAi4l4ILYAtxYCxqygE=; b=g7/P4g1QfOKzrpU++fXNbSyhy2 5R4TVStMFsHkcpODO27PZ+w0mjd6/HvIOBPyiouU9Ht2blWQpXjvO95/SYAQ/BWvqdFNj0JsGr2co WdnKaAiIzzFU0CsG5QiQZUCSFaUsM5Nfs+z311uFFHuQshvrBuewpsWnBV7wtPPWQCVlNi9A8xtWZ 4XtpnozBwcxLc6ZiU2KH0/2YHCPGXbSkylZMRg/tPBPoXYl2dO4H4DU9ePwHkUPzus6qijqteneD9 SmpSx2A755DuiUTxxiCKGBj+RkukofD1xg50my1YN6jhmlS5Bfc+NpDJGXV6cjAkDxhLsfv/boXZW fd6eBgrA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBJa-00000007G7U-3wSS; Wed, 12 Feb 2025 11:54:02 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq2-000000079iu-148f for linux-arm-kernel@bombadil.infradead.org; Wed, 12 Feb 2025 11:23:30 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Sender:Reply-To:Content-ID:Content-Description; bh=SK4XvykyJ6Zn+vIxSK9vSx/zpAi4l4ILYAtxYCxqygE=; b=AaYDmye1PBa+Zo2pJhpPuzEoIN jKe6ytlZBhoLESNjXkWfW1joJhGWqnEI2mWHrpkThZh2lRselb+gp083HwNXwOhWu24OgxgYMo9EU m2fg429iW5mMzSj0kYa04HE5Agw/6tZ8L0WCZU02ItuSlZ9Ph37Oqg3m1WX7YVoWCp9mM75mdwiWV Ufl2QtiL4r5iuO3q8EhEN44rztKhBmwmJanC63+kLyZrc6SKadDukJo3j11sqZYKCZ7xunjt1u7dd UnkfHgD4/bKvhcNpP21qGYXDf+zcXum1xQOHAclUAWbWRWMEbFX/OvI4PXI+fbNuaBk3/BaLQRiSX /P5BiMyw==; Received: from mail-lj1-x236.google.com ([2a00:1450:4864:20::236]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiApz-00000000npC-1HQJ for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:29 +0000 Received: by mail-lj1-x236.google.com with SMTP id 38308e7fff4ca-30761be8fa7so60302561fa.2 for ; Wed, 12 Feb 2025 03:23:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359406; x=1739964206; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=SK4XvykyJ6Zn+vIxSK9vSx/zpAi4l4ILYAtxYCxqygE=; b=WblPLfXJezDuAeKQHnLXddAkSBbDlndbmbnPxDpR61AkCkG9Lv+cbiHzkG1bauPEJH 9e//NZhDinAcmp8cNDnUlXDMWjD0nYQz9QvKBeRBMZdXmSrhvykNlApwLVU5QcHhOuXN OFDx372fJacOzPW495nA0ZD+bH8MD4ZOUOI+Gq8lULGywfi5WpZulNf+bkVsnXafFca8 3y2FS+vngbRL1DPL9KY+OV9ufDX/3Gcz9Dm8lrFrqIZ+RKCGTJ1dBFIrFPvslpp8gnbe HJX81rlkIxjaGqQDJZ7XNBNXO0zswBDtBKdeX1YakC3PHg2COXT57Dpvkw7cI7uC8ze6 4yeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359406; x=1739964206; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SK4XvykyJ6Zn+vIxSK9vSx/zpAi4l4ILYAtxYCxqygE=; b=A6usirTD7CopPsmWsRuZkMXRTymXrxJSduHPeKbv+3Dyz9fdzu6Qf0Pz36VC0W/xP9 NI7ufzmucnORNLCrqqWBebEkCkTeBUQHlGDaY96PSli2sKsHzOMyfQHAwWK45cScc16e XaLnJc/YPXAEuSbiN46xa7stmd3BLvHSZTz8IDzkdmKoD/hSTawieIOXjgDGt4Bnb6DU RhMmgwOhD6QYyhvXBaQG0UJdlT47foTn8YqNkwvjRwx1ik/IAxmZLiUVxhsx4T9LKWq8 qMCYqMGhDmpFiAFjDp9czDfarG1xi8OognchPcD5QIG45uc8n1pJb4TUSJmfH1/WaQSH /xxA== X-Gm-Message-State: AOJu0YyB5pyrJtv9mP2L2qUMvFpwgIs8ojbxge9iyNQl8BJHro1assJ5 EuhMytBG1eqmRUWDL898fE1PUCU58T9+q2sf4mbk2fffUtSUw+N112X3QXqumhM= X-Gm-Gg: ASbGnctXstwlBfdpMVsPZyhgstYzfEPXahnV1VleCRKZeUN3amUAAj7r0bnGK0O3ZvZ MamNCj968Sqt97QpH6sMWQe5HEDcWBaOCoAgBC2SxxsRD/WNQ7S+1atwo+CvAfYEJ/mRbuAvKa9 gcjFIuPSl7zBNppUR9qVp5i8fT/AIbwW1cDcUuX4aukCpr6g8HLF7CcjY/OyKSlKsRcJQ+VaXFb wLdOTRj9MyC8KZXqWokNGTjYe00DFxWxYRhLgsX7Mk6jC7zOsG03WA+WnKAes6ldnGMUQfM87i4 Dviv2iNoXpMW7TSdM++B0y948Q== X-Google-Smtp-Source: AGHT+IGZGX3RxfrWPejP/YO09nPgCO79pvl584D11RVURMnv/+ySzxg9C6vPdPm/xocAoi8IjTVcCQ== X-Received: by 2002:a2e:be87:0:b0:308:f0c9:c4cf with SMTP id 38308e7fff4ca-309036f2cd8mr10642481fa.33.1739359405778; Wed, 12 Feb 2025 03:23:25 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:25 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:13 +0100 Subject: [PATCH v4 19/31] ARM: entry: Remove fast and offset register restore MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-19-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_112327_629250_C811998D X-CRM114-Status: GOOD ( 13.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now all remaining invocations of restore_user_regs has fast = 0 and offset = 0 set, so drop these parameters to the macro. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 2 +- arch/arm/kernel/entry-common.S | 2 +- arch/arm/kernel/entry-header.S | 29 +++++++++-------------------- 3 files changed, 11 insertions(+), 22 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 460aa92c3c1f50de905550acf363c58f509bfe0a..3cd5a80b47758867d1db6c3971030370bca1551a 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -497,7 +497,7 @@ __fiq_usr: mov r0, sp @ struct pt_regs *regs bl handle_fiq_as_nmi get_thread_info tsk - restore_user_regs fast = 0, offset = 0 + restore_user_regs UNWIND(.fnend ) ENDPROC(__fiq_usr) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index dddbd6ba1eb9fcbc2c0d81f24e1a9b3231558d7f..d27e34d7e97a079d62e7b778cf7cf97c302e4763 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -81,7 +81,7 @@ no_work_pending: #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack #endif - restore_user_regs fast = 0, offset = 0 + restore_user_regs ENDPROC(ret_to_user_from_irq) ENDPROC(ret_to_user) diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 50c0b55adc7421e7be123c9d00f94b1ebb93ff9e..51c7f93c1ded1db56f173c7aaed8094ef011ff5a 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -290,7 +290,7 @@ .endm - .macro restore_user_regs, fast = 0, offset = 0 + .macro restore_user_regs #if defined(CONFIG_CPU_32v6K) && \ (!defined(CONFIG_CPU_V6) || defined(CONFIG_SMP)) #ifdef CONFIG_CPU_V6 @@ -309,8 +309,8 @@ ALT_UP_B(.L1_\@) #ifndef CONFIG_THUMB2_KERNEL @ ARM mode restore mov r2, sp - ldr r1, [r2, #\offset + S_PSR] @ get calling cpsr - ldr lr, [r2, #\offset + S_PC]! @ get pc + ldr r1, [r2, S_PSR] @ get calling cpsr + ldr lr, [r2, S_PC]! @ get pc tst r1, #PSR_I_BIT | 0x0f bne 1f msr spsr_cxsf, r1 @ save in spsr_svc @@ -318,31 +318,24 @@ ALT_UP_B(.L1_\@) @ We must avoid clrex due to Cortex-A15 erratum #830321 strex r1, r2, [r2] @ clear the exclusive monitor #endif - .if \fast - ldmdb r2, {r1 - lr}^ @ get calling r1 - lr - .else ldmdb r2, {r0 - lr}^ @ get calling r0 - lr - .endif mov r0, r0 @ ARMv5T and earlier require a nop @ after ldm {}^ - add sp, sp, #\offset + PT_REGS_SIZE + add sp, sp, #PT_REGS_SIZE movs pc, lr @ return & move spsr_svc into cpsr 1: bug "Returning to usermode but unexpected PSR bits set?", \@ #elif defined(CONFIG_CPU_V7M) @ V7M restore. @ Note that we don't need to do clrex here as clearing the local @ monitor is part of the exception entry and exit sequence. - .if \offset - add sp, #\offset - .endif - v7m_exception_slow_exit ret_r0 = \fast + v7m_exception_slow_exit ret_r0 = 0 #else @ Thumb mode restore mov r2, sp - load_user_sp_lr r2, r3, \offset + S_SP @ calling sp, lr - ldr r1, [sp, #\offset + S_PSR] @ get calling cpsr - ldr lr, [sp, #\offset + S_PC] @ get pc - add sp, sp, #\offset + S_SP + load_user_sp_lr r2, r3, S_SP @ calling sp, lr + ldr r1, [sp, S_PSR] @ get calling cpsr + ldr lr, [sp, S_PC] @ get pc + add sp, sp, S_SP tst r1, #PSR_I_BIT | 0x0f bne 1f msr spsr_cxsf, r1 @ save in spsr_svc @@ -350,11 +343,7 @@ ALT_UP_B(.L1_\@) @ We must avoid clrex due to Cortex-A15 erratum #830321 strex r1, r2, [sp] @ clear the exclusive monitor - .if \fast - ldmdb sp, {r1 - r12} @ get calling r1 - r12 - .else ldmdb sp, {r0 - r12} @ get calling r0 - r12 - .endif add sp, sp, #PT_REGS_SIZE - S_SP movs pc, lr @ return & move spsr_svc into cpsr 1: bug "Returning to usermode but unexpected PSR bits set?", \@ From patchwork Wed Feb 12 11:23:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971816 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 300D3C02198 for ; Wed, 12 Feb 2025 11:52:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=hLPL0cxSmhQjzTuSazqL5ULZpLc4BXlPwpKFYiv7qfk=; b=2A/nKa8vyebVqpYRzK9l/nFriI BqVeGfROve0YhGSJNWuX0LAHSUeGi7GANb0S7XMazUS8eEOR55CutBgnHZ0ISf8haa3czinVhyTiV VvWy2quw6eMbNbUZUvRGqr+N84wWaMboLNYloOiAN20yZwWFHLWmHwzGTiR/D631mh+EOuycFAE/t aJWZJcbBuuVuV/JmPOjE075LNkH7SJqINak70QPyB9pX1AH+Vv/I0R0IRFOLTsUUtoZhSHg9ruCXa 6o0RwFkDuXLgLVa1vqnwmn+xZBJ5xSyRTw12Y+5lZesWV7CSJiJY4LAPDWhwohQdGJDzAFVAbcEQJ F/Eq/Umg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBIC-00000007Fuv-0eL2; Wed, 12 Feb 2025 11:52:36 +0000 Received: from mail-lj1-f182.google.com ([209.85.208.182]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq0-000000079iA-42GF for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:30 +0000 Received: by mail-lj1-f182.google.com with SMTP id 38308e7fff4ca-308f141518cso31075111fa.2 for ; Wed, 12 Feb 2025 03:23:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359407; x=1739964207; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=hLPL0cxSmhQjzTuSazqL5ULZpLc4BXlPwpKFYiv7qfk=; b=HBgBQ2EFvoJ+HkjmGM09CvJAYiEIr55mirwt33914rQX+aDNG2dg+86CSs0eAQ+tiF G0AE83RUEXsAvDRtBmStXHKj4VpkzM5VSjJLrXgaaxsyleeIMMR4VKzktbbiVrtuZ9oe AAgvNbaaK0llEwtCS1BYwK2ZYuLWB4I292OWYsT9RuewVhsrh9bdwtgTUkKPgk0HWAiX uBigA5ZYhgJz5EdoHGFEcZEPwVNWdKdeujpm2apL4vsZygfNbI57Sh+ZV7Zo1JPme1Qq vajGsIpL2CUxKZRFiODMAZ8pCdQX33BwefsJaSsev6/snQSkkSW11OX3AfHIq5tf5YLt lM6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359407; x=1739964207; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hLPL0cxSmhQjzTuSazqL5ULZpLc4BXlPwpKFYiv7qfk=; b=skpwdSFY3W8xi/hnRfYAg+46phtM+9IR+JmQfuPLV7vwCcKhZV7qArb+1MF3Sjxc59 5LF0NgaKQs+Nvpsq1M9Zi928MXV1ZSiXHofhJuSI0YnN7u9jWuHjVg1nmRkLQAZGPSZT 5VkYUuHm9WJME8anmWQnk4X6pmyzyzSWBtOpfYsY6xnypWmswBKw9lCl+hyNYPxROmCc kbLrNb3MLsBn2QfMPAiI2V2oc0EumY7acm4P1TG9NgOQQa9SfM5dAcbHwZIO62HIkxLw GbYcmOz8GoZpk286L4+qBnC6DeAecsonX6en+ZVYMxMa/hLcXicNyUm/IO46d0tV49fL 0VxQ== X-Gm-Message-State: AOJu0YxMrtBluyVUjr83R0YRc15pfakmwBB//FiQqsv+CuiuRJIwYQHz 4uugUbTZGB7m0z2feMgB5g4BAfbm6FJ4oGxPCEndghy010SZAHMGAhPhwz+U7kg= X-Gm-Gg: ASbGncsbAKklcWxtB6NNd49FMVm7N8nvrpMjXmkaKN8trZZ3d/bLnNVdoOIEvZatVNH Sz7EgPdWG1/BB35J7UGjqexLr8Eha2CjsiguxcCW0bC2/SHaWMFGhDAc/0tCTOiig30HUh0+OyK EASM95+v0cpZD2ljUsZ1w+bxcDg1HTVdC56LXCylYYXVjKSxbYr2wnojBukZRhWUmceg2F14y3j e1xbkI8bPMlDh0KEaId+WjQCthEl32p4Mgsuhhy1H+W36rh7gjMbWLplC9rWjIG/v0JZLncf7y7 xJ+g9RrWI0WRBpkzwWqRaG5kzA== X-Google-Smtp-Source: AGHT+IHQpMybVh2UGY3q+b/5gNXwdYp/Tf5GyuWxuwv3bDkab39Fdfp6CaiE28gYUS4P0fFCrwjZ/A== X-Received: by 2002:a05:651c:1991:b0:2ff:c3a2:f408 with SMTP id 38308e7fff4ca-30904fe65b4mr8430881fa.12.1739359406796; Wed, 12 Feb 2025 03:23:26 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:26 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:14 +0100 Subject: [PATCH v4 20/31] ARM: entry: Untangle ret_fast_syscall/to_user MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-20-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032329_041096_B67741C4 X-CRM114-Status: GOOD ( 12.10 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The return to userspace is a bit hard to follow. To prepare the ground for further refactorings we rewrite the functions to ret_fast_syscall and ret_to_user are mostly stand-alone symbols. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index d27e34d7e97a079d62e7b778cf7cf97c302e4763..ece9215717143427de7572a18a6dbe620a9193e9 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -44,20 +44,23 @@ ret_fast_syscall: bl syscall_exit_to_user_mode ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing movs r1, r1, lsl #16 - beq no_work_pending - UNWIND(.fnend ) -ENDPROC(ret_fast_syscall) - - /* Slower path - fall through to work_pending */ + beq 2f tst r1, #_TIF_SYSCALL_WORK - beq slow_work_pending + beq 1f + b ret_to_user -slow_work_pending: - mov r0, sp @ 'regs' +1: mov r0, sp @ 'regs' bl do_work_pending - b no_work_pending + +2: asm_irqentry_exit_to_user_mode + +#ifdef CONFIG_GCC_PLUGIN_STACKLEAK + bl stackleak_erase_on_task_stack +#endif + restore_user_regs + UNWIND(.fnend ) ENDPROC(ret_fast_syscall) /* @@ -74,7 +77,9 @@ ENTRY(ret_to_user) ENTRY(ret_to_user_from_irq) ldr r1, [tsk, #TI_FLAGS] movs r1, r1, lsl #16 - bne slow_work_pending + beq no_work_pending + mov r0, sp @ 'regs' + bl do_work_pending no_work_pending: asm_irqentry_exit_to_user_mode From patchwork Wed Feb 12 11:23:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971819 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7C9D3C02198 for ; Wed, 12 Feb 2025 11:57:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=GFYj8M8sWUilipZ79moNvoJmNP0x1CtymAnRzXm2Cm0=; b=Lnv48tOKcGDatKhCfRAKDkoKwT /+YFjvqoDNaZC60aAM1fmJ45A9Aj0Cjed6hPRkY0twaGLZf+TKSpQzCLpmndsai7DWABGo/B29bcC VVb9Fp72aaywwgo+ky0E52cXL8WdnPnTuQz9699u/X2uTrjC23xv+Wwj8fgF1xN+uegRCAjo3bZ9V IA7/BK4GSEnKyzBHIPPxTz9t5DBMyBx5NkZcZlMsozfTWptmSAzt8C+oa6CezwxHQxINEcm7/A7IC zM7sCCi12TS4NQnmcJjNY8W28pWu/dMjgw2SjDXO95k5qZu89JVk0188gVmlAxWSBgwsQcFYRULe4 17/0pmEQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBMN-00000007GQX-1PjF; Wed, 12 Feb 2025 11:56:55 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq5-000000079ji-13s3 for linux-arm-kernel@bombadil.infradead.org; Wed, 12 Feb 2025 11:23:33 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Sender:Reply-To:Content-ID:Content-Description; bh=GFYj8M8sWUilipZ79moNvoJmNP0x1CtymAnRzXm2Cm0=; b=HVOSwTqb5GpvXYRPQyEAnTmbCV o1YAVkaz1uWRMvc2O/2+8gd4L94YdnHWY4U9KUqf8yyNG00fpWU/lg5heICwYgxy9w1TqQT5jVApc oOcMW8mze4MOSoYVQ3tWYqXQbRvrjSyQ4REPzDFXCERnxght5CKmHfmU7ynIqU83UuLONRdv1nxtv wjn1BxeZf3ikdHU/R8/qwdqOo5jnIfx/F70nyBuNF+MRmkQV0d92SLibbd44rRvEZH9CtjybqEwgz 0D2VBzpEW3vQaA5R04Z20rkj3pWAd3/cqYTxb8eu4CXRlbEZiKfGY1VEl61uAXLE8dd0wbqsoiO44 jnC3rXyw==; Received: from mail-lj1-x22e.google.com ([2a00:1450:4864:20::22e]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq2-00000000nq5-1T7r for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:32 +0000 Received: by mail-lj1-x22e.google.com with SMTP id 38308e7fff4ca-30761be8fa8so65769751fa.2 for ; Wed, 12 Feb 2025 03:23:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359408; x=1739964208; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=GFYj8M8sWUilipZ79moNvoJmNP0x1CtymAnRzXm2Cm0=; b=ktTN9v5LAdAXN72fB0g/AALcv88DyCjIaBQ8dGwlh/oWnfY4vP2uhI6epz6ZcnT9nL 25MgkfkXOO8oPnD+paVR+sXEfLfbJgDizkGXEzmuqvA5OdV643/z8Z/TRolFRydFIYtV lyMAE1DkXZnRCCsEeBWYDBDbAzVAW2v1YjjwylYMBebgrWUH38Xbkmt1BLFJJmsnSdw0 tEox7X1usmwsW42XAsXyEzhQoDPonEI+wgDcHYNFN+AzDVX9Xh+fCdPjUMNPmlfQqZFA /Kay4yRk/q1Nq5m+dooMNP9mS24kgdR1sU2Qqck1AyhHDKkf/bSAiR2lhn6U1bK2GRZ4 hhOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359408; x=1739964208; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GFYj8M8sWUilipZ79moNvoJmNP0x1CtymAnRzXm2Cm0=; b=wMYPZhezD+5Kt6eJsEX8ZKG8QXIKicNW0QHxVww3nMYZWA0NsOCKyb7HUzKFSrkpsd Q5i3L5s8Jf2sDLf+wPArFH5wVHgUEUA1HFmZDwW47PN5R4w+Aq8NJCWO6icZ40Tx7GVP kM3A3uVDU8hvCdlBIA0GZDEDzqDT00hjF3ocgguKXp+EHQ/DVX1JrCzgUWo841gZ6RcY diG8uB7zrQdMxFRoujFIU/lJUzOV7BJiSOfacj8X7+H0Lswrn61aMA8Fpcdfqj8y3wsW ifiilQ5UVEpuuQCjrWhq5AHz1yX4Wb3uek/FxjKuHIZKUjO7Xs+smbhnf7D98ueEGTnd t4yg== X-Gm-Message-State: AOJu0YzTdQTPj6Oi9a2Go6vWTO0T2Gzvkuu+S/aSJs+/hwwfdRWUOx5w WUnse3p6umIwt8iKgwx7GA2oErgN3lG/8habe2krLSJyfNH9mDiSMWXBy1OBlFg= X-Gm-Gg: ASbGnctHxtKA4Dez+KWBSx7ucy2Tu05WbDDSThkjg9T6ZV9AAiSkBYKZOjqkJ57AmmJ tE8OINNMwh7hXIX9Uegu/tG8alYZBBrDu6JsdOSP3TrcQuzwShU7OjBtGIyvSwvyGRC427hbv2x ryNUX+YUPXc7+GbhRL5TymIgfEW5xV7nK1qbWMzSv3rF1iJtLVDp50ERPlipXsnz2tObbKpJEf0 NCtSybQJmgOgKGtggILjiIlQhWZg/oBt2I4erLn8TnSTyrWOrTdzJGdITvZfquxHoSp2obhhNlt rcpT5AwMYViRhP5/K2MijZpxOg== X-Google-Smtp-Source: AGHT+IFtx+JLgvxuRlhMSLR6Ap6cqBlv8BsXuPeR5u4M9sE4qMRSRRT4UspKmtEyZVl6ngpQPtOzZA== X-Received: by 2002:a2e:be9b:0:b0:307:46f3:4e92 with SMTP id 38308e7fff4ca-309036ea8f1mr13238611fa.32.1739359407816; Wed, 12 Feb 2025 03:23:27 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:27 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:15 +0100 Subject: [PATCH v4 21/31] ARM: entry: Do not double-call exit functions MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-21-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_112330_632766_42C117C8 X-CRM114-Status: GOOD ( 17.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This is a semantic change to ret_fast_syscall: if there is no work pending the function used to jump to ret_to_user, instead we proceed to return from the syscall. If we jump to ret_to_user, IRQs are enabled and we call syscall_exit_to_user_mode a second time; at this point a this just means a second call to rseq_syscall(), but it seems wrong and we want syscall_exit_to_user_mode to be called exactly once. Apparently the ARM kernel has under some circumstances called rseq_syscall() twice on the exit to userspace without side effects. ret_to_user also checks for pending work a second time under ret_to_user_from_irq, but this isn't even needed any more on the fast return path: this was needed because the syscall invocation enabled interrupts when doing syscall tracing, and all tracing invocation has been moved over to C in earlier patches. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index ece9215717143427de7572a18a6dbe620a9193e9..33bc9e7c5b32f62474724f1d47f97af24dba53bf 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -49,7 +49,7 @@ ret_fast_syscall: tst r1, #_TIF_SYSCALL_WORK beq 1f - b ret_to_user + b 2f 1: mov r0, sp @ 'regs' bl do_work_pending From patchwork Wed Feb 12 11:23:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971820 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A1AF7C02198 for ; Wed, 12 Feb 2025 11:58:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=0l05ZDEo5bKH98lcf/z2wVpVbfg6jDUAnt9TMH5HSeU=; b=ZucUiRajVStTZVHV5bzXPjGMTk rdsW1QTiN3H15oKW797bC1hssB/aeV0KoGf38CnQtDo6AzeKzhhFjKHYTIXkACfrvwBJqZuSPrEuf G0jT/UERqdtNY1JHE+VIiZHTis8IM/WpB6v1tc8C+x/yAynw1qjrDzA/vpC8oGzNogeY+7Y/hkWNh mHLzkEXpLbT9HFkPdngXjEmAULiTX0avxNUBDNVnRdM2zjg4F288Fv9pTErXDhxsW13L5DnK8D4HA gYwavDU/P9AZPWgFwKOS2MO9A3vkEtFyQ+Pg5kGcveW1GMEdYjqBL1UlWq7UA0iajQa+3eeeJWgIx KZYVKzqg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBNl-00000007GeY-02pU; Wed, 12 Feb 2025 11:58:21 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq5-000000079jg-0nEO for linux-arm-kernel@bombadil.infradead.org; Wed, 12 Feb 2025 11:23:33 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Sender:Reply-To:Content-ID:Content-Description; bh=0l05ZDEo5bKH98lcf/z2wVpVbfg6jDUAnt9TMH5HSeU=; b=XmO5F1q+gY2/XhOjWbCfv8590s HMaC5r5sGGEtQnUa4IJ0tcAlB7b+3voNYIfZlGay5Z9Fh6/dgKlcAFBMMu8WWmOqaleZjrx5MtrHs vlfXUzRS6IeyhoEDC7wFbwMHEz6Bo/wxjKXUA1/se+IYw2mXIrQPIIl0uGzNJz46KkQPfLOSEMfMV gi/gTbdLcRdJoWzX8219O6x4Blo3h03RsVqC5c8cuBcJgMLHRjJxMmLbs3zqFEC9d5EAdlreLQyQk wHoeFC3f2gNbDJyDn9CnNQ2h+WGb18b/ly4HQC+hoi69jpW9DvjlXOPxouHk2CmCnpixPhAHO2PqD 8D04dlnw==; Received: from mail-lj1-x22c.google.com ([2a00:1450:4864:20::22c]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq2-00000000nqR-1EHA for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:32 +0000 Received: by mail-lj1-x22c.google.com with SMTP id 38308e7fff4ca-308e3bd8286so39089821fa.1 for ; Wed, 12 Feb 2025 03:23:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359409; x=1739964209; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=0l05ZDEo5bKH98lcf/z2wVpVbfg6jDUAnt9TMH5HSeU=; b=O6pPW+GSbHSgVaW/ygdtzJGuikGqjm07wvDLzZH6Tng031/xRcmxjga7Co4ScHayeP aFrmAFKixvV/DGWdUdotb7yoTZ9F5Jw0tlQj8AFcmiYggiLulP9lTZaVeb2QJG3Ie4Ro lbVh7UmDC9uX6d8dR6EnWwZ+iA1tNkRHLrciScIvB2tBV9by4RSERWKH4FAPPi1D/2+J lcZ7Z4S3Up/PDJtJ4Ixf3ctAPD5FHCYqec+IWJ5vztglhjTnRLmpFCXbx53/sf1RFhof h7l8bnfrYn1sqpKc0BWghi0b3dM/HdKaryTnVLt4T/sofXzafJFKJsZTxJUME7R0OB1P mO5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359409; x=1739964209; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0l05ZDEo5bKH98lcf/z2wVpVbfg6jDUAnt9TMH5HSeU=; b=AHJCiSX/mcDSe2o+ZhrgeVJ1kNg9p9KrViS25crjHzKou2yOAiBbORlWpte9LZFaOI R6FDmuGqd9BVeOl0YGNiQ8IIh/gOSyYXO0UaxdKbpx5UHsM+B7OAzuSP2I0qzovpnBHT HqbJ18cti0r7doIpJYZSvOeOtG8PzhRBtsmwAMP+HzwJ7oNdMeHQ7e70xqBAoXUH9T/1 AdxFom7Ydjp9M8zZyJM5Fby1X7bDmpt+6QXSE5x+7BVgIzuLfcRbjp7+b7o2QuZ9JUtX FAnksvVcHFETxp3DsD0JgXbaLoK9fgKJYGngnqQV+B2sOCmLxW/mpFG0WbOKmQTtQ9dS KwdA== X-Gm-Message-State: AOJu0Yw0QvQNSR8xF2tYzXcoiucnZ6kgSKlfMi+VBtKozLVvg3oATjST d9qRDaZK9XZxJ/96TEDmUkSlGJC6OhAOr6U4Q6IHmBAlb+gC6QcQ8+4Jr+Zzoyn0EK67kAYdvQS m X-Gm-Gg: ASbGnctUfbJgIC3A4R27z9ICUZKXmXY1m9JNitOCUxd/ZKpEH/MTiHICxuyNypo/EvZ Ls8Jj49+RvPzUvjbta/LUROc0nFlZz6KVDAUw0+xaKNmjX7O+qBmnqh8GiUmtsG3oSIyAuaihwT tu8Zj7hYF26jjE0ao60PewG+JfFO3yPoZrMO6M46VfeZxIhsVaKR1VckU10ZyzkdrRJOuQEPFsW WIObBXaaqSK5CfMTm6opiZhuQ+iOR5iT/zF0LIySBxfPbbr0LnBICR++eQIKMI3GAImwnng28+V zcAVGksJG3ldfIr8rD/pRZBZjg== X-Google-Smtp-Source: AGHT+IGSxmUmsIGZU6BTQUkSp0x8dKEvIYRNw21ZDleo+Ua51tS+VPbbp2jic1YKzEdrANQKDuDojQ== X-Received: by 2002:a2e:bcc7:0:b0:307:2b3e:a4a9 with SMTP id 38308e7fff4ca-3090366fe24mr12618001fa.20.1739359408813; Wed, 12 Feb 2025 03:23:28 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:28 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:16 +0100 Subject: [PATCH v4 22/31] ARM: entry: Move work processing to C MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-22-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_112330_580331_A19A7CA4 X-CRM114-Status: GOOD ( 14.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org At the end of a syscall software interrupt and every time we exit an interrupt handler we check if there is work to do, i.e. if the current thread (userspace or kernel daemon) has set the _TIF_SYSCALL_WORK flag. Move this processing over to the C entry code. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 14 +------------- arch/arm/kernel/entry.c | 10 ++++++++++ 2 files changed, 11 insertions(+), 13 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 33bc9e7c5b32f62474724f1d47f97af24dba53bf..5385574c4339714f3d8b5ab6894f27466b89f932 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -42,19 +42,7 @@ ret_fast_syscall: /* do_rseq_syscall needs interrupts enabled. */ mov r0, sp @ 'regs' bl syscall_exit_to_user_mode - ldr r1, [tsk, #TI_FLAGS] @ re-check for syscall tracing - movs r1, r1, lsl #16 - beq 2f - - tst r1, #_TIF_SYSCALL_WORK - beq 1f - - b 2f - -1: mov r0, sp @ 'regs' - bl do_work_pending - -2: asm_irqentry_exit_to_user_mode + asm_irqentry_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 47233804733752aef0f2fec6f54badeab18e98d0..0e3960844b94b14325eb00ce7dd791b36c43ac3c 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -1,10 +1,16 @@ // SPDX-License-Identifier: GPL-2.0 #include +#include #include #include #include #include +static inline bool has_syscall_work(unsigned long flags) +{ + return unlikely(flags & _TIF_SYSCALL_WORK); +} + long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) { trace_hardirqs_on(); @@ -18,8 +24,12 @@ long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) void syscall_exit_to_user_mode(struct pt_regs *regs) { + unsigned long flags = read_thread_flags(); + rseq_syscall(regs); local_irq_disable(); + if (has_syscall_work(flags)) + do_work_pending(regs, flags); } noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) From patchwork Wed Feb 12 11:23:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971818 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7AD87C02198 for ; Wed, 12 Feb 2025 11:55:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=XZE98juq0oGv0Xstj1U7TzgmKhYjLwozMlzxeXVPHlA=; b=LxeQF11l49IV0sZIj1Men8IGlo 3UpWe497A35JNvuy4S9GEl5PsP5UQLjH8jvjq9QPsV2WfcCtFUP7BMDxZdg35eqArulhYf6AcIG3s ZMtVfO7R6HuvsUvr4ZTQV4F0uqo+8ltJBbyWVeUuMEpATditv92UYhVihwqk7t9ac0KdbkyF67Meo ZEV89zhdvJqm0m6BXTcuRm5dFBreGVoO/ne51A+ROoEzk/14J4qAOhG/TaEHQw/r99g/MCiv0DwVm +gi2PmAkmZjWlz98Q+zA6aS6GoKcR3HLbjBJLjfKcpFCF2UlnohpI7Uh9a4mw+P/UVNEWVcdpDjRq ZMRxqoUQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBKy-00000007GHf-2bhz; Wed, 12 Feb 2025 11:55:28 +0000 Received: from mail-lj1-x232.google.com ([2a00:1450:4864:20::232]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq3-000000079jE-2wEi for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:32 +0000 Received: by mail-lj1-x232.google.com with SMTP id 38308e7fff4ca-307325f2436so61495901fa.0 for ; Wed, 12 Feb 2025 03:23:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359410; x=1739964210; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=XZE98juq0oGv0Xstj1U7TzgmKhYjLwozMlzxeXVPHlA=; b=j/vrQdQP2Hg1aBykExme0LEtPwZNIBBKGr2P/IkcMwj2pLBTdwagfcq8yH5ML4kY+r y2Tq/Asn342D5xmMB0w48dcN8CMrCJakGWtbVvKIg2srxyZmvUzOU8a7FxzqOzsmFsqk YIb0bkqrZ5pA6JDrR4i6Hghvkm8lvETZXan7pQF+fd9qFARcGzb6W6QzcUCMl7KcEXFs KY6QQ9a36/OfkqKqVI4lDq4dMhLdZkMA5A1vCBXhaI6yQ7qwp4CICPD3MKbq2elzXHuC dOkbZULVvtj4OKL4G4hcP5rQsrFGtomwfJw23ilmqCHzjaGxcX2K/dqs+sMb7Gldxyok yEGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359410; x=1739964210; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XZE98juq0oGv0Xstj1U7TzgmKhYjLwozMlzxeXVPHlA=; b=Q7uiLoRj7oZNcHIihZvyhIvTOixZcPcDx66rqrXZO5Rc7gZRM0UXLe5SHWQ0Tk5lgq SB/u1du3eaeqo6WtHhCY9J0h+63nUs4hi+Q4m9VnZU579sJGWvcwrHeBnFHwP09P7rOm /gBnE2F5ZnI6MopdbJTjAid+XpwuXuO9NlU096HTs27SEwk0BTAcF/W0RlvBBMln2nGD UgXoZYCrDrl9cS6V+juo+PxMey/7Qdnv3AMVu+I4rT/1iah6H2VaMROwNcH3p2Lrllll V/n+4BKEyp5yu4JiGhCmQJf4HmoSpk//MQsZ92kI84NRJ9U6J2WitXSaaO/Va7W+xQpl ISsA== X-Gm-Message-State: AOJu0YzYxGwFzH46gPs5zCTr2mm+TcNdwt3JZcZccGWBtEf++5o7c+dX +wQZNtp67LMnhz8oM/ZoMxTztd9vFqMjEn9rEdnymPE06NaugLp4g7SxjxZbb0R4ThJl53OxC8i J X-Gm-Gg: ASbGncs4jY7K+o9nr0ohBGaMCFphPs7jZwWv/GgJtOFh5C408hYixjsQZRSxn5l8uw0 2zO0TTPK5KiN+omqej53ZvoTfCHLVKRCYXNoYY7VsnppgAxqi/gJND60OiKG6dX2Nw7BUihCa44 N0SN1gQCIJuvA9jPEgBWlwl90o3LzNCRXH16hvleLuV/h0KV+o88euRfsgw3wyBz1Q3kVkRwHay KfEOiBnsVTf1AleIG/lKNxJkEiQMRAPQoHKPRnHkUHWWxn3lzohMIJ17d3ieq2DxRofThp01Xje Ak45eiaeqbfm1rngaGJCasU2Dw== X-Google-Smtp-Source: AGHT+IHrTlZ5pCNMcRck2dAefduLfG8aeY2J1acA2/5dWt4A8Fi7HvaayN1bzLZYfwPYvudmJmpuSw== X-Received: by 2002:a2e:be86:0:b0:306:124c:69d5 with SMTP id 38308e7fff4ca-309037615d5mr11191991fa.34.1739359409801; Wed, 12 Feb 2025 03:23:29 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:29 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:17 +0100 Subject: [PATCH v4 23/31] ARM: entry: Stop exiting syscalls like IRQs MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-23-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032331_785079_9C0197A5 X-CRM114-Status: GOOD ( 14.94 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The syscalls are issued as software interrupts, and to this point they were sharing code with the other exception handlers, but this does not work with generic entry. Make syscall_exit_to_user_mode do what irqentry_exit_to_user_mode does at it's tail, and drop the invocations of irqentry_exit_to_user_mode from the syscall path so that these are now exception-exclusive. Split ret_to_user and ret_to_user_from_irq into two distinct execution paths. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 14 ++++++++++++-- arch/arm/kernel/entry.c | 4 ++++ 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 5385574c4339714f3d8b5ab6894f27466b89f932..e2ac6d3216b6828d948d60c424ff95ccdad12331 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -42,7 +42,6 @@ ret_fast_syscall: /* do_rseq_syscall needs interrupts enabled. */ mov r0, sp @ 'regs' bl syscall_exit_to_user_mode - asm_irqentry_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack @@ -62,6 +61,18 @@ ENTRY(ret_to_user) enable_irq_notrace @ enable interrupts mov r0, sp @ 'regs' bl syscall_exit_to_user_mode + ldr r1, [tsk, #TI_FLAGS] + movs r1, r1, lsl #16 + beq 1f + mov r0, sp @ 'regs' + bl do_work_pending +1: +#ifdef CONFIG_GCC_PLUGIN_STACKLEAK + bl stackleak_erase_on_task_stack +#endif + restore_user_regs +ENDPROC(ret_to_user) + ENTRY(ret_to_user_from_irq) ldr r1, [tsk, #TI_FLAGS] movs r1, r1, lsl #16 @@ -76,7 +87,6 @@ no_work_pending: #endif restore_user_regs ENDPROC(ret_to_user_from_irq) -ENDPROC(ret_to_user) /* * This is how we return from a fork. diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 0e3960844b94b14325eb00ce7dd791b36c43ac3c..88a7a699306ab61153d57333c26266b9161f5466 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -30,6 +30,10 @@ void syscall_exit_to_user_mode(struct pt_regs *regs) local_irq_disable(); if (has_syscall_work(flags)) do_work_pending(regs, flags); + + trace_hardirqs_on(); + /* This context tracking call has inverse naming */ + user_enter_callable(); } noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) From patchwork Wed Feb 12 11:23:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971821 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BCCA5C02198 for ; Wed, 12 Feb 2025 11:59:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=B1th+DnYZSKT2go08vz+Ftvr7FlZ7mKXRvbFUdOlbPI=; b=4FYbrtBIjgNCuJPGg88pJC5mht EncIsS2BWefYF3oHTprRUeoX/qsHWl4q7c9Y5eMTHtt/1GLua/PZjTHL0qq7YYGtz6QsbE+NButd9 3xghwCLgvSfSyjMIX0wuF/hMXsLg3YyBhZIxsyh+17FgP88L7LPJFnwkS5BqKiLeB7J3Gy212HpWf DNW6kqAn64xxSLTo5KtOhjTK+yLxzY6Ad/sBYr1l2Goa0GiUtRrZ/LZ7Ip3fSJtW+xY+cT4koKeia gETklxseclA45CD049y72UpFSi4oog6sA4lczQZsZsZeE2Z+gJOypxPqwPVN3r0aAUvsRYcYGVmbi 1VdCZehw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBP9-00000007Gw6-3Gmo; Wed, 12 Feb 2025 11:59:47 +0000 Received: from mail-lj1-x22d.google.com ([2a00:1450:4864:20::22d]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq4-000000079jQ-39YC for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:33 +0000 Received: by mail-lj1-x22d.google.com with SMTP id 38308e7fff4ca-308e9ff235bso30051021fa.2 for ; Wed, 12 Feb 2025 03:23:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359411; x=1739964211; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=B1th+DnYZSKT2go08vz+Ftvr7FlZ7mKXRvbFUdOlbPI=; b=PKKrkjjYR5/0VKMV87yJhVv1/Oy8aQyD84zACDyYfl/cIdjJKoOi7ipmh/d2EQxAoE R0vXVBM7QPguJteas4gJ9umTSV+eD37RLPw5LonZqQDGxMcIK47Y+OticDWml6KJ7T4b KXh0De1oExQl/n0Tj2LT9eug1G3ISbS+1PRcj+y3+QAMTkNmV+hXujd3JQAembZfztAK zw7ublf0ww3lCnin+zhxdV8il8rs72fRtpNAizZaUUbSco2+0IFpvIa7uAbFNttjv0Ib RsB6Dr2nbbVsKMGuD1wyjgkc9EvWXvjCcGFD2S5pC19/y8OYOOofjWWjUEHe8JxbLeRq dvaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359411; x=1739964211; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B1th+DnYZSKT2go08vz+Ftvr7FlZ7mKXRvbFUdOlbPI=; b=hY9yvba0i6IPZIshfqw0rSBYRw4B5v51dGSj1AZwKop8hJEnevHCLOh+4A8x5r2V14 FmQgtzWOYX1P8Hnnag6ETRn+irtI6RjfW4dzX4h61J/ibggroq+8OkZmGD92KNgNB+tn rTHeGp1A77ejod53VBP4kMtNkife0O69S9gRDGHo25JXeZzLpw6y6oDjHC+IjGAJGzbw dS6YlOIltB+ONbswhJpbxRL5tfK47CNqo+siuc2O5/Gn0qPY/M78gOW9F4yg2pbBFwnd CAWQrw4oltzDMyorYI2d9D5GtwZNGEAR4tMwAVgwhQtDZTWnnyF4RJvB8lLY7rgNj86U Zu/w== X-Gm-Message-State: AOJu0YxylacwctMgGKcuZ0an/uCqBAu9KV3g6ZsMewfHf9Yw9uMAfgqk dn80AFU8CcxQi29/GBMKUlJAgYG3YEzBELMOuUYtYNLVEoaOyp+qrn+f8N86dvI= X-Gm-Gg: ASbGncsmXkOutw5/t07p/IKQUnSIdQK0zqt7oQlk2ilI6Xms8hIF6/vF+wJvZKh2Rmv xtRoiNopm8zwiJg5txqnAY8Hwu+vT9elc2WvsP5zQ7Oot9Fm5dhT9Oywi+CipuBPfRWhSA67cxI yGs8mvaeh84rh8hF/dQvVwPhf04We9cpLmj5DD65NPfl8Ag0Ury4ZZx1hjUh+ecTMAo7CcBPW2a JSVQU8KvURRlxDNNII9tsIaGKRjS/2rOdHxI/jjL3KjCphu4trjf3VEesxEOz34kD6Q7kC2bmCl w6I/TGQlqqeTyOi1nKH4YaeTVg== X-Google-Smtp-Source: AGHT+IH0PjnzWpFgj3aInaPvyhRs2W2GlVVhEZ9bNbf1pRiBVZ5r7dcyK4H7rv4O7JS5XNCNKpDFKg== X-Received: by 2002:a2e:a58a:0:b0:308:fd11:7705 with SMTP id 38308e7fff4ca-3090363079amr11272811fa.2.1739359410842; Wed, 12 Feb 2025 03:23:30 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:30 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:18 +0100 Subject: [PATCH v4 24/31] ARM: entry: Complete syscall and IRQ transition to C MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-24-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032332_807301_34FA8F38 X-CRM114-Status: GOOD ( 17.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This moves over the last few lines of assembly to C. The subtle change is that in return to userspace from syscall (SWI) or interrupt, we need to call do_work_pending() as soon as the thread flags are != 0, just checking for work with _TIF_SYSCALL_WORK is not enough (the machine will freeze if we do that). This is because do_work_pending() does not just handle work: it handles _TIF_NEED_RESCHED, _TIF_SIGPENDING, _TIF_NOTIFY_SIGNAL and _TIF_UPROBE as well. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 13 +------------ arch/arm/kernel/entry.c | 19 +++++++++++++------ arch/arm/kernel/signal.c | 3 +-- 3 files changed, 15 insertions(+), 20 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index e2ac6d3216b6828d948d60c424ff95ccdad12331..6b0f86786a7d9e656ecbe8c741f2edbcbafd2915 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -61,12 +61,7 @@ ENTRY(ret_to_user) enable_irq_notrace @ enable interrupts mov r0, sp @ 'regs' bl syscall_exit_to_user_mode - ldr r1, [tsk, #TI_FLAGS] - movs r1, r1, lsl #16 - beq 1f - mov r0, sp @ 'regs' - bl do_work_pending -1: + #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack #endif @@ -74,12 +69,6 @@ ENTRY(ret_to_user) ENDPROC(ret_to_user) ENTRY(ret_to_user_from_irq) - ldr r1, [tsk, #TI_FLAGS] - movs r1, r1, lsl #16 - beq no_work_pending - mov r0, sp @ 'regs' - bl do_work_pending -no_work_pending: asm_irqentry_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 88a7a699306ab61153d57333c26266b9161f5466..d7fdb9df33312dd2a914b59512b701748f33b597 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -6,11 +6,6 @@ #include #include -static inline bool has_syscall_work(unsigned long flags) -{ - return unlikely(flags & _TIF_SYSCALL_WORK); -} - long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) { trace_hardirqs_on(); @@ -28,7 +23,11 @@ void syscall_exit_to_user_mode(struct pt_regs *regs) rseq_syscall(regs); local_irq_disable(); - if (has_syscall_work(flags)) + /* + * It really matters that we check for flags != 0 and not + * just for pending work here! + */ + if (flags) do_work_pending(regs, flags); trace_hardirqs_on(); @@ -45,6 +44,14 @@ noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) { + unsigned long flags = read_thread_flags(); + + /* + * It really matters that we check for flags != 0 and not + * just for pending work here! + */ + if (flags) + do_work_pending(regs, flags); trace_hardirqs_on(); /* This context tracking call has inverse naming */ user_enter_callable(); diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index 03b20637a2e1ca6f327223c9132915a97a9c4697..a39ee14ec5b70b3738ecef537709326ef1e8217c 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -598,8 +598,7 @@ static void arch_do_signal_or_restart(struct pt_regs *regs) return; } -asmlinkage void -do_work_pending(struct pt_regs *regs, unsigned int thread_flags) +void do_work_pending(struct pt_regs *regs, unsigned int thread_flags) { /* * The assembly code enters us with IRQs off, but it hasn't From patchwork Wed Feb 12 11:23:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971824 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F3BD9C02198 for ; Wed, 12 Feb 2025 12:04:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NA+LzTmhZnGJ5F/6+si6C2L1JDWL6V8/wsE3A9CcZck=; b=Xk1c1gszDFnZLpHktJlvg7SHbn zC95at4UTBz7z5/0Qtv1s2M+ZCAZYF0FHxkFK/2ZQhQV0oIv1Or94e8lX1DNxIzdTV03CEdR6C0DL TQnlxxAsnCJ/7hYhiixPThNUMyD+48UnH6sSXjRbLmzNI44BK3gMAsWkPRyrLS30B67Qw53Jjxe6o UKohp/QRNIgkhdvbIaJu8ChqYYrB21wD6Zvcpf1yOSaEZq16fqrn+426cJkcBAC+1ots+63AR1B8Y KzokLXENA5FDs6GuXSwdCTpkbbFP1hg2UEe9pfB+r1WGtwJkfqXtA3fs/MjSlR018JCC5eRbU4s4i WJ8iGH/A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBTL-00000007Hli-0OnH; Wed, 12 Feb 2025 12:04:07 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq9-000000079lQ-0vPQ for linux-arm-kernel@bombadil.infradead.org; Wed, 12 Feb 2025 11:23:37 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Sender:Reply-To:Content-ID:Content-Description; bh=NA+LzTmhZnGJ5F/6+si6C2L1JDWL6V8/wsE3A9CcZck=; b=QiQPG5HLtNiOS88dCIMdhCfd4a K0m7f7Uguf2u9kiyBwg7jBA6x32eUob89JVCr0i3zoqm4HLsg4k4Qwd4udTF4zfwourL7S26yOMbX 9RqcI2duP0SfbXfZtE9p+UapduMMMkxJT/VIGM4kICm8qepfSqWMZCPmjn3hyCxNr13lLGMReXXyx 8qR9Us1UvigA41eguVkYdHmf9GiqLH61D0pVX0Us3GYzbEoMN6C0fad1ArDTwWiEOU1BcQfs6viri kpLd/3kTjoS5uLiiNxb6W/R3h5bwXADmhex9kHnDF8PZeglKRE7SPJE2kN8FVVUSATGOLMfNWAP7F sUERxHmQ==; Received: from mail-lj1-x22f.google.com ([2a00:1450:4864:20::22f]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq6-00000000nrb-1ipZ for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:36 +0000 Received: by mail-lj1-x22f.google.com with SMTP id 38308e7fff4ca-308e92c3779so7143121fa.0 for ; Wed, 12 Feb 2025 03:23:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359412; x=1739964212; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=NA+LzTmhZnGJ5F/6+si6C2L1JDWL6V8/wsE3A9CcZck=; b=ItjTaydWZbG+HsSx+N5DQHTkWamt8UN5ptm2DMuKm/Iytss4MvICVuzPuNSarMCp6l 3zgdsceU76MNptxWps7h7Pej0IlWJYnenad9pALprrPV09B9kKdmxt2m+UlMXiHIvyqa GYPyAzGXvVNpH0la45g4iemG41+tw6h6CsyDf+SCS41FsPDAwbm/VKYT0O61lR8PQEnb gqOVGC0b5mFWOKoCggZFw6P+TMlzTPHmsysHspgISQNtLvbaZSMNOWktFEqWRHpFCkDX nfe2Ax2S9FT15iw5jH7YH5ajlfsBUvFoqS0c/u2h00tXdONBJhNpaJyA/BHE7ZSCZ4r3 AS5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359412; x=1739964212; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NA+LzTmhZnGJ5F/6+si6C2L1JDWL6V8/wsE3A9CcZck=; b=pk4vf3IYbq44ZbjtqvmVUOusHexwZJPMel5s29LbFOUzzO5B4ss3rGdibQg+0DCzpA O9D+ISlie7QKykjyOSKypugZIpN2+elnvrUycLIacinJSJNj6hFYFJXjZtlesCces3WL bS0o3HJo85f57pZ/fTqDi3TLdv6bxNGPboZJooxRU2HbLeVlCLlXHtupABdYEOsuNQUp QKCMJ3ijDAGKQ+EoZxOyPcXZ+tHyXBps5XNSzfSjIRN5Ks9vgriGFTHbkZPO14F6/m7z WCuEBRX1qnk5ZMUBaKU4EnMIAKKcUyXTRMfDv9mqvxtA9TsxxzjEjhg9d+C4fmaNiE6y hsmw== X-Gm-Message-State: AOJu0YzH+B7kFspLSfk+R7das8NMlA6WeppFo78OY01nM4C2s3VdJ7LN Jn4b83ZZTgMcZA/0zFFKoK7JkB3WTeXMM3oG+lM68OpOlBK5ftaDjnyiZIvjzmI= X-Gm-Gg: ASbGncvHt95wMzTnc7ZvmiU/ncBaKJwnAnDNnz4GyKe/yfekkGyn/GBjxtzuPDRr+QI Nf6/ioAe9bbRRBZFhK9qQ8UeZO5VDevZnAw0kO4IOmET/lMV03o/4ozxMVC2WO86azIXTrici5h SDVKraRXycACjze5rN2L6kKgQdq0rNiZBxdBnYRY9AyooQD6383bFkykMPoHM1v9iQha6Mok5wI exWeQgEF722AasHoVvN0xYb3TiJsE6MM3s0e6lhLyCE8bK18mTqAe3EqnL5qoHNoClTid/l3JKu F8w8/0UBDIbJFN0ozzLrcCFEhg== X-Google-Smtp-Source: AGHT+IEplwuuVF82kBeGhT/TWo/XqlzuKUbj10/w8fQqnJV4OOCk8n2PPTKgIfpGykjHMApLmTymWA== X-Received: by 2002:a2e:be85:0:b0:300:182b:91a0 with SMTP id 38308e7fff4ca-309045927b9mr9380691fa.7.1739359411856; Wed, 12 Feb 2025 03:23:31 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:31 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:19 +0100 Subject: [PATCH v4 25/31] ARM: entry: Create irqentry calls from kernel mode MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-25-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_112334_663456_C86CBAB7 X-CRM114-Status: GOOD ( 14.11 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This adds a few calls to C when exceptions enter/exit from the kernel itself, so these can be used for generic entry. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 2 ++ arch/arm/kernel/entry-armv.S | 3 +++ arch/arm/kernel/entry-header.S | 5 +++++ arch/arm/kernel/entry.c | 8 ++++++++ 4 files changed, 18 insertions(+) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index 8b8cc9a1eefbd00dd75d4e344576fc184a413768..a78bc5054b093120e903f08369fb8b1a9feb7f81 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -12,5 +12,7 @@ long syscall_enter_from_user_mode(struct pt_regs *regs, long); void syscall_exit_to_user_mode(struct pt_regs *regs); void irqentry_enter_from_user_mode(struct pt_regs *regs); void irqentry_exit_to_user_mode(struct pt_regs *regs); +void irqentry_enter_from_kernel_mode(struct pt_regs *regs); +void irqentry_exit_to_kernel_mode(struct pt_regs *regs); #endif /* __ASM_ENTRY_H__ */ diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 3cd5a80b47758867d1db6c3971030370bca1551a..f64c4cc8beda6bcd469e6ff1a1f337d52dbbaf9c 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -205,6 +205,9 @@ ENDPROC(__und_invalid) get_thread_info tsk uaccess_entry tsk, r0, r1, r2, \uaccess + mov r0, sp @ 'regs' + bl irqentry_enter_from_kernel_mode + .if \trace #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_off diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 51c7f93c1ded1db56f173c7aaed8094ef011ff5a..49a9c5cf6fd5fbb917f2ada6c0d6cc400b7d3fb3 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -210,6 +210,7 @@ .else @ IRQs off again before pulling preserved data off the stack disable_irq_notrace + #ifdef CONFIG_TRACE_IRQFLAGS tst \rpsr, #PSR_I_BIT bleq trace_hardirqs_on @@ -217,6 +218,10 @@ blne trace_hardirqs_off #endif .endif + + mov r0, sp @ 'regs' + bl irqentry_exit_to_kernel_mode + uaccess_exit tsk, r0, r1 #ifndef CONFIG_THUMB2_KERNEL diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index d7fdb9df33312dd2a914b59512b701748f33b597..674b5adcec0001b7d075d6936bfb4e318cb7ce74 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -56,3 +56,11 @@ noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) /* This context tracking call has inverse naming */ user_enter_callable(); } + +noinstr void irqentry_enter_from_kernel_mode(struct pt_regs *regs) +{ +} + +noinstr void irqentry_exit_to_kernel_mode(struct pt_regs *regs) +{ +} From patchwork Wed Feb 12 11:23:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971822 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D5E2DC02198 for ; Wed, 12 Feb 2025 12:01:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=8L1+xbNej+7+BbJW1uipXHwDEUpYT2HP3XoeW+FBwgo=; b=awHBGbpPwPAzkhi+Uve13RGkuD GmlLF+/CoCuKQEuChlIbnSk+UpPFIk8+STbDz+007ZRd1pjCCIfgBnIs5AEGZb7EfVAA3yHg28Sca JmHzdm8e5IZeweeQStr5hXEBgehZqYbdzdnIpi9Z5sE2ghl1yFY4N88AV2YY4cIp5N7hA0eq4njal /ghMhi45IMnW55QqoSf5dniYedCZV1b5yGav38tT4EvvjMR8q1ZE9/4ttVZKuHUglakwxml5smq+w wXe9w8lbQ13FLYTusrrNLbCz/QOggsBVLAq+H/IZXBINQwqsc1k7IVXU32OsLm1HqqW0zAyxF8y2R 15EAhwiw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBQX-00000007HHn-2MOy; Wed, 12 Feb 2025 12:01:13 +0000 Received: from mail-lj1-f170.google.com ([209.85.208.170]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq6-000000079kM-3YMV for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:35 +0000 Received: by mail-lj1-f170.google.com with SMTP id 38308e7fff4ca-30761be8fa7so60303531fa.2 for ; Wed, 12 Feb 2025 03:23:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359413; x=1739964213; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=8L1+xbNej+7+BbJW1uipXHwDEUpYT2HP3XoeW+FBwgo=; b=BkPAdvBNUUuPdAEH6cTXWH4c6ni1hwJiEYlpZmSYA0Z3RtMsVqzsnFuTBGgzpktzKm bkqOOmpiaI5cBvrvjuZacqR9wVSAT8749Ii9rpck0QCZWABZrpcFXixNemEgHqCGgehL QyOpKcd5kpjAAE5VIcxEvAWwmQ7JkyTSqeI7PME4gG3gXz4vhL4bOBVqQJ1eRwpCKuk8 YkopfHmuugACnOsCkvf9AJbBChxi8QuRq9wQVwJ1MDmLoohpbVy/ej8yOeA9EF4tiXjP 5gfoa7hZDHYcXRmFg34uj+fQ4EG3LnNe7393oHZLSuRQyfZ6Zmpu8qvw1dluQ/eIOjDG m9Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359413; x=1739964213; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8L1+xbNej+7+BbJW1uipXHwDEUpYT2HP3XoeW+FBwgo=; b=bQLERl8NG6WUYFleyAjLLfpiRGgsXlIqxrbD1a06YiHew7T7Iy1UyviSiWIe+XrXDl DTjQkiJMlc+Fw1xSWI2YdD1iv12BIwpXIYoomOu69C47GTC6MBntVj5cn40gVDFR1MWs IViFZX0dc2+SJ0lbA+2sUWYnRJ+lq1tgiY9TYK/vik59Zye2PpPFkpxUzvHkpopENcDX HGooNiMlRHes5jJvKI4+g5qBDxGH8/QeFtOOSgS7gdjwK2Xrf2Ebj+j7cTJBY7B942FO 8ZZPkWc5h3qRdV01558y7XV4jW+BtEwli5u3HE8K3RN2LuFoRGlnusW3Cq02enHBfSQn qzlQ== X-Gm-Message-State: AOJu0YzHDZPlPahvAQz2xILBOiQsJTujLOiHMNAppJS6jLHr6ALzUaqA Rr6Ru2QkRq/AUMRtgqd4Jle6BBBstic/IJF94bEkMoyU/R/iXSuEIyN6jiQrOFo= X-Gm-Gg: ASbGncsaY5olHjH4GNywmToJojwQUcw+1prl+ntQfHShHe2yfZ95otPmqZvCTeA05QI Ugw2y/Rd05mB4V1HJQmGlfqUUgXyzpOdlwxWyV5pevZrFDdKY+VksN0lUs42q2jAiAezEAO0Gxp tksa+ocXIgc0jGxqBE/4PJg7u7dDvrRF4HqTnRZRnyQCGOm+VA0MLD3i5xyrtFmdJUyv3licyCB VAz89ZoR9cthxlVcogQFJm3uz61jZlLksfFRbHNEBLIQCzN0TKm7cezbKMX1/ynQDVZfnCZh4yZ DNf0qDMPw8gn1HD0cX9rWDpmoA== X-Google-Smtp-Source: AGHT+IGztsILaaYaXNkeSXwRXUmkpO930r7CtwcGUJLwBA+MhfurzFmeCaR5cjXsMc204D4O1MAizg== X-Received: by 2002:a2e:be22:0:b0:302:3003:97e with SMTP id 38308e7fff4ca-309036dbf25mr11765111fa.30.1739359412849; Wed, 12 Feb 2025 03:23:32 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:32 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:20 +0100 Subject: [PATCH v4 26/31] ARM: entry: Move in-kernel hardirq tracing to C MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-26-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032334_887742_D740DF51 X-CRM114-Status: GOOD ( 14.62 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Move the code tracing hardirqs on/off into the C callbacks for irqentry_enter_from_kernel_mode() and irqentry_exit_to_kernel_mode(). The semantic difference occurred is that we alsways check the PSR_I_BIT to determine if (hard) interrupts were enabled or not. The assembly has a tweak to avoid this if we are exiting an IRQ since it is obvious that IRQs must have been enabled to get there, but for simplicity we just check it for all exceptions. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 13 ++++--------- arch/arm/kernel/entry-header.S | 19 ++----------------- arch/arm/kernel/entry.c | 5 +++++ 3 files changed, 11 insertions(+), 26 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index f64c4cc8beda6bcd469e6ff1a1f337d52dbbaf9c..2a789c8834b93475c32dcb6ba5854e24ddd8d6e9 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -158,7 +158,7 @@ ENDPROC(__und_invalid) #define SPFIX(code...) #endif - .macro svc_entry, stack_hole=0, trace=1, uaccess=1, overflow_check=1 + .macro svc_entry, stack_hole=0 uaccess=1, overflow_check=1 UNWIND(.fnstart ) sub sp, sp, #(SVC_REGS_SIZE + \stack_hole) THUMB( add sp, r1 ) @ get SP in a GPR without @@ -208,11 +208,6 @@ ENDPROC(__und_invalid) mov r0, sp @ 'regs' bl irqentry_enter_from_kernel_mode - .if \trace -#ifdef CONFIG_TRACE_IRQFLAGS - bl trace_hardirqs_off -#endif - .endif .endm .align 5 @@ -239,7 +234,7 @@ __irq_svc: blne svc_preempt #endif - svc_exit r5, irq = 1 @ return from exception + svc_exit r5 @ return from exception UNWIND(.fnend ) ENDPROC(__irq_svc) @@ -303,7 +298,7 @@ ENDPROC(__pabt_svc) .align 5 __fiq_svc: - svc_entry trace=0 + svc_entry mov r0, sp @ struct pt_regs *regs bl handle_fiq_as_nmi svc_exit_via_fiq @@ -321,7 +316,7 @@ ENDPROC(__fiq_svc) @ .align 5 __fiq_abt: - svc_entry trace=0 + svc_entry ARM( msr cpsr_c, #ABT_MODE | PSR_I_BIT | PSR_F_BIT ) THUMB( mov r0, #ABT_MODE | PSR_I_BIT | PSR_F_BIT ) diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index 49a9c5cf6fd5fbb917f2ada6c0d6cc400b7d3fb3..cfaf14d71378ba14bbb2a42cd36d48a23838eee1 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -199,26 +199,11 @@ .endm - .macro svc_exit, rpsr, irq = 0 - .if \irq != 0 - @ IRQs already off -#ifdef CONFIG_TRACE_IRQFLAGS - @ The parent context IRQs must have been enabled to get here in - @ the first place, so there's no point checking the PSR I bit. - bl trace_hardirqs_on -#endif - .else + .macro svc_exit, rpsr + @ IRQs off again before pulling preserved data off the stack disable_irq_notrace -#ifdef CONFIG_TRACE_IRQFLAGS - tst \rpsr, #PSR_I_BIT - bleq trace_hardirqs_on - tst \rpsr, #PSR_I_BIT - blne trace_hardirqs_off -#endif - .endif - mov r0, sp @ 'regs' bl irqentry_exit_to_kernel_mode diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 674b5adcec0001b7d075d6936bfb4e318cb7ce74..1e1284cc4caed6e602ce36e812d535e6fe324f34 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -59,8 +59,13 @@ noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) noinstr void irqentry_enter_from_kernel_mode(struct pt_regs *regs) { + trace_hardirqs_off(); } noinstr void irqentry_exit_to_kernel_mode(struct pt_regs *regs) { + if (interrupts_enabled(regs)) + trace_hardirqs_on(); + else + trace_hardirqs_off(); } From patchwork Wed Feb 12 11:23:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971823 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 539A7C021A0 for ; Wed, 12 Feb 2025 12:02:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=PgPYwUz2C3jq2K678fDOOKopXFthUCF1z0KBZvQ8kgo=; b=m6mvK9ikTp2TMDCkc2B5ozrv1F 5BA7i9FYTCjDUqzXEGyXnq3a4mLELbXGK9cfRv/K/uezPhVYwRXHunnGq7OlNJll/sF+v52aCiHBQ 6gY6uwj0KfRxJeYM23HqHSFW8SesbiHH0mG1W/N514LNR7IttB+8QiofvpKX3j8WQNCF/3N8G1/p6 V2yfXyrAy2vLwo5ILTfE1Clw3B9LFOTKn5W/69nSeyc6bV9i+4I1y/s74o+3417eSvUDOTAZ9381/ OT7BnwSqNaOHmriCk1x2xCkb6uXFzkN/hhwDzRWDU9U3+cX7jN+kKVrgDqOnR1CAynsE3Yh9Iz3Qp RU/9CtjQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBRw-00000007HY7-1p5S; Wed, 12 Feb 2025 12:02:40 +0000 Received: from mail-lj1-x229.google.com ([2a00:1450:4864:20::229]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq7-000000079kc-3yPv for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:37 +0000 Received: by mail-lj1-x229.google.com with SMTP id 38308e7fff4ca-308eebd4938so31230691fa.0 for ; Wed, 12 Feb 2025 03:23:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359414; x=1739964214; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=PgPYwUz2C3jq2K678fDOOKopXFthUCF1z0KBZvQ8kgo=; b=ImZVcorrhG6j8oIOb52Ah0vBD0Sw9kBwVnbe6jrF/A/xnalwAsD8gzORzkBw/1s8qP MgmykK0F4VIyYjfBURdhOd7y8Ip36MQyKFfVat6MkMLVUURI3cONPv/gVBXpaxx6lA3L 6sBiaL3Y4Q1xwdQdB4ARSFze0wPLbSz64zXdgBbYqkWTkjNvpsugXAErj81dMNwijZaw YPfJsW8eiVWljG1hIBoSPicyOJmGzT4ZWYUPwvA+EWSkU0P7EupKAX3/jm51esvkomUT a4wpz1tpJeqY/2PMuy3Y3x8CFs8GV4WBt4zsl4zNCKRszVvnU/ZPBjx9/gcP2uRkqpWl Jtrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359414; x=1739964214; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PgPYwUz2C3jq2K678fDOOKopXFthUCF1z0KBZvQ8kgo=; b=vHlZOkaYuMaP3msI4mcoiSI5E285pC33mFTAkq/gzqmrT6rWV0zyT3Jh7VrWkL0i0N nHY8J9MnK9fYi2dbVqGPEb+/GI5ekfmPq81IOe+cIWNsk4VdDkodAJ6JFJehl3kyHsad U66bi0uRjSaQ/rwEcNFEa/mUyamouUiLkvYGcY9Vavawv3f6C1sf5daEWASOUiXxZ1VK LwUN4xtbr672/s/v6IkgWqmnp4XNR7JndPy5Nd4tJlegObEut7FkgeIi/OmTRdE7L82V MexKlcLya/inclbm2PmuDIC38RIIdKNUzCfbQQePCKP+CzxmLtKuLPGATmEwv2yGbheL XhOw== X-Gm-Message-State: AOJu0YxJDaT/1ftAvx5EM44ZtC3PnZksXmqebFaD4nGNtp/e3gZrznBK u7WHIKHtGyhM6BsqRSsubjqyBvoVcopB7O7jkOS1THR+NSSXnCXHTBfLrKr29W0= X-Gm-Gg: ASbGncsBehEv1LSSpxyJ1p13EG6iSyXburVXWXhtCR4Skn1C4XI9fwLjmRpt2tG6FfW Nn0XSyb9XJKVq8HMoUexdER2ql4eDVDWM2wdj707LOqkuzsLjzuJeSXj2teXjIZ/ZQzwtFVWaEh krFZksSJ13MXkbzdGzL0ff+NjdpUc/97ZngQbGHnXMYJNW//RWxu8A8Jd48Y1SDaT8b9HPPxMnb R532Q7fkLyDg4ACzjKDg8MHbjvVGDvnHGqwABrVC5PE0/z1qQhF7fWL3+96a2REF7NtxGkAT4g1 p5knqOaVGYBHhuKpYLbTQezUYA== X-Google-Smtp-Source: AGHT+IGF2XSQHUsYlAH9QvRI7MOgHMAfLQ2S1iDSbZ5eNFwUYo7EQJgkizHhQjqYAo/v6F8JXXJM2Q== X-Received: by 2002:a2e:bc84:0:b0:308:ffa1:8915 with SMTP id 38308e7fff4ca-309037ac73fmr9166291fa.5.1739359413983; Wed, 12 Feb 2025 03:23:33 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:33 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:21 +0100 Subject: [PATCH v4 27/31] ARM: irq: Add irqstack helper MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-27-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032336_035908_B6E7F243 X-CRM114-Status: GOOD ( 14.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a helper to dispatch IRQ execution to the IRQ stack: call_on_irq_stack() so we can explicitly issue handle_irq() on the IRQ stack from a C program. Cc: Ard Biesheuvel Signed-off-by: Linus Walleij --- arch/arm/kernel/irq.c | 6 ++++++ arch/arm/kernel/irq.h | 2 ++ 2 files changed, 8 insertions(+) diff --git a/arch/arm/kernel/irq.c b/arch/arm/kernel/irq.c index e1993e28a9ecfd80b55b2677253ac582467e6c14..f99d6b24d8ff56f901e4a817c61a1e57b7e532cc 100644 --- a/arch/arm/kernel/irq.c +++ b/arch/arm/kernel/irq.c @@ -43,6 +43,7 @@ #include #include +#include "irq.h" #include "reboot.h" unsigned long irq_err_count; @@ -71,6 +72,11 @@ static void __init init_irq_stacks(void) } } +void call_on_irq_stack(void (*fn)(void *), void *arg) +{ + call_with_stack(fn, arg, __this_cpu_read(irq_stack_ptr)); +} + #ifdef CONFIG_SOFTIRQ_ON_OWN_STACK static void ____do_softirq(void *arg) { diff --git a/arch/arm/kernel/irq.h b/arch/arm/kernel/irq.h new file mode 100644 index 0000000000000000000000000000000000000000..80dd5bfe6403d68fc62fab30666d5645748d329f --- /dev/null +++ b/arch/arm/kernel/irq.h @@ -0,0 +1,2 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +void call_on_irq_stack(void (*fn)(void *), void *arg); From patchwork Wed Feb 12 11:23:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971825 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0CE9AC02198 for ; Wed, 12 Feb 2025 12:05:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=f3uDcPAUn17u5guwAmKffFv3WeYri61KKtnRD7KLFFU=; b=kspIoV3e67WeoVQiyoLabtteZe 3TTJyLC5orJBeED798d29YFupS8f0A0hMZnBXjNGAukqehjncjGYp3PCOEtXOXwtUA1Otv/0AtBaV CtSx7Qq4qw+c5nqWGB6PueVLiU8OyAIZhRpyAyp/RWDadu+UlHAiDtVlXEAAZzFEKRhS1WwYp4Evf 4p+5ivXoRLhY1aocC3Z6gqwSt5T5BDhMd1dLAGdhylj0IFOIn+YR12U4i92j6rAWxJp/QLXSqr1bY JSiM82FPmIwRHluzVWTo3Zw62OoAyhpdCSTb/2vJo5FRNvyi44aeWUn3EC2wiKOTPbXzkNN1V600d 5fr6lR7A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBUi-00000007Hy1-3eFL; Wed, 12 Feb 2025 12:05:32 +0000 Received: from mail-lj1-f172.google.com ([209.85.208.172]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAq9-000000079lN-2BHX for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:39 +0000 Received: by mail-lj1-f172.google.com with SMTP id 38308e7fff4ca-308edbc368cso32851811fa.1 for ; Wed, 12 Feb 2025 03:23:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359415; x=1739964215; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=f3uDcPAUn17u5guwAmKffFv3WeYri61KKtnRD7KLFFU=; b=Zl9XJGE4dd+qmkPt7UctP+p3VErtiL4NUQjL5DfDwe3fV+2jDoDkOmfWxzvzQMyxpS pU8rkdHeqhFSNfrVDHhJ9tRRyFpgDyy3bFxRyCATYxK4od4FNbODRJ2bhJpamiJiPmPt mjBJlFFpC2R1jj+GpHTLoLbiJrKS6JisQ7VculBj6l6DW2Kq0AIHAy04RnB8Sl6nCrJV tJa3ycPSewbHTXECExn89WL1TFob717sf4Ic6TgF1MjoXtDawcG3yfOi87M5SF8T7U5/ IVLaRIQPF9NqbLSZBchM8ES6tu0o3jMKTlZKbGSisf0J7ZxKOVCHq20zZn3Y91O9bo04 SQzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359415; x=1739964215; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=f3uDcPAUn17u5guwAmKffFv3WeYri61KKtnRD7KLFFU=; b=xUfW080mYXZHOb1HOFbxjesFQ53V57n8FbsMZ13pcEg9Ud5LuJiGI9c7j0TSwktHdS B0aYlWuO8U2qGxSB8wBe9Lg8yeSxY8zM7PmpEjoAjifDnGmsZfXM4DxAoDRBVpzw/9nh GhLFYAfjVWnFBXt6y2WfEhS6O1EConbUbmC5mRjw3zwcW+hIp7CB9NotvNXwJfKvDea0 1j/VFunQCSRWdklQkgHY0zxJX7mCv8r+MmpGzS+nWloKfxVmFsX76qqvaPA1zWWnADRm pmK5NP05t440dy2tOJKscZkvmBXjJpNPnPvXPPW276KavgKTB82pU39pVHjgXHtP8p1y 6qVQ== X-Gm-Message-State: AOJu0Yx4VbW1I++PwtegE/PzFqag0wk5mfGZUt1wf0K+DpgujB9kk+L0 TAtrVdCBc5INPvUWe6fWf9KPWCHYPtupWim9cXIAka3xQsExW8lG5HG5g/BCpmk= X-Gm-Gg: ASbGncthvL8bR1cymPdV31AL4YOn4g1X5OWKmknyXN48i35z1cg4mgipBFgso8o7aHL TGS8nJW0xUFgKQnAvqwyxYXMjD9C5QC1o4GkYE3Exkejim1cCT+KeOAXLBZw2ZJBNBH9jnJ+v7L zEg8aII3j+LrhT85u8GRvNV6UMUzrkd9XwmDAHJqLJzKv2KqFliBxQcs/buh/rZeVEtLC6AOU6w hxqI9ymPiiNcPwK2sja9amg19USdSPHb9KQZbGuUfW/vVvUVsPGddBTFGURxYOXPLCvO4zgbyOs VI447FF/QP8MV6WamPA2oEcviQ== X-Google-Smtp-Source: AGHT+IFmyIoqF6QqB+73fCEmlj7J35c6YAHltdfZCDJIcH19FAT/hkU/hCN7mO1wHl3rV0Vb0E7YYQ== X-Received: by 2002:a2e:bea4:0:b0:308:eb34:1037 with SMTP id 38308e7fff4ca-309050915d3mr8754251fa.23.1739359415134; Wed, 12 Feb 2025 03:23:35 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:34 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:22 +0100 Subject: [PATCH v4 28/31] ARM: entry: Convert to generic entry MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-28-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032337_662114_E3C920BB X-CRM114-Status: GOOD ( 28.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This rewrites ARM to use the generic entry. All of the irqentry_* callbacks are collapsed and made to call into the generic entry code that handle the interrupts storing regs context before entry and restoring it after. The syscall_enter_from_user_mode(), syscall_exit_to_user_mode(), do_work_pending() constructs are replaced with generic code. The syscall invocation now enables syscall tracing "work" using the generic callbacks and no extra actions are needed to enable/disable syscall tracing. The old context tracking with user_exit_callable() user_enter_callable() is no longer used. Only CSKY and Xtensa use this deprecated mechanism now. handle_fiq_as_nmi() is retired as the generic entry provides the same functionality in irqentry_nmi_enter() and irqentry_nmi_exit(). Like with the ARM64 proposed patch for generic entry, it is necessary to create a small call "arm_exit_to_user_mode()" that is used for things such as ret_from_fork, i.e. when we create a completely new execution context with a botched return from interrupt. The elaborate checking and switching to IRQ stack in the irq_handler macro was reimplemented in C for this exercise: this was easiest and there was already code making use of the IRQ stack from C for soft interrupts. I have dropped the context tracking (i.e. calling irqentry_enter() and looking for trace conditions) on dabt/pabt i.e. do_DataAbort and do_PrefetchAbort code paths. These are addressed in a separate patch for development/review reasons. I added stubs for PTRACE_SYSEMU and PTRACE_SYSEMU_SINGLESTEP returning -EIO like the UM arch does, since those syscalls are required by generic entry. If someone wants to implement them, the placeholders are there. Signed-off-by: Linus Walleij --- arch/arm/Kconfig | 1 + arch/arm/include/asm/entry-common.h | 66 ++++++++++++++++++++++++++ arch/arm/include/asm/entry.h | 13 ++--- arch/arm/include/asm/ptrace.h | 3 -- arch/arm/include/asm/signal.h | 2 - arch/arm/include/asm/syscall.h | 7 ++- arch/arm/include/asm/thread_info.h | 17 +------ arch/arm/include/asm/traps.h | 1 - arch/arm/include/uapi/asm/ptrace.h | 2 + arch/arm/kernel/entry-armv.S | 50 ++++--------------- arch/arm/kernel/entry-common.S | 34 +++---------- arch/arm/kernel/entry-header.S | 18 ------- arch/arm/kernel/entry.c | 94 ++++++++++++++++++------------------ arch/arm/kernel/process.c | 5 +- arch/arm/kernel/ptrace.c | 95 ++----------------------------------- arch/arm/kernel/signal.c | 31 +----------- arch/arm/kernel/syscall.c | 30 ++++-------- arch/arm/kernel/traps.c | 25 ---------- 18 files changed, 161 insertions(+), 333 deletions(-) diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig index 835b5f100e926e54bab4be7802aeada4af37a145..564575f52cf7ec8296938944fa5296a4acc2a598 100644 --- a/arch/arm/Kconfig +++ b/arch/arm/Kconfig @@ -69,6 +69,7 @@ config ARM select GENERIC_CPU_AUTOPROBE select GENERIC_CPU_DEVICES select GENERIC_EARLY_IOREMAP + select GENERIC_ENTRY select GENERIC_IDLE_POLL_SETUP select GENERIC_IRQ_MULTI_HANDLER select GENERIC_IRQ_PROBE diff --git a/arch/arm/include/asm/entry-common.h b/arch/arm/include/asm/entry-common.h new file mode 100644 index 0000000000000000000000000000000000000000..7b1f108512db0001a65911af0df4f48197d99e75 --- /dev/null +++ b/arch/arm/include/asm/entry-common.h @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _ASM_ARM_ENTRY_COMMON_H +#define _ASM_ARM_ENTRY_COMMON_H + +#include + +#include + +enum ptrace_syscall_dir { + PTRACE_SYSCALL_ENTER = 0, + PTRACE_SYSCALL_EXIT, +}; + +static inline unsigned long +arch_prepare_report_syscall_entry(struct pt_regs *regs) +{ + unsigned long ip; + + /* + * IP is used to denote syscall entry/exit: + * IP = 0 -> entry + */ + ip = regs->ARM_ip; + regs->ARM_ip = PTRACE_SYSCALL_ENTER; + + return ip; +} +#define arch_prepare_report_syscall_entry arch_prepare_report_syscall_entry + +static inline void +arch_post_report_syscall_entry(struct pt_regs *regs, + unsigned long saved_reg, long ret) +{ + regs->ARM_ip = saved_reg; +} +#define arch_post_report_syscall_entry arch_post_report_syscall_entry + + +static inline unsigned long +arch_prepare_report_syscall_exit(struct pt_regs *regs, + unsigned long work) +{ + unsigned long ip; + + /* + * IP is used to denote syscall entry/exit: + * IP = 1 -> exit + */ + ip = regs->ARM_ip; + regs->ARM_ip = PTRACE_SYSCALL_EXIT; + + return ip; +} +#define arch_prepare_report_syscall_exit arch_prepare_report_syscall_exit + +static inline void +arch_post_report_syscall_exit(struct pt_regs *regs, + unsigned long saved_reg, + unsigned long work) +{ + regs->ARM_ip = saved_reg; +} +#define arch_post_report_syscall_exit arch_post_report_syscall_exit + +#endif /* _ASM_ARM_ENTRY_COMMON_H */ diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index a78bc5054b093120e903f08369fb8b1a9feb7f81..df2dd14ab51a586b83b6da6bbf9bd99858c1ddf4 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -4,15 +4,8 @@ struct pt_regs; -/* - * These are copies of generic entry headers so we can transition - * to generic entry once they are semantically equivalent. - */ -long syscall_enter_from_user_mode(struct pt_regs *regs, long); -void syscall_exit_to_user_mode(struct pt_regs *regs); -void irqentry_enter_from_user_mode(struct pt_regs *regs); -void irqentry_exit_to_user_mode(struct pt_regs *regs); -void irqentry_enter_from_kernel_mode(struct pt_regs *regs); -void irqentry_exit_to_kernel_mode(struct pt_regs *regs); +void arm_irq_handler(struct pt_regs *regs, int mode); +void arm_fiq_handler(struct pt_regs *regs); +void arm_exit_to_user_mode(struct pt_regs *regs); #endif /* __ASM_ENTRY_H__ */ diff --git a/arch/arm/include/asm/ptrace.h b/arch/arm/include/asm/ptrace.h index a01c66f0ad907882af2a383e53de8aca68c551e5..8a7de4020cbc016f743dcf515b9fa83b9bd3e3f0 100644 --- a/arch/arm/include/asm/ptrace.h +++ b/arch/arm/include/asm/ptrace.h @@ -200,8 +200,5 @@ static inline unsigned long it_advance(unsigned long cpsr) return cpsr; } -int syscall_trace_enter(struct pt_regs *regs); -void syscall_trace_exit(struct pt_regs *regs); - #endif /* __ASSEMBLY__ */ #endif diff --git a/arch/arm/include/asm/signal.h b/arch/arm/include/asm/signal.h index bb753cd3e29fc54186543c48b2aee7b221301a37..45885383646c9f166b0d7ee80080b78776b515e7 100644 --- a/arch/arm/include/asm/signal.h +++ b/arch/arm/include/asm/signal.h @@ -23,6 +23,4 @@ typedef struct { #include -void do_work_pending(struct pt_regs *regs, unsigned int thread_flags); - #endif diff --git a/arch/arm/include/asm/syscall.h b/arch/arm/include/asm/syscall.h index b94fd7ce17f82f0baf06b74800245112e1f13002..a6c8f0e799bb8bf3833161dec70fc7a1aaa513f6 100644 --- a/arch/arm/include/asm/syscall.h +++ b/arch/arm/include/asm/syscall.h @@ -19,7 +19,12 @@ extern const unsigned long sys_call_table[]; -int invoke_syscall(void *table, struct pt_regs *regs, int scno); +void invoke_syscall(void *table, struct pt_regs *regs, int scno); + +static inline bool arch_syscall_is_vdso_sigreturn(struct pt_regs *regs) +{ + return false; +} static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) diff --git a/arch/arm/include/asm/thread_info.h b/arch/arm/include/asm/thread_info.h index d8a45c5a10496aaf806bfeaa0353d5e8985bd6f5..84e58a9cdab63ad264c2cd2bad64239d1912cbe7 100644 --- a/arch/arm/include/asm/thread_info.h +++ b/arch/arm/include/asm/thread_info.h @@ -74,6 +74,7 @@ struct thread_info { #ifdef CONFIG_ARM_THUMBEE unsigned long thumbee_state; /* ThumbEE Handler Base register */ #endif + unsigned long syscall_work; /* SYSCALL_WORK_ flags */ }; #define INIT_THREAD_INFO(tsk) \ @@ -149,30 +150,14 @@ extern int vfp_restore_user_hwstate(struct user_vfp *, #define TIF_USING_IWMMXT 17 #define TIF_MEMDIE 18 /* is terminating due to OOM killer */ #define TIF_RESTORE_SIGMASK 19 -#define TIF_SYSCALL_TRACE 20 /* syscall trace active */ -#define TIF_SYSCALL_AUDIT 21 /* syscall auditing active */ -#define TIF_SYSCALL_TRACEPOINT 22 /* syscall tracepoint instrumentation */ -#define TIF_SECCOMP 23 /* seccomp syscall filtering active */ - #define _TIF_SIGPENDING (1 << TIF_SIGPENDING) #define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED) #define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME) #define _TIF_UPROBE (1 << TIF_UPROBE) -#define _TIF_SYSCALL_TRACE (1 << TIF_SYSCALL_TRACE) -#define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT) -#define _TIF_SYSCALL_TRACEPOINT (1 << TIF_SYSCALL_TRACEPOINT) -#define _TIF_SECCOMP (1 << TIF_SECCOMP) #define _TIF_NOTIFY_SIGNAL (1 << TIF_NOTIFY_SIGNAL) #define _TIF_USING_IWMMXT (1 << TIF_USING_IWMMXT) -/* Checks for any syscall work in entry-common.S */ -#define _TIF_SYSCALL_WORK (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_AUDIT | \ - _TIF_SYSCALL_TRACEPOINT | _TIF_SECCOMP) - -/* - * Change these and you break ASM code in entry-common.S - */ #define _TIF_WORK_MASK (_TIF_NEED_RESCHED | _TIF_SIGPENDING | \ _TIF_NOTIFY_RESUME | _TIF_UPROBE | \ _TIF_NOTIFY_SIGNAL) diff --git a/arch/arm/include/asm/traps.h b/arch/arm/include/asm/traps.h index b888912c2450e0e8eec8139166c7292f283545c7..f8695104c72f2f450092839c7c50920e540e594b 100644 --- a/arch/arm/include/asm/traps.h +++ b/arch/arm/include/asm/traps.h @@ -38,7 +38,6 @@ extern void *vectors_page; asmlinkage void dump_backtrace_stm(u32 *stack, u32 instruction, const char *loglvl); asmlinkage void do_undefinstr(struct pt_regs *regs); -asmlinkage void handle_fiq_as_nmi(struct pt_regs *regs); asmlinkage void bad_mode(struct pt_regs *regs, int reason); int arm_syscall(int no, struct pt_regs *regs); asmlinkage void baddataabort(int code, unsigned long instr, struct pt_regs *regs); diff --git a/arch/arm/include/uapi/asm/ptrace.h b/arch/arm/include/uapi/asm/ptrace.h index 8896c23ccba78ed7e10c7d1b9955124314496dc6..336a5beb282f360e875703045ee833974bbd68b2 100644 --- a/arch/arm/include/uapi/asm/ptrace.h +++ b/arch/arm/include/uapi/asm/ptrace.h @@ -33,6 +33,8 @@ #define PTRACE_GETHBPREGS 29 #define PTRACE_SETHBPREGS 30 #define PTRACE_GETFDPIC 31 +#define PTRACE_SYSEMU 32 +#define PTRACE_SYSEMU_SINGLESTEP 33 #define PTRACE_GETFDPIC_EXEC 0 #define PTRACE_GETFDPIC_INTERP 1 diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 2a789c8834b93475c32dcb6ba5854e24ddd8d6e9..498a22f5fa8a6516d1eab584f0523c3d6c6e4926 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -36,35 +36,6 @@ #define RELOC_TEXT_NONE #endif -/* - * Interrupt handling. - */ - .macro irq_handler, from_user:req - mov r1, sp - ldr_this_cpu r2, irq_stack_ptr, r2, r3 - .if \from_user == 0 - @ - @ If we took the interrupt while running in the kernel, we may already - @ be using the IRQ stack, so revert to the original value in that case. - @ - subs r3, r2, r1 @ SP above bottom of IRQ stack? - rsbscs r3, r3, #THREAD_SIZE @ ... and below the top? -#ifdef CONFIG_VMAP_STACK - ldr_va r3, high_memory, cc @ End of the linear region - cmpcc r3, r1 @ Stack pointer was below it? -#endif - bcc 0f @ If not, switch to the IRQ stack - mov r0, r1 - bl generic_handle_arch_irq - b 1f -0: - .endif - - mov_l r0, generic_handle_arch_irq - bl call_with_stack -1: - .endm - .macro pabt_helper @ PABORT handler takes pt_regs in r2, fault address in r4 and psr in r5 #ifdef MULTI_PABORT @@ -205,9 +176,6 @@ ENDPROC(__und_invalid) get_thread_info tsk uaccess_entry tsk, r0, r1, r2, \uaccess - mov r0, sp @ 'regs' - bl irqentry_enter_from_kernel_mode - .endm .align 5 @@ -223,7 +191,9 @@ ENDPROC(__dabt_svc) .align 5 __irq_svc: svc_entry - irq_handler from_user=0 + mov r0, sp @ regs + mov r1, #0 @ from kernel mode + bl arm_irq_handler #ifdef CONFIG_PREEMPTION ldr r8, [tsk, #TI_PREEMPT] @ get preempt count @@ -300,7 +270,7 @@ ENDPROC(__pabt_svc) __fiq_svc: svc_entry mov r0, sp @ struct pt_regs *regs - bl handle_fiq_as_nmi + bl arm_fiq_handler svc_exit_via_fiq UNWIND(.fnend ) ENDPROC(__fiq_svc) @@ -329,7 +299,7 @@ __fiq_abt: stmfd sp!, {r1 - r2} add r0, sp, #8 @ struct pt_regs *regs - bl handle_fiq_as_nmi + bl arm_fiq_handler ldmfd sp!, {r1 - r2} ARM( msr cpsr_c, #ABT_MODE | PSR_I_BIT | PSR_F_BIT ) @@ -420,7 +390,6 @@ ENDPROC(__fiq_abt) .align 5 __dabt_usr: usr_entry uaccess=0 - asm_irqentry_enter_from_user_mode kuser_cmpxchg_check mov r2, sp dabt_helper @@ -431,9 +400,10 @@ ENDPROC(__dabt_usr) .align 5 __irq_usr: usr_entry - asm_irqentry_enter_from_user_mode kuser_cmpxchg_check - irq_handler from_user=1 + mov r0, sp @ regs + mov r1, #1 @ from user mode + bl arm_irq_handler get_thread_info tsk mov why, #0 b ret_to_user_from_irq @@ -445,7 +415,6 @@ ENDPROC(__irq_usr) .align 5 __und_usr: usr_entry uaccess=0 - asm_irqentry_enter_from_user_mode @ IRQs must be enabled before attempting to read the instruction from @ user space since that could cause a page/translation fault if the @@ -470,7 +439,6 @@ ENDPROC(__und_usr) .align 5 __pabt_usr: usr_entry - asm_irqentry_enter_from_user_mode mov r2, sp @ regs pabt_helper UNWIND(.fnend ) @@ -493,7 +461,7 @@ __fiq_usr: usr_entry kuser_cmpxchg_check mov r0, sp @ struct pt_regs *regs - bl handle_fiq_as_nmi + bl arm_fiq_handler get_thread_info tsk restore_user_regs UNWIND(.fnend ) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index 6b0f86786a7d9e656ecbe8c741f2edbcbafd2915..a5a0e757ab23d151485092a078af98966e039869 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -39,10 +39,6 @@ ret_fast_syscall: UNWIND(.fnstart ) UNWIND(.cantunwind ) add sp, sp, #(S_R0 + S_OFF) - /* do_rseq_syscall needs interrupts enabled. */ - mov r0, sp @ 'regs' - bl syscall_exit_to_user_mode - #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack #endif @@ -60,7 +56,7 @@ ENTRY(ret_to_user) /* do_rseq_syscall needs interrupts enabled. */ enable_irq_notrace @ enable interrupts mov r0, sp @ 'regs' - bl syscall_exit_to_user_mode + bl arm_exit_to_user_mode #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack @@ -69,8 +65,6 @@ ENTRY(ret_to_user) ENDPROC(ret_to_user) ENTRY(ret_to_user_from_irq) - asm_irqentry_exit_to_user_mode - #ifdef CONFIG_GCC_PLUGIN_STACKLEAK bl stackleak_erase_on_task_stack #endif @@ -92,9 +86,10 @@ SYM_TYPED_FUNC_START(ret_from_fork_asm) mov r2, r5 mov r3, r4 bl ret_from_fork - mov r0, #0 -1: get_thread_info tsk - b ret_to_user +#ifdef CONFIG_GCC_PLUGIN_STACKLEAK + bl stackleak_erase_on_task_stack +#endif + restore_user_regs SYM_FUNC_END(ret_from_fork_asm) /*============================================================================= @@ -210,29 +205,12 @@ ENTRY(vector_swi) #else str scno, [tsk, #TI_ABI_SYSCALL] #endif - - /* - * Calling out to C to be careful to save and restore registers. - * This call could modify the syscall number. scno is r7 so we - * do not save and restore r7. - */ - mov r0, sp @ regs - mov r1, scno - push {r4 - r6, r8 - r10, lr} - bl syscall_enter_from_user_mode - pop {r4 - r6, r8 - r10, lr} - mov scno, r0 - mov r1, sp @ put regs into r1 stmdb sp!, {r4, r5} @ push fifth and sixth args mov r0, tbl mov r2, scno @ syscall number from r7 bl invoke_syscall - cmp r0, #0 - beq ret_fast_syscall - /* This path taken when tracing */ - add sp, sp, #(S_R0 + S_OFF) - b ret_to_user + b ret_fast_syscall #if defined(CONFIG_OABI_COMPAT) || !defined(CONFIG_AEABI) /* diff --git a/arch/arm/kernel/entry-header.S b/arch/arm/kernel/entry-header.S index cfaf14d71378ba14bbb2a42cd36d48a23838eee1..abd927f1cc3a4d813cc48bc046191f49f3c0e533 100644 --- a/arch/arm/kernel/entry-header.S +++ b/arch/arm/kernel/entry-header.S @@ -203,10 +203,6 @@ @ IRQs off again before pulling preserved data off the stack disable_irq_notrace - - mov r0, sp @ 'regs' - bl irqentry_exit_to_kernel_mode - uaccess_exit tsk, r0, r1 #ifndef CONFIG_THUMB2_KERNEL @@ -340,20 +336,6 @@ ALT_UP_B(.L1_\@) #endif /* !CONFIG_THUMB2_KERNEL */ .endm -/* - * Context tracking and other mode transitions. Used to instrument transitions - * between user and kernel mode. -*/ - .macro asm_irqentry_enter_from_user_mode - mov r0, sp @ regs - bl irqentry_enter_from_user_mode - .endm - - .macro asm_irqentry_exit_to_user_mode - mov r0, sp @ regs - bl irqentry_exit_to_user_mode - .endm - /* * These are the registers used in the syscall handler, and allow us to * have in theory up to 7 arguments to a function - r0 to r6. diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 1e1284cc4caed6e602ce36e812d535e6fe324f34..08b8680233e5aae6e477f9bf15d7f06d7fd3e5af 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -1,71 +1,71 @@ // SPDX-License-Identifier: GPL-2.0 #include -#include -#include #include +#include +#include +#include #include +#include #include +#include -long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall) -{ - trace_hardirqs_on(); - local_irq_enable(); - /* This context tracking call has inverse naming */ - user_exit_callable(); - - /* This will optionally be modified later */ - return syscall; -} +#include "irq.h" -void syscall_exit_to_user_mode(struct pt_regs *regs) +static void noinstr handle_arm_irq(void *data) { - unsigned long flags = read_thread_flags(); + struct pt_regs *regs = data; + struct pt_regs *old_regs; - rseq_syscall(regs); - local_irq_disable(); - /* - * It really matters that we check for flags != 0 and not - * just for pending work here! - */ - if (flags) - do_work_pending(regs, flags); + irq_enter_rcu(); + old_regs = set_irq_regs(regs); - trace_hardirqs_on(); - /* This context tracking call has inverse naming */ - user_enter_callable(); -} + handle_arch_irq(regs); -noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs) -{ - trace_hardirqs_off(); - /* This context tracking call has inverse naming */ - user_exit_callable(); + set_irq_regs(old_regs); + irq_exit_rcu(); } -noinstr void irqentry_exit_to_user_mode(struct pt_regs *regs) +noinstr void arm_irq_handler(struct pt_regs *regs, int mode) { - unsigned long flags = read_thread_flags(); + irqentry_state_t state = irqentry_enter(regs); /* - * It really matters that we check for flags != 0 and not - * just for pending work here! + * If we are executing in usermode, or kernel process context + * (on the thread stack) then switch to the IRQ stack. Else we + * are already on the IRQ stack (or the overflow stack) and we + * can just proceed to handle the IRQ. */ - if (flags) - do_work_pending(regs, flags); - trace_hardirqs_on(); - /* This context tracking call has inverse naming */ - user_enter_callable(); + if (mode == 1) + call_on_irq_stack(handle_arm_irq, regs); + else if (on_thread_stack()) + call_on_irq_stack(handle_arm_irq, regs); + else + handle_arm_irq(regs); + + irqentry_exit(regs, state); } -noinstr void irqentry_enter_from_kernel_mode(struct pt_regs *regs) +/* + * Handle FIQ similarly to NMI on x86 systems. + * + * The runtime environment for NMIs is extremely restrictive + * (NMIs can pre-empt critical sections meaning almost all locking is + * forbidden) meaning this default FIQ handling must only be used in + * circumstances where non-maskability improves robustness, such as + * watchdog or debug logic. + * + * This handler is not appropriate for general purpose use in drivers + * platform code and can be overrideen using set_fiq_handler. + */ +noinstr void arm_fiq_handler(struct pt_regs *regs) { - trace_hardirqs_off(); + irqentry_state_t state = irqentry_nmi_enter(regs); + + irqentry_nmi_exit(regs, state); } -noinstr void irqentry_exit_to_kernel_mode(struct pt_regs *regs) +asmlinkage void arm_exit_to_user_mode(struct pt_regs *regs) { - if (interrupts_enabled(regs)) - trace_hardirqs_on(); - else - trace_hardirqs_off(); + local_irq_disable(); + irqentry_exit_to_user_mode(regs); } diff --git a/arch/arm/kernel/process.c b/arch/arm/kernel/process.c index 81c1c3f988344185917cf4e53d0d8ee47ae912f4..a59ad5fdaaf51f653c352884bbdd10d1211b2669 100644 --- a/arch/arm/kernel/process.c +++ b/arch/arm/kernel/process.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include @@ -248,8 +249,8 @@ __visible void ret_from_fork(struct task_struct *prev, struct pt_regs *regs, * execve() syscall. */ } - - /* syscall_exit_to_user_mode(regs); here once we switch to generic entry */ + local_irq_enable(); + syscall_exit_to_user_mode(regs); } int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) diff --git a/arch/arm/kernel/ptrace.c b/arch/arm/kernel/ptrace.c index ac7b98ae47249b4a00fc1cb871bb2d309bfd8c88..d1e5a78c616b0e71f9466c201b74e06ce0d7a08f 100644 --- a/arch/arm/kernel/ptrace.c +++ b/arch/arm/kernel/ptrace.c @@ -27,9 +27,6 @@ #include #include -#define CREATE_TRACE_POINTS -#include - #define REG_PC 15 #define REG_PSR 16 /* @@ -737,6 +734,11 @@ long arch_ptrace(struct task_struct *child, long request, ret = ptrace_write_user(child, addr, data); break; + case PTRACE_SYSEMU: + case PTRACE_SYSEMU_SINGLESTEP: + ret = -EIO; + break; + case PTRACE_GETREGS: ret = copy_regset_to_user(child, &user_arm_view, REGSET_GPR, @@ -821,90 +823,3 @@ long arch_ptrace(struct task_struct *child, long request, return ret; } - -enum ptrace_syscall_dir { - PTRACE_SYSCALL_ENTER = 0, - PTRACE_SYSCALL_EXIT, -}; - -static void report_syscall_enter(struct pt_regs *regs) -{ - unsigned long ip; - - /* - * IP is used to denote syscall entry/exit: - * IP = 0 -> entry - */ - ip = regs->ARM_ip; - regs->ARM_ip = PTRACE_SYSCALL_ENTER; - - if (ptrace_report_syscall_entry(regs)) - current_thread_info()->abi_syscall = -1; - - regs->ARM_ip = ip; -} - -static void report_syscall_exit(struct pt_regs *regs) -{ - unsigned long ip; - - /* - * IP is used to denote syscall entry/exit: - * IP = 1 -> exit - */ - ip = regs->ARM_ip; - regs->ARM_ip = PTRACE_SYSCALL_EXIT; - - ptrace_report_syscall_exit(regs, 0); - - regs->ARM_ip = ip; -} - -asmlinkage int syscall_trace_enter(struct pt_regs *regs) -{ - int scno; - - if (test_thread_flag(TIF_SYSCALL_TRACE)) - report_syscall_enter(regs); - - /* Do seccomp after ptrace; syscall may have changed. */ -#ifdef CONFIG_HAVE_ARCH_SECCOMP_FILTER - if (secure_computing() == -1) - return -1; -#else - /* XXX: remove this once OABI gets fixed */ - secure_computing_strict(syscall_get_nr(current, regs)); -#endif - - /* Tracer or seccomp may have changed syscall. */ - scno = syscall_get_nr(current, regs); - - if (test_thread_flag(TIF_SYSCALL_TRACEPOINT)) - trace_sys_enter(regs, scno); - - audit_syscall_entry(scno, regs->ARM_r0, regs->ARM_r1, regs->ARM_r2, - regs->ARM_r3); - - return scno; -} - -void syscall_trace_exit(struct pt_regs *regs) -{ - /* - * Audit the syscall before anything else, as a debugger may - * come in and change the current registers. - */ - audit_syscall_exit(regs); - - /* - * Note that we haven't updated the ->syscall field for the - * current thread. This isn't a problem because it will have - * been set on syscall entry and there hasn't been an opportunity - * for a PTRACE_SET_SYSCALL since then. - */ - if (test_thread_flag(TIF_SYSCALL_TRACEPOINT)) - trace_sys_exit(regs, regs_return_value(regs)); - - if (test_thread_flag(TIF_SYSCALL_TRACE)) - report_syscall_exit(regs); -} diff --git a/arch/arm/kernel/signal.c b/arch/arm/kernel/signal.c index a39ee14ec5b70b3738ecef537709326ef1e8217c..35d2bb3cd2b442dac164548037262e065fbfe12a 100644 --- a/arch/arm/kernel/signal.c +++ b/arch/arm/kernel/signal.c @@ -4,6 +4,7 @@ * * Copyright (C) 1995-2009 Russell King */ +#include #include #include #include @@ -535,7 +536,7 @@ static void handle_signal(struct ksignal *ksig, struct pt_regs *regs) * the kernel can handle, and then we build all the user-level signal handling * stack-frames in one go after that. */ -static void arch_do_signal_or_restart(struct pt_regs *regs) +void arch_do_signal_or_restart(struct pt_regs *regs) { unsigned int retval = 0, continue_addr = 0, restart_addr = 0; bool syscall = (syscall_get_nr(current, regs) != -1); @@ -598,34 +599,6 @@ static void arch_do_signal_or_restart(struct pt_regs *regs) return; } -void do_work_pending(struct pt_regs *regs, unsigned int thread_flags) -{ - /* - * The assembly code enters us with IRQs off, but it hasn't - * informed the tracing code of that for efficiency reasons. - * Update the trace code with the current status. - */ - trace_hardirqs_off(); - do { - if (likely(thread_flags & _TIF_NEED_RESCHED)) { - schedule(); - } else { - if (unlikely(!user_mode(regs))) - return; - local_irq_enable(); - if (thread_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL)) { - arch_do_signal_or_restart(regs); - } else if (thread_flags & _TIF_UPROBE) { - uprobe_notify_resume(regs); - } else { - resume_user_mode_work(regs); - } - } - local_irq_disable(); - thread_flags = read_thread_flags(); - } while (thread_flags & _TIF_WORK_MASK); -} - struct page *get_signal_page(void) { unsigned long ptr; diff --git a/arch/arm/kernel/syscall.c b/arch/arm/kernel/syscall.c index 6f518ec3870fb36a64d1b671b20f96d63a52bf78..ed3ab51283c06c1398ece2ad3ee1fae16cd03ee8 100644 --- a/arch/arm/kernel/syscall.c +++ b/arch/arm/kernel/syscall.c @@ -1,25 +1,21 @@ // SPDX-License-Identifier: GPL-2.0 +#include #include #include -static inline bool has_syscall_work(unsigned long flags) -{ - return unlikely(flags & _TIF_SYSCALL_WORK); -} - int invoke_syscall_asm(void *table, struct pt_regs *regs, int scno); __ADDRESSABLE(invoke_syscall_asm); -__visible int invoke_syscall(void *table, struct pt_regs *regs, int scno) +__visible void invoke_syscall(void *table, struct pt_regs *regs, int scno) { - unsigned long flags = read_thread_flags(); int ret; - if (has_syscall_work(flags)) { - scno = syscall_trace_enter(regs); - if (scno == -1) - goto trace_exit_nosave; + scno = syscall_enter_from_user_mode(regs, scno); + /* When tracing syscall -1 means "skip syscall" */ + if (scno < 0) { + ret = 0; + goto exit_save; } if (scno < NR_syscalls) { @@ -35,13 +31,7 @@ __visible int invoke_syscall(void *table, struct pt_regs *regs, int scno) ret = sys_ni_syscall(); exit_save: - /* Save return value from syscall */ - regs->ARM_r0 = ret; - if (!has_syscall_work(flags)) - return 0; - -trace_exit_nosave: - local_irq_enable(); - syscall_trace_exit(regs); - return 1; + syscall_set_return_value(current, regs, 0, ret); + + syscall_exit_to_user_mode(regs); } diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index 942c77767919466d5b89d83d8eb77032b119473e..57e74b49c52533babe177a8a2e0160a83c9bde05 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -502,31 +502,6 @@ asmlinkage void do_undefinstr(struct pt_regs *regs) } NOKPROBE_SYMBOL(do_undefinstr) -/* - * Handle FIQ similarly to NMI on x86 systems. - * - * The runtime environment for NMIs is extremely restrictive - * (NMIs can pre-empt critical sections meaning almost all locking is - * forbidden) meaning this default FIQ handling must only be used in - * circumstances where non-maskability improves robustness, such as - * watchdog or debug logic. - * - * This handler is not appropriate for general purpose use in drivers - * platform code and can be overrideen using set_fiq_handler. - */ -asmlinkage void __exception_irq_entry handle_fiq_as_nmi(struct pt_regs *regs) -{ - struct pt_regs *old_regs = set_irq_regs(regs); - - nmi_enter(); - - /* nop. FIQ handlers for special arch/arm features can be added here. */ - - nmi_exit(); - - set_irq_regs(old_regs); -} - /* * bad_mode handles the impossible case in the vectors. If you see one of * these, then it's extremely serious, and could mean you have buggy hardware. From patchwork Wed Feb 12 11:23:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971826 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DBE61C02198 for ; Wed, 12 Feb 2025 12:07:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=hG0/vjAYb5Ulc6jjnKhm2XeOXEYOwE4Q8HmW4VgPla8=; b=KYvsWi7UZyZb/5vhk8egTErTxo dupiUsOgd91WaV924Wjq2eqWVZVyt9nhvxj/1muPDQoP7y/B5CaFjeIYVoSTVv/EhwKHHjrGnHtJ4 m5nJHn/OgaEvPWt/4edOVn28TcjPM6ve/jI7ZjwTb+fabcrwRFYimMFpgXuQHP3N2ArRzaLVuA1Ci GX71buHmMdTvkXdDl3DreEOfQpG8soTDs7+yBNRP1S6cYKhdEMlXvbp8DNCsGCalF2zEGqp1B9shf 9bxzCz2leuofv5rhM+kZzSrMgUBJ0PRml1O93o3MmoAZIAUX6Ul+lZMpDpmLrHwW41AFfsCoyjM4X 4/9TeAfg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBW7-00000007IG2-2bP3; Wed, 12 Feb 2025 12:06:59 +0000 Received: from mail-lj1-f172.google.com ([209.85.208.172]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAqA-000000079mL-2C3Q for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:40 +0000 Received: by mail-lj1-f172.google.com with SMTP id 38308e7fff4ca-30797730cbdso65114181fa.3 for ; Wed, 12 Feb 2025 03:23:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359417; x=1739964217; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=hG0/vjAYb5Ulc6jjnKhm2XeOXEYOwE4Q8HmW4VgPla8=; b=PBshQgdoqdvZDwuFzVHGsa9Q1PkZvliZdN2cxWVNaxOV6tAddlOi5P/OSTLmiPhEH2 DGxd/SP5XAefSTVsjhNgEXvWnGKC2xXWd7/eIo2HGtvVz44d89D8GHgZTicBH8lHixUi zq1JpfW8wN5vfUTUPo0F6b/gyQoIloyczHz+d2VRG52xO3B1Vc+CnzQRjwfjHIdBKuSX QtoJJQxkKWFiyemHzGKt9BgwWoDkyFe1iSSKyK8spa4frqeDRRm5+M8IU1SUYm3W0ysJ eMn1wFJ5m6U1PsalXU3/RwIWoCaryUpoBo93zl8Xp0KmdVTN4e1KoQv2M2Iwq3pPXo6e oS2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359417; x=1739964217; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hG0/vjAYb5Ulc6jjnKhm2XeOXEYOwE4Q8HmW4VgPla8=; b=IX0gQQjGTsuSofir+Gxwg7sjJU6lYStiiaZLvOevwmKZmNQ1x4oCUI4n/JEaKKxN+Z j/p2jMYbiVjr6yJvz9lRyfrN9uNBr5lpxxCx8s0HZEMoyWPpYaky1493AgSlVWUeSsCa 3sFcjlUXM9FXdo8jC90rxQXTUCqecf/NqJEVjolRqfL3zsuOD65v0wdsQI0J52MHIMzt DwvnqplLGZ6yTkX3z1gAja0FwRgshoqxz+pwJSO90bmCENC7xoAbKp3ZGoEuOSSQIrWU cs6Ln9d748gPzr2B20QzmArS9u4TaVJhYguH3PxzTrbK/CGOKrnm2LuAJ4V6OoFgS8nF EEpg== X-Gm-Message-State: AOJu0YxNJyR3OO6PrZM2OjyPT52C9qULc9CWaM9iHUKTb8Ig44tSLdpr ficz3pWszn5XFgnH8+P6ILM9kWe/eNdZoIMT7yzOHZ70jh20KTU85Rk0PgMlvZg= X-Gm-Gg: ASbGncsqc2gTb/YQGkRyjeotXTWuKOMMzx2fAt70SnB6DtMFD/tbh6zrAzkFDUNxdRk w+g5+vmzs6zHjPVyoj1HCtlkX1oC3VKBuKBr3lFOFUdjx6b3JY6BJxRpUmsk74hr4K0WixE/aTj ej+Nqi2+Hm4APdDuh9/6WJ64LjXG6YRU2YC8MkG9KOBPXQfIciwE8N7+34hjGL4OZ0Lwt9qzlMH 8EJU7NsActMyxm+jY1cmyvhbsfINKCwGmxPrZm8GykL0Rb+lC/oYO39z7advhdcKPKgOOu02DHs IbMvcedQ19g88iONkhliFV42bA== X-Google-Smtp-Source: AGHT+IGP37hz0YvESaBREy9HQSg86iSAViPZV6mIcF+1tjXX1uTruztxqRmlhZCOlY4ljwki9BmazQ== X-Received: by 2002:a05:651c:4ca:b0:300:1f2d:97a with SMTP id 38308e7fff4ca-309036b6cb8mr9621591fa.16.1739359416424; Wed, 12 Feb 2025 03:23:36 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:35 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:23 +0100 Subject: [PATCH v4 29/31] ARM: entry: Handle dabt, pabt, and und as interrupts MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-29-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032338_724115_FF7FDF05 X-CRM114-Status: GOOD ( 21.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org While it isn't entirely intuitive, it appears that any kind of exception such as data or prefetch abort ("page faults") need to be handled as some kind of "interrupts" when using generic entry. At least this is what other platforms are doing. The same goes for undefined instruction handling, i.e. floating point emulation in the kernel (und-exceptions). This is necessary for the context checking to pass: without this patch, a whole slew of warnings start to trigger from syscall_exit_to_user_mode_prepare() CT_WARN_ON(ct_state() != CT_STATE_KERNEL), i.e. syscalls seems to exit from user mode to user mode (not good), because the page faults screws up the context tracker. This patch restores the order. If this seems like the previous patch introduces a regression that is then fixed in this patch, it can simply be squashed into the former: having this rewrite separately surely makes development and review easier. Signed-off-by: Linus Walleij --- arch/arm/include/asm/entry.h | 3 +++ arch/arm/include/asm/traps.h | 2 +- arch/arm/kernel/entry-armv.S | 7 +++---- arch/arm/kernel/entry.c | 31 +++++++++++++++++++++++++++++++ arch/arm/kernel/traps.c | 2 +- arch/arm/mm/abort-ev4.S | 2 +- arch/arm/mm/abort-ev4t.S | 2 +- arch/arm/mm/abort-ev5t.S | 4 ++-- arch/arm/mm/abort-ev5tj.S | 6 +++--- arch/arm/mm/abort-ev6.S | 2 +- arch/arm/mm/abort-ev7.S | 2 +- arch/arm/mm/abort-lv4t.S | 36 ++++++++++++++++++------------------ arch/arm/mm/abort-macro.S | 2 +- arch/arm/mm/abort-nommu.S | 2 +- arch/arm/mm/fault.c | 4 ++-- arch/arm/mm/fault.h | 8 ++++---- arch/arm/mm/pabort-legacy.S | 2 +- arch/arm/mm/pabort-v6.S | 2 +- arch/arm/mm/pabort-v7.S | 2 +- 19 files changed, 77 insertions(+), 44 deletions(-) diff --git a/arch/arm/include/asm/entry.h b/arch/arm/include/asm/entry.h index df2dd14ab51a586b83b6da6bbf9bd99858c1ddf4..de722084d5309194e02cb87db64ec32b9e6cd627 100644 --- a/arch/arm/include/asm/entry.h +++ b/arch/arm/include/asm/entry.h @@ -4,6 +4,9 @@ struct pt_regs; +void arm_und_handler(struct pt_regs *regs); +void arm_dabt_handler(unsigned long addr, unsigned int fsr, struct pt_regs *regs); +void arm_pabt_handler(unsigned long addr, unsigned int ifsr, struct pt_regs *regs); void arm_irq_handler(struct pt_regs *regs, int mode); void arm_fiq_handler(struct pt_regs *regs); void arm_exit_to_user_mode(struct pt_regs *regs); diff --git a/arch/arm/include/asm/traps.h b/arch/arm/include/asm/traps.h index f8695104c72f2f450092839c7c50920e540e594b..0ca98271e10a210225c1714f5b24668cebab2def 100644 --- a/arch/arm/include/asm/traps.h +++ b/arch/arm/include/asm/traps.h @@ -37,7 +37,7 @@ extern void ptrace_break(struct pt_regs *regs); extern void *vectors_page; asmlinkage void dump_backtrace_stm(u32 *stack, u32 instruction, const char *loglvl); -asmlinkage void do_undefinstr(struct pt_regs *regs); +void do_undefinstr(struct pt_regs *regs); asmlinkage void bad_mode(struct pt_regs *regs, int reason); int arm_syscall(int no, struct pt_regs *regs); asmlinkage void baddataabort(int code, unsigned long instr, struct pt_regs *regs); diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index 498a22f5fa8a6516d1eab584f0523c3d6c6e4926..ae2f952beea7611f0abc7bd299fc944335a21219 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -230,7 +230,7 @@ __und_fault: ldr r2, [r0, #S_PC] sub r2, r2, r1 str r2, [r0, #S_PC] - b do_undefinstr + b arm_und_handler ENDPROC(__und_fault) .align 5 @@ -449,9 +449,8 @@ __pabt_usr: ENTRY(ret_from_exception) UNWIND(.fnstart ) UNWIND(.cantunwind ) - get_thread_info tsk - mov why, #0 - b ret_to_user + disable_irq_notrace + b ret_to_user_from_irq UNWIND(.fnend ) ENDPROC(__pabt_usr) ENDPROC(ret_from_exception) diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 08b8680233e5aae6e477f9bf15d7f06d7fd3e5af..01e4339ccdb4951e04a89fb91ad7c6e1991e09af 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -8,8 +8,39 @@ #include #include #include +#include #include "irq.h" +#include "../mm/fault.h" + +noinstr asmlinkage void arm_und_handler(struct pt_regs *regs) +{ + irqentry_state_t state = irqentry_enter(regs); + + do_undefinstr(regs); + + irqentry_exit(regs, state); +} + +noinstr asmlinkage void arm_dabt_handler(unsigned long addr, unsigned int fsr, + struct pt_regs *regs) +{ + irqentry_state_t state = irqentry_enter(regs); + + do_DataAbort(addr, fsr, regs); + + irqentry_exit(regs, state); +} + +noinstr asmlinkage void arm_pabt_handler(unsigned long addr, unsigned int ifsr, + struct pt_regs *regs) +{ + irqentry_state_t state = irqentry_enter(regs); + + do_PrefetchAbort(addr, ifsr, regs); + + irqentry_exit(regs, state); +} static void noinstr handle_arm_irq(void *data) { diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index 57e74b49c52533babe177a8a2e0160a83c9bde05..d3a689bd05955f1ae46a6341e456bb097e831311 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -449,7 +449,7 @@ int call_undef_hook(struct pt_regs *regs, unsigned int instr) return fn ? fn(regs, instr) : 1; } -asmlinkage void do_undefinstr(struct pt_regs *regs) +noinstr void do_undefinstr(struct pt_regs *regs) { unsigned int instr; void __user *pc; diff --git a/arch/arm/mm/abort-ev4.S b/arch/arm/mm/abort-ev4.S index a10bcb89594dd38ce31ca30bf97d68cf421afc42..68d8ae2f4e9c0955c9f08112b4ff1887867d550a 100644 --- a/arch/arm/mm/abort-ev4.S +++ b/arch/arm/mm/abort-ev4.S @@ -24,4 +24,4 @@ ENTRY(v4_early_abort) bic r1, r1, #1 << 11 | 1 << 10 @ clear bits 11 and 10 of FSR tst r3, #1 << 20 @ L = 1 -> write? orreq r1, r1, #1 << 11 @ yes. - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev4t.S b/arch/arm/mm/abort-ev4t.S index 14743a2f6997fcae0eeb55f53dec07b7b2ec227c..76c6cfa79e2a32e34b6f35a15c71c1af407956f3 100644 --- a/arch/arm/mm/abort-ev4t.S +++ b/arch/arm/mm/abort-ev4t.S @@ -25,4 +25,4 @@ ENTRY(v4t_early_abort) bic r1, r1, #1 << 11 | 1 << 10 @ clear bits 11 and 10 of FSR tst r3, #1 << 20 @ check write orreq r1, r1, #1 << 11 - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev5t.S b/arch/arm/mm/abort-ev5t.S index 98c523118820798668bf04d065ad86ea05fb6d2c..fda1e54debc44b380fb3f3af95c1f198a9cc008a 100644 --- a/arch/arm/mm/abort-ev5t.S +++ b/arch/arm/mm/abort-ev5t.S @@ -25,7 +25,7 @@ ENTRY(v5t_early_abort) uaccess_disable ip @ disable user access bic r1, r1, #1 << 11 @ clear bits 11 of FSR teq_ldrd tmp=ip, insn=r3 @ insn was LDRD? - beq do_DataAbort @ yes + beq arm_dabt_handler @ yes tst r3, #1 << 20 @ check write orreq r1, r1, #1 << 11 - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev5tj.S b/arch/arm/mm/abort-ev5tj.S index fec72f4fbaf508597d826e58d0dc084ee6e58dd0..a786a7d69e6ae23aa1125ccd41deaaaa743601e1 100644 --- a/arch/arm/mm/abort-ev5tj.S +++ b/arch/arm/mm/abort-ev5tj.S @@ -22,12 +22,12 @@ ENTRY(v5tj_early_abort) mrc p15, 0, r0, c6, c0, 0 @ get FAR bic r1, r1, #1 << 11 | 1 << 10 @ clear bits 11 and 10 of FSR tst r5, #PSR_J_BIT @ Java? - bne do_DataAbort + bne arm_dabt_handler do_thumb_abort fsr=r1, pc=r4, psr=r5, tmp=r3 ldreq r3, [r4] @ read aborted ARM instruction uaccess_disable ip @ disable userspace access teq_ldrd tmp=ip, insn=r3 @ insn was LDRD? - beq do_DataAbort @ yes + beq arm_dabt_handler @ yes tst r3, #1 << 20 @ L = 0 -> write orreq r1, r1, #1 << 11 @ yes. - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev6.S b/arch/arm/mm/abort-ev6.S index 836dc1299243415faaac1a93dce35ac46ae1a7da..2366236053eb86484cb179ff572f06db6559426b 100644 --- a/arch/arm/mm/abort-ev6.S +++ b/arch/arm/mm/abort-ev6.S @@ -42,4 +42,4 @@ ENTRY(v6_early_abort) orreq r1, r1, #1 << 11 @ yes. #endif 1: uaccess_disable ip @ disable userspace access - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-ev7.S b/arch/arm/mm/abort-ev7.S index 53fb41c24774db985eddc370904aa7666527f538..72b6eba101555651d7afd749fd5d7715a2964372 100644 --- a/arch/arm/mm/abort-ev7.S +++ b/arch/arm/mm/abort-ev7.S @@ -18,5 +18,5 @@ ENTRY(v7_early_abort) mrc p15, 0, r1, c5, c0, 0 @ get FSR mrc p15, 0, r0, c6, c0, 0 @ get FAR uaccess_disable ip @ disable userspace access - b do_DataAbort + b arm_dabt_handler ENDPROC(v7_early_abort) diff --git a/arch/arm/mm/abort-lv4t.S b/arch/arm/mm/abort-lv4t.S index fbd60a120f6684c56c63cea10b00200765473f1d..3fef9c1479c17bb07652e43e0b8b0683e00c4b2f 100644 --- a/arch/arm/mm/abort-lv4t.S +++ b/arch/arm/mm/abort-lv4t.S @@ -46,8 +46,8 @@ ENTRY(v4t_late_abort) /* 9 */ b .data_arm_ldmstm @ ldm*b rn, /* a */ b .data_unknown /* b */ b .data_unknown -/* c */ b do_DataAbort @ ldc rd, [rn], #m @ Same as ldr rd, [rn], #m -/* d */ b do_DataAbort @ ldc rd, [rn, #m] +/* c */ b arm_dabt_handler @ ldc rd, [rn], #m @ Same as ldr rd, [rn], #m +/* d */ b arm_dabt_handler @ ldc rd, [rn, #m] /* e */ b .data_unknown /* f */ b .data_unknown @@ -60,7 +60,7 @@ ENTRY(v4t_late_abort) .data_arm_ldmstm: tst r8, #1 << 21 @ check writeback bit - beq do_DataAbort @ no writeback -> no fixup + beq arm_dabt_handler @ no writeback -> no fixup str r9, [sp, #-4]! mov r7, #0x11 orr r7, r7, #0x1100 @@ -81,11 +81,11 @@ ENTRY(v4t_late_abort) addeq r7, r7, r6, lsl #2 @ Undo decrement str r7, [r2, r9, lsr #14] @ Put register 'Rn' ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler .data_arm_lateldrhpre: tst r8, #1 << 21 @ Check writeback bit - beq do_DataAbort @ No writeback -> no fixup + beq arm_dabt_handler @ No writeback -> no fixup .data_arm_lateldrhpost: str r9, [sp, #-4]! and r9, r8, #0x00f @ get Rm / low nibble of immediate value @@ -101,14 +101,14 @@ ENTRY(v4t_late_abort) addeq r7, r7, r6 @ Undo decrement str r7, [r2, r9, lsr #14] @ Put register 'Rn' ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler .data_arm_lateldrpreconst: tst r8, #1 << 21 @ check writeback bit - beq do_DataAbort @ no writeback -> no fixup + beq arm_dabt_handler @ no writeback -> no fixup .data_arm_lateldrpostconst: movs r6, r8, lsl #20 @ Get offset - beq do_DataAbort @ zero -> no fixup + beq arm_dabt_handler @ zero -> no fixup str r9, [sp, #-4]! and r9, r8, #15 << 16 @ Extract 'n' from instruction ldr r7, [r2, r9, lsr #14] @ Get register 'Rn' @@ -117,11 +117,11 @@ ENTRY(v4t_late_abort) addeq r7, r7, r6, lsr #20 @ Undo decrement str r7, [r2, r9, lsr #14] @ Put register 'Rn' ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler .data_arm_lateldrprereg: tst r8, #1 << 21 @ check writeback bit - beq do_DataAbort @ no writeback -> no fixup + beq arm_dabt_handler @ no writeback -> no fixup .data_arm_lateldrpostreg: and r7, r8, #15 @ Extract 'm' from instruction ldr r6, [r2, r7, lsl #2] @ Get register 'Rm' @@ -180,10 +180,10 @@ ENTRY(v4t_late_abort) /* 3 */ b .data_unknown /* 4 */ b .data_unknown /* 5 */ b .data_thumb_reg -/* 6 */ b do_DataAbort -/* 7 */ b do_DataAbort -/* 8 */ b do_DataAbort -/* 9 */ b do_DataAbort +/* 6 */ b arm_dabt_handler +/* 7 */ b arm_dabt_handler +/* 8 */ b arm_dabt_handler +/* 9 */ b arm_dabt_handler /* A */ b .data_unknown /* B */ b .data_thumb_pushpop /* C */ b .data_thumb_ldmstm @@ -193,10 +193,10 @@ ENTRY(v4t_late_abort) .data_thumb_reg: tst r8, #1 << 9 - beq do_DataAbort + beq arm_dabt_handler tst r8, #1 << 10 @ If 'S' (signed) bit is set movne r1, #0 @ it must be a load instr - b do_DataAbort + b arm_dabt_handler .data_thumb_pushpop: tst r8, #1 << 10 @@ -217,7 +217,7 @@ ENTRY(v4t_late_abort) subne r7, r7, r6, lsl #2 @ decrement SP if POP str r7, [r2, #13 << 2] ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler .data_thumb_ldmstm: str r9, [sp, #-4]! @@ -234,4 +234,4 @@ ENTRY(v4t_late_abort) sub r7, r7, r6, lsl #2 @ always decrement str r7, [r2, r9, lsr #6] ldr r9, [sp], #4 - b do_DataAbort + b arm_dabt_handler diff --git a/arch/arm/mm/abort-macro.S b/arch/arm/mm/abort-macro.S index bacf53fd0b70c6307e74ef8601d8dcc7db292700..30bb5e2135862902951b5077b0b860fd12ed8c28 100644 --- a/arch/arm/mm/abort-macro.S +++ b/arch/arm/mm/abort-macro.S @@ -20,7 +20,7 @@ orreq \tmp, \tmp, #1 << 11 @ Set L-bit if yes tst \tmp, #1 << 11 @ L = 0 -> write orreq \fsr, \fsr, #1 << 11 @ yes. - b do_DataAbort + b arm_dabt_handler not_thumb: .endm diff --git a/arch/arm/mm/abort-nommu.S b/arch/arm/mm/abort-nommu.S index 6e2366a263219b379f4fdb43cc8e5413fb36e52a..573a09529ac7794f8004680ab350750de7c8e15d 100644 --- a/arch/arm/mm/abort-nommu.S +++ b/arch/arm/mm/abort-nommu.S @@ -17,5 +17,5 @@ ENTRY(nommu_early_abort) mov r0, #0 @ clear r0, r1 (no FSR/FAR) mov r1, #0 - b do_DataAbort + b arm_dabt_handler ENDPROC(nommu_early_abort) diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index ab01b51de5590bff6c2acaf446d01c589a5f7987..78afb701b34a17d8d841762944eb5f8af6843dc4 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -588,7 +588,7 @@ hook_fault_code(int nr, int (*fn)(unsigned long, unsigned int, struct pt_regs *) /* * Dispatch a data abort to the relevant handler. */ -asmlinkage void +void do_DataAbort(unsigned long addr, unsigned int fsr, struct pt_regs *regs) { const struct fsr_info *inf = fsr_info + fsr_fs(fsr); @@ -618,7 +618,7 @@ hook_ifault_code(int nr, int (*fn)(unsigned long, unsigned int, struct pt_regs * ifsr_info[nr].name = name; } -asmlinkage void +void do_PrefetchAbort(unsigned long addr, unsigned int ifsr, struct pt_regs *regs) { const struct fsr_info *inf = ifsr_info + fsr_fs(ifsr); diff --git a/arch/arm/mm/fault.h b/arch/arm/mm/fault.h index e8f8c190254442a015a6cbc179602589cfddb0ea..36a7e7138c310a8311458207d2b694cb44c63b93 100644 --- a/arch/arm/mm/fault.h +++ b/arch/arm/mm/fault.h @@ -37,9 +37,9 @@ static inline int fsr_fs(unsigned int fsr) void do_bad_area(unsigned long addr, unsigned int fsr, struct pt_regs *regs); void early_abt_enable(void); -asmlinkage void do_DataAbort(unsigned long addr, unsigned int fsr, - struct pt_regs *regs); -asmlinkage void do_PrefetchAbort(unsigned long addr, unsigned int ifsr, - struct pt_regs *regs); +void do_DataAbort(unsigned long addr, unsigned int fsr, + struct pt_regs *regs); +void do_PrefetchAbort(unsigned long addr, unsigned int ifsr, + struct pt_regs *regs); #endif /* __ARCH_ARM_FAULT_H */ diff --git a/arch/arm/mm/pabort-legacy.S b/arch/arm/mm/pabort-legacy.S index b2ffce4201062e3ec2045364ddc454cf706bab8d..26c62e568638d37133661f4f5b8dec430593f8eb 100644 --- a/arch/arm/mm/pabort-legacy.S +++ b/arch/arm/mm/pabort-legacy.S @@ -18,5 +18,5 @@ ENTRY(legacy_pabort) mov r0, r4 mov r1, #5 - b do_PrefetchAbort + b arm_pabt_handler ENDPROC(legacy_pabort) diff --git a/arch/arm/mm/pabort-v6.S b/arch/arm/mm/pabort-v6.S index 8686265dc9418b29381942bfd87a937a3234d46e..25abd11a35253bf464fe9fd7fc14fd11bc564dcd 100644 --- a/arch/arm/mm/pabort-v6.S +++ b/arch/arm/mm/pabort-v6.S @@ -18,5 +18,5 @@ ENTRY(v6_pabort) mov r0, r4 mrc p15, 0, r1, c5, c0, 1 @ get IFSR - b do_PrefetchAbort + b arm_pabt_handler ENDPROC(v6_pabort) diff --git a/arch/arm/mm/pabort-v7.S b/arch/arm/mm/pabort-v7.S index 9c70b1a21dc9204f24524df9905fbc077a82f2dc..e05c7d44d307adeba6759213374b2aa328c693da 100644 --- a/arch/arm/mm/pabort-v7.S +++ b/arch/arm/mm/pabort-v7.S @@ -18,5 +18,5 @@ ENTRY(v7_pabort) mrc p15, 0, r0, c6, c0, 2 @ get IFAR mrc p15, 0, r1, c5, c0, 1 @ get IFSR - b do_PrefetchAbort + b arm_pabt_handler ENDPROC(v7_pabort) From patchwork Wed Feb 12 11:23:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971827 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 71C33C02198 for ; Wed, 12 Feb 2025 12:08:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=BLyynCt450Utx2Z4sB12bYa6WPePLKZjdTj9AHDawWQ=; b=AB5iER/cCLVGRM5rAFA0JKbJxN 86/ZcBsxFnijdxWYg5/5pVsXHRRPgP2tXrPXLx7tH/B2G1SvkNjglKP0tfm6ZBDPPhenwwPgKsql7 ZSACCMGwv02hMh9nj6satsxjt/zxFYaPGMUlGZW67PYAaeYLcGf8b6Vq1Y0AwU48hGUKSc+HZ6bbR ifXvnlcUg3IPIVQt7XUzyqIc6R1IXsnMk/5HIcx2NdbK1ENMxyhvrd5yaFc/EangLCx7otuI/rCP1 /wIYaBBVQrd+P+tdP1HTW6j7wzkwHUu0ApI5Ezo4saR+R/7nxNNFqXcJNOqvcz8YWdhTkiXUxrtCL iD/LdESw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBXV-00000007ISD-1gLw; Wed, 12 Feb 2025 12:08:25 +0000 Received: from mail-lj1-f177.google.com ([209.85.208.177]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAqB-000000079mS-237n for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:41 +0000 Received: by mail-lj1-f177.google.com with SMTP id 38308e7fff4ca-30615661f98so71117691fa.2 for ; Wed, 12 Feb 2025 03:23:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359418; x=1739964218; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=BLyynCt450Utx2Z4sB12bYa6WPePLKZjdTj9AHDawWQ=; b=XqIM3N7dRWs/RCDXqausiARgL0lWjdFabaTCGFS0IE02mA0K6/XSlJl1T+1UK0ajqI qKwg6oN27GBpCveEyulPLarduDxzsCn+bkUXlm9OMICow+TqYudbQMOw5k0LYRKrpi5L PqonTY9eWfmDTuX8MpnTWEQZReDc0zBXh1bGTQ2H6WhXCElJu0zpo6RTUV2W4oeXoLjE k9qVacbjCDnzD05OOoS0wvg7hGfgxhv7O5biUKRf05IyFb6Onh7TMqjRMnQKzDPmW86q NmtN4H1fwFjl7KB4kIqETQ9/8E6ctfOi1irZJyybncD9ZPt0oLnDJUYn7niZHrRVOF9I Y9aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359418; x=1739964218; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BLyynCt450Utx2Z4sB12bYa6WPePLKZjdTj9AHDawWQ=; b=d+HMbwITlpitLD3yd6RTVqwbFIRuMd9egaEO6d5FNypYuNbALk20ageYmQxr7jxx09 IKPlIhXnxSB84dEu56+Mi4qM7qLY5LfpO60/zjUuDvLaNkSc/OMLF58vyYC7wVDIxfZZ 2BPUUG00PLDEpOtbMTk6XCR3Uf6j0ToljiSx97FtkIrBvZn6nlCBM+zaUYiXuNyxyb5E HXlzWqR2125grjo7hmZ0n5b4Nuoy+fybNrjeFWaUPYaNdM+iBrgu+j23TYZ/XiZ4eVtt BAN/8B9LY9X8bDKP97lgJpCfVjyc9AbQ1Toc9j2DfjhM+DFfExEdBU3RVboFJLQ10g9V IB8w== X-Gm-Message-State: AOJu0YyOiDgNA0el6ZIv0pbMUBOmQd6JdlCDy+M5M1r96KkAWbKjfhZb 4A1bimgtR7z8lVEDvMZp0+sf8QhKjtjwy1Ak+NDehSEDzAKZPepR5L2aSUJqEnY= X-Gm-Gg: ASbGncsYbrdygHbNGmgFyh/oQctzBzKmPsbvk9NS+FGuUVLsSfHPNqxIrc0SKT9q1VA zOJAsNqOC3gdfF1LOGFv2fnxA+kn+BaoH0mu/5HbaTRS3ktIpMYz2ygqJ9BwCBPVl8lFxL5QZaW E6WZIwpEURp4FD5kMy2UVwh5kP+Sdo8ThW2LxbJ4Fat0ARjyOvFdEukh/Uzi9j+rVIR+ZvgyBV/ cE/ITGs5nl3vsCp2F2hbup5tk3/xmiqlWg+BfA3NYKFTDTbQ6KeFi3whBKLy30IECEkRcR5LBRs OAzqOux0UKIY4nuGFHlQaJCFcg== X-Google-Smtp-Source: AGHT+IHK2ire6eSOXUjIyEB1DADW6wNdjzmfwgrMquJUSGQom2k6V3tykQc82oC2UKLm7kJWGSeVIQ== X-Received: by 2002:a05:651c:1b06:b0:2fa:cdd1:4f16 with SMTP id 38308e7fff4ca-3090364236dmr12530571fa.14.1739359417574; Wed, 12 Feb 2025 03:23:37 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:36 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:24 +0100 Subject: [PATCH v4 30/31] ARM: entry: Block IRQs in early IRQ context MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-30-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032339_536840_AE80BF56 X-CRM114-Status: GOOD ( 19.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When dabt, pabt or und exceptions occur on ARM, ordinary interrupts (IRQs) can still happen. This isn't nice for the kernels context tracker, which expect (when using generic entry at least) that any nested IRQs happens between irqentry_enter() and irqentry_exit(), else it thinks something is fishy. This change blocks interrupts in the pabt, dabt, und and abt exception paths (all of them really) by unconditionally setting PSR_I_BIT in the early exception handler, until after context has been established with irqentry_enter() and before it is exited with irqentry_exit(). Inside the context-tracked exception handler we enable IRQs again, and once we leave it we disable them while exiting the exception. The local_irq_disable() in bad_mode() can be dropped since we are now disabling IRQs in the early assembly exception handler for all exceptions. This seems like not perfect: it seems an interrupt could still occur right before CPSR is set, or right after the userspace registers are restored in ret_from_exception. I would like to know if there is some way to set up these exceptions to inherently block IRQs when handled, until we explicitly allow them between irqentry_enter() and irqentry_exit() or if this is simply the best we can do on ARM for these exceptions to make the context tracker happy. Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-armv.S | 6 +----- arch/arm/kernel/entry.c | 18 ++++++++++++++++++ arch/arm/kernel/traps.c | 1 - 3 files changed, 19 insertions(+), 6 deletions(-) diff --git a/arch/arm/kernel/entry-armv.S b/arch/arm/kernel/entry-armv.S index ae2f952beea7611f0abc7bd299fc944335a21219..3dae35b0bb3f440ecaf157a45687bf4690fb8f88 100644 --- a/arch/arm/kernel/entry-armv.S +++ b/arch/arm/kernel/entry-armv.S @@ -416,11 +416,6 @@ ENDPROC(__irq_usr) __und_usr: usr_entry uaccess=0 - @ IRQs must be enabled before attempting to read the instruction from - @ user space since that could cause a page/translation fault if the - @ page table was modified by another CPU. - enable_irq - tst r5, #PSR_T_BIT @ Thumb mode? mov r1, #2 @ set insn size to 2 for Thumb bne 0f @ handle as Thumb undef exception @@ -847,6 +842,7 @@ vector_\name: @ mrs r0, cpsr eor r0, r0, #(\mode ^ SVC_MODE | PSR_ISETSTATE) + orr r0, r0, #PSR_I_BIT msr spsr_cxsf, r0 @ diff --git a/arch/arm/kernel/entry.c b/arch/arm/kernel/entry.c index 01e4339ccdb4951e04a89fb91ad7c6e1991e09af..3881670e0987ee40be6fff32f412edcf4f3ec80a 100644 --- a/arch/arm/kernel/entry.c +++ b/arch/arm/kernel/entry.c @@ -17,8 +17,18 @@ noinstr asmlinkage void arm_und_handler(struct pt_regs *regs) { irqentry_state_t state = irqentry_enter(regs); + /* + * IRQs must be enabled before attempting to read the instruction from + * user space since that could cause a page/translation fault if the + * page table was modified by another CPU. + */ + + local_irq_enable(); + do_undefinstr(regs); + local_irq_disable(); + irqentry_exit(regs, state); } @@ -27,8 +37,12 @@ noinstr asmlinkage void arm_dabt_handler(unsigned long addr, unsigned int fsr, { irqentry_state_t state = irqentry_enter(regs); + local_irq_enable(); + do_DataAbort(addr, fsr, regs); + local_irq_disable(); + irqentry_exit(regs, state); } @@ -37,8 +51,12 @@ noinstr asmlinkage void arm_pabt_handler(unsigned long addr, unsigned int ifsr, { irqentry_state_t state = irqentry_enter(regs); + local_irq_enable(); + do_PrefetchAbort(addr, ifsr, regs); + local_irq_disable(); + irqentry_exit(regs, state); } diff --git a/arch/arm/kernel/traps.c b/arch/arm/kernel/traps.c index d3a689bd05955f1ae46a6341e456bb097e831311..b2eaab9cc4977c448f77faa4e6bc22a0749d1f9c 100644 --- a/arch/arm/kernel/traps.c +++ b/arch/arm/kernel/traps.c @@ -515,7 +515,6 @@ asmlinkage void bad_mode(struct pt_regs *regs, int reason) pr_crit("Bad mode in %s handler detected\n", handler[reason]); die("Oops - bad mode", regs, 0); - local_irq_disable(); panic("bad mode"); } From patchwork Wed Feb 12 11:23:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Linus Walleij X-Patchwork-Id: 13971828 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8A2F9C02198 for ; Wed, 12 Feb 2025 12:10:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Pd5myOXJmBrq2ZireMN1LAxoO4XGu12StPrpGUT11PQ=; b=4k2G4Vqe6C+MBOiAN2DsNeT5u/ haYUndXhTpbmaqAkcFZZemgK7g8B8FsYrIr8unReLOaBQpk86pbIXC13Fzdhiy+JtkUtGLibpS9e9 TQO7/n8jAcA2qC1KFcwecx5UpSr0rfWCXwnD3VPsYUMxkAnmEe7u0XDMpeMRmph2hGsBYN7GLaC7h uKLJ28pf5+DzolHBPEMnJRgyaU2oKRAJi9zfRPzDDMuAoADmUGEfG4FuB7lJyEfnCgpLTKWhxBl7A +/qXR1O9/yO76jKjQdoiZhdQpkN5GYrM/c6MtuoWp72l4f28g1iEtyLMWw8UdQNG/i9cV8bywlzPo KDVJBG9Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tiBYt-00000007IcC-2wsD; Wed, 12 Feb 2025 12:09:51 +0000 Received: from mail-lj1-x233.google.com ([2a00:1450:4864:20::233]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tiAqC-000000079n6-2JMS for linux-arm-kernel@lists.infradead.org; Wed, 12 Feb 2025 11:23:41 +0000 Received: by mail-lj1-x233.google.com with SMTP id 38308e7fff4ca-308edbc368cso32852301fa.1 for ; Wed, 12 Feb 2025 03:23:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1739359419; x=1739964219; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Pd5myOXJmBrq2ZireMN1LAxoO4XGu12StPrpGUT11PQ=; b=nbHhR4d5J5rOQJ4GFk7ee8CA/aHFfgkhZb8KyNi0hVfCJNloRswedrX8lb7DqDV1Zw 1jikUg4MUIVHMaO4nVMm6plZhkp16BUqtnKvKl9XszpDu/RrSOyz4l8Ed9Z8f9Uqd2Dj DURs9WnaF1vdv5ty5sLAUf5NBZfqjj6jaqYTnOhpNGelnbn7me9jVnsuf5U8kj7Tlowo MVQttlr6BMve/gitEq9daMzRZPT/dtZoSovsvaTgiHdPYv1weNeqYAXgzKd/ILqJFxTz 7I5f02NqhTEOa/0YAtpIX8tgX19KjcaALGaUHtF7EfIlAubR+Ru+jkcdKt4Gyiikswni twWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739359419; x=1739964219; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Pd5myOXJmBrq2ZireMN1LAxoO4XGu12StPrpGUT11PQ=; b=wyrfow1O38irAyEqnxKV8MV3rhbvdLVJQaTrKk6B4hpZWsbnGHrAztGiJmuBjmn2sX zXav1B9a9Ah36JvG2qj/gwJwAY3/O6QxDzKtBP3DTqlrsET+RKAUqO9tvG47ffQwj3CG ilHhFIy1AmQiSsZOpbJqI1UyT/qadekVTNaJKaLP4/D8HOKrXQjMUCkWkCqkTr15yKWf 2oVnOi5+irZz/sNMxYv58hjWyJN5bDmROA7iw7tC3NeXzT12UnF8ir0Asddgz5CuBtMY eWhl6DC6sK2TG5QS1OG9DEyRysawP8zZTSOcUJUDUqWmOweG0Uz5DBq4+voLmPVF3mXJ holQ== X-Gm-Message-State: AOJu0YwzhYAvSFC5s9lpVCVdjTNSi+/hXpXBYj2WP+gpAOoy6zpoogFP L9dq/mVQ0CMTdIF9n7FC+rUpC8Ek/ilN5Nnaely5ww+US9JDgZydcQ9/KPFtUYU= X-Gm-Gg: ASbGncvcZ84ItegPcTJszT5jzDbst6dY2k1hVHg7UdyvSCkPlpxReXiSVERDZA53xnQ 8+rHbvRdHQIp2UyEbSh14N+ozVvKN2wEcWHdr/UMqBJJEUA6nhp9HB+wy41Yl2Qk3JUQ1WdRzM1 BriwABBhrcImGEenvNCHnYJUdN7rKbIu2UkV9HEgM5lZnYWQqgS1ZYaAQLKrzcE22bEC6tSGTXU qNWQ9my0yEqYPQ+fk9u3bHNXQUgm5urS9afTmNlYO03lpODFDcbaMHtyLKUexFci9Qa6rimUiAF uQJh5YQ9ch6pnmiTX8RDlgwPhg== X-Google-Smtp-Source: AGHT+IFnBCvVu/VGrB3ufwIm9PIPjMNjQwi218jFm+vuC3QKd2lwYi8qU7J4iPr0yHXGhJ+sLSoDTg== X-Received: by 2002:a05:651c:1991:b0:2ff:c3a2:f408 with SMTP id 38308e7fff4ca-30904fe65b4mr8433611fa.12.1739359418597; Wed, 12 Feb 2025 03:23:38 -0800 (PST) Received: from [192.168.1.140] ([85.235.12.238]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-307de1a6531sm18310511fa.45.2025.02.12.03.23.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Feb 2025 03:23:38 -0800 (PST) From: Linus Walleij Date: Wed, 12 Feb 2025 12:23:25 +0100 Subject: [PATCH v4 31/31] ARM: entry: Straighten syscall returns MIME-Version: 1.0 Message-Id: <20250212-arm-generic-entry-v4-31-a457ff0a61d6@linaro.org> References: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> In-Reply-To: <20250212-arm-generic-entry-v4-0-a457ff0a61d6@linaro.org> To: Dmitry Vyukov , Oleg Nesterov , Russell King , Kees Cook , Andy Lutomirski , Will Drewry , Frederic Weisbecker , "Paul E. McKenney" , Jinjie Ruan , Arnd Bergmann , Ard Biesheuvel , Al Viro Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Linus Walleij X-Mailer: b4 0.14.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250212_032340_592496_4D69B33F X-CRM114-Status: GOOD ( 15.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Since ret_fast_syscall() is now only called in two spots in vector_swi, we can inline the function into vector_swi, and skip the separate call, placing the code from ret_fast_syscall at the end of vector_swi and one of them inside an ifdef that gets compiled out for !EABI. TODO: is the UNWIND(.fnstart/.cantunwind/.fnend) really needed on vector_swi after this change or can I just drop it? I do not understand unwind well enough :/ Signed-off-by: Linus Walleij --- arch/arm/kernel/entry-common.S | 32 ++++++++++++++------------------ 1 file changed, 14 insertions(+), 18 deletions(-) diff --git a/arch/arm/kernel/entry-common.S b/arch/arm/kernel/entry-common.S index a5a0e757ab23d151485092a078af98966e039869..f9c38f5311851e299234997796cd78de06a5c531 100644 --- a/arch/arm/kernel/entry-common.S +++ b/arch/arm/kernel/entry-common.S @@ -30,21 +30,6 @@ saved_pc .req lr .section .entry.text,"ax",%progbits .align 5 -/* - * As we will need to call out to some C functions, - * we save r0 first to avoid needing to save registers around each C function - * call. - */ -ret_fast_syscall: - UNWIND(.fnstart ) - UNWIND(.cantunwind ) - add sp, sp, #(S_R0 + S_OFF) -#ifdef CONFIG_GCC_PLUGIN_STACKLEAK - bl stackleak_erase_on_task_stack -#endif - restore_user_regs - UNWIND(.fnend ) -ENDPROC(ret_fast_syscall) /* * "slow" syscall return path. "why" tells us if this was a real syscall. @@ -122,6 +107,8 @@ ENDPROC(vector_bhb_bpiall_swi) #endif .align 5 ENTRY(vector_swi) + UNWIND(.fnstart ) + UNWIND(.cantunwind ) #ifdef CONFIG_CPU_V7M v7m_exception_entry #else @@ -210,9 +197,9 @@ ENTRY(vector_swi) mov r0, tbl mov r2, scno @ syscall number from r7 bl invoke_syscall - b ret_fast_syscall - #if defined(CONFIG_OABI_COMPAT) || !defined(CONFIG_AEABI) + b 9002 + /* * We failed to handle a fault trying to access the page * containing the swi instruction, but we're not really in a @@ -225,8 +212,17 @@ ENTRY(vector_swi) sub lr, saved_pc, #4 str lr, [sp, #S_PC] get_thread_info tsk - b ret_fast_syscall +9002: + /* Exit !AEABI clause */ +#endif + + /* Return from syscall */ + add sp, sp, #(S_R0 + S_OFF) +#ifdef CONFIG_GCC_PLUGIN_STACKLEAK + bl stackleak_erase_on_task_stack #endif + restore_user_regs + UNWIND(.fnend ) ENDPROC(vector_swi) .ltorg