From patchwork Thu Oct 17 09:25:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 13839692 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 589A8D21262 for ; Thu, 17 Oct 2024 09:34:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=dUtkdmdtkNCtxVODk8OoZF9PuEdsHwBwvFab/eA2Lbc=; b=s5Vw0yI2p5bpRKGjIZZ3wv9skD zjzzu9tEOdeLJMBcR8yGByDRVxnAoSq3RldKrxsUfy73wnAEjTU9heKZexhGMO1uMpsM/Q01yTDuq jxCbvph2RJIICeydArxLJujBCpXJiqeLnbkiHC4unM88VdLSrQD1pwXr2KMM+G2IRkXDjN9OWOTyZ IjtN+fEH9YIast24X75W8pi0PJQ/UDiF4km8cG3OgGZ8nop1JCivIvDRtkkeFLeQdcyf+c9OGw8Is 4aTSLqYDVPdgwo9KZoXZhyCDDgtusxxBDzRf4JLnUSeYkxu8R39a4bZQiUdMlO+HhBlEUCrgsSYSX Hto0Mp5w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1Mtd-0000000EMMx-33U0; Thu, 17 Oct 2024 09:34:17 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1MlS-0000000EKf2-1EFh for linux-arm-kernel@lists.infradead.org; Thu, 17 Oct 2024 09:25:51 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4996A1477; Thu, 17 Oct 2024 02:26:17 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 987DD3F528; Thu, 17 Oct 2024 02:25:45 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, jpoimboe@kernel.org, kaleshsingh@google.com, madvenka@linux.microsoft.com, mark.rutland@arm.com, maz@kernel.org, mbenes@suse.cz, puranjay12@gmail.com, will@kernel.org Subject: [PATCH v2 01/10] arm64: pt_regs: assert pt_regs is a multiple of 16 bytes Date: Thu, 17 Oct 2024 10:25:29 +0100 Message-Id: <20241017092538.1859841-2-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20241017092538.1859841-1-mark.rutland@arm.com> References: <20241017092538.1859841-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241017_022550_413243_EBA46276 X-CRM114-Status: GOOD ( 12.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To ensure that the stack is correctly aligned when branching to C code, we require that struct pt_regs is a multiple of 16 bytes, as noted in a comment. Add an explicit assertion for this, so that any accidental violation of this requirement will be caught by the compiler. Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Reviewed-by: Miroslav Benes Reviewed-by: Puranjay Mohan Cc: Ard Biesheuvel Cc: Catalin Marinas Cc: Josh Poimboeuf Cc: Kalesh Singh Cc: Madhavan T. Venkataraman Cc: Marc Zyngier Cc: Will Deacon --- arch/arm64/include/asm/ptrace.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index 0abe975d68a8e..10eca3a3fed49 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -149,8 +149,7 @@ static inline unsigned long pstate_to_compat_psr(const unsigned long pstate) /* * This struct defines the way the registers are stored on the stack during an - * exception. Note that sizeof(struct pt_regs) has to be a multiple of 16 (for - * stack alignment). struct user_pt_regs must form a prefix of struct pt_regs. + * exception. struct user_pt_regs must form a prefix of struct pt_regs. */ struct pt_regs { union { @@ -180,6 +179,9 @@ struct pt_regs { u64 exit_rcu; }; +/* For correct stack alignment, pt_regs has to be a multiple of 16 bytes. */ +static_assert(IS_ALIGNED(sizeof(struct pt_regs), 16)); + static inline bool in_syscall(struct pt_regs const *regs) { return regs->syscallno != NO_SYSCALL; From patchwork Thu Oct 17 09:25:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 13839693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 41495D21269 for ; Thu, 17 Oct 2024 09:36:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=+Ngg3IaJUb0Pd0LXWUJ/jr41wwgVIoxrzHv2BdSZS4c=; b=AP6/JQz5o+NgTgnWft3InqvYc1 TSkyBLyHP2jHkRr8dU4v5Li5JToefipnf+Z1HpTmFHU4BYJH4DdZMEEQE7lYK2RQSLQKQs9bDxCF3 hoL3DPt3Mg6koZsFqw5R/ijFqST/PrMV5K98oCn1XhuRsCrlZP2kD30xQLUuZVhG5H3BD5YDeJbCf otH9SrN9/t3oBMtgVrwwgkaEzrMGS2iWLzB7sinV4WP0EkSZlYUrzfRj/AaF0HkUCYCt9DZhtQScN CGLFgYKjVIGXtbaYBVTOaXrKYOsa6Kk9Liii92MRAB7VxlpwwDFrugciR1dP88+Kw+W66lNIee0dG pxemxZtQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1Mv3-0000000EMaY-2imD; Thu, 17 Oct 2024 09:35:45 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1MlT-0000000EKg2-1uGA for linux-arm-kernel@lists.infradead.org; Thu, 17 Oct 2024 09:25:53 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 30AC0150C; Thu, 17 Oct 2024 02:26:20 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7F6AE3F528; Thu, 17 Oct 2024 02:25:48 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, jpoimboe@kernel.org, kaleshsingh@google.com, madvenka@linux.microsoft.com, mark.rutland@arm.com, maz@kernel.org, mbenes@suse.cz, puranjay12@gmail.com, will@kernel.org Subject: [PATCH v2 02/10] arm64: pt_regs: remove stale big-endian layout Date: Thu, 17 Oct 2024 10:25:30 +0100 Message-Id: <20241017092538.1859841-3-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20241017092538.1859841-1-mark.rutland@arm.com> References: <20241017092538.1859841-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241017_022551_767547_37C2C726 X-CRM114-Status: GOOD ( 14.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org For historical reasons the layout of struct pt_regs depends on the configured endianness, with the order of the 'syscallno' and 'unused2' fields varying dependent upon whether __AARCH64EB__ is defined. We no longer depend on the order of these two fields and can remove the ifdeffery. The current conditional layout was introduced in commit: 35d0e6fb4d219d64 ("arm64: syscallno is secretly an int, make it official") At the time, this was necessary so that the entry assembly could use a single STP instruction to save the pt_regs::{orig_x0,syscallno} fields, without logic that was conditional on the endianness of the kernel: | el0_svc_naked: | stp x0, xscno, [sp, #S_ORIG_X0] // save the original x0 and syscall number This logic was converted to C in commit: f37099b6992a0b81 ("arm64: convert syscall trace logic to C") Since that commit, we no longer manipulate pt_regs::orig_x0 from assembly, and only manipulate pt_regs::syscallno as a 32-bit quantity early in the kernel_entry assembly: | /* Not in a syscall by default (el0_svc overwrites for real syscall) */ | .if \el == 0 | mov w21, #NO_SYSCALL | str w21, [sp, #S_SYSCALLNO] | .endif Given the above, there's no longer a need for the layout of pt_regs::{syscallno,unused2} to depend on the endianness of the kernel. This patch removes the ifdeffery and places 'syscallno' before 'unused2' regardless of the endianess of the kernel. At the same time, 'unused2' is renamed to 'unused', as it is the only unused field within pt_regs. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Reviewed-by: Miroslav Benes Reviewed-by: Puranjay Mohan Cc: Ard Biesheuvel Cc: Catalin Marinas Cc: Josh Poimboeuf Cc: Kalesh Singh Cc: Madhavan T. Venkataraman Cc: Marc Zyngier Cc: Will Deacon --- arch/arm64/include/asm/ptrace.h | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index 10eca3a3fed49..1ae671fe64d86 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -162,13 +162,9 @@ struct pt_regs { }; }; u64 orig_x0; -#ifdef __AARCH64EB__ - u32 unused2; s32 syscallno; -#else - s32 syscallno; - u32 unused2; -#endif + u32 unused; + u64 sdei_ttbr1; /* Only valid when ARM64_HAS_GIC_PRIO_MASKING is enabled. */ u64 pmr_save; From patchwork Thu Oct 17 09:25:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 13839694 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6AFC0D21262 for ; Thu, 17 Oct 2024 09:37:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=1VYY4uhckpZMsOs5SbsANmUfu9mRaB+kDZjCL8kw+s0=; b=idPDlJfdZ5QA7makck5VU4t+yB CqeQs73aBOcXyG/tGL7hllPforSryaCwwZkHeCnLcfUNTH/zMqzu2t2XGXsCU7zeteSOeRqGrjFlm 9ZjIqW0Lfk6vHTwm8DU905Fz6yPEyXGS9nwvK1RwE7+RoL7QfwXsCDfE+bowRfuQJNPQSav06htjQ 0BeNrgZtQ48ewBc75w5Ls+ATKcJLSbCWI7TC0Z7SGAERGJLvrzkizPuvDnHZZDxKiNHkk2kpDr6u2 kYlo5fDRY5LiaGiuF+AIKglK2L0Ezq5m1KiCcF7JFa8nPvtufAK+XznlM2/4LUHrkQ2X5FxCMICe5 I6fDxboQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1MwT-0000000EMmF-3lbl; Thu, 17 Oct 2024 09:37:14 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1MlV-0000000EKge-2ul0 for linux-arm-kernel@lists.infradead.org; Thu, 17 Oct 2024 09:25:55 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0EA1FFEC; Thu, 17 Oct 2024 02:26:23 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 5DF903F528; Thu, 17 Oct 2024 02:25:51 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, jpoimboe@kernel.org, kaleshsingh@google.com, madvenka@linux.microsoft.com, mark.rutland@arm.com, maz@kernel.org, mbenes@suse.cz, puranjay12@gmail.com, will@kernel.org Subject: [PATCH v2 03/10] arm64: pt_regs: rename "pmr_save" -> "pmr" Date: Thu, 17 Oct 2024 10:25:31 +0100 Message-Id: <20241017092538.1859841-4-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20241017092538.1859841-1-mark.rutland@arm.com> References: <20241017092538.1859841-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241017_022553_841397_AE5CB552 X-CRM114-Status: GOOD ( 12.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The pt_regs::pmr_save field is weirdly named relative to all other pt_regs fields, with a '_save' suffix that doesn't make anything clearer and only leads to more typing to access the field. Remove the '_save' suffix. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Reviewed-by: Miroslav Benes Reviewed-by: Puranjay Mohan Cc: Ard Biesheuvel Cc: Catalin Marinas Cc: Josh Poimboeuf Cc: Kalesh Singh Cc: Madhavan T. Venkataraman Cc: Marc Zyngier Cc: Will Deacon --- arch/arm64/include/asm/daifflags.h | 2 +- arch/arm64/include/asm/processor.h | 2 +- arch/arm64/include/asm/ptrace.h | 4 ++-- arch/arm64/kernel/asm-offsets.c | 2 +- arch/arm64/kernel/entry.S | 4 ++-- arch/arm64/kernel/process.c | 2 +- 6 files changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/daifflags.h b/arch/arm64/include/asm/daifflags.h index 55f57dfa8e2fe..fbb5c99eb2f9d 100644 --- a/arch/arm64/include/asm/daifflags.h +++ b/arch/arm64/include/asm/daifflags.h @@ -132,7 +132,7 @@ static inline void local_daif_inherit(struct pt_regs *regs) trace_hardirqs_on(); if (system_uses_irq_prio_masking()) - gic_write_pmr(regs->pmr_save); + gic_write_pmr(regs->pmr); /* * We can't use local_daif_restore(regs->pstate) here as diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 1438424f00643..03b99164f7f46 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -293,7 +293,7 @@ static inline void start_thread_common(struct pt_regs *regs, unsigned long pc) regs->pc = pc; if (system_uses_irq_prio_masking()) - regs->pmr_save = GIC_PRIO_IRQON; + regs->pmr = GIC_PRIO_IRQON; } static inline void start_thread(struct pt_regs *regs, unsigned long pc, diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index 1ae671fe64d86..e82e6e47ac9a4 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -167,7 +167,7 @@ struct pt_regs { u64 sdei_ttbr1; /* Only valid when ARM64_HAS_GIC_PRIO_MASKING is enabled. */ - u64 pmr_save; + u64 pmr; u64 stackframe[2]; /* Only valid for some EL1 exceptions. */ @@ -211,7 +211,7 @@ static inline void forget_syscall(struct pt_regs *regs) #define irqs_priority_unmasked(regs) \ (system_uses_irq_prio_masking() ? \ - (regs)->pmr_save == GIC_PRIO_IRQON : \ + (regs)->pmr == GIC_PRIO_IRQON : \ true) #define interrupts_enabled(regs) \ diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 27de1dddb0abe..eb7fb2f9b9274 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -79,7 +79,7 @@ int main(void) DEFINE(S_PSTATE, offsetof(struct pt_regs, pstate)); DEFINE(S_SYSCALLNO, offsetof(struct pt_regs, syscallno)); DEFINE(S_SDEI_TTBR1, offsetof(struct pt_regs, sdei_ttbr1)); - DEFINE(S_PMR_SAVE, offsetof(struct pt_regs, pmr_save)); + DEFINE(S_PMR, offsetof(struct pt_regs, pmr)); DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe)); DEFINE(PT_REGS_SIZE, sizeof(struct pt_regs)); BLANK(); diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 7ef0e127b149f..a84ce95ad96ce 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -315,7 +315,7 @@ alternative_if_not ARM64_HAS_GIC_PRIO_MASKING alternative_else_nop_endif mrs_s x20, SYS_ICC_PMR_EL1 - str x20, [sp, #S_PMR_SAVE] + str x20, [sp, #S_PMR] mov x20, #GIC_PRIO_IRQON | GIC_PRIO_PSR_I_SET msr_s SYS_ICC_PMR_EL1, x20 @@ -342,7 +342,7 @@ alternative_if_not ARM64_HAS_GIC_PRIO_MASKING b .Lskip_pmr_restore\@ alternative_else_nop_endif - ldr x20, [sp, #S_PMR_SAVE] + ldr x20, [sp, #S_PMR] msr_s SYS_ICC_PMR_EL1, x20 /* Ensure priority change is seen by redistributor */ diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 0540653fbf382..2951b5ba19378 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -227,7 +227,7 @@ void __show_regs(struct pt_regs *regs) printk("sp : %016llx\n", sp); if (system_uses_irq_prio_masking()) - printk("pmr_save: %08llx\n", regs->pmr_save); + printk("pmr: %08llx\n", regs->pmr); i = top_reg; From patchwork Thu Oct 17 09:25:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 13839700 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 38E6BD21269 for ; Thu, 17 Oct 2024 09:39:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=cGgcn/qQ9//CaFcbU5BbSJvrPdp17u/RzV4ge1BaI7o=; b=kvlZWbTY+cXtOmBbEEd8eJBh3G UNOoQ0ZGUdHGno4zHQXhso3acIzKMR5qivmjOX9Hc5/TQCDSVy0XzmcWpMjjr2cmDQJP9Epv1yu/c Flqnn+7wdsDt/B754EwrsmkQOb3iX0lOWbwKukCRFqa8YSnOmYLa1+82LcB07vuXLgzNSPfToLTMl eUYqGYOofZhdMTycG+fHJCEdr2WCqL099Tx+fAVVOLCrpCxm6TQrdzMp1ICwveaQszGl9c6K+u4+n Btk/r492xvVH0+4GHSTWa9FV7FibS8+Jn1T4rR2SrWeJ20F+Xb/MWKlqAFKTFur6JNT4esApff91l 3H+wLQHw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1Mxw-0000000EMw0-2PTw; Thu, 17 Oct 2024 09:38:44 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1MlZ-0000000EKhL-1hd9 for linux-arm-kernel@lists.infradead.org; Thu, 17 Oct 2024 09:25:58 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E234F1477; Thu, 17 Oct 2024 02:26:25 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3CA133F528; Thu, 17 Oct 2024 02:25:54 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, jpoimboe@kernel.org, kaleshsingh@google.com, madvenka@linux.microsoft.com, mark.rutland@arm.com, maz@kernel.org, mbenes@suse.cz, puranjay12@gmail.com, will@kernel.org Subject: [PATCH v2 04/10] arm64: pt_regs: swap 'unused' and 'pmr' fields Date: Thu, 17 Oct 2024 10:25:32 +0100 Message-Id: <20241017092538.1859841-5-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20241017092538.1859841-1-mark.rutland@arm.com> References: <20241017092538.1859841-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241017_022557_514267_3F7E8BAB X-CRM114-Status: GOOD ( 12.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In subsequent patches we'll want to add an additional u64 to struct pt_regs. To make space, this patch swaps the 'unused' and 'pmr' fields, as the 'pmr' value only requires bits[7:0] and can safely fit into a u32, which frees up a 64-bit unused field. The 'lockdep_hardirqs' and 'exit_rcu' fields should eventually be moved out of pt_regs and managed locally within entry-common.c, so I've left those as-is for the moment. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Reviewed-by: Miroslav Benes Reviewed-by: Puranjay Mohan Cc: Ard Biesheuvel Cc: Josh Poimboeuf Cc: Kalesh Singh Cc: Madhavan T. Venkataraman Cc: Marc Zyngier Cc: Will Deacon --- arch/arm64/include/asm/ptrace.h | 6 +++--- arch/arm64/kernel/entry.S | 4 ++-- arch/arm64/kernel/process.c | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index e82e6e47ac9a4..92531aeba5310 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -163,11 +163,11 @@ struct pt_regs { }; u64 orig_x0; s32 syscallno; - u32 unused; + u32 pmr; u64 sdei_ttbr1; - /* Only valid when ARM64_HAS_GIC_PRIO_MASKING is enabled. */ - u64 pmr; + u64 unused; + u64 stackframe[2]; /* Only valid for some EL1 exceptions. */ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index a84ce95ad96ce..fa6d6d5ca5e02 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -315,7 +315,7 @@ alternative_if_not ARM64_HAS_GIC_PRIO_MASKING alternative_else_nop_endif mrs_s x20, SYS_ICC_PMR_EL1 - str x20, [sp, #S_PMR] + str w20, [sp, #S_PMR] mov x20, #GIC_PRIO_IRQON | GIC_PRIO_PSR_I_SET msr_s SYS_ICC_PMR_EL1, x20 @@ -342,7 +342,7 @@ alternative_if_not ARM64_HAS_GIC_PRIO_MASKING b .Lskip_pmr_restore\@ alternative_else_nop_endif - ldr x20, [sp, #S_PMR] + ldr w20, [sp, #S_PMR] msr_s SYS_ICC_PMR_EL1, x20 /* Ensure priority change is seen by redistributor */ diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 2951b5ba19378..c722c1be6fa59 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -227,7 +227,7 @@ void __show_regs(struct pt_regs *regs) printk("sp : %016llx\n", sp); if (system_uses_irq_prio_masking()) - printk("pmr: %08llx\n", regs->pmr); + printk("pmr: %08x\n", regs->pmr); i = top_reg; From patchwork Thu Oct 17 09:25:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 13839701 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AC88AD21269 for ; Thu, 17 Oct 2024 09:40:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=oD+2qgVbqgnEXonqg5HfWQV3mdO2FIwjkvL5OVl1txU=; b=ylWM9HN0Bt0YJJnLz5gyFwbQUA 0ebJ9cnBdmxsyIsDws4Ici6A+kgQMKDa0QxOVkt6/9RsdsLWsUDUmKl7BXj1wbNblAjVkyhx773n+ 4Ok65bPlVjbqEB/LqAqFEPMGXq76q3pIi55qlVznk+ahqK4HS/xVzIp85ueis4VA09QN2pAtoJV0l 3eGzyaGyBQ1qRefMESW3CpGc8FylTRBkj4QbjwVxG6Jo7cjfRktQwMXZO5oO+THZAyyX5s5VI5lAR B2NWBezzHMV2w0A7s87UFYDlQ00o2c7IvkjGnuuOTxhbsdNS8wFjkvfpbrwA5w2tvzTQoSDgUnSC+ 7MvRsfYA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1MzM-0000000EN9l-23CM; Thu, 17 Oct 2024 09:40:12 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1Mlb-0000000EKht-1Z7Z for linux-arm-kernel@lists.infradead.org; Thu, 17 Oct 2024 09:26:00 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B37DBFEC; Thu, 17 Oct 2024 02:26:28 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0D8203F528; Thu, 17 Oct 2024 02:25:56 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, jpoimboe@kernel.org, kaleshsingh@google.com, madvenka@linux.microsoft.com, mark.rutland@arm.com, maz@kernel.org, mbenes@suse.cz, puranjay12@gmail.com, will@kernel.org Subject: [PATCH v2 05/10] arm64: use a common struct frame_record Date: Thu, 17 Oct 2024 10:25:33 +0100 Message-Id: <20241017092538.1859841-6-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20241017092538.1859841-1-mark.rutland@arm.com> References: <20241017092538.1859841-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241017_022559_524969_B44839CF X-CRM114-Status: GOOD ( 19.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently the signal handling code has its own struct frame_record, the definition of struct pt_regs open-codes a frame record as an array, and the kernel unwinder hard-codes frame record offsets. Move to a common struct frame_record that can be used throughout the kernel. Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Reviewed-by: Miroslav Benes Reviewed-by: Puranjay Mohan Cc: Ard Biesheuvel Cc: Josh Poimboeuf Cc: Kalesh Singh Cc: Madhavan T. Venkataraman Cc: Marc Zyngier Cc: Will Deacon --- arch/arm64/include/asm/ptrace.h | 4 +++- arch/arm64/include/asm/stacktrace/common.h | 8 +++++--- arch/arm64/include/asm/stacktrace/frame.h | 13 +++++++++++++ arch/arm64/kernel/process.c | 2 +- arch/arm64/kernel/signal.c | 5 ----- arch/arm64/kernel/stacktrace.c | 2 +- 6 files changed, 23 insertions(+), 11 deletions(-) create mode 100644 arch/arm64/include/asm/stacktrace/frame.h diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index 92531aeba5310..89c02f85f4b11 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -98,6 +98,8 @@ #include #include +#include + /* sizeof(struct user) for AArch32 */ #define COMPAT_USER_SZ 296 @@ -168,7 +170,7 @@ struct pt_regs { u64 sdei_ttbr1; u64 unused; - u64 stackframe[2]; + struct frame_record stackframe; /* Only valid for some EL1 exceptions. */ u64 lockdep_hardirqs; diff --git a/arch/arm64/include/asm/stacktrace/common.h b/arch/arm64/include/asm/stacktrace/common.h index f63dc654e545f..7fab6876e4970 100644 --- a/arch/arm64/include/asm/stacktrace/common.h +++ b/arch/arm64/include/asm/stacktrace/common.h @@ -137,21 +137,23 @@ static inline int unwind_consume_stack(struct unwind_state *state, static inline int unwind_next_frame_record(struct unwind_state *state) { + struct frame_record *record; unsigned long fp = state->fp; int err; if (fp & 0x7) return -EINVAL; - err = unwind_consume_stack(state, fp, 16); + err = unwind_consume_stack(state, fp, sizeof(*record)); if (err) return err; /* * Record this frame record's values. */ - state->fp = READ_ONCE(*(unsigned long *)(fp)); - state->pc = READ_ONCE(*(unsigned long *)(fp + 8)); + record = (struct frame_record *)fp; + state->fp = READ_ONCE(record->fp); + state->pc = READ_ONCE(record->lr); return 0; } diff --git a/arch/arm64/include/asm/stacktrace/frame.h b/arch/arm64/include/asm/stacktrace/frame.h new file mode 100644 index 0000000000000..6397bc847f147 --- /dev/null +++ b/arch/arm64/include/asm/stacktrace/frame.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __ASM_STACKTRACE_FRAME_H +#define __ASM_STACKTRACE_FRAME_H + +/* + * A standard AAPCS64 frame record. + */ +struct frame_record { + u64 fp; + u64 lr; +}; + +#endif /* __ASM_STACKTRACE_FRAME_H */ diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index c722c1be6fa59..d45fd114eac3f 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -419,7 +419,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) * For the benefit of the unwinder, set up childregs->stackframe * as the final frame for the new task. */ - p->thread.cpu_context.fp = (unsigned long)childregs->stackframe; + p->thread.cpu_context.fp = (unsigned long)&childregs->stackframe; ptrace_hw_copy_thread(p); diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 5619869475304..2c47f9a0e40ba 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -42,11 +42,6 @@ struct rt_sigframe { struct ucontext uc; }; -struct frame_record { - u64 fp; - u64 lr; -}; - struct rt_sigframe_user_layout { struct rt_sigframe __user *sigframe; struct frame_record __user *next_frame; diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 2729faaee4b4c..ffe8e4f549566 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -145,7 +145,7 @@ kunwind_next(struct kunwind_state *state) int err; /* Final frame; nothing to unwind */ - if (fp == (unsigned long)task_pt_regs(tsk)->stackframe) + if (fp == (unsigned long)&task_pt_regs(tsk)->stackframe) return -ENOENT; err = unwind_next_frame_record(&state->common); From patchwork Thu Oct 17 09:25:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 13839702 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A91F7D2126A for ; Thu, 17 Oct 2024 09:41:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=w9APiXnGK8s37GVXGBVi3iVOcry8GZaQ/bVzMVA0rKI=; b=nBJ+VU7eteNsXU+oWk2c2SBeS4 ebTsR8nVW4Zpxfh/bQXURLVYJ6JPAtitLuH//CI4TsAu06WIKNFOc5KFF91bX5fId9lLrRUkMxNo6 k7W9NGMS29+WJdvXGVc9axWQz399OpFXzLMRaX5zylF6gl07MrZjTTX0JS1VZqTltCd4fYR3FAQH7 7HmV9QmFb4rDAIj2YxATM33eE8kFOsdhLC0T+4ZdsSo6Cp0bhA9fB6r3T+KZKxlibN9q4ybAds7Tx mqCt7vXgUXOwy71sHzOhjk+yyMG1fR+KBCvXRv8uwEZ3IGfH1xCbTYfL6C8ZgxWNqPzwIwNfzpVX+ Pi17E1mg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1N0l-0000000ENMU-3dkG; Thu, 17 Oct 2024 09:41:39 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1Mle-0000000EKiQ-0O26 for linux-arm-kernel@lists.infradead.org; Thu, 17 Oct 2024 09:26:04 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6AFE61477; Thu, 17 Oct 2024 02:26:31 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id BA0953F528; Thu, 17 Oct 2024 02:25:59 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, jpoimboe@kernel.org, kaleshsingh@google.com, madvenka@linux.microsoft.com, mark.rutland@arm.com, maz@kernel.org, mbenes@suse.cz, puranjay12@gmail.com, will@kernel.org Subject: [PATCH v2 06/10] arm64: stacktrace: move dump_backtrace() to kunwind_stack_walk() Date: Thu, 17 Oct 2024 10:25:34 +0100 Message-Id: <20241017092538.1859841-7-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20241017092538.1859841-1-mark.rutland@arm.com> References: <20241017092538.1859841-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241017_022602_228647_968F12B9 X-CRM114-Status: GOOD ( 10.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently dump_backtrace() can only print the PC value at each step of the unwind, as this is all the information that arch_stack_walk() passes to the dump_backtrace_entry() callback. In future we'd like to print some additional information, such as the origin of entries (e.g. PC, LR, FP) and/or the reliability thereof. In preparation for doing so, this patch moves dump_backtrace() over to kunwind_stack_walk(), which passes the full kunwind_state to the callback. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Reviewed-by: Miroslav Benes Reviewed-by: Puranjay Mohan Cc: Ard Biesheuvel Cc: Josh Poimboeuf Cc: Kalesh Singh Cc: Madhavan T. Venkataraman Cc: Marc Zyngier Cc: Will Deacon --- arch/arm64/kernel/stacktrace.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index ffe8e4f549566..22f8709f0e763 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -294,10 +294,10 @@ noinline noinstr void arch_bpf_stack_walk(bool (*consume_entry)(void *cookie, u6 kunwind_stack_walk(arch_bpf_unwind_consume_entry, &data, current, NULL); } -static bool dump_backtrace_entry(void *arg, unsigned long where) +static bool dump_backtrace_entry(const struct kunwind_state *state, void *arg) { char *loglvl = arg; - printk("%s %pSb\n", loglvl, (void *)where); + printk("%s %pSb\n", loglvl, (void *)state->common.pc); return true; } @@ -316,7 +316,7 @@ void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk, return; printk("%sCall trace:\n", loglvl); - arch_stack_walk(dump_backtrace_entry, (void *)loglvl, tsk, regs); + kunwind_stack_walk(dump_backtrace_entry, (void *)loglvl, tsk, regs); put_task_stack(tsk); } From patchwork Thu Oct 17 09:25:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 13839715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8986BD21269 for ; Thu, 17 Oct 2024 09:43:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=bESDMZS3NcigCp8PSG1TV6cr8jtTfHTirn75c9qQLFs=; b=v9yU7baS3t18w/xivqVius2Eo3 HoOGUdH2FfoS+FlZZPFEhy7brBBmyWhfPEPtPmu9/PugZjLVjo85Cbnra4OfI29birD8fvU/Hbv/r pLROwUST3UxfCZt4cbcqNWoyVXZ20FAfQ+taehsYFevpCgoKdyT0pRkmhp0rHAUsX61b/c5PbJQDG QBVfLZaBSXOBiXjcwQuhZ5TgfrYyIcEQ7gRdTP/D7VF8vruUHE9lc7oKgaETqqupNnUFMN2lGVMlU ZNXa5MwwhNRNY5pokvKp/ue3vjXJDouQFPkWAbIxRO+Cp70uQFOGGz8mEAxCLO0N7JKzQNlw1bElZ SAKoA6hw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1N2G-0000000ENsT-44CL; Thu, 17 Oct 2024 09:43:12 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1Mlh-0000000EKiw-0pXR for linux-arm-kernel@lists.infradead.org; Thu, 17 Oct 2024 09:26:06 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 719B31477; Thu, 17 Oct 2024 02:26:34 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id BF9553F528; Thu, 17 Oct 2024 02:26:02 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, jpoimboe@kernel.org, kaleshsingh@google.com, madvenka@linux.microsoft.com, mark.rutland@arm.com, maz@kernel.org, mbenes@suse.cz, puranjay12@gmail.com, will@kernel.org Subject: [PATCH v2 07/10] arm64: stacktrace: report source of unwind data Date: Thu, 17 Oct 2024 10:25:35 +0100 Message-Id: <20241017092538.1859841-8-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20241017092538.1859841-1-mark.rutland@arm.com> References: <20241017092538.1859841-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241017_022605_349431_9AF2942C X-CRM114-Status: GOOD ( 17.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When analysing a stacktrace it can be useful to know where an unwound PC came from, as in some situations certain sources may be suspect or known to be unreliable. In future it would also be useful to track this so that certain unwind steps can be performed in a stateful manner. For example when unwinding across an exception boundary, we'd ideally unwind pt_regs::pc, then pt_regs::lr, then the next frame record. This patch adds an enumerated set of unwind sources, tracks this during the unwind, and updates dump_backtrace() to log these for interesting unwind steps. The interesting sources recorded are: "C" - the PC came from the caller of an unwind function. "T" - the PC came from thread_saved_pc() for a blocked task. "P" - the PC came from a pt_regs::pc. "U" - the PC came from an unknown source (indicates an unwinder error). ... with nothing recorded when the PC came from a frame_record::pc as this is the vastly common case and logging this would make it difficult to spot the more interesting cases. For example, when triggering a backtrace via magic-sysrq + L, the CPU handling the sysrq will have a backtrace whose first element is the caller (C) of dump_backtrace(): | Call trace: | show_stack+0x18/0x30 (C) | dump_stack_lvl+0x60/0x80 | dump_stack+0x18/0x24 | nmi_cpu_backtrace+0xfc/0x140 | ... ... and other CPUs will have a backtrace whose first element is their pt_regs::pc (P) at the instant the backtrace IPI was taken: | Call trace: | _raw_spin_unlock_irqrestore+0x8/0x50 (P) | wake_up_process+0x18/0x24 | process_timeout+0x14/0x20 | call_timer_fn.isra.0+0x24/0x80 | ... Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Reviewed-by: Miroslav Benes Reviewed-by: Puranjay Mohan Cc: Ard Biesheuvel Cc: Josh Poimboeuf Cc: Kalesh Singh Cc: Madhavan T. Venkataraman Cc: Marc Zyngier Cc: Will Deacon --- arch/arm64/kernel/stacktrace.c | 47 +++++++++++++++++++++++++++++++--- 1 file changed, 43 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 22f8709f0e763..83ab37e13159e 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -20,6 +20,14 @@ #include #include +enum kunwind_source { + KUNWIND_SOURCE_UNKNOWN, + KUNWIND_SOURCE_FRAME, + KUNWIND_SOURCE_CALLER, + KUNWIND_SOURCE_TASK, + KUNWIND_SOURCE_REGS_PC, +}; + /* * Kernel unwind state * @@ -37,6 +45,7 @@ struct kunwind_state { #ifdef CONFIG_KRETPROBES struct llist_node *kr_cur; #endif + enum kunwind_source source; }; static __always_inline void @@ -45,6 +54,7 @@ kunwind_init(struct kunwind_state *state, { unwind_init_common(&state->common); state->task = task; + state->source = KUNWIND_SOURCE_UNKNOWN; } /* @@ -62,6 +72,7 @@ kunwind_init_from_regs(struct kunwind_state *state, state->common.fp = regs->regs[29]; state->common.pc = regs->pc; + state->source = KUNWIND_SOURCE_REGS_PC; } /* @@ -79,6 +90,7 @@ kunwind_init_from_caller(struct kunwind_state *state) state->common.fp = (unsigned long)__builtin_frame_address(1); state->common.pc = (unsigned long)__builtin_return_address(0); + state->source = KUNWIND_SOURCE_CALLER; } /* @@ -99,6 +111,7 @@ kunwind_init_from_task(struct kunwind_state *state, state->common.fp = thread_saved_fp(task); state->common.pc = thread_saved_pc(task); + state->source = KUNWIND_SOURCE_TASK; } static __always_inline int @@ -148,9 +161,19 @@ kunwind_next(struct kunwind_state *state) if (fp == (unsigned long)&task_pt_regs(tsk)->stackframe) return -ENOENT; - err = unwind_next_frame_record(&state->common); - if (err) - return err; + switch (state->source) { + case KUNWIND_SOURCE_FRAME: + case KUNWIND_SOURCE_CALLER: + case KUNWIND_SOURCE_TASK: + case KUNWIND_SOURCE_REGS_PC: + err = unwind_next_frame_record(&state->common); + if (err) + return err; + state->source = KUNWIND_SOURCE_FRAME; + break; + default: + return -EINVAL; + } state->common.pc = ptrauth_strip_kernel_insn_pac(state->common.pc); @@ -294,10 +317,26 @@ noinline noinstr void arch_bpf_stack_walk(bool (*consume_entry)(void *cookie, u6 kunwind_stack_walk(arch_bpf_unwind_consume_entry, &data, current, NULL); } +static const char *state_source_string(const struct kunwind_state *state) +{ + switch (state->source) { + case KUNWIND_SOURCE_FRAME: return NULL; + case KUNWIND_SOURCE_CALLER: return "C"; + case KUNWIND_SOURCE_TASK: return "T"; + case KUNWIND_SOURCE_REGS_PC: return "P"; + default: return "U"; + } +} + static bool dump_backtrace_entry(const struct kunwind_state *state, void *arg) { + const char *source = state_source_string(state); char *loglvl = arg; - printk("%s %pSb\n", loglvl, (void *)state->common.pc); + printk("%s %pSb%s%s%s\n", loglvl, + (void *)state->common.pc, + source ? " (" : "", + source ? source : "", + source ? ")" : ""); return true; } From patchwork Thu Oct 17 09:25:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 13839716 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 752AED21269 for ; Thu, 17 Oct 2024 09:44:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=7LJ+YLf0BwqUkC0mTf7O5hNS0S8STTI1qDzjKylgYXg=; b=WofBh5rSq9hQ+azY4kpcSj6SSJ TNth0UOQpdV2w3qQe21n0eY++2bBAxZTI6ARdaTTHy9Ojft0prE3uvtxkzwbpYPC3lDGq647FJuWC GAEqk0rMATsnh6GL4DnePcNXy+BQ6SS3hg+EaJmXZxLF13XxNiOQG710N0CzVEbQx5k9SkEGUmKW3 ixGsBxhvLjG2i8LGDIolY0Q9mvfZW5tOd9FUgGa423z6r7r0ORc+F4Ptm+KjO5VacWKubEXxoDLXJ jTfKiVlmR+SimdQj5GjlBqVojHvxZOOHaRaKefm8O/XlDzL0TaTFJowHbxIVkkzbuFhWWM8ltud/D mzKa1c/Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1N3k-0000000EOJy-2KtY; Thu, 17 Oct 2024 09:44:44 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1Mlk-0000000EKjQ-03n4 for linux-arm-kernel@lists.infradead.org; Thu, 17 Oct 2024 09:26:10 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 40FB4FEC; Thu, 17 Oct 2024 02:26:37 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 901623F528; Thu, 17 Oct 2024 02:26:05 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, jpoimboe@kernel.org, kaleshsingh@google.com, madvenka@linux.microsoft.com, mark.rutland@arm.com, maz@kernel.org, mbenes@suse.cz, puranjay12@gmail.com, will@kernel.org Subject: [PATCH v2 08/10] arm64: stacktrace: report recovered PCs Date: Thu, 17 Oct 2024 10:25:36 +0100 Message-Id: <20241017092538.1859841-9-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20241017092538.1859841-1-mark.rutland@arm.com> References: <20241017092538.1859841-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241017_022608_162158_219BC4CF X-CRM114-Status: GOOD ( 16.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When analysing a stacktrace it can be useful to know whether an unwound PC has been rewritten by fgraph or kretprobes, as in some situations these may be suspect or be known to be unreliable. This patch adds flags to track when an unwind entry has recovered the PC from fgraph and/or kretprobes, and updates dump_backtrace() to log when this is the case. The flags recorded are: "F" - the PC was recovered from fgraph "K" - the PC was recovered from kretprobes These flags are recorded and logged in addition to the original source of the unwound PC. For example, with the ftrace_graph profiler enabled globally, and kretprobes installed on generic_handle_domain_irq() and do_interrupt_handler(), a backtrace triggered by magic-sysrq + L reports: | Call trace: | show_stack+0x20/0x40 (CF) | dump_stack_lvl+0x60/0x80 (F) | dump_stack+0x18/0x28 | nmi_cpu_backtrace+0xfc/0x140 | nmi_trigger_cpumask_backtrace+0x1c8/0x200 | arch_trigger_cpumask_backtrace+0x20/0x40 | sysrq_handle_showallcpus+0x24/0x38 (F) | __handle_sysrq+0xa8/0x1b0 (F) | handle_sysrq+0x38/0x50 (F) | pl011_int+0x460/0x5a8 (F) | __handle_irq_event_percpu+0x60/0x220 (F) | handle_irq_event+0x54/0xc0 (F) | handle_fasteoi_irq+0xa8/0x1d0 (F) | generic_handle_domain_irq+0x34/0x58 (F) | gic_handle_irq+0x54/0x140 (FK) | call_on_irq_stack+0x24/0x58 (F) | do_interrupt_handler+0x88/0xa0 | el1_interrupt+0x34/0x68 (FK) | el1h_64_irq_handler+0x18/0x28 | el1h_64_irq+0x64/0x68 | default_idle_call+0x34/0x180 | do_idle+0x204/0x268 | cpu_startup_entry+0x40/0x50 (F) | rest_init+0xe4/0xf0 | start_kernel+0x744/0x750 | __primary_switched+0x80/0x90 Note that as these flags are reported next to the recovered PC value, they appear on the callers of instrumented functions. For example gic_handle_irq() has a "K" marker because generic_handle_domain_irq() was instrumented with kretprobes and had its return address rewritten. Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Reviewed-by: Miroslav Benes Reviewed-by: Puranjay Mohan Cc: Ard Biesheuvel Cc: Josh Poimboeuf Cc: Kalesh Singh Cc: Madhavan T. Venkataraman Cc: Marc Zyngier Cc: Will Deacon --- arch/arm64/kernel/stacktrace.c | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 83ab37e13159e..f8e231683dad9 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -28,6 +28,14 @@ enum kunwind_source { KUNWIND_SOURCE_REGS_PC, }; +union unwind_flags { + unsigned long all; + struct { + unsigned long fgraph : 1, + kretprobe : 1; + }; +}; + /* * Kernel unwind state * @@ -46,6 +54,7 @@ struct kunwind_state { struct llist_node *kr_cur; #endif enum kunwind_source source; + union unwind_flags flags; }; static __always_inline void @@ -55,6 +64,7 @@ kunwind_init(struct kunwind_state *state, unwind_init_common(&state->common); state->task = task; state->source = KUNWIND_SOURCE_UNKNOWN; + state->flags.all = 0; } /* @@ -127,6 +137,7 @@ kunwind_recover_return_address(struct kunwind_state *state) if (WARN_ON_ONCE(state->common.pc == orig_pc)) return -EINVAL; state->common.pc = orig_pc; + state->flags.fgraph = 1; } #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ @@ -137,6 +148,7 @@ kunwind_recover_return_address(struct kunwind_state *state) (void *)state->common.fp, &state->kr_cur); state->common.pc = orig_pc; + state->flags.kretprobe = 1; } #endif /* CONFIG_KRETPROBES */ @@ -157,6 +169,8 @@ kunwind_next(struct kunwind_state *state) unsigned long fp = state->common.fp; int err; + state->flags.all = 0; + /* Final frame; nothing to unwind */ if (fp == (unsigned long)&task_pt_regs(tsk)->stackframe) return -ENOENT; @@ -331,12 +345,18 @@ static const char *state_source_string(const struct kunwind_state *state) static bool dump_backtrace_entry(const struct kunwind_state *state, void *arg) { const char *source = state_source_string(state); + union unwind_flags flags = state->flags; + bool has_info = source || flags.all; char *loglvl = arg; - printk("%s %pSb%s%s%s\n", loglvl, + + printk("%s %pSb%s%s%s%s%s\n", loglvl, (void *)state->common.pc, - source ? " (" : "", + has_info ? " (" : "", source ? source : "", - source ? ")" : ""); + flags.fgraph ? "F" : "", + flags.kretprobe ? "K" : "", + has_info ? ")" : ""); + return true; } From patchwork Thu Oct 17 09:25:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 13839717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CEB32D2126E for ; Thu, 17 Oct 2024 09:46:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=b9TSuIi5lXTJLC3mm+HFFHF6e72sr6VnlrolsFYua2M=; b=pPo08MPrwyhJna4uYPNuLO/K08 UlOF0ZC/BHt2VDHbUNxw74CiEhINf3PvdPJKGN3jZB7wNVXZIaKNRn07UZM7yoI0CeOEWXJXwUN8R 0bZZcqLHNPi6gBWl58AM1XmXXjnozXRndzU6pH42Os5X+d9JibPahUq1I8UZPCWMLNXhI7+1X1ES4 mqXSbjmTtAAedbfmvy6MIpPniUlnsdNI6R+5ZNM+QaQweElipzK/9po4QO1cgdNV41ihEdsFrpooc HfHydvtgFf33ITrwNdKHKid/zuzh8ylZ54+qJjcWegu/1Fddw80GUxjiggDr1oDRai086qjLGl9VX L4JMka2w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1N59-0000000EORU-1fYC; Thu, 17 Oct 2024 09:46:11 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1Mlm-0000000EKju-3S9n for linux-arm-kernel@lists.infradead.org; Thu, 17 Oct 2024 09:26:12 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 067801477; Thu, 17 Oct 2024 02:26:40 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 558C13F528; Thu, 17 Oct 2024 02:26:08 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, jpoimboe@kernel.org, kaleshsingh@google.com, madvenka@linux.microsoft.com, mark.rutland@arm.com, maz@kernel.org, mbenes@suse.cz, puranjay12@gmail.com, will@kernel.org Subject: [PATCH v2 09/10] arm64: stacktrace: split unwind_consume_stack() Date: Thu, 17 Oct 2024 10:25:37 +0100 Message-Id: <20241017092538.1859841-10-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20241017092538.1859841-1-mark.rutland@arm.com> References: <20241017092538.1859841-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241017_022610_992223_117B402C X-CRM114-Status: GOOD ( 22.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When unwinding stacks, we use unwind_consume_stack() to both find whether an object (e.g. a frame record) is on an accessible stack *and* to update the stack boundaries. This works fine today since we only care about one type of object which does not overlap other objects. In subsequent patches we'll want to check whether an object (e.g a frame record) is on the stack and follow this up by accessing a larger object containing the first (e.g. a pt_regs with an embedded frame record). To make that pattern easier to implement, this patch reworks unwind_find_next_stack() and unwind_consume_stack() so that the former can be used to check if an object is on any accessible stack, and the latter is purely used to update the stack boundaries. As unwind_find_next_stack() is modified to also check the stack currently being unwound, it is renamed to unwind_find_stack(). There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Reviewed-by: Miroslav Benes Reviewed-by: Puranjay Mohan Cc: Ard Biesheuvel Cc: Josh Poimboeuf Cc: Kalesh Singh Cc: Madhavan T. Venkataraman Cc: Marc Zyngier Cc: Will Deacon --- arch/arm64/include/asm/stacktrace/common.h | 68 +++++++++++++--------- 1 file changed, 39 insertions(+), 29 deletions(-) diff --git a/arch/arm64/include/asm/stacktrace/common.h b/arch/arm64/include/asm/stacktrace/common.h index 7fab6876e4970..821a8fdd31af4 100644 --- a/arch/arm64/include/asm/stacktrace/common.h +++ b/arch/arm64/include/asm/stacktrace/common.h @@ -60,13 +60,27 @@ static inline void unwind_init_common(struct unwind_state *state) state->stack = stackinfo_get_unknown(); } -static struct stack_info *unwind_find_next_stack(const struct unwind_state *state, - unsigned long sp, - unsigned long size) +/** + * unwind_find_stack() - Find the accessible stack which entirely contains an + * object. + * + * @state: the current unwind state. + * @sp: the base address of the object. + * @size: the size of the object. + * + * Return: a pointer to the relevant stack_info if found; NULL otherwise. + */ +static struct stack_info *unwind_find_stack(struct unwind_state *state, + unsigned long sp, + unsigned long size) { - for (int i = 0; i < state->nr_stacks; i++) { - struct stack_info *info = &state->stacks[i]; + struct stack_info *info = &state->stack; + if (stackinfo_on_stack(info, sp, size)) + return info; + + for (int i = 0; i < state->nr_stacks; i++) { + info = &state->stacks[i]; if (stackinfo_on_stack(info, sp, size)) return info; } @@ -75,36 +89,31 @@ static struct stack_info *unwind_find_next_stack(const struct unwind_state *stat } /** - * unwind_consume_stack() - Check if an object is on an accessible stack, - * updating stack boundaries so that future unwind steps cannot consume this - * object again. + * unwind_consume_stack() - Update stack boundaries so that future unwind steps + * cannot consume this object again. * * @state: the current unwind state. + * @info: the stack_info of the stack containing the object. * @sp: the base address of the object. * @size: the size of the object. * * Return: 0 upon success, an error code otherwise. */ -static inline int unwind_consume_stack(struct unwind_state *state, - unsigned long sp, - unsigned long size) +static inline void unwind_consume_stack(struct unwind_state *state, + struct stack_info *info, + unsigned long sp, + unsigned long size) { - struct stack_info *next; - - if (stackinfo_on_stack(&state->stack, sp, size)) - goto found; - - next = unwind_find_next_stack(state, sp, size); - if (!next) - return -EINVAL; + struct stack_info tmp; /* * Stack transitions are strictly one-way, and once we've * transitioned from one stack to another, it's never valid to * unwind back to the old stack. * - * Remove the current stack from the list of stacks so that it cannot - * be found on a subsequent transition. + * Destroy the old stack info so that it cannot be found upon a + * subsequent transition. If the stack has not changed, we'll + * immediately restore the current stack info. * * Note that stacks can nest in several valid orders, e.g. * @@ -115,16 +124,15 @@ static inline int unwind_consume_stack(struct unwind_state *state, * ... so we do not check the specific order of stack * transitions. */ - state->stack = *next; - *next = stackinfo_get_unknown(); + tmp = *info; + *info = stackinfo_get_unknown(); + state->stack = tmp; -found: /* * Future unwind steps can only consume stack above this frame record. * Update the current stack to start immediately above it. */ state->stack.low = sp + size; - return 0; } /** @@ -137,16 +145,18 @@ static inline int unwind_consume_stack(struct unwind_state *state, static inline int unwind_next_frame_record(struct unwind_state *state) { + struct stack_info *info; struct frame_record *record; unsigned long fp = state->fp; - int err; if (fp & 0x7) return -EINVAL; - err = unwind_consume_stack(state, fp, sizeof(*record)); - if (err) - return err; + info = unwind_find_stack(state, fp, sizeof(*record)); + if (!info) + return -EINVAL; + + unwind_consume_stack(state, info, fp, sizeof(*record)); /* * Record this frame record's values. From patchwork Thu Oct 17 09:25:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 13839756 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ACA97D21269 for ; Thu, 17 Oct 2024 09:47:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Nj6PrUdkOwepPcQ0D+rOepBAO40p4P7jic+25PLaCvI=; b=36B9+4wfMZ1+v2CtkinGpWUqWC b/xT9pAxFddF1r9lY9kydOWigG6rhMNzkYoWUY+M6F1s8IM6TvSTBBT/gzgTRyxyVOMKpt4qvepb6 8zZMea8nd1DSh9JtfkwxbbIDovfgw25rloMr0Bgw6CR8zu5YhgDZAp9IdK7FLtAVmK/Cm7Cl5cQns iIaColGEpjYXcqVUJZPQaQ4yKdjBIGqW5K0+Tq2RDum/ko+HHnLiUFpkMNFn4+ZUaDCVv7H2MNnfe ZuC9gN8u5iM2fp55sAOV/RwSmakB/U303hWL0tWRoBlkuPUpeZM3JO0BGJKfk8uHxasnudVA9P2R7 0SqpBwQg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1N6b-0000000EOmS-1OdJ; Thu, 17 Oct 2024 09:47:41 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1t1Mlq-0000000EKkb-2UY4 for linux-arm-kernel@lists.infradead.org; Thu, 17 Oct 2024 09:26:16 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E53B8FEC; Thu, 17 Oct 2024 02:26:43 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 403553F528; Thu, 17 Oct 2024 02:26:12 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ardb@kernel.org, broonie@kernel.org, catalin.marinas@arm.com, jpoimboe@kernel.org, kaleshsingh@google.com, madvenka@linux.microsoft.com, mark.rutland@arm.com, maz@kernel.org, mbenes@suse.cz, puranjay12@gmail.com, will@kernel.org Subject: [PATCH v2 10/10] arm64: stacktrace: unwind exception boundaries Date: Thu, 17 Oct 2024 10:25:38 +0100 Message-Id: <20241017092538.1859841-11-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20241017092538.1859841-1-mark.rutland@arm.com> References: <20241017092538.1859841-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241017_022614_749887_DE4F4112 X-CRM114-Status: GOOD ( 32.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When arm64's stack unwinder encounters an exception boundary, it uses the pt_regs::stackframe created by the entry code, which has a copy of the PC and FP at the time the exception was taken. The unwinder doesn't know anything about pt_regs, and reports the PC from the stackframe, but does not report the LR. The LR is only guaranteed to contain the return address at function call boundaries, and can be used as a scratch register at other times, so the LR at an exception boundary may or may not be a legitimate return address. It would be useful to report the LR value regardless, as it can be helpful when debugging, and in future it will be helpful for reliable stacktrace support. This patch changes the way we unwind across exception boundaries, allowing both the PC and LR to be reported. The entry code creates a frame_record_meta structure embedded within pt_regs, which the unwinder uses to find the pt_regs. The unwinder can then extract pt_regs::pc and pt_regs::lr as two separate unwind steps before continuing with a regular walk of frame records. When a PC is unwound from pt_regs::lr, dump_backtrace() will log this with an "L" marker so that it can be identified easily. For example, an unwind across an exception boundary will appear as follows: | el1h_64_irq+0x6c/0x70 | _raw_spin_unlock_irqrestore+0x10/0x60 (P) | __aarch64_insn_write+0x6c/0x90 (L) | aarch64_insn_patch_text_nosync+0x28/0x80 ... with a (P) entry for pt_regs::pc, and an (L) entry for pt_regs:lr. Note that the LR may be stale at the point of the exception, for example, shortly after a return: | el1h_64_irq+0x6c/0x70 | default_idle_call+0x34/0x180 (P) | default_idle_call+0x28/0x180 (L) | do_idle+0x204/0x268 ... where the LR points a few instructions before the current PC. This plays nicely with all the other unwind metadata tracking. With the ftrace_graph profiler enabled globally, and kretprobes installed on generic_handle_domain_irq() and do_interrupt_handler(), a backtrace triggered by magic-sysrq + L reports: | Call trace: | show_stack+0x20/0x40 (CF) | dump_stack_lvl+0x60/0x80 (F) | dump_stack+0x18/0x28 | nmi_cpu_backtrace+0xfc/0x140 | nmi_trigger_cpumask_backtrace+0x1c8/0x200 | arch_trigger_cpumask_backtrace+0x20/0x40 | sysrq_handle_showallcpus+0x24/0x38 (F) | __handle_sysrq+0xa8/0x1b0 (F) | handle_sysrq+0x38/0x50 (F) | pl011_int+0x460/0x5a8 (F) | __handle_irq_event_percpu+0x60/0x220 (F) | handle_irq_event+0x54/0xc0 (F) | handle_fasteoi_irq+0xa8/0x1d0 (F) | generic_handle_domain_irq+0x34/0x58 (F) | gic_handle_irq+0x54/0x140 (FK) | call_on_irq_stack+0x24/0x58 (F) | do_interrupt_handler+0x88/0xa0 | el1_interrupt+0x34/0x68 (FK) | el1h_64_irq_handler+0x18/0x28 | el1h_64_irq+0x6c/0x70 | default_idle_call+0x34/0x180 (P) | default_idle_call+0x28/0x180 (L) | do_idle+0x204/0x268 | cpu_startup_entry+0x3c/0x50 (F) | rest_init+0xe4/0xf0 | start_kernel+0x744/0x750 | __primary_switched+0x88/0x98 Signed-off-by: Mark Rutland Reviewed-by: Mark Brown Reviewed-by: Miroslav Benes Reviewed-by: Puranjay Mohan Cc: Ard Biesheuvel Cc: Josh Poimboeuf Cc: Kalesh Singh Cc: Madhavan T. Venkataraman Cc: Marc Zyngier Cc: Will Deacon --- arch/arm64/include/asm/ptrace.h | 4 +- arch/arm64/include/asm/stacktrace/frame.h | 35 +++++++ arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/entry.S | 12 ++- arch/arm64/kernel/head.S | 3 + arch/arm64/kernel/process.c | 1 + arch/arm64/kernel/stacktrace.c | 121 ++++++++++++++++++++-- 7 files changed, 158 insertions(+), 19 deletions(-) diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index 89c02f85f4b11..47ff8654c5ec1 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -168,9 +168,7 @@ struct pt_regs { u32 pmr; u64 sdei_ttbr1; - u64 unused; - - struct frame_record stackframe; + struct frame_record_meta stackframe; /* Only valid for some EL1 exceptions. */ u64 lockdep_hardirqs; diff --git a/arch/arm64/include/asm/stacktrace/frame.h b/arch/arm64/include/asm/stacktrace/frame.h index 6397bc847f147..0ee0f6ba0fd86 100644 --- a/arch/arm64/include/asm/stacktrace/frame.h +++ b/arch/arm64/include/asm/stacktrace/frame.h @@ -3,6 +3,30 @@ #define __ASM_STACKTRACE_FRAME_H /* + * - FRAME_META_TYPE_NONE + * + * This value is reserved. + * + * - FRAME_META_TYPE_FINAL + * + * The record is the last entry on the stack. + * Unwinding should terminate successfully. + * + * - FRAME_META_TYPE_PT_REGS + * + * The record is embedded within a struct pt_regs, recording the registers at + * an arbitrary point in time. + * Unwinding should consume pt_regs::pc, followed by pt_regs::lr. + * + * Note: all other values are reserved and should result in unwinding + * terminating with an error. + */ +#define FRAME_META_TYPE_NONE 0 +#define FRAME_META_TYPE_FINAL 1 +#define FRAME_META_TYPE_PT_REGS 2 + +#ifndef __ASSEMBLY__ +/* * A standard AAPCS64 frame record. */ struct frame_record { @@ -10,4 +34,15 @@ struct frame_record { u64 lr; }; +/* + * A metadata frame record indicating a special unwind. + * The record::{fp,lr} fields must be zero to indicate the presence of + * metadata. + */ +struct frame_record_meta { + struct frame_record record; + u64 type; +}; +#endif /* __ASSEMBLY */ + #endif /* __ASM_STACKTRACE_FRAME_H */ diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index eb7fb2f9b9274..021f04f97fde5 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -81,6 +81,7 @@ int main(void) DEFINE(S_SDEI_TTBR1, offsetof(struct pt_regs, sdei_ttbr1)); DEFINE(S_PMR, offsetof(struct pt_regs, pmr)); DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe)); + DEFINE(S_STACKFRAME_TYPE, offsetof(struct pt_regs, stackframe.type)); DEFINE(PT_REGS_SIZE, sizeof(struct pt_regs)); BLANK(); #ifdef CONFIG_DYNAMIC_FTRACE_WITH_ARGS diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index fa6d6d5ca5e02..5ae2a34b50bda 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -25,6 +25,7 @@ #include #include #include +#include #include #include #include @@ -284,15 +285,16 @@ alternative_else_nop_endif stp lr, x21, [sp, #S_LR] /* - * For exceptions from EL0, create a final frame record. - * For exceptions from EL1, create a synthetic frame record so the - * interrupted code shows up in the backtrace. + * Create a metadata frame record. The unwinder will use this to + * identify and unwind exception boundaries. */ - .if \el == 0 stp xzr, xzr, [sp, #S_STACKFRAME] + .if \el == 0 + mov x0, #FRAME_META_TYPE_FINAL .else - stp x29, x22, [sp, #S_STACKFRAME] + mov x0, #FRAME_META_TYPE_PT_REGS .endif + str x0, [sp, #S_STACKFRAME_TYPE] add x29, sp, #S_STACKFRAME #ifdef CONFIG_ARM64_SW_TTBR0_PAN diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index cb68adcabe078..5ab1970ee5436 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -32,6 +32,7 @@ #include #include #include +#include #include #include @@ -199,6 +200,8 @@ SYM_CODE_END(preserve_boot_args) sub sp, sp, #PT_REGS_SIZE stp xzr, xzr, [sp, #S_STACKFRAME] + mov \tmp1, #FRAME_META_TYPE_FINAL + str \tmp1, [sp, #S_STACKFRAME_TYPE] add x29, sp, #S_STACKFRAME scs_load_current diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index d45fd114eac3f..29904c829de25 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -409,6 +409,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) */ memset(childregs, 0, sizeof(struct pt_regs)); childregs->pstate = PSR_MODE_EL1h | PSR_IL_BIT; + childregs->stackframe.type = FRAME_META_TYPE_FINAL; p->thread.cpu_context.x19 = (unsigned long)args->fn; p->thread.cpu_context.x20 = (unsigned long)args->fn_arg; diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index f8e231683dad9..caef85462acb6 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -26,6 +26,7 @@ enum kunwind_source { KUNWIND_SOURCE_CALLER, KUNWIND_SOURCE_TASK, KUNWIND_SOURCE_REGS_PC, + KUNWIND_SOURCE_REGS_LR, }; union unwind_flags { @@ -55,6 +56,7 @@ struct kunwind_state { #endif enum kunwind_source source; union unwind_flags flags; + struct pt_regs *regs; }; static __always_inline void @@ -65,6 +67,7 @@ kunwind_init(struct kunwind_state *state, state->task = task; state->source = KUNWIND_SOURCE_UNKNOWN; state->flags.all = 0; + state->regs = NULL; } /* @@ -80,6 +83,7 @@ kunwind_init_from_regs(struct kunwind_state *state, { kunwind_init(state, current); + state->regs = regs; state->common.fp = regs->regs[29]; state->common.pc = regs->pc; state->source = KUNWIND_SOURCE_REGS_PC; @@ -155,6 +159,103 @@ kunwind_recover_return_address(struct kunwind_state *state) return 0; } +static __always_inline +int kunwind_next_regs_pc(struct kunwind_state *state) +{ + struct stack_info *info; + unsigned long fp = state->common.fp; + struct pt_regs *regs; + + regs = container_of((u64 *)fp, struct pt_regs, stackframe.record.fp); + + info = unwind_find_stack(&state->common, (unsigned long)regs, sizeof(*regs)); + if (!info) + return -EINVAL; + + unwind_consume_stack(&state->common, info, (unsigned long)regs, + sizeof(*regs)); + + state->regs = regs; + state->common.pc = regs->pc; + state->common.fp = regs->regs[29]; + state->source = KUNWIND_SOURCE_REGS_PC; + return 0; +} + +static __always_inline int +kunwind_next_regs_lr(struct kunwind_state *state) +{ + /* + * The stack for the regs was consumed by kunwind_next_regs_pc(), so we + * cannot consume that again here, but we know the regs are safe to + * access. + */ + state->common.pc = state->regs->regs[30]; + state->common.fp = state->regs->regs[29]; + state->regs = NULL; + state->source = KUNWIND_SOURCE_REGS_LR; + + return 0; +} + +static __always_inline int +kunwind_next_frame_record_meta(struct kunwind_state *state) +{ + struct task_struct *tsk = state->task; + unsigned long fp = state->common.fp; + struct frame_record_meta *meta; + struct stack_info *info; + + info = unwind_find_stack(&state->common, fp, sizeof(*meta)); + if (!info) + return -EINVAL; + + meta = (struct frame_record_meta *)fp; + switch (READ_ONCE(meta->type)) { + case FRAME_META_TYPE_FINAL: + if (meta == &task_pt_regs(tsk)->stackframe) + return -ENOENT; + WARN_ON_ONCE(1); + return -EINVAL; + case FRAME_META_TYPE_PT_REGS: + return kunwind_next_regs_pc(state); + default: + WARN_ON_ONCE(1); + return -EINVAL; + } +} + +static __always_inline int +kunwind_next_frame_record(struct kunwind_state *state) +{ + unsigned long fp = state->common.fp; + struct frame_record *record; + struct stack_info *info; + unsigned long new_fp, new_pc; + + if (fp & 0x7) + return -EINVAL; + + info = unwind_find_stack(&state->common, fp, sizeof(*record)); + if (!info) + return -EINVAL; + + record = (struct frame_record *)fp; + new_fp = READ_ONCE(record->fp); + new_pc = READ_ONCE(record->lr); + + if (!new_fp && !new_pc) + return kunwind_next_frame_record_meta(state); + + unwind_consume_stack(&state->common, info, fp, sizeof(*record)); + + state->common.fp = new_fp; + state->common.pc = new_pc; + state->source = KUNWIND_SOURCE_FRAME; + + return 0; +} + /* * Unwind from one frame record (A) to the next frame record (B). * @@ -165,30 +266,27 @@ kunwind_recover_return_address(struct kunwind_state *state) static __always_inline int kunwind_next(struct kunwind_state *state) { - struct task_struct *tsk = state->task; - unsigned long fp = state->common.fp; int err; state->flags.all = 0; - /* Final frame; nothing to unwind */ - if (fp == (unsigned long)&task_pt_regs(tsk)->stackframe) - return -ENOENT; - switch (state->source) { case KUNWIND_SOURCE_FRAME: case KUNWIND_SOURCE_CALLER: case KUNWIND_SOURCE_TASK: + case KUNWIND_SOURCE_REGS_LR: + err = kunwind_next_frame_record(state); + break; case KUNWIND_SOURCE_REGS_PC: - err = unwind_next_frame_record(&state->common); - if (err) - return err; - state->source = KUNWIND_SOURCE_FRAME; + err = kunwind_next_regs_lr(state); break; default: - return -EINVAL; + err = -EINVAL; } + if (err) + return err; + state->common.pc = ptrauth_strip_kernel_insn_pac(state->common.pc); return kunwind_recover_return_address(state); @@ -338,6 +436,7 @@ static const char *state_source_string(const struct kunwind_state *state) case KUNWIND_SOURCE_CALLER: return "C"; case KUNWIND_SOURCE_TASK: return "T"; case KUNWIND_SOURCE_REGS_PC: return "P"; + case KUNWIND_SOURCE_REGS_LR: return "L"; default: return "U"; } }