From patchwork Sun May 16 04:00:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12260151 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82063C433B4 for ; Sun, 16 May 2021 04:04:43 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 01C0660C3E for ; Sun, 16 May 2021 04:04:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 01C0660C3E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=5SYq6iROLh1qjsqH/vN6MA/4BnHgGHqiMQfoeHfk9SA=; b=SD0yYFROlilaSy0PWebR8UsKE kv2eES5jklmRqKn5aqriEvRI+RXPL0lg0PsMLXsNNxvc9YZCK0+ZlBuajLHqwkqdwDG0UobjmM+LG YKZDEyhDmIL8xVsGgERX4NbWRj1dIflO6Xv/qE9U3xoPewMnf+ljPw9lkwglsIooXgmB9zwmPWZXw G25qko08XjRc48vTO4cjdQv8pEofBMOa3RJADxgm0IyS0NTBEphazS8TLA4zu5ddhH1u4X9esJuwc 8XM5ioz0WOAFEeeKzWkR90d3bg/QJ0zRK/Bqio7CQb90nO2xlsp326i8w6RKw/5sQ9H9IvPRcfLPv p2jGGY4Fw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1li7yC-00BVxR-6O; Sun, 16 May 2021 04:01:37 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1li7xF-00BVuW-M6 for linux-arm-kernel@desiato.infradead.org; Sun, 16 May 2021 04:00:53 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender: Reply-To:Cc:Content-Type:Content-ID:Content-Description; bh=Dz0YnSe+6/RNJMKZ7AszyCNm1xp4eykmTR9aH7S0FJM=; b=OqAbHIaPdwFvU9JVJrmtJTnQLc FZMtj53kMfyU8FAQTBdWEs5+8t/xaX2UJ2/dXhdbHV7nn/0ThASAMqD29k82IEs58VJOkxZ8JEQH5 zjO7+wGDXivb60ntumGYZyESAjlPTOmgNjuz1dCn7Ow0sVSCnakDceO7sxuZItcOgfy1uhheptOjr jhnB6Qf9LGlopkllyAQd5GXTRrAeExx3ISRSW7xNi2BuI1jRfZ2XjWGlLwWb3PcMhRWzC2GG4DEDF cZr4QRrcdHCvN5sJiGjxsSKXrRbgGSDnsRCoQcVvi29kqu5BEe1GXnNWJkSw+Z7TOxSiXICrcqifJ D4lW4Cdw==; Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1li7xC-00ClH2-7W for linux-arm-kernel@lists.infradead.org; Sun, 16 May 2021 04:00:36 +0000 Received: from x64host.home (unknown [47.187.223.33]) by linux.microsoft.com (Postfix) with ESMTPSA id 2268D20B8006; Sat, 15 May 2021 21:00:31 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 2268D20B8006 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1621137631; bh=Dz0YnSe+6/RNJMKZ7AszyCNm1xp4eykmTR9aH7S0FJM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=GMcyFxcaYX8ZIE/ZdYxSnasZZUTSox1Yci1qO2nQ0UCupX+ELPZ0SlrCDCRYatm/P Uhnr0/CQ4XpcpDqdPYkTi8409gaYkAyjnZ5JD46jYu+l8819HSsKvUmqRabYYGvXrv E5YnBY7/oax0DrIUMFBa82d/SJv9xw/vxZqiU9mo= From: madvenka@linux.microsoft.com To: broonie@kernel.org, mark.rutland@arm.com, jpoimboe@redhat.com, ardb@kernel.org, jthierry@redhat.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, pasha.tatashin@soleen.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v4 1/2] arm64: Introduce stack trace reliability checks in the unwinder Date: Sat, 15 May 2021 23:00:17 -0500 Message-Id: <20210516040018.128105-2-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210516040018.128105-1-madvenka@linux.microsoft.com> References: <68eeda61b3e9579d65698a884b26c8632025e503> <20210516040018.128105-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210515_210034_395583_FF903AB3 X-CRM114-Status: GOOD ( 19.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" The unwinder should check for the presence of various features and conditions that can render the stack trace unreliable and mark the the stack trace as unreliable for the benefit of the caller. Introduce the first reliability check - If a return PC is not a valid kernel text address, consider the stack trace unreliable. It could be some generated code. Other reliability checks will be added in the future. Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/include/asm/stacktrace.h | 4 ++++ arch/arm64/kernel/stacktrace.c | 35 ++++++++++++++++++++++++----- 2 files changed, 34 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index eb29b1fe8255..f1eab6b029f7 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -49,6 +49,8 @@ struct stack_info { * * @graph: When FUNCTION_GRAPH_TRACER is selected, holds the index of a * replacement lr value in the ftrace graph stack. + * + * @reliable: Is this stack frame reliable? */ struct stackframe { unsigned long fp; @@ -59,6 +61,7 @@ struct stackframe { #ifdef CONFIG_FUNCTION_GRAPH_TRACER int graph; #endif + bool reliable; }; extern int unwind_frame(struct task_struct *tsk, struct stackframe *frame); @@ -169,6 +172,7 @@ static inline void start_backtrace(struct stackframe *frame, bitmap_zero(frame->stacks_done, __NR_STACK_TYPES); frame->prev_fp = 0; frame->prev_type = STACK_TYPE_UNKNOWN; + frame->reliable = true; } #endif /* __ASM_STACKTRACE_H */ diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index d55bdfb7789c..d38232cab3ee 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -44,21 +44,29 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) unsigned long fp = frame->fp; struct stack_info info; + frame->reliable = true; + /* Terminal record; nothing to unwind */ if (!fp) return -ENOENT; - if (fp & 0xf) + if (fp & 0xf) { + frame->reliable = false; return -EINVAL; + } if (!tsk) tsk = current; - if (!on_accessible_stack(tsk, fp, &info)) + if (!on_accessible_stack(tsk, fp, &info)) { + frame->reliable = false; return -EINVAL; + } - if (test_bit(info.type, frame->stacks_done)) + if (test_bit(info.type, frame->stacks_done)) { + frame->reliable = false; return -EINVAL; + } /* * As stacks grow downward, any valid record on the same stack must be @@ -74,8 +82,10 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) * stack. */ if (info.type == frame->prev_type) { - if (fp <= frame->prev_fp) + if (fp <= frame->prev_fp) { + frame->reliable = false; return -EINVAL; + } } else { set_bit(frame->prev_type, frame->stacks_done); } @@ -100,14 +110,29 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) * So replace it to an original value. */ ret_stack = ftrace_graph_get_ret_stack(tsk, frame->graph++); - if (WARN_ON_ONCE(!ret_stack)) + if (WARN_ON_ONCE(!ret_stack)) { + frame->reliable = false; return -EINVAL; + } frame->pc = ret_stack->ret; } #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ frame->pc = ptrauth_strip_insn_pac(frame->pc); + /* + * Check the return PC for conditions that make unwinding unreliable. + * In each case, mark the stack trace as such. + */ + + /* + * Make sure that the return address is a proper kernel text address. + * A NULL or invalid return address probably means there's some + * generated code which __kernel_text_address() doesn't know about. + */ + if (!__kernel_text_address(frame->pc)) + frame->reliable = false; + return 0; } NOKPROBE_SYMBOL(unwind_frame); From patchwork Sun May 16 04:00:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12260155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26040C433B4 for ; Sun, 16 May 2021 04:05:03 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9F5F660C3E for ; Sun, 16 May 2021 04:05:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9F5F660C3E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/qiZdlzfPetckuEFnwK1SLQ515r21LZEhFd54wHaHco=; b=Dj48FIbUfRPNW+qCjL44KcEdy uNoC7IX7R0RIiVbYqv3pCAvYblNyA8q7MNz73O5nPYvJVIMk3VAbvb6fsIH4QIJzKsHIRuDPNnrgs S/asW3MGU72K9hdXPKufod5ggjrfH52HDm76siWMZQxKMCy0PlLK1ATVJokL3BXIQR2lT1FjoCmr+ efHwpoXLkAKBC5EJpxvtZCLbkfLwczqVvXbLFzNF0JCIqb5VurbebAqUSfe6bL80EqI/p9gEiBylF 6ne1XW33qbK/QHqO3tdw/JJqsgdFpKNkQFQcazMuHaKxFxQyBMhuM05rayLCaL7T61vSGPKeKmA27 efdLmPMFA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1li7zK-00BW1z-Jx; Sun, 16 May 2021 04:02:46 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1li7xG-00BVuY-Sa for linux-arm-kernel@desiato.infradead.org; Sun, 16 May 2021 04:00:53 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender: Reply-To:Cc:Content-Type:Content-ID:Content-Description; bh=PYPweDHqRbvaXjH+hEzoM6fCWoLSO1BbHSyYQNeiAMk=; b=H/j17LRBd1KNFaWvy67my2mn8D oh2fCJpV53ZFE136ouD5duTt5nVpGKaRHmcpSC0L8GfwLGJ34VWsFSQhkKykw9wtizQC7hcFM6I2J omkQQ+M6rtUwC4/vWoXhzMmn44Y+/BKbCrT0S1mmnz6kVE7Udsi8fvlotjGGGoH7tVc3CWvQOH7jW aC9DhSyfpEkGltJx8UB8naiT+Yl7d1cHFfWovxfd+CB1avXP+6VmkaN0zhxkMTyEGpvCCowmZlJY5 pNxG0oQafXaAH74Xgh6LN0pVyg9Cr/WNvxMOCZe6lmFLWY7VOHoXADHsx+c6HbYRBAHLl3IhZY2QK 8CxTdZSw==; Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1li7xC-00ClH4-C5 for linux-arm-kernel@lists.infradead.org; Sun, 16 May 2021 04:00:37 +0000 Received: from x64host.home (unknown [47.187.223.33]) by linux.microsoft.com (Postfix) with ESMTPSA id 225D420B8008; Sat, 15 May 2021 21:00:32 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 225D420B8008 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1621137632; bh=PYPweDHqRbvaXjH+hEzoM6fCWoLSO1BbHSyYQNeiAMk=; h=From:To:Subject:Date:In-Reply-To:References:From; b=QyEgrz8a9rVQovyNyJ/dLIxwtCAnzK1ZnN5qpjEPwfKau0EWGxXXLSwnnW8u86VUS u91FVwf7wRqQnGXWJwXXeI4lFm2udMShxrRJOsj0RP+0SpDdx+gD59Ym35yhzx2IS5 wCp5srgruELA42KE6obHzTkdaxOeJSvgqgxvfNDY= From: madvenka@linux.microsoft.com To: broonie@kernel.org, mark.rutland@arm.com, jpoimboe@redhat.com, ardb@kernel.org, jthierry@redhat.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, pasha.tatashin@soleen.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [RFC PATCH v4 2/2] arm64: Create a list of SYM_CODE functions, blacklist them in the unwinder Date: Sat, 15 May 2021 23:00:18 -0500 Message-Id: <20210516040018.128105-3-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210516040018.128105-1-madvenka@linux.microsoft.com> References: <68eeda61b3e9579d65698a884b26c8632025e503> <20210516040018.128105-1-madvenka@linux.microsoft.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210515_210034_490336_D06001E8 X-CRM114-Status: GOOD ( 24.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" The unwinder should check if the return PC falls in any function that is considered unreliable from an unwinding perspective. If it does, mark the stack trace unreliable. Function types ============== The compiler generates code for C functions and assigns the type STT_FUNC to them. Assembly functions are manually assigned a type: - STT_FUNC for functions defined with SYM_FUNC*() macros - STT_NONE for functions defined with SYM_CODE*() macros In the future, STT_FUNC functions will be analyzed by objtool and "fixed" as necessary. So, they are not "interesting" to the reliable unwinder in the kernel. That leaves SYM_CODE*() functions. These contain low-level code that is difficult or impossible for objtool to analyze. So, objtool ignores them leaving them to the reliable unwinder. These functions must be blacklisted for unwinding in some way. Blacklisting functions ====================== Define a SYM_CODE_END() macro for arm64 that adds the function address range to a new section called "sym_code_functions". Linker file =========== Include the "sym_code_functions" section under initdata in vmlinux.lds.S. Initialization ============== Define an early_initcall() to copy the function address ranges from the "sym_code_functions" section to an array by the same name. Unwinder check ============== Define a function called unwinder_blacklisted() that compares a return PC with sym_code_functions[]. If there is a match, then mark the stack trace as unreliable. Call unwinder_blacklisted() from unwind_frame(). Signed-off-by: Madhavan T. Venkataraman --- arch/arm64/include/asm/linkage.h | 12 ++++++ arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/stacktrace.c | 61 ++++++++++++++++++++++++++++++- arch/arm64/kernel/vmlinux.lds.S | 7 ++++ 4 files changed, 80 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/linkage.h b/arch/arm64/include/asm/linkage.h index ba89a9af820a..3b5f1fd332b0 100644 --- a/arch/arm64/include/asm/linkage.h +++ b/arch/arm64/include/asm/linkage.h @@ -60,4 +60,16 @@ SYM_FUNC_END(x); \ SYM_FUNC_END_ALIAS(__pi_##x) +/* + * Record the address range of each SYM_CODE function in a struct code_range + * in a special section. + */ +#define SYM_CODE_END(name) \ + SYM_END(name, SYM_T_NONE) ;\ + 99: ;\ + .pushsection "sym_code_functions", "aw" ;\ + .quad name ;\ + .quad 99b ;\ + .popsection + #endif diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 2f36b16a5b5d..29cb566f65ec 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -20,5 +20,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __sym_code_functions_start[], __sym_code_functions_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index d38232cab3ee..f488425cacf1 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -18,6 +18,52 @@ #include #include +struct code_range { + unsigned long start; + unsigned long end; +}; + +static struct code_range *sym_code_functions; +static int num_sym_code_functions; + +int __init init_sym_code_functions(void) +{ + size_t size; + + size = (unsigned long)__sym_code_functions_end - + (unsigned long)__sym_code_functions_start; + + sym_code_functions = kmalloc(size, GFP_KERNEL); + if (!sym_code_functions) + return -ENOMEM; + + memcpy(sym_code_functions, __sym_code_functions_start, size); + /* Update num_sym_code_functions after copying sym_code_functions. */ + smp_mb(); + num_sym_code_functions = size / sizeof(struct code_range); + + return 0; +} +early_initcall(init_sym_code_functions); + +static bool unwinder_blacklisted(unsigned long pc) +{ + const struct code_range *range; + int i; + + /* + * If sym_code_functions[] were sorted, a binary search could be + * done to make this more performant. + */ + for (i = 0; i < num_sym_code_functions; i++) { + range = &sym_code_functions[i]; + if (pc >= range->start && pc < range->end) + return true; + } + + return false; +} + /* * AArch64 PCS assigns the frame pointer to x29. * @@ -130,7 +176,20 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) * A NULL or invalid return address probably means there's some * generated code which __kernel_text_address() doesn't know about. */ - if (!__kernel_text_address(frame->pc)) + if (!__kernel_text_address(frame->pc)) { + frame->reliable = false; + return 0; + } + + /* + * If the final frame has been reached, there is no more unwinding + * to do. There is no need to check if the return PC is blacklisted + * by the unwinder. + */ + if (!frame->fp) + return 0; + + if (unwinder_blacklisted(frame->pc)) frame->reliable = false; return 0; diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 7eea7888bb02..32e8d57397a1 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -103,6 +103,12 @@ jiffies = jiffies_64; #define TRAMP_TEXT #endif +#define SYM_CODE_FUNCTIONS \ + . = ALIGN(16); \ + __sym_code_functions_start = .; \ + KEEP(*(sym_code_functions)) \ + __sym_code_functions_end = .; + /* * The size of the PE/COFF section that covers the kernel image, which * runs from _stext to _edata, must be a round multiple of the PE/COFF @@ -218,6 +224,7 @@ SECTIONS CON_INITCALL INIT_RAM_FS *(.init.altinstructions .init.bss) /* from the EFI stub */ + SYM_CODE_FUNCTIONS } .exit.data : { EXIT_DATA