From patchwork Tue Apr 20 18:44:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Madhavan T. Venkataraman" X-Patchwork-Id: 12214937 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2810FC433ED for ; Tue, 20 Apr 2021 18:46:54 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 76BCC613D3 for ; Tue, 20 Apr 2021 18:46:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 76BCC613D3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=XDJqtmVdpk0hpEdQT18DZLvMxA+cZF2u/D6wRFARHis=; b=lazNKyi4suL8daLGR5wTJImJW m4Tl/xHqV6ZYagoy6+3BA5AapwLJwad+9acukjORXOQKtJZ8CVNS0HxVGZU2X0h2SRU6UgH45wzOy O3MsVKh2iICqJCczzgKiDcpGyz+0GztX+px+aqqnmg6RS3zJL5/gPjVsCVGWE3U4mlTLFSM1MVwo2 f8yHwOpgrDnAi/cJY0Gzf3Aezmn9Czm09/AbE1QLON0DvxlOqVWfsZWyTJrTsSSGHhgVxqoj6rD0v g0GbIXMw8+Z5lf3CYNqepOTyA4IbYICW3+iwEq8Sa3xWQ2FncWm9w1iKGEhI3Eg8/tOinFuIlNJpV uaIdRiQ/g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lYvN7-00CsGr-5h; Tue, 20 Apr 2021 18:45:19 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lYvN3-00CsGd-BH for linux-arm-kernel@desiato.infradead.org; Tue, 20 Apr 2021 18:45:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender: Reply-To:Cc:Content-Type:Content-ID:Content-Description; bh=0DaS2ZAaR3tWrI17aOD2elYuIto8dgrtjnc7v28PGzg=; b=aNjC1V7aTHEw8hCKk2L7yCNX3t vn6ozYUFrIISwH9xMiuXjrHscPsZwabbkH+7U2cLOVTDpYU+aRUH5vP4KOSw5tYz7UtKK3Ma/In/e ay3OgemTXYyKNxPuN4vSCBrgzrizvgfm5OnQIVxm/93P9rOcYBjb6Ha92cNZunZhhjy+TAj0LTenu 4YkxiE03Ozay3+gmRx5lGib6Q5dTQ9bqfprLKGg2alSosI5Hw7jV5YLiwtiHPRBDAHxnEjv3SIdbD eGZpkrFFoWDwH1BOFxfq1WkTa70cS9vLn9IuaxKBDii7nwe6FNITEL9wLwDzFwWBFpH7gw7Xcq6pe d6ftOUVQ==; Received: from linux.microsoft.com ([13.77.154.182]) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lYvMy-00CM8P-9B for linux-arm-kernel@lists.infradead.org; Tue, 20 Apr 2021 18:45:12 +0000 Received: from x64host.home (unknown [47.187.223.33]) by linux.microsoft.com (Postfix) with ESMTPSA id 9617020B8001; Tue, 20 Apr 2021 11:45:03 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 9617020B8001 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1618944304; bh=0DaS2ZAaR3tWrI17aOD2elYuIto8dgrtjnc7v28PGzg=; h=From:To:Subject:Date:In-Reply-To:References:From; b=f8rQP2uXAZ3fqh9zU4UD8IGGpIaTLG0Yy2/SA8BY4qeqhSalsev4rMQZXTEW08Dzf qPROV9sDwhw9YggGmCQ7S5i0q65iu0v+z37KzljImaHuc6XshuvYWimq4vPkKHwv/X oOJRKyXnra3mok8S8i9GM4Tb/cqMjPl30OSl77n4= From: madvenka@linux.microsoft.com To: broonie@kernel.org, jpoimboe@redhat.com, mark.rutland@arm.com, jthierry@redhat.com, catalin.marinas@arm.com, will@kernel.org, jmorris@namei.org, pasha.tatashin@soleen.com, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org, madvenka@linux.microsoft.com Subject: [PATCH v3 0/1] arm64: Implement stack trace termination record Date: Tue, 20 Apr 2021 13:44:46 -0500 Message-Id: <20210420184447.16306-1-madvenka@linux.microsoft.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <80cac661608c8d3623328b37b9b1696f63f45968> References: <80cac661608c8d3623328b37b9b1696f63f45968> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210420_114508_461264_BAC12BDE X-CRM114-Status: GOOD ( 13.50 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: "Madhavan T. Venkataraman" Reliable stacktracing requires that we identify when a stacktrace is terminated early. We can do this by ensuring all tasks have a final frame record at a known location on their task stack, and checking that this is the final frame record in the chain. All tasks have a pt_regs structure right after the task stack in the stack page. The pt_regs structure contains a stackframe field. Make this stackframe field the final frame in the task stack so all stack traces end at a fixed stack offset. For kernel tasks, this is simple to understand. For user tasks, there is some extra detail. User tasks get created via fork() et al. Once they return from fork, they enter the kernel only on an EL0 exception. In arm64, system calls are also EL0 exceptions. The EL0 exception handler uses the task pt_regs mentioned above to save register state and call different exception functions. All stack traces from EL0 exception code must end at the pt_regs. So, make pt_regs->stackframe the final frame in the EL0 exception stack. To summarize, task_pt_regs(task)->stackframe will always be the final frame in a stack trace. Sample stack traces =================== Showing just the last couple of frames in each stack trace to show how the stack trace ends. Primary CPU idle task ===================== ... [ 0.077109] rest_init+0x108/0x144 [ 0.077188] arch_call_rest_init+0x18/0x24 [ 0.077220] start_kernel+0x3ac/0x3e4 [ 0.077293] __primary_switched+0xac/0xb0 Secondary CPU idle task ======================= ... [ 0.077264] secondary_start_kernel+0x228/0x388 [ 0.077326] __secondary_switched+0x80/0x84 Sample kernel thread ==================== ... [ 24.543250] kernel_init+0xa4/0x164 [ 24.561850] ret_from_fork+0x10/0x18 Write system call (EL0 exception) ================= (using a test driver called callfd) [ 1160.628723] callfd_stack+0x3c/0x70 [ 1160.628768] callfd_op+0x35c/0x3a8 [ 1160.628791] callfd_write+0x5c/0xc8 [ 1160.628813] vfs_write+0x104/0x3b8 [ 1160.628837] ksys_write+0xd0/0x188 [ 1160.628859] __arm64_sys_write+0x4c/0x60 [ 1160.628883] el0_svc_common.constprop.0+0xa8/0x240 [ 1160.628904] do_el0_svc+0x40/0xa8 [ 1160.628921] el0_svc+0x2c/0x78 [ 1160.628942] el0_sync_handler+0xb0/0xb8 [ 1160.628962] el0_sync+0x17c/0x180 NULL pointer dereference exception (EL1 exception) ================================== [ 1160.637984] callfd_stack+0x3c/0x70 [ 1160.638015] die_kernel_fault+0x80/0x108 [ 1160.638042] do_page_fault+0x520/0x600 [ 1160.638075] do_translation_fault+0xa8/0xdc [ 1160.638102] do_mem_abort+0x68/0x100 [ 1160.638120] el1_abort+0x40/0x60 [ 1160.638138] el1_sync_handler+0xac/0xc8 [ 1160.638157] el1_sync+0x74/0x100 [ 1160.638174] 0x0 <=== NULL pointer dereference [ 1160.638189] callfd_write+0x5c/0xc8 [ 1160.638211] vfs_write+0x104/0x3b8 [ 1160.638234] ksys_write+0xd0/0x188 [ 1160.638278] __arm64_sys_write+0x4c/0x60 [ 1160.638325] el0_svc_common.constprop.0+0xa8/0x240 [ 1160.638358] do_el0_svc+0x40/0xa8 [ 1160.638379] el0_svc+0x2c/0x78 [ 1160.638409] el0_sync_handler+0xb0/0xb8 [ 1160.638452] el0_sync+0x17c/0x180 Timer interrupt (EL1 exception) =============== Secondary CPU idle task interrupted by the timer interrupt: [ 1160.702949] callfd_callback: [ 1160.703006] callfd_stack+0x3c/0x70 [ 1160.703060] callfd_callback+0x30/0x40 [ 1160.703087] call_timer_fn+0x48/0x220 [ 1160.703113] run_timer_softirq+0x7cc/0xc70 [ 1160.703144] __do_softirq+0x1ec/0x608 [ 1160.703166] irq_exit+0x138/0x180 [ 1160.703193] __handle_domain_irq+0x8c/0xf0 [ 1160.703218] gic_handle_irq+0xec/0x410 [ 1160.703253] el1_irq+0xc0/0x180 [ 1160.703278] arch_local_irq_enable+0xc/0x28 [ 1160.703329] default_idle_call+0x54/0x1d8 [ 1160.703355] do_idle+0x2d8/0x350 [ 1160.703388] cpu_startup_entry+0x2c/0x98 [ 1160.703412] secondary_start_kernel+0x238/0x388 [ 1160.703446] __secondary_switched+0x80/0x84 --- Changelog: v3: - Added Reviewed-by: Mark Brown . - Fixed an extra space after a cast reported by checkpatch --strict. - Synced with mainline tip. v2: - Changed some wordings as suggested by Mark Rutland. - Removed the synthetic return PC for idle tasks. Changed the branches to start_kernel() and secondary_start_kernel() to calls so that they will have a proper return PC. v1: - Set up task_pt_regs(current)->stackframe as the final frame when a new task is initialized in copy_thread(). - Create pt_regs for the idle tasks and set up pt_regs->stackframe as the final frame for the idle tasks. - Set up task_pt_regs(current)->stackframe as the final frame in the EL0 exception handler so the EL0 exception stack trace ends there. - Terminate the stack trace successfully in unwind_frame() when the FP reaches task_pt_regs(current)->stackframe. - The stack traces (above) in the kernel will terminate at the correct place. Debuggers may show an extra record 0x0 at the end for pt_regs->stackframe. That said, I did not see that extra frame when I did stack traces using gdb. Testing: - Functional validation on a ThunderX system. Previous versions and discussion ================================ v2: https://lore.kernel.org/linux-arm-kernel/20210402032404.47239-1-madvenka@linux.microsoft.com/ v1: https://lore.kernel.org/linux-arm-kernel/20210324184607.120948-1-madvenka@linux.microsoft.com/ Madhavan T. Venkataraman (1): arm64: Implement stack trace termination record arch/arm64/kernel/entry.S | 8 +++++--- arch/arm64/kernel/head.S | 29 +++++++++++++++++++++++------ arch/arm64/kernel/process.c | 5 +++++ arch/arm64/kernel/stacktrace.c | 10 +++++----- 4 files changed, 38 insertions(+), 14 deletions(-) base-commit: bf05bf16c76bb44ab5156223e1e58e26dfe30a88