From patchwork Mon Jun 26 07:47:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: D Scott Phillips X-Patchwork-Id: 13292443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 22BC8EB64D7 for ; Mon, 26 Jun 2023 07:51:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=ujTW08wsMF7pYEJ6OpYAAUw5gQQ6rgvyJdvlQH6yw64=; b=1jF40TD3Pmgjmi BNb6ILXT6av2r9ZsFYAq67ffdpRI29fzR61QKgWxvkLUFanGqgRv0YJs37CfqyK0tqCs18c1Am5vC 3vwBuCLgZrkW6p4CAUhoFtnpjfMA5CesgwWl/VbWx8bexmOd9CHGpgeM6Q+FJfXHaUlhy2X4hwknk Pjc8h/VHkD6cxexYLT5CISdTxl4GcKzGE8+YhAff5m+gxWkf5fz0SjeB2iMe+KInKzV8LkCkBjdmc ogaYLfiU4g2SjnCy3tUt3q8rAU+Et2PUDlwhhx+tgZa8k6K2vs/rX+f6hWxq/enm+pMGH+8SQ75Zk k+XNTQjnP9TSdCJaD/0A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qDgzf-009aWn-2P; Mon, 26 Jun 2023 07:50:39 +0000 Received: from mail-mw2nam12on2072a.outbound.protection.outlook.com ([2a01:111:f400:fe5a::72a] helo=NAM12-MW2-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qDgzd-009aVs-0Q for linux-arm-kernel@lists.infradead.org; Mon, 26 Jun 2023 07:50:38 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kTV92AiRLtPQ1Ge423QCFofgE94yJC03esIZXg+awvf6mw80BdwumkQ6XFdhB6yB5IsR+KdYMFnqg+ahOAwzO+6EWaQvbpReXyaxiZjjcGHBH62vpj3MIKRaTTeQqT3+Qhxr5aESlEctMhAT1TiC0na4qPF/+oPFsiG41WG0eRhI8jBn68SAeqPG4tIApPNyYFHKv6/kuoZ73BZhI2iFVuG25cMUxQFBaJLVi6Lg9qmLshUpkGuTwxAYXSu38UbB86DJkFJd9PNrNGN0hQ2x6kMCa3QE/8Az+JHTnnlykT6t16wyPixTh1KbJAPxofvMxwvFC1e4/6aEZBAKnS2HUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fz+RWvXcbBb7bdBUnonUbfoHVYXcJo937rmQ02J74ug=; b=W2CSCF2sPdY/fVviA8Pmg2Pc4xCew7bg9LJLfoWkBRtf+a/BL60+oE+Q5nZ0863NwOWi14Gxry5luYfN3kpRA1tEYoJ2N454FouT9wPhCHDCgdJ4avMMT/+7siVfGZ+s1/V+U+TKPGltf9r/BhdcQP1OrT3qxhD+oCmqI/9FRpFSbkE4bQpyTXkNzNmh9/sYSItepSyu0CtOrwbLhL9Z8y4EbbXPhOo0W/Rozu+x1kNfVSDK7KyDpLNI/dbjl8pS9R84hHlmCl3cGklkmGRRy7szfwP9LKCRZU0kxbW+RZdVEkE476KDz8Lc8w+xH+N9YFLUtq6N0gmLDSt3Ain3PA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=os.amperecomputing.com; dmarc=pass action=none header.from=os.amperecomputing.com; dkim=pass header.d=os.amperecomputing.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=os.amperecomputing.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fz+RWvXcbBb7bdBUnonUbfoHVYXcJo937rmQ02J74ug=; b=kEofo6+h5H/7UV5h1eBd5C6gBOJpi+3J1vXnM8LHisf1Mmbj2uaqTStGeb9puijMc/903GCYyCO7/BA6MqFrYnFblrhotSuqvK7BIw0XrhB3x2oFkwgGSCxXLAh4JXf//i+6XqbEtu9N0bWs9TZjeLMBt0WGuh9WzGmcf7m6DzU= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=os.amperecomputing.com; Received: from MWHPR0101MB2893.prod.exchangelabs.com (2603:10b6:301:33::25) by SJ2PR01MB8322.prod.exchangelabs.com (2603:10b6:a03:536::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6477.29; Mon, 26 Jun 2023 07:50:25 +0000 Received: from MWHPR0101MB2893.prod.exchangelabs.com ([fe80::c3b4:c7ab:46e:476]) by MWHPR0101MB2893.prod.exchangelabs.com ([fe80::c3b4:c7ab:46e:476%2]) with mapi id 15.20.6521.026; Mon, 26 Jun 2023 07:50:24 +0000 From: D Scott Phillips To: linux-arm-kernel@lists.infradead.org, James Morse Cc: Catalin Marinas , Will Deacon , linux-kernel@vger.kernel.org, Darren Hart Subject: [PATCH v5] arm64: sdei: abort running SDEI handlers during crash Date: Mon, 26 Jun 2023 00:47:48 -0700 Message-ID: <20230626074748.2785-1-scott@os.amperecomputing.com> X-Mailer: git-send-email 2.41.0 X-ClientProxiedBy: CH2PR05CA0069.namprd05.prod.outlook.com (2603:10b6:610:38::46) To MWHPR0101MB2893.prod.exchangelabs.com (2603:10b6:301:33::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWHPR0101MB2893:EE_|SJ2PR01MB8322:EE_ X-MS-Office365-Filtering-Correlation-Id: ca7e3984-afe2-49ef-7800-08db7619ffed X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kVFnmeXY6gLYs56mI8lvb6pRP8tE/7GMzakD4najoTBSJaK0LlewiJ9wTc82s4ew6obKOJ6BIwpGfPQcREZ9qhhbQXs2L27DgsL/leabaFob1eArkcBYRR98d6Dt5a+/33kVYEq98LoOx71l5MdXJlGyFuHq9bICQcSqclbxXrU2xD4d1DtIYeJ4D5qTmZ89JFzO7KE7e7tmPcfwXThYfQZBxzLRgaYKRfSaHtekBvYw02mngZLLOQpQoAOEcbX45wOHuP2fOuFYAoTHIbUIQVPe3gb1T1cuKUQLnpoLr+UhcX48l+L26YJcAb7D0ymhE8WpLXMIISvzs2gdGnDKd2OyR22mdVzlnx6Kg452l6XBuwgiacboW3Mo+OJLsJCfiid/fs8pxrfEil8NBjtanQRvLWBR8P8moXU8D3Wp+EUjHQRBkeGQsNJS2usiUm93SGZBjbkpZSnBAD4wxQpFgsS+KCOT74OeH8hgb/IQzg9CYLVtLekNLJ8vISX2DZmgo4UA9H0lIYR8uLOjKIjaS+ZokJKWwvKKpGyUkogFCJkXUzzdyjR8mqnYR2Y1grd1+RYVCodKA4DtLTCLn3nTVGx2/ZOHSyyyPCf9upKvaXI8WhHcMSgF7+ztublFHuy/ZoCmrlMboAtawz9W9BRSTw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MWHPR0101MB2893.prod.exchangelabs.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(396003)(39850400004)(366004)(136003)(346002)(376002)(451199021)(186003)(6666004)(478600001)(26005)(1076003)(6512007)(6506007)(2616005)(83380400001)(107886003)(966005)(52116002)(6486002)(38350700002)(38100700002)(6916009)(66556008)(66476007)(66946007)(4326008)(41300700001)(86362001)(2906002)(316002)(8936002)(8676002)(5660300002)(54906003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 14Y+iDLbtsbCutpvWY4RFRU1XIL++Tk7gib2VeAkF9ggG5brPrnwZrpeH7iJA3FZCJCF4h2i8oltnx49nv4yriUJfbcezrIfJ+7LSYdf4GsXJwoi4RzdG/0g3FMKAn+Ko4/z4ueZep/hOVs1lDugFN4tPOuSoMUjcoSSP0skmtt2U0l/HHKVnCjPTVudZTzxh8q24hy8oktGGiyG3V6JDLVxlEZsI3LVf6+nlGfWLOr7L2/CGy/IY1ga3VjMF6FDhfdmcxE7KsPcmlP5s0h/CPWb9tRvQydZtAAKLIChszzedi40s0gV6ft83Bhjte0AIN8h/qm1uTNjY8YetwN9Gs/Ld9HZIFN6ysrOUkNNhunHnFB8YxVUn1g9gpow9kyII8Vw6Ot4b7FbQVN5z4AvGZlAJ2n8vBrFAVhyuJely3fE4GQcapJHNOC4ONguhBtW/9i6Rgf/ZJwgiAZk/rdr+dGgO8QPiour8Imt/78xtyDN7ViiT6VbtVlm+GYWi6JC8rKowgdV8RODXwyceQzuTnR+mDcxsrXel/b1sFzp7emnC2iiQ0FwhysKfglgJWPMp5mdePFFpw6BlLCIQ3C4Ta0yCV9A3F7L13HkDFEXJal/rH5BCBOQL8jlguHPkLI28Ug+5oaBkQsDRbz/RInbm9YsyQ0/7mYdK1eqcDoGWev1GUjdeR6OidQHGlwssxwZCno3Pb4p/4AKdJn5+dIYgSf7anRK/UimnOiSB1bM/C4mtL50WQ+aHkFZAHkfjPTZRV5x6EMcuVmRZWFCkAXcMsTe1hbU2FnhVWF1Sg+rbc4hN406cL+jJrqgD1WjSn3Bcw9Wk7cz67RyNWYNjrCY8nojxmVWkwKwEW3W5fk4n2QVwq9RbFXva1pXiFnusykGlO9VBlDwBVIyWPAAQUTfqOl9T9DSgznzocpb5uGhw6Nbx5nNPiLefsbbrZijQe3kYY5DayfOCnyRPyoJ4aYXNCMkZv4shhcw3aq+x6A4fdUjuLaRvgZbJBa4r6WH2I3+WfHVgNs0g64DPreGgx26Voz771PdTvpLQNGXssJOLKn5IXgWG99dHUCrFwFWTIE07eWfjNjv2D7Ori0SOUuevB4/OGT6D5nEurCtq4kuQNP3BiTFrFUSe/MZ3bJN9fHjIyqWMbfHj5bvUXXqW6lys0tGIypP5lvJou8hO1G2KSBCO8GsgHnhoQvlrgyZZzsiayQ1vmerqyaQr/j3/8mz0h3xDXincBIKg0dhiObacM2LupD4h/Hz0z61/yW8EQDQFtZEB0utgz8ZUJOVK+DNePjQw8x96rtHAXubmHOrdz7cC0JJ+Cwv5rIq9zoB2pz5fyJjjXZKtlHWpO1yJwsphUlfNy18FWDxqX2H6EvIr5KXNBodew22ebojCZMt1WdIEQsNvM4M8/2DudIZBECDseVNJ5oVWq2xY6QiQtZNASWIDtkdEncm/Tdih/cXbFcBJc6qAK/xDYiX/p5cfd6jJ+lFElFP4AA7hFnACKo+hZRZ37ih/OEjDhUhjGR048Ma5etgl536K1VuXSfAG6ge5jQsLgQJYryrwH0cIwKx+q4GokFJKA7Cus2rDABd4oA7Js9UbOy7ZhI7uWopD+7aQw== X-OriginatorOrg: os.amperecomputing.com X-MS-Exchange-CrossTenant-Network-Message-Id: ca7e3984-afe2-49ef-7800-08db7619ffed X-MS-Exchange-CrossTenant-AuthSource: MWHPR0101MB2893.prod.exchangelabs.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jun 2023 07:50:24.4826 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1QQ7Dx6FYu0h1BVFcghKvxPPnTNQJKjuunZB9n/nmcPewRBpAT8g9KopLE0Azql+CZUYtG9gzMOyazEM83rbuGX91GNbU9WLS1gcPLTijuRXkMLKuvrCd9SvhpxI/iOc X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR01MB8322 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230626_005037_206726_FB9EB66C X-CRM114-Status: GOOD ( 22.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Interrupts are blocked in SDEI context, per the SDEI spec: "The client interrupts cannot preempt the event handler." If we crashed in the SDEI handler-running context (as with ACPI's AGDI) then we need to clean up the SDEI state before proceeding to the crash kernel so that the crash kernel can have working interrupts. Track the active SDEI handler per-cpu so that we can COMPLETE_AND_RESUME the handler, discarding the interrupted context. Fixes: f5df26961853 ("arm64: kernel: Add arch-specific SDEI entry code and CPU masking") Signed-off-by: D Scott Phillips Reviewed-by: James Morse Cc: stable@vger.kernel.org --- Changes since v4: - unconditionally include asm/sdei.h from linux/arm_sdei.h v4 Link: https://lore.kernel.org/linux-arm-kernel/20230625234033.672594-1-scott@os.amperecomputing.com/ Changes since v3: - Fixed messed up #ifdef logic in entry.S - Moved sdei_handler_abort() logic from smp.c to sdei.c v3 Link: https://lore.kernel.org/linux-arm-kernel/20230607195546.2896-1-scott@os.amperecomputing.com/ Changes since v2: - Dropped the patch fiddling with the sdei conduit. v2 Link: https://lore.kernel.org/linux-arm-kernel/20230329202519.6110-1-scott@os.amperecomputing.com/ Changes since v1: - Store the active SDEI event being handled per-cpu, use the per-cpu active handler information to know when to abort. - Add prints before attempting to abort sdei handlers. v1 Link: https://lore.kernel.org/linux-arm-kernel/20230204000851.3871-1-scott@os.amperecomputing.com/ arch/arm64/include/asm/sdei.h | 11 +++++++++++ arch/arm64/kernel/entry.S | 27 +++++++++++++++++++++++++-- arch/arm64/kernel/sdei.c | 22 ++++++++++++++++++++++ arch/arm64/kernel/smp.c | 8 ++++---- include/linux/arm_sdei.h | 2 -- 5 files changed, 62 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/sdei.h b/arch/arm64/include/asm/sdei.h index 4292d9bafb9d..98786108c493 100644 --- a/arch/arm64/include/asm/sdei.h +++ b/arch/arm64/include/asm/sdei.h @@ -17,6 +17,9 @@ #include +DECLARE_PER_CPU(struct sdei_registered_event *, sdei_active_normal_event); +DECLARE_PER_CPU(struct sdei_registered_event *, sdei_active_critical_event); + extern unsigned long sdei_exit_mode; /* Software Delegated Exception entry point from firmware*/ @@ -29,6 +32,14 @@ asmlinkage void __sdei_asm_entry_trampoline(unsigned long event_num, unsigned long pc, unsigned long pstate); +#ifdef CONFIG_ARM_SDE_INTERFACE +/* Abort a running handler. Context is discarded. */ +void sdei_handler_abort(void); +void __sdei_handler_abort(void); +#else +static inline void sdei_handler_abort(void) { } +#endif + /* * The above entry point does the minimum to call C code. This function does * anything else, before calling the driver. diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index ab2a6e33c052..1b4a65a33186 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -1003,9 +1003,13 @@ SYM_CODE_START(__sdei_asm_handler) mov x19, x1 -#if defined(CONFIG_VMAP_STACK) || defined(CONFIG_SHADOW_CALL_STACK) + /* Store the registered-event for crash_smp_send_stop() */ ldrb w4, [x19, #SDEI_EVENT_PRIORITY] -#endif + cbnz w4, 1f + adr_this_cpu dst=x5, sym=sdei_active_normal_event, tmp=x6 + b 2f +1: adr_this_cpu dst=x5, sym=sdei_active_critical_event, tmp=x6 +2: str x19, [x5] #ifdef CONFIG_VMAP_STACK /* @@ -1072,6 +1076,14 @@ SYM_CODE_START(__sdei_asm_handler) ldr_l x2, sdei_exit_mode + /* Clear the registered-event seen by crash_smp_send_stop() */ + ldrb w3, [x4, #SDEI_EVENT_PRIORITY] + cbnz w3, 1f + adr_this_cpu dst=x5, sym=sdei_active_normal_event, tmp=x6 + b 2f +1: adr_this_cpu dst=x5, sym=sdei_active_critical_event, tmp=x6 +2: str xzr, [x5] + alternative_if_not ARM64_UNMAP_KERNEL_AT_EL0 sdei_handler_exit exit_mode=x2 alternative_else_nop_endif @@ -1082,4 +1094,15 @@ alternative_else_nop_endif #endif SYM_CODE_END(__sdei_asm_handler) NOKPROBE(__sdei_asm_handler) + +SYM_CODE_START(__sdei_handler_abort) + mov_q x0, SDEI_1_0_FN_SDEI_EVENT_COMPLETE_AND_RESUME + adr x1, 1f + ldr_l x2, sdei_exit_mode + sdei_handler_exit exit_mode=x2 + // exit the handler and jump to the next instruction. + // Exit will stomp x0-x17, PSTATE, ELR_ELx, and SPSR_ELx. +1: ret +SYM_CODE_END(__sdei_handler_abort) +NOKPROBE(__sdei_handler_abort) #endif /* CONFIG_ARM_SDE_INTERFACE */ diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index 830be01af32d..fae7269090f8 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -47,6 +47,9 @@ DEFINE_PER_CPU(unsigned long *, sdei_shadow_call_stack_normal_ptr); DEFINE_PER_CPU(unsigned long *, sdei_shadow_call_stack_critical_ptr); #endif +DEFINE_PER_CPU(struct sdei_registered_event *, sdei_active_normal_event); +DEFINE_PER_CPU(struct sdei_registered_event *, sdei_active_critical_event); + static void _free_sdei_stack(unsigned long * __percpu *ptr, int cpu) { unsigned long *p; @@ -262,3 +265,22 @@ unsigned long __kprobes do_sdei_event(struct pt_regs *regs, return vbar + 0x480; } + +void sdei_handler_abort(void) +{ + /* + * If the crash happened in an SDEI event handler then we need to + * finish the handler with the firmware so that we can have working + * interrupts in the crash kernel. + */ + if (__this_cpu_read(sdei_active_critical_event)) { + pr_warn("still in SDEI critical event context, attempting to finish handler.\n"); + __sdei_handler_abort(); + __this_cpu_write(sdei_active_critical_event, NULL); + } + if (__this_cpu_read(sdei_active_normal_event)) { + pr_warn("still in SDEI normal event context, attempting to finish handler.\n"); + __sdei_handler_abort(); + __this_cpu_write(sdei_active_normal_event, NULL); + } +} diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index d00d4cbb31b1..c6b882e589e6 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -1048,10 +1048,8 @@ void crash_smp_send_stop(void) * If this cpu is the only one alive at this point in time, online or * not, there are no stop messages to be sent around, so just back out. */ - if (num_other_online_cpus() == 0) { - sdei_mask_local_cpu(); - return; - } + if (num_other_online_cpus() == 0) + goto skip_ipi; cpumask_copy(&mask, cpu_online_mask); cpumask_clear_cpu(smp_processor_id(), &mask); @@ -1070,7 +1068,9 @@ void crash_smp_send_stop(void) pr_warn("SMP: failed to stop secondary CPUs %*pbl\n", cpumask_pr_args(&mask)); +skip_ipi: sdei_mask_local_cpu(); + sdei_handler_abort(); } bool smp_crash_stop_failed(void) diff --git a/include/linux/arm_sdei.h b/include/linux/arm_sdei.h index 14dc461b0e82..d534a8bb9359 100644 --- a/include/linux/arm_sdei.h +++ b/include/linux/arm_sdei.h @@ -7,9 +7,7 @@ #include -#ifdef CONFIG_ARM_SDE_INTERFACE #include -#endif /* Arch code should override this to set the entry point from firmware... */ #ifndef sdei_arch_get_entry_point