From patchwork Thu Aug 17 02:24:27 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hoeun Ryu X-Patchwork-Id: 9904831 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7B98A60244 for ; Thu, 17 Aug 2017 02:30:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 64B35205A4 for ; Thu, 17 Aug 2017 02:30:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 59012283FE; Thu, 17 Aug 2017 02:30:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.6 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, DKIM_VALID, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C156B205A4 for ; Thu, 17 Aug 2017 02:30:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=j95dIlSrN1IOIaUQDYB+HP5hV0EWCovvvZ3cPRi3GY0=; b=sch Ba2edqGw5EV3hxm84qGTpV4mmL31/S78+iR1t2abwSn50kdgu0VDDGxQeTeRpXHka2LG4Z6EI8Ft5 h6I8YvYlQFGib2vaPh6AiugmirHuTU5rOoWOB7MuX5ecrpOSb+FecYQQEwskWgbNfi01pktq5E8mS 66kblHeq3YgCmUt/hWp7SGUUL3yXfsUCrXfBz/Ry7hNl9L/JPl6ZfMidRM2g9kkzDtZ93UR6JiBGc 02uPuFD8J3AOaTfSiRN0ZrlJlM1RU4mEWuqIpQgxmSylcIWC8oXfrw9RnArHI6ELEpGQlYbmyxxsF y0zzh+rLqZ/um/m8swGdhmVi2WyTTVA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1diAZm-0000xo-UM; Thu, 17 Aug 2017 02:30:26 +0000 Received: from mail-pg0-x243.google.com ([2607:f8b0:400e:c05::243]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1diAZi-0007xX-1i for linux-arm-kernel@lists.infradead.org; Thu, 17 Aug 2017 02:30:24 +0000 Received: by mail-pg0-x243.google.com with SMTP id y129so7383797pgy.3 for ; Wed, 16 Aug 2017 19:30:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=uysMFpqH3v+3J8gU/8uBKwzdvj+E+ynOfUQoxdyVcIE=; b=XWTO+yO6DbBdRSWf2pArr5B7CV0/xAc9Dfv8stWGlpImSlP92k6X4J7WHikh8ffdBw dubGrEUtedLJUq6cIyBNt96Q0bmsqQ4uYh4O4pAG6jC36EyoP6bko2s1qzc6MJYRYufh Ol7iPyMlmOoEfchYrRQWDPUjSy4ZsPMOXrODVG7Z/CkiUnVTIL48c7T5VUxh5K/GUouU vL6iRrCPVnStjDnh2Oo/0NRq65LmH1qvhSK6AA8Kd5rFt8ky4oxMsqRLvYXfARAbSzsg Yntb7BX1Polr3DcGlhkKy7P/9LP4PDOCCiZdz3u+wuitpCYoAsE/Drks6aeHJ2xcxwv2 JaOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=uysMFpqH3v+3J8gU/8uBKwzdvj+E+ynOfUQoxdyVcIE=; b=p0pRXxWaOZaDPc3JDKGapu1YmDXyBcJxq2ZJke1WIf8Q59CHPAQ6BsDBzCo+xLDesE pZpSDz93vb7LIIj4MYjo1dKq/gN9qP/MGz/Q8z2JcXBMm4UbD5cspL+I1M4nUi+TZXzK RsulwTdndb3Ovlsls0YhnZFQPB1UR/9cQFV/dgw3LsW9mgy/U6H3VVfw5iDjSwwp1k4A stnrtrfx51Tue3Bj0wCpm1VeZM3YIFtjI74r/P1ySJP9zal0aVTAd6kKFNPFJIuyYNb2 AihaL50N6L1v7G756HRMXeMtqTLBoLSM3EM7eG48G6Aokf6TiA50y09eomgyaBKxFqj2 aPQA== X-Gm-Message-State: AHYfb5i6PDbv0AL3SC8aN7t7T+WgTOAxCpDHwRD1SNVAswz4A8SXXZxL rFL9vFPAJcqZwg== X-Received: by 10.99.142.74 with SMTP id k71mr210767pge.86.1502937001196; Wed, 16 Aug 2017 19:30:01 -0700 (PDT) Received: from hryu-OptiPlex-9020.corp.ad.wrs.com ([61.107.31.52]) by smtp.gmail.com with ESMTPSA id e198sm3962857pfh.36.2017.08.16.19.29.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 16 Aug 2017 19:30:00 -0700 (PDT) From: Hoeun Ryu To: Catalin Marinas , Will Deacon , James Morse , Mark Rutland , AKASHI Takahiro , Robin Murphy , Ard Biesheuvel , Ingo Molnar , "Peter Zijlstra (Intel)" , Mark Salter , Suzuki K Poulose , David Daney , Rob Herring , Dmitry Torokhov , Thomas Gleixner Subject: [PATCHv3] arm64:kexec: have own crash_smp_send_stop() for crash dump for nonpanic cores Date: Thu, 17 Aug 2017 11:24:27 +0900 Message-Id: <1502936722-549-1-git-send-email-hoeun.ryu@gmail.com> X-Mailer: git-send-email 2.7.4 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170816_193022_296286_598BF107 X-CRM114-Status: GOOD ( 14.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Hoeun Ryu , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Commit 0ee5941 : (x86/panic: replace smp_send_stop() with kdump friendly version in panic path) introduced crash_smp_send_stop() which is a weak function and can be overridden by architecture codes to fix the side effect caused by commit f06e515 : (kernel/panic.c: add "crash_kexec_post_ notifiers" option). ARM64 architecture uses the weak version function and the problem is that the weak function simply calls smp_send_stop() which makes other CPUs offline and takes away the chance to save crash information for nonpanic CPUs in machine_crash_shutdown() when crash_kexec_post_notifiers kernel option is enabled. Calling smp_send_crash_stop() in machine_crash_shutdown() is useless because all nonpanic CPUs are already offline by smp_send_stop() in this case and smp_send_crash_stop() only works against online CPUs. The result is that secondary CPUs registers are not saved by crash_save_cpu() and the vmcore file misreports these CPUs as being offline. crash_smp_send_stop() is implemented to fix this problem by replacing the existing smp_send_crash_stop() and adding a check for multiple calling to the function. The function (strong symbol version) saves crash information for nonpanic CPUs and machine_crash_shutdown() tries to save crash information for nonpanic CPUs only when crash_kexec_post_notifiers kernel option is disabled. * crash_kexec_post_notifiers : false panic() __crash_kexec() machine_crash_shutdown() crash_smp_send_stop() <= save crash dump for nonpanic cores * crash_kexec_post_notifiers : true panic() crash_smp_send_stop() <= save crash dump for nonpanic cores __crash_kexec() machine_crash_shutdown() crash_smp_send_stop() <= just return. Signed-off-by: Hoeun Ryu Reviewed-by: James Morse Tested-by: James Morse --- v3: - fix typos in the commit log. - modify commit log about the result of this problem. - add Tested-by/Reviewed-by: James Morse. v2: - replace the existing smp_send_crash_stop() with crash_smp_send_stop() and adding called-twice logic to it. - modify the commit message. arch/arm64/include/asm/smp.h | 2 +- arch/arm64/kernel/machine_kexec.c | 2 +- arch/arm64/kernel/smp.c | 12 +++++++++++- 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/smp.h b/arch/arm64/include/asm/smp.h index 55f08c5..f82b447 100644 --- a/arch/arm64/include/asm/smp.h +++ b/arch/arm64/include/asm/smp.h @@ -148,7 +148,7 @@ static inline void cpu_panic_kernel(void) */ bool cpus_are_stuck_in_kernel(void); -extern void smp_send_crash_stop(void); +extern void crash_smp_send_stop(void); extern bool smp_crash_stop_failed(void); #endif /* ifndef __ASSEMBLY__ */ diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 481f54a..11121f6 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -252,7 +252,7 @@ void machine_crash_shutdown(struct pt_regs *regs) local_irq_disable(); /* shutdown non-crashing cpus */ - smp_send_crash_stop(); + crash_smp_send_stop(); /* for crashing cpu */ crash_save_cpu(regs, smp_processor_id()); diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index dc66e6e..73d8f5e 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -977,11 +977,21 @@ void smp_send_stop(void) } #ifdef CONFIG_KEXEC_CORE -void smp_send_crash_stop(void) +void crash_smp_send_stop(void) { + static int cpus_stopped; cpumask_t mask; unsigned long timeout; + /* + * This function can be called twice in panic path, but obviously + * we execute this only once. + */ + if (cpus_stopped) + return; + + cpus_stopped = 1; + if (num_online_cpus() == 1) return;