From patchwork Sun Dec 10 21:54:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Russell King (Oracle)" X-Patchwork-Id: 10104027 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1F5A160329 for ; Sun, 10 Dec 2017 21:55:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D34F292A0 for ; Sun, 10 Dec 2017 21:55:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F1D14292A2; Sun, 10 Dec 2017 21:55:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 30178292A0 for ; Sun, 10 Dec 2017 21:55:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=TwyblZg+49iEPhfRN71lHwlq1TIQNO1+mPMYRxKpZRA=; b=V3LVQ16lbIynpS zJLCMN9GMH10VBq++PMTHzr5MbCkzKtabdpqUNrNPSeFhlummZFU66iQUCMYtaYZOmNWuXHNwQ4xE gCU9dDx5GNvU+migA/RHWzppzH6vvcqqNNKyUDQ4Ie9TrDw9fbqf+s/vbgIv27dzkU1z5dh+Z84HR c4YL4zYFJAcRWDUOZqMPbKBYPP3J/YND9nFnGZahur4Ff3pXqAxwtu6jJXWCuw80GKByi3Z3SA1q2 YGdVt0Vl83d1OBLW7CsjvtRPkaJ6m2nIhtplClIhxjCcUi+2bg+iO5rmmM0l1z1VAbowhIft8p407 3eRDRyod8qxUTWC3XSUg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1eO9Yz-0006vy-Qm; Sun, 10 Dec 2017 21:55:09 +0000 Received: from pandora.armlinux.org.uk ([2001:4d48:ad52:3201:214:fdff:fe10:1be6]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1eO9Yv-0005hK-Sk for linux-arm-kernel@lists.infradead.org; Sun, 10 Dec 2017 21:55:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=armlinux.org.uk; s=pandora-2014; h=Sender:In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=fEv4H6U+I6ngMLj+mbTSl0jfVuaOg26/F7h//gl9yCo=; b=WusnX2FYFRqBtzahO3aXUOhEp/7O2zb+wrQK0UJBrZtUSOvc8aiDpS7q+VqZUsc8d2JGCZ9YKa3LDSZ0T4H1q+CxrO76LQKKs6C1ig/slL4HEYr6LLcU1dKLQI828baxNyqdujsdtb8QpVFeG0f9n0ahq3n5VR7S4EZZssUZHvY=; Received: from n2100.armlinux.org.uk ([2001:4d48:ad52:3201:214:fdff:fe10:4f86]:45313) by pandora.armlinux.org.uk with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.82_1-5b7a7c0-XX) (envelope-from ) id 1eO9YV-0008Q3-Pv; Sun, 10 Dec 2017 21:54:39 +0000 Received: from linux by n2100.armlinux.org.uk with local (Exim 4.76) (envelope-from ) id 1eO9YQ-0001Q2-1w; Sun, 10 Dec 2017 21:54:34 +0000 Date: Sun, 10 Dec 2017 21:54:33 +0000 From: Russell King - ARM Linux To: "Paul E. McKenney" Subject: Re: WARNING: suspicious RCU usage Message-ID: <20171210215433.GQ10595@n2100.armlinux.org.uk> References: <20171210113930.zimywmxzsutwzzmz@linux-u7w5.ap.freescale.net> <20171210120012.GM10595@n2100.armlinux.org.uk> <20171210190727.GJ7829@linux.vnet.ibm.com> <20171210193438.GP10595@n2100.armlinux.org.uk> <20171210213930.GL7829@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20171210213930.GL7829@linux.vnet.ibm.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20171210_135506_460262_D180DE45 X-CRM114-Status: GOOD ( 29.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peng Fan , fabio.estevam@nxp.com, shawnguo@kernel.org, aisheng.dong@nxp.com, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP On Sun, Dec 10, 2017 at 01:39:30PM -0800, Paul E. McKenney wrote: > On Sun, Dec 10, 2017 at 07:34:39PM +0000, Russell King - ARM Linux wrote: > > On Sun, Dec 10, 2017 at 11:07:27AM -0800, Paul E. McKenney wrote: > > > On Sun, Dec 10, 2017 at 12:00:12PM +0000, Russell King - ARM Linux wrote: > > > > +Paul > > > > > > > > Annoyingly, it looks like calling "complete()" from a dying CPU is > > > > triggering the RCU usage warning. From what I remember, this is an > > > > old problem, and we still have no better solution for this other than > > > > to persist with the warning. > > > > > > I thought that this issue was resolved with tglx's use of IPIs from > > > the outgoing CPU. Or is this due to an additional complete() from the > > > ARM code? If so, could it also use tglx's IPI trick? > > > > I don't think it was tglx's IPI trick, I've had code sitting in my tree > > for a while for it, but it has its own set of problems which are not > > resolvable: > > > > 1. it needs more IPIs than we have available on all platforms > > OK, I will ask the stupid question... Is it possible to multiplex > the IPIs, for example, by using smp_call_function_single()? > > > 2. there's some optional locking in the GIC driver that cause problems > > for the cpu dying path. > > On this, I must plead ignorance. > > > The concensus last time around was that the IPI solution is a non- > > starter, so the seven year proven-reliable solution (disregarding the > > RCU warning) persists because I don't think anyone came up with a > > better solution. > > Seven years already? Sigh, I don't have the heart to check because > I wouldn't want to find out that it has actually been longer. ;-) *Sigh* thanks for what you just said, you do realise that you've just said that you don't believe what I write in emails? FFS. Is there any point in continuing to discuss this if that's the case? Really? commit 3c030beabf937b1d3b4ecaedfd1fb2f1e2aa0c70 Author: Russell King Date: Tue Nov 30 11:07:35 2010 +0000 ARM: CPU hotplug: move cpu_killed completion to core code We always need to wait for the dying CPU to reach a safe state before taking it down, irrespective of the requirements of the platform. Move the completion code into the ARM SMP hotplug code rather than having each platform re-implement this. Signed-off-by: Russell King So, 30th Nov 2010 to 10th Dec 2017 is seven years, one week and three days to be exact. diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c index a30c4094db3a..8c81ff9b3732 100644 --- a/arch/arm/kernel/smp.c +++ b/arch/arm/kernel/smp.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include @@ -238,12 +239,20 @@ int __cpu_disable(void) return 0; } +static DECLARE_COMPLETION(cpu_died); + /* * called on the thread which is asking for a CPU to be shutdown - * waits until shutdown has completed, or it is timed out. */ void __cpu_die(unsigned int cpu) { + if (!wait_for_completion_timeout(&cpu_died, msecs_to_jiffies(5000))) { + pr_err("CPU%u: cpu didn't die\n", cpu); + return; + } + printk(KERN_NOTICE "CPU%u: shutdown\n", cpu); + if (!platform_cpu_kill(cpu)) printk("CPU%u: unable to kill\n", cpu); } @@ -263,9 +272,12 @@ void __ref cpu_die(void) local_irq_disable(); idle_task_exit(); + /* Tell __cpu_die() that this CPU is now safe to dispose of */ + complete(&cpu_died); + /* * actual CPU shutdown procedure is at least platform (if not - * CPU) specific + * CPU) specific. */ platform_cpu_die(cpu);