[arm] Use common outgoing-CPU-notification code

Commit Message

Paul E. McKenney June 11, 2019, 7:24 p.m. UTC

This commit removes the open-coded CPU-offline notification with new
common code.  In particular, this change avoids calling scheduler code
using RCU from an offline CPU that RCU is ignoring.  This is a minimal
change.  A more intrusive change might invoke the cpu_check_up_prepare()
and cpu_set_state_online() functions at CPU-online time, which would
allow onlining throw an error if the CPU did not go offline properly.

Signed-off-by: Paul E. McKenney <[e-mail]>
Cc: [e-mail]
Cc: Russell King <[e-mail]>
Cc: Mark Rutland <[e-mail]>
Tested-by: Dietmar Eggemann <[e-mail]>

Comments

Mark Rutland June 17, 2019, 11:58 a.m. UTC | #1

On Tue, Jun 11, 2019 at 12:24:10PM -0700, Paul E. McKenney wrote:
> This commit removes the open-coded CPU-offline notification with new
> common code.  In particular, this change avoids calling scheduler code
> using RCU from an offline CPU that RCU is ignoring.  This is a minimal
> change.  A more intrusive change might invoke the cpu_check_up_prepare()
> and cpu_set_state_online() functions at CPU-online time, which would
> allow onlining throw an error if the CPU did not go offline properly.
> 
> Signed-off-by: Paul E. McKenney <[e-mail]>
> Cc: [e-mail]
> Cc: Russell King <[e-mail]>
> Cc: Mark Rutland <[e-mail]>
> Tested-by: Dietmar Eggemann <[e-mail]>

FWIW:

Acked-by: Mark Rutland <[e-mail]>

On the assumption that Russell is ok with this, I think this should be
dropped into the ARM patch system [1].

Paul, are you familiar with that, or would you prefer that someone else
submits the patch there? I can do so if you'd like.

Thanks,
Mark.

[1] https://www.armlinux.org.uk/developer/patches/info.php

> 
> diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
> index ebc53804d57b..8687d619260f 100644
> --- a/arch/arm/kernel/smp.c
> +++ b/arch/arm/kernel/smp.c
> @@ -267,15 +267,13 @@ int __cpu_disable(void)
>  	return 0;
>  }
>  
> -static DECLARE_COMPLETION(cpu_died);
> -
>  /*
>   * called on the thread which is asking for a CPU to be shutdown -
>   * waits until shutdown has completed, or it is timed out.
>   */
>  void __cpu_die(unsigned int cpu)
>  {
> -	if (!wait_for_completion_timeout(&cpu_died, msecs_to_jiffies(5000))) {
> +	if (!cpu_wait_death(cpu, 5)) {
>  		pr_err("CPU%u: cpu didn't die\n", cpu);
>  		return;
>  	}
> @@ -322,7 +320,7 @@ void arch_cpu_idle_dead(void)
>  	 * this returns, power and/or clocks can be removed at any point
>  	 * from this CPU and its cache by platform_cpu_kill().
>  	 */
> -	complete(&cpu_died);
> +	(void)cpu_report_death();
>  
>  	/*
>  	 * Ensure that the cache lines associated with that completion are
>

Paul E. McKenney June 17, 2019, 1:06 p.m. UTC | #2

On Mon, Jun 17, 2019 at 12:58:19PM +0100, Mark Rutland wrote:
> On Tue, Jun 11, 2019 at 12:24:10PM -0700, Paul E. McKenney wrote:
> > This commit removes the open-coded CPU-offline notification with new
> > common code.  In particular, this change avoids calling scheduler code
> > using RCU from an offline CPU that RCU is ignoring.  This is a minimal
> > change.  A more intrusive change might invoke the cpu_check_up_prepare()
> > and cpu_set_state_online() functions at CPU-online time, which would
> > allow onlining throw an error if the CPU did not go offline properly.
> > 
> > Signed-off-by: Paul E. McKenney <[e-mail]>
> > Cc: [e-mail]
> > Cc: Russell King <[e-mail]>
> > Cc: Mark Rutland <[e-mail]>
> > Tested-by: Dietmar Eggemann <[e-mail]>
> 
> FWIW:
> 
> Acked-by: Mark Rutland <[e-mail]>
> 
> On the assumption that Russell is ok with this, I think this should be
> dropped into the ARM patch system [1].
> 
> Paul, are you familiar with that, or would you prefer that someone else
> submits the patch there? I can do so if you'd like.

I never have used this system, so please do drop it in there!  Let me
know when you have done so, and I will then drop it from -rcu.

							Thanx, Paul

> Thanks,
> Mark.
> 
> [1] https://www.armlinux.org.uk/developer/patches/info.php
> 
> > 
> > diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
> > index ebc53804d57b..8687d619260f 100644
> > --- a/arch/arm/kernel/smp.c
> > +++ b/arch/arm/kernel/smp.c
> > @@ -267,15 +267,13 @@ int __cpu_disable(void)
> >  	return 0;
> >  }
> >  
> > -static DECLARE_COMPLETION(cpu_died);
> > -
> >  /*
> >   * called on the thread which is asking for a CPU to be shutdown -
> >   * waits until shutdown has completed, or it is timed out.
> >   */
> >  void __cpu_die(unsigned int cpu)
> >  {
> > -	if (!wait_for_completion_timeout(&cpu_died, msecs_to_jiffies(5000))) {
> > +	if (!cpu_wait_death(cpu, 5)) {
> >  		pr_err("CPU%u: cpu didn't die\n", cpu);
> >  		return;
> >  	}
> > @@ -322,7 +320,7 @@ void arch_cpu_idle_dead(void)
> >  	 * this returns, power and/or clocks can be removed at any point
> >  	 * from this CPU and its cache by platform_cpu_kill().
> >  	 */
> > -	complete(&cpu_died);
> > +	(void)cpu_report_death();
> >  
> >  	/*
> >  	 * Ensure that the cache lines associated with that completion are
> > 
>

Mark Rutland June 26, 2019, 4:42 p.m. UTC | #3

On Mon, Jun 17, 2019 at 06:06:57AM -0700, Paul E. McKenney wrote:
> On Mon, Jun 17, 2019 at 12:58:19PM +0100, Mark Rutland wrote:
> > On Tue, Jun 11, 2019 at 12:24:10PM -0700, Paul E. McKenney wrote:
> > > This commit removes the open-coded CPU-offline notification with new
> > > common code.  In particular, this change avoids calling scheduler code
> > > using RCU from an offline CPU that RCU is ignoring.  This is a minimal
> > > change.  A more intrusive change might invoke the cpu_check_up_prepare()
> > > and cpu_set_state_online() functions at CPU-online time, which would
> > > allow onlining throw an error if the CPU did not go offline properly.
> > > 
> > > Signed-off-by: Paul E. McKenney <[e-mail]>
> > > Cc: [e-mail]
> > > Cc: Russell King <[e-mail]>
> > > Cc: Mark Rutland <[e-mail]>
> > > Tested-by: Dietmar Eggemann <[e-mail]>
> > 
> > FWIW:
> > 
> > Acked-by: Mark Rutland <[e-mail]>
> > 
> > On the assumption that Russell is ok with this, I think this should be
> > dropped into the ARM patch system [1].
> > 
> > Paul, are you familiar with that, or would you prefer that someone else
> > submits the patch there? I can do so if you'd like.
> 
> I never have used this system, so please do drop it in there!  Let me
> know when you have done so, and I will then drop it from -rcu.

After testing that multi_v7_defconfig built, I've just submitted this as
8872/1:

  https://www.armlinux.org.uk/developer/patches/viewpatch.php?id=8872/1

Thanks,
Mark.

Paul E. McKenney June 26, 2019, 5:49 p.m. UTC | #4

On Wed, Jun 26, 2019 at 05:42:00PM +0100, Mark Rutland wrote:
> On Mon, Jun 17, 2019 at 06:06:57AM -0700, Paul E. McKenney wrote:
> > On Mon, Jun 17, 2019 at 12:58:19PM +0100, Mark Rutland wrote:
> > > On Tue, Jun 11, 2019 at 12:24:10PM -0700, Paul E. McKenney wrote:
> > > > This commit removes the open-coded CPU-offline notification with new
> > > > common code.  In particular, this change avoids calling scheduler code
> > > > using RCU from an offline CPU that RCU is ignoring.  This is a minimal
> > > > change.  A more intrusive change might invoke the cpu_check_up_prepare()
> > > > and cpu_set_state_online() functions at CPU-online time, which would
> > > > allow onlining throw an error if the CPU did not go offline properly.
> > > > 
> > > > Signed-off-by: Paul E. McKenney <[e-mail]>
> > > > Cc: [e-mail]
> > > > Cc: Russell King <[e-mail]>
> > > > Cc: Mark Rutland <[e-mail]>
> > > > Tested-by: Dietmar Eggemann <[e-mail]>
> > > 
> > > FWIW:
> > > 
> > > Acked-by: Mark Rutland <[e-mail]>
> > > 
> > > On the assumption that Russell is ok with this, I think this should be
> > > dropped into the ARM patch system [1].
> > > 
> > > Paul, are you familiar with that, or would you prefer that someone else
> > > submits the patch there? I can do so if you'd like.
> > 
> > I never have used this system, so please do drop it in there!  Let me
> > know when you have done so, and I will then drop it from -rcu.
> 
> After testing that multi_v7_defconfig built, I've just submitted this as
> 8872/1:
> 
>   https://www.armlinux.org.uk/developer/patches/viewpatch.php?id=8872/1

Very good, thank you!  I will drop this from my -rcu tree.

							Thanx, Paul

Patch

diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
index ebc53804d57b..8687d619260f 100644
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -267,15 +267,13 @@  int __cpu_disable(void)
 	return 0;
 }
 
-static DECLARE_COMPLETION(cpu_died);
-
 /*
  * called on the thread which is asking for a CPU to be shutdown -
  * waits until shutdown has completed, or it is timed out.
  */
 void __cpu_die(unsigned int cpu)
 {
-	if (!wait_for_completion_timeout(&cpu_died, msecs_to_jiffies(5000))) {
+	if (!cpu_wait_death(cpu, 5)) {
 		pr_err("CPU%u: cpu didn't die\n", cpu);
 		return;
 	}
@@ -322,7 +320,7 @@  void arch_cpu_idle_dead(void)
 	 * this returns, power and/or clocks can be removed at any point
 	 * from this CPU and its cache by platform_cpu_kill().
 	 */
-	complete(&cpu_died);
+	(void)cpu_report_death();
 
 	/*
 	 * Ensure that the cache lines associated with that completion are