| Message ID | 1364549049-29278-1-git-send-email-ning.n.jiang@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 03/29/13 02:24, ning.n.jiang@gmail.com wrote: > From: Ning Jiang <ning.n.jiang@gmail.com> > > Currently there are two problems when we try to stop local timer. > First, it calls set_mode function directly so mode state is not > updated for the clock event device. Second, it makes the device > unused instead of shutdown. What device is this a problem on? I believe this only matters to drivers which enable their timer in their set_next_event() callback? But even then, does anything actually happen because the interrupt should have been disabled in the local timer stop callback. > > A subtle error will happen because of it. When a cpu is plugged out > it will stop the local timer. It will call tick_nohz_idle_enter() > in idle thread afterwards. It will cancel the sched timer and try > to reprogram the next event. This is wrong since the local timer > is supposed to be stopped. > > The right way to stop the local timer is to shutdown it by calling > clockevents_set_mode(). Thus when we try to reprogram the clock > event device, it will return directly without doing anything since > the clock mode is CLOCK_EVT_MODE_SHUTDOWN. While this prevents the set_next_event() callback from being called on a dying CPU, wouldn't it be better to fix this problem in the core code once instead of fixing it many times in each local timer driver? It doesn't seem to make much sense to program an event on a CPU that is about to die, so why do we do that?
On 03/29/2013 10:24 AM, ning.n.jiang@gmail.com wrote: > From: Ning Jiang <ning.n.jiang@gmail.com> > > Currently there are two problems when we try to stop local timer. > First, it calls set_mode function directly so mode state is not > updated for the clock event device. Second, it makes the device > unused instead of shutdown. > > A subtle error will happen because of it. When a cpu is plugged out > it will stop the local timer. It will call tick_nohz_idle_enter() > in idle thread afterwards. It will cancel the sched timer and try > to reprogram the next event. This is wrong since the local timer > is supposed to be stopped. > > The right way to stop the local timer is to shutdown it by calling > clockevents_set_mode(). Thus when we try to reprogram the clock > event device, it will return directly without doing anything since > the clock mode is CLOCK_EVT_MODE_SHUTDOWN. > > Signed-off-by: Ning Jiang <ning.n.jiang@gmail.com> > --- > arch/arm/kernel/smp_twd.c | 2 +- > arch/arm/mach-exynos/mct.c | 2 +- > arch/arm/mach-msm/timer.c | 2 +- > drivers/clocksource/arm_arch_timer.c | 2 +- > drivers/clocksource/time-armada-370-xp.c | 2 +- > 5 files changed, 5 insertions(+), 5 deletions(-) > > diff --git a/arch/arm/kernel/smp_twd.c b/arch/arm/kernel/smp_twd.c > index 3f25650..c1d4ab4 100644 > --- a/arch/arm/kernel/smp_twd.c > +++ b/arch/arm/kernel/smp_twd.c > @@ -92,7 +92,7 @@ static int twd_timer_ack(void) > > static void twd_timer_stop(struct clock_event_device *clk) > { > - twd_set_mode(CLOCK_EVT_MODE_UNUSED, clk); > + clockevents_set_mode(clk, CLOCK_EVT_MODE_SHUTDOWN); > disable_percpu_irq(clk->irq); Wouldn't be clockevents_shutdown more adequate here ? The next event will be also set.
On 03/29/2013 10:24 AM, ning.n.jiang@gmail.com wrote: > From: Ning Jiang <ning.n.jiang@gmail.com> > > Currently there are two problems when we try to stop local timer. > First, it calls set_mode function directly so mode state is not > updated for the clock event device. Second, it makes the device > unused instead of shutdown. > > A subtle error will happen because of it. When a cpu is plugged out > it will stop the local timer. It will call tick_nohz_idle_enter() > in idle thread afterwards. It will cancel the sched timer and try > to reprogram the next event. This is wrong since the local timer > is supposed to be stopped. > > The right way to stop the local timer is to shutdown it by calling > clockevents_set_mode(). Thus when we try to reprogram the clock > event device, it will return directly without doing anything since > the clock mode is CLOCK_EVT_MODE_SHUTDOWN. > > Signed-off-by: Ning Jiang <ning.n.jiang@gmail.com> > --- Don't you see a kernel BUG with this patch ? In the clockevents code, there is: /** * clockevents_notify - notification about relevant events */ void clockevents_notify(unsigned long reason, void *arg) { ... case CLOCK_EVT_NOTIFY_CPU_DEAD: ... cpu = *((int *)arg); list_for_each_entry_safe(dev, tmp, &clockevent_devices, list) { if (cpumask_test_cpu(cpu, dev->cpumask) && cpumask_weight(dev->cpumask) == 1 && !tick_is_broadcast_device(dev)) { BUG_ON(dev->mode != CLOCK_EVT_MODE_UNUSED); ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ list_del(&dev->list); } } break; ... } This is called triggered from hrtimer_cpu_notify with the CPU_DEAD event.
2013/4/1 Daniel Lezcano <daniel.lezcano@linaro.org>: > On 03/29/2013 10:24 AM, ning.n.jiang@gmail.com wrote: >> From: Ning Jiang <ning.n.jiang@gmail.com> >> >> Currently there are two problems when we try to stop local timer. >> First, it calls set_mode function directly so mode state is not >> updated for the clock event device. Second, it makes the device >> unused instead of shutdown. >> >> A subtle error will happen because of it. When a cpu is plugged out >> it will stop the local timer. It will call tick_nohz_idle_enter() >> in idle thread afterwards. It will cancel the sched timer and try >> to reprogram the next event. This is wrong since the local timer >> is supposed to be stopped. >> >> The right way to stop the local timer is to shutdown it by calling >> clockevents_set_mode(). Thus when we try to reprogram the clock >> event device, it will return directly without doing anything since >> the clock mode is CLOCK_EVT_MODE_SHUTDOWN. >> >> Signed-off-by: Ning Jiang <ning.n.jiang@gmail.com> >> --- >> arch/arm/kernel/smp_twd.c | 2 +- >> arch/arm/mach-exynos/mct.c | 2 +- >> arch/arm/mach-msm/timer.c | 2 +- >> drivers/clocksource/arm_arch_timer.c | 2 +- >> drivers/clocksource/time-armada-370-xp.c | 2 +- >> 5 files changed, 5 insertions(+), 5 deletions(-) >> >> diff --git a/arch/arm/kernel/smp_twd.c b/arch/arm/kernel/smp_twd.c >> index 3f25650..c1d4ab4 100644 >> --- a/arch/arm/kernel/smp_twd.c >> +++ b/arch/arm/kernel/smp_twd.c >> @@ -92,7 +92,7 @@ static int twd_timer_ack(void) >> >> static void twd_timer_stop(struct clock_event_device *clk) >> { >> - twd_set_mode(CLOCK_EVT_MODE_UNUSED, clk); >> + clockevents_set_mode(clk, CLOCK_EVT_MODE_SHUTDOWN); >> disable_percpu_irq(clk->irq); > > Wouldn't be clockevents_shutdown more adequate here ? The next event > will be also set. You're right. clockevents_shutdown seems more appropriate here. I'll submit a revised patch for it. > -- > <http://www.linaro.org/> Linaro.org ? Open source software for ARM SoCs > > Follow Linaro: <http://www.facebook.com/pages/Linaro> Facebook | > <http://twitter.com/#!/linaroorg> Twitter | > <http://www.linaro.org/linaro-blog/> Blog > -- To unsubscribe from this list: send the line "unsubscribe linux-samsung-soc" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2013/4/1 Daniel Lezcano <daniel.lezcano@linaro.org>: > On 03/29/2013 10:24 AM, ning.n.jiang@gmail.com wrote: >> From: Ning Jiang <ning.n.jiang@gmail.com> >> >> Currently there are two problems when we try to stop local timer. >> First, it calls set_mode function directly so mode state is not >> updated for the clock event device. Second, it makes the device >> unused instead of shutdown. >> >> A subtle error will happen because of it. When a cpu is plugged out >> it will stop the local timer. It will call tick_nohz_idle_enter() >> in idle thread afterwards. It will cancel the sched timer and try >> to reprogram the next event. This is wrong since the local timer >> is supposed to be stopped. >> >> The right way to stop the local timer is to shutdown it by calling >> clockevents_set_mode(). Thus when we try to reprogram the clock >> event device, it will return directly without doing anything since >> the clock mode is CLOCK_EVT_MODE_SHUTDOWN. >> >> Signed-off-by: Ning Jiang <ning.n.jiang@gmail.com> >> --- > > > Don't you see a kernel BUG with this patch ? > > In the clockevents code, there is: > > /** > * clockevents_notify - notification about relevant events > */ > void clockevents_notify(unsigned long reason, void *arg) > { > ... > case CLOCK_EVT_NOTIFY_CPU_DEAD: > ... > cpu = *((int *)arg); > list_for_each_entry_safe(dev, tmp, &clockevent_devices, > list) { > if (cpumask_test_cpu(cpu, dev->cpumask) && > cpumask_weight(dev->cpumask) == 1 && > !tick_is_broadcast_device(dev)) { > BUG_ON(dev->mode != > CLOCK_EVT_MODE_UNUSED); > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > list_del(&dev->list); > } > } > break; > ... > } > > This is called triggered from hrtimer_cpu_notify with the CPU_DEAD event. > The clockevents_do_notify() in clockevents_notify() will call tick_notify() which will call tick_shutdown() in turn. tick_shutdown() will set clock event device mode to UNUSED. So no panic afterwards. void clockevents_notify(unsigned long reason, void *arg) { ... clockevents_do_notify(reason, arg); switch (reason) { case CLOCK_EVT_NOTIFY_CPU_DEAD: ... list_for_each_entry_safe(dev, tmp, &clockevent_devices, list) { if (cpumask_test_cpu(cpu, dev->cpumask) && cpumask_weight(dev->cpumask) == 1 && !tick_is_broadcast_device(dev)) { BUG_ON(dev->mode != CLOCK_EVT_MODE_UNUSED); list_del(&dev->list); } } .... } raw_spin_unlock_irqrestore(&clockevents_lock, flags); } -- To unsubscribe from this list: send the line "unsubscribe linux-samsung-soc" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
2013/4/1 Ning Jiang <ning.n.jiang@gmail.com>: > 2013/4/1 Daniel Lezcano <daniel.lezcano@linaro.org>: >> On 03/29/2013 10:24 AM, ning.n.jiang@gmail.com wrote: >>> From: Ning Jiang <ning.n.jiang@gmail.com> >>> >>> Currently there are two problems when we try to stop local timer. >>> First, it calls set_mode function directly so mode state is not >>> updated for the clock event device. Second, it makes the device >>> unused instead of shutdown. >>> >>> A subtle error will happen because of it. When a cpu is plugged out >>> it will stop the local timer. It will call tick_nohz_idle_enter() >>> in idle thread afterwards. It will cancel the sched timer and try >>> to reprogram the next event. This is wrong since the local timer >>> is supposed to be stopped. >>> >>> The right way to stop the local timer is to shutdown it by calling >>> clockevents_set_mode(). Thus when we try to reprogram the clock >>> event device, it will return directly without doing anything since >>> the clock mode is CLOCK_EVT_MODE_SHUTDOWN. >>> >>> Signed-off-by: Ning Jiang <ning.n.jiang@gmail.com> >>> --- >>> arch/arm/kernel/smp_twd.c | 2 +- >>> arch/arm/mach-exynos/mct.c | 2 +- >>> arch/arm/mach-msm/timer.c | 2 +- >>> drivers/clocksource/arm_arch_timer.c | 2 +- >>> drivers/clocksource/time-armada-370-xp.c | 2 +- >>> 5 files changed, 5 insertions(+), 5 deletions(-) >>> >>> diff --git a/arch/arm/kernel/smp_twd.c b/arch/arm/kernel/smp_twd.c >>> index 3f25650..c1d4ab4 100644 >>> --- a/arch/arm/kernel/smp_twd.c >>> +++ b/arch/arm/kernel/smp_twd.c >>> @@ -92,7 +92,7 @@ static int twd_timer_ack(void) >>> >>> static void twd_timer_stop(struct clock_event_device *clk) >>> { >>> - twd_set_mode(CLOCK_EVT_MODE_UNUSED, clk); >>> + clockevents_set_mode(clk, CLOCK_EVT_MODE_SHUTDOWN); >>> disable_percpu_irq(clk->irq); >> >> Wouldn't be clockevents_shutdown more adequate here ? The next event >> will be also set. > > You're right. clockevents_shutdown seems more appropriate here. I'll > submit a revised patch for it. > Here attached the revised patch. Any more comments?
diff --git a/arch/arm/kernel/smp_twd.c b/arch/arm/kernel/smp_twd.c index 3f25650..c1d4ab4 100644 --- a/arch/arm/kernel/smp_twd.c +++ b/arch/arm/kernel/smp_twd.c @@ -92,7 +92,7 @@ static int twd_timer_ack(void) static void twd_timer_stop(struct clock_event_device *clk) { - twd_set_mode(CLOCK_EVT_MODE_UNUSED, clk); + clockevents_set_mode(clk, CLOCK_EVT_MODE_SHUTDOWN); disable_percpu_irq(clk->irq); } diff --git a/arch/arm/mach-exynos/mct.c b/arch/arm/mach-exynos/mct.c index c9d6650..b2d0d46 100644 --- a/arch/arm/mach-exynos/mct.c +++ b/arch/arm/mach-exynos/mct.c @@ -429,7 +429,7 @@ static int __cpuinit exynos4_local_timer_setup(struct clock_event_device *evt) static void exynos4_local_timer_stop(struct clock_event_device *evt) { unsigned int cpu = smp_processor_id(); - evt->set_mode(CLOCK_EVT_MODE_UNUSED, evt); + clockevents_set_mode(evt, CLOCK_EVT_MODE_SHUTDOWN); if (mct_int_type == MCT_INT_SPI) if (cpu == 0) remove_irq(evt->irq, &mct_tick0_event_irq); diff --git a/arch/arm/mach-msm/timer.c b/arch/arm/mach-msm/timer.c index 2969027..9b0cc66 100644 --- a/arch/arm/mach-msm/timer.c +++ b/arch/arm/mach-msm/timer.c @@ -152,7 +152,7 @@ static int __cpuinit msm_local_timer_setup(struct clock_event_device *evt) static void msm_local_timer_stop(struct clock_event_device *evt) { - evt->set_mode(CLOCK_EVT_MODE_UNUSED, evt); + clockevents_set_mode(evt, CLOCK_EVT_MODE_SHUTDOWN); disable_percpu_irq(evt->irq); } diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c index d7ad425..beecb26 100644 --- a/drivers/clocksource/arm_arch_timer.c +++ b/drivers/clocksource/arm_arch_timer.c @@ -242,7 +242,7 @@ static void __cpuinit arch_timer_stop(struct clock_event_device *clk) disable_percpu_irq(arch_timer_ppi[PHYS_NONSECURE_PPI]); } - clk->set_mode(CLOCK_EVT_MODE_UNUSED, clk); + clockevents_set_mode(clk, CLOCK_EVT_MODE_SHUTDOWN); } static int __cpuinit arch_timer_cpu_notify(struct notifier_block *self, diff --git a/drivers/clocksource/time-armada-370-xp.c b/drivers/clocksource/time-armada-370-xp.c index 47a6730..bfb2971 100644 --- a/drivers/clocksource/time-armada-370-xp.c +++ b/drivers/clocksource/time-armada-370-xp.c @@ -201,7 +201,7 @@ static int __cpuinit armada_370_xp_timer_setup(struct clock_event_device *evt) static void armada_370_xp_timer_stop(struct clock_event_device *evt) { - evt->set_mode(CLOCK_EVT_MODE_UNUSED, evt); + clockevents_set_mode(evt, CLOCK_EVT_MODE_SHUTDOWN); disable_percpu_irq(evt->irq); }