diff mbox series

[2/2] clocksource/drivers/timer-ti-dm: Handle dra7 timer wrap errata i940

Message ID 20210323074326.28302-3-tony@atomide.com (mailing list archive)
State New, archived
Headers show
Series Fixes for for dra7 timer wrap errata i940 | expand

Commit Message

Tony Lindgren March 23, 2021, 7:43 a.m. UTC
There is a timer wrap issue on dra7 for the ARM architected timer.
In a typical clock configuration the timer fails to wrap after 388 days.

To work around the issue, we need to use timer-ti-dm percpu timers instead.

Let's configure dmtimer3 and 4 as percpu timers by default, and warn about
the issue if the dtb is not configured properly.

Let's do this as a single patch so it can be backported to v5.8 and later
kernels easily. Note that this patch depends on earlier timer-ti-dm
systimer posted mode fixes, and a preparatory clockevent patch
"clocksource/drivers/timer-ti-dm: Prepare to handle dra7 timer wrap issue".

For more information, please see the errata for "AM572x Sitara Processors
Silicon Revisions 1.1, 2.0":

https://www.ti.com/lit/er/sprz429m/sprz429m.pdf

The concept is based on earlier reference patches done by Tero Kristo and
Keerthy.

Cc: Keerthy <j-keerthy@ti.com>
Cc: Tero Kristo <kristo@kernel.org>
Signed-off-by: Tony Lindgren <tony@atomide.com>
---
 arch/arm/boot/dts/dra7-l4.dtsi             |  4 +-
 arch/arm/boot/dts/dra7.dtsi                | 20 ++++++
 drivers/clocksource/timer-ti-dm-systimer.c | 76 ++++++++++++++++++++++
 include/linux/cpuhotplug.h                 |  1 +
 4 files changed, 99 insertions(+), 2 deletions(-)

Comments

Drew Fustini Jan. 29, 2022, 9:05 p.m. UTC | #1
On Tue, Mar 23, 2021 at 09:43:26AM +0200, Tony Lindgren wrote:
> There is a timer wrap issue on dra7 for the ARM architected timer.
> In a typical clock configuration the timer fails to wrap after 388 days.
> 
> To work around the issue, we need to use timer-ti-dm percpu timers instead.
> 
> Let's configure dmtimer3 and 4 as percpu timers by default, and warn about
> the issue if the dtb is not configured properly.

Hi Tony,

This causes a conflict for IPU2 which is using timer 3 and 4.

From arch/arm/boot/dts/dra7-ipu-dsp-common.dtsi:

  &ipu2 {
          mboxes = <&mailbox6 &mbox_ipu2_ipc3x>;
          ti,timers = <&timer3>;
          ti,watchdog-timers = <&timer4>, <&timer9>;
  };

I noticed an error ("could not get timer platform device") when booting
mainline on a BeagleBoard X15 (AM578):

  omap-rproc 55020000.ipu: assigned reserved memory node ipu2-memory@95800000
  remoteproc remoteproc1: 55020000.ipu is available
  remoteproc remoteproc1: powering up 55020000.ipu
  remoteproc remoteproc1: Booting fw image dra7-ipu2-fw.xem4, size 3747220
  omap-rproc 55020000.ipu: could not get timer platform device
  omap-rproc 55020000.ipu: omap_rproc_enable_timers failed: -19
  remoteproc remoteproc1: can't start rproc 55020000.ipu: -19

I switched this errata fix to use timer 15 and 16 instead which resolves
the error.  Do you think that is an acceptable solution?

If so, I will post the patch to the list.

Thanks,
Drew
Tony Lindgren Jan. 31, 2022, 10:16 a.m. UTC | #2
Hi,

* Drew Fustini <dfustini@baylibre.com> [220129 21:05]:
> On Tue, Mar 23, 2021 at 09:43:26AM +0200, Tony Lindgren wrote:
> > There is a timer wrap issue on dra7 for the ARM architected timer.
> > In a typical clock configuration the timer fails to wrap after 388 days.
> > 
> > To work around the issue, we need to use timer-ti-dm percpu timers instead.
> > 
> > Let's configure dmtimer3 and 4 as percpu timers by default, and warn about
> > the issue if the dtb is not configured properly.
> 
> Hi Tony,
> 
> This causes a conflict for IPU2 which is using timer 3 and 4.
> 
> From arch/arm/boot/dts/dra7-ipu-dsp-common.dtsi:
> 
>   &ipu2 {
>           mboxes = <&mailbox6 &mbox_ipu2_ipc3x>;
>           ti,timers = <&timer3>;
>           ti,watchdog-timers = <&timer4>, <&timer9>;
>   };

OK, sorry I missed that part.

> I noticed an error ("could not get timer platform device") when booting
> mainline on a BeagleBoard X15 (AM578):
> 
>   omap-rproc 55020000.ipu: assigned reserved memory node ipu2-memory@95800000
>   remoteproc remoteproc1: 55020000.ipu is available
>   remoteproc remoteproc1: powering up 55020000.ipu
>   remoteproc remoteproc1: Booting fw image dra7-ipu2-fw.xem4, size 3747220
>   omap-rproc 55020000.ipu: could not get timer platform device
>   omap-rproc 55020000.ipu: omap_rproc_enable_timers failed: -19
>   remoteproc remoteproc1: can't start rproc 55020000.ipu: -19
> 
> I switched this errata fix to use timer 15 and 16 instead which resolves
> the error.  Do you think that is an acceptable solution?

I think the only difference is that timers 15 and 16 are in l4_per3 instead
of l4_per1. I doubt that matters as they are pretty much always clocked in
this case. If you want to check you can run cyclictest :)

> If so, I will post the patch to the list.

OK thanks,

Tony
Drew Fustini Feb. 2, 2022, 8:54 p.m. UTC | #3
On Mon, Jan 31, 2022 at 12:16:06PM +0200, Tony Lindgren wrote:
> Hi,
> 
> * Drew Fustini <dfustini@baylibre.com> [220129 21:05]:
> > On Tue, Mar 23, 2021 at 09:43:26AM +0200, Tony Lindgren wrote:
> > > There is a timer wrap issue on dra7 for the ARM architected timer.
> > > In a typical clock configuration the timer fails to wrap after 388 days.
> > > 
> > > To work around the issue, we need to use timer-ti-dm percpu timers instead.
> > > 
> > > Let's configure dmtimer3 and 4 as percpu timers by default, and warn about
> > > the issue if the dtb is not configured properly.
> > 
> > Hi Tony,
> > 
> > This causes a conflict for IPU2 which is using timer 3 and 4.
> > 
> > From arch/arm/boot/dts/dra7-ipu-dsp-common.dtsi:
> > 
> >   &ipu2 {
> >           mboxes = <&mailbox6 &mbox_ipu2_ipc3x>;
> >           ti,timers = <&timer3>;
> >           ti,watchdog-timers = <&timer4>, <&timer9>;
> >   };
> 
> OK, sorry I missed that part.
> 
> > I noticed an error ("could not get timer platform device") when booting
> > mainline on a BeagleBoard X15 (AM578):
> > 
> >   omap-rproc 55020000.ipu: assigned reserved memory node ipu2-memory@95800000
> >   remoteproc remoteproc1: 55020000.ipu is available
> >   remoteproc remoteproc1: powering up 55020000.ipu
> >   remoteproc remoteproc1: Booting fw image dra7-ipu2-fw.xem4, size 3747220
> >   omap-rproc 55020000.ipu: could not get timer platform device
> >   omap-rproc 55020000.ipu: omap_rproc_enable_timers failed: -19
> >   remoteproc remoteproc1: can't start rproc 55020000.ipu: -19
> > 
> > I switched this errata fix to use timer 15 and 16 instead which resolves
> > the error.  Do you think that is an acceptable solution?
> 
> I think the only difference is that timers 15 and 16 are in l4_per3 instead
> of l4_per1. I doubt that matters as they are pretty much always clocked in
> this case. If you want to check you can run cyclictest :)

I ran this with existing errata fix with dmtimer 3 and 4:

root@am57xx-evm:~# cyclictest --mlockall --smp --priority=80 --interval=200 --distance=0
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.02 0.03 0.05 

T: 0 ( 1449) P:80 I:200 C: 800368 Min:      0 Act:   32 Avg:   22 Max:     128
T: 1 ( 1450) P:80 I:200 C: 800301 Min:      0 Act:   12 Avg:   23 Max:      70

And the results after my switch to dmtimer 15 and 16:

root@am57xx-evm:~# cyclictest --mlockall --smp --priority=80 --interval=200 --distance=0
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 0.36 0.19 0.07

T: 0 ( 1711) P:80 I:200 C: 759599 Min:      0 Act:    6 Avg:   22 Max:     108
T: 1 ( 1712) P:80 I:200 C: 759539 Min:      0 Act:   19 Avg:   23 Max:      79

This doesn't appear to show any latency regression.

Any other options for cyclictest that that you'd recommend trying?

Thank you,
Drew
diff mbox series

Patch

diff --git a/arch/arm/boot/dts/dra7-l4.dtsi b/arch/arm/boot/dts/dra7-l4.dtsi
--- a/arch/arm/boot/dts/dra7-l4.dtsi
+++ b/arch/arm/boot/dts/dra7-l4.dtsi
@@ -1168,7 +1168,7 @@  timer2: timer@0 {
 			};
 		};
 
-		target-module@34000 {			/* 0x48034000, ap 7 46.0 */
+		timer3_target: target-module@34000 {	/* 0x48034000, ap 7 46.0 */
 			compatible = "ti,sysc-omap4-timer", "ti,sysc";
 			reg = <0x34000 0x4>,
 			      <0x34010 0x4>;
@@ -1195,7 +1195,7 @@  timer3: timer@0 {
 			};
 		};
 
-		target-module@36000 {			/* 0x48036000, ap 9 4e.0 */
+		timer4_target: target-module@36000 {	/* 0x48036000, ap 9 4e.0 */
 			compatible = "ti,sysc-omap4-timer", "ti,sysc";
 			reg = <0x36000 0x4>,
 			      <0x36010 0x4>;
diff --git a/arch/arm/boot/dts/dra7.dtsi b/arch/arm/boot/dts/dra7.dtsi
--- a/arch/arm/boot/dts/dra7.dtsi
+++ b/arch/arm/boot/dts/dra7.dtsi
@@ -46,6 +46,7 @@  aliases {
 
 	timer {
 		compatible = "arm,armv7-timer";
+		status = "disabled";	/* See ARM architected timer wrap erratum i940 */
 		interrupts = <GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(2) | IRQ_TYPE_LEVEL_LOW)>,
 			     <GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(2) | IRQ_TYPE_LEVEL_LOW)>,
 			     <GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(2) | IRQ_TYPE_LEVEL_LOW)>,
@@ -1241,3 +1242,22 @@  timer@0 {
 		assigned-clock-parents = <&sys_32k_ck>;
 	};
 };
+
+/* Local timers, see ARM architected timer wrap erratum i940 */
+&timer3_target {
+	ti,no-reset-on-init;
+	ti,no-idle;
+	timer@0 {
+		assigned-clocks = <&l4per_clkctrl DRA7_L4PER_TIMER3_CLKCTRL 24>;
+		assigned-clock-parents = <&timer_sys_clk_div>;
+	};
+};
+
+&timer4_target {
+	ti,no-reset-on-init;
+	ti,no-idle;
+	timer@0 {
+		assigned-clocks = <&l4per_clkctrl DRA7_L4PER_TIMER4_CLKCTRL 24>;
+		assigned-clock-parents = <&timer_sys_clk_div>;
+	};
+};
diff --git a/drivers/clocksource/timer-ti-dm-systimer.c b/drivers/clocksource/timer-ti-dm-systimer.c
--- a/drivers/clocksource/timer-ti-dm-systimer.c
+++ b/drivers/clocksource/timer-ti-dm-systimer.c
@@ -2,6 +2,7 @@ 
 #include <linux/clk.h>
 #include <linux/clocksource.h>
 #include <linux/clockchips.h>
+#include <linux/cpuhotplug.h>
 #include <linux/interrupt.h>
 #include <linux/io.h>
 #include <linux/iopoll.h>
@@ -630,6 +631,78 @@  static int __init dmtimer_clockevent_init(struct device_node *np)
 	return error;
 }
 
+/* Dmtimer as percpu timer. See dra7 ARM architected timer wrap erratum i940 */
+static DEFINE_PER_CPU(struct dmtimer_clockevent, dmtimer_percpu_timer);
+
+static int __init dmtimer_percpu_timer_init(struct device_node *np, int cpu)
+{
+	struct dmtimer_clockevent *clkevt;
+	int error;
+
+	if (!cpu_possible(cpu))
+		return -EINVAL;
+
+	if (!of_property_read_bool(np->parent, "ti,no-reset-on-init") ||
+	    !of_property_read_bool(np->parent, "ti,no-idle"))
+		pr_warn("Incomplete dtb for percpu dmtimer %pOF\n", np->parent);
+
+	clkevt = per_cpu_ptr(&dmtimer_percpu_timer, cpu);
+
+	error = dmtimer_clkevt_init_common(clkevt, np, CLOCK_EVT_FEAT_ONESHOT,
+					   cpumask_of(cpu), "percpu-dmtimer",
+					   500);
+	if (error)
+		return error;
+
+	return 0;
+}
+
+/* See TRM for timer internal resynch latency */
+static int omap_dmtimer_starting_cpu(unsigned int cpu)
+{
+	struct dmtimer_clockevent *clkevt = per_cpu_ptr(&dmtimer_percpu_timer, cpu);
+	struct clock_event_device *dev = &clkevt->dev;
+	struct dmtimer_systimer *t = &clkevt->t;
+
+	clockevents_config_and_register(dev, t->rate, 3, ULONG_MAX);
+	irq_force_affinity(dev->irq, cpumask_of(cpu));
+
+	return 0;
+}
+
+static int __init dmtimer_percpu_timer_startup(void)
+{
+	struct dmtimer_clockevent *clkevt = per_cpu_ptr(&dmtimer_percpu_timer, 0);
+	struct dmtimer_systimer *t = &clkevt->t;
+
+	if (t->sysc) {
+		cpuhp_setup_state(CPUHP_AP_TI_GP_TIMER_STARTING,
+				  "clockevents/omap/gptimer:starting",
+				  omap_dmtimer_starting_cpu, NULL);
+	}
+
+	return 0;
+}
+subsys_initcall(dmtimer_percpu_timer_startup);
+
+static int __init dmtimer_percpu_quirk_init(struct device_node *np, u32 pa)
+{
+	struct device_node *arm_timer;
+
+	arm_timer = of_find_compatible_node(NULL, NULL, "arm,armv7-timer");
+	if (of_device_is_available(arm_timer)) {
+		pr_warn_once("ARM architected timer wrap issue i940 detected\n");
+		return 0;
+	}
+
+	if (pa == 0x48034000)		/* dra7 dmtimer3 */
+		return dmtimer_percpu_timer_init(np, 0);
+	else if (pa == 0x48036000)	/* dra7 dmtimer4 */
+		return dmtimer_percpu_timer_init(np, 1);
+
+	return 0;
+}
+
 /* Clocksource */
 static struct dmtimer_clocksource *
 to_dmtimer_clocksource(struct clocksource *cs)
@@ -763,6 +836,9 @@  static int __init dmtimer_systimer_init(struct device_node *np)
 	if (clockevent == pa)
 		return dmtimer_clockevent_init(np);
 
+	if (of_machine_is_compatible("ti,dra7"))
+		return dmtimer_percpu_quirk_init(np, pa);
+
 	return 0;
 }
 
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -135,6 +135,7 @@  enum cpuhp_state {
 	CPUHP_AP_RISCV_TIMER_STARTING,
 	CPUHP_AP_CLINT_TIMER_STARTING,
 	CPUHP_AP_CSKY_TIMER_STARTING,
+	CPUHP_AP_TI_GP_TIMER_STARTING,
 	CPUHP_AP_HYPERV_TIMER_STARTING,
 	CPUHP_AP_KVM_STARTING,
 	CPUHP_AP_KVM_ARM_VGIC_INIT_STARTING,