diff mbox

ARM: mvebu: pass the coherency availability information at init time

Message ID 1433992764-12753-1-git-send-email-gerg@uclinux.org (mailing list archive)
State New, archived
Headers show

Commit Message

Greg Ungerer June 11, 2015, 3:19 a.m. UTC
From: Greg Ungerer <gerg@uclinux.org>

This patch is specifically meant to be applied to stable tree linux-3.10.y.

IO coherency should not be used on the Armada 370 SoC, due to all
the necessary pre-requisites for reliable operation not being met
(such as write allocate cache policy, shareable pages, SMP bit set).

Commit e55355453600a33bb5ca4f71f2d7214875f3b061 ("ARM: mvebu: disable
I/O coherency on non-SMP situations on Armada 370/375/38x/XP") was
intended to disable IO coherency for the Armada 370. However it only
disables the CPU side IO coherency. The mbus driver (drivers/bus/
mvebu-mbus.c) still passes the IO coherency attributes through the
dram chip selects and onto driver memory window attributes. It
does this based on looking directly into the device tree (looking
for "marvell,coherency-fabric").

To fix we pass the coherency availability information (whether enabled
or not) to the mbus driver at init time. This is done in the same way
that it was done for mainline kernels in commit
5686a1e5aa436c49187a60052d5885fb1f541ce6 ("bus: mvebu: pass the
coherency availability information at init time").

Having the IO coherency enabled on the Armada 370 SoC causes rare
unreliable system behavior. It is not easy to consistently reproduce
problems caused by this. Best method I have seen is heavy network
load resulting in kernel dumps due to corrupted memory.

Signed-off-by: Greg Ungerer <gerg@uclinux.org>
Acked-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 arch/arm/mach-dove/common.c         |  2 +-
 arch/arm/mach-kirkwood/common.c     |  2 +-
 arch/arm/mach-mv78xx0/common.c      |  4 ++--
 arch/arm/mach-mvebu/armada-370-xp.c |  3 ++-
 arch/arm/mach-mvebu/coherency.c     | 15 +++++++++++++++
 arch/arm/mach-mvebu/coherency.h     |  1 +
 arch/arm/mach-orion5x/common.c      |  2 +-
 drivers/bus/mvebu-mbus.c            |  5 ++---
 include/linux/mbus.h                |  2 +-
 9 files changed, 26 insertions(+), 10 deletions(-)

Comments

Greg Kroah-Hartman June 11, 2015, 3:45 a.m. UTC | #1
On Thu, Jun 11, 2015 at 01:19:24PM +1000, gerg@uclinux.org wrote:
> From: Greg Ungerer <gerg@uclinux.org>
> 
> This patch is specifically meant to be applied to stable tree linux-3.10.y.

Why?  What's wrong with taking the exact specific upstream patches
instead?

Your descriptions here:

> IO coherency should not be used on the Armada 370 SoC, due to all
> the necessary pre-requisites for reliable operation not being met
> (such as write allocate cache policy, shareable pages, SMP bit set).
> 
> Commit e55355453600a33bb5ca4f71f2d7214875f3b061 ("ARM: mvebu: disable
> I/O coherency on non-SMP situations on Armada 370/375/38x/XP") was
> intended to disable IO coherency for the Armada 370. However it only
> disables the CPU side IO coherency. The mbus driver (drivers/bus/
> mvebu-mbus.c) still passes the IO coherency attributes through the
> dram chip selects and onto driver memory window attributes. It
> does this based on looking directly into the device tree (looking
> for "marvell,coherency-fabric").
> 
> To fix we pass the coherency availability information (whether enabled
> or not) to the mbus driver at init time. This is done in the same way
> that it was done for mainline kernels in commit
> 5686a1e5aa436c49187a60052d5885fb1f541ce6 ("bus: mvebu: pass the
> coherency availability information at init time").
> 
> Having the IO coherency enabled on the Armada 370 SoC causes rare
> unreliable system behavior. It is not easy to consistently reproduce
> problems caused by this. Best method I have seen is heavy network
> load resulting in kernel dumps due to corrupted memory.

I don't understand the issue here, I really don't want to not take
patches that are not in Linus's tree, sorry.

greg k-h
Greg Ungerer June 11, 2015, 4:04 a.m. UTC | #2
On 11/06/15 13:45, Greg KH wrote:
> On Thu, Jun 11, 2015 at 01:19:24PM +1000, gerg@uclinux.org wrote:
>> From: Greg Ungerer <gerg@uclinux.org>
>>
>> This patch is specifically meant to be applied to stable tree linux-3.10.y.
> 
> Why?  What's wrong with taking the exact specific upstream patches
> instead?

The exact patch mentioned below ("5686a1e5aa4") will not apply.
Too much of the code around it has changed. This does the same
thing in the same away taking into account the changes around it.


> Your descriptions here:
> 
>> IO coherency should not be used on the Armada 370 SoC, due to all
>> the necessary pre-requisites for reliable operation not being met
>> (such as write allocate cache policy, shareable pages, SMP bit set).
>>
>> Commit e55355453600a33bb5ca4f71f2d7214875f3b061 ("ARM: mvebu: disable
>> I/O coherency on non-SMP situations on Armada 370/375/38x/XP") was
>> intended to disable IO coherency for the Armada 370. However it only
>> disables the CPU side IO coherency. The mbus driver (drivers/bus/
>> mvebu-mbus.c) still passes the IO coherency attributes through the
>> dram chip selects and onto driver memory window attributes. It
>> does this based on looking directly into the device tree (looking
>> for "marvell,coherency-fabric").
>>
>> To fix we pass the coherency availability information (whether enabled
>> or not) to the mbus driver at init time. This is done in the same way
>> that it was done for mainline kernels in commit
>> 5686a1e5aa436c49187a60052d5885fb1f541ce6 ("bus: mvebu: pass the
>> coherency availability information at init time").
>>
>> Having the IO coherency enabled on the Armada 370 SoC causes rare
>> unreliable system behavior. It is not easy to consistently reproduce
>> problems caused by this. Best method I have seen is heavy network
>> load resulting in kernel dumps due to corrupted memory.
> 
> I don't understand the issue here, I really don't want to not take
> patches that are not in Linus's tree, sorry.

Sure.

Regards
Greg
Thomas Petazzoni June 11, 2015, 7:25 a.m. UTC | #3
Greg, Greg,

On Thu, 11 Jun 2015 14:04:18 +1000, Greg Ungerer wrote:

> > Why?  What's wrong with taking the exact specific upstream patches
> > instead?
> 
> The exact patch mentioned below ("5686a1e5aa4") will not apply.
> Too much of the code around it has changed. This does the same
> thing in the same away taking into account the changes around it.

As the original author of 5686a1e5aa4 ("bus: mvebu: pass the coherency
availability information at init time"), I can confirm that it will
clearly not apply as is on 3.10. What Greg Ungerer is proposing here is
a backport of 5686a1e5aa4 to 3.10.

The mistake is that 5686a1e5aa4 should have been marked for stable
really.

Best regards,

Thomas
Greg Kroah-Hartman June 11, 2015, 2:51 p.m. UTC | #4
On Thu, Jun 11, 2015 at 09:25:49AM +0200, Thomas Petazzoni wrote:
> Greg, Greg,
> 
> On Thu, 11 Jun 2015 14:04:18 +1000, Greg Ungerer wrote:
> 
> > > Why?  What's wrong with taking the exact specific upstream patches
> > > instead?
> > 
> > The exact patch mentioned below ("5686a1e5aa4") will not apply.
> > Too much of the code around it has changed. This does the same
> > thing in the same away taking into account the changes around it.
> 
> As the original author of 5686a1e5aa4 ("bus: mvebu: pass the coherency
> availability information at init time"), I can confirm that it will
> clearly not apply as is on 3.10. What Greg Ungerer is proposing here is
> a backport of 5686a1e5aa4 to 3.10.

What about 3.14-stable?

And if this is just a simple backport, that should have been stated
here.

thanks,

greg k-h
Thomas Petazzoni June 11, 2015, 3:14 p.m. UTC | #5
Greg,

On Thu, 11 Jun 2015 07:51:29 -0700, Greg KH wrote:

> > As the original author of 5686a1e5aa4 ("bus: mvebu: pass the coherency
> > availability information at init time"), I can confirm that it will
> > clearly not apply as is on 3.10. What Greg Ungerer is proposing here is
> > a backport of 5686a1e5aa4 to 3.10.
> 
> What about 3.14-stable?

5686a1e5aa4 was merged in 3.16, and it should be backported all the way
up to when Armada 370 support was merged with I/O coherency support,
i.e in 3.8.

Best regards,

Thomas
Greg Ungerer June 12, 2015, 1:19 a.m. UTC | #6
On 12/06/15 00:51, Greg KH wrote:
> On Thu, Jun 11, 2015 at 09:25:49AM +0200, Thomas Petazzoni wrote:
>> Greg, Greg,
>>
>> On Thu, 11 Jun 2015 14:04:18 +1000, Greg Ungerer wrote:
>>
>>>> Why?  What's wrong with taking the exact specific upstream patches
>>>> instead?
>>>
>>> The exact patch mentioned below ("5686a1e5aa4") will not apply.
>>> Too much of the code around it has changed. This does the same
>>> thing in the same away taking into account the changes around it.
>>
>> As the original author of 5686a1e5aa4 ("bus: mvebu: pass the coherency
>> availability information at init time"), I can confirm that it will
>> clearly not apply as is on 3.10. What Greg Ungerer is proposing here is
>> a backport of 5686a1e5aa4 to 3.10.
> 
> What about 3.14-stable?

As Thomas pointed out, yes. Due to file movements and other changes
neither this patch (for 3.10.y) or the original commit 5686a1e5aa4
apply cleanly to 3.14.y.

How do you want to handle that for 3.14.y?


> And if this is just a simple backport, that should have been stated
> here.

The part that said "This is done in the same way that it was done
for mainline kernels in commit" was meant to convey that meaning.
Do you want an updated patch with those exact words, "simple
backport", in the commit message?

Regards
Greg
Greg Kroah-Hartman June 30, 2015, 12:31 a.m. UTC | #7
On Fri, Jun 12, 2015 at 11:19:43AM +1000, Greg Ungerer wrote:
> On 12/06/15 00:51, Greg KH wrote:
> > On Thu, Jun 11, 2015 at 09:25:49AM +0200, Thomas Petazzoni wrote:
> >> Greg, Greg,
> >>
> >> On Thu, 11 Jun 2015 14:04:18 +1000, Greg Ungerer wrote:
> >>
> >>>> Why?  What's wrong with taking the exact specific upstream patches
> >>>> instead?
> >>>
> >>> The exact patch mentioned below ("5686a1e5aa4") will not apply.
> >>> Too much of the code around it has changed. This does the same
> >>> thing in the same away taking into account the changes around it.
> >>
> >> As the original author of 5686a1e5aa4 ("bus: mvebu: pass the coherency
> >> availability information at init time"), I can confirm that it will
> >> clearly not apply as is on 3.10. What Greg Ungerer is proposing here is
> >> a backport of 5686a1e5aa4 to 3.10.
> > 
> > What about 3.14-stable?
> 
> As Thomas pointed out, yes. Due to file movements and other changes
> neither this patch (for 3.10.y) or the original commit 5686a1e5aa4
> apply cleanly to 3.14.y.
> 
> How do you want to handle that for 3.14.y?

I need a backport for 3.14.y as well.

And I need a signed-off-by: from the subsystem maintainers that this
backport is acceptable, as it's so different from what is in Linus's
tree, before I can take it.

thanks,

greg k-h
Greg Ungerer June 30, 2015, 1:09 p.m. UTC | #8
On 30/06/15 10:31, Greg KH wrote:
> On Fri, Jun 12, 2015 at 11:19:43AM +1000, Greg Ungerer wrote:
>> On 12/06/15 00:51, Greg KH wrote:
>>> On Thu, Jun 11, 2015 at 09:25:49AM +0200, Thomas Petazzoni wrote:
>>>> Greg, Greg,
>>>>
>>>> On Thu, 11 Jun 2015 14:04:18 +1000, Greg Ungerer wrote:
>>>>
>>>>>> Why?  What's wrong with taking the exact specific upstream patches
>>>>>> instead?
>>>>> The exact patch mentioned below ("5686a1e5aa4") will not apply.
>>>>> Too much of the code around it has changed. This does the same
>>>>> thing in the same away taking into account the changes around it.
>>>> As the original author of 5686a1e5aa4 ("bus: mvebu: pass the coherency
>>>> availability information at init time"), I can confirm that it will
>>>> clearly not apply as is on 3.10. What Greg Ungerer is proposing here is
>>>> a backport of 5686a1e5aa4 to 3.10.
>>> What about 3.14-stable?
>> As Thomas pointed out, yes. Due to file movements and other changes
>> neither this patch (for 3.10.y) or the original commit 5686a1e5aa4
>> apply cleanly to 3.14.y.
>>
>> How do you want to handle that for 3.14.y?
> I need a backport for 3.14.y as well.
>
> And I need a signed-off-by: from the subsystem maintainers that this
> backport is acceptable, as it's so different from what is in Linus's
> tree, before I can take it.

Ok, I will prepare a 3.14 port. I will send to all recipients of this mail,
that should catch all those who need to sign off on it.

I have generated a version 2 of the original 3.10 patch. No change to
the code diffs, but it changes the commit message to include all of the
original commit followed by a brief description of the back port. Perhaps
this is better?

Regards
Greg
Greg Kroah-Hartman June 30, 2015, 4:48 p.m. UTC | #9
On Tue, Jun 30, 2015 at 11:09:03PM +1000, Greg Ungerer wrote:
> 
> 
> On 30/06/15 10:31, Greg KH wrote:
> >On Fri, Jun 12, 2015 at 11:19:43AM +1000, Greg Ungerer wrote:
> >>On 12/06/15 00:51, Greg KH wrote:
> >>>On Thu, Jun 11, 2015 at 09:25:49AM +0200, Thomas Petazzoni wrote:
> >>>>Greg, Greg,
> >>>>
> >>>>On Thu, 11 Jun 2015 14:04:18 +1000, Greg Ungerer wrote:
> >>>>
> >>>>>>Why?  What's wrong with taking the exact specific upstream patches
> >>>>>>instead?
> >>>>>The exact patch mentioned below ("5686a1e5aa4") will not apply.
> >>>>>Too much of the code around it has changed. This does the same
> >>>>>thing in the same away taking into account the changes around it.
> >>>>As the original author of 5686a1e5aa4 ("bus: mvebu: pass the coherency
> >>>>availability information at init time"), I can confirm that it will
> >>>>clearly not apply as is on 3.10. What Greg Ungerer is proposing here is
> >>>>a backport of 5686a1e5aa4 to 3.10.
> >>>What about 3.14-stable?
> >>As Thomas pointed out, yes. Due to file movements and other changes
> >>neither this patch (for 3.10.y) or the original commit 5686a1e5aa4
> >>apply cleanly to 3.14.y.
> >>
> >>How do you want to handle that for 3.14.y?
> >I need a backport for 3.14.y as well.
> >
> >And I need a signed-off-by: from the subsystem maintainers that this
> >backport is acceptable, as it's so different from what is in Linus's
> >tree, before I can take it.
> 
> Ok, I will prepare a 3.14 port. I will send to all recipients of this mail,
> that should catch all those who need to sign off on it.
> 
> I have generated a version 2 of the original 3.10 patch. No change to
> the code diffs, but it changes the commit message to include all of the
> original commit followed by a brief description of the back port. Perhaps
> this is better?

Yes, much better, thanks, I'll go queue those up later today.

greg k-h
diff mbox

Patch

diff --git a/arch/arm/mach-dove/common.c b/arch/arm/mach-dove/common.c
index e2b5da0..8d4f5dc 100644
--- a/arch/arm/mach-dove/common.c
+++ b/arch/arm/mach-dove/common.c
@@ -226,7 +226,7 @@  void __init dove_init_early(void)
 	orion_time_set_base(TIMER_VIRT_BASE);
 	mvebu_mbus_init("marvell,dove-mbus",
 			BRIDGE_WINS_BASE, BRIDGE_WINS_SZ,
-			DOVE_MC_WINS_BASE, DOVE_MC_WINS_SZ);
+			DOVE_MC_WINS_BASE, DOVE_MC_WINS_SZ, 0);
 }
 
 static int __init dove_find_tclk(void)
diff --git a/arch/arm/mach-kirkwood/common.c b/arch/arm/mach-kirkwood/common.c
index f389228..4f6831e 100644
--- a/arch/arm/mach-kirkwood/common.c
+++ b/arch/arm/mach-kirkwood/common.c
@@ -530,7 +530,7 @@  void __init kirkwood_init_early(void)
 
 	mvebu_mbus_init("marvell,kirkwood-mbus",
 			BRIDGE_WINS_BASE, BRIDGE_WINS_SZ,
-			DDR_WINDOW_CPU_BASE, DDR_WINDOW_CPU_SZ);
+			DDR_WINDOW_CPU_BASE, DDR_WINDOW_CPU_SZ, 0);
 }
 
 int kirkwood_tclk;
diff --git a/arch/arm/mach-mv78xx0/common.c b/arch/arm/mach-mv78xx0/common.c
index 749a7f8..4722c98 100644
--- a/arch/arm/mach-mv78xx0/common.c
+++ b/arch/arm/mach-mv78xx0/common.c
@@ -337,11 +337,11 @@  void __init mv78xx0_init_early(void)
 	if (mv78xx0_core_index() == 0)
 		mvebu_mbus_init("marvell,mv78xx0-mbus",
 				BRIDGE_WINS_CPU0_BASE, BRIDGE_WINS_SZ,
-				DDR_WINDOW_CPU0_BASE, DDR_WINDOW_CPU_SZ);
+				DDR_WINDOW_CPU0_BASE, DDR_WINDOW_CPU_SZ, 0);
 	else
 		mvebu_mbus_init("marvell,mv78xx0-mbus",
 				BRIDGE_WINS_CPU1_BASE, BRIDGE_WINS_SZ,
-				DDR_WINDOW_CPU1_BASE, DDR_WINDOW_CPU_SZ);
+				DDR_WINDOW_CPU1_BASE, DDR_WINDOW_CPU_SZ, 0);
 }
 
 void __init_refok mv78xx0_timer_init(void)
diff --git a/arch/arm/mach-mvebu/armada-370-xp.c b/arch/arm/mach-mvebu/armada-370-xp.c
index 1c48890..4377c34 100644
--- a/arch/arm/mach-mvebu/armada-370-xp.c
+++ b/arch/arm/mach-mvebu/armada-370-xp.c
@@ -66,7 +66,8 @@  void __init armada_370_xp_init_early(void)
 			ARMADA_370_XP_MBUS_WINS_BASE,
 			ARMADA_370_XP_MBUS_WINS_SIZE,
 			ARMADA_370_XP_SDRAM_WINS_BASE,
-			ARMADA_370_XP_SDRAM_WINS_SIZE);
+			ARMADA_370_XP_SDRAM_WINS_SIZE,
+			coherency_available());
 
 #ifdef CONFIG_CACHE_L2X0
 	l2x0_of_init(0, ~0UL);
diff --git a/arch/arm/mach-mvebu/coherency.c b/arch/arm/mach-mvebu/coherency.c
index 3ee701f..ea26ebb 100644
--- a/arch/arm/mach-mvebu/coherency.c
+++ b/arch/arm/mach-mvebu/coherency.c
@@ -137,6 +137,20 @@  static struct notifier_block mvebu_hwcc_platform_nb = {
 	.notifier_call = mvebu_hwcc_platform_notifier,
 };
 
+/*
+ * Keep track of whether we have IO hardware coherency enabled or not.
+ * On Armada 370's we will not be using it for example. We need to make
+ * that available [through coherency_available()] so the mbus controller
+ * doesn't enable the IO coherency bit in the attribute bits of the
+ * chip selects.
+ */
+static int coherency_enabled;
+
+int coherency_available(void)
+{
+	return coherency_enabled;
+}
+
 int __init coherency_init(void)
 {
 	struct device_node *np;
@@ -170,6 +184,7 @@  int __init coherency_init(void)
 		coherency_base = of_iomap(np, 0);
 		coherency_cpu_base = of_iomap(np, 1);
 		set_cpu_coherent(cpu_logical_map(smp_processor_id()), 0);
+		coherency_enabled = 1;
 		bus_register_notifier(&platform_bus_type,
 					&mvebu_hwcc_platform_nb);
 	}
diff --git a/arch/arm/mach-mvebu/coherency.h b/arch/arm/mach-mvebu/coherency.h
index 2f42813..1501a4e 100644
--- a/arch/arm/mach-mvebu/coherency.h
+++ b/arch/arm/mach-mvebu/coherency.h
@@ -19,6 +19,7 @@  int coherency_get_cpu_count(void);
 #endif
 
 int set_cpu_coherent(int cpu_id, int smp_group_id);
+int coherency_available(void);
 int coherency_init(void);
 
 #endif	/* __MACH_370_XP_COHERENCY_H */
diff --git a/arch/arm/mach-orion5x/common.c b/arch/arm/mach-orion5x/common.c
index f8a6db9..04877392 100644
--- a/arch/arm/mach-orion5x/common.c
+++ b/arch/arm/mach-orion5x/common.c
@@ -213,7 +213,7 @@  void __init orion5x_init_early(void)
 		mbus_soc_name = NULL;
 	mvebu_mbus_init(mbus_soc_name, ORION5X_BRIDGE_WINS_BASE,
 			ORION5X_BRIDGE_WINS_SZ,
-			ORION5X_DDR_WINS_BASE, ORION5X_DDR_WINS_SZ);
+			ORION5X_DDR_WINS_BASE, ORION5X_DDR_WINS_SZ, 0);
 }
 
 void orion5x_setup_wins(void)
diff --git a/drivers/bus/mvebu-mbus.c b/drivers/bus/mvebu-mbus.c
index 711dcf4..7c43782 100644
--- a/drivers/bus/mvebu-mbus.c
+++ b/drivers/bus/mvebu-mbus.c
@@ -838,7 +838,7 @@  fs_initcall(mvebu_mbus_debugfs_init);
 int __init mvebu_mbus_init(const char *soc, phys_addr_t mbuswins_phys_base,
 			   size_t mbuswins_size,
 			   phys_addr_t sdramwins_phys_base,
-			   size_t sdramwins_size)
+			   size_t sdramwins_size, int is_coherent)
 {
 	struct mvebu_mbus_state *mbus = &mbus_state;
 	const struct of_device_id *of_id;
@@ -865,8 +865,7 @@  int __init mvebu_mbus_init(const char *soc, phys_addr_t mbuswins_phys_base,
 		return -ENOMEM;
 	}
 
-	if (of_find_compatible_node(NULL, NULL, "marvell,coherency-fabric"))
-		mbus->hw_io_coherency = 1;
+	mbus->hw_io_coherency = is_coherent;
 
 	for (win = 0; win < mbus->soc->num_wins; win++)
 		mvebu_mbus_disable_window(mbus, win);
diff --git a/include/linux/mbus.h b/include/linux/mbus.h
index dba482e..e80b9c7 100644
--- a/include/linux/mbus.h
+++ b/include/linux/mbus.h
@@ -67,6 +67,6 @@  int mvebu_mbus_add_window(const char *devname, phys_addr_t base,
 int mvebu_mbus_del_window(phys_addr_t base, size_t size);
 int mvebu_mbus_init(const char *soc, phys_addr_t mbus_phys_base,
 		    size_t mbus_size, phys_addr_t sdram_phys_base,
-		    size_t sdram_size);
+		    size_t sdram_size, int is_coherent);
 
 #endif /* __LINUX_MBUS_H */