Message ID | 87ppqgyrmc.fsf@natisbad.org (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
Hi Arnaud, On 04/11/2013 19:00, Arnaud Ebalard wrote: > Hi, > > I am currently playing with a NETGEAR ReadyNAS 2120. It is powered by a > 2-core Armada XP SoC, more precisely a 78230 (this is what NETGEAR > advertises about the product; see the u-boot and /proc/cpuinfo info at > the end of this message). > > I spent some time this week-end trying to figure out why I was unable to > see the Fresco Logic FL1009 XHCI controller (PCIe Gen II x1) connected > to the PCIe bus on a 3.12-rc7 with a custom .dts: Even after enabling > all available ports referenced in armada-xp-mv78230.dtsi, I still only > see the 2 Marvell SATA controllers (on Lane 0 and 1 of port 0) but not > the FL1009. > > At some point, because I suspected a PCIe definition issue, I decided to > modify my .dts and make it inherit from armada-xp-mv78460.dtsi with all > PCIe ports enabled. At that point, here is what I got: > > root@thin:~# lspci > 00:01.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 > 00:02.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 > 00:03.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 > 00:04.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 > 00:05.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 > 00:06.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 > 00:07.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 > 00:08.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 > 00:09.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 > 00:0a.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 > 01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9170 (rev 12) > 02:00.0 SATA controller: Marvell Technology Group Ltd. Device 9170 (rev 12) > 03:00.0 Memory controller: Marvell Technology Group Ltd. Device 7846 (rev 02) > 04:00.0 Memory controller: Marvell Technology Group Ltd. Device 7846 (rev 02) > 05:00.0 USB controller: Fresco Logic FL1009 USB 3.0 Host Controller (rev 02) > 06:00.0 Memory controller: Marvell Technology Group Ltd. Device 7846 (rev 02) > 07:00.0 Memory controller: Marvell Technology Group Ltd. Device 7846 (rev 02) > 08:00.0 Memory controller: Marvell Technology Group Ltd. Device 7846 (rev 02) > 09:00.0 Memory controller: Marvell Technology Group Ltd. Device 7846 (rev 02) > 0a:00.0 Memory controller: Marvell Technology Group Ltd. Device 7846 (rev 02) > > Then, I took a look at the [laconic] public documentation on the various > flavours of the Armada XP (mv7823, mv78260 and mv78460) regarding PCIe > aspects. I noticed that [1] reports a x1 second PCIe unit for mv78230 > when [2] reports one x4 second unit for mv78230. FWIW, armada-xp-mv78230.dtsi > currently states the second unit is x4/x1. > > [1]: http://www.marvell.com/embedded-processors/armada-xp/ > [2]: http://www.marvell.com/embedded-processors/armada-xp/assets/Marvell-ArmadaXP-SoC-product%20brief.pdf Unless I missed something I can confirm your assumption with the doc I have about the hardware specification oof the MV78230, I can quote a small part: "PCIe unit 0 may be configured to x4 or quad x1 lanes. PCIe unit 1 is always x1." I didn't work on the PCIe port and just follow the thread about it, so I can be wrong, that's why I would like that Thomas give his opinion on the subject. Thanks for your feedback, Gregory > > I started modifying the 3.12-rc7 armada-xp-mv78230.dtsi considering [1] > was right and [2] was wrong. With the attached patch, here is what I get > with a .dts based on the modified armada-xp-mv78230.dtsi. Simply put, > it works (can see FL1009 and use it): > > root@thin:~# lspci > 00:01.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 > 00:02.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 > 00:05.0 PCI bridge: Marvell Technology Group Ltd. Device 7846 > 01:00.0 SATA controller: Marvell Technology Group Ltd. Device 9170 (rev 12) > 02:00.0 SATA controller: Marvell Technology Group Ltd. Device 9170 (rev 12) > 03:00.0 USB controller: Fresco Logic FL1009 USB 3.0 Host Controller (rev 02) > > At that point, my .dts contains the following: > > pcie-controller { > status = "okay"; > > /* Connected to first Marvell 88SE9170 SATA controller */ > pcie@1,0 { > /* Port 0, Lane 0 */ > status = "okay"; > }; > > /* Connected to second Marvell 88SE9170 SATA controller */ > pcie@2,0 { > /* Port 0, Lane 1 */ > status = "okay"; > }; > > /* Connected to Fresco Logic FL1009 USB 3.0 controller */ > pcie@5,0 { > /* Port 1, Lane 0 */ > status = "okay"; > }; > }; > > As I am way out of my league on the topic and do not have access to any > proper documentation, can someone give me some hints (based on symptoms > and working patch) on what may be happening? If I am correct on the root > cause, can someone comment on the patch? > > Cheers, > > a+ > > > For the records, here is what NETGEAR u-boot reports: > > U-Boot 2011.12 (Apr 09 2013 - 10:24:15) Marvell version: 2012_Q4.0 > Netgear version: 04/09/2013 ReadyNAS-2120 V1.7.8 > > Board: DB-78460-BP rev 2.0 > SoC: MV78230 B0 > running 2 CPUs > Custom configuration > CPU: Marvell PJ4B (584) v7 (Rev 2) LE > CPU 0 > CPU @ 1200 [MHz] > L2 @ 600 [MHz] > TClock @ 250 [MHz] > DDR @ 600 [MHz] > DDR 32Bit Width, FastPath Memory Access > DDR ECC Disabled > DRAM: 2 GiB > > and what I get on current 3.12-rc7 kernel: > > / # cat /proc/cpuinfo > Processor : Marvell PJ4Bv7 Processor rev 2 (v7l) > processor : 0 > BogoMIPS : 1196.85 > > processor : 1 > BogoMIPS : 1196.85 > > Features : swp half thumb fastmult vfp edsp vfpv3 > CPU implementer : 0x56 > CPU architecture: 7 > CPU variant : 0x2 > CPU part : 0x584 > CPU revision : 2 > > Hardware : Marvell Armada XP Development Board > Revision : 0000 > Serial : 0000000000000000 > > > diff --git a/arch/arm/boot/dts/armada-xp-mv78230.dtsi b/arch/arm/boot/dts/armada-xp-mv78230.dtsi > index 0358a33..9dc7381 100644 > --- a/arch/arm/boot/dts/armada-xp-mv78230.dtsi > +++ b/arch/arm/boot/dts/armada-xp-mv78230.dtsi > @@ -47,7 +47,7 @@ > /* > * MV78230 has 2 PCIe units Gen2.0: One unit can be > * configured as x4 or quad x1 lanes. One unit is > - * x4/x1. > + * x1 only. > */ > pcie-controller { > compatible = "marvell,armada-xp-pcie"; > @@ -61,10 +61,10 @@ > > ranges = > <0x82000000 0 0x40000 MBUS_ID(0xf0, 0x01) 0x40000 0 0x00002000 /* Port 0.0 registers */ > - 0x82000000 0 0x42000 MBUS_ID(0xf0, 0x01) 0x42000 0 0x00002000 /* Port 2.0 registers */ > 0x82000000 0 0x44000 MBUS_ID(0xf0, 0x01) 0x44000 0 0x00002000 /* Port 0.1 registers */ > 0x82000000 0 0x48000 MBUS_ID(0xf0, 0x01) 0x48000 0 0x00002000 /* Port 0.2 registers */ > 0x82000000 0 0x4c000 MBUS_ID(0xf0, 0x01) 0x4c000 0 0x00002000 /* Port 0.3 registers */ > + 0x82000000 0 0x80000 MBUS_ID(0xf0, 0x01) 0x80000 0 0x00002000 /* Port 1.0 registers */ > 0x82000000 0x1 0 MBUS_ID(0x04, 0xe8) 0 1 0 /* Port 0.0 MEM */ > 0x81000000 0x1 0 MBUS_ID(0x04, 0xe0) 0 1 0 /* Port 0.0 IO */ > 0x82000000 0x2 0 MBUS_ID(0x04, 0xd8) 0 1 0 /* Port 0.1 MEM */ > @@ -73,8 +73,8 @@ > 0x81000000 0x3 0 MBUS_ID(0x04, 0xb0) 0 1 0 /* Port 0.2 IO */ > 0x82000000 0x4 0 MBUS_ID(0x04, 0x78) 0 1 0 /* Port 0.3 MEM */ > 0x81000000 0x4 0 MBUS_ID(0x04, 0x70) 0 1 0 /* Port 0.3 IO */ > - 0x82000000 0x9 0 MBUS_ID(0x04, 0xf8) 0 1 0 /* Port 2.0 MEM */ > - 0x81000000 0x9 0 MBUS_ID(0x04, 0xf0) 0 1 0 /* Port 2.0 IO */>; > + 0x82000000 0x5 0 MBUS_ID(0x08, 0xe8) 0 1 0 /* Port 1.0 MEM */ > + 0x81000000 0x5 0 MBUS_ID(0x08, 0xe0) 0 1 0 /* Port 1.0 IO */>; > > pcie@1,0 { > device_type = "pci"; > @@ -144,20 +144,20 @@ > status = "disabled"; > }; > > - pcie@9,0 { > + pcie@5,0 { > device_type = "pci"; > - assigned-addresses = <0x82000800 0 0x42000 0 0x2000>; > - reg = <0x4800 0 0 0 0>; > + assigned-addresses = <0x82000800 0 0x80000 0 0x2000>; > + reg = <0x2800 0 0 0 0>; > #address-cells = <3>; > #size-cells = <2>; > #interrupt-cells = <1>; > - ranges = <0x82000000 0 0 0x82000000 0x9 0 1 0 > - 0x81000000 0 0 0x81000000 0x9 0 1 0>; > + ranges = <0x82000000 0 0 0x82000000 0x5 0 1 0 > + 0x81000000 0 0 0x81000000 0x5 0 1 0>; > interrupt-map-mask = <0 0 0 0>; > - interrupt-map = <0 0 0 0 &mpic 99>; > - marvell,pcie-port = <2>; > + interrupt-map = <0 0 0 0 &mpic 62>; > + marvell,pcie-port = <1>; > marvell,pcie-lane = <0>; > - clocks = <&gateclk 26>; > + clocks = <&gateclk 9>; > status = "disabled"; > }; > }; >
Dear Arnaud Ebalard, On Mon, 04 Nov 2013 19:00:11 +0100, Arnaud Ebalard wrote: > Then, I took a look at the [laconic] public documentation on the various > flavours of the Armada XP (mv7823, mv78260 and mv78460) regarding PCIe > aspects. I noticed that [1] reports a x1 second PCIe unit for mv78230 > when [2] reports one x4 second unit for mv78230. FWIW, armada-xp-mv78230.dtsi > currently states the second unit is x4/x1. Thanks for the great investigation! There is indeed something wrong in our .dts files, and the datasheet doesn't make everything entirely clear. The functional datasheet (which covers the three SoC variants 78230, 78260 and 78460) says: MV78230 2 PCIe units Gen2.0. One unit can be configured as x4 or quad x1 lanes. The other unit is x1. MV78260 3 PCIe units Gen2.0 Two units can be configured as x4 or quad x1 lanes. One unit is x4/x1. MV78460 4 PCIe units Gen2.0 Two of those units can be configured as x4 or quad x1 lanes. The other 2 units are x4 or x1. but for those variants (78230 and 78260) that don't have *all* PCIe interfaces, it doesn't say *which* ones are available from the set of 10 possibles interfaces in 78460, and therefore the 10 register areas that are available. The only place where we (Gregory pointed me in the right direction) could find some relation between the SoC variants and the available PCIe interfaces is in the hardware manual of each SoC variant, where the SERDES lanes are described. For MV78230, it says: PCIe 0.0, 0.1, 0.2, 0.3, 1.0, 1.1 and 1.2 For MV78260, it says: PCIe 0.0, 0.1, 0.2, 0.3, 1.0, 1.1, 1.2, 1.3 and 2 x4 For MV78460, it says PCIe 0.0, 0.1, 0.2, 0.3, 1.0, 1.1, 1.2, 1.3, 2 x4 and 3 x4 What it says for 78230 it a bit weird, because I don't see why it mentions 1.1 and 1.2 since the second PCIe interface on 78230 is said to be x1. Maybe that's a mistake in the datasheet, I'll try to get a confirmation about this. But indeed it seems like the second PCIe interface on 78230 is 1.0 and not 2.0, as you found out. So your patch below looks correct to me, and the mv78260.dtsi should also be modified to expose the correct PCIe interfaces. (We've mostly had access to 78460 hardware until now, and the only 78260 hardware I had access to only exposes the PCIe 0.0 interface, so I couldn't test the other ones). > diff --git a/arch/arm/boot/dts/armada-xp-mv78230.dtsi b/arch/arm/boot/dts/armada-xp-mv78230.dtsi > index 0358a33..9dc7381 100644 > --- a/arch/arm/boot/dts/armada-xp-mv78230.dtsi > +++ b/arch/arm/boot/dts/armada-xp-mv78230.dtsi > @@ -47,7 +47,7 @@ > /* > * MV78230 has 2 PCIe units Gen2.0: One unit can be > * configured as x4 or quad x1 lanes. One unit is > - * x4/x1. > + * x1 only. Ack. > */ > pcie-controller { > compatible = "marvell,armada-xp-pcie"; > @@ -61,10 +61,10 @@ > > ranges = > <0x82000000 0 0x40000 MBUS_ID(0xf0, 0x01) 0x40000 0 0x00002000 /* Port 0.0 registers */ > - 0x82000000 0 0x42000 MBUS_ID(0xf0, 0x01) 0x42000 0 0x00002000 /* Port 2.0 registers */ > 0x82000000 0 0x44000 MBUS_ID(0xf0, 0x01) 0x44000 0 0x00002000 /* Port 0.1 registers */ > 0x82000000 0 0x48000 MBUS_ID(0xf0, 0x01) 0x48000 0 0x00002000 /* Port 0.2 registers */ > 0x82000000 0 0x4c000 MBUS_ID(0xf0, 0x01) 0x4c000 0 0x00002000 /* Port 0.3 registers */ > + 0x82000000 0 0x80000 MBUS_ID(0xf0, 0x01) 0x80000 0 0x00002000 /* Port 1.0 registers */ Ack. > 0x82000000 0x1 0 MBUS_ID(0x04, 0xe8) 0 1 0 /* Port 0.0 MEM */ > 0x81000000 0x1 0 MBUS_ID(0x04, 0xe0) 0 1 0 /* Port 0.0 IO */ > 0x82000000 0x2 0 MBUS_ID(0x04, 0xd8) 0 1 0 /* Port 0.1 MEM */ > @@ -73,8 +73,8 @@ > 0x81000000 0x3 0 MBUS_ID(0x04, 0xb0) 0 1 0 /* Port 0.2 IO */ > 0x82000000 0x4 0 MBUS_ID(0x04, 0x78) 0 1 0 /* Port 0.3 MEM */ > 0x81000000 0x4 0 MBUS_ID(0x04, 0x70) 0 1 0 /* Port 0.3 IO */ > - 0x82000000 0x9 0 MBUS_ID(0x04, 0xf8) 0 1 0 /* Port 2.0 MEM */ > - 0x81000000 0x9 0 MBUS_ID(0x04, 0xf0) 0 1 0 /* Port 2.0 IO */>; > + 0x82000000 0x5 0 MBUS_ID(0x08, 0xe8) 0 1 0 /* Port 1.0 MEM */ > + 0x81000000 0x5 0 MBUS_ID(0x08, 0xe0) 0 1 0 /* Port 1.0 IO */>; Ack. > > pcie@1,0 { > device_type = "pci"; > @@ -144,20 +144,20 @@ > status = "disabled"; > }; > > - pcie@9,0 { > + pcie@5,0 { > device_type = "pci"; > - assigned-addresses = <0x82000800 0 0x42000 0 0x2000>; > - reg = <0x4800 0 0 0 0>; > + assigned-addresses = <0x82000800 0 0x80000 0 0x2000>; > + reg = <0x2800 0 0 0 0>; > #address-cells = <3>; > #size-cells = <2>; > #interrupt-cells = <1>; > - ranges = <0x82000000 0 0 0x82000000 0x9 0 1 0 > - 0x81000000 0 0 0x81000000 0x9 0 1 0>; > + ranges = <0x82000000 0 0 0x82000000 0x5 0 1 0 > + 0x81000000 0 0 0x81000000 0x5 0 1 0>; > interrupt-map-mask = <0 0 0 0>; > - interrupt-map = <0 0 0 0 &mpic 99>; > - marvell,pcie-port = <2>; > + interrupt-map = <0 0 0 0 &mpic 62>; > + marvell,pcie-port = <1>; > marvell,pcie-lane = <0>; > - clocks = <&gateclk 26>; > + clocks = <&gateclk 9>; > status = "disabled"; > }; > }; This all looks good. Would you mind submitting a proper patch with your SoB ? Of course, if you also want to fix 78260, feel free to do so. If not, then please let me know so I can send a patch. Again, thanks a lot for your investigation! Thomas
Arnaud, On Tue, 5 Nov 2013 10:48:45 +0100, Thomas Petazzoni wrote: > The only place where we (Gregory pointed me in the right direction) > could find some relation between the SoC variants and the available PCIe > interfaces is in the hardware manual of each SoC variant, where the > SERDES lanes are described. > > For MV78230, it says: > > PCIe 0.0, 0.1, 0.2, 0.3, 1.0, 1.1 and 1.2 [...] > What it says for 78230 it a bit weird, because I don't see why it > mentions 1.1 and 1.2 since the second PCIe interface on 78230 is said > to be x1. Maybe that's a mistake in the datasheet, I'll try to get a > confirmation about this. I've updated the MV78230 hardware datasheet that we had, and indeed the latest version available has this correct: MV78230 has PCIe 0.0, 0.1, 0.2, 0.3 and 1.0, which means a x4/x1 PCIe interface and a x1 one. The "1.1" and "1.2" indicated in an earlier version of the datasheet were a mistake, apparently. Bottom line: it means that the patch you submitted and that I Acked-by a few minutes ago is correct. Best regards, Thomas
diff --git a/arch/arm/boot/dts/armada-xp-mv78230.dtsi b/arch/arm/boot/dts/armada-xp-mv78230.dtsi index 0358a33..9dc7381 100644 --- a/arch/arm/boot/dts/armada-xp-mv78230.dtsi +++ b/arch/arm/boot/dts/armada-xp-mv78230.dtsi @@ -47,7 +47,7 @@ /* * MV78230 has 2 PCIe units Gen2.0: One unit can be * configured as x4 or quad x1 lanes. One unit is - * x4/x1. + * x1 only. */ pcie-controller { compatible = "marvell,armada-xp-pcie"; @@ -61,10 +61,10 @@ ranges = <0x82000000 0 0x40000 MBUS_ID(0xf0, 0x01) 0x40000 0 0x00002000 /* Port 0.0 registers */ - 0x82000000 0 0x42000 MBUS_ID(0xf0, 0x01) 0x42000 0 0x00002000 /* Port 2.0 registers */ 0x82000000 0 0x44000 MBUS_ID(0xf0, 0x01) 0x44000 0 0x00002000 /* Port 0.1 registers */ 0x82000000 0 0x48000 MBUS_ID(0xf0, 0x01) 0x48000 0 0x00002000 /* Port 0.2 registers */ 0x82000000 0 0x4c000 MBUS_ID(0xf0, 0x01) 0x4c000 0 0x00002000 /* Port 0.3 registers */ + 0x82000000 0 0x80000 MBUS_ID(0xf0, 0x01) 0x80000 0 0x00002000 /* Port 1.0 registers */ 0x82000000 0x1 0 MBUS_ID(0x04, 0xe8) 0 1 0 /* Port 0.0 MEM */ 0x81000000 0x1 0 MBUS_ID(0x04, 0xe0) 0 1 0 /* Port 0.0 IO */ 0x82000000 0x2 0 MBUS_ID(0x04, 0xd8) 0 1 0 /* Port 0.1 MEM */ @@ -73,8 +73,8 @@ 0x81000000 0x3 0 MBUS_ID(0x04, 0xb0) 0 1 0 /* Port 0.2 IO */ 0x82000000 0x4 0 MBUS_ID(0x04, 0x78) 0 1 0 /* Port 0.3 MEM */ 0x81000000 0x4 0 MBUS_ID(0x04, 0x70) 0 1 0 /* Port 0.3 IO */ - 0x82000000 0x9 0 MBUS_ID(0x04, 0xf8) 0 1 0 /* Port 2.0 MEM */ - 0x81000000 0x9 0 MBUS_ID(0x04, 0xf0) 0 1 0 /* Port 2.0 IO */>; + 0x82000000 0x5 0 MBUS_ID(0x08, 0xe8) 0 1 0 /* Port 1.0 MEM */ + 0x81000000 0x5 0 MBUS_ID(0x08, 0xe0) 0 1 0 /* Port 1.0 IO */>; pcie@1,0 { device_type = "pci"; @@ -144,20 +144,20 @@ status = "disabled"; }; - pcie@9,0 { + pcie@5,0 { device_type = "pci"; - assigned-addresses = <0x82000800 0 0x42000 0 0x2000>; - reg = <0x4800 0 0 0 0>; + assigned-addresses = <0x82000800 0 0x80000 0 0x2000>; + reg = <0x2800 0 0 0 0>; #address-cells = <3>; #size-cells = <2>; #interrupt-cells = <1>; - ranges = <0x82000000 0 0 0x82000000 0x9 0 1 0 - 0x81000000 0 0 0x81000000 0x9 0 1 0>; + ranges = <0x82000000 0 0 0x82000000 0x5 0 1 0 + 0x81000000 0 0 0x81000000 0x5 0 1 0>; interrupt-map-mask = <0 0 0 0>; - interrupt-map = <0 0 0 0 &mpic 99>; - marvell,pcie-port = <2>; + interrupt-map = <0 0 0 0 &mpic 62>; + marvell,pcie-port = <1>; marvell,pcie-lane = <0>; - clocks = <&gateclk 26>; + clocks = <&gateclk 9>; status = "disabled"; }; };