| Message ID | 20190515072747.39941-2-xiaowei.bao@nxp.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [1/3] dt-bindings: pci: layerscape-pci: add compatible strings "fsl, ls1028a-pcie" | expand |
On Wed, May 15, 2019 at 9:36 AM Xiaowei Bao <xiaowei.bao@nxp.com> wrote: > Signed-off-by: Xiaowei Bao <xiaowei.bao@nxp.com> > --- > arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi | 52 ++++++++++++++++++++++++ > 1 files changed, 52 insertions(+), 0 deletions(-) > > diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > index b045812..50b579b 100644 > --- a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > +++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > @@ -398,6 +398,58 @@ > status = "disabled"; > }; > > + pcie@3400000 { > + compatible = "fsl,ls1028a-pcie"; > + reg = <0x00 0x03400000 0x0 0x00100000 /* controller registers */ > + 0x80 0x00000000 0x0 0x00002000>; /* configuration space */ > + reg-names = "regs", "config"; > + interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>, /* PME interrupt */ > + <GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>; /* aer interrupt */ > + interrupt-names = "pme", "aer"; > + #address-cells = <3>; > + #size-cells = <2>; > + device_type = "pci"; > + dma-coherent; > + num-lanes = <4>; > + bus-range = <0x0 0xff>; > + ranges = <0x81000000 0x0 0x00000000 0x80 0x00010000 0x0 0x00010000 /* downstream I/O */ > + 0x82000000 0x0 0x40000000 0x80 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */ Are you sure there is no support for 64-bit BARs or prefetchable memory? Is this a hardware bug, or something that can be fixed in firmware? Arnd
Hi Arnd, -----Original Message----- From: Arnd Bergmann <arnd@arndb.de> Sent: 2019年5月15日 16:05 To: Xiaowei Bao <xiaowei.bao@nxp.com> Cc: Bjorn Helgaas <bhelgaas@google.com>; Rob Herring <robh+dt@kernel.org>; Mark Rutland <mark.rutland@arm.com>; Shawn Guo <shawnguo@kernel.org>; Leo Li <leoyang.li@nxp.com>; Kishon <kishon@ti.com>; Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; gregkh <gregkh@linuxfoundation.org>; M.h. Lian <minghuan.lian@nxp.com>; Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang <roy.zang@nxp.com>; Kate Stewart <kstewart@linuxfoundation.org>; Philippe Ombredanne <pombredanne@nexb.com>; Shawn Lin <shawn.lin@rock-chips.com>; linux-pci <linux-pci@vger.kernel.org>; DTML <devicetree@vger.kernel.org>; Linux Kernel Mailing List <linux-kernel@vger.kernel.org>; Linux ARM <linux-arm-kernel@lists.infradead.org>; linuxppc-dev <linuxppc-dev@lists.ozlabs.org> Subject: [EXT] Re: [PATCH 2/3] arm64: dts: ls1028a: Add PCIe controller DT nodes Caution: EXT Email On Wed, May 15, 2019 at 9:36 AM Xiaowei Bao <xiaowei.bao@nxp.com> wrote: > Signed-off-by: Xiaowei Bao <xiaowei.bao@nxp.com> > --- > arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi | 52 ++++++++++++++++++++++++ > 1 files changed, 52 insertions(+), 0 deletions(-) > > diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > index b045812..50b579b 100644 > --- a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > +++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > @@ -398,6 +398,58 @@ > status = "disabled"; > }; > > + pcie@3400000 { > + compatible = "fsl,ls1028a-pcie"; > + reg = <0x00 0x03400000 0x0 0x00100000 /* controller registers */ > + 0x80 0x00000000 0x0 0x00002000>; /* configuration space */ > + reg-names = "regs", "config"; > + interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>, /* PME interrupt */ > + <GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>; /* aer interrupt */ > + interrupt-names = "pme", "aer"; > + #address-cells = <3>; > + #size-cells = <2>; > + device_type = "pci"; > + dma-coherent; > + num-lanes = <4>; > + bus-range = <0x0 0xff>; > + ranges = <0x81000000 0x0 0x00000000 0x80 0x00010000 0x0 0x00010000 /* downstream I/O */ > + 0x82000000 0x0 0x40000000 0x80 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */ Are you sure there is no support for 64-bit BARs or prefetchable memory? [Xiaowei Bao] sorry for late reply, Thought that our Layerscape platform has not added prefetchable memory support in DTS, so this platform has not been added, I will submit a separate patch to add prefetchable memory support for all Layerscape platforms. Of course, the prefetchable PCIE device can work in our boards, because the RC will assign non-prefetchable memory for this device. We reserve 1G no-prefetchable memory for PCIE device, it is enough for general devices. Is this a hardware bug, or something that can be fixed in firmware? [Xiaowei Bao] this is not a hardware bug, our HW support the 64-bit prefetchable memory. Arnd
On Fri, May 17, 2019 at 5:21 AM Xiaowei Bao <xiaowei.bao@nxp.com> wrote: > -----Original Message----- > From: Arnd Bergmann <arnd@arndb.de> > On Wed, May 15, 2019 at 9:36 AM Xiaowei Bao <xiaowei.bao@nxp.com> wrote: > > Signed-off-by: Xiaowei Bao <xiaowei.bao@nxp.com> > > --- > > arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi | 52 ++++++++++++++++++++++++ > > 1 files changed, 52 insertions(+), 0 deletions(-) > > > > diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > > index b045812..50b579b 100644 > > --- a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > > +++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > > @@ -398,6 +398,58 @@ > > status = "disabled"; > > }; > > > > + pcie@3400000 { > > + compatible = "fsl,ls1028a-pcie"; > > + reg = <0x00 0x03400000 0x0 0x00100000 /* controller registers */ > > + 0x80 0x00000000 0x0 0x00002000>; /* configuration space */ > > + reg-names = "regs", "config"; > > + interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>, /* PME interrupt */ > > + <GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>; /* aer interrupt */ > > + interrupt-names = "pme", "aer"; > > + #address-cells = <3>; > > + #size-cells = <2>; > > + device_type = "pci"; > > + dma-coherent; > > + num-lanes = <4>; > > + bus-range = <0x0 0xff>; > > + ranges = <0x81000000 0x0 0x00000000 0x80 0x00010000 0x0 0x00010000 /* downstream I/O */ > > + 0x82000000 0x0 0x40000000 0x80 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */ > > Are you sure there is no support for 64-bit BARs or prefetchable memory? > [Xiaowei Bao] sorry for late reply, Thought that our Layerscape platform has not added prefetchable memory support in DTS, so this platform has not been added, I will submit a separate patch to add prefetchable memory support for all Layerscape platforms. Ok, thanks. > Of course, the prefetchable PCIE device can work in our boards, because the RC will > assign non-prefetchable memory for this device. We reserve 1G no-prefetchable > memory for PCIE device, it is enough for general devices. Sure, many devices work just fine, this is mostly a question of supporting those devices that do require multiple gigabytes, or that need prefetchable memory semantics to get the expected performance. GPUs are the obvious example, but I think there are others (infiniband?). Arnd
On Fri, 17 May 2019 at 10:59, Arnd Bergmann <arnd@arndb.de> wrote: > > On Fri, May 17, 2019 at 5:21 AM Xiaowei Bao <xiaowei.bao@nxp.com> wrote: > > -----Original Message----- > > From: Arnd Bergmann <arnd@arndb.de> > > On Wed, May 15, 2019 at 9:36 AM Xiaowei Bao <xiaowei.bao@nxp.com> wrote: > > > Signed-off-by: Xiaowei Bao <xiaowei.bao@nxp.com> > > > --- > > > arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi | 52 ++++++++++++++++++++++++ > > > 1 files changed, 52 insertions(+), 0 deletions(-) > > > > > > diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > > > index b045812..50b579b 100644 > > > --- a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > > > +++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > > > @@ -398,6 +398,58 @@ > > > status = "disabled"; > > > }; > > > > > > + pcie@3400000 { > > > + compatible = "fsl,ls1028a-pcie"; > > > + reg = <0x00 0x03400000 0x0 0x00100000 /* controller registers */ > > > + 0x80 0x00000000 0x0 0x00002000>; /* configuration space */ > > > + reg-names = "regs", "config"; > > > + interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>, /* PME interrupt */ > > > + <GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>; /* aer interrupt */ > > > + interrupt-names = "pme", "aer"; > > > + #address-cells = <3>; > > > + #size-cells = <2>; > > > + device_type = "pci"; > > > + dma-coherent; > > > + num-lanes = <4>; > > > + bus-range = <0x0 0xff>; > > > + ranges = <0x81000000 0x0 0x00000000 0x80 0x00010000 0x0 0x00010000 /* downstream I/O */ > > > + 0x82000000 0x0 0x40000000 0x80 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */ > > > > Are you sure there is no support for 64-bit BARs or prefetchable memory? > > [Xiaowei Bao] sorry for late reply, Thought that our Layerscape platform has not added prefetchable memory support in DTS, so this platform has not been added, I will submit a separate patch to add prefetchable memory support for all Layerscape platforms. > > Ok, thanks. > > > Of course, the prefetchable PCIE device can work in our boards, because the RC will > > assign non-prefetchable memory for this device. We reserve 1G no-prefetchable > > memory for PCIE device, it is enough for general devices. > > Sure, many devices work just fine, this is mostly a question of supporting those > devices that do require multiple gigabytes, or that need prefetchable memory > semantics to get the expected performance. GPUs are the obvious example, > but I think there are others (infiniband?). > Some implementations of the Synopsys dw PCIe IP contain a 'root port' (within quotes because it is not actually a root port but an arbitrary set of MMIO registers that looks like a type 01 config region) that does not permit the prefetchable bridge window BAR to be written (a thing which is apparently permitted by the PCIe spec). So while the host bridge is capable of supporting more than one MMIO BAR window, the OS visible software interface does not expose this functionality In practice, it probably doesn't matter, since the driver uses the same iATU attributes for prefetchable and non-prefetchable windows, but I guess 1 GB of MMIO BAR space is a bit restrictive for modern systems.
Hi Arndt， -----Original Message----- From: Arnd Bergmann <arnd@arndb.de> Sent: 2019年5月17日 16:59 To: Xiaowei Bao <xiaowei.bao@nxp.com> Cc: Bjorn Helgaas <bhelgaas@google.com>; Rob Herring <robh+dt@kernel.org>; Mark Rutland <mark.rutland@arm.com>; Shawn Guo <shawnguo@kernel.org>; Leo Li <leoyang.li@nxp.com>; Kishon <kishon@ti.com>; Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>; gregkh <gregkh@linuxfoundation.org>; M.h. Lian <minghuan.lian@nxp.com>; Mingkai Hu <mingkai.hu@nxp.com>; Roy Zang <roy.zang@nxp.com>; Kate Stewart <kstewart@linuxfoundation.org>; Philippe Ombredanne <pombredanne@nexb.com>; Shawn Lin <shawn.lin@rock-chips.com>; linux-pci <linux-pci@vger.kernel.org>; DTML <devicetree@vger.kernel.org>; Linux Kernel Mailing List <linux-kernel@vger.kernel.org>; Linux ARM <linux-arm-kernel@lists.infradead.org>; linuxppc-dev <linuxppc-dev@lists.ozlabs.org> Subject: Re: [EXT] Re: [PATCH 2/3] arm64: dts: ls1028a: Add PCIe controller DT nodes Caution: EXT Email On Fri, May 17, 2019 at 5:21 AM Xiaowei Bao <xiaowei.bao@nxp.com> wrote: > -----Original Message----- > From: Arnd Bergmann <arnd@arndb.de> > On Wed, May 15, 2019 at 9:36 AM Xiaowei Bao <xiaowei.bao@nxp.com> wrote: > > Signed-off-by: Xiaowei Bao <xiaowei.bao@nxp.com> > > --- > > arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi | 52 ++++++++++++++++++++++++ > > 1 files changed, 52 insertions(+), 0 deletions(-) > > > > diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > > b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > > index b045812..50b579b 100644 > > --- a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > > +++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi > > @@ -398,6 +398,58 @@ > > status = "disabled"; > > }; > > > > + pcie@3400000 { > > + compatible = "fsl,ls1028a-pcie"; > > + reg = <0x00 0x03400000 0x0 0x00100000 /* controller registers */ > > + 0x80 0x00000000 0x0 0x00002000>; /* configuration space */ > > + reg-names = "regs", "config"; > > + interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>, /* PME interrupt */ > > + <GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>; /* aer interrupt */ > > + interrupt-names = "pme", "aer"; > > + #address-cells = <3>; > > + #size-cells = <2>; > > + device_type = "pci"; > > + dma-coherent; > > + num-lanes = <4>; > > + bus-range = <0x0 0xff>; > > + ranges = <0x81000000 0x0 0x00000000 0x80 0x00010000 0x0 0x00010000 /* downstream I/O */ > > + 0x82000000 0x0 0x40000000 0x80 > > + 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */ > > Are you sure there is no support for 64-bit BARs or prefetchable memory? > [Xiaowei Bao] sorry for late reply, Thought that our Layerscape platform has not added prefetchable memory support in DTS, so this platform has not been added, I will submit a separate patch to add prefetchable memory support for all Layerscape platforms. Ok, thanks. > Of course, the prefetchable PCIE device can work in our boards, > because the RC will assign non-prefetchable memory for this device. We > reserve 1G no-prefetchable memory for PCIE device, it is enough for general devices. Sure, many devices work just fine, this is mostly a question of supporting those devices that do require multiple gigabytes, or that need prefetchable memory semantics to get the expected performance. GPUs are the obvious example, but I think there are others (infiniband?). [Xiaowei Bao] sorry, I don't know much about infiniband and GPU, as you said, I think many devices works fine with this DTS, I will add the prefetchable memory entry in DTS future and submit another patch. Arnd
diff --git a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi index b045812..50b579b 100644 --- a/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi +++ b/arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi @@ -398,6 +398,58 @@ status = "disabled"; }; + pcie@3400000 { + compatible = "fsl,ls1028a-pcie"; + reg = <0x00 0x03400000 0x0 0x00100000 /* controller registers */ + 0x80 0x00000000 0x0 0x00002000>; /* configuration space */ + reg-names = "regs", "config"; + interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>, /* PME interrupt */ + <GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>; /* aer interrupt */ + interrupt-names = "pme", "aer"; + #address-cells = <3>; + #size-cells = <2>; + device_type = "pci"; + dma-coherent; + num-lanes = <4>; + bus-range = <0x0 0xff>; + ranges = <0x81000000 0x0 0x00000000 0x80 0x00010000 0x0 0x00010000 /* downstream I/O */ + 0x82000000 0x0 0x40000000 0x80 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */ + msi-parent = <&its>; + #interrupt-cells = <1>; + interrupt-map-mask = <0 0 0 7>; + interrupt-map = <0000 0 0 1 &gic GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>, + <0000 0 0 2 &gic GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>, + <0000 0 0 3 &gic GIC_SPI 111 IRQ_TYPE_LEVEL_HIGH>, + <0000 0 0 4 &gic GIC_SPI 112 IRQ_TYPE_LEVEL_HIGH>; + status = "disabled"; + }; + + pcie@3500000 { + compatible = "fsl,ls1028a-pcie"; + reg = <0x00 0x03500000 0x0 0x00100000 /* controller registers */ + 0x88 0x00000000 0x0 0x00002000>; /* configuration space */ + reg-names = "regs", "config"; + interrupts = <GIC_SPI 113 IRQ_TYPE_LEVEL_HIGH>, + <GIC_SPI 114 IRQ_TYPE_LEVEL_HIGH>; + interrupt-names = "pme", "aer"; + #address-cells = <3>; + #size-cells = <2>; + device_type = "pci"; + dma-coherent; + num-lanes = <4>; + bus-range = <0x0 0xff>; + ranges = <0x81000000 0x0 0x00000000 0x88 0x00010000 0x0 0x00010000 /* downstream I/O */ + 0x82000000 0x0 0x40000000 0x88 0x40000000 0x0 0x40000000>; /* non-prefetchable memory */ + msi-parent = <&its>; + #interrupt-cells = <1>; + interrupt-map-mask = <0 0 0 7>; + interrupt-map = <0000 0 0 1 &gic GIC_SPI 114 IRQ_TYPE_LEVEL_HIGH>, + <0000 0 0 2 &gic GIC_SPI 115 IRQ_TYPE_LEVEL_HIGH>, + <0000 0 0 3 &gic GIC_SPI 116 IRQ_TYPE_LEVEL_HIGH>, + <0000 0 0 4 &gic GIC_SPI 117 IRQ_TYPE_LEVEL_HIGH>; + status = "disabled"; + }; + pcie@1f0000000 { /* Integrated Endpoint Root Complex */ compatible = "pci-host-ecam-generic"; reg = <0x01 0xf0000000 0x0 0x100000>;
LS1028a implements 2 PCIe 3.0 controllers. Signed-off-by: Xiaowei Bao <xiaowei.bao@nxp.com> --- arch/arm64/boot/dts/freescale/fsl-ls1028a.dtsi | 52 ++++++++++++++++++++++++ 1 files changed, 52 insertions(+), 0 deletions(-)