| Message ID | 20210526133457.3102393-1-punitagrawal@gmail.com (mailing list archive) |
|---|---|
| State | Not Applicable |
| Delegated to: | Lorenzo Pieralisi |
| Headers | show |
| Series | arm64: dts: rockchip: Update PCI host bridge window to 32-bit address memory | expand |
On Wed, May 26, 2021 at 8:35 AM Punit Agrawal <punitagrawal@gmail.com> wrote: > > The PCIe host bridge on RK3399 advertises a single address range > marked as 64-bit memory even though it lies entirely below 4GB. While > previously, the OF PCI range parser treated 64-bit ranges more > leniently (i.e., as 32-bit), since commit 9d57e61bf723 ("of/pci: Add > IORESOURCE_MEM_64 to resource flags for 64-bit memory addresses") the > code takes a stricter view and treats the ranges as advertised in the > device tree (i.e, as 64-bit). > > The change in behaviour causes failure when allocating bus addresses > to devices connected behind a PCI-to-PCI bridge that require > non-prefetchable memory ranges. The allocation failure was observed > for certain Samsung NVMe drives connected to RockPro64 boards. > > Update the host bridge window attributes to treat it as 32-bit address > memory. This fixes the allocation failure observed since commit > 9d57e61bf723. > > Reported-by: Alexandru Elisei <alexandru.elisei@arm.com> > Link: https://lore.kernel.org/r/7a1e2ebc-f7d8-8431-d844-41a9c36a8911@arm.com > Suggested-by: Robin Murphy <robin.murphy@arm.com> > Signed-off-by: Punit Agrawal <punitagrawal@gmail.com> > Cc: Heiko Stuebner <heiko@sntech.de> > Cc: Rob Herring <robh+dt@kernel.org> > --- > Hi, > > The patch fixes the failure observed with detecting certain Samsung > NVMe drives on RK3399 based boards. > > Hopefully, the folks on this thread can provide some input on the > reason the host bridge window was originally marked as 64-bit or if > there are any downsides to applying the patch. We can't require *only* a DT update to fix this. Ideally, the Rockchip PCI driver should clear the 64-bit flag in the resources though I'm not sure if the bridge driver would have access early enough. This is not the first time we've had to work-around RK3399 PCI DT (see commit d1ac0002dd297069 "of: address: Work around missing device_type property in pcie nodes"). Rob
Hi Rob, Thanks for taking a look. Rob Herring <robh+dt@kernel.org> writes: > On Wed, May 26, 2021 at 8:35 AM Punit Agrawal <punitagrawal@gmail.com> wrote: >> >> The PCIe host bridge on RK3399 advertises a single address range >> marked as 64-bit memory even though it lies entirely below 4GB. While >> previously, the OF PCI range parser treated 64-bit ranges more >> leniently (i.e., as 32-bit), since commit 9d57e61bf723 ("of/pci: Add >> IORESOURCE_MEM_64 to resource flags for 64-bit memory addresses") the >> code takes a stricter view and treats the ranges as advertised in the >> device tree (i.e, as 64-bit). >> >> The change in behaviour causes failure when allocating bus addresses >> to devices connected behind a PCI-to-PCI bridge that require >> non-prefetchable memory ranges. The allocation failure was observed >> for certain Samsung NVMe drives connected to RockPro64 boards. >> >> Update the host bridge window attributes to treat it as 32-bit address >> memory. This fixes the allocation failure observed since commit >> 9d57e61bf723. >> >> Reported-by: Alexandru Elisei <alexandru.elisei@arm.com> >> Link: https://lore.kernel.org/r/7a1e2ebc-f7d8-8431-d844-41a9c36a8911@arm.com >> Suggested-by: Robin Murphy <robin.murphy@arm.com> >> Signed-off-by: Punit Agrawal <punitagrawal@gmail.com> >> Cc: Heiko Stuebner <heiko@sntech.de> >> Cc: Rob Herring <robh+dt@kernel.org> >> --- >> Hi, >> >> The patch fixes the failure observed with detecting certain Samsung >> NVMe drives on RK3399 based boards. >> >> Hopefully, the folks on this thread can provide some input on the >> reason the host bridge window was originally marked as 64-bit or if >> there are any downsides to applying the patch. > > We can't require *only* a DT update to fix this. Ideally, the Rockchip > PCI driver should clear the 64-bit flag in the resources though I'm > not sure if the bridge driver would have access early enough. Following the discussion in the other thread, I tested the following changes to fixup 64-bit flag for non-prefetchable memory resources that fit below 4GB. If the changes look good, I'll send it out as a proper patch later today. ---->8---- diff --git a/drivers/pci/of.c b/drivers/pci/of.c index da5b414d585a..b9d0bee5a088 100644 --- a/drivers/pci/of.c +++ b/drivers/pci/of.c @@ -565,10 +565,14 @@ static int pci_parse_request_of_pci_ranges(struct device *dev, case IORESOURCE_MEM: res_valid |= !(res->flags & IORESOURCE_PREFETCH); - if (!(res->flags & IORESOURCE_PREFETCH)) + if (!(res->flags & IORESOURCE_PREFETCH)) { if (upper_32_bits(resource_size(res))) dev_warn(dev, "Memory resource size exceeds max for 32 bits\n"); - + if ((res->flags & IORESOURCE_MEM_64) && !upper_32_bits(res->end)) { + dev_warn(dev, "Overriding 64-bit flag for non-prefetchable memory below 4GB\n"); + res->flags &= ~IORESOURCE_MEM_64; + } + } break; } }
diff --git a/arch/arm64/boot/dts/rockchip/rk3399.dtsi b/arch/arm64/boot/dts/rockchip/rk3399.dtsi index 634a91af8e83..4b854eb21f72 100644 --- a/arch/arm64/boot/dts/rockchip/rk3399.dtsi +++ b/arch/arm64/boot/dts/rockchip/rk3399.dtsi @@ -227,7 +227,7 @@ pcie0: pcie@f8000000 { <&pcie_phy 2>, <&pcie_phy 3>; phy-names = "pcie-phy-0", "pcie-phy-1", "pcie-phy-2", "pcie-phy-3"; - ranges = <0x83000000 0x0 0xfa000000 0x0 0xfa000000 0x0 0x1e00000>, + ranges = <0x82000000 0x0 0xfa000000 0x0 0xfa000000 0x0 0x1e00000>, <0x81000000 0x0 0xfbe00000 0x0 0xfbe00000 0x0 0x100000>; resets = <&cru SRST_PCIE_CORE>, <&cru SRST_PCIE_MGMT>, <&cru SRST_PCIE_MGMT_STICKY>, <&cru SRST_PCIE_PIPE>,
The PCIe host bridge on RK3399 advertises a single address range marked as 64-bit memory even though it lies entirely below 4GB. While previously, the OF PCI range parser treated 64-bit ranges more leniently (i.e., as 32-bit), since commit 9d57e61bf723 ("of/pci: Add IORESOURCE_MEM_64 to resource flags for 64-bit memory addresses") the code takes a stricter view and treats the ranges as advertised in the device tree (i.e, as 64-bit). The change in behaviour causes failure when allocating bus addresses to devices connected behind a PCI-to-PCI bridge that require non-prefetchable memory ranges. The allocation failure was observed for certain Samsung NVMe drives connected to RockPro64 boards. Update the host bridge window attributes to treat it as 32-bit address memory. This fixes the allocation failure observed since commit 9d57e61bf723. Reported-by: Alexandru Elisei <alexandru.elisei@arm.com> Link: https://lore.kernel.org/r/7a1e2ebc-f7d8-8431-d844-41a9c36a8911@arm.com Suggested-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Punit Agrawal <punitagrawal@gmail.com> Cc: Heiko Stuebner <heiko@sntech.de> Cc: Rob Herring <robh+dt@kernel.org> --- Hi, The patch fixes the failure observed with detecting certain Samsung NVMe drives on RK3399 based boards. Hopefully, the folks on this thread can provide some input on the reason the host bridge window was originally marked as 64-bit or if there are any downsides to applying the patch. Thanks, Punit arch/arm64/boot/dts/rockchip/rk3399.dtsi | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)