| Message ID | 1462700001-30086-2-git-send-email-jchandra@broadcom.com (mailing list archive) |
|---|---|
| State | New, archived |
| Delegated to: | Bjorn Helgaas |
| Headers | show |
On Sun, May 08, 2016 at 03:03:21PM +0530, Jayachandran C wrote: > The Broadcom Vulcan PCI topology is slightly unusual, for a multi-node > system, it looks like: > > [bus 0] > | > +--[node 0 PCI bridge 0.0.0] > | | > | [bus 1] > | +---[SoC PCI devices 1.4.x, 1.5.x] ........ > | +---[Glue bridges 1.a.x, 1.b.x] \ > | | .....{node 0 GIC ITS} > | | / > | +----[SoC PCIe controller root ports] > | | \...... {SMMUv3 0..3} > | | > | +---- [external PCI devices] > +- [node 1 PCI bridge 0.0.1] > | | > | [bus n - similar to bus 1 above] > ... > > The for_each_dma_alias call for external PCI devices should not go > beyond the PCIe controllers where the SMMU and ITS is associated, going > further above to the glue bridges or the node bridges will find aliases > which are not valid. For internal devices, the association is at the > node level bridge and the alias search should not go further. I like the line of reasoning that we should not iterate higher in the hierarchy than where the translation hardware is attached. That seems like a reasonable thing in general, not just for this hardware. Is there some generic way to learn where the translation hardware is attached? If there is, we could make pci_for_each_dma_alias() use that information, and maybe we could solve the problem for other systems as well as for Vulcan. > To handle this quirk, we mark the SoC PCIe bridges and node PCI bridge > with flag PCI_DEV_FLAGS_DMA_ALIAS_ROOT so that the alias and RID > calculations are correct. > > Signed-off-by: Jayachandran C <jchandra@broadcom.com> > --- > drivers/pci/quirks.c | 14 ++++++++++++++ > 1 file changed, 14 insertions(+) > > diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c > index 8e67802..62664b5 100644 > --- a/drivers/pci/quirks.c > +++ b/drivers/pci/quirks.c > @@ -3734,6 +3734,20 @@ DECLARE_PCI_FIXUP_HEADER(0x1283, 0x8892, quirk_use_pcie_bridge_dma_alias); > DECLARE_PCI_FIXUP_HEADER(0x8086, 0x244e, quirk_use_pcie_bridge_dma_alias); > > /* > + * The IOMMU and interrupt controller of Broadcom vulcan are associated > + * not at the root bus, but at a bridge below. The DMA alias search should > + * not go above the bridge at which the association exists. > + */ > +static void quirk_bridge_brcm_vulcan_pcie_root(struct pci_dev *pdev) > +{ > + pdev->dev_flags |= PCI_DEV_FLAGS_DMA_ALIAS_ROOT; > +} > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, 0x9000, > + quirk_bridge_brcm_vulcan_pcie_root); > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, 0x9084, > + quirk_bridge_brcm_vulcan_pcie_root); > + > +/* > * Intersil/Techwell TW686[4589]-based video capture cards have an empty (zero) > * class code. Fix it. > */ > -- > 1.9.1 > -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[Adding ARM GIC and SMMU maintainers] On Sat, Jun 11, 2016 at 10:54 PM, Bjorn Helgaas <helgaas@kernel.org> wrote: > On Sun, May 08, 2016 at 03:03:21PM +0530, Jayachandran C wrote: >> The Broadcom Vulcan PCI topology is slightly unusual, for a multi-node >> system, it looks like: >> >> [bus 0] >> | >> +--[node 0 PCI bridge 0.0.0] >> | | >> | [bus 1] >> | +---[SoC PCI devices 1.4.x, 1.5.x] ........ >> | +---[Glue bridges 1.a.x, 1.b.x] \ >> | | .....{node 0 GIC ITS} >> | | / >> | +----[SoC PCIe controller root ports] >> | | \...... {SMMUv3 0..3} >> | | >> | +---- [external PCI devices] >> +- [node 1 PCI bridge 0.0.1] >> | | >> | [bus n - similar to bus 1 above] >> ... >> >> The for_each_dma_alias call for external PCI devices should not go >> beyond the PCIe controllers where the SMMU and ITS is associated, going >> further above to the glue bridges or the node bridges will find aliases >> which are not valid. For internal devices, the association is at the >> node level bridge and the alias search should not go further. > > I like the line of reasoning that we should not iterate higher in the > hierarchy than where the translation hardware is attached. That seems > like a reasonable thing in general, not just for this hardware. > > Is there some generic way to learn where the translation hardware is > attached? If there is, we could make pci_for_each_dma_alias() use > that information, and maybe we could solve the problem for other > systems as well as for Vulcan. There was discussion on using the return value of the callback function to stop the alias search when we reach the bridge to which the IOMMU or MSI controller is attached[1]. There was an iommu patch proposed to handle this for the device tree case[2], It is possible that we can do something similar for MSI and extend it to work when ACPI IORT is used as well. I will spend some time to see if this can be done. JC. [1] http://www.spinics.net/lists/linux-pci/msg51093.html [2] http://www.spinics.net/lists/arm-kernel/msg508266.html -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index 8e67802..62664b5 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -3734,6 +3734,20 @@ DECLARE_PCI_FIXUP_HEADER(0x1283, 0x8892, quirk_use_pcie_bridge_dma_alias); DECLARE_PCI_FIXUP_HEADER(0x8086, 0x244e, quirk_use_pcie_bridge_dma_alias); /* + * The IOMMU and interrupt controller of Broadcom vulcan are associated + * not at the root bus, but at a bridge below. The DMA alias search should + * not go above the bridge at which the association exists. + */ +static void quirk_bridge_brcm_vulcan_pcie_root(struct pci_dev *pdev) +{ + pdev->dev_flags |= PCI_DEV_FLAGS_DMA_ALIAS_ROOT; +} +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, 0x9000, + quirk_bridge_brcm_vulcan_pcie_root); +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_BROADCOM, 0x9084, + quirk_bridge_brcm_vulcan_pcie_root); + +/* * Intersil/Techwell TW686[4589]-based video capture cards have an empty (zero) * class code. Fix it. */
The Broadcom Vulcan PCI topology is slightly unusual, for a multi-node system, it looks like: [bus 0] | +--[node 0 PCI bridge 0.0.0] | | | [bus 1] | +---[SoC PCI devices 1.4.x, 1.5.x] ........ | +---[Glue bridges 1.a.x, 1.b.x] \ | | .....{node 0 GIC ITS} | | / | +----[SoC PCIe controller root ports] | | \...... {SMMUv3 0..3} | | | +---- [external PCI devices] +- [node 1 PCI bridge 0.0.1] | | | [bus n - similar to bus 1 above] ... The for_each_dma_alias call for external PCI devices should not go beyond the PCIe controllers where the SMMU and ITS is associated, going further above to the glue bridges or the node bridges will find aliases which are not valid. For internal devices, the association is at the node level bridge and the alias search should not go further. To handle this quirk, we mark the SoC PCIe bridges and node PCI bridge with flag PCI_DEV_FLAGS_DMA_ALIAS_ROOT so that the alias and RID calculations are correct. Signed-off-by: Jayachandran C <jchandra@broadcom.com> --- drivers/pci/quirks.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)