| Message ID | f18738b7096fb06a5865214ddc299ce99630673b.1480644928.git.dhdang@apm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 12/01/2016 09:27 PM, Duc Dang wrote: > PCIe controllers in X-Gene SoCs is not ECAM compliant: software > needs to configure additional controller's register to address > device at bus:dev:function. > > The quirk will discover controller MMIO register space and configure > controller registers to select and address the target secondary device. > > The quirk will only be applied for X-Gene PCIe MCFG table with > OEM revison 1, 2, 3 or 4 (PCIe controller v1 and v2 on X-Gene SoCs). > > Signed-off-by: Duc Dang <dhdang@apm.com> So far I've tested this on an HPE ProLiant m400 (Moonshot) cartridge and will test it on some other reference platforms soon. Bootlog for the m400 attached in case Bjorn wants to see the output. Here's what I see in /proc/iomem btw on that platform: # cat /proc/iomem 10520000-10523fff : APMC0D18:00 10520000-10523fff : APMC0D18:00 10524000-10527fff : APMC0D17:00 10540000-1054a0ff : APMC0D01:00 10546000-10546fff : APMC0D50:00 1054a000-1054a00f : APMC0D12:03 1054a000-1054a00f : APMC0D12:02 1054a000-1054a00f : APMC0D12:01 1054a000-1054a00f : APMC0D12:00 17000000-17000fff : APMC0D01:00 17001000-17001fff : APMC0D01:00 17001000-170013ff : APMC0D15:00 17001000-170013ff : APMC0D15:00 1701c000-1701cfff : APMC0D14:00 1a800000-1a800fff : APMC0D0D:00 1a800000-1a800fff : APMC0D0D:00 1c000200-1c0002ff : APMC0D06:00 1c021000-1c0210ff : APMC0D08:00 1c021000-1c02101f : serial 1c024000-1c024fff : APMC0D07:00 1f230000-1f230fff : APMC0D0D:00 1f230000-1f230fff : APMC0D0D:00 1f23d000-1f23dfff : APMC0D0D:00 1f23d000-1f23dfff : APMC0D0D:00 1f23e000-1f23efff : APMC0D0D:00 1f23e000-1f23efff : APMC0D0D:00 1f2a0000-1f31ffff : APMC0D06:00 1f500000-1f50ffff : PCI Bus 0000:00 1f500000-1f50ffff : PNP0A08:00 78800000-78800fff : APMC0D13:00 78800000-78800fff : APMC0D12:03 78800000-78800fff : APMC0D12:02 78800000-78800fff : APMC0D12:01 78800000-78800fff : APMC0D12:00 78800000-78800fff : APMC0D11:00 78800000-78800fff : APMC0D10:03 78800000-78800fff : APMC0D10:02 78800000-78800fff : APMC0D10:01 78800000-78800fff : APMC0D10:00 79000000-798fffff : APMC0D0E:00 7c000000-7c1fffff : APMC0D12:00 7c200000-7c3fffff : APMC0D12:01 7c400000-7c5fffff : APMC0D12:02 7c600000-7c7fffff : APMC0D12:03 7e000000-7e000fff : APMC0D13:00 7e200000-7e200fff : APMC0D10:03 7e200000-7e200fff : APMC0D10:02 7e200000-7e200fff : APMC0D10:01 7e200000-7e200fff : APMC0D10:00 7e600000-7e600fff : APMC0D11:00 7e700000-7e700fff : APMC0D10:03 7e700000-7e700fff : APMC0D10:02 7e700000-7e700fff : APMC0D10:01 7e700000-7e700fff : APMC0D10:00 7e720000-7e720fff : APMC0D10:03 7e720000-7e720fff : APMC0D10:02 7e720000-7e720fff : APMC0D10:01 7e720000-7e720fff : APMC0D10:00 7e800000-7e800fff : APMC0D10:00 7e840000-7e840fff : APMC0D10:01 7e880000-7e880fff : APMC0D10:02 7e8c0000-7e8c0fff : APMC0D10:03 7e930000-7e930fff : APMC0D13:00 4000000000-4001ffffff : System RAM 4000080000-4000c3ffff : Kernel code 4000db0000-400165ffff : Kernel data 40023a0000-4ff733ffff : System RAM 4ff7340000-4ff77cffff : reserved 4ff77d0000-4ff79cffff : System RAM 4ff79d0000-4ff7e7ffff : reserved 4ff7e80000-4ff7e8ffff : System RAM 4ff7e90000-4ff7efffff : reserved 4ff7f10000-4ff800ffff : reserved 4ff8010000-4fffffffff : System RAM a020000000-a03fffffff : PCI Bus 0000:00 a020000000-a0201fffff : PCI Bus 0000:01 a020000000-a0200fffff : 0000:01:00.0 a020000000-a0200fffff : mlx4_core a020100000-a0201fffff : 0000:01:00.0 a060000000-a07fffffff : PCI Bus 0000:00 a0d0000000-a0dfffffff : PCI ECAM a110000000-a14fffffff : PCI Bus 0000:00 a110000000-a121ffffff : PCI Bus 0000:01 a110000000-a111ffffff : 0000:01:00.0 a110000000-a111ffffff : mlx4_core a112000000-a121ffffff : 0000:01:00.0 Adding a Tested-by for the record: Tested-by: Jon Masters <jcm@redhat.com> Jon.
On Thu, Dec 1, 2016 at 11:12 PM, Jon Masters <jcm@redhat.com> wrote: > On 12/01/2016 09:27 PM, Duc Dang wrote: >> PCIe controllers in X-Gene SoCs is not ECAM compliant: software >> needs to configure additional controller's register to address >> device at bus:dev:function. >> >> The quirk will discover controller MMIO register space and configure >> controller registers to select and address the target secondary device. >> >> The quirk will only be applied for X-Gene PCIe MCFG table with >> OEM revison 1, 2, 3 or 4 (PCIe controller v1 and v2 on X-Gene SoCs). >> >> Signed-off-by: Duc Dang <dhdang@apm.com> > > So far I've tested this on an HPE ProLiant m400 (Moonshot) cartridge > and will test it on some other reference platforms soon. Bootlog for > the m400 attached in case Bjorn wants to see the output. Here's > what I see in /proc/iomem btw on that platform: > > # cat /proc/iomem > 10520000-10523fff : APMC0D18:00 > 10520000-10523fff : APMC0D18:00 > 10524000-10527fff : APMC0D17:00 > 10540000-1054a0ff : APMC0D01:00 > 10546000-10546fff : APMC0D50:00 > 1054a000-1054a00f : APMC0D12:03 > 1054a000-1054a00f : APMC0D12:02 > 1054a000-1054a00f : APMC0D12:01 > 1054a000-1054a00f : APMC0D12:00 > 17000000-17000fff : APMC0D01:00 > 17001000-17001fff : APMC0D01:00 > 17001000-170013ff : APMC0D15:00 > 17001000-170013ff : APMC0D15:00 > 1701c000-1701cfff : APMC0D14:00 > 1a800000-1a800fff : APMC0D0D:00 > 1a800000-1a800fff : APMC0D0D:00 > 1c000200-1c0002ff : APMC0D06:00 > 1c021000-1c0210ff : APMC0D08:00 > 1c021000-1c02101f : serial > 1c024000-1c024fff : APMC0D07:00 > 1f230000-1f230fff : APMC0D0D:00 > 1f230000-1f230fff : APMC0D0D:00 > 1f23d000-1f23dfff : APMC0D0D:00 > 1f23d000-1f23dfff : APMC0D0D:00 > 1f23e000-1f23efff : APMC0D0D:00 > 1f23e000-1f23efff : APMC0D0D:00 > 1f2a0000-1f31ffff : APMC0D06:00 > 1f500000-1f50ffff : PCI Bus 0000:00 > 1f500000-1f50ffff : PNP0A08:00 > 78800000-78800fff : APMC0D13:00 > 78800000-78800fff : APMC0D12:03 > 78800000-78800fff : APMC0D12:02 > 78800000-78800fff : APMC0D12:01 > 78800000-78800fff : APMC0D12:00 > 78800000-78800fff : APMC0D11:00 > 78800000-78800fff : APMC0D10:03 > 78800000-78800fff : APMC0D10:02 > 78800000-78800fff : APMC0D10:01 > 78800000-78800fff : APMC0D10:00 > 79000000-798fffff : APMC0D0E:00 > 7c000000-7c1fffff : APMC0D12:00 > 7c200000-7c3fffff : APMC0D12:01 > 7c400000-7c5fffff : APMC0D12:02 > 7c600000-7c7fffff : APMC0D12:03 > 7e000000-7e000fff : APMC0D13:00 > 7e200000-7e200fff : APMC0D10:03 > 7e200000-7e200fff : APMC0D10:02 > 7e200000-7e200fff : APMC0D10:01 > 7e200000-7e200fff : APMC0D10:00 > 7e600000-7e600fff : APMC0D11:00 > 7e700000-7e700fff : APMC0D10:03 > 7e700000-7e700fff : APMC0D10:02 > 7e700000-7e700fff : APMC0D10:01 > 7e700000-7e700fff : APMC0D10:00 > 7e720000-7e720fff : APMC0D10:03 > 7e720000-7e720fff : APMC0D10:02 > 7e720000-7e720fff : APMC0D10:01 > 7e720000-7e720fff : APMC0D10:00 > 7e800000-7e800fff : APMC0D10:00 > 7e840000-7e840fff : APMC0D10:01 > 7e880000-7e880fff : APMC0D10:02 > 7e8c0000-7e8c0fff : APMC0D10:03 > 7e930000-7e930fff : APMC0D13:00 > 4000000000-4001ffffff : System RAM > 4000080000-4000c3ffff : Kernel code > 4000db0000-400165ffff : Kernel data > 40023a0000-4ff733ffff : System RAM > 4ff7340000-4ff77cffff : reserved > 4ff77d0000-4ff79cffff : System RAM > 4ff79d0000-4ff7e7ffff : reserved > 4ff7e80000-4ff7e8ffff : System RAM > 4ff7e90000-4ff7efffff : reserved > 4ff7f10000-4ff800ffff : reserved > 4ff8010000-4fffffffff : System RAM > a020000000-a03fffffff : PCI Bus 0000:00 > a020000000-a0201fffff : PCI Bus 0000:01 > a020000000-a0200fffff : 0000:01:00.0 > a020000000-a0200fffff : mlx4_core > a020100000-a0201fffff : 0000:01:00.0 > a060000000-a07fffffff : PCI Bus 0000:00 > a0d0000000-a0dfffffff : PCI ECAM > a110000000-a14fffffff : PCI Bus 0000:00 > a110000000-a121ffffff : PCI Bus 0000:01 > a110000000-a111ffffff : 0000:01:00.0 > a110000000-a111ffffff : mlx4_core > a112000000-a121ffffff : 0000:01:00.0 > > Adding a Tested-by for the record: > > Tested-by: Jon Masters <jcm@redhat.com> Thanks a lot for testing this, Jon. > > Jon. > > -- > Computer Architect | Sent from my Fedora powered laptop > Regards, Duc Dang.
You're welcome. (Unrelated) Note that I added a console= and earlycon in my test (and got the baud rate wrong for the console but nevermind...was ssh'd in after the earlycon output I cared about anyway) because of some other cleanup work for the SPCR parsing that apparently is still not quite fixed for upstream, or rather, there is a need to match on the 32-bit access required for the UART and that isn't happening so it's not getting setup. Folks are tracking that one and fixing it though.
On Thu, Dec 01, 2016 at 06:27:07PM -0800, Duc Dang wrote: > PCIe controllers in X-Gene SoCs is not ECAM compliant: software > needs to configure additional controller's register to address > device at bus:dev:function. > > The quirk will discover controller MMIO register space and configure > controller registers to select and address the target secondary device. > > The quirk will only be applied for X-Gene PCIe MCFG table with > OEM revison 1, 2, 3 or 4 (PCIe controller v1 and v2 on X-Gene SoCs). > > Signed-off-by: Duc Dang <dhdang@apm.com> Tested on our Linaro moonshot Tested-by: Graeme Gregory <graeme.gregory@linaro.org> Thanks > --- > v4: > - Rebase on top of pci/ecam tree > - Introduce xgene_get_csr_resource to discover MMIO register space > (per Mark's test code). > - Refactor .init functions for pci_ecam_ops to reuse common code > - Introduce pcie_bus_to_port to extract X-Gene 'port' information > from 'bus' device. > - Kconfig/Makefile changes to only compile required code for ACPI > v3: > - Rebase on top of pci/ecam-v6 tree. > - Use DEFINE_RES_MEM_NAMED to declare controller register space > with name "PCIe CSR" > v2: > RFC v2: https://patchwork.ozlabs.org/patch/686846/ > v1: > RFC v1: https://patchwork.kernel.org/patch/9337115/ > --- > drivers/acpi/pci_mcfg.c | 25 +++++++++ > drivers/pci/host/Kconfig | 4 +- > drivers/pci/host/Makefile | 2 +- > drivers/pci/host/pci-xgene.c | 127 ++++++++++++++++++++++++++++++++++++++++--- > include/linux/pci-ecam.h | 2 + > 5 files changed, 150 insertions(+), 10 deletions(-) > > diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c > index d34d196..7319188 100644 > --- a/drivers/acpi/pci_mcfg.c > +++ b/drivers/acpi/pci_mcfg.c > @@ -108,6 +108,31 @@ struct mcfg_fixup { > THUNDER_ECAM_QUIRK(2, 11), > THUNDER_ECAM_QUIRK(2, 12), > THUNDER_ECAM_QUIRK(2, 13), > + > +#define XGENE_V1_ECAM_MCFG(rev, seg) \ > + {"APM ", "XGENE ", rev, seg, MCFG_BUS_ANY, \ > + &xgene_v1_pcie_ecam_ops } > +#define XGENE_V2_ECAM_MCFG(rev, seg) \ > + {"APM ", "XGENE ", rev, seg, MCFG_BUS_ANY, \ > + &xgene_v2_pcie_ecam_ops } > + /* X-Gene SoC with v1 PCIe controller */ > + XGENE_V1_ECAM_MCFG(1, 0), > + XGENE_V1_ECAM_MCFG(1, 1), > + XGENE_V1_ECAM_MCFG(1, 2), > + XGENE_V1_ECAM_MCFG(1, 3), > + XGENE_V1_ECAM_MCFG(1, 4), > + XGENE_V1_ECAM_MCFG(2, 0), > + XGENE_V1_ECAM_MCFG(2, 1), > + XGENE_V1_ECAM_MCFG(2, 2), > + XGENE_V1_ECAM_MCFG(2, 3), > + XGENE_V1_ECAM_MCFG(2, 4), > + /* X-Gene SoC with v2.1 PCIe controller */ > + XGENE_V2_ECAM_MCFG(3, 0), > + XGENE_V2_ECAM_MCFG(3, 1), > + /* X-Gene SoC with v2.2 PCIe controller */ > + XGENE_V2_ECAM_MCFG(4, 0), > + XGENE_V2_ECAM_MCFG(4, 1), > + XGENE_V2_ECAM_MCFG(4, 2), > }; > > static char mcfg_oem_id[ACPI_OEM_ID_SIZE]; > diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig > index c983892..1fb5518 100644 > --- a/drivers/pci/host/Kconfig > +++ b/drivers/pci/host/Kconfig > @@ -133,8 +133,8 @@ config PCIE_XILINX > > config PCI_XGENE > bool "X-Gene PCIe controller" > - depends on ARCH_XGENE > - depends on OF > + depends on ARM64 > + depends on OF || (ACPI && PCI_QUIRKS) > select PCIEPORTBUS > help > Say Y here if you want internal PCI support on APM X-Gene SoC. > diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile > index 639494a..6cc84b4 100644 > --- a/drivers/pci/host/Makefile > +++ b/drivers/pci/host/Makefile > @@ -15,7 +15,7 @@ obj-$(CONFIG_PCIE_SPEAR13XX) += pcie-spear13xx.o > obj-$(CONFIG_PCI_KEYSTONE) += pci-keystone-dw.o pci-keystone.o > obj-$(CONFIG_PCIE_XILINX) += pcie-xilinx.o > obj-$(CONFIG_PCIE_XILINX_NWL) += pcie-xilinx-nwl.o > -obj-$(CONFIG_PCI_XGENE) += pci-xgene.o > +obj-$(CONFIG_ARM64) += pci-xgene.o > obj-$(CONFIG_PCI_XGENE_MSI) += pci-xgene-msi.o > obj-$(CONFIG_PCI_LAYERSCAPE) += pci-layerscape.o > obj-$(CONFIG_PCI_VERSATILE) += pci-versatile.o > diff --git a/drivers/pci/host/pci-xgene.c b/drivers/pci/host/pci-xgene.c > index 1de23d7..6edcac7 100644 > --- a/drivers/pci/host/pci-xgene.c > +++ b/drivers/pci/host/pci-xgene.c > @@ -27,6 +27,8 @@ > #include <linux/of_irq.h> > #include <linux/of_pci.h> > #include <linux/pci.h> > +#include <linux/pci-acpi.h> > +#include <linux/pci-ecam.h> > #include <linux/platform_device.h> > #include <linux/slab.h> > > @@ -64,7 +66,9 @@ > /* PCIe IP version */ > #define XGENE_PCIE_IP_VER_UNKN 0 > #define XGENE_PCIE_IP_VER_1 1 > +#define XGENE_PCIE_IP_VER_2 2 > > +#if defined(CONFIG_PCI_XGENE) || (defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)) > struct xgene_pcie_port { > struct device_node *node; > struct device *dev; > @@ -91,13 +95,24 @@ static inline u32 pcie_bar_low_val(u32 addr, u32 flags) > return (addr & PCI_BASE_ADDRESS_MEM_MASK) | flags; > } > > +static inline struct xgene_pcie_port *pcie_bus_to_port(struct pci_bus *bus) > +{ > + struct pci_config_window *cfg; > + > + if (acpi_disabled) > + return (struct xgene_pcie_port *)(bus->sysdata); > + > + cfg = bus->sysdata; > + return (struct xgene_pcie_port *)(cfg->priv); > +} > + > /* > * When the address bit [17:16] is 2'b01, the Configuration access will be > * treated as Type 1 and it will be forwarded to external PCIe device. > */ > static void __iomem *xgene_pcie_get_cfg_base(struct pci_bus *bus) > { > - struct xgene_pcie_port *port = bus->sysdata; > + struct xgene_pcie_port *port = pcie_bus_to_port(bus); > > if (bus->number >= (bus->primary + 1)) > return port->cfg_base + AXI_EP_CFG_ACCESS; > @@ -111,7 +126,7 @@ static void __iomem *xgene_pcie_get_cfg_base(struct pci_bus *bus) > */ > static void xgene_pcie_set_rtdid_reg(struct pci_bus *bus, uint devfn) > { > - struct xgene_pcie_port *port = bus->sysdata; > + struct xgene_pcie_port *port = pcie_bus_to_port(bus); > unsigned int b, d, f; > u32 rtdid_val = 0; > > @@ -158,7 +173,7 @@ static void __iomem *xgene_pcie_map_bus(struct pci_bus *bus, unsigned int devfn, > static int xgene_pcie_config_read32(struct pci_bus *bus, unsigned int devfn, > int where, int size, u32 *val) > { > - struct xgene_pcie_port *port = bus->sysdata; > + struct xgene_pcie_port *port = pcie_bus_to_port(bus); > > if (pci_generic_config_read32(bus, devfn, where & ~0x3, 4, val) != > PCIBIOS_SUCCESSFUL) > @@ -182,13 +197,104 @@ static int xgene_pcie_config_read32(struct pci_bus *bus, unsigned int devfn, > > return PCIBIOS_SUCCESSFUL; > } > +#endif > > -static struct pci_ops xgene_pcie_ops = { > - .map_bus = xgene_pcie_map_bus, > - .read = xgene_pcie_config_read32, > - .write = pci_generic_config_write32, > +#if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS) > +static int xgene_get_csr_resource(struct acpi_device *adev, > + struct resource *res) > +{ > + struct device *dev = &adev->dev; > + struct resource_entry *entry; > + struct list_head list; > + unsigned long flags; > + int ret; > + > + INIT_LIST_HEAD(&list); > + flags = IORESOURCE_MEM; > + ret = acpi_dev_get_resources(adev, &list, > + acpi_dev_filter_resource_type_cb, > + (void *) flags); > + if (ret < 0) { > + dev_err(dev, "failed to parse _CRS method, error code %d\n", > + ret); > + return ret; > + } > + > + if (ret == 0) { > + dev_err(dev, "no IO and memory resources present in _CRS\n"); > + return -EINVAL; > + } > + > + entry = list_first_entry(&list, struct resource_entry, node); > + *res = *entry->res; > + acpi_dev_free_resource_list(&list); > + return 0; > +} > + > +static int xgene_pcie_ecam_init(struct pci_config_window *cfg, u32 ipversion) > +{ > + struct acpi_device *adev = to_acpi_device(cfg->parent); > + struct device *dev = cfg->parent; > + struct xgene_pcie_port *port; > + struct resource csr; > + int ret; > + > + port = devm_kzalloc(dev, sizeof(*port), GFP_KERNEL); > + if (!port) > + return -ENOMEM; > + > + ret = xgene_get_csr_resource(adev, &csr); > + if (ret) { > + dev_err(dev, "can't get CSR resource\n"); > + kfree(port); > + return ret; > + } > + port->csr_base = devm_ioremap_resource(dev, &csr); > + if (IS_ERR(port->csr_base)) { > + kfree(port); > + return -ENOMEM; > + } > + > + port->cfg_base = cfg->win; > + port->version = ipversion; > + > + cfg->priv = port; > + > + return 0; > +} > + > +static int xgene_v1_pcie_ecam_init(struct pci_config_window *cfg) > +{ > + return xgene_pcie_ecam_init(cfg, XGENE_PCIE_IP_VER_1); > +} > + > +struct pci_ecam_ops xgene_v1_pcie_ecam_ops = { > + .bus_shift = 16, > + .init = xgene_v1_pcie_ecam_init, > + .pci_ops = { > + .map_bus = xgene_pcie_map_bus, > + .read = xgene_pcie_config_read32, > + .write = pci_generic_config_write, > + } > +}; > + > +static int xgene_v2_pcie_ecam_init(struct pci_config_window *cfg) > +{ > + return xgene_pcie_ecam_init(cfg, XGENE_PCIE_IP_VER_2); > +} > + > +struct pci_ecam_ops xgene_v2_pcie_ecam_ops = { > + .bus_shift = 16, > + .init = xgene_v2_pcie_ecam_init, > + .pci_ops = { > + .map_bus = xgene_pcie_map_bus, > + .read = xgene_pcie_config_read32, > + .write = pci_generic_config_write, > + } > }; > +#endif > > +#if defined(CONFIG_PCI_XGENE) > static u64 xgene_pcie_set_ib_mask(struct xgene_pcie_port *port, u32 addr, > u32 flags, u64 size) > { > @@ -521,6 +627,12 @@ static int xgene_pcie_setup(struct xgene_pcie_port *port, > return 0; > } > > +static struct pci_ops xgene_pcie_ops = { > + .map_bus = xgene_pcie_map_bus, > + .read = xgene_pcie_config_read32, > + .write = pci_generic_config_write32, > +}; > + > static int xgene_pcie_probe_bridge(struct platform_device *pdev) > { > struct device *dev = &pdev->dev; > @@ -591,3 +703,4 @@ static int xgene_pcie_probe_bridge(struct platform_device *pdev) > .probe = xgene_pcie_probe_bridge, > }; > builtin_platform_driver(xgene_pcie_driver); > +#endif > diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h > index 00eb8eb..f0d2b94 100644 > --- a/include/linux/pci-ecam.h > +++ b/include/linux/pci-ecam.h > @@ -64,6 +64,8 @@ void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn, > extern struct pci_ecam_ops hisi_pcie_ops; /* HiSilicon */ > extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 1.x & 2.x */ > extern struct pci_ecam_ops pci_thunder_ecam_ops; /* Cavium ThunderX 1.x */ > +extern struct pci_ecam_ops xgene_v1_pcie_ecam_ops; /* APM X-Gene PCIe v1 */ > +extern struct pci_ecam_ops xgene_v2_pcie_ecam_ops; /* APM X-Gene PCIe v2.x */ > #endif > > #ifdef CONFIG_PCI_HOST_GENERIC > -- > 1.9.1 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Fri, Dec 2, 2016 at 12:11 AM, Jon Masters <jcm@redhat.com> wrote: > You're welcome. > > (Unrelated) Note that I added a console= and earlycon in my test (and got the baud rate wrong for the console but nevermind...was ssh'd in after the earlycon output I cared about anyway) because of some other cleanup work for the SPCR parsing that apparently is still not quite fixed for upstream, or rather, there is a need to match on the 32-bit access required for the UART and that isn't happening so it's not getting setup. Folks are tracking that one and fixing it though. I don't see this console issue on X-Gene1 (Mustang board). I tried with X-Gene 2 as well. I used both console=ttyS0,115200 and earlycon=uart8250,mmio32,0x1c020000. Are you setting baudrate to 115200 or something else? > > -- > Computer Architect | Sent from my 64-bit #ARM Powered phone > >> On Dec 2, 2016, at 02:37, Duc Dang <dhdang@apm.com> wrote: >> >>> On Thu, Dec 1, 2016 at 11:12 PM, Jon Masters <jcm@redhat.com> wrote: >>>> On 12/01/2016 09:27 PM, Duc Dang wrote: >>>> PCIe controllers in X-Gene SoCs is not ECAM compliant: software >>>> needs to configure additional controller's register to address >>>> device at bus:dev:function. >>>> >>>> The quirk will discover controller MMIO register space and configure >>>> controller registers to select and address the target secondary device. >>>> >>>> The quirk will only be applied for X-Gene PCIe MCFG table with >>>> OEM revison 1, 2, 3 or 4 (PCIe controller v1 and v2 on X-Gene SoCs). >>>> >>>> Signed-off-by: Duc Dang <dhdang@apm.com> >>> >>> So far I've tested this on an HPE ProLiant m400 (Moonshot) cartridge >>> and will test it on some other reference platforms soon. Bootlog for >>> the m400 attached in case Bjorn wants to see the output. Here's >>> what I see in /proc/iomem btw on that platform: >>> >>> # cat /proc/iomem >>> 10520000-10523fff : APMC0D18:00 >>> 10520000-10523fff : APMC0D18:00 >>> 10524000-10527fff : APMC0D17:00 >>> 10540000-1054a0ff : APMC0D01:00 >>> 10546000-10546fff : APMC0D50:00 >>> 1054a000-1054a00f : APMC0D12:03 >>> 1054a000-1054a00f : APMC0D12:02 >>> 1054a000-1054a00f : APMC0D12:01 >>> 1054a000-1054a00f : APMC0D12:00 >>> 17000000-17000fff : APMC0D01:00 >>> 17001000-17001fff : APMC0D01:00 >>> 17001000-170013ff : APMC0D15:00 >>> 17001000-170013ff : APMC0D15:00 >>> 1701c000-1701cfff : APMC0D14:00 >>> 1a800000-1a800fff : APMC0D0D:00 >>> 1a800000-1a800fff : APMC0D0D:00 >>> 1c000200-1c0002ff : APMC0D06:00 >>> 1c021000-1c0210ff : APMC0D08:00 >>> 1c021000-1c02101f : serial >>> 1c024000-1c024fff : APMC0D07:00 >>> 1f230000-1f230fff : APMC0D0D:00 >>> 1f230000-1f230fff : APMC0D0D:00 >>> 1f23d000-1f23dfff : APMC0D0D:00 >>> 1f23d000-1f23dfff : APMC0D0D:00 >>> 1f23e000-1f23efff : APMC0D0D:00 >>> 1f23e000-1f23efff : APMC0D0D:00 >>> 1f2a0000-1f31ffff : APMC0D06:00 >>> 1f500000-1f50ffff : PCI Bus 0000:00 >>> 1f500000-1f50ffff : PNP0A08:00 >>> 78800000-78800fff : APMC0D13:00 >>> 78800000-78800fff : APMC0D12:03 >>> 78800000-78800fff : APMC0D12:02 >>> 78800000-78800fff : APMC0D12:01 >>> 78800000-78800fff : APMC0D12:00 >>> 78800000-78800fff : APMC0D11:00 >>> 78800000-78800fff : APMC0D10:03 >>> 78800000-78800fff : APMC0D10:02 >>> 78800000-78800fff : APMC0D10:01 >>> 78800000-78800fff : APMC0D10:00 >>> 79000000-798fffff : APMC0D0E:00 >>> 7c000000-7c1fffff : APMC0D12:00 >>> 7c200000-7c3fffff : APMC0D12:01 >>> 7c400000-7c5fffff : APMC0D12:02 >>> 7c600000-7c7fffff : APMC0D12:03 >>> 7e000000-7e000fff : APMC0D13:00 >>> 7e200000-7e200fff : APMC0D10:03 >>> 7e200000-7e200fff : APMC0D10:02 >>> 7e200000-7e200fff : APMC0D10:01 >>> 7e200000-7e200fff : APMC0D10:00 >>> 7e600000-7e600fff : APMC0D11:00 >>> 7e700000-7e700fff : APMC0D10:03 >>> 7e700000-7e700fff : APMC0D10:02 >>> 7e700000-7e700fff : APMC0D10:01 >>> 7e700000-7e700fff : APMC0D10:00 >>> 7e720000-7e720fff : APMC0D10:03 >>> 7e720000-7e720fff : APMC0D10:02 >>> 7e720000-7e720fff : APMC0D10:01 >>> 7e720000-7e720fff : APMC0D10:00 >>> 7e800000-7e800fff : APMC0D10:00 >>> 7e840000-7e840fff : APMC0D10:01 >>> 7e880000-7e880fff : APMC0D10:02 >>> 7e8c0000-7e8c0fff : APMC0D10:03 >>> 7e930000-7e930fff : APMC0D13:00 >>> 4000000000-4001ffffff : System RAM >>> 4000080000-4000c3ffff : Kernel code >>> 4000db0000-400165ffff : Kernel data >>> 40023a0000-4ff733ffff : System RAM >>> 4ff7340000-4ff77cffff : reserved >>> 4ff77d0000-4ff79cffff : System RAM >>> 4ff79d0000-4ff7e7ffff : reserved >>> 4ff7e80000-4ff7e8ffff : System RAM >>> 4ff7e90000-4ff7efffff : reserved >>> 4ff7f10000-4ff800ffff : reserved >>> 4ff8010000-4fffffffff : System RAM >>> a020000000-a03fffffff : PCI Bus 0000:00 >>> a020000000-a0201fffff : PCI Bus 0000:01 >>> a020000000-a0200fffff : 0000:01:00.0 >>> a020000000-a0200fffff : mlx4_core >>> a020100000-a0201fffff : 0000:01:00.0 >>> a060000000-a07fffffff : PCI Bus 0000:00 >>> a0d0000000-a0dfffffff : PCI ECAM >>> a110000000-a14fffffff : PCI Bus 0000:00 >>> a110000000-a121ffffff : PCI Bus 0000:01 >>> a110000000-a111ffffff : 0000:01:00.0 >>> a110000000-a111ffffff : mlx4_core >>> a112000000-a121ffffff : 0000:01:00.0 >>> >>> Adding a Tested-by for the record: >>> >>> Tested-by: Jon Masters <jcm@redhat.com> >> >> Thanks a lot for testing this, Jon. >>> >>> Jon. >>> >>> -- >>> Computer Architect | Sent from my Fedora powered laptop >>> >> Regards, >> Duc Dang.
On 12/02/2016 02:39 PM, Duc Dang wrote: > On Fri, Dec 2, 2016 at 12:11 AM, Jon Masters <jcm@redhat.com> wrote: >> You're welcome. >> >> (Unrelated) Note that I added a console= and earlycon in my test (and got the baud rate wrong for the console but nevermind...was ssh'd in after the earlycon output I cared about anyway) because of some other cleanup work for the SPCR parsing that apparently is still not quite fixed for upstream, or rather, there is a need to match on the 32-bit access required for the UART and that isn't happening so it's not getting setup. Folks are tracking that one and fixing it though. > > I don't see this console issue on X-Gene1 (Mustang board). I tried > with X-Gene 2 as well. I used both console=ttyS0,115200 and > earlycon=uart8250,mmio32,0x1c020000. Are you setting baudrate to > 115200 or something else? It's an m400 issue in that their SPCR needs updating to convey the required 32-bit access width for the 8250 dw IP or similar. It's one of those things someone described the other day and it made sense but I haven't yet dug into the exact situation, other than that I know the access width in the m400 ACPI table isn't quite right. My understanding is that it's in hand. I'll catch up on exactly what's up, check with my own mustangs, and followup separately. Jon.
On Sat, Dec 03, 2016 at 05:06:31AM -0500, Jon Masters wrote: > Hi Duc, all, > > (and changing the subject and trimming/adjusting the CC) > > On 12/02/2016 02:39 PM, Duc Dang wrote: > > On Fri, Dec 2, 2016 at 12:11 AM, Jon Masters <jcm@redhat.com> wrote: > >> You're welcome. > >> > >> (Unrelated) Note that I added a console= and earlycon in my test (and > >> got the baud rate wrong for the console but nevermind...was ssh'd in > >> after the earlycon output I cared about anyway) because of some other > >> cleanup work for the SPCR parsing that apparently is still not quite > >> fixed for upstream, or rather, there is a need to match on the 32-bit > >> access required for the UART and that isn't happening so it's not > >> getting setup. Folks are tracking that one and fixing it though. > > > I don't see this console issue on X-Gene1 (Mustang board). I tried > > with X-Gene 2 as well. I used both console=ttyS0,115200 and > > earlycon=uart8250,mmio32,0x1c020000. Are you setting baudrate to > > 115200 or something else? > > Let me clarify. What I meant above is that when I generated the boot > log, I had specified earlycon and console parameters, but had fat > fingered the baud rate (m400 uses 9600, mustang uses 115200 baud). > That's what I was referring to in the original text above. > > HOWEVER... > > There is a broader problem with X-Gene SPCR support. The problem is > that the 16550 UART in X-Gene requires the 32-bit access quirk (the > iotype is set to UPIO_MEM32 for the APMC0D08 device). This means > that when univ8250_console_match runs later, it will compare the > iotype (MEM32) with the type previously registered with the > kernel when the earlycon setup the preferred console. > > The earlycon preferred console will parse the SPCR and find: > > iotype = table->serial_port.space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY ? > "mmio" : "io"; > > Which sets it to "mmio" (not "mmio32"). There is a DBG2 (the table > referenced by the SPCR that provides the actual port types for all > modern revisions of the SPCR with revision2+, as required by Linux) > port subtype for non-compliant SBSA ARM Generic UARTs that require > 32-bit accesses, and this is ACPI_DBG2_ARM_SBSA_32BIT (type 0xd). > > However that only applies for "pl011" devices, and doesn't provide > for 16550 UARTs that require 32-bit accessors. > > So you'll end up with Linux thinking it's registering a non-32-bit > mmio preferred console during earlycon setup and then never match > against that later in the 8250_core/8250_dw match function by > virtue of the fact that these differ. > > There are a couple of possibilities: > > 1). Perhaps (for some reason) the IP actually does support sub-32-bit > access and the iotype simply needs to be changed to reflect this. > That would be the easiest option. But it's been this way for a > long time in various codebases, so I would be pleasantly > surprised if this were the case. Let me/us know :) > > 2). We find some way during SPCR setup to quirk for X-Gene as a non > standard 16550 and set it up as an mmio32 iotype. > Aleksey is going to send a patch for review that uses the register width field in the Generic Address Structure to select mmio vs mmio32 for the 8250 UARTS. The registers in xgene are 8 bits wide with a padding of 24 dont care bits, they do allow AFAIK 8 bit address, but they are at 32bit spacing unlike real 16550 at 8 bit spacing. > 3). We get the DBG2 table updated to add a subtype of the 16650 > called something like "(deprecated) 16550 subset supporting > only 32-bit accesses". Then we add this to Linux and get > the firmware updated on systems to switch to this type. We > would probably still want to quirk for existing machines. > > Perhaps I'm missing something. I would love for that to be true, > but I don't think it is. I think we need a subtype of the 16550 > defined that encapsulates this mmio32 requirement properly. To > that end, I've preemptively asked some friends at MS for a > favor to look into adding a new subtype for this. > > Let me know what you think is the best path... :) > I think the new subtype is the way to go as we can also use this to detect that we have no knowledge of the clocks set by firmware so we should not attempt a baud rate change like we can on 16550. Thanks Graeme
On Sat, 2016-12-03 at 05:06 -0500, Jon Masters wrote: > Hi Duc, all, > > (and changing the subject and trimming/adjusting the CC) > > On 12/02/2016 02:39 PM, Duc Dang wrote: > > > > On Fri, Dec 2, 2016 at 12:11 AM, Jon Masters <jcm@redhat.com> wrote: > > > > > > You're welcome. > > > > > > (Unrelated) Note that I added a console= and earlycon in my test (and > > > got the baud rate wrong for the console but nevermind...was ssh'd in > > > after the earlycon output I cared about anyway) because of some other > > > cleanup work for the SPCR parsing that apparently is still not quite > > > fixed for upstream, or rather, there is a need to match on the 32-bit > > > access required for the UART and that isn't happening so it's not > > > getting setup. Folks are tracking that one and fixing it though. > > > > I don't see this console issue on X-Gene1 (Mustang board). I tried > > with X-Gene 2 as well. I used both console=ttyS0,115200 and > > earlycon=uart8250,mmio32,0x1c020000. Are you setting baudrate to > > 115200 or something else? > Let me clarify. What I meant above is that when I generated the boot > log, I had specified earlycon and console parameters, but had fat > fingered the baud rate (m400 uses 9600, mustang uses 115200 baud). > That's what I was referring to in the original text above. > > HOWEVER... > > There is a broader problem with X-Gene SPCR support. The problem is > that the 16550 UART in X-Gene requires the 32-bit access quirk (the > iotype is set to UPIO_MEM32 for the APMC0D08 device). This means > that when univ8250_console_match runs later, it will compare the > iotype (MEM32) with the type previously registered with the > kernel when the earlycon setup the preferred console. Linaro has a kernel patch which looks at the bit_width field of the port address: Author: Aleksey Makarov <aleksey.makarov@linaro.org> Date: Thu Apr 28 19:52:38 2016 +0300 serial: SPCR: check bit width for the 16550 UART The SPCR in 3.06.25 firmware has a bit_width field set to 32 and with the above patch, I don't need to use console=. But HP firmware on m400 sets bit width to 8 so that needs a firmware fix to work with above. > > The earlycon preferred console will parse the SPCR and find: > > iotype = table->serial_port.space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY ? > "mmio" : "io"; > > Which sets it to "mmio" (not "mmio32"). There is a DBG2 (the table > referenced by the SPCR that provides the actual port types for all > modern revisions of the SPCR with revision2+, as required by Linux) > port subtype for non-compliant SBSA ARM Generic UARTs that require > 32-bit accesses, and this is ACPI_DBG2_ARM_SBSA_32BIT (type 0xd). > > However that only applies for "pl011" devices, and doesn't provide > for 16550 UARTs that require 32-bit accessors. > > So you'll end up with Linux thinking it's registering a non-32-bit > mmio preferred console during earlycon setup and then never match > against that later in the 8250_core/8250_dw match function by > virtue of the fact that these differ. > > There are a couple of possibilities: > > 1). Perhaps (for some reason) the IP actually does support sub-32-bit > access and the iotype simply needs to be changed to reflect this. > That would be the easiest option. But it's been this way for a > long time in various codebases, so I would be pleasantly > surprised if this were the case. Let me/us know :) > > 2). We find some way during SPCR setup to quirk for X-Gene as a non > standard 16550 and set it up as an mmio32 iotype. > > 3). We get the DBG2 table updated to add a subtype of the 16650 > called something like "(deprecated) 16550 subset supporting > only 32-bit accesses". Then we add this to Linux and get > the firmware updated on systems to switch to this type. We > would probably still want to quirk for existing machines. > > Perhaps I'm missing something. I would love for that to be true, > but I don't think it is. I think we need a subtype of the 16550 > defined that encapsulates this mmio32 requirement properly. To > that end, I've preemptively asked some friends at MS for a > favor to look into adding a new subtype for this. > > Let me know what you think is the best path... :) > > Thanks, > > Jon. >
Hi Mark, On 12/03/2016 12:15 PM, Mark Salter wrote: > On Sat, 2016-12-03 at 05:06 -0500, Jon Masters wrote: >> There is a broader problem with X-Gene SPCR support. The problem is >> that the 16550 UART in X-Gene requires the 32-bit access quirk (the >> iotype is set to UPIO_MEM32 for the APMC0D08 device). This means >> that when univ8250_console_match runs later, it will compare the >> iotype (MEM32) with the type previously registered with the >> kernel when the earlycon setup the preferred console. > > Linaro has a kernel patch which looks at the bit_width field of the > port address: > > Author: Aleksey Makarov <aleksey.makarov@linaro.org> > Date: Thu Apr 28 19:52:38 2016 +0300 > > serial: SPCR: check bit width for the 16550 UART > > The SPCR in 3.06.25 firmware has a bit_width field set to 32 and with the > above patch, I don't need to use console=. But HP firmware on m400 sets > bit width to 8 so that needs a firmware fix to work with above. Indeed, thanks. Graeme also mentioned that last night. I think this a good solution for the moment, but still that it makes sense to have a new 16650 UART type defined in the DBG2 spec for those requiring 32-bit access (to mirror the type D for pl011). I heard back from Microsoft this morning that they're looking. Jon.
On Sat, Dec 3, 2016 at 12:33 PM, Jon Masters <jcm@redhat.com> wrote: > Hi Mark, > > On 12/03/2016 12:15 PM, Mark Salter wrote: >> On Sat, 2016-12-03 at 05:06 -0500, Jon Masters wrote: > >>> There is a broader problem with X-Gene SPCR support. The problem is >>> that the 16550 UART in X-Gene requires the 32-bit access quirk (the >>> iotype is set to UPIO_MEM32 for the APMC0D08 device). This means >>> that when univ8250_console_match runs later, it will compare the >>> iotype (MEM32) with the type previously registered with the >>> kernel when the earlycon setup the preferred console. >> >> Linaro has a kernel patch which looks at the bit_width field of the >> port address: >> >> Author: Aleksey Makarov <aleksey.makarov@linaro.org> >> Date: Thu Apr 28 19:52:38 2016 +0300 >> >> serial: SPCR: check bit width for the 16550 UART >> >> The SPCR in 3.06.25 firmware has a bit_width field set to 32 and with the >> above patch, I don't need to use console=. But HP firmware on m400 sets >> bit width to 8 so that needs a firmware fix to work with above. Will this patch be posted for upstream? > > Indeed, thanks. Graeme also mentioned that last night. I think this > a good solution for the moment, but still that it makes sense to > have a new 16650 UART type defined in the DBG2 spec for those > requiring 32-bit access (to mirror the type D for pl011). I > heard back from Microsoft this morning that they're looking. Yes, thanks, Jon. It will be nice to have a new 16550 UART type for those requiring 32-bit access. > > Jon. > > -- > Computer Architect | Sent from my Fedora powered laptop > Regards, Duc Dang.
diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c index d34d196..7319188 100644 --- a/drivers/acpi/pci_mcfg.c +++ b/drivers/acpi/pci_mcfg.c @@ -108,6 +108,31 @@ struct mcfg_fixup { THUNDER_ECAM_QUIRK(2, 11), THUNDER_ECAM_QUIRK(2, 12), THUNDER_ECAM_QUIRK(2, 13), + +#define XGENE_V1_ECAM_MCFG(rev, seg) \ + {"APM ", "XGENE ", rev, seg, MCFG_BUS_ANY, \ + &xgene_v1_pcie_ecam_ops } +#define XGENE_V2_ECAM_MCFG(rev, seg) \ + {"APM ", "XGENE ", rev, seg, MCFG_BUS_ANY, \ + &xgene_v2_pcie_ecam_ops } + /* X-Gene SoC with v1 PCIe controller */ + XGENE_V1_ECAM_MCFG(1, 0), + XGENE_V1_ECAM_MCFG(1, 1), + XGENE_V1_ECAM_MCFG(1, 2), + XGENE_V1_ECAM_MCFG(1, 3), + XGENE_V1_ECAM_MCFG(1, 4), + XGENE_V1_ECAM_MCFG(2, 0), + XGENE_V1_ECAM_MCFG(2, 1), + XGENE_V1_ECAM_MCFG(2, 2), + XGENE_V1_ECAM_MCFG(2, 3), + XGENE_V1_ECAM_MCFG(2, 4), + /* X-Gene SoC with v2.1 PCIe controller */ + XGENE_V2_ECAM_MCFG(3, 0), + XGENE_V2_ECAM_MCFG(3, 1), + /* X-Gene SoC with v2.2 PCIe controller */ + XGENE_V2_ECAM_MCFG(4, 0), + XGENE_V2_ECAM_MCFG(4, 1), + XGENE_V2_ECAM_MCFG(4, 2), }; static char mcfg_oem_id[ACPI_OEM_ID_SIZE]; diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig index c983892..1fb5518 100644 --- a/drivers/pci/host/Kconfig +++ b/drivers/pci/host/Kconfig @@ -133,8 +133,8 @@ config PCIE_XILINX config PCI_XGENE bool "X-Gene PCIe controller" - depends on ARCH_XGENE - depends on OF + depends on ARM64 + depends on OF || (ACPI && PCI_QUIRKS) select PCIEPORTBUS help Say Y here if you want internal PCI support on APM X-Gene SoC. diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile index 639494a..6cc84b4 100644 --- a/drivers/pci/host/Makefile +++ b/drivers/pci/host/Makefile @@ -15,7 +15,7 @@ obj-$(CONFIG_PCIE_SPEAR13XX) += pcie-spear13xx.o obj-$(CONFIG_PCI_KEYSTONE) += pci-keystone-dw.o pci-keystone.o obj-$(CONFIG_PCIE_XILINX) += pcie-xilinx.o obj-$(CONFIG_PCIE_XILINX_NWL) += pcie-xilinx-nwl.o -obj-$(CONFIG_PCI_XGENE) += pci-xgene.o +obj-$(CONFIG_ARM64) += pci-xgene.o obj-$(CONFIG_PCI_XGENE_MSI) += pci-xgene-msi.o obj-$(CONFIG_PCI_LAYERSCAPE) += pci-layerscape.o obj-$(CONFIG_PCI_VERSATILE) += pci-versatile.o diff --git a/drivers/pci/host/pci-xgene.c b/drivers/pci/host/pci-xgene.c index 1de23d7..6edcac7 100644 --- a/drivers/pci/host/pci-xgene.c +++ b/drivers/pci/host/pci-xgene.c @@ -27,6 +27,8 @@ #include <linux/of_irq.h> #include <linux/of_pci.h> #include <linux/pci.h> +#include <linux/pci-acpi.h> +#include <linux/pci-ecam.h> #include <linux/platform_device.h> #include <linux/slab.h> @@ -64,7 +66,9 @@ /* PCIe IP version */ #define XGENE_PCIE_IP_VER_UNKN 0 #define XGENE_PCIE_IP_VER_1 1 +#define XGENE_PCIE_IP_VER_2 2 +#if defined(CONFIG_PCI_XGENE) || (defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS)) struct xgene_pcie_port { struct device_node *node; struct device *dev; @@ -91,13 +95,24 @@ static inline u32 pcie_bar_low_val(u32 addr, u32 flags) return (addr & PCI_BASE_ADDRESS_MEM_MASK) | flags; } +static inline struct xgene_pcie_port *pcie_bus_to_port(struct pci_bus *bus) +{ + struct pci_config_window *cfg; + + if (acpi_disabled) + return (struct xgene_pcie_port *)(bus->sysdata); + + cfg = bus->sysdata; + return (struct xgene_pcie_port *)(cfg->priv); +} + /* * When the address bit [17:16] is 2'b01, the Configuration access will be * treated as Type 1 and it will be forwarded to external PCIe device. */ static void __iomem *xgene_pcie_get_cfg_base(struct pci_bus *bus) { - struct xgene_pcie_port *port = bus->sysdata; + struct xgene_pcie_port *port = pcie_bus_to_port(bus); if (bus->number >= (bus->primary + 1)) return port->cfg_base + AXI_EP_CFG_ACCESS; @@ -111,7 +126,7 @@ static void __iomem *xgene_pcie_get_cfg_base(struct pci_bus *bus) */ static void xgene_pcie_set_rtdid_reg(struct pci_bus *bus, uint devfn) { - struct xgene_pcie_port *port = bus->sysdata; + struct xgene_pcie_port *port = pcie_bus_to_port(bus); unsigned int b, d, f; u32 rtdid_val = 0; @@ -158,7 +173,7 @@ static void __iomem *xgene_pcie_map_bus(struct pci_bus *bus, unsigned int devfn, static int xgene_pcie_config_read32(struct pci_bus *bus, unsigned int devfn, int where, int size, u32 *val) { - struct xgene_pcie_port *port = bus->sysdata; + struct xgene_pcie_port *port = pcie_bus_to_port(bus); if (pci_generic_config_read32(bus, devfn, where & ~0x3, 4, val) != PCIBIOS_SUCCESSFUL) @@ -182,13 +197,104 @@ static int xgene_pcie_config_read32(struct pci_bus *bus, unsigned int devfn, return PCIBIOS_SUCCESSFUL; } +#endif -static struct pci_ops xgene_pcie_ops = { - .map_bus = xgene_pcie_map_bus, - .read = xgene_pcie_config_read32, - .write = pci_generic_config_write32, +#if defined(CONFIG_ACPI) && defined(CONFIG_PCI_QUIRKS) +static int xgene_get_csr_resource(struct acpi_device *adev, + struct resource *res) +{ + struct device *dev = &adev->dev; + struct resource_entry *entry; + struct list_head list; + unsigned long flags; + int ret; + + INIT_LIST_HEAD(&list); + flags = IORESOURCE_MEM; + ret = acpi_dev_get_resources(adev, &list, + acpi_dev_filter_resource_type_cb, + (void *) flags); + if (ret < 0) { + dev_err(dev, "failed to parse _CRS method, error code %d\n", + ret); + return ret; + } + + if (ret == 0) { + dev_err(dev, "no IO and memory resources present in _CRS\n"); + return -EINVAL; + } + + entry = list_first_entry(&list, struct resource_entry, node); + *res = *entry->res; + acpi_dev_free_resource_list(&list); + return 0; +} + +static int xgene_pcie_ecam_init(struct pci_config_window *cfg, u32 ipversion) +{ + struct acpi_device *adev = to_acpi_device(cfg->parent); + struct device *dev = cfg->parent; + struct xgene_pcie_port *port; + struct resource csr; + int ret; + + port = devm_kzalloc(dev, sizeof(*port), GFP_KERNEL); + if (!port) + return -ENOMEM; + + ret = xgene_get_csr_resource(adev, &csr); + if (ret) { + dev_err(dev, "can't get CSR resource\n"); + kfree(port); + return ret; + } + port->csr_base = devm_ioremap_resource(dev, &csr); + if (IS_ERR(port->csr_base)) { + kfree(port); + return -ENOMEM; + } + + port->cfg_base = cfg->win; + port->version = ipversion; + + cfg->priv = port; + + return 0; +} + +static int xgene_v1_pcie_ecam_init(struct pci_config_window *cfg) +{ + return xgene_pcie_ecam_init(cfg, XGENE_PCIE_IP_VER_1); +} + +struct pci_ecam_ops xgene_v1_pcie_ecam_ops = { + .bus_shift = 16, + .init = xgene_v1_pcie_ecam_init, + .pci_ops = { + .map_bus = xgene_pcie_map_bus, + .read = xgene_pcie_config_read32, + .write = pci_generic_config_write, + } +}; + +static int xgene_v2_pcie_ecam_init(struct pci_config_window *cfg) +{ + return xgene_pcie_ecam_init(cfg, XGENE_PCIE_IP_VER_2); +} + +struct pci_ecam_ops xgene_v2_pcie_ecam_ops = { + .bus_shift = 16, + .init = xgene_v2_pcie_ecam_init, + .pci_ops = { + .map_bus = xgene_pcie_map_bus, + .read = xgene_pcie_config_read32, + .write = pci_generic_config_write, + } }; +#endif +#if defined(CONFIG_PCI_XGENE) static u64 xgene_pcie_set_ib_mask(struct xgene_pcie_port *port, u32 addr, u32 flags, u64 size) { @@ -521,6 +627,12 @@ static int xgene_pcie_setup(struct xgene_pcie_port *port, return 0; } +static struct pci_ops xgene_pcie_ops = { + .map_bus = xgene_pcie_map_bus, + .read = xgene_pcie_config_read32, + .write = pci_generic_config_write32, +}; + static int xgene_pcie_probe_bridge(struct platform_device *pdev) { struct device *dev = &pdev->dev; @@ -591,3 +703,4 @@ static int xgene_pcie_probe_bridge(struct platform_device *pdev) .probe = xgene_pcie_probe_bridge, }; builtin_platform_driver(xgene_pcie_driver); +#endif diff --git a/include/linux/pci-ecam.h b/include/linux/pci-ecam.h index 00eb8eb..f0d2b94 100644 --- a/include/linux/pci-ecam.h +++ b/include/linux/pci-ecam.h @@ -64,6 +64,8 @@ void __iomem *pci_ecam_map_bus(struct pci_bus *bus, unsigned int devfn, extern struct pci_ecam_ops hisi_pcie_ops; /* HiSilicon */ extern struct pci_ecam_ops thunder_pem_ecam_ops; /* Cavium ThunderX 1.x & 2.x */ extern struct pci_ecam_ops pci_thunder_ecam_ops; /* Cavium ThunderX 1.x */ +extern struct pci_ecam_ops xgene_v1_pcie_ecam_ops; /* APM X-Gene PCIe v1 */ +extern struct pci_ecam_ops xgene_v2_pcie_ecam_ops; /* APM X-Gene PCIe v2.x */ #endif #ifdef CONFIG_PCI_HOST_GENERIC
PCIe controllers in X-Gene SoCs is not ECAM compliant: software needs to configure additional controller's register to address device at bus:dev:function. The quirk will discover controller MMIO register space and configure controller registers to select and address the target secondary device. The quirk will only be applied for X-Gene PCIe MCFG table with OEM revison 1, 2, 3 or 4 (PCIe controller v1 and v2 on X-Gene SoCs). Signed-off-by: Duc Dang <dhdang@apm.com> --- v4: - Rebase on top of pci/ecam tree - Introduce xgene_get_csr_resource to discover MMIO register space (per Mark's test code). - Refactor .init functions for pci_ecam_ops to reuse common code - Introduce pcie_bus_to_port to extract X-Gene 'port' information from 'bus' device. - Kconfig/Makefile changes to only compile required code for ACPI v3: - Rebase on top of pci/ecam-v6 tree. - Use DEFINE_RES_MEM_NAMED to declare controller register space with name "PCIe CSR" v2: RFC v2: https://patchwork.ozlabs.org/patch/686846/ v1: RFC v1: https://patchwork.kernel.org/patch/9337115/ --- drivers/acpi/pci_mcfg.c | 25 +++++++++ drivers/pci/host/Kconfig | 4 +- drivers/pci/host/Makefile | 2 +- drivers/pci/host/pci-xgene.c | 127 ++++++++++++++++++++++++++++++++++++++++--- include/linux/pci-ecam.h | 2 + 5 files changed, 150 insertions(+), 10 deletions(-)