| Message ID | 1359399397-29729-8-git-send-email-thomas.petazzoni@free-electrons.com (mailing list archive) |
|---|---|
| State | New, archived |
| Delegated to: | Bjorn Helgaas |
| Headers | show |
On Monday 28 January 2013, Thomas Petazzoni wrote: > From: Thierry Reding <thierry.reding@avionic-design.de> > > [Thomas Petazzoni: > - Simplify capabilities handling. > - Move to a separate file. > - Fix mask used when writing a 4 bytes value.] > > Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de> > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> Not even a description why this is needed? This patch (together with patch 8) seems like the most controversial one of the series, so you should better provide a really good reason why we would emulate something in software rather than using whatever hardware is there. Arnd -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On 01/28/2013 01:18 PM, Arnd Bergmann wrote: > On Monday 28 January 2013, Thomas Petazzoni wrote: >> From: Thierry Reding <thierry.reding@avionic-design.de> >> >> [Thomas Petazzoni: >> - Simplify capabilities handling. >> - Move to a separate file. >> - Fix mask used when writing a 4 bytes value.] >> >> Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de> >> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> > > Not even a description why this is needed? > > This patch (together with patch 8) seems like the most controversial > one of the series, so you should better provide a really good reason > why we would emulate something in software rather than using whatever > hardware is there. At least on Tegra, there is no HW that exposes PCI configuration registers for the host bridge itself. Only the root ports have exposed PCI configuration registers. There was some debate re: whether a host bridge device needed to exist or not. This patch makes such a device exist if it's required. -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Jan 28, 2013 at 03:03:55PM -0700, Stephen Warren wrote: > On 01/28/2013 01:18 PM, Arnd Bergmann wrote: > > On Monday 28 January 2013, Thomas Petazzoni wrote: > >> From: Thierry Reding <thierry.reding@avionic-design.de> > >> > >> [Thomas Petazzoni: > >> - Simplify capabilities handling. > >> - Move to a separate file. > >> - Fix mask used when writing a 4 bytes value.] > >> > >> Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de> > >> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> > > > > Not even a description why this is needed? > > > > This patch (together with patch 8) seems like the most controversial > > one of the series, so you should better provide a really good reason > > why we would emulate something in software rather than using whatever > > hardware is there. > > At least on Tegra, there is no HW that exposes PCI configuration > registers for the host bridge itself. Only the root ports have exposed > PCI configuration registers. There was some debate re: whether a host > bridge device needed to exist or not. This patch makes such a device > exist if it's required. If Linux will discover properly (I strongly suspect it does) without the host bridge, then I would say to ditch this... The PCI-E standard requires a host bridge device, but if Linux doesn't require it then there is no reason to emulate one. That would simplify the question of PCI IDs - for Marvell's case and the sw root port bridge we can just copy the IDs from the bogus config space of the HW. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Dear Arnd Bergmann, On Mon, 28 Jan 2013 20:18:17 +0000, Arnd Bergmann wrote: > Not even a description why this is needed? > > This patch (together with patch 8) seems like the most controversial > one of the series, so you should better provide a really good reason > why we would emulate something in software rather than using whatever > hardware is there. Hum, you're right. In fact, the very reason why I'm adding an emulated host bridge and emulated PCI-to-PCI bridges is simply because this was one of the main suggestion raised during the review of the first revision of this patch set. Thomas
Dear Jason Gunthorpe, On Mon, 28 Jan 2013 15:09:04 -0700, Jason Gunthorpe wrote: > If Linux will discover properly (I strongly suspect it does) without > the host bridge, then I would say to ditch this... > > The PCI-E standard requires a host bridge device, but if Linux doesn't > require it then there is no reason to emulate one. > > That would simplify the question of PCI IDs - for Marvell's case and > the sw root port bridge we can just copy the IDs from the bogus config > space of the HW. Not sure what you mean in this last paragraph. In this second version, I really rely on the emulated PCI-to-PCI bridges for the resource allocation. I give the Linux PCI core a global range of addresses for memory regions and a global range of addresses for I/O regions, and then I let Linux do the allocation of ranges on a per bridge basis, depending on the devices detected downstream. And at the end, I use those allocated ranges to set up the address decoding windows. This all comes from your suggestions during the review of the first revision of this patch set. Thomas
On Mon, Jan 28, 2013 at 11:18:29PM +0100, Thomas Petazzoni wrote: > > That would simplify the question of PCI IDs - for Marvell's case and > > the sw root port bridge we can just copy the IDs from the bogus config > > space of the HW. > > Not sure what you mean in this last paragraph. In this second version, > I really rely on the emulated PCI-to-PCI bridges for the resource I'm refering to your earlier question about what PCI IDs to use for the SW emulated devices. If there is no need for the host bridge then you only need 1 PCI ID (for the root port bridge) and you can probably fairly safely re-use the one in the Marvell config space of the HW. > allocation. I give the Linux PCI core a global range of addresses for > memory regions and a global range of addresses for I/O regions, and > then I let Linux do the allocation of ranges on a per bridge basis, > depending on the devices detected downstream. And at the end, I use > those allocated ranges to set up the address decoding windows. Yes, that all seems OK to me. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Dear Jason Gunthorpe, On Mon, 28 Jan 2013 15:23:48 -0700, Jason Gunthorpe wrote: > I'm refering to your earlier question about what PCI IDs to use for > the SW emulated devices. If there is no need for the host bridge then > you only need 1 PCI ID (for the root port bridge) and you can probably > fairly safely re-use the one in the Marvell config space of the HW. Ah, ok, I see. But isn't a host bridge needed to bind all the PCI-to-PCI bridges under a single bus, in order to get the global resource assignment I was referring to? Regarding the PCI IDs, I have started to work with Marvell to see what is possible. I, unfortunately, haven't received the answer for now. Best regards, Thomas
On Mon, Jan 28, 2013 at 11:30:48PM +0100, Thomas Petazzoni wrote: > Dear Jason Gunthorpe, > > On Mon, 28 Jan 2013 15:23:48 -0700, Jason Gunthorpe wrote: > > > I'm refering to your earlier question about what PCI IDs to use for > > the SW emulated devices. If there is no need for the host bridge then > > you only need 1 PCI ID (for the root port bridge) and you can probably > > fairly safely re-use the one in the Marvell config space of the HW. > > Ah, ok, I see. But isn't a host bridge needed to bind all the > PCI-to-PCI bridges under a single bus, in order to get the global > resource assignment I was referring to? The PCI-E spec requires it, but AFAIK it doesn't actually *do* anything on Linux, and Linux doesn't require it. I thought Thierry did this experiment and decided it wasn't necessary: > The reason is that with the latest bindings the matching of root > ports to device tree nodes works as-is and nothing else indicates > that the emulated host bridge is actually required to make any of > this work. So in order not to introduce unneeded code I've left it > out for now. If somebody decides that we actually need this host > bridge (for standards compliance or whatnot) it could easily be > added back. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Jan 28, 2013 at 3:09 PM, Jason Gunthorpe <jgunthorpe@obsidianresearch.com> wrote: > On Mon, Jan 28, 2013 at 03:03:55PM -0700, Stephen Warren wrote: >> On 01/28/2013 01:18 PM, Arnd Bergmann wrote: >> > On Monday 28 January 2013, Thomas Petazzoni wrote: >> >> From: Thierry Reding <thierry.reding@avionic-design.de> >> >> >> >> [Thomas Petazzoni: >> >> - Simplify capabilities handling. >> >> - Move to a separate file. >> >> - Fix mask used when writing a 4 bytes value.] >> >> >> >> Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de> >> >> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> >> > >> > Not even a description why this is needed? >> > >> > This patch (together with patch 8) seems like the most controversial >> > one of the series, so you should better provide a really good reason >> > why we would emulate something in software rather than using whatever >> > hardware is there. >> >> At least on Tegra, there is no HW that exposes PCI configuration >> registers for the host bridge itself. Only the root ports have exposed >> PCI configuration registers. There was some debate re: whether a host >> bridge device needed to exist or not. This patch makes such a device >> exist if it's required. Host bridges are not actually PCI devices on any architecture. The upstream side of a host bridge is by definition not on a PCI bus. On some architectures, it *looks* like the host bridge is a PCI device because it responds to PCI config accesses and you can get to configuration registers that way. But it isn't really; you can't enumerate host bridges by using normal PCI device enumeration because you have to somehow discover the root bus and the method of doing config accesses to it. That is all outside the scope of PCI. Even on the architectures where host bridges appear in PCI config space, the only reason that works is because we assume a config access mechanism that works for domain 0. We can't discover bridges in other domains without help. > If Linux will discover properly (I strongly suspect it does) without > the host bridge, then I would say to ditch this... > > The PCI-E standard requires a host bridge device, but if Linux doesn't > require it then there is no reason to emulate one. I agree that you don't need to emulate anything in the sense of making config space accessors as this patch does. However, I think you *should* use pci_scan_root_bus() (maybe you do already; I haven't read all these patches), which requires that you know the configuration of the host bridge, i.e., the config access mechanism, the bus number range below the host bridge, and the I/O and MMIO apertures through the bridge. The PCI core builds a logical host bridge structure internally from that information, and that's all Linux really needs. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Mon, Jan 28, 2013 at 07:40:16PM -0700, Bjorn Helgaas wrote: > On Mon, Jan 28, 2013 at 3:09 PM, Jason Gunthorpe > <jgunthorpe@obsidianresearch.com> wrote: > > On Mon, Jan 28, 2013 at 03:03:55PM -0700, Stephen Warren wrote: > >> On 01/28/2013 01:18 PM, Arnd Bergmann wrote: > >> > On Monday 28 January 2013, Thomas Petazzoni wrote: > >> >> From: Thierry Reding <thierry.reding@avionic-design.de> > >> >> > >> >> [Thomas Petazzoni: > >> >> - Simplify capabilities handling. > >> >> - Move to a separate file. > >> >> - Fix mask used when writing a 4 bytes value.] > >> >> > >> >> Signed-off-by: Thierry Reding <thierry.reding@avionic-design.de> > >> >> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> > >> > > >> > Not even a description why this is needed? > >> > > >> > This patch (together with patch 8) seems like the most controversial > >> > one of the series, so you should better provide a really good reason > >> > why we would emulate something in software rather than using whatever > >> > hardware is there. > >> > >> At least on Tegra, there is no HW that exposes PCI configuration > >> registers for the host bridge itself. Only the root ports have exposed > >> PCI configuration registers. There was some debate re: whether a host > >> bridge device needed to exist or not. This patch makes such a device > >> exist if it's required. > > Host bridges are not actually PCI devices on any architecture. The > upstream side of a host bridge is by definition not on a PCI bus. On > some architectures, it *looks* like the host bridge is a PCI device > because it responds to PCI config accesses and you can get to Sure, you can't discover domains through any standard means, but once you have found a domain (notably a way to issue config transactions) then the PCI-E standard actually does place requirements on what config transactions should return: - 0:00.0 is a host bridge config space. - 0:XX.X will be one of: - A root complex internal function, with some restrictions this is basically a PCI end device - A PCI-PCI bridge with various mandatory capability headers. One of these must show up for every physical PCI-E link on the root complex. This collection of stuff on bus 0 is called the 'root complex'. This is new in PCI-E, PCI-X and PCI didn't have such requirements. SOC vendors are taking various liberties with their PCI-E implementations. - nvidia followed the standard but did not include the host bridge at 0:00.0 - Marvell ignored everything about the root complex config space behavior :) There are two patch sets in this subject, one for nvidia tegra and one for Marvell, both presenting to Linux a view of the HW that matches what the PCI-E spec describes - specifically that there is one domain, and each PCI-E link/controller shows up as a PCI-PCI bridge on bus 0. In this model, there is no 'host bridge aperture' hardware, each PCI-E link has a dedicated aperture and control of that aperture is through the PCI-PCI bridge window registers, again as PCI-E specifies. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Dear Jason Gunthorpe, On Mon, 28 Jan 2013 15:51:05 -0700, Jason Gunthorpe wrote: > > > I'm refering to your earlier question about what PCI IDs to use for > > > the SW emulated devices. If there is no need for the host bridge then > > > you only need 1 PCI ID (for the root port bridge) and you can probably > > > fairly safely re-use the one in the Marvell config space of the HW. > > > > Ah, ok, I see. But isn't a host bridge needed to bind all the > > PCI-to-PCI bridges under a single bus, in order to get the global > > resource assignment I was referring to? > > The PCI-E spec requires it, but AFAIK it doesn't actually *do* > anything on Linux, and Linux doesn't require it. > > I thought Thierry did this experiment and decided it wasn't necessary: Could you detail what would be visible PCI bus topology if I remove the emulated PCI host bridge? (And keeping one PCI-to-PCI bridge per PCIe interface) ? I'm just trying to understand what it would look like, in terms of "lspci -t" output, because for now, it's not clear to me how everything would fit together with the emulated host bridge. Thanks, Thomas
On Tue, Jan 29, 2013 at 11:01:19AM +0100, Thomas Petazzoni wrote: > Dear Jason Gunthorpe, > > On Mon, 28 Jan 2013 15:51:05 -0700, Jason Gunthorpe wrote: > > > > > I'm refering to your earlier question about what PCI IDs to use for > > > > the SW emulated devices. If there is no need for the host bridge then > > > > you only need 1 PCI ID (for the root port bridge) and you can probably > > > > fairly safely re-use the one in the Marvell config space of the HW. > > > > > > Ah, ok, I see. But isn't a host bridge needed to bind all the > > > PCI-to-PCI bridges under a single bus, in order to get the global > > > resource assignment I was referring to? > > > > The PCI-E spec requires it, but AFAIK it doesn't actually *do* > > anything on Linux, and Linux doesn't require it. > > > > I thought Thierry did this experiment and decided it wasn't necessary: > > Could you detail what would be visible PCI bus topology if I remove the > emulated PCI host bridge? (And keeping one PCI-to-PCI bridge per PCIe > interface) ? So if this is what you have now... -[0000:00]-+-00.0 <- Host bridge +-10.0 <- Bridge +-11.0 <- Bridge +-12.0 <- Bridge Then removing the config space at 00.0 (ie the host bridge) will give you: -[0000:00]-+-10.0 <- Bridge +-11.0 <- Bridge +-12.0 <- Bridge The 'host bridge' isn't a bridge in the PCI-PCI sense, it is just a normal device at bus 0, device 0, function 0. Jason -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Dear Jason Gunthorpe, On Tue, 29 Jan 2013 10:42:05 -0700, Jason Gunthorpe wrote: > So if this is what you have now... > > -[0000:00]-+-00.0 <- Host bridge > +-10.0 <- Bridge > +-11.0 <- Bridge > +-12.0 <- Bridge > > Then removing the config space at 00.0 (ie the host bridge) will give > you: > > -[0000:00]-+-10.0 <- Bridge > +-11.0 <- Bridge > +-12.0 <- Bridge > > The 'host bridge' isn't a bridge in the PCI-PCI sense, it is just a > normal device at bus 0, device 0, function 0. Ok, thanks I'll try this! Thomas
diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index 6d51aa6..f7548e2 100644 --- a/drivers/pci/Kconfig +++ b/drivers/pci/Kconfig @@ -119,3 +119,6 @@ config PCI_IOAPIC config PCI_LABEL def_bool y if (DMI || ACPI) select NLS + +config PCI_SW_HOST_BRIDGE + bool diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile index 0c3efcf..44ce914 100644 --- a/drivers/pci/Makefile +++ b/drivers/pci/Makefile @@ -15,6 +15,9 @@ obj-$(CONFIG_PCIEPORTBUS) += pcie/ obj-$(CONFIG_PCI_IOAPIC) += ioapic.o +# Emulated PCI elements +obj-$(CONFIG_PCI_SW_HOST_BRIDGE) += sw-host-bridge.o + # Build the PCI Hotplug drivers if we were asked to obj-$(CONFIG_HOTPLUG_PCI) += hotplug/ ifdef CONFIG_HOTPLUG_PCI diff --git a/drivers/pci/sw-host-bridge.c b/drivers/pci/sw-host-bridge.c new file mode 100644 index 0000000..b5a2aed --- /dev/null +++ b/drivers/pci/sw-host-bridge.c @@ -0,0 +1,144 @@ +/* + * Implementation of a simple emulated PCI host bridge. + * + * Thierry Reding <thierry.reding@avionic-design.de> + * Thomas Petazzoni <thomas.petazzoni@free-electrons.com> + * + * This file is licensed under the terms of the GNU General Public + * License version 2. This program is licensed "as is" without any + * warranty of any kind, whether express or implied. + */ + +#include <linux/kernel.h> +#include <linux/init.h> +#include <linux/pci.h> +#include <linux/module.h> + +int pci_sw_host_bridge_init(struct pci_sw_host_bridge *bridge) +{ + unsigned int i; + + if (!bridge) + return -EINVAL; + + bridge->vendor = 0x0000; + bridge->device = 0x0000; + + bridge->command = 0x0000; + bridge->status = PCI_STATUS_CAP_LIST; + + bridge->class = PCI_CLASS_BRIDGE_HOST; + bridge->interface = 0x00; + bridge->revision = 0x00; + + bridge->bist = 0x00; + bridge->header_type = PCI_HEADER_TYPE_NORMAL; + bridge->latency_timer = 0x00; + bridge->cache_line_size = 0x10; + + for (i = 0; i < 6; i++) + bridge->bar[i] = 0x00000000; + + bridge->subsystem_vendor = 0x0000; + bridge->subsystem_device = 0x0000; + + return 0; +} +EXPORT_SYMBOL_GPL(pci_sw_host_bridge_init); + +int pci_sw_host_bridge_read(struct pci_sw_host_bridge *bridge, + unsigned int where, int size, u32 *value) +{ + switch (where & ~3) { + case PCI_VENDOR_ID: + *value = bridge->device << 16 | bridge->vendor; + break; + + case PCI_COMMAND: + *value = bridge->status << 16 | bridge->command; + break; + + case PCI_STATUS: + *value = 0; + break; + + case PCI_CLASS_REVISION: + *value = bridge->class << 16 | bridge->interface << 8 | + bridge->revision; + break; + + case PCI_CACHE_LINE_SIZE: + *value = bridge->bist << 24 | bridge->header_type << 16 | + bridge->latency_timer << 8 | bridge->cache_line_size; + break; + + case PCI_BASE_ADDRESS_0 ... PCI_BASE_ADDRESS_5: + *value = bridge->bar[((where & ~3) - PCI_BASE_ADDRESS_0) / 4]; + break; + + case PCI_CARDBUS_CIS: + *value = 0; + break; + + case PCI_SUBSYSTEM_VENDOR_ID: + *value = bridge->subsystem_device << 16 | + bridge->subsystem_vendor; + break; + + case PCI_ROM_ADDRESS: + *value = 0; + break; + + case PCI_INTERRUPT_LINE: + break; + + default: + *value = 0xffffffff; + return PCIBIOS_BAD_REGISTER_NUMBER; + } + + if (size == 2) + *value = (*value >> (8 * (where & 3))) & 0xffff; + else if (size == 1) + *value = (*value >> (8 * (where & 3))) & 0xff; + + return PCIBIOS_SUCCESSFUL; +} +EXPORT_SYMBOL_GPL(pci_sw_host_bridge_read); + +int pci_sw_host_bridge_write(struct pci_sw_host_bridge *bridge, + unsigned int where, int size, u32 value) +{ + u32 mask, reg; + int err; + + if (size == 4) + mask = 0x0; + else if (size == 2) + mask = ~(0xffff << ((where & 3) * 8)); + else if (size == 1) + mask = ~(0xff << ((where & 3) * 8)); + else + return PCIBIOS_BAD_REGISTER_NUMBER; + + err = pci_sw_host_bridge_read(bridge, where & ~3, 4, ®); + if (err) + return err; + + value = (reg & mask) | value << ((where & 3) * 8); + + switch (where & ~3) { + case PCI_COMMAND: + bridge->command = value & 0xffff; + bridge->status = value >> 16; + break; + + case PCI_BASE_ADDRESS_0 ... PCI_BASE_ADDRESS_5: + bridge->bar[((where & ~3) - PCI_BASE_ADDRESS_0) / 4] = value; + break; + } + + return PCIBIOS_SUCCESSFUL; +} +EXPORT_SYMBOL_GPL(pci_sw_host_bridge_write); + diff --git a/include/linux/pci.h b/include/linux/pci.h index 15472d6..c93e258 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1841,4 +1841,27 @@ static inline struct eeh_dev *pci_dev_to_eeh_dev(struct pci_dev *pdev) */ struct pci_dev *pci_find_upstream_pcie_bridge(struct pci_dev *pdev); +struct pci_sw_host_bridge { + u16 vendor; + u16 device; + u16 command; + u16 status; + u16 class; + u8 interface; + u8 revision; + u8 bist; + u8 header_type; + u8 latency_timer; + u8 cache_line_size; + u32 bar[6]; + u16 subsystem_vendor; + u16 subsystem_device; +}; + +extern int pci_sw_host_bridge_init(struct pci_sw_host_bridge *bridge); +extern int pci_sw_host_bridge_read(struct pci_sw_host_bridge *bridge, + unsigned int where, int size, u32 *value); +extern int pci_sw_host_bridge_write(struct pci_sw_host_bridge *bridge, + unsigned int where, int size, u32 value); + #endif /* LINUX_PCI_H */