Message ID | 1425603699-13416-3-git-send-email-rjui@broadcom.com (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
On Thu, 2015-03-05 at 17:01 -0800, Ray Jui wrote: > --- a/drivers/pci/host/Kconfig > +++ b/drivers/pci/host/Kconfig > @@ -106,4 +106,21 @@ config PCI_VERSATILE > bool "ARM Versatile PB PCI controller" > depends on ARCH_VERSATILE > > +config PCIE_IPROC > + bool "Broadcom iProc PCIe controller" bool symbol. > + help > + This enables the iProc PCIe core controller support for Broadcom's > + iProc family of SoCs. An appropriate bus interface driver also needs > + to be enabled > + > +config PCIE_IPROC_PLTFM > + bool "Broadcom iProc PCIe platform bus driver" Another bool symbol. > + depends on ARCH_BCM_IPROC || COMPILE_TEST > + depends on OF > + select PCIE_IPROC > + default ARCH_BCM_IPROC > + help > + Say Y here if you want to use the Broadcom iProc PCIe controller > + through the generic platform bus interface > + > endmenu > --- a/drivers/pci/host/Makefile > +++ b/drivers/pci/host/Makefile > +obj-$(CONFIG_PCIE_IPROC) += pcie-iproc.o > +obj-$(CONFIG_PCIE_IPROC_PLTFM) += pcie-iproc-pltfm.o Both objectfiles will never be part of a module. > --- /dev/null > +++ b/drivers/pci/host/pcie-iproc-pltfm.c > +#include <linux/module.h> Is this needed? > +MODULE_DEVICE_TABLE(of, iproc_pcie_of_match_table); This macro will be preprocessed away, won't it? > +static struct platform_driver iproc_pcie_pltfm_driver = { > + .driver = { > + .name = "iproc-pcie", > + .of_match_table = of_match_ptr(iproc_pcie_of_match_table), > + .suppress_bind_attrs = true, > + }, > + .probe = iproc_pcie_pltfm_probe, > +}; > +module_platform_driver(iproc_pcie_pltfm_driver); Perhaps it's clearer to have make this a call to platform_driver_register(), put that in a wrapper function, and invoke that wrapper function through device_initcall() or similar. I haven't actually tested that, so the details could be off. > +MODULE_AUTHOR("Ray Jui <rjui@broadcom.com>"); > +MODULE_DESCRIPTION("Broadcom iPROC PCIe platform driver"); > +MODULE_LICENSE("GPL v2"); And these three macros will, effectively, be preprocessed away. > --- /dev/null > +++ b/drivers/pci/host/pcie-iproc.c > +#include <linux/module.h> See above. > +MODULE_AUTHOR("Ray Jui <rjui@broadcom.com>"); > +MODULE_DESCRIPTION("Broadcom iPROC PCIe common driver"); > +MODULE_LICENSE("GPL v2"); Ditto. Paul Bolle
Hi Paul, On 3/6/2015 3:00 AM, Paul Bolle wrote: > On Thu, 2015-03-05 at 17:01 -0800, Ray Jui wrote: >> --- a/drivers/pci/host/Kconfig >> +++ b/drivers/pci/host/Kconfig >> @@ -106,4 +106,21 @@ config PCI_VERSATILE >> bool "ARM Versatile PB PCI controller" >> depends on ARCH_VERSATILE >> >> +config PCIE_IPROC >> + bool "Broadcom iProc PCIe controller" > > bool symbol. > >> + help >> + This enables the iProc PCIe core controller support for Broadcom's >> + iProc family of SoCs. An appropriate bus interface driver also needs >> + to be enabled >> + >> +config PCIE_IPROC_PLTFM >> + bool "Broadcom iProc PCIe platform bus driver" > > Another bool symbol. > >> + depends on ARCH_BCM_IPROC || COMPILE_TEST >> + depends on OF >> + select PCIE_IPROC >> + default ARCH_BCM_IPROC >> + help >> + Say Y here if you want to use the Broadcom iProc PCIe controller >> + through the generic platform bus interface >> + >> endmenu > >> --- a/drivers/pci/host/Makefile >> +++ b/drivers/pci/host/Makefile > >> +obj-$(CONFIG_PCIE_IPROC) += pcie-iproc.o >> +obj-$(CONFIG_PCIE_IPROC_PLTFM) += pcie-iproc-pltfm.o > > Both objectfiles will never be part of a module. > >> --- /dev/null >> +++ b/drivers/pci/host/pcie-iproc-pltfm.c > >> +#include <linux/module.h> > > Is this needed? > >> +MODULE_DEVICE_TABLE(of, iproc_pcie_of_match_table); > > This macro will be preprocessed away, won't it? > >> +static struct platform_driver iproc_pcie_pltfm_driver = { >> + .driver = { >> + .name = "iproc-pcie", >> + .of_match_table = of_match_ptr(iproc_pcie_of_match_table), >> + .suppress_bind_attrs = true, >> + }, >> + .probe = iproc_pcie_pltfm_probe, >> +}; >> +module_platform_driver(iproc_pcie_pltfm_driver); > > Perhaps it's clearer to have make this a call to > platform_driver_register(), put that in a wrapper function, and invoke > that wrapper function through device_initcall() or similar. I haven't > actually tested that, so the details could be off. > >> +MODULE_AUTHOR("Ray Jui <rjui@broadcom.com>"); >> +MODULE_DESCRIPTION("Broadcom iPROC PCIe platform driver"); >> +MODULE_LICENSE("GPL v2"); > > And these three macros will, effectively, be preprocessed away. > >> --- /dev/null >> +++ b/drivers/pci/host/pcie-iproc.c > >> +#include <linux/module.h> > > See above. > >> +MODULE_AUTHOR("Ray Jui <rjui@broadcom.com>"); >> +MODULE_DESCRIPTION("Broadcom iPROC PCIe common driver"); >> +MODULE_LICENSE("GPL v2"); > > Ditto. > > > Paul Bolle > So every single PCIe host driver under drivers/pci/host/* has their config flag of "bool" type, and all of them except pci-keystone-dw.c have these MODULE based macros in their driver. While I agree with you that these macros will preprocessed away while compiled as statically linked in, I thought it's a convention to have these macros in the driver. At least it provides information on the author, driver description, and license (although one can also argue you can find all of those info from the maintainer list, Kconfig, and license header). Would you be able to sort this out with a developer or subsystem maintainer who is familiar with this matter and let us know what is the convention for MODULE macros usage in a driver when its config flag is set to "bool" instead of "tristate"? Thanks, Ray
On Fri, Mar 6, 2015 at 9:26 AM, Ray Jui <rjui@broadcom.com> wrote: > Hi Paul, > > On 3/6/2015 3:00 AM, Paul Bolle wrote: >> On Thu, 2015-03-05 at 17:01 -0800, Ray Jui wrote: >>> --- a/drivers/pci/host/Kconfig >>> +++ b/drivers/pci/host/Kconfig >>> @@ -106,4 +106,21 @@ config PCI_VERSATILE >>> bool "ARM Versatile PB PCI controller" >>> depends on ARCH_VERSATILE >>> >>> +config PCIE_IPROC >>> + bool "Broadcom iProc PCIe controller" >> >> bool symbol. >> >>> + help >>> + This enables the iProc PCIe core controller support for Broadcom's >>> + iProc family of SoCs. An appropriate bus interface driver also needs >>> + to be enabled >>> + >>> +config PCIE_IPROC_PLTFM >>> + bool "Broadcom iProc PCIe platform bus driver" >> >> Another bool symbol. >> >>> + depends on ARCH_BCM_IPROC || COMPILE_TEST >>> + depends on OF >>> + select PCIE_IPROC >>> + default ARCH_BCM_IPROC >>> + help >>> + Say Y here if you want to use the Broadcom iProc PCIe controller >>> + through the generic platform bus interface >>> + >>> endmenu >> >>> --- a/drivers/pci/host/Makefile >>> +++ b/drivers/pci/host/Makefile >> >>> +obj-$(CONFIG_PCIE_IPROC) += pcie-iproc.o >>> +obj-$(CONFIG_PCIE_IPROC_PLTFM) += pcie-iproc-pltfm.o >> >> Both objectfiles will never be part of a module. >> >>> --- /dev/null >>> +++ b/drivers/pci/host/pcie-iproc-pltfm.c >> >>> +#include <linux/module.h> >> >> Is this needed? >> >>> +MODULE_DEVICE_TABLE(of, iproc_pcie_of_match_table); >> >> This macro will be preprocessed away, won't it? >> >>> +static struct platform_driver iproc_pcie_pltfm_driver = { >>> + .driver = { >>> + .name = "iproc-pcie", >>> + .of_match_table = of_match_ptr(iproc_pcie_of_match_table), >>> + .suppress_bind_attrs = true, >>> + }, >>> + .probe = iproc_pcie_pltfm_probe, >>> +}; >>> +module_platform_driver(iproc_pcie_pltfm_driver); >> >> Perhaps it's clearer to have make this a call to >> platform_driver_register(), put that in a wrapper function, and invoke >> that wrapper function through device_initcall() or similar. I haven't >> actually tested that, so the details could be off. >> >>> +MODULE_AUTHOR("Ray Jui <rjui@broadcom.com>"); >>> +MODULE_DESCRIPTION("Broadcom iPROC PCIe platform driver"); >>> +MODULE_LICENSE("GPL v2"); >> >> And these three macros will, effectively, be preprocessed away. >> >>> --- /dev/null >>> +++ b/drivers/pci/host/pcie-iproc.c >> >>> +#include <linux/module.h> >> >> See above. >> >>> +MODULE_AUTHOR("Ray Jui <rjui@broadcom.com>"); >>> +MODULE_DESCRIPTION("Broadcom iPROC PCIe common driver"); >>> +MODULE_LICENSE("GPL v2"); >> >> Ditto. >> >> >> Paul Bolle >> > > So every single PCIe host driver under drivers/pci/host/* has their > config flag of "bool" type, and all of them except pci-keystone-dw.c > have these MODULE based macros in their driver. While I agree with you > that these macros will preprocessed away while compiled as statically > linked in, I thought it's a convention to have these macros in the > driver. At least it provides information on the author, driver > description, and license (although one can also argue you can find all > of those info from the maintainer list, Kconfig, and license header). > > Would you be able to sort this out with a developer or subsystem > maintainer who is familiar with this matter and let us know what is the > convention for MODULE macros usage in a driver when its config flag is > set to "bool" instead of "tristate"? Is there a technical reason why the driver can't be a module? We can suppress module unloading if unloading properly is hard (i see you already suppress unbinding via sysfs), but we should be able to load it after kernel booted, no? Thanks, Dmitry
Hi Dmitry, On 3/6/2015 9:35 AM, Dmitry Torokhov wrote: > On Fri, Mar 6, 2015 at 9:26 AM, Ray Jui <rjui@broadcom.com> wrote: >> Hi Paul, >> >> On 3/6/2015 3:00 AM, Paul Bolle wrote: >>> On Thu, 2015-03-05 at 17:01 -0800, Ray Jui wrote: >>>> --- a/drivers/pci/host/Kconfig >>>> +++ b/drivers/pci/host/Kconfig >>>> @@ -106,4 +106,21 @@ config PCI_VERSATILE >>>> bool "ARM Versatile PB PCI controller" >>>> depends on ARCH_VERSATILE >>>> >>>> +config PCIE_IPROC >>>> + bool "Broadcom iProc PCIe controller" >>> >>> bool symbol. >>> >>>> + help >>>> + This enables the iProc PCIe core controller support for Broadcom's >>>> + iProc family of SoCs. An appropriate bus interface driver also needs >>>> + to be enabled >>>> + >>>> +config PCIE_IPROC_PLTFM >>>> + bool "Broadcom iProc PCIe platform bus driver" >>> >>> Another bool symbol. >>> >>>> + depends on ARCH_BCM_IPROC || COMPILE_TEST >>>> + depends on OF >>>> + select PCIE_IPROC >>>> + default ARCH_BCM_IPROC >>>> + help >>>> + Say Y here if you want to use the Broadcom iProc PCIe controller >>>> + through the generic platform bus interface >>>> + >>>> endmenu >>> >>>> --- a/drivers/pci/host/Makefile >>>> +++ b/drivers/pci/host/Makefile >>> >>>> +obj-$(CONFIG_PCIE_IPROC) += pcie-iproc.o >>>> +obj-$(CONFIG_PCIE_IPROC_PLTFM) += pcie-iproc-pltfm.o >>> >>> Both objectfiles will never be part of a module. >>> >>>> --- /dev/null >>>> +++ b/drivers/pci/host/pcie-iproc-pltfm.c >>> >>>> +#include <linux/module.h> >>> >>> Is this needed? >>> >>>> +MODULE_DEVICE_TABLE(of, iproc_pcie_of_match_table); >>> >>> This macro will be preprocessed away, won't it? >>> >>>> +static struct platform_driver iproc_pcie_pltfm_driver = { >>>> + .driver = { >>>> + .name = "iproc-pcie", >>>> + .of_match_table = of_match_ptr(iproc_pcie_of_match_table), >>>> + .suppress_bind_attrs = true, >>>> + }, >>>> + .probe = iproc_pcie_pltfm_probe, >>>> +}; >>>> +module_platform_driver(iproc_pcie_pltfm_driver); >>> >>> Perhaps it's clearer to have make this a call to >>> platform_driver_register(), put that in a wrapper function, and invoke >>> that wrapper function through device_initcall() or similar. I haven't >>> actually tested that, so the details could be off. >>> >>>> +MODULE_AUTHOR("Ray Jui <rjui@broadcom.com>"); >>>> +MODULE_DESCRIPTION("Broadcom iPROC PCIe platform driver"); >>>> +MODULE_LICENSE("GPL v2"); >>> >>> And these three macros will, effectively, be preprocessed away. >>> >>>> --- /dev/null >>>> +++ b/drivers/pci/host/pcie-iproc.c >>> >>>> +#include <linux/module.h> >>> >>> See above. >>> >>>> +MODULE_AUTHOR("Ray Jui <rjui@broadcom.com>"); >>>> +MODULE_DESCRIPTION("Broadcom iPROC PCIe common driver"); >>>> +MODULE_LICENSE("GPL v2"); >>> >>> Ditto. >>> >>> >>> Paul Bolle >>> >> >> So every single PCIe host driver under drivers/pci/host/* has their >> config flag of "bool" type, and all of them except pci-keystone-dw.c >> have these MODULE based macros in their driver. While I agree with you >> that these macros will preprocessed away while compiled as statically >> linked in, I thought it's a convention to have these macros in the >> driver. At least it provides information on the author, driver >> description, and license (although one can also argue you can find all >> of those info from the maintainer list, Kconfig, and license header). >> >> Would you be able to sort this out with a developer or subsystem >> maintainer who is familiar with this matter and let us know what is the >> convention for MODULE macros usage in a driver when its config flag is >> set to "bool" instead of "tristate"? > > Is there a technical reason why the driver can't be a module? We can > suppress module unloading if unloading properly is hard (i see you > already suppress unbinding via sysfs), but we should be able to load > it after kernel booted, no? > > Thanks, > Dmitry > Although I have not tested it, but to my best knowledge there shouldn't be any technical issue by making the PCIe iProc driver a loadable module and installing the module later after the kernel finishes booting. But I wonder why I haven't seen any PCIe host driver being allowed to be compiled as module? Maybe there's no obvious benefit of doing that. People typically opt to load slave devices as module but tend to keep the bus or host devices compiled in. And to allow a PCI host driver to be compiled as module, some PCI functions need to have their symbols exported: ERROR: "pci_common_swizzle" [drivers/pci/host/pcie-iproc.ko] undefined! ERROR: "pci_fixup_irqs" [drivers/pci/host/pcie-iproc.ko] undefined! ERROR: "pci_assign_unassigned_bus_resources" [drivers/pci/host/pcie-iproc.ko] undefined! ERROR: "pci_remove_root_bus" [drivers/pci/host/pcie-iproc.ko] undefined! ERROR: "pci_stop_root_bus" [drivers/pci/host/pcie-iproc.ko] undefined! ERROR: "pci_create_root_bus" [drivers/pci/host/pcie-iproc.ko] undefined! Maybe Bjorn can help to shed some light here? Should I go ahead and export_symbol these PCI functions and make the iProc PCI driver "tristate"? Thanks, Ray
On Friday 06 March 2015 10:00:34 Ray Jui wrote: > > > > Although I have not tested it, but to my best knowledge there shouldn't > be any technical issue by making the PCIe iProc driver a loadable module > and installing the module later after the kernel finishes booting. > > But I wonder why I haven't seen any PCIe host driver being allowed to be > compiled as module? Maybe there's no obvious benefit of doing that. > People typically opt to load slave devices as module but tend to keep > the bus or host devices compiled in. > > And to allow a PCI host driver to be compiled as module, some PCI > functions need to have their symbols exported: > > ERROR: "pci_common_swizzle" [drivers/pci/host/pcie-iproc.ko] undefined! > ERROR: "pci_fixup_irqs" [drivers/pci/host/pcie-iproc.ko] undefined! > ERROR: "pci_assign_unassigned_bus_resources" > [drivers/pci/host/pcie-iproc.ko] undefined! > ERROR: "pci_remove_root_bus" [drivers/pci/host/pcie-iproc.ko] undefined! > ERROR: "pci_stop_root_bus" [drivers/pci/host/pcie-iproc.ko] undefined! > ERROR: "pci_create_root_bus" [drivers/pci/host/pcie-iproc.ko] undefined! > > Maybe Bjorn can help to shed some light here? Should I go ahead and > export_symbol these PCI functions and make the iProc PCI driver "tristate"? My best guess is that no loadable driver ever tried to use these and we should indeed just export them. The call to pci_assign_unassigned_bus_resources could be avoided by using pci_rescan_bus(), which would simplify the driver a little, but for some reason, nothing else uses that, so I'm not sure about it. The pci_remove_root_bus/pci_stop_root_bus calls are rarely used anywhere else. Are they functional to the point where you could unload a pci host driver that is a loadable module? If so, that would be wonderful. Arnd
Hi Arnd, On 3/9/2015 5:35 AM, Arnd Bergmann wrote: > On Friday 06 March 2015 10:00:34 Ray Jui wrote: >>> >> >> Although I have not tested it, but to my best knowledge there shouldn't >> be any technical issue by making the PCIe iProc driver a loadable module >> and installing the module later after the kernel finishes booting. >> >> But I wonder why I haven't seen any PCIe host driver being allowed to be >> compiled as module? Maybe there's no obvious benefit of doing that. >> People typically opt to load slave devices as module but tend to keep >> the bus or host devices compiled in. >> >> And to allow a PCI host driver to be compiled as module, some PCI >> functions need to have their symbols exported: >> >> ERROR: "pci_common_swizzle" [drivers/pci/host/pcie-iproc.ko] undefined! >> ERROR: "pci_fixup_irqs" [drivers/pci/host/pcie-iproc.ko] undefined! >> ERROR: "pci_assign_unassigned_bus_resources" >> [drivers/pci/host/pcie-iproc.ko] undefined! >> ERROR: "pci_remove_root_bus" [drivers/pci/host/pcie-iproc.ko] undefined! >> ERROR: "pci_stop_root_bus" [drivers/pci/host/pcie-iproc.ko] undefined! >> ERROR: "pci_create_root_bus" [drivers/pci/host/pcie-iproc.ko] undefined! >> >> Maybe Bjorn can help to shed some light here? Should I go ahead and >> export_symbol these PCI functions and make the iProc PCI driver "tristate"? > > My best guess is that no loadable driver ever tried to use these and > we should indeed just export them. True. I can export those symbols. > > The call to pci_assign_unassigned_bus_resources could be avoided by using > pci_rescan_bus(), which would simplify the driver a little, but for > some reason, nothing else uses that, so I'm not sure about it. > > The pci_remove_root_bus/pci_stop_root_bus calls are rarely used anywhere > else. Are they functional to the point where you could unload a pci > host driver that is a loadable module? If so, that would be > wonderful. It is quite wonderful, :) I added driver remove function support and tested it out. I'm able to install/remove/install the iProc PCIe platform driver as a kernel loadable module. pci_remove_root_bus/pci_stop_root_bus works fine. See logs below. -bash-3.2# insmod pcie-iproc-pltfm.ko [ 54.058546] PCI host bridge /pcie@18012000 ranges: [ 54.064584] IO 0x28000000..0x2800ffff -> 0x00000000 [ 54.071003] MEM 0x20000000..0x23ffffff -> 0x20000000 [ 54.330194] iproc-pcie 18012000.pcie: PCI host bridge to bus 0000:00 [ 54.338138] pci_bus 0000:00: root bus resource [bus 00-ff] [ 54.344983] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] [ 54.352683] pci_bus 0000:00: root bus resource [mem 0x20000000-0x23ffffff] [ 54.361254] iproc-pcie 18012000.pcie: link: UP [ 54.367086] PCI: bus0: Fast back to back transfers disabled [ 54.374046] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring [ 54.400237] PCI: bus1: Fast back to back transfers disabled [ 54.407274] pci 0000:00:00.0: BAR 8: assigned [mem 0x20000000-0x200fffff] [ 54.415766] pci 0000:01:00.0: BAR 1: assigned [mem 0x20000000-0x2007ffff] [ 54.424241] pci 0000:01:00.0: BAR 6: assigned [mem 0x20080000-0x200bffff pref] [ 54.433245] pci 0000:01:00.0: BAR 0: assigned [mem 0x200c0000-0x200dffff] [ 54.441710] pci 0000:01:00.0: BAR 3: assigned [mem 0x200e0000-0x200e3fff] [ 54.450173] pci 0000:01:00.0: BAR 2: no space for [io size 0x0020] [ 54.457979] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0020] [ 54.466217] pci 0000:00:00.0: PCI bridge to [bus 01] [ 54.472404] pci 0000:00:00.0: bridge window [mem 0x20000000-0x200fffff] [ 54.480945] pcieport 0000:00:00.0: enabling device (0140 -> 0142) [ 54.488767] e1000e 0000:01:00.0: enabling device (0140 -> 0142) [ 54.496315] e1000e 0000:01:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode [ 54.507958] e1000e 0000:01:00.0 0000:01:00.0 (uninitialized): Failed to initialize MSI-X interrupts. Falling back to MSI interru. [ 54.522913] e1000e 0000:01:00.0 0000:01:00.0 (uninitialized): Failed to initialize MSI interrupts. Falling back to legacy interr. [ 54.643486] e1000e 0000:01:00.0 eth0: registered PHC clock [ 54.650379] e1000e 0000:01:00.0 eth0: (PCI Express:2.5GT/s:Width x1) 68:05:ca:28:19:e8 [ 54.660254] e1000e 0000:01:00.0 eth0: Intel(R) PRO/1000 Network Connection [ 54.668834] e1000e 0000:01:00.0 eth0: MAC: 3, PHY: 8, PBA No: E46981-008 -bash-3.2# rmmod pcie-iproc-pltfm [ 66.465834] e1000e 0000:01:00.0 eth0: removed PHC -bash-3.2# insmod pcie-iproc-pltfm.ko [ 71.043195] PCI host bridge /pcie@18012000 ranges: [ 71.049235] IO 0x28000000..0x2800ffff -> 0x00000000 [ 71.055652] MEM 0x20000000..0x23ffffff -> 0x20000000 [ 71.315192] iproc-pcie 18012000.pcie: PCI host bridge to bus 0000:00 [ 71.323135] pci_bus 0000:00: root bus resource [bus 00-ff] [ 71.329979] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] [ 71.337679] pci_bus 0000:00: root bus resource [mem 0x20000000-0x23ffffff] [ 71.346249] iproc-pcie 18012000.pcie: link: UP [ 71.352072] PCI: bus0: Fast back to back transfers disabled [ 71.375261] PCI: bus1: Fast back to back transfers disabled [ 71.382291] pci 0000:00:00.0: BAR 8: assigned [mem 0x20000000-0x200fffff] [ 71.390782] pci 0000:01:00.0: BAR 1: assigned [mem 0x20000000-0x2007ffff] [ 71.399258] pci 0000:01:00.0: BAR 6: assigned [mem 0x20080000-0x200bffff pref] [ 71.408263] pci 0000:01:00.0: BAR 0: assigned [mem 0x200c0000-0x200dffff] [ 71.416728] pci 0000:01:00.0: BAR 3: assigned [mem 0x200e0000-0x200e3fff] [ 71.425190] pci 0000:01:00.0: BAR 2: no space for [io size 0x0020] [ 71.432996] pci 0000:01:00.0: BAR 2: failed to assign [io size 0x0020] [ 71.441235] pci 0000:00:00.0: PCI bridge to [bus 01] [ 71.447422] pci 0000:00:00.0: bridge window [mem 0x20000000-0x200fffff] [ 71.456125] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt [ 71.464812] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt [ 71.473411] e1000e 0000:01:00.0: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode [ 71.485069] e1000e 0000:01:00.0 0000:01:00.0 (uninitialized): Failed to initialize MSI-X interrupts. Falling back to MSI interru. [ 71.500025] e1000e 0000:01:00.0 0000:01:00.0 (uninitialized): Failed to initialize MSI interrupts. Falling back to legacy interr. [ 71.624549] e1000e 0000:01:00.0 eth0: registered PHC clock [ 71.631444] e1000e 0000:01:00.0 eth0: (PCI Express:2.5GT/s:Width x1) 68:05:ca:28:19:e8 [ 71.641314] e1000e 0000:01:00.0 eth0: Intel(R) PRO/1000 Network Connection [ 71.649894] e1000e 0000:01:00.0 eth0: MAC: 3, PHY: 8, PBA No: E46981-008 > > Arnd > I'll submit patch series v4 with loadable module support. Thanks, Ray
diff --git a/drivers/pci/host/Kconfig b/drivers/pci/host/Kconfig index 7b892a9..e10c3e3 100644 --- a/drivers/pci/host/Kconfig +++ b/drivers/pci/host/Kconfig @@ -106,4 +106,21 @@ config PCI_VERSATILE bool "ARM Versatile PB PCI controller" depends on ARCH_VERSATILE +config PCIE_IPROC + bool "Broadcom iProc PCIe controller" + help + This enables the iProc PCIe core controller support for Broadcom's + iProc family of SoCs. An appropriate bus interface driver also needs + to be enabled + +config PCIE_IPROC_PLTFM + bool "Broadcom iProc PCIe platform bus driver" + depends on ARCH_BCM_IPROC || COMPILE_TEST + depends on OF + select PCIE_IPROC + default ARCH_BCM_IPROC + help + Say Y here if you want to use the Broadcom iProc PCIe controller + through the generic platform bus interface + endmenu diff --git a/drivers/pci/host/Makefile b/drivers/pci/host/Makefile index e61d91c..2e02d20 100644 --- a/drivers/pci/host/Makefile +++ b/drivers/pci/host/Makefile @@ -13,3 +13,5 @@ obj-$(CONFIG_PCIE_XILINX) += pcie-xilinx.o obj-$(CONFIG_PCI_XGENE) += pci-xgene.o obj-$(CONFIG_PCI_LAYERSCAPE) += pci-layerscape.o obj-$(CONFIG_PCI_VERSATILE) += pci-versatile.o +obj-$(CONFIG_PCIE_IPROC) += pcie-iproc.o +obj-$(CONFIG_PCIE_IPROC_PLTFM) += pcie-iproc-pltfm.o diff --git a/drivers/pci/host/pcie-iproc-pltfm.c b/drivers/pci/host/pcie-iproc-pltfm.c new file mode 100644 index 0000000..20d5287 --- /dev/null +++ b/drivers/pci/host/pcie-iproc-pltfm.c @@ -0,0 +1,101 @@ +/* + * Copyright (C) 2015 Broadcom Corporation + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation version 2. + * + * This program is distributed "as is" WITHOUT ANY WARRANTY of any + * kind, whether express or implied; without even the implied warranty + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include <linux/kernel.h> +#include <linux/pci.h> +#include <linux/clk.h> +#include <linux/module.h> +#include <linux/slab.h> +#include <linux/interrupt.h> +#include <linux/platform_device.h> +#include <linux/of_address.h> +#include <linux/of_pci.h> +#include <linux/of_irq.h> +#include <linux/of_platform.h> +#include <linux/phy/phy.h> + +#include "pcie-iproc.h" + +static int __init iproc_pcie_pltfm_probe(struct platform_device *pdev) +{ + struct iproc_pcie *pcie; + struct device_node *np = pdev->dev.of_node; + struct resource reg; + resource_size_t iobase = 0; + LIST_HEAD(res); + int ret; + + pcie = devm_kzalloc(&pdev->dev, sizeof(struct iproc_pcie), GFP_KERNEL); + if (!pcie) + return -ENOMEM; + + pcie->dev = &pdev->dev; + platform_set_drvdata(pdev, pcie); + + ret = of_address_to_resource(np, 0, ®); + if (ret < 0) { + dev_err(pcie->dev, "unable to obtain controller resources\n"); + return ret; + } + + pcie->base = devm_ioremap(pcie->dev, reg.start, resource_size(®)); + if (!pcie->base) { + dev_err(pcie->dev, "unable to map controller registers\n"); + return -ENOMEM; + } + + /* PHY use is optional */ + pcie->phy = devm_phy_get(&pdev->dev, "pcie-phy"); + if (IS_ERR(pcie->phy)) { + if (PTR_ERR(pcie->phy) == -EPROBE_DEFER) + return -EPROBE_DEFER; + pcie->phy = NULL; + } + + ret = of_pci_get_host_bridge_resources(np, 0, 0xff, &res, &iobase); + if (ret) { + dev_err(pcie->dev, + "unable to get PCI host bridge resources\n"); + return ret; + } + + pcie->resources = &res; + + ret = iproc_pcie_setup(pcie); + if (ret) { + dev_err(pcie->dev, "PCIe controller setup failed\n"); + return ret; + } + + return 0; +} + +static const struct of_device_id iproc_pcie_of_match_table[] = { + { .compatible = "brcm,iproc-pcie", }, + { /* sentinel */ } +}; +MODULE_DEVICE_TABLE(of, iproc_pcie_of_match_table); + +static struct platform_driver iproc_pcie_pltfm_driver = { + .driver = { + .name = "iproc-pcie", + .of_match_table = of_match_ptr(iproc_pcie_of_match_table), + .suppress_bind_attrs = true, + }, + .probe = iproc_pcie_pltfm_probe, +}; +module_platform_driver(iproc_pcie_pltfm_driver); + +MODULE_AUTHOR("Ray Jui <rjui@broadcom.com>"); +MODULE_DESCRIPTION("Broadcom iPROC PCIe platform driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/pci/host/pcie-iproc.c b/drivers/pci/host/pcie-iproc.c new file mode 100644 index 0000000..36c3357 --- /dev/null +++ b/drivers/pci/host/pcie-iproc.c @@ -0,0 +1,337 @@ +/* + * Copyright (C) 2014 Hauke Mehrtens <hauke@hauke-m.de> + * Copyright (C) 2015 Broadcom Corporatcommon ion + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation version 2. + * + * This program is distributed "as is" WITHOUT ANY WARRANTY of any + * kind, whether express or implied; without even the implied warranty + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#include <linux/kernel.h> +#include <linux/pci.h> +#include <linux/msi.h> +#include <linux/clk.h> +#include <linux/module.h> +#include <linux/mbus.h> +#include <linux/slab.h> +#include <linux/delay.h> +#include <linux/interrupt.h> +#include <linux/platform_device.h> +#include <linux/of_address.h> +#include <linux/of_pci.h> +#include <linux/of_irq.h> +#include <linux/of_platform.h> +#include <linux/phy/phy.h> + +#include "pcie-iproc.h" + +#define CLK_CONTROL_OFFSET 0x000 +#define EP_MODE_SURVIVE_PERST_SHIFT 1 +#define EP_MODE_SURVIVE_PERST BIT(EP_MODE_SURVIVE_PERST_SHIFT) +#define RC_PCIE_RST_OUTPUT_SHIFT 0 +#define RC_PCIE_RST_OUTPUT BIT(RC_PCIE_RST_OUTPUT_SHIFT) + +#define CFG_IND_ADDR_OFFSET 0x120 +#define CFG_IND_ADDR_MASK 0x00001ffc + +#define CFG_IND_DATA_OFFSET 0x124 + +#define CFG_ADDR_OFFSET 0x1f8 +#define CFG_ADDR_BUS_NUM_SHIFT 20 +#define CFG_ADDR_BUS_NUM_MASK 0x0ff00000 +#define CFG_ADDR_DEV_NUM_SHIFT 15 +#define CFG_ADDR_DEV_NUM_MASK 0x000f8000 +#define CFG_ADDR_FUNC_NUM_SHIFT 12 +#define CFG_ADDR_FUNC_NUM_MASK 0x00007000 +#define CFG_ADDR_REG_NUM_SHIFT 2 +#define CFG_ADDR_REG_NUM_MASK 0x00000ffc +#define CFG_ADDR_CFG_TYPE_SHIFT 0 +#define CFG_ADDR_CFG_TYPE_MASK 0x00000003 + +#define CFG_DATA_OFFSET 0x1fc + +#define SYS_RC_INTX_EN 0x330 +#define SYS_RC_INTX_MASK 0xf + +static inline struct iproc_pcie *sys_to_pcie(struct pci_sys_data *sys) +{ + return sys->private_data; +} + +#define INVALID_ACCESS_OFFSET 0xffffffff +static u32 iproc_pcie_cfg_base(struct iproc_pcie *pcie, int busno, + unsigned int devfn, int where) +{ + int slot = PCI_SLOT(devfn); + int fn = PCI_FUNC(devfn); + u32 val; + + /* root complex access */ + if (busno == 0) { + if (slot >= 1) + return INVALID_ACCESS_OFFSET; + writel(where & CFG_IND_ADDR_MASK, + pcie->base + CFG_IND_ADDR_OFFSET); + return CFG_IND_DATA_OFFSET; + } + + if (fn > 1) + return INVALID_ACCESS_OFFSET; + + /* EP device access */ + val = (busno << CFG_ADDR_BUS_NUM_SHIFT) | + (slot << CFG_ADDR_DEV_NUM_SHIFT) | + (fn << CFG_ADDR_FUNC_NUM_SHIFT) | + (where & CFG_ADDR_REG_NUM_MASK) | + (1 & CFG_ADDR_CFG_TYPE_MASK); + writel(val, pcie->base + CFG_ADDR_OFFSET); + + return CFG_DATA_OFFSET; +} + +#define INVALID_CFG_RD 0xffffffff +static int iproc_pcie_read_conf(struct iproc_pcie *pcie, int busno, + unsigned int devfn, int where, int size, + u32 *val) +{ + u32 offset, mask, shift; + + *val = INVALID_CFG_RD; + + if (size != 1 && size != 2 && size != 4) + return PCIBIOS_BAD_REGISTER_NUMBER; + else if ((size == 2) && (where & 1)) + return PCIBIOS_BAD_REGISTER_NUMBER; + else if ((size == 4) && (where & 3)) + return PCIBIOS_BAD_REGISTER_NUMBER; + + offset = iproc_pcie_cfg_base(pcie, busno, devfn, where); + if (offset == INVALID_ACCESS_OFFSET) + return PCIBIOS_BAD_REGISTER_NUMBER; + + *val = readl(pcie->base + offset); + + if (size != 4) { + mask = (1 << (size * BITS_PER_BYTE)) - 1; + shift = (where % 4) * BITS_PER_BYTE; + *val = (*val >> shift) & mask; + } + + dev_dbg(pcie->dev, + "conf rd: busn=%d devfn=%u where=%d size=%d val=0x%08x\n", + busno, devfn, where, size, *val); + + return PCIBIOS_SUCCESSFUL; +} + +static int iproc_pcie_write_conf(struct iproc_pcie *pcie, int busno, + unsigned int devfn, int where, int size, + u32 val) +{ + u32 offset, mask, shift, data; + + if (size != 1 && size != 2 && size != 4) + return PCIBIOS_BAD_REGISTER_NUMBER; + else if ((size == 2) && (where & 1)) + return PCIBIOS_BAD_REGISTER_NUMBER; + else if ((size == 4) && (where & 3)) + return PCIBIOS_BAD_REGISTER_NUMBER; + + offset = iproc_pcie_cfg_base(pcie, busno, devfn, where); + if (offset == INVALID_ACCESS_OFFSET) + return PCIBIOS_BAD_REGISTER_NUMBER; + + data = readl(pcie->base + offset); + + if (size != 4) { + mask = (1 << (size * BITS_PER_BYTE)) - 1; + shift = (where % 4) * BITS_PER_BYTE; + data &= ~(mask << shift); + data |= (val & mask) << shift; + } else { + data = val; + } + + writel(data, pcie->base + offset); + + dev_dbg(pcie->dev, + "conf wr: busn=%d devfn=%u where=%d size=%d data=0x%08x\n", + busno, devfn, where, size, data); + + return PCIBIOS_SUCCESSFUL; +} + +static int iproc_pcie_read(struct pci_bus *bus, unsigned int devfn, + int where, int size, u32 *val) +{ + struct iproc_pcie *pcie = sys_to_pcie(bus->sysdata); + + return iproc_pcie_read_conf(pcie, bus->number, devfn, where, size, + val); +} + +static int iproc_pcie_write(struct pci_bus *bus, unsigned int devfn, + int where, int size, u32 val) +{ + struct iproc_pcie *pcie = sys_to_pcie(bus->sysdata); + + return iproc_pcie_write_conf(pcie, bus->number, devfn, where, size, + val); +} + +static struct pci_ops iproc_pcie_ops = { + .read = iproc_pcie_read, + .write = iproc_pcie_write, +}; + +static void iproc_pcie_reset(struct iproc_pcie *pcie) +{ + u32 val; + + /* + * Configure the PCIe controller as root complex and send a downstream + * reset + */ + val = EP_MODE_SURVIVE_PERST | RC_PCIE_RST_OUTPUT; + writel(val, pcie->base + CLK_CONTROL_OFFSET); + udelay(250); + val &= ~EP_MODE_SURVIVE_PERST; + writel(val, pcie->base + CLK_CONTROL_OFFSET); + msleep(250); +} + +static int iproc_pcie_check_link(struct iproc_pcie *pcie, struct pci_bus *bus) +{ + u8 hdr_type; + u32 link_ctrl; + u16 pos, link_status; + int link_is_active = 0; + + /* make sure we are not in EP mode */ + pci_bus_read_config_byte(bus, 0, PCI_HEADER_TYPE, &hdr_type); + if ((hdr_type & 0x7f) != PCI_HEADER_TYPE_BRIDGE) { + dev_err(pcie->dev, "in EP mode, hdr=0x08%x\n", hdr_type); + return -EFAULT; + } + + /* force class to PCI_CLASS_BRIDGE_PCI (0x0604) */ + pci_bus_write_config_word(bus, 0, PCI_CLASS_DEVICE, + PCI_CLASS_BRIDGE_PCI); + + /* check link status to see if link is active */ + pos = pci_bus_find_capability(bus, 0, PCI_CAP_ID_EXP); + pci_bus_read_config_word(bus, 0, pos + PCI_EXP_LNKSTA, &link_status); + if (link_status & PCI_EXP_LNKSTA_NLW) + link_is_active = 1; + + if (!link_is_active) { + /* try GEN 1 link speed */ +#define PCI_LINK_STATUS_CTRL_2_OFFSET 0x0dc +#define PCI_TARGET_LINK_SPEED_MASK 0xf +#define PCI_TARGET_LINK_SPEED_GEN2 0x2 +#define PCI_TARGET_LINK_SPEED_GEN1 0x1 + pci_bus_read_config_dword(bus, 0, + PCI_LINK_STATUS_CTRL_2_OFFSET, + &link_ctrl); + if ((link_ctrl & PCI_TARGET_LINK_SPEED_MASK) == + PCI_TARGET_LINK_SPEED_GEN2) { + link_ctrl &= ~PCI_TARGET_LINK_SPEED_MASK; + link_ctrl |= PCI_TARGET_LINK_SPEED_GEN1; + pci_bus_write_config_dword(bus, 0, + PCI_LINK_STATUS_CTRL_2_OFFSET, + link_ctrl); + msleep(100); + + pos = pci_bus_find_capability(bus, 0, PCI_CAP_ID_EXP); + pci_bus_read_config_word(bus, 0, pos + PCI_EXP_LNKSTA, + &link_status); + if (link_status & PCI_EXP_LNKSTA_NLW) + link_is_active = 1; + } + } + + dev_info(pcie->dev, "link: %s\n", link_is_active ? "UP" : "DOWN"); + + return link_is_active ? 0 : -ENODEV; +} + +static void iproc_pcie_enable(struct iproc_pcie *pcie) +{ + writel(SYS_RC_INTX_MASK, pcie->base + SYS_RC_INTX_EN); +} + +int iproc_pcie_setup(struct iproc_pcie *pcie) +{ + int ret; + struct pci_bus *bus; + + if (!pcie || !pcie->dev || !pcie->base) + return -EINVAL; + + if (pcie->phy) { + ret = phy_init(pcie->phy); + if (ret) { + dev_err(pcie->dev, "unable to initialize PCIe PHY\n"); + return ret; + } + + ret = phy_power_on(pcie->phy); + if (ret) { + dev_err(pcie->dev, "unable to power on PCIe PHY\n"); + goto err_exit_phy; + } + + } + + iproc_pcie_reset(pcie); + + pcie->sysdata.private_data = pcie; + + bus = pci_create_root_bus(pcie->dev, 0, &iproc_pcie_ops, + &pcie->sysdata, pcie->resources); + if (!bus) { + dev_err(pcie->dev, "unable to create PCI root bus\n"); + ret = -ENOMEM; + goto err_power_off_phy; + } + + ret = iproc_pcie_check_link(pcie, bus); + if (ret) { + dev_err(pcie->dev, "no PCIe EP device detected\n"); + goto err_rm_root_bus; + } + + iproc_pcie_enable(pcie); + + pci_scan_child_bus(bus); + pci_assign_unassigned_bus_resources(bus); + pci_bus_add_devices(bus); + + pci_fixup_irqs(pci_common_swizzle, of_irq_parse_and_map_pci); + + return 0; + +err_rm_root_bus: + pci_lock_rescan_remove(); + pci_stop_root_bus(bus); + pci_remove_root_bus(bus); + pci_unlock_rescan_remove(); + +err_power_off_phy: + if (pcie->phy) + phy_power_off(pcie->phy); +err_exit_phy: + if (pcie->phy) + phy_exit(pcie->phy); + + return ret; +} + +MODULE_AUTHOR("Ray Jui <rjui@broadcom.com>"); +MODULE_DESCRIPTION("Broadcom iPROC PCIe common driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/pci/host/pcie-iproc.h b/drivers/pci/host/pcie-iproc.h new file mode 100644 index 0000000..4e2437e --- /dev/null +++ b/drivers/pci/host/pcie-iproc.h @@ -0,0 +1,39 @@ +/* + * Copyright (C) 2014-2015 Broadcom Corporation + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation version 2. + * + * This program is distributed "as is" WITHOUT ANY WARRANTY of any + * kind, whether express or implied; without even the implied warranty + * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + */ + +#ifndef _PCIE_IPROC_H +#define _PCIE_IPROC_H + +#define IPROC_PCIE_MAX_NUM_IRQS 6 + +/** + * iProc PCIe device + * @dev: pointer to device data structure + * @base: PCIe host controller I/O register base + * @resources: linked list of all PCI resources + * @sysdata: Per PCI controller data + * @phy: optional PHY device that controls the Serdes + * @irqs: interrupt IDs + */ +struct iproc_pcie { + struct device *dev; + void __iomem *base; + struct list_head *resources; + struct pci_sys_data sysdata; + struct phy *phy; + int irqs[IPROC_PCIE_MAX_NUM_IRQS]; +}; + +extern int iproc_pcie_setup(struct iproc_pcie *pcie); + +#endif /* _PCIE_IPROC_H */