| Message ID | 1473779140-4016-2-git-send-email-keith.busch@intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Delegated to: | Bjorn Helgaas |
| Headers | show |
On Tue, Sep 13, 2016 at 09:05:40AM -0600, Keith Busch wrote: > This patch adds a new function to set PCI domain specific options as > devices are added. The usage included in this patch is for LED indicator > control in VMD domains, but may be extended in the future as new domain > specific options are required. > > PCIe LED Slot Control in a VMD domain is repurposed to a non-standard > implementation. As such, all devices in a VMD domain will be flagged so > pciehp does not attempt to use LED indicators. This user_led flag > has pciehp provide a different sysfs entry for user exclusive control > over the domain's slot indicators. > > In order to determine if a bus is within a PCI domain, the patch appends > a bool to the pci_sysdata structure that the VMD driver sets during > initialization. > > Requested-by: Kapil Karkra <kapil.karkra@intel.com> > Tested-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com> > Signed-off-by: Keith Busch <keith.busch@intel.com> Applied on pci/hotplug for v4.9, thanks! I made the necessary changes to match the renaming I did in the first patch, and I also used plain old "#ifdef" instead of "#if IS_ENABLED" since the rest of the file uses the former style. If there's a reason to switch, we should change the whole file in a separate patch so we can explain the rationale. Please check it out and make sure everything you need made it in. > --- > > No change from previous version of this patch; just part of the series. > > arch/x86/include/asm/pci.h | 14 ++++++++++++++ > arch/x86/pci/common.c | 7 +++++++ > arch/x86/pci/vmd.c | 1 + > 3 files changed, 22 insertions(+) > > diff --git a/arch/x86/include/asm/pci.h b/arch/x86/include/asm/pci.h > index 9ab7507..1411dbe 100644 > --- a/arch/x86/include/asm/pci.h > +++ b/arch/x86/include/asm/pci.h > @@ -23,6 +23,9 @@ struct pci_sysdata { > #ifdef CONFIG_PCI_MSI_IRQ_DOMAIN > void *fwnode; /* IRQ domain for MSI assignment */ > #endif > +#if IS_ENABLED(CONFIG_VMD) > + bool vmd_domain; /* True if in Intel VMD domain */ > +#endif > }; > > extern int pci_routeirq; > @@ -56,6 +59,17 @@ static inline void *_pci_root_bus_fwnode(struct pci_bus *bus) > #define pci_root_bus_fwnode _pci_root_bus_fwnode > #endif > > +static inline bool is_vmd(struct pci_bus *bus) > +{ > +#if IS_ENABLED(CONFIG_VMD) > + struct pci_sysdata *sd = bus->sysdata; > + > + return sd->vmd_domain; > +#else > + return false; > +#endif > +} > + > /* Can be used to override the logic in pci_scan_bus for skipping > already-configured bus numbers - to be used for buggy BIOSes > or architectures with incomplete PCI setup by the loader */ > diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c > index 7b6a9d1..ccf696c 100644 > --- a/arch/x86/pci/common.c > +++ b/arch/x86/pci/common.c > @@ -677,6 +677,12 @@ static void set_dma_domain_ops(struct pci_dev *pdev) > static void set_dma_domain_ops(struct pci_dev *pdev) {} > #endif > > +static void set_dev_domain_options(struct pci_dev *pdev) > +{ > + if (is_vmd(pdev->bus)) > + pdev->user_leds = 1; > +} > + > int pcibios_add_device(struct pci_dev *dev) > { > struct setup_data *data; > @@ -707,6 +713,7 @@ int pcibios_add_device(struct pci_dev *dev) > iounmap(data); > } > set_dma_domain_ops(dev); > + set_dev_domain_options(dev); > return 0; > } > > diff --git a/arch/x86/pci/vmd.c b/arch/x86/pci/vmd.c > index b814ca6..a021b7b 100644 > --- a/arch/x86/pci/vmd.c > +++ b/arch/x86/pci/vmd.c > @@ -596,6 +596,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd) > .parent = res, > }; > > + sd->vmd_domain = true; > sd->domain = vmd_find_free_domain(); > if (sd->domain < 0) > return sd->domain; > -- > 2.7.2 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-pci" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Sep 23, 2016 at 09:34:41AM -0500, Bjorn Helgaas wrote: > I made the necessary changes to match the renaming I did in the first > patch, and I also used plain old "#ifdef" instead of "#if IS_ENABLED" > since the rest of the file uses the former style. If there's a reason > to switch, we should change the whole file in a separate patch so we > can explain the rationale. The check was "IS_ENABLED" because VMD can be a loadable module, which fails the ifdef check. I see Fengguang 0'dayed it using the module configuration option. I can send you a fix based on your pci/hotplug branch, or revert and apply the original patch if you prefer. BTW, you had asked me not to split a series when incremental fixes touched a single patch. I didn't resend the whole series here, and while you got the right patches, I apologize for making it more difficult to find. -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Sep 23, 2016 at 12:57:02PM -0400, Keith Busch wrote: > On Fri, Sep 23, 2016 at 09:34:41AM -0500, Bjorn Helgaas wrote: > > I made the necessary changes to match the renaming I did in the first > > patch, and I also used plain old "#ifdef" instead of "#if IS_ENABLED" > > since the rest of the file uses the former style. If there's a reason > > to switch, we should change the whole file in a separate patch so we > > can explain the rationale. > > The check was "IS_ENABLED" because VMD can be a loadable module, which > fails the ifdef check. I see Fengguang 0'dayed it using the module > configuration option. I can send you a fix based on your pci/hotplug > branch, or revert and apply the original patch if you prefer. I didn't realize VMD could be a loadable module, and I didn't realize that would make a difference for the config symbol. BTW, the "Volume Management Device Driver" config item appears by itself in the top-level menuconfig menu. That seems a little ... presumptuous; is it what you intended? It took me a while, but I did eventually figure out why #ifdef doesn't work -- we generate a different include/generated/autoconf.h symbol for modules: built-in loadable module --------------------- --------------------------- .config CONFIG_VMD=y CONFIG_VMD=m autoconf.h #define CONFIG_VMD 1 #define CONFIG_VMD_MODULE 1 Anyway, I fixed it by using IS_ENABLED() again. I might propose a comment update to help anybody else who stumbles over this. It was kind of annoying to puzzle this out. > BTW, you had asked me not to split a series when incremental fixes > touched a single patch. I didn't resend the whole series here, and while > you got the right patches, I apologize for making it more difficult to find. No problem, I was just paying more attention this time :) Except for IS_ENABLED(), anyway. Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Fri, Sep 23, 2016 at 02:12:23PM -0500, Bjorn Helgaas wrote: > BTW, the "Volume Management Device Driver" config item appears by > itself in the top-level menuconfig menu. That seems a little ... > presumptuous; is it what you intended? Not really intended, but I didn't really know any better at the time. :). I think drivers/pci/host/ is a more appropriate home for this driver. I'll send a patch to relocate it, along with necessary Kconfig updates. -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[+cc Nirmal, Jim, Paul, Blazej] On Tue, Sep 13, 2016 at 09:05:40AM -0600, Keith Busch wrote: > This patch adds a new function to set PCI domain specific options as > devices are added. The usage included in this patch is for LED indicator > control in VMD domains, but may be extended in the future as new domain > specific options are required. > > PCIe LED Slot Control in a VMD domain is repurposed to a non-standard > implementation. As such, all devices in a VMD domain will be flagged so > pciehp does not attempt to use LED indicators. This user_led flag > has pciehp provide a different sysfs entry for user exclusive control > over the domain's slot indicators. > > In order to determine if a bus is within a PCI domain, the patch appends > a bool to the pci_sysdata structure that the VMD driver sets during > initialization. This eventually turned into https://git.kernel.org/linus/3161832d58c7 ("x86/PCI: VMD: Request userspace control of PCIe hotplug indicators") More questions about this, prompted by Blazej's recent regression report: https://lore.kernel.org/r/20240722141440.7210-1-blazej.kucman@intel.com I assume this patch was prompted by NVMe devices behind a VMD? And the non-standard slot indicator usage is specifically related to VMD Root Ports? Isn't it possible to add non-NVMe devices behind VMD, e.g., a switch in an external enclosure where pciehp manages a switch Downstream Port with standard slot indicators? I'm wondering if pdev->hotplug_user_indicators should be more narrowly targeted to just VMD Root Ports. If there's any possibility of a Downstream Port behind VMD with standard indicators, users are going to be very confused when the sysfs "attention" file is basically backwards from normal. IIUC writing 0 to "attention" when hotplug_user_indicators is set writes 0 ("reserved") to AIC, when it would otherwise write 11b ("off"). I'm also wondering whether there's a way to do this in the vmd driver instead of in arch/x86/pci/common.c, but that's a secondary question. > arch/x86/include/asm/pci.h | 14 ++++++++++++++ > arch/x86/pci/common.c | 7 +++++++ > arch/x86/pci/vmd.c | 1 + > 3 files changed, 22 insertions(+) > > diff --git a/arch/x86/include/asm/pci.h b/arch/x86/include/asm/pci.h > index 9ab7507..1411dbe 100644 > --- a/arch/x86/include/asm/pci.h > +++ b/arch/x86/include/asm/pci.h > @@ -23,6 +23,9 @@ struct pci_sysdata { > #ifdef CONFIG_PCI_MSI_IRQ_DOMAIN > void *fwnode; /* IRQ domain for MSI assignment */ > #endif > +#if IS_ENABLED(CONFIG_VMD) > + bool vmd_domain; /* True if in Intel VMD domain */ > +#endif > }; > > extern int pci_routeirq; > @@ -56,6 +59,17 @@ static inline void *_pci_root_bus_fwnode(struct pci_bus *bus) > #define pci_root_bus_fwnode _pci_root_bus_fwnode > #endif > > +static inline bool is_vmd(struct pci_bus *bus) > +{ > +#if IS_ENABLED(CONFIG_VMD) > + struct pci_sysdata *sd = bus->sysdata; > + > + return sd->vmd_domain; > +#else > + return false; > +#endif > +} > + > /* Can be used to override the logic in pci_scan_bus for skipping > already-configured bus numbers - to be used for buggy BIOSes > or architectures with incomplete PCI setup by the loader */ > diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c > index 7b6a9d1..ccf696c 100644 > --- a/arch/x86/pci/common.c > +++ b/arch/x86/pci/common.c > @@ -677,6 +677,12 @@ static void set_dma_domain_ops(struct pci_dev *pdev) > static void set_dma_domain_ops(struct pci_dev *pdev) {} > #endif > > +static void set_dev_domain_options(struct pci_dev *pdev) > +{ > + if (is_vmd(pdev->bus)) > + pdev->user_leds = 1; > +} > + > int pcibios_add_device(struct pci_dev *dev) > { > struct setup_data *data; > @@ -707,6 +713,7 @@ int pcibios_add_device(struct pci_dev *dev) > iounmap(data); > } > set_dma_domain_ops(dev); > + set_dev_domain_options(dev); > return 0; > } > > diff --git a/arch/x86/pci/vmd.c b/arch/x86/pci/vmd.c > index b814ca6..a021b7b 100644 > --- a/arch/x86/pci/vmd.c > +++ b/arch/x86/pci/vmd.c > @@ -596,6 +596,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd) > .parent = res, > }; > > + sd->vmd_domain = true; > sd->domain = vmd_find_free_domain(); > if (sd->domain < 0) > return sd->domain; > -- > 2.7.2
diff --git a/arch/x86/include/asm/pci.h b/arch/x86/include/asm/pci.h index 9ab7507..1411dbe 100644 --- a/arch/x86/include/asm/pci.h +++ b/arch/x86/include/asm/pci.h @@ -23,6 +23,9 @@ struct pci_sysdata { #ifdef CONFIG_PCI_MSI_IRQ_DOMAIN void *fwnode; /* IRQ domain for MSI assignment */ #endif +#if IS_ENABLED(CONFIG_VMD) + bool vmd_domain; /* True if in Intel VMD domain */ +#endif }; extern int pci_routeirq; @@ -56,6 +59,17 @@ static inline void *_pci_root_bus_fwnode(struct pci_bus *bus) #define pci_root_bus_fwnode _pci_root_bus_fwnode #endif +static inline bool is_vmd(struct pci_bus *bus) +{ +#if IS_ENABLED(CONFIG_VMD) + struct pci_sysdata *sd = bus->sysdata; + + return sd->vmd_domain; +#else + return false; +#endif +} + /* Can be used to override the logic in pci_scan_bus for skipping already-configured bus numbers - to be used for buggy BIOSes or architectures with incomplete PCI setup by the loader */ diff --git a/arch/x86/pci/common.c b/arch/x86/pci/common.c index 7b6a9d1..ccf696c 100644 --- a/arch/x86/pci/common.c +++ b/arch/x86/pci/common.c @@ -677,6 +677,12 @@ static void set_dma_domain_ops(struct pci_dev *pdev) static void set_dma_domain_ops(struct pci_dev *pdev) {} #endif +static void set_dev_domain_options(struct pci_dev *pdev) +{ + if (is_vmd(pdev->bus)) + pdev->user_leds = 1; +} + int pcibios_add_device(struct pci_dev *dev) { struct setup_data *data; @@ -707,6 +713,7 @@ int pcibios_add_device(struct pci_dev *dev) iounmap(data); } set_dma_domain_ops(dev); + set_dev_domain_options(dev); return 0; } diff --git a/arch/x86/pci/vmd.c b/arch/x86/pci/vmd.c index b814ca6..a021b7b 100644 --- a/arch/x86/pci/vmd.c +++ b/arch/x86/pci/vmd.c @@ -596,6 +596,7 @@ static int vmd_enable_domain(struct vmd_dev *vmd) .parent = res, }; + sd->vmd_domain = true; sd->domain = vmd_find_free_domain(); if (sd->domain < 0) return sd->domain;