| Message ID | 20211126222700.862407977@linutronix.de (mailing list archive) |
|---|---|
| Headers | show |
| Series | genirq/msi, PCI/MSI: Spring cleaning - Part 1 | expand |
On Sat, Nov 27, 2021 at 02:18:34AM +0100, Thomas Gleixner wrote: > The [PCI] MSI code has gained quite some warts over time. A recent > discussion unearthed a shortcoming: the lack of support for expanding > PCI/MSI-X vectors after initialization of MSI-X. > > PCI/MSI-X has no requirement to setup all vectors when MSI-X is enabled in > the device. The non-used vectors have just to be masked in the vector > table. For PCI/MSI this is not possible because the number of vectors > cannot be changed after initialization. > > The PCI/MSI code, but also the core MSI irq domain code are built around > the assumption that all required vectors are installed at initialization > time and freed when the device is shut down by the driver. > > Supporting dynamic expansion at least for MSI-X is important for VFIO so > that the host side interrupts for passthrough devices can be installed on > demand. > > This is the first part of a large (total 101 patches) series which > refactors the [PCI]MSI infrastructure to make runtime expansion of MSI-X > vectors possible. The last part (10 patches) provide this functionality. > > The first part is mostly a cleanup which consolidates code, moves the PCI > MSI code into a separate directory and splits it up into several parts. > > No functional change intended except for patch 2/N which changes the > behaviour of pci_get_vector()/affinity() to get rid of the assumption that > the provided index is the "index" into the descriptor list instead of using > it as the actual MSI[X] index as seen by the hardware. This would break > users of sparse allocated MSI-X entries, but non of them use these > functions. I don't know all the irqdomain stuff all that well anymore, but I read through all the patches and only noticed a small spello [patch 02/22] PCI/MSI: Fix pci_irq_vector()/pci_irq_get_attinity() ^^^^ ff It all seems good, I especially like the splitting of msi.c and removal of ops.. Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Thanks, Jason
On 27.11.21 02:18, Thomas Gleixner wrote: > The [PCI] MSI code has gained quite some warts over time. A recent > discussion unearthed a shortcoming: the lack of support for expanding > PCI/MSI-X vectors after initialization of MSI-X. > > PCI/MSI-X has no requirement to setup all vectors when MSI-X is enabled in > the device. The non-used vectors have just to be masked in the vector > table. For PCI/MSI this is not possible because the number of vectors > cannot be changed after initialization. > > The PCI/MSI code, but also the core MSI irq domain code are built around > the assumption that all required vectors are installed at initialization > time and freed when the device is shut down by the driver. > > Supporting dynamic expansion at least for MSI-X is important for VFIO so > that the host side interrupts for passthrough devices can be installed on > demand. > > This is the first part of a large (total 101 patches) series which > refactors the [PCI]MSI infrastructure to make runtime expansion of MSI-X > vectors possible. The last part (10 patches) provide this functionality. > > The first part is mostly a cleanup which consolidates code, moves the PCI > MSI code into a separate directory and splits it up into several parts. > > No functional change intended except for patch 2/N which changes the > behaviour of pci_get_vector()/affinity() to get rid of the assumption that > the provided index is the "index" into the descriptor list instead of using > it as the actual MSI[X] index as seen by the hardware. This would break > users of sparse allocated MSI-X entries, but non of them use these > functions. > > This series is based on 5.16-rc2 and also available via git: > > git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git msi-v1-part-1 Tested with Xen (PV dom0, PV guest, PVH guest, HVM guest). You can add my: Tested-by: Juergen Gross <jgross@suse.com> Juergen
On 11/27/21 02:18, Thomas Gleixner wrote: > The [PCI] MSI code has gained quite some warts over time. A recent > discussion unearthed a shortcoming: the lack of support for expanding > PCI/MSI-X vectors after initialization of MSI-X. > > PCI/MSI-X has no requirement to setup all vectors when MSI-X is enabled in > the device. The non-used vectors have just to be masked in the vector > table. For PCI/MSI this is not possible because the number of vectors > cannot be changed after initialization. > > The PCI/MSI code, but also the core MSI irq domain code are built around > the assumption that all required vectors are installed at initialization > time and freed when the device is shut down by the driver. > > Supporting dynamic expansion at least for MSI-X is important for VFIO so > that the host side interrupts for passthrough devices can be installed on > demand. > > This is the first part of a large (total 101 patches) series which > refactors the [PCI]MSI infrastructure to make runtime expansion of MSI-X > vectors possible. The last part (10 patches) provide this functionality. > > The first part is mostly a cleanup which consolidates code, moves the PCI > MSI code into a separate directory and splits it up into several parts. > > No functional change intended except for patch 2/N which changes the > behaviour of pci_get_vector()/affinity() to get rid of the assumption that > the provided index is the "index" into the descriptor list instead of using > it as the actual MSI[X] index as seen by the hardware. This would break > users of sparse allocated MSI-X entries, but non of them use these > functions. > > This series is based on 5.16-rc2 and also available via git: > > git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git msi-v1-part-1 > > For the curious who can't wait for the next part to arrive the full series > is available via: > > git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git msi-v1-part-4 After fixing the compile failures, I didn't see any regressions on these platforms : PowerNV, pSeries under KVM and PowerVM, using POWER8/9 processors. Thanks, C. > Thanks, > > tglx > --- > arch/powerpc/platforms/4xx/msi.c | 281 ------------ > b/Documentation/driver-api/pci/pci.rst | 2 > b/arch/mips/pci/msi-octeon.c | 32 - > b/arch/powerpc/platforms/4xx/Makefile | 1 > b/arch/powerpc/platforms/cell/axon_msi.c | 2 > b/arch/powerpc/platforms/powernv/pci-ioda.c | 4 > b/arch/powerpc/platforms/pseries/msi.c | 6 > b/arch/powerpc/sysdev/Kconfig | 6 > b/arch/s390/pci/pci_irq.c | 4 > b/arch/sparc/kernel/pci_msi.c | 4 > b/arch/x86/hyperv/irqdomain.c | 55 -- > b/arch/x86/include/asm/x86_init.h | 6 > b/arch/x86/include/asm/xen/hypervisor.h | 8 > b/arch/x86/kernel/apic/msi.c | 8 > b/arch/x86/kernel/x86_init.c | 12 > b/arch/x86/pci/xen.c | 19 > b/drivers/irqchip/irq-gic-v2m.c | 1 > b/drivers/irqchip/irq-gic-v3-its-pci-msi.c | 1 > b/drivers/irqchip/irq-gic-v3-mbi.c | 1 > b/drivers/net/wireless/ath/ath11k/pci.c | 2 > b/drivers/pci/Makefile | 3 > b/drivers/pci/msi/Makefile | 7 > b/drivers/pci/msi/irqdomain.c | 267 +++++++++++ > b/drivers/pci/msi/legacy.c | 79 +++ > b/drivers/pci/msi/msi.c | 645 ++++------------------------ > b/drivers/pci/msi/msi.h | 39 + > b/drivers/pci/msi/pcidev_msi.c | 43 + > b/drivers/pci/pci-sysfs.c | 7 > b/drivers/pci/xen-pcifront.c | 2 > b/include/linux/msi.h | 135 ++--- > b/include/linux/pci.h | 1 > b/kernel/irq/msi.c | 41 + > 32 files changed, 696 insertions(+), 1028 deletions(-) >
The [PCI] MSI code has gained quite some warts over time. A recent discussion unearthed a shortcoming: the lack of support for expanding PCI/MSI-X vectors after initialization of MSI-X. PCI/MSI-X has no requirement to setup all vectors when MSI-X is enabled in the device. The non-used vectors have just to be masked in the vector table. For PCI/MSI this is not possible because the number of vectors cannot be changed after initialization. The PCI/MSI code, but also the core MSI irq domain code are built around the assumption that all required vectors are installed at initialization time and freed when the device is shut down by the driver. Supporting dynamic expansion at least for MSI-X is important for VFIO so that the host side interrupts for passthrough devices can be installed on demand. This is the first part of a large (total 101 patches) series which refactors the [PCI]MSI infrastructure to make runtime expansion of MSI-X vectors possible. The last part (10 patches) provide this functionality. The first part is mostly a cleanup which consolidates code, moves the PCI MSI code into a separate directory and splits it up into several parts. No functional change intended except for patch 2/N which changes the behaviour of pci_get_vector()/affinity() to get rid of the assumption that the provided index is the "index" into the descriptor list instead of using it as the actual MSI[X] index as seen by the hardware. This would break users of sparse allocated MSI-X entries, but non of them use these functions. This series is based on 5.16-rc2 and also available via git: git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git msi-v1-part-1 For the curious who can't wait for the next part to arrive the full series is available via: git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git msi-v1-part-4 Thanks, tglx --- arch/powerpc/platforms/4xx/msi.c | 281 ------------ b/Documentation/driver-api/pci/pci.rst | 2 b/arch/mips/pci/msi-octeon.c | 32 - b/arch/powerpc/platforms/4xx/Makefile | 1 b/arch/powerpc/platforms/cell/axon_msi.c | 2 b/arch/powerpc/platforms/powernv/pci-ioda.c | 4 b/arch/powerpc/platforms/pseries/msi.c | 6 b/arch/powerpc/sysdev/Kconfig | 6 b/arch/s390/pci/pci_irq.c | 4 b/arch/sparc/kernel/pci_msi.c | 4 b/arch/x86/hyperv/irqdomain.c | 55 -- b/arch/x86/include/asm/x86_init.h | 6 b/arch/x86/include/asm/xen/hypervisor.h | 8 b/arch/x86/kernel/apic/msi.c | 8 b/arch/x86/kernel/x86_init.c | 12 b/arch/x86/pci/xen.c | 19 b/drivers/irqchip/irq-gic-v2m.c | 1 b/drivers/irqchip/irq-gic-v3-its-pci-msi.c | 1 b/drivers/irqchip/irq-gic-v3-mbi.c | 1 b/drivers/net/wireless/ath/ath11k/pci.c | 2 b/drivers/pci/Makefile | 3 b/drivers/pci/msi/Makefile | 7 b/drivers/pci/msi/irqdomain.c | 267 +++++++++++ b/drivers/pci/msi/legacy.c | 79 +++ b/drivers/pci/msi/msi.c | 645 ++++------------------------ b/drivers/pci/msi/msi.h | 39 + b/drivers/pci/msi/pcidev_msi.c | 43 + b/drivers/pci/pci-sysfs.c | 7 b/drivers/pci/xen-pcifront.c | 2 b/include/linux/msi.h | 135 ++--- b/include/linux/pci.h | 1 b/kernel/irq/msi.c | 41 + 32 files changed, 696 insertions(+), 1028 deletions(-)