mbox series

[V5,00/13] Introduce vfio_pci_core subsystem

Message ID 20210826103912.128972-1-yishaih@nvidia.com (mailing list archive)
Headers show
Series Introduce vfio_pci_core subsystem | expand

Message

Yishai Hadas Aug. 26, 2021, 10:38 a.m. UTC 
From Max Gurtovoy:
====================
This series splits the vfio_pci driver into two parts, a PCI driver and
a subsystem driver that will also be library of code. The main PCI
driver, vfio_pci.ko, will remain as before and it will use the library
module vfio_pci_core.ko to help create the vfio_device.

This series is intended to solve the issues that were raised in the
previous attempts for extending vfio-pci for device specific
functionality:

1.
https://lore.kernel.org/kvm/20200518024202.13996-1-yan.y.zhao@intel.com
   by Yan Zhao
2.
https://lore.kernel.org/kvm/20210702095849.1610-1-shameerali.kolothum.thodi@huawei.com
   by Longfang Liu

Also to support proposed future changes to virtio and other common
protocols to support migration:

https://lists.oasis-open.org/archives/virtio-comment/202106/msg00044.html

This subsystem framework will also ease adding new device specific
functionality to VFIO devices in the future by allowing another module
to provide the pci_driver that can setup a number of details before
registering to the VFIO subsystem, such as injecting its own operations.

This series also extends the "driver_override" mechanism. A flag is
added for PCI drivers that will declare themselves as "driver_override"
capable which sends their match table to the modules.alias file but
otherwise leaves them outside of the normal driver core auto-binding
world, like vfio_pci.

In order to get the best match for "driver_override" drivers, one can
create a userspace program to inspect the modules.alias, an example can
be found at:

https://github.com/maxgurtovoy/linux_tools/blob/main/vfio/bind_vfio_pci_driver.py

Which finds the 'best match' according to a simple algorithm: "the
driver with the fewest '*' matches wins."

For example, the vfio-pci driver will match to any pci device. So it
will have the maximal '*' matches.

In case we are looking for a match to a mlx5 based device, we'll have a
match to vfio-pci.ko and mlx5-vfio-pci.ko. We'll prefer mlx5-vfio-pci.ko
since it will have less '*' matches (probably vendor and device IDs will
match). This will work in the future for NVMe/Virtio devices that can
match according to a class code or other criteria.

v5:
- Add Reviewed-by: Christoph Hellwig and Acked-by: Bjorn Helgaas to the
  relevant patches.

Patch #9:
- Rephrase commit log as was suggested by Bjorn Helgaas.

Patch #10:
- Use a switch statement to enforce a valid override_only value as was
  asked by Alex Williamson.

v4:
Patch #6:
- Delete might_sleep() from the vfio_pci_sriov_configure() path as the
  annotation through mutex_lock() which is used down the road is enough
  on its own.

Patch #13:
- Add include/linux/vfio_pci_core.h to the MAINTAINERS file as was
  previously asked by Alex Williamson.

v3:
Patch #6:
- Upon error flow, print PF driver name instead of hard-coded vfio-pci.
Patch #9:
- Split into two patches and follow the notes given by Bjorn Helgaas.

v2:
Patch #6:
- Drop DRIVER_VERSION as it's useless and not required any more.

Patch #9:
- Follow Bjorn Helgaas suggestion to enable having "vfio_" prefix in
  modules.alias file without the unnecessary VFIO connection in
  pci_match_device.

- Add the sequence of commands/algorithm that is required by
  userspace to discover the matching driver to the commit message to let
  the patch documentation be self-contained.

Patch #12:
- Save compatibility with Kconfig as was asked in the mailing list.
- Drop DRIVER_VERSION as it's useless and not required any more.

Yishai

Jason Gunthorpe (2):
  vfio: Use select for eventfd
  vfio: Use kconfig if XX/endif blocks instead of repeating 'depends on'

Max Gurtovoy (10):
  vfio/pci: Rename vfio_pci.c to vfio_pci_core.c
  vfio/pci: Rename vfio_pci_private.h to vfio_pci_core.h
  vfio/pci: Rename vfio_pci_device to vfio_pci_core_device
  vfio/pci: Rename ops functions to fit core namings
  vfio/pci: Include vfio header in vfio_pci_core.h
  vfio/pci: Split the pci_driver code out of vfio_pci_core.c
  vfio/pci: Move igd initialization to vfio_pci.c
  PCI: Add 'override_only' field to struct pci_device_id
  PCI / VFIO: Add 'override_only' support for VFIO PCI sub system
  vfio/pci: Introduce vfio_pci_core.ko

Yishai Hadas (1):
  vfio/pci: Move module parameters to vfio_pci.c

 Documentation/PCI/pci.rst                     |    1 +
 MAINTAINERS                                   |    1 +
 drivers/pci/pci-driver.c                      |   28 +-
 drivers/vfio/Kconfig                          |   29 +-
 drivers/vfio/fsl-mc/Kconfig                   |    3 +-
 drivers/vfio/mdev/Kconfig                     |    1 -
 drivers/vfio/pci/Kconfig                      |   40 +-
 drivers/vfio/pci/Makefile                     |    8 +-
 drivers/vfio/pci/vfio_pci.c                   | 2262 +----------------
 drivers/vfio/pci/vfio_pci_config.c            |   70 +-
 drivers/vfio/pci/vfio_pci_core.c              | 2158 ++++++++++++++++
 drivers/vfio/pci/vfio_pci_igd.c               |   19 +-
 drivers/vfio/pci/vfio_pci_intrs.c             |   42 +-
 drivers/vfio/pci/vfio_pci_rdwr.c              |   18 +-
 drivers/vfio/pci/vfio_pci_zdev.c              |    4 +-
 drivers/vfio/platform/Kconfig                 |    6 +-
 drivers/vfio/platform/reset/Kconfig           |    4 +-
 include/linux/mod_devicetable.h               |    6 +
 include/linux/pci.h                           |   29 +
 .../linux/vfio_pci_core.h                     |   89 +-
 scripts/mod/devicetable-offsets.c             |    1 +
 scripts/mod/file2alias.c                      |   17 +-
 22 files changed, 2525 insertions(+), 2311 deletions(-)
 create mode 100644 drivers/vfio/pci/vfio_pci_core.c
 rename drivers/vfio/pci/vfio_pci_private.h => include/linux/vfio_pci_core.h (56%)

Comments

Alex Williamson Aug. 26, 2021, 10:21 p.m. UTC | #1 
On Thu, 26 Aug 2021 13:38:59 +0300
Yishai Hadas <yishaih@nvidia.com> wrote:
> 
> Jason Gunthorpe (2):
>   vfio: Use select for eventfd
>   vfio: Use kconfig if XX/endif blocks instead of repeating 'depends on'
> 
> Max Gurtovoy (10):
>   vfio/pci: Rename vfio_pci.c to vfio_pci_core.c
>   vfio/pci: Rename vfio_pci_private.h to vfio_pci_core.h
>   vfio/pci: Rename vfio_pci_device to vfio_pci_core_device
>   vfio/pci: Rename ops functions to fit core namings
>   vfio/pci: Include vfio header in vfio_pci_core.h
>   vfio/pci: Split the pci_driver code out of vfio_pci_core.c
>   vfio/pci: Move igd initialization to vfio_pci.c
>   PCI: Add 'override_only' field to struct pci_device_id
>   PCI / VFIO: Add 'override_only' support for VFIO PCI sub system
>   vfio/pci: Introduce vfio_pci_core.ko
> 
> Yishai Hadas (1):
>   vfio/pci: Move module parameters to vfio_pci.c
> 
>  Documentation/PCI/pci.rst                     |    1 +
>  MAINTAINERS                                   |    1 +
>  drivers/pci/pci-driver.c                      |   28 +-
>  drivers/vfio/Kconfig                          |   29 +-
>  drivers/vfio/fsl-mc/Kconfig                   |    3 +-
>  drivers/vfio/mdev/Kconfig                     |    1 -
>  drivers/vfio/pci/Kconfig                      |   40 +-
>  drivers/vfio/pci/Makefile                     |    8 +-
>  drivers/vfio/pci/vfio_pci.c                   | 2262 +----------------
>  drivers/vfio/pci/vfio_pci_config.c            |   70 +-
>  drivers/vfio/pci/vfio_pci_core.c              | 2158 ++++++++++++++++
>  drivers/vfio/pci/vfio_pci_igd.c               |   19 +-
>  drivers/vfio/pci/vfio_pci_intrs.c             |   42 +-
>  drivers/vfio/pci/vfio_pci_rdwr.c              |   18 +-
>  drivers/vfio/pci/vfio_pci_zdev.c              |    4 +-
>  drivers/vfio/platform/Kconfig                 |    6 +-
>  drivers/vfio/platform/reset/Kconfig           |    4 +-
>  include/linux/mod_devicetable.h               |    6 +
>  include/linux/pci.h                           |   29 +
>  .../linux/vfio_pci_core.h                     |   89 +-
>  scripts/mod/devicetable-offsets.c             |    1 +
>  scripts/mod/file2alias.c                      |   17 +-
>  22 files changed, 2525 insertions(+), 2311 deletions(-)
>  create mode 100644 drivers/vfio/pci/vfio_pci_core.c
>  rename drivers/vfio/pci/vfio_pci_private.h => include/linux/vfio_pci_core.h (56%)

Applied to vfio next branch for v5.15.  Thanks,

Alex
Xu, Terrence Aug. 30, 2021, 3:40 p.m. UTC | #2 
> -----Original Message-----
> From: Alex Williamson <alex.williamson@redhat.com>
> Sent: Friday, August 27, 2021 6:21 AM
> 
> On Thu, 26 Aug 2021 13:38:59 +0300
> Yishai Hadas <yishaih@nvidia.com> wrote:
> >
> > Jason Gunthorpe (2):
> >   vfio: Use select for eventfd
> >   vfio: Use kconfig if XX/endif blocks instead of repeating 'depends on'
> >
> > Max Gurtovoy (10):
> >   vfio/pci: Rename vfio_pci.c to vfio_pci_core.c
> >   vfio/pci: Rename vfio_pci_private.h to vfio_pci_core.h
> >   vfio/pci: Rename vfio_pci_device to vfio_pci_core_device
> >   vfio/pci: Rename ops functions to fit core namings
> >   vfio/pci: Include vfio header in vfio_pci_core.h
> >   vfio/pci: Split the pci_driver code out of vfio_pci_core.c
> >   vfio/pci: Move igd initialization to vfio_pci.c
> >   PCI: Add 'override_only' field to struct pci_device_id
> >   PCI / VFIO: Add 'override_only' support for VFIO PCI sub system
> >   vfio/pci: Introduce vfio_pci_core.ko
> >
> > Yishai Hadas (1):
> >   vfio/pci: Move module parameters to vfio_pci.c
> >
> >  Documentation/PCI/pci.rst                     |    1 +
> >  MAINTAINERS                                   |    1 +
> >  drivers/pci/pci-driver.c                      |   28 +-
> >  drivers/vfio/Kconfig                          |   29 +-
> >  drivers/vfio/fsl-mc/Kconfig                   |    3 +-
> >  drivers/vfio/mdev/Kconfig                     |    1 -
> >  drivers/vfio/pci/Kconfig                      |   40 +-
> >  drivers/vfio/pci/Makefile                     |    8 +-
> >  drivers/vfio/pci/vfio_pci.c                   | 2262 +----------------
> >  drivers/vfio/pci/vfio_pci_config.c            |   70 +-
> >  drivers/vfio/pci/vfio_pci_core.c              | 2158 ++++++++++++++++
> >  drivers/vfio/pci/vfio_pci_igd.c               |   19 +-
> >  drivers/vfio/pci/vfio_pci_intrs.c             |   42 +-
> >  drivers/vfio/pci/vfio_pci_rdwr.c              |   18 +-
> >  drivers/vfio/pci/vfio_pci_zdev.c              |    4 +-
> >  drivers/vfio/platform/Kconfig                 |    6 +-
> >  drivers/vfio/platform/reset/Kconfig           |    4 +-
> >  include/linux/mod_devicetable.h               |    6 +
> >  include/linux/pci.h                           |   29 +
> >  .../linux/vfio_pci_core.h                     |   89 +-
> >  scripts/mod/devicetable-offsets.c             |    1 +
> >  scripts/mod/file2alias.c                      |   17 +-
> >  22 files changed, 2525 insertions(+), 2311 deletions(-)  create mode
> > 100644 drivers/vfio/pci/vfio_pci_core.c  rename
> > drivers/vfio/pci/vfio_pci_private.h => include/linux/vfio_pci_core.h
> > (56%)
> Applied to vfio next branch for v5.15.  Thanks,
> 
> Alex
We verified the Intel KVMGT and GVT-d KVM feature, no regression be introduced by these patch series.
Tested-by: Terrence Xu <terrence.xu@intel.com>

Thanks
Terrence