[v7,00/17] vfio-user server in QEMU

Message

Jag Raman March 25, 2022, 7:19 p.m. UTC

Hi,

This is v7 of the server side changes to enable vfio-user in QEMU.

Thank you very much for reviewing the last revision of this series!

In this revision, we've dropped the patches concerning migration
due to a major changes in the VFIO PCI protocol. We are changing the
client, library and server to conform with the updated spec, and
will send the migration patches for review separately.

The review of client side changes are happening parallelly [1]. The
avocado test in Patch 17 of this series depends on the client, as
such it has to wait until client changes are available. However,
we run the avocado test, among others, to confirm that the server
is working as expected - we do it before submitting patches for
review each time. The following repo applies this series on the
latest client, in case anyone would like to run vfio-user.
repo: https://github.com/oracle/qemu
branch: vfio-user-client-v7server
launcher: scripts/vfiouser-launcher.py
[1]: https://lore.kernel.org/all/6d8ae2[e-mail]/T/

We've made the following changes in this revision:

[PATCH v7 02/17] qdev: unplug blocker for devices
  - corrects comments to prevent creation of new section for
    unplug blocker
  - adds an assert to device_finalize() to confirm that device
    does not have unplug blockers
  - moves the unplug blocker functions to hw/core/qdev.c
  - moves test for unplug blocker to qdev_unplug() from qmp_device_del()

[PATCH v7 05/17] configure: require cmake 3.19 or newer
  - new in this series

[PATCH v7 06/17] vfio-user: build library
  - configure script sets cmake_required flag for vfio user

[PATCH v7 07/17] vfio-user: define vfio-user-server object
  - adds auto-shutdown sub-option to the remote machine
  - adds boolean auto_shutdown to TYPE_REMOTE_MACHINE's class
  - adds vfu_object_auto_shutdown() helper function to
    vfio-user-obj.c to query the auto-shutdown property
    from the machine
  - reworks VFU_OBJECT_ERROR() & vfu_object_finalize() to use
    the helper function above.
  - updates QEMU version to 7.1 in commentary for VfioUserServerProperties

[PATCH v7 08/17] vfio-user: instantiate vfio-user context
  - moves phase_check() after the check for machine type
    in vfu_object_init()
  - sets o->vfu_ctx to NULL in vfu_object_finalize()

[PATCH v7 09/17] vfio-user: find and init PCI device
  - holds a reference to attached device in the server,
    and unrefs it during cleanup

[PATCH v7 10/17] vfio-user: run vfio-user context
  - updates QEMU version to 7.1 in commentary for VFU_CLIENT_HANGUP

[PATCH v7 11/17] vfio-user: handle PCI config space accesses
  - adds a comment explaining that writes to BAR register in
    config space doesn't create conflicting memory regions

[PATCH v7 12/17] vfio-user: IOMMU support for remote device
  - passes IOMMU table as an opaque data to pci_setup_iommu()
  - adds locking to access the table to enable concurrent access
  - retains the global data structure as remote_iommu_del_device()
    needs it because it doesn't have access to the opaque data
  - removes redundant check to confirm if hash table is present
    in remote_iommu_find_add_as()
  - adds remote_iommu_del_elem() to free IOMMU entry when it
    is removed from the table

[PATCH v7 14/17] vfio-user: handle PCI BAR accesses
  - adjusts the 'offset' in vfu_object_bar_rw() - MemoryRegion
    returned by memory_region_find() could be a subregion of the
    root memory region referenced by pci_dev->io_regions[pci_bar].memory.
    'offset' input argument is relative to the root region, whereas it
    must be relative to the subregion before access.
  - adds warning for out-of-range access

[PATCH v7 15/17] vfio-user: handle device interrupts
  - moves msi_nonbroken initialization to hw/remote/machine.c
  - adds irq_opaque to PCIDevice which the interrupt notification
    could use; drops global hash table which map device to
    vfio-user context
  - removes NULL function pointer test in msi_notify and msix_notify
  - vfu_object_msi_notify() asserts that IRQ vector is allocated

[PATCH v7 16/17] vfio-user: handle reset of remote device
  - adds comment to explain lost connection handling

[PATCH v7 17/17] vfio-user: avocado tests for vfio-user
  - drops the migration test

Dropped the following patches:
configure, meson: override C compiler for cmake
softmmu/vl: defer backend init
vfio-user: register handlers to facilitate migration

We are looking forward to your comments.

Thank you very much!

Jagannathan Raman (17):
  tests/avocado: Specify target VM argument to helper routines
  qdev: unplug blocker for devices
  remote/machine: add HotplugHandler for remote machine
  remote/machine: add vfio-user property
  configure: require cmake 3.19 or newer
  vfio-user: build library
  vfio-user: define vfio-user-server object
  vfio-user: instantiate vfio-user context
  vfio-user: find and init PCI device
  vfio-user: run vfio-user context
  vfio-user: handle PCI config space accesses
  vfio-user: IOMMU support for remote device
  vfio-user: handle DMA mappings
  vfio-user: handle PCI BAR accesses
  vfio-user: handle device interrupts
  vfio-user: handle reset of remote device
  vfio-user: avocado tests for vfio-user

 configure                                  |  36 +-
 meson.build                                |  44 +-
 qapi/misc.json                             |  23 +
 qapi/qom.json                              |  20 +-
 include/exec/memory.h                      |   3 +
 include/hw/pci/pci.h                       |  10 +
 include/hw/qdev-core.h                     |  29 +
 include/hw/remote/iommu.h                  |  18 +
 include/hw/remote/machine.h                |  10 +-
 include/hw/remote/vfio-user-obj.h          |   6 +
 hw/core/qdev.c                             |  24 +
 hw/pci/msi.c                               |  11 +-
 hw/pci/msix.c                              |  10 +-
 hw/remote/iommu.c                          |  95 +++
 hw/remote/machine.c                        |  73 +-
 hw/remote/vfio-user-obj.c                  | 847 +++++++++++++++++++++
 softmmu/physmem.c                          |   4 +-
 softmmu/qdev-monitor.c                     |   4 +
 stubs/vfio-user-obj.c                      |   6 +
 tests/qtest/fuzz/generic_fuzz.c            |   9 +-
 .gitlab-ci.d/buildtest.yml                 |   2 +
 .gitmodules                                |   3 +
 Kconfig.host                               |   4 +
 MAINTAINERS                                |   6 +
 hw/remote/Kconfig                          |   4 +
 hw/remote/meson.build                      |   4 +
 hw/remote/trace-events                     |  11 +
 meson_options.txt                          |   3 +
 stubs/meson.build                          |   1 +
 subprojects/libvfio-user                   |   1 +
 tests/avocado/avocado_qemu/__init__.py     |  14 +-
 tests/avocado/vfio-user.py                 | 164 ++++
 tests/docker/dockerfiles/centos8.docker    |   2 +
 tests/docker/dockerfiles/ubuntu2004.docker |   2 +
 34 files changed, 1483 insertions(+), 20 deletions(-)
 create mode 100644 include/hw/remote/iommu.h
 create mode 100644 include/hw/remote/vfio-user-obj.h
 create mode 100644 hw/remote/iommu.c
 create mode 100644 hw/remote/vfio-user-obj.c
 create mode 100644 stubs/vfio-user-obj.c
 create mode 160000 subprojects/libvfio-user
 create mode 100644 tests/avocado/vfio-user.py