[v3,00/12] vfio-user server in QEMU

Message

Jag Raman Oct. 11, 2021, 5:31 a.m. UTC

Based-on: <[e-mail]>

Hi,

Thank you very much for reviewing the patches in the previous revision!

We have addressed most of comments from v2.

We are working on MSI-x support (used by PCIe devices such as “nvme”)
and a couple of comments in the migration patches. We hope these two
items will be in the next revision of the patches.

Please the list below for changes since last revision:

[PATCH RFC v3 01/12] configure, meson: override C compiler for cmake
  - New patch in this rev

[PATCH RFC v3 02/12] vfio-user: build library
  - fixed indentation issue
  - dropped separate MAINTAINERS section for submodule. using
    existing section to avoid checkpatch warning

[PATCH RFC v3 03/12] vfio-user: define vfio-user-server object
  - Changed socket parameter type to SocketAddress from str
  - renamed object as vfio-user-server
  - renamed devid option as device
  - Added CONFIG_VFIO_USER_SERVER option

[PATCH RFC v3 04/12] vfio-user: instantiate vfio-user context
  - removed errno.h include
  - documented reason for using a machine init done notifier
  - don’t call vfu_destroy_ctx() if context is not initialized

[PATCH RFC v3 05/12] vfio-user: find and init PCI device
  - registering for PCI Express devices as well. Tested with
    “nvme” device in QEMU.
  - MSI-x is not supported yet, PCI-e devices using INTx
    presently. Will add a patch shortly to address MSI-x

[PATCH RFC v3 06/12] vfio-user: run vfio-user context
  - Removed the separate QemuThread for attach as it was blocking
  - Using qemu_set_fd_handler() for attach’ing context

[PATCH RFC v3 07/12] vfio-user: handle PCI config space accesses
  - Using pci_host_config_read_common/write/common() instead of pci_default_read/write_config()
  - The read_common/write_common() should take care of endianness

[PATCH RFC v3 09/12] vfio-user: handle PCI BAR accesses
  - Added support for 64-bit BARs

[PATCH RFC v3 11/12] vfio-user: register handlers to facilitate migration
  - Fixed size handling in vfu_mig_buf_read()
  - Moved qemu_remote_savevm() from “precopy” phase to “stop_and_copy”
  - Opening “vfu_mig_file” just before use
  - TODO: Working with Nutanix on issues concerning the “resume”
    phase and size of Migration BAR (the size of migration BAR shouldn’t
    matter when using vfu_migration_callbacks_t).

Please share your comments.

Thank you!

Jagannathan Raman (12):
  configure, meson: override C compiler for cmake
  vfio-user: build library
  vfio-user: define vfio-user-server object
  vfio-user: instantiate vfio-user context
  vfio-user: find and init PCI device
  vfio-user: run vfio-user context
  vfio-user: handle PCI config space accesses
  vfio-user: handle DMA mappings
  vfio-user: handle PCI BAR accesses
  vfio-user: handle device interrupts
  vfio-user: register handlers to facilitate migration
  vfio-user: acceptance test

 configure                                  |  15 +-
 meson.build                                |  39 +
 qapi/qom.json                              |  20 +-
 include/hw/remote/iohub.h                  |   2 +
 migration/savevm.h                         |   2 +
 hw/remote/iohub.c                          |   5 +
 hw/remote/vfio-user-obj.c                  | 868 +++++++++++++++++++++
 migration/savevm.c                         |  73 ++
 .gitlab-ci.d/buildtest.yml                 |   2 +
 .gitmodules                                |   3 +
 MAINTAINERS                                |   3 +
 hw/remote/Kconfig                          |   5 +
 hw/remote/meson.build                      |   3 +
 hw/remote/trace-events                     |  11 +
 subprojects/libvfio-user                   |   1 +
 tests/acceptance/multiprocess.py           |   2 +
 tests/acceptance/vfio-user.py              |  96 +++
 tests/docker/dockerfiles/centos8.docker    |   2 +
 tests/docker/dockerfiles/ubuntu2004.docker |   2 +
 19 files changed, 1151 insertions(+), 3 deletions(-)
 create mode 100644 hw/remote/vfio-user-obj.c
 create mode 160000 subprojects/libvfio-user
 create mode 100644 tests/acceptance/vfio-user.py

Comments

Stefan Hajnoczi Oct. 27, 2021, 6:33 p.m. UTC | #1

On Mon, Oct 11, 2021 at 01:31:05AM -0400, Jagannathan Raman wrote:
> We have addressed most of comments from v2.
> 
> We are working on MSI-x support (used by PCIe devices such as “nvme”)
> and a couple of comments in the migration patches. We hope these two
> items will be in the next revision of the patches.

Please add an object-add vfio-user-server and object-del
vfio-user-server test case. The code is currently written in a way that
is incompatible with hotplug. Even if you don't need hotplug right away
it's worth implementing it immediately to avoid baking in assumptions
that will be hard to fix later.

Please try running the tests on a big-endian host, if possible. That may
help shake out remaining endianness issues.

Stefan