[v4,00/13] Consolidate common vdpa members in VhostVDPAShared

Message

Eugenio Perez Martin Dec. 21, 2023, 5:43 p.m. UTC

Current memory operations like pinning may take a lot of time at the
destination.  Currently they are done after the source of the migration is
stopped, and before the workload is resumed at the destination.  This is a
period where neigher traffic can flow, nor the VM workload can continue
(downtime).

We can do better as we know the memory layout of the guest RAM at the
destination from the moment the migration starts.  Moving that operation allows
QEMU to communicate the kernel the maps while the workload is still running in
the source, so Linux can start mapping them.  Ideally, all IOMMU is configured,
but if the vDPA parent driver uses on-chip IOMMU and .set_map we're still
saving all the pinning time.

This is a first required step to consolidate all the members in a common
struct.  This is needed because the destination does not know what vhost_vdpa
struct will have the registered listener member, so it is easier to place them
in a shared struct rather to keep them in vhost_vdpa struct.

v4:
* Actually fix the SIGSEGV, unregistering shared->listener in devices
  with vq_index > 1 while shared has been freed in net/vhost-vdpa.c
  freeing the device with vq_index == 0. Keeping the free in 0 as fd is
  closed at index 0.

v3:
* Cherry-pick Si-Wei's fixes.
* Only memory_listener_unregister at vhost_vdpa_cleanup in the last dev.
  SIGSEGV detected by both Lei Yang [1] and Si-Wei [2].

v2:
* Avoid repeated setting shared->shadow_data by squashing Si-Wei's patch
  [2].

v1 from RFC:
* Fix vhost_vdpa_net_cvq_start checking for always_svq instead of
  shadow_data.  This could cause CVQ not being shadowed if
  vhost_vdpa_net_cvq_start was called in the middle of a migration.

[1] https://patchwork.kernel.org/project/qemu-devel/cover/[e-mail]/
[2] https://patchwork.kernel.org/project/qemu-devel/patch/[e-mail]/

Eugenio Pérez (13):
  vdpa: add VhostVDPAShared
  vdpa: move iova tree to the shared struct
  vdpa: move iova_range to vhost_vdpa_shared
  vdpa: move shadow_data to vhost_vdpa_shared
  vdpa: use vdpa shared for tracing
  vdpa: move file descriptor to vhost_vdpa_shared
  vdpa: move iotlb_batch_begin_sent to vhost_vdpa_shared
  vdpa: move backend_cap to vhost_vdpa_shared
  vdpa: remove msg type of vhost_vdpa
  vdpa: move iommu_list to vhost_vdpa_shared
  vdpa: use VhostVDPAShared in vdpa_dma_map and unmap
  vdpa: use dev_shared in vdpa_iommu
  vdpa: move memory listener to vhost_vdpa_shared

 include/hw/virtio/vhost-vdpa.h |  36 +++++---
 hw/virtio/vdpa-dev.c           |   7 +-
 hw/virtio/vhost-vdpa.c         | 160 +++++++++++++++++----------------
 net/vhost-vdpa.c               | 116 ++++++++++++------------
 hw/virtio/trace-events         |  14 +--
 5 files changed, 173 insertions(+), 160 deletions(-)