vhost: set mem table before device start

Commit Message

Yajun Wu Nov. 9, 2022, 2:53 a.m. UTC

Today guest memory information (through VHOST_USER_SET_MEM_TABLE)
is sent out during vhost device start. Due to this delay, memory
pinning is delayed. For 4G guest memory, a VFIO driver usually
takes around 400+msec to pin the memory.

This time is accounted towards the VM downtime. When live migrating
a VM, vhost device start is occuring in vmstate load stage.

Moving set mem table just after VM bootup, before device start can
let backend have enough time to pin the guest memory before starting
the device. This improvements reduces VM downtime by 400+msec.

Signed-off-by: Yajun Wu <[e-mail]>
Acked-by: Parav Pandit <[e-mail]>
---
 hw/virtio/vhost.c | 8 ++++++++
 1 file changed, 8 insertions(+)

Patch

diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c
index f758f177bb..73e473cd84 100644
--- a/hw/virtio/vhost.c
+++ b/hw/virtio/vhost.c
@@ -539,6 +539,14 @@  static void vhost_commit(MemoryListener *listener)
     }
 
     if (!dev->started) {
+        /* Backend can pin memory before device start, reduce LM downtime */
+        if (dev->vhost_ops->backend_type == VHOST_BACKEND_TYPE_USER &&
+            dev->n_mem_sections) {
+            r = dev->vhost_ops->vhost_set_mem_table(dev, dev->mem);
+            if (r < 0) {
+                VHOST_OPS_DEBUG(r, "vhost_set_mem_table failed");
+            }
+        }
         goto out;
     }