| Message ID | 20220712105501.688628-1-yajunw@nvidia.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | vhost-user: Support vhost_dev_start | expand |
On Tue, Jul 12, 2022 at 01:54:59PM +0300, Yajun Wu wrote: > The motivation of adding vhost-user vhost_dev_start support is to > improve backend configuration speed and reduce live migration VM > downtime. > > Today VQ configuration is issued one by one. For virtio net with > multi-queue support, backend needs to update RSS (Receive side > scaling) on every rx queue enable. Updating RSS is time-consuming > (typical time like 7ms). > > Implement already defined vhost status and message in the vhost > specification [1]. > (a) VHOST_USER_PROTOCOL_F_STATUS > (b) VHOST_USER_SET_STATUS > (c) VHOST_USER_GET_STATUS > > Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK for > device start and reset(0) for device stop. > > On reception of the DRIVER_OK message, backend can apply the needed setting > only once (instead of incremental) and also utilize parallelism on enabling > queues. > > This improves QEMU's live migration downtime with vhost user backend > implementation by great margin, specially for the large number of VQs of 64 > from 800 msec to 250 msec. > > Another change is to move the device start routines after finishing all the > necessary device and VQ configuration, further aligning to the virtio > specification for "device initialization sequence". I think it's best to merge this after the 7.1 release. I've tagged this but pls ping me after the release just to make sure it's not lost. Thanks! > [1] https://qemu-project.gitlab.io/qemu/interop/vhost-user.html#introduction > > v2: > - add setting status bit VIRTIO_CONFIG_S_FEATURES_OK > - avoid adding status bits already set > > Yajun Wu (2): > vhost: Change the sequence of device start > vhost-user: Support vhost_dev_start > > hw/block/vhost-user-blk.c | 18 ++++++---- > hw/net/vhost_net.c | 12 +++---- > hw/virtio/vhost-user.c | 74 ++++++++++++++++++++++++++++++++++++++- > 3 files changed, 90 insertions(+), 14 deletions(-) > > -- > 2.27.0
Michael, 7.1 released, please merge. -----Original Message----- From: Michael S. Tsirkin <mst@redhat.com> Sent: Tuesday, July 26, 2022 10:56 PM To: Yajun Wu <yajunw@nvidia.com> Cc: qemu-devel@nongnu.org; Parav Pandit <parav@nvidia.com> Subject: Re: [PATCH v2 0/2] vhost-user: Support vhost_dev_start External email: Use caution opening links or attachments On Tue, Jul 12, 2022 at 01:54:59PM +0300, Yajun Wu wrote: > The motivation of adding vhost-user vhost_dev_start support is to > improve backend configuration speed and reduce live migration VM > downtime. > > Today VQ configuration is issued one by one. For virtio net with > multi-queue support, backend needs to update RSS (Receive side > scaling) on every rx queue enable. Updating RSS is time-consuming > (typical time like 7ms). > > Implement already defined vhost status and message in the vhost > specification [1]. > (a) VHOST_USER_PROTOCOL_F_STATUS > (b) VHOST_USER_SET_STATUS > (c) VHOST_USER_GET_STATUS > > Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK for > device start and reset(0) for device stop. > > On reception of the DRIVER_OK message, backend can apply the needed > setting only once (instead of incremental) and also utilize > parallelism on enabling queues. > > This improves QEMU's live migration downtime with vhost user backend > implementation by great margin, specially for the large number of VQs > of 64 from 800 msec to 250 msec. > > Another change is to move the device start routines after finishing > all the necessary device and VQ configuration, further aligning to the > virtio specification for "device initialization sequence". I think it's best to merge this after the 7.1 release. I've tagged this but pls ping me after the release just to make sure it's not lost. Thanks! > [1] > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fqemu > -project.gitlab.io%2Fqemu%2Finterop%2Fvhost-user.html%23introduction&a > mp;data=05%7C01%7Cyajunw%40nvidia.com%7Cb526f8237f7a4531d6eb08da6f16fc > e9%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637944441784266470%7CU > nknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha > WwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zQ5%2F6pYP0riRzArdOWyaARNn > 6s9NC8vIeIgj%2BB03EAM%3D&reserved=0 > > v2: > - add setting status bit VIRTIO_CONFIG_S_FEATURES_OK > - avoid adding status bits already set > > Yajun Wu (2): > vhost: Change the sequence of device start > vhost-user: Support vhost_dev_start > > hw/block/vhost-user-blk.c | 18 ++++++---- > hw/net/vhost_net.c | 12 +++---- > hw/virtio/vhost-user.c | 74 ++++++++++++++++++++++++++++++++++++++- > 3 files changed, 90 insertions(+), 14 deletions(-) > > -- > 2.27.0
Michael, Don't forget to merge. Thanks! -----Original Message----- From: Yajun Wu Sent: Monday, September 5, 2022 12:58 PM To: Michael S. Tsirkin <mst@redhat.com> Cc: qemu-devel@nongnu.org; Parav Pandit <parav@nvidia.com> Subject: RE: [PATCH v2 0/2] vhost-user: Support vhost_dev_start Michael, 7.1 released, please merge. -----Original Message----- From: Michael S. Tsirkin <mst@redhat.com> Sent: Tuesday, July 26, 2022 10:56 PM To: Yajun Wu <yajunw@nvidia.com> Cc: qemu-devel@nongnu.org; Parav Pandit <parav@nvidia.com> Subject: Re: [PATCH v2 0/2] vhost-user: Support vhost_dev_start External email: Use caution opening links or attachments On Tue, Jul 12, 2022 at 01:54:59PM +0300, Yajun Wu wrote: > The motivation of adding vhost-user vhost_dev_start support is to > improve backend configuration speed and reduce live migration VM > downtime. > > Today VQ configuration is issued one by one. For virtio net with > multi-queue support, backend needs to update RSS (Receive side > scaling) on every rx queue enable. Updating RSS is time-consuming > (typical time like 7ms). > > Implement already defined vhost status and message in the vhost > specification [1]. > (a) VHOST_USER_PROTOCOL_F_STATUS > (b) VHOST_USER_SET_STATUS > (c) VHOST_USER_GET_STATUS > > Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK for > device start and reset(0) for device stop. > > On reception of the DRIVER_OK message, backend can apply the needed > setting only once (instead of incremental) and also utilize > parallelism on enabling queues. > > This improves QEMU's live migration downtime with vhost user backend > implementation by great margin, specially for the large number of VQs > of 64 from 800 msec to 250 msec. > > Another change is to move the device start routines after finishing > all the necessary device and VQ configuration, further aligning to the > virtio specification for "device initialization sequence". I think it's best to merge this after the 7.1 release. I've tagged this but pls ping me after the release just to make sure it's not lost. Thanks! > [1] > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fqemu > -project.gitlab.io%2Fqemu%2Finterop%2Fvhost-user.html%23introduction&a > mp;data=05%7C01%7Cyajunw%40nvidia.com%7Cb526f8237f7a4531d6eb08da6f16fc > e9%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C637944441784266470%7CU > nknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha > WwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zQ5%2F6pYP0riRzArdOWyaARNn > 6s9NC8vIeIgj%2BB03EAM%3D&reserved=0 > > v2: > - add setting status bit VIRTIO_CONFIG_S_FEATURES_OK > - avoid adding status bits already set > > Yajun Wu (2): > vhost: Change the sequence of device start > vhost-user: Support vhost_dev_start > > hw/block/vhost-user-blk.c | 18 ++++++---- > hw/net/vhost_net.c | 12 +++---- > hw/virtio/vhost-user.c | 74 ++++++++++++++++++++++++++++++++++++++- > 3 files changed, 90 insertions(+), 14 deletions(-) > > -- > 2.27.0