| Message ID | 20221108170755.92768-1-eperezma@redhat.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | ASID support in vhost-vdpa net | expand |
On Tue, Nov 08, 2022 at 06:07:45PM +0100, Eugenio Pérez wrote: > Control VQ is the way net devices use to send changes to the device state, like > the number of active queues or its mac address. > > QEMU needs to intercept this queue so it can track these changes and is able to > migrate the device. It can do it from 1576dbb5bbc4 ("vdpa: Add x-svq to > NetdevVhostVDPAOptions"). However, to enable x-svq implies to shadow all VirtIO > device's virtqueues, which will damage performance. > > This series adds address space isolation, so the device and the guest > communicate directly with them (passthrough) and CVQ communication is split in > two: The guest communicates with QEMU and QEMU forwards the commands to the > device. > > Comments are welcome. Thanks! This is not 7.2 material, right? > v6: > - Do not allocate SVQ resources like file descriptors if SVQ cannot be used. > - Disable shadow CVQ if the device does not support it because of net > features. > > v5: > - Move vring state in vhost_vdpa_get_vring_group instead of using a > parameter. > - Rename VHOST_VDPA_NET_CVQ_PASSTHROUGH to VHOST_VDPA_NET_DATA_ASID > > v4: > - Rebased on last CVQ start series, that allocated CVQ cmd bufs at load > - Squash vhost_vdpa_cvq_group_is_independent. > - Do not check for cvq index on vhost_vdpa_net_prepare, we only have one > that callback registered in that NetClientInfo. > - Add comment specifying behavior if device does not support _F_ASID > - Update headers to a later Linux commit to not to remove SETUP_RNG_SEED > > v3: > - Do not return an error but just print a warning if vdpa device initialization > returns failure while getting AS num of VQ groups > - Delete extra newline > > v2: > - Much as commented on series [1], handle vhost_net backend through > NetClientInfo callbacks instead of directly. > - Fix not freeing SVQ properly when device does not support CVQ > - Add BIT_ULL missed checking device's backend feature for _F_ASID. > > Eugenio Pérez (10): > vdpa: Use v->shadow_vqs_enabled in vhost_vdpa_svqs_start & stop > vhost: set SVQ device call handler at SVQ start > vhost: Allocate SVQ device file descriptors at device start > vdpa: add vhost_vdpa_net_valid_svq_features > vdpa: move SVQ vring features check to net/ > vdpa: Allocate SVQ unconditionally > vdpa: Add asid parameter to vhost_vdpa_dma_map/unmap > vdpa: Store x-svq parameter in VhostVDPAState > vdpa: Add listener_shadow_vq to vhost_vdpa > vdpa: Always start CVQ in SVQ mode > > include/hw/virtio/vhost-vdpa.h | 10 +- > hw/virtio/vhost-shadow-virtqueue.c | 35 +----- > hw/virtio/vhost-vdpa.c | 114 ++++++++++--------- > net/vhost-vdpa.c | 171 ++++++++++++++++++++++++++--- > hw/virtio/trace-events | 4 +- > 5 files changed, 222 insertions(+), 112 deletions(-) > > -- > 2.31.1 >
On Thu, Nov 10, 2022 at 1:25 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > On Tue, Nov 08, 2022 at 06:07:45PM +0100, Eugenio Pérez wrote: > > Control VQ is the way net devices use to send changes to the device state, like > > the number of active queues or its mac address. > > > > QEMU needs to intercept this queue so it can track these changes and is able to > > migrate the device. It can do it from 1576dbb5bbc4 ("vdpa: Add x-svq to > > NetdevVhostVDPAOptions"). However, to enable x-svq implies to shadow all VirtIO > > device's virtqueues, which will damage performance. > > > > This series adds address space isolation, so the device and the guest > > communicate directly with them (passthrough) and CVQ communication is split in > > two: The guest communicates with QEMU and QEMU forwards the commands to the > > device. > > > > Comments are welcome. Thanks! > > > This is not 7.2 material, right? > No it is not. I should have noted it after PATCH in the subject, sorry. > > v6: > > - Do not allocate SVQ resources like file descriptors if SVQ cannot be used. > > - Disable shadow CVQ if the device does not support it because of net > > features. > > > > v5: > > - Move vring state in vhost_vdpa_get_vring_group instead of using a > > parameter. > > - Rename VHOST_VDPA_NET_CVQ_PASSTHROUGH to VHOST_VDPA_NET_DATA_ASID > > > > v4: > > - Rebased on last CVQ start series, that allocated CVQ cmd bufs at load > > - Squash vhost_vdpa_cvq_group_is_independent. > > - Do not check for cvq index on vhost_vdpa_net_prepare, we only have one > > that callback registered in that NetClientInfo. > > - Add comment specifying behavior if device does not support _F_ASID > > - Update headers to a later Linux commit to not to remove SETUP_RNG_SEED > > > > v3: > > - Do not return an error but just print a warning if vdpa device initialization > > returns failure while getting AS num of VQ groups > > - Delete extra newline > > > > v2: > > - Much as commented on series [1], handle vhost_net backend through > > NetClientInfo callbacks instead of directly. > > - Fix not freeing SVQ properly when device does not support CVQ > > - Add BIT_ULL missed checking device's backend feature for _F_ASID. > > > > Eugenio Pérez (10): > > vdpa: Use v->shadow_vqs_enabled in vhost_vdpa_svqs_start & stop > > vhost: set SVQ device call handler at SVQ start > > vhost: Allocate SVQ device file descriptors at device start > > vdpa: add vhost_vdpa_net_valid_svq_features > > vdpa: move SVQ vring features check to net/ > > vdpa: Allocate SVQ unconditionally > > vdpa: Add asid parameter to vhost_vdpa_dma_map/unmap > > vdpa: Store x-svq parameter in VhostVDPAState > > vdpa: Add listener_shadow_vq to vhost_vdpa > > vdpa: Always start CVQ in SVQ mode > > > > include/hw/virtio/vhost-vdpa.h | 10 +- > > hw/virtio/vhost-shadow-virtqueue.c | 35 +----- > > hw/virtio/vhost-vdpa.c | 114 ++++++++++--------- > > net/vhost-vdpa.c | 171 ++++++++++++++++++++++++++--- > > hw/virtio/trace-events | 4 +- > > 5 files changed, 222 insertions(+), 112 deletions(-) > > > > -- > > 2.31.1 > > >
Control VQ is the way net devices use to send changes to the device state, like the number of active queues or its mac address. QEMU needs to intercept this queue so it can track these changes and is able to migrate the device. It can do it from 1576dbb5bbc4 ("vdpa: Add x-svq to NetdevVhostVDPAOptions"). However, to enable x-svq implies to shadow all VirtIO device's virtqueues, which will damage performance. This series adds address space isolation, so the device and the guest communicate directly with them (passthrough) and CVQ communication is split in two: The guest communicates with QEMU and QEMU forwards the commands to the device. Comments are welcome. Thanks! v6: - Do not allocate SVQ resources like file descriptors if SVQ cannot be used. - Disable shadow CVQ if the device does not support it because of net features. v5: - Move vring state in vhost_vdpa_get_vring_group instead of using a parameter. - Rename VHOST_VDPA_NET_CVQ_PASSTHROUGH to VHOST_VDPA_NET_DATA_ASID v4: - Rebased on last CVQ start series, that allocated CVQ cmd bufs at load - Squash vhost_vdpa_cvq_group_is_independent. - Do not check for cvq index on vhost_vdpa_net_prepare, we only have one that callback registered in that NetClientInfo. - Add comment specifying behavior if device does not support _F_ASID - Update headers to a later Linux commit to not to remove SETUP_RNG_SEED v3: - Do not return an error but just print a warning if vdpa device initialization returns failure while getting AS num of VQ groups - Delete extra newline v2: - Much as commented on series [1], handle vhost_net backend through NetClientInfo callbacks instead of directly. - Fix not freeing SVQ properly when device does not support CVQ - Add BIT_ULL missed checking device's backend feature for _F_ASID. Eugenio Pérez (10): vdpa: Use v->shadow_vqs_enabled in vhost_vdpa_svqs_start & stop vhost: set SVQ device call handler at SVQ start vhost: Allocate SVQ device file descriptors at device start vdpa: add vhost_vdpa_net_valid_svq_features vdpa: move SVQ vring features check to net/ vdpa: Allocate SVQ unconditionally vdpa: Add asid parameter to vhost_vdpa_dma_map/unmap vdpa: Store x-svq parameter in VhostVDPAState vdpa: Add listener_shadow_vq to vhost_vdpa vdpa: Always start CVQ in SVQ mode include/hw/virtio/vhost-vdpa.h | 10 +- hw/virtio/vhost-shadow-virtqueue.c | 35 +----- hw/virtio/vhost-vdpa.c | 114 ++++++++++--------- net/vhost-vdpa.c | 171 ++++++++++++++++++++++++++--- hw/virtio/trace-events | 4 +- 5 files changed, 222 insertions(+), 112 deletions(-)