| Message ID | 20221105083629.1058-1-longpeng2@huawei.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | add generic vDPA device support | expand |
On Sat, Nov 05, 2022 at 04:36:25PM +0800, Longpeng(Mike) wrote: > From: Longpeng <longpeng2@huawei.com> > > Hi guys, > > With the generic vDPA device, QEMU won't need to touch the device > types any more, such like vfio. With this kind of passthrough migration is completely MIA right? Better add a blocker... And given this is there an advantage over VFIO? > We can use the generic vDPA device as follow: > -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X > Or > -M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \ > vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x > Changes v6 -> v7: > (v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html) > - rebase. [Jason] > - add documentation . [Stefan] > > Changes v5 -> v6: > Patch 2: > - Turn to the original approach in the RFC to initialize the > virtio_pci_id_info array. [Michael] > https://lore.kernel.org/all/20220105005900.860-2-longpeng2@huawei.com/ > Patch 3: > - Fix logical error of exception handler around the post_init. > [Stefano] > - Fix some coding style warnings. [Stefano] > Patch 4: > - Fix some coding style warnings. [Stefano] > > Changes v4 -> v5: > Patch 3: > - remove vhostfd [Jason] > - support virtio-mmio [Jason] > > Changes v3 -> v4: > v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg877015.html > - reorganize the series [Stefano] > - fix some typos [Stefano] > - fix logical error in vhost_vdpa_device_realize [Stefano] > > Changes v2 -> v3 > Patch 4 & 5: > - only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng] > - s/VQS_NUM/VQS_COUNT [Stefano] > - check both vdpa_dev_fd and vdpa_dev [Stefano] > Patch 6: > - move all steps into vhost_vdpa_device_unrealize. [Stefano] > > Changes RFC -> v2 > Patch 1: > - rename 'pdev_id' to 'trans_devid' [Michael] > - only use transitional device id for the devices > listed in the spec [Michael] > - use macros to make the id_info table clearer [Longpeng] > - add some modern devices in the id_info table [Longpeng] > Patch 2: > - remove the GET_VECTORS_NUM command [Jason] > Patch 4: > - expose vdpa_dev_fd as a QOM preperty [Stefan] > - introduce vhost_vdpa_device_get_u32 as a common > function to make the code clearer [Stefan] > - fix the misleading description of 'dc->desc' [Stefano] > Patch 5: > - check returned number of virtqueues [Stefan] > Patch 6: > - init s->num_queues [Stefano] > - free s->dev.vqs [Stefano] > > > Longpeng (Mike) (4): > virtio: get class_id and pci device id by the virtio id > vdpa: add vdpa-dev support > vdpa: add vdpa-dev-pci support > docs: Add generic vhost-vdpa device documentation > > docs/system/devices/vhost-vdpa-device.rst | 43 +++ > hw/virtio/Kconfig | 5 + > hw/virtio/meson.build | 2 + > hw/virtio/vdpa-dev-pci.c | 102 ++++++ > hw/virtio/vdpa-dev.c | 377 ++++++++++++++++++++++ > hw/virtio/virtio-pci.c | 88 +++++ > include/hw/virtio/vdpa-dev.h | 43 +++ > include/hw/virtio/virtio-pci.h | 5 + > 8 files changed, 665 insertions(+) > create mode 100644 docs/system/devices/vhost-vdpa-device.rst > create mode 100644 hw/virtio/vdpa-dev-pci.c > create mode 100644 hw/virtio/vdpa-dev.c > create mode 100644 include/hw/virtio/vdpa-dev.h > > -- > 2.23.0
在 2022/11/6 0:43, Michael S. Tsirkin 写道: > On Sat, Nov 05, 2022 at 04:36:25PM +0800, Longpeng(Mike) wrote: >> From: Longpeng <longpeng2@huawei.com> >> >> Hi guys, >> >> With the generic vDPA device, QEMU won't need to touch the device >> types any more, such like vfio. > > With this kind of passthrough migration is completely MIA right? > Better add a blocker... Oh, I missed the "vdpa-dev: mark the device as unmigratable" since v4 and I'll add it in the next version. We'll support passthrough migration in the next step. We have already written a demo that can migrate between some offloading cards. > And given this is there an advantage over VFIO? I think the answer is the same as "why we need vDPA" if we compare it with VFIO. > >> We can use the generic vDPA device as follow: >> -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X >> Or >> -M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \ >> vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x > >> Changes v6 -> v7: >> (v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html) >> - rebase. [Jason] >> - add documentation . [Stefan] >> >> Changes v5 -> v6: >> Patch 2: >> - Turn to the original approach in the RFC to initialize the >> virtio_pci_id_info array. [Michael] >> https://lore.kernel.org/all/20220105005900.860-2-longpeng2@huawei.com/ >> Patch 3: >> - Fix logical error of exception handler around the post_init. >> [Stefano] >> - Fix some coding style warnings. [Stefano] >> Patch 4: >> - Fix some coding style warnings. [Stefano] >> >> Changes v4 -> v5: >> Patch 3: >> - remove vhostfd [Jason] >> - support virtio-mmio [Jason] >> >> Changes v3 -> v4: >> v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg877015.html >> - reorganize the series [Stefano] >> - fix some typos [Stefano] >> - fix logical error in vhost_vdpa_device_realize [Stefano] >> >> Changes v2 -> v3 >> Patch 4 & 5: >> - only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng] >> - s/VQS_NUM/VQS_COUNT [Stefano] >> - check both vdpa_dev_fd and vdpa_dev [Stefano] >> Patch 6: >> - move all steps into vhost_vdpa_device_unrealize. [Stefano] >> >> Changes RFC -> v2 >> Patch 1: >> - rename 'pdev_id' to 'trans_devid' [Michael] >> - only use transitional device id for the devices >> listed in the spec [Michael] >> - use macros to make the id_info table clearer [Longpeng] >> - add some modern devices in the id_info table [Longpeng] >> Patch 2: >> - remove the GET_VECTORS_NUM command [Jason] >> Patch 4: >> - expose vdpa_dev_fd as a QOM preperty [Stefan] >> - introduce vhost_vdpa_device_get_u32 as a common >> function to make the code clearer [Stefan] >> - fix the misleading description of 'dc->desc' [Stefano] >> Patch 5: >> - check returned number of virtqueues [Stefan] >> Patch 6: >> - init s->num_queues [Stefano] >> - free s->dev.vqs [Stefano] >> >> >> Longpeng (Mike) (4): >> virtio: get class_id and pci device id by the virtio id >> vdpa: add vdpa-dev support >> vdpa: add vdpa-dev-pci support >> docs: Add generic vhost-vdpa device documentation >> >> docs/system/devices/vhost-vdpa-device.rst | 43 +++ >> hw/virtio/Kconfig | 5 + >> hw/virtio/meson.build | 2 + >> hw/virtio/vdpa-dev-pci.c | 102 ++++++ >> hw/virtio/vdpa-dev.c | 377 ++++++++++++++++++++++ >> hw/virtio/virtio-pci.c | 88 +++++ >> include/hw/virtio/vdpa-dev.h | 43 +++ >> include/hw/virtio/virtio-pci.h | 5 + >> 8 files changed, 665 insertions(+) >> create mode 100644 docs/system/devices/vhost-vdpa-device.rst >> create mode 100644 hw/virtio/vdpa-dev-pci.c >> create mode 100644 hw/virtio/vdpa-dev.c >> create mode 100644 include/hw/virtio/vdpa-dev.h >> >> -- >> 2.23.0 > > .
On Sun, Nov 06, 2022 at 08:17:07AM +0800, Longpeng (Mike, Cloud Infrastructure Service Product Dept.) wrote: > > > 在 2022/11/6 0:43, Michael S. Tsirkin 写道: > > On Sat, Nov 05, 2022 at 04:36:25PM +0800, Longpeng(Mike) wrote: > > > From: Longpeng <longpeng2@huawei.com> > > > > > > Hi guys, > > > > > > With the generic vDPA device, QEMU won't need to touch the device > > > types any more, such like vfio. > > > > With this kind of passthrough migration is completely MIA right? > > Better add a blocker... > > Oh, I missed the "vdpa-dev: mark the device as unmigratable" since v4 and > I'll add it in the next version. > > We'll support passthrough migration in the next step. We have already > written a demo that can migrate between some offloading cards. Hmm ok. Backend disconnect can't work though, can it? State is by necessity lost when backend crashes. > > And given this is there an advantage over VFIO? > > I think the answer is the same as "why we need vDPA" if we compare it with > VFIO. The answer is mostly because you can migrate and support backend disconnect, no? > > > > > We can use the generic vDPA device as follow: > > > -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X > > > Or > > > -M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \ > > > vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x > > > > > Changes v6 -> v7: > > > (v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html) > > > - rebase. [Jason] > > > - add documentation . [Stefan] > > > > > > Changes v5 -> v6: > > > Patch 2: > > > - Turn to the original approach in the RFC to initialize the > > > virtio_pci_id_info array. [Michael] > > > https://lore.kernel.org/all/20220105005900.860-2-longpeng2@huawei.com/ > > > Patch 3: > > > - Fix logical error of exception handler around the post_init. > > > [Stefano] > > > - Fix some coding style warnings. [Stefano] > > > Patch 4: > > > - Fix some coding style warnings. [Stefano] > > > > > > Changes v4 -> v5: > > > Patch 3: > > > - remove vhostfd [Jason] > > > - support virtio-mmio [Jason] > > > > > > Changes v3 -> v4: > > > v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg877015.html > > > - reorganize the series [Stefano] > > > - fix some typos [Stefano] > > > - fix logical error in vhost_vdpa_device_realize [Stefano] > > > > > > Changes v2 -> v3 > > > Patch 4 & 5: > > > - only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng] > > > - s/VQS_NUM/VQS_COUNT [Stefano] > > > - check both vdpa_dev_fd and vdpa_dev [Stefano] > > > Patch 6: > > > - move all steps into vhost_vdpa_device_unrealize. [Stefano] > > > > > > Changes RFC -> v2 > > > Patch 1: > > > - rename 'pdev_id' to 'trans_devid' [Michael] > > > - only use transitional device id for the devices > > > listed in the spec [Michael] > > > - use macros to make the id_info table clearer [Longpeng] > > > - add some modern devices in the id_info table [Longpeng] > > > Patch 2: > > > - remove the GET_VECTORS_NUM command [Jason] > > > Patch 4: > > > - expose vdpa_dev_fd as a QOM preperty [Stefan] > > > - introduce vhost_vdpa_device_get_u32 as a common > > > function to make the code clearer [Stefan] > > > - fix the misleading description of 'dc->desc' [Stefano] > > > Patch 5: > > > - check returned number of virtqueues [Stefan] > > > Patch 6: > > > - init s->num_queues [Stefano] > > > - free s->dev.vqs [Stefano] > > > > > > > > > Longpeng (Mike) (4): > > > virtio: get class_id and pci device id by the virtio id > > > vdpa: add vdpa-dev support > > > vdpa: add vdpa-dev-pci support > > > docs: Add generic vhost-vdpa device documentation > > > > > > docs/system/devices/vhost-vdpa-device.rst | 43 +++ > > > hw/virtio/Kconfig | 5 + > > > hw/virtio/meson.build | 2 + > > > hw/virtio/vdpa-dev-pci.c | 102 ++++++ > > > hw/virtio/vdpa-dev.c | 377 ++++++++++++++++++++++ > > > hw/virtio/virtio-pci.c | 88 +++++ > > > include/hw/virtio/vdpa-dev.h | 43 +++ > > > include/hw/virtio/virtio-pci.h | 5 + > > > 8 files changed, 665 insertions(+) > > > create mode 100644 docs/system/devices/vhost-vdpa-device.rst > > > create mode 100644 hw/virtio/vdpa-dev-pci.c > > > create mode 100644 hw/virtio/vdpa-dev.c > > > create mode 100644 include/hw/virtio/vdpa-dev.h > > > > > > -- > > > 2.23.0 > > > > .
在 2022/11/6 13:22, Michael S. Tsirkin 写道: > On Sun, Nov 06, 2022 at 08:17:07AM +0800, Longpeng (Mike, Cloud Infrastructure Service Product Dept.) wrote: >> >> >> 在 2022/11/6 0:43, Michael S. Tsirkin 写道: >>> On Sat, Nov 05, 2022 at 04:36:25PM +0800, Longpeng(Mike) wrote: >>>> From: Longpeng <longpeng2@huawei.com> >>>> >>>> Hi guys, >>>> >>>> With the generic vDPA device, QEMU won't need to touch the device >>>> types any more, such like vfio. >>> >>> With this kind of passthrough migration is completely MIA right? >>> Better add a blocker... >> >> Oh, I missed the "vdpa-dev: mark the device as unmigratable" since v4 and >> I'll add it in the next version. >> >> We'll support passthrough migration in the next step. We have already >> written a demo that can migrate between some offloading cards. > > Hmm ok. Backend disconnect can't work though, can it? State > is by necessity lost when backend crashes. >Yes, it can't. >>> And given this is there an advantage over VFIO? >> >> I think the answer is the same as "why we need vDPA" if we compare it with >> VFIO. > > The answer is mostly because you can migrate and support backend > disconnect, no? > Migrating between different hardware is the first consideration in our requirement, supporting backend disconnect is a low priority. >>> >>>> We can use the generic vDPA device as follow: >>>> -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X >>>> Or >>>> -M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \ >>>> vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x >>> >>>> Changes v6 -> v7: >>>> (v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html) >>>> - rebase. [Jason] >>>> - add documentation . [Stefan] >>>> >>>> Changes v5 -> v6: >>>> Patch 2: >>>> - Turn to the original approach in the RFC to initialize the >>>> virtio_pci_id_info array. [Michael] >>>> https://lore.kernel.org/all/20220105005900.860-2-longpeng2@huawei.com/ >>>> Patch 3: >>>> - Fix logical error of exception handler around the post_init. >>>> [Stefano] >>>> - Fix some coding style warnings. [Stefano] >>>> Patch 4: >>>> - Fix some coding style warnings. [Stefano] >>>> >>>> Changes v4 -> v5: >>>> Patch 3: >>>> - remove vhostfd [Jason] >>>> - support virtio-mmio [Jason] >>>> >>>> Changes v3 -> v4: >>>> v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg877015.html >>>> - reorganize the series [Stefano] >>>> - fix some typos [Stefano] >>>> - fix logical error in vhost_vdpa_device_realize [Stefano] >>>> >>>> Changes v2 -> v3 >>>> Patch 4 & 5: >>>> - only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng] >>>> - s/VQS_NUM/VQS_COUNT [Stefano] >>>> - check both vdpa_dev_fd and vdpa_dev [Stefano] >>>> Patch 6: >>>> - move all steps into vhost_vdpa_device_unrealize. [Stefano] >>>> >>>> Changes RFC -> v2 >>>> Patch 1: >>>> - rename 'pdev_id' to 'trans_devid' [Michael] >>>> - only use transitional device id for the devices >>>> listed in the spec [Michael] >>>> - use macros to make the id_info table clearer [Longpeng] >>>> - add some modern devices in the id_info table [Longpeng] >>>> Patch 2: >>>> - remove the GET_VECTORS_NUM command [Jason] >>>> Patch 4: >>>> - expose vdpa_dev_fd as a QOM preperty [Stefan] >>>> - introduce vhost_vdpa_device_get_u32 as a common >>>> function to make the code clearer [Stefan] >>>> - fix the misleading description of 'dc->desc' [Stefano] >>>> Patch 5: >>>> - check returned number of virtqueues [Stefan] >>>> Patch 6: >>>> - init s->num_queues [Stefano] >>>> - free s->dev.vqs [Stefano] >>>> >>>> >>>> Longpeng (Mike) (4): >>>> virtio: get class_id and pci device id by the virtio id >>>> vdpa: add vdpa-dev support >>>> vdpa: add vdpa-dev-pci support >>>> docs: Add generic vhost-vdpa device documentation >>>> >>>> docs/system/devices/vhost-vdpa-device.rst | 43 +++ >>>> hw/virtio/Kconfig | 5 + >>>> hw/virtio/meson.build | 2 + >>>> hw/virtio/vdpa-dev-pci.c | 102 ++++++ >>>> hw/virtio/vdpa-dev.c | 377 ++++++++++++++++++++++ >>>> hw/virtio/virtio-pci.c | 88 +++++ >>>> include/hw/virtio/vdpa-dev.h | 43 +++ >>>> include/hw/virtio/virtio-pci.h | 5 + >>>> 8 files changed, 665 insertions(+) >>>> create mode 100644 docs/system/devices/vhost-vdpa-device.rst >>>> create mode 100644 hw/virtio/vdpa-dev-pci.c >>>> create mode 100644 hw/virtio/vdpa-dev.c >>>> create mode 100644 include/hw/virtio/vdpa-dev.h >>>> >>>> -- >>>> 2.23.0 >>> >>> . > > > .
On Sun, Nov 06, 2022 at 09:11:39PM +0800, Longpeng (Mike, Cloud Infrastructure Service Product Dept.) wrote: > > > 在 2022/11/6 13:22, Michael S. Tsirkin 写道: > > On Sun, Nov 06, 2022 at 08:17:07AM +0800, Longpeng (Mike, Cloud Infrastructure Service Product Dept.) wrote: > > > > > > > > > 在 2022/11/6 0:43, Michael S. Tsirkin 写道: > > > > On Sat, Nov 05, 2022 at 04:36:25PM +0800, Longpeng(Mike) wrote: > > > > > From: Longpeng <longpeng2@huawei.com> > > > > > > > > > > Hi guys, > > > > > > > > > > With the generic vDPA device, QEMU won't need to touch the device > > > > > types any more, such like vfio. > > > > > > > > With this kind of passthrough migration is completely MIA right? > > > > Better add a blocker... > > > > > > Oh, I missed the "vdpa-dev: mark the device as unmigratable" since v4 and > > > I'll add it in the next version. > > > > > > We'll support passthrough migration in the next step. We have already > > > written a demo that can migrate between some offloading cards. > > > > Hmm ok. Backend disconnect can't work though, can it? State > > is by necessity lost when backend crashes. > > Yes, it can't. > > > > > And given this is there an advantage over VFIO? > > > > > > I think the answer is the same as "why we need vDPA" if we compare it with > > > VFIO. > > > > The answer is mostly because you can migrate and support backend > > disconnect, no? > > > Migrating between different hardware is the first consideration in our > requirement, supporting backend disconnect is a low priority. I dislike non-orthogonal features though ... And the advantage of keeping it out of process with qemu is I presume security? > > > > > > > > > We can use the generic vDPA device as follow: > > > > > -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X > > > > > Or > > > > > -M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \ > > > > > vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x > > > > > > > > > Changes v6 -> v7: > > > > > (v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html) > > > > > - rebase. [Jason] > > > > > - add documentation . [Stefan] > > > > > > > > > > Changes v5 -> v6: > > > > > Patch 2: > > > > > - Turn to the original approach in the RFC to initialize the > > > > > virtio_pci_id_info array. [Michael] > > > > > https://lore.kernel.org/all/20220105005900.860-2-longpeng2@huawei.com/ > > > > > Patch 3: > > > > > - Fix logical error of exception handler around the post_init. > > > > > [Stefano] > > > > > - Fix some coding style warnings. [Stefano] > > > > > Patch 4: > > > > > - Fix some coding style warnings. [Stefano] > > > > > > > > > > Changes v4 -> v5: > > > > > Patch 3: > > > > > - remove vhostfd [Jason] > > > > > - support virtio-mmio [Jason] > > > > > > > > > > Changes v3 -> v4: > > > > > v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg877015.html > > > > > - reorganize the series [Stefano] > > > > > - fix some typos [Stefano] > > > > > - fix logical error in vhost_vdpa_device_realize [Stefano] > > > > > > > > > > Changes v2 -> v3 > > > > > Patch 4 & 5: > > > > > - only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng] > > > > > - s/VQS_NUM/VQS_COUNT [Stefano] > > > > > - check both vdpa_dev_fd and vdpa_dev [Stefano] > > > > > Patch 6: > > > > > - move all steps into vhost_vdpa_device_unrealize. [Stefano] > > > > > > > > > > Changes RFC -> v2 > > > > > Patch 1: > > > > > - rename 'pdev_id' to 'trans_devid' [Michael] > > > > > - only use transitional device id for the devices > > > > > listed in the spec [Michael] > > > > > - use macros to make the id_info table clearer [Longpeng] > > > > > - add some modern devices in the id_info table [Longpeng] > > > > > Patch 2: > > > > > - remove the GET_VECTORS_NUM command [Jason] > > > > > Patch 4: > > > > > - expose vdpa_dev_fd as a QOM preperty [Stefan] > > > > > - introduce vhost_vdpa_device_get_u32 as a common > > > > > function to make the code clearer [Stefan] > > > > > - fix the misleading description of 'dc->desc' [Stefano] > > > > > Patch 5: > > > > > - check returned number of virtqueues [Stefan] > > > > > Patch 6: > > > > > - init s->num_queues [Stefano] > > > > > - free s->dev.vqs [Stefano] > > > > > > > > > > > > > > > Longpeng (Mike) (4): > > > > > virtio: get class_id and pci device id by the virtio id > > > > > vdpa: add vdpa-dev support > > > > > vdpa: add vdpa-dev-pci support > > > > > docs: Add generic vhost-vdpa device documentation > > > > > > > > > > docs/system/devices/vhost-vdpa-device.rst | 43 +++ > > > > > hw/virtio/Kconfig | 5 + > > > > > hw/virtio/meson.build | 2 + > > > > > hw/virtio/vdpa-dev-pci.c | 102 ++++++ > > > > > hw/virtio/vdpa-dev.c | 377 ++++++++++++++++++++++ > > > > > hw/virtio/virtio-pci.c | 88 +++++ > > > > > include/hw/virtio/vdpa-dev.h | 43 +++ > > > > > include/hw/virtio/virtio-pci.h | 5 + > > > > > 8 files changed, 665 insertions(+) > > > > > create mode 100644 docs/system/devices/vhost-vdpa-device.rst > > > > > create mode 100644 hw/virtio/vdpa-dev-pci.c > > > > > create mode 100644 hw/virtio/vdpa-dev.c > > > > > create mode 100644 include/hw/virtio/vdpa-dev.h > > > > > > > > > > -- > > > > > 2.23.0 > > > > > > > > . > > > > > > .
在 2022/11/6 21:47, Michael S. Tsirkin 写道: > On Sun, Nov 06, 2022 at 09:11:39PM +0800, Longpeng (Mike, Cloud Infrastructure Service Product Dept.) wrote: >> >> >> 在 2022/11/6 13:22, Michael S. Tsirkin 写道: >>> On Sun, Nov 06, 2022 at 08:17:07AM +0800, Longpeng (Mike, Cloud Infrastructure Service Product Dept.) wrote: >>>> >>>> >>>> 在 2022/11/6 0:43, Michael S. Tsirkin 写道: >>>>> On Sat, Nov 05, 2022 at 04:36:25PM +0800, Longpeng(Mike) wrote: >>>>>> From: Longpeng <longpeng2@huawei.com> >>>>>> >>>>>> Hi guys, >>>>>> >>>>>> With the generic vDPA device, QEMU won't need to touch the device >>>>>> types any more, such like vfio. >>>>> >>>>> With this kind of passthrough migration is completely MIA right? >>>>> Better add a blocker... >>>> >>>> Oh, I missed the "vdpa-dev: mark the device as unmigratable" since v4 and >>>> I'll add it in the next version. >>>> >>>> We'll support passthrough migration in the next step. We have already >>>> written a demo that can migrate between some offloading cards. >>> >>> Hmm ok. Backend disconnect can't work though, can it? State >>> is by necessity lost when backend crashes. >>> Yes, it can't. >> >>>>> And given this is there an advantage over VFIO? >>>> >>>> I think the answer is the same as "why we need vDPA" if we compare it with >>>> VFIO. >>> >>> The answer is mostly because you can migrate and support backend >>> disconnect, no? >>> >> Migrating between different hardware is the first consideration in our >> requirement, supporting backend disconnect is a low priority. > > I dislike non-orthogonal features though ... > And the advantage of keeping it out of process with qemu is > I presume security? > Yes, this is one of the reasons. The TCB of the generic vdpa device is smaller than the existing vdpa device (needs to use the virtio-net/blk/scsi emulation codes). Besides, the generic vdpa device can support any virtio device, but the existing vdpa device only supports virtio-net yet. Though the existing vdpa device is more powerful and the generic vdpa device would miss some features, it can be an alternative for some users. >>>>> >>>>>> We can use the generic vDPA device as follow: >>>>>> -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X >>>>>> Or >>>>>> -M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \ >>>>>> vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x >>>>> >>>>>> Changes v6 -> v7: >>>>>> (v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html) >>>>>> - rebase. [Jason] >>>>>> - add documentation . [Stefan] >>>>>> >>>>>> Changes v5 -> v6: >>>>>> Patch 2: >>>>>> - Turn to the original approach in the RFC to initialize the >>>>>> virtio_pci_id_info array. [Michael] >>>>>> https://lore.kernel.org/all/20220105005900.860-2-longpeng2@huawei.com/ >>>>>> Patch 3: >>>>>> - Fix logical error of exception handler around the post_init. >>>>>> [Stefano] >>>>>> - Fix some coding style warnings. [Stefano] >>>>>> Patch 4: >>>>>> - Fix some coding style warnings. [Stefano] >>>>>> >>>>>> Changes v4 -> v5: >>>>>> Patch 3: >>>>>> - remove vhostfd [Jason] >>>>>> - support virtio-mmio [Jason] >>>>>> >>>>>> Changes v3 -> v4: >>>>>> v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg877015.html >>>>>> - reorganize the series [Stefano] >>>>>> - fix some typos [Stefano] >>>>>> - fix logical error in vhost_vdpa_device_realize [Stefano] >>>>>> >>>>>> Changes v2 -> v3 >>>>>> Patch 4 & 5: >>>>>> - only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng] >>>>>> - s/VQS_NUM/VQS_COUNT [Stefano] >>>>>> - check both vdpa_dev_fd and vdpa_dev [Stefano] >>>>>> Patch 6: >>>>>> - move all steps into vhost_vdpa_device_unrealize. [Stefano] >>>>>> >>>>>> Changes RFC -> v2 >>>>>> Patch 1: >>>>>> - rename 'pdev_id' to 'trans_devid' [Michael] >>>>>> - only use transitional device id for the devices >>>>>> listed in the spec [Michael] >>>>>> - use macros to make the id_info table clearer [Longpeng] >>>>>> - add some modern devices in the id_info table [Longpeng] >>>>>> Patch 2: >>>>>> - remove the GET_VECTORS_NUM command [Jason] >>>>>> Patch 4: >>>>>> - expose vdpa_dev_fd as a QOM preperty [Stefan] >>>>>> - introduce vhost_vdpa_device_get_u32 as a common >>>>>> function to make the code clearer [Stefan] >>>>>> - fix the misleading description of 'dc->desc' [Stefano] >>>>>> Patch 5: >>>>>> - check returned number of virtqueues [Stefan] >>>>>> Patch 6: >>>>>> - init s->num_queues [Stefano] >>>>>> - free s->dev.vqs [Stefano] >>>>>> >>>>>> >>>>>> Longpeng (Mike) (4): >>>>>> virtio: get class_id and pci device id by the virtio id >>>>>> vdpa: add vdpa-dev support >>>>>> vdpa: add vdpa-dev-pci support >>>>>> docs: Add generic vhost-vdpa device documentation >>>>>> >>>>>> docs/system/devices/vhost-vdpa-device.rst | 43 +++ >>>>>> hw/virtio/Kconfig | 5 + >>>>>> hw/virtio/meson.build | 2 + >>>>>> hw/virtio/vdpa-dev-pci.c | 102 ++++++ >>>>>> hw/virtio/vdpa-dev.c | 377 ++++++++++++++++++++++ >>>>>> hw/virtio/virtio-pci.c | 88 +++++ >>>>>> include/hw/virtio/vdpa-dev.h | 43 +++ >>>>>> include/hw/virtio/virtio-pci.h | 5 + >>>>>> 8 files changed, 665 insertions(+) >>>>>> create mode 100644 docs/system/devices/vhost-vdpa-device.rst >>>>>> create mode 100644 hw/virtio/vdpa-dev-pci.c >>>>>> create mode 100644 hw/virtio/vdpa-dev.c >>>>>> create mode 100644 include/hw/virtio/vdpa-dev.h >>>>>> >>>>>> -- >>>>>> 2.23.0 >>>>> >>>>> . >>> >>> >>> . > > > .
From: Longpeng <longpeng2@huawei.com> Hi guys, With the generic vDPA device, QEMU won't need to touch the device types any more, such like vfio. We can use the generic vDPA device as follow: -device vhost-vdpa-device-pci,vhostdev=/dev/vhost-vdpa-X Or -M microvm -m 512m -smp 2 -kernel ... -initrd ... -device \ vhost-vdpa-device,vhostdev=/dev/vhost-vdpa-x Changes v6 -> v7: (v6: https://mail.gnu.org/archive/html/qemu-devel/2022-05/msg02821.html) - rebase. [Jason] - add documentation . [Stefan] Changes v5 -> v6: Patch 2: - Turn to the original approach in the RFC to initialize the virtio_pci_id_info array. [Michael] https://lore.kernel.org/all/20220105005900.860-2-longpeng2@huawei.com/ Patch 3: - Fix logical error of exception handler around the post_init. [Stefano] - Fix some coding style warnings. [Stefano] Patch 4: - Fix some coding style warnings. [Stefano] Changes v4 -> v5: Patch 3: - remove vhostfd [Jason] - support virtio-mmio [Jason] Changes v3 -> v4: v3: https://www.mail-archive.com/qemu-devel@nongnu.org/msg877015.html - reorganize the series [Stefano] - fix some typos [Stefano] - fix logical error in vhost_vdpa_device_realize [Stefano] Changes v2 -> v3 Patch 4 & 5: - only call vdpa ioctls in vdpa-dev.c [Stefano, Longpeng] - s/VQS_NUM/VQS_COUNT [Stefano] - check both vdpa_dev_fd and vdpa_dev [Stefano] Patch 6: - move all steps into vhost_vdpa_device_unrealize. [Stefano] Changes RFC -> v2 Patch 1: - rename 'pdev_id' to 'trans_devid' [Michael] - only use transitional device id for the devices listed in the spec [Michael] - use macros to make the id_info table clearer [Longpeng] - add some modern devices in the id_info table [Longpeng] Patch 2: - remove the GET_VECTORS_NUM command [Jason] Patch 4: - expose vdpa_dev_fd as a QOM preperty [Stefan] - introduce vhost_vdpa_device_get_u32 as a common function to make the code clearer [Stefan] - fix the misleading description of 'dc->desc' [Stefano] Patch 5: - check returned number of virtqueues [Stefan] Patch 6: - init s->num_queues [Stefano] - free s->dev.vqs [Stefano] Longpeng (Mike) (4): virtio: get class_id and pci device id by the virtio id vdpa: add vdpa-dev support vdpa: add vdpa-dev-pci support docs: Add generic vhost-vdpa device documentation docs/system/devices/vhost-vdpa-device.rst | 43 +++ hw/virtio/Kconfig | 5 + hw/virtio/meson.build | 2 + hw/virtio/vdpa-dev-pci.c | 102 ++++++ hw/virtio/vdpa-dev.c | 377 ++++++++++++++++++++++ hw/virtio/virtio-pci.c | 88 +++++ include/hw/virtio/vdpa-dev.h | 43 +++ include/hw/virtio/virtio-pci.h | 5 + 8 files changed, 665 insertions(+) create mode 100644 docs/system/devices/vhost-vdpa-device.rst create mode 100644 hw/virtio/vdpa-dev-pci.c create mode 100644 hw/virtio/vdpa-dev.c create mode 100644 include/hw/virtio/vdpa-dev.h