mbox series

[v5,0/4] Virtio shared dma-buf

Message ID 20230802090824.91688-1-aesteve@redhat.com (mailing list archive)
Headers show
Series Virtio shared dma-buf | expand

Message

Albert Esteve Aug. 2, 2023, 9:08 a.m. UTC
v1 link -> https://lists.gnu.org/archive/html/qemu-devel/2023-05/msg00598.html
v2 link -> https://lists.gnu.org/archive/html/qemu-devel/2023-05/msg04530.html
v3 link -> https://lists.gnu.org/archive/html/qemu-devel/2023-05/msg06126.html
v4 link -> https://lists.gnu.org/archive/html/qemu-devel/2023-06/msg05174.html
v4 -> v5:
- Allow shared table to hold pointers for vhost devices, in a struct that defines the types that the table can store
- New message VHOST_USER_GET_SHARED_OBJECT to retrieve objects stored in vhost backends
- Minor additions to support the previous items (e.g. new test usecases).

This patch covers the required steps to add support for virtio cross-device resource sharing[1],
which support is already available in the kernel.

The main usecase will be sharing dma buffers from virtio-gpu devices (as the exporter
-see VIRTIO_GPU_CMD_RESOURCE_ASSIGN_UUID in [2]), to virtio-video (under discussion)
devices (as the buffer-user or importer). Therefore, even though virtio specs talk about
resources or objects[3], this patch adds the infrastructure with dma-bufs in mind.
Note that virtio specs let the devices themselves define what a vitio object is.

These are the main parts that are covered in the patch:

- Add hash function to uuid module
- Shared resources table, to hold all resources that can be shared in the host and their assigned UUID,
  or pointers to the backend holding the resource
- Internal shared table API for virtio devices to add, lookup and remove resources
- Unit test to verify the API
- New messages to the vhost-user protocol to allow backend to interact with the shared
  table API through the control socket
- New vhost-user feature bit to enable shared objects feature

Applies cleanly to 38a6de80b917b2a822cff0e38d83563ab401c890

[1] - https://lwn.net/Articles/828988/
[2] - https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/virtio-v1.2-csd01.html#x1-3730006
[3] - https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/virtio-v1.2-csd01.html#x1-10500011

Albert Esteve (4):
  uuid: add a hash function
  virtio-dmabuf: introduce virtio-dmabuf
  vhost-user: add shared_object msg
  vhost-user: refactor send_resp code

 MAINTAINERS                               |   7 +
 docs/interop/vhost-user.rst               |  57 +++++++
 hw/display/meson.build                    |   1 +
 hw/display/virtio-dmabuf.c                | 136 +++++++++++++++++
 hw/virtio/vhost-user.c                    | 174 ++++++++++++++++++++--
 include/hw/virtio/vhost-backend.h         |   3 +
 include/hw/virtio/virtio-dmabuf.h         | 103 +++++++++++++
 include/qemu/uuid.h                       |   2 +
 subprojects/libvhost-user/libvhost-user.c | 118 +++++++++++++++
 subprojects/libvhost-user/libvhost-user.h |  55 ++++++-
 tests/unit/meson.build                    |   1 +
 tests/unit/test-uuid.c                    |  27 ++++
 tests/unit/test-virtio-dmabuf.c           | 137 +++++++++++++++++
 util/uuid.c                               |  14 ++
 14 files changed, 821 insertions(+), 14 deletions(-)
 create mode 100644 hw/display/virtio-dmabuf.c
 create mode 100644 include/hw/virtio/virtio-dmabuf.h
 create mode 100644 tests/unit/test-virtio-dmabuf.c

Comments

Albert Esteve Aug. 21, 2023, 12:37 p.m. UTC | #1
Hi all,

A little bump for this patch, sorry for the extra noise.

Regards,
Albert


On Wed, Aug 2, 2023 at 11:08 AM Albert Esteve <aesteve@redhat.com> wrote:

> v1 link ->
> https://lists.gnu.org/archive/html/qemu-devel/2023-05/msg00598.html
> v2 link ->
> https://lists.gnu.org/archive/html/qemu-devel/2023-05/msg04530.html
> v3 link ->
> https://lists.gnu.org/archive/html/qemu-devel/2023-05/msg06126.html
> v4 link ->
> https://lists.gnu.org/archive/html/qemu-devel/2023-06/msg05174.html
> v4 -> v5:
> - Allow shared table to hold pointers for vhost devices, in a struct that
> defines the types that the table can store
> - New message VHOST_USER_GET_SHARED_OBJECT to retrieve objects stored in
> vhost backends
> - Minor additions to support the previous items (e.g. new test usecases).
>
> This patch covers the required steps to add support for virtio
> cross-device resource sharing[1],
> which support is already available in the kernel.
>
> The main usecase will be sharing dma buffers from virtio-gpu devices (as
> the exporter
> -see VIRTIO_GPU_CMD_RESOURCE_ASSIGN_UUID in [2]), to virtio-video (under
> discussion)
> devices (as the buffer-user or importer). Therefore, even though virtio
> specs talk about
> resources or objects[3], this patch adds the infrastructure with dma-bufs
> in mind.
> Note that virtio specs let the devices themselves define what a vitio
> object is.
>
> These are the main parts that are covered in the patch:
>
> - Add hash function to uuid module
> - Shared resources table, to hold all resources that can be shared in the
> host and their assigned UUID,
>   or pointers to the backend holding the resource
> - Internal shared table API for virtio devices to add, lookup and remove
> resources
> - Unit test to verify the API
> - New messages to the vhost-user protocol to allow backend to interact
> with the shared
>   table API through the control socket
> - New vhost-user feature bit to enable shared objects feature
>
> Applies cleanly to 38a6de80b917b2a822cff0e38d83563ab401c890
>
> [1] - https://lwn.net/Articles/828988/
> [2] -
> https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/virtio-v1.2-csd01.html#x1-3730006
> [3] -
> https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/virtio-v1.2-csd01.html#x1-10500011
>
> Albert Esteve (4):
>   uuid: add a hash function
>   virtio-dmabuf: introduce virtio-dmabuf
>   vhost-user: add shared_object msg
>   vhost-user: refactor send_resp code
>
>  MAINTAINERS                               |   7 +
>  docs/interop/vhost-user.rst               |  57 +++++++
>  hw/display/meson.build                    |   1 +
>  hw/display/virtio-dmabuf.c                | 136 +++++++++++++++++
>  hw/virtio/vhost-user.c                    | 174 ++++++++++++++++++++--
>  include/hw/virtio/vhost-backend.h         |   3 +
>  include/hw/virtio/virtio-dmabuf.h         | 103 +++++++++++++
>  include/qemu/uuid.h                       |   2 +
>  subprojects/libvhost-user/libvhost-user.c | 118 +++++++++++++++
>  subprojects/libvhost-user/libvhost-user.h |  55 ++++++-
>  tests/unit/meson.build                    |   1 +
>  tests/unit/test-uuid.c                    |  27 ++++
>  tests/unit/test-virtio-dmabuf.c           | 137 +++++++++++++++++
>  util/uuid.c                               |  14 ++
>  14 files changed, 821 insertions(+), 14 deletions(-)
>  create mode 100644 hw/display/virtio-dmabuf.c
>  create mode 100644 include/hw/virtio/virtio-dmabuf.h
>  create mode 100644 tests/unit/test-virtio-dmabuf.c
>
> --
> 2.40.0
>
>
Michael S. Tsirkin Sept. 5, 2023, 8:45 p.m. UTC | #2
I was hoping for some acks from Gerd or anyone else with a clue
about graphics, but as that doesn't seem to happen I'll merge.
Thanks!

On Mon, Aug 21, 2023 at 02:37:56PM +0200, Albert Esteve wrote:
> Hi all,
> 
> A little bump for this patch, sorry for the extra noise.
> 
> Regards,
> Albert
> 
> 
> On Wed, Aug 2, 2023 at 11:08 AM Albert Esteve <aesteve@redhat.com> wrote:
> 
>     v1 link -> https://lists.gnu.org/archive/html/qemu-devel/2023-05/
>     msg00598.html
>     v2 link -> https://lists.gnu.org/archive/html/qemu-devel/2023-05/
>     msg04530.html
>     v3 link -> https://lists.gnu.org/archive/html/qemu-devel/2023-05/
>     msg06126.html
>     v4 link -> https://lists.gnu.org/archive/html/qemu-devel/2023-06/
>     msg05174.html
>     v4 -> v5:
>     - Allow shared table to hold pointers for vhost devices, in a struct that
>     defines the types that the table can store
>     - New message VHOST_USER_GET_SHARED_OBJECT to retrieve objects stored in
>     vhost backends
>     - Minor additions to support the previous items (e.g. new test usecases).
> 
>     This patch covers the required steps to add support for virtio cross-device
>     resource sharing[1],
>     which support is already available in the kernel.
> 
>     The main usecase will be sharing dma buffers from virtio-gpu devices (as
>     the exporter
>     -see VIRTIO_GPU_CMD_RESOURCE_ASSIGN_UUID in [2]), to virtio-video (under
>     discussion)
>     devices (as the buffer-user or importer). Therefore, even though virtio
>     specs talk about
>     resources or objects[3], this patch adds the infrastructure with dma-bufs
>     in mind.
>     Note that virtio specs let the devices themselves define what a vitio
>     object is.
> 
>     These are the main parts that are covered in the patch:
> 
>     - Add hash function to uuid module
>     - Shared resources table, to hold all resources that can be shared in the
>     host and their assigned UUID,
>       or pointers to the backend holding the resource
>     - Internal shared table API for virtio devices to add, lookup and remove
>     resources
>     - Unit test to verify the API
>     - New messages to the vhost-user protocol to allow backend to interact with
>     the shared
>       table API through the control socket
>     - New vhost-user feature bit to enable shared objects feature
> 
>     Applies cleanly to 38a6de80b917b2a822cff0e38d83563ab401c890
> 
>     [1] - https://lwn.net/Articles/828988/
>     [2] - https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/
>     virtio-v1.2-csd01.html#x1-3730006
>     [3] - https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/
>     virtio-v1.2-csd01.html#x1-10500011
> 
>     Albert Esteve (4):
>       uuid: add a hash function
>       virtio-dmabuf: introduce virtio-dmabuf
>       vhost-user: add shared_object msg
>       vhost-user: refactor send_resp code
> 
>      MAINTAINERS                               |   7 +
>      docs/interop/vhost-user.rst               |  57 +++++++
>      hw/display/meson.build                    |   1 +
>      hw/display/virtio-dmabuf.c                | 136 +++++++++++++++++
>      hw/virtio/vhost-user.c                    | 174 ++++++++++++++++++++--
>      include/hw/virtio/vhost-backend.h         |   3 +
>      include/hw/virtio/virtio-dmabuf.h         | 103 +++++++++++++
>      include/qemu/uuid.h                       |   2 +
>      subprojects/libvhost-user/libvhost-user.c | 118 +++++++++++++++
>      subprojects/libvhost-user/libvhost-user.h |  55 ++++++-
>      tests/unit/meson.build                    |   1 +
>      tests/unit/test-uuid.c                    |  27 ++++
>      tests/unit/test-virtio-dmabuf.c           | 137 +++++++++++++++++
>      util/uuid.c                               |  14 ++
>      14 files changed, 821 insertions(+), 14 deletions(-)
>      create mode 100644 hw/display/virtio-dmabuf.c
>      create mode 100644 include/hw/virtio/virtio-dmabuf.h
>      create mode 100644 tests/unit/test-virtio-dmabuf.c
> 
>     --
>     2.40.0
> 
>
Philippe Mathieu-Daudé Sept. 6, 2023, 6:13 a.m. UTC | #3
Hi Michael,

On 5/9/23 22:45, Michael S. Tsirkin wrote:
> I was hoping for some acks from Gerd or anyone else with a clue
> about graphics, but as that doesn't seem to happen I'll merge.
> Thanks!

I made few late comments. Patch #3 doesn't build (thus
break git-bisections). I also have some concern about locking.
I'd rather see a v6, do you mind dropping v5 from your queue?

Thanks,

Phil.

> On Mon, Aug 21, 2023 at 02:37:56PM +0200, Albert Esteve wrote:
>> Hi all,
>>
>> A little bump for this patch, sorry for the extra noise.
>>
>> Regards,
>> Albert
Albert Esteve Sept. 6, 2023, 9:39 a.m. UTC | #4
On Wed, Sep 6, 2023 at 8:13 AM Philippe Mathieu-Daudé <philmd@linaro.org>
wrote:

> Hi Michael,
>
> On 5/9/23 22:45, Michael S. Tsirkin wrote:
> > I was hoping for some acks from Gerd or anyone else with a clue
> > about graphics, but as that doesn't seem to happen I'll merge.
> > Thanks!
>
> I made few late comments. Patch #3 doesn't build (thus
> break git-bisections). I also have some concern about locking.
> I'd rather see a v6, do you mind dropping v5 from your queue?
>
> Thanks,
>
> Phil.
>

I have the v6 ready. I will wait for Michael response and post it,
to ensure that I do not step on his toes.
Thank you both!


>
> > On Mon, Aug 21, 2023 at 02:37:56PM +0200, Albert Esteve wrote:
> >> Hi all,
> >>
> >> A little bump for this patch, sorry for the extra noise.
> >>
> >> Regards,
> >> Albert
>
>
Philippe Mathieu-Daudé Sept. 6, 2023, 10:16 a.m. UTC | #5
On 6/9/23 11:39, Albert Esteve wrote:
> 
> 
> On Wed, Sep 6, 2023 at 8:13 AM Philippe Mathieu-Daudé <philmd@linaro.org 
> <mailto:philmd@linaro.org>> wrote:
> 
>     Hi Michael,
> 
>     On 5/9/23 22:45, Michael S. Tsirkin wrote:
>      > I was hoping for some acks from Gerd or anyone else with a clue
>      > about graphics, but as that doesn't seem to happen I'll merge.
>      > Thanks!
> 
>     I made few late comments. Patch #3 doesn't build (thus
>     break git-bisections). I also have some concern about locking.
>     I'd rather see a v6, do you mind dropping v5 from your queue?
> 
>     Thanks,
> 
>     Phil.
> 
> 
> I have the v6 ready. I will wait for Michael response and post it,
> to ensure that I do not step on his toes.
> Thank you both!

If you have them ready, don't wait to post them, because:
- we can review them
- Michael can directly pick it instead of dropping the current
   one (you'd have to wait his next PR)