From patchwork Wed Mar 22 14:21:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Albert Esteve X-Patchwork-Id: 13184154 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2C97BC6FD1C for ; Wed, 22 Mar 2023 14:23:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pezLa-0007pA-Ad; Wed, 22 Mar 2023 10:21:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLW-0007ns-LL for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:21:46 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLU-0007Ft-2S for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:21:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679494903; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vx20FkO4qrTT1t0kX/w4FLGP9efJRwvPTxvvgxnVEMY=; b=cNiaztnapSIqtpSg1lX/viYwdHSz/nh6e9USApL7IFRL22nsuMINaBHxXHyhPBVfLoehhd YFvy67cNTjogDOmsJ5lf5MrrTdwRuTP0k1p+wFnyUwchikbNJXt0HJq4Woi9gdYmgB9gG9 s1Z+NOS34HlO1wFTqKaiTGEfuPOn/Xo= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-128-cQH5Kai0OKGOM5f678BHcw-1; Wed, 22 Mar 2023 10:21:39 -0400 X-MC-Unique: cQH5Kai0OKGOM5f678BHcw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 967E985A5A3; Wed, 22 Mar 2023 14:21:39 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.226.254]) by smtp.corp.redhat.com (Postfix) with ESMTP id EDE1240C6E67; Wed, 22 Mar 2023 14:21:36 +0000 (UTC) From: Albert Esteve To: qemu-devel@nongnu.org Cc: Thomas Huth , fmartine@redhat.com, Gerd Hoffmann , eballetb@redhat.com, Albert Esteve , alex.bennee@linaro.org, Laurent Vivier , "Michael S. Tsirkin" , sgarzare@redhat.com, =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , peter.griffin@linaro.org, Paolo Bonzini Subject: [PATCH 01/12] docs: Add a vhost-virtio-video rst file Date: Wed, 22 Mar 2023 15:21:21 +0100 Message-Id: <20230322142132.22909-2-aesteve@redhat.com> In-Reply-To: <20230322142132.22909-1-aesteve@redhat.com> References: <20230322142132.22909-1-aesteve@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Griffin Add rst file with a brief description of the daemon and a cheat sheet of commands. Signed-off-by: Peter Griffin Signed-off-by: Albert Esteve --- docs/system/devices/vhost-user-video.rst | 124 +++++++++++++++++++++++ 1 file changed, 124 insertions(+) create mode 100644 docs/system/devices/vhost-user-video.rst diff --git a/docs/system/devices/vhost-user-video.rst b/docs/system/devices/vhost-user-video.rst new file mode 100644 index 0000000000..ff0a8fe5c7 --- /dev/null +++ b/docs/system/devices/vhost-user-video.rst @@ -0,0 +1,124 @@ +===================== +QEMU vhost-user-video +===================== + +Overview +-------- + +This vmm translates from virtio-video v3 protocol and writes +to a v4l2 mem2mem stateful decoder/encoder device [1]. v3 was +chosen as that is what the virtio-video Linux frontend driver +currently implements. + +The primary goal so far is to enable development of virtio-video +frontend driver using purely open source software. Using vicodec +v4l2 stateful decoder on the host for testing then allows a pure +virtual environment for development and testing. + +Currently the vmm only supports v4l2 stateful devices, and the +intention is it will be used with Arm SoCs that implement stateful +decode/encode devices such as Qcom Venus, RPi, MediaTek etc. + +A Qemu + vicodec setup for virtio-video should also allow for +CI systems like kernelci, lkft to test the virtio-video interface +easily. + +Currently support for VAAPI or decoding via libavcodec or similar +libraries is not implemented, but this could be added in the future. + +Some example commands are provided below on how to run the daemon +and achieve a video decode using vicodec and a link to some test +content. + +[1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/dev-decoder.html + +[2] https://lwn.net/Articles/760650/ + +Examples +-------- + +Guest Linux kernel modules: + +:: + + CONFIG_MEDIA_SUPPORT=y + CONFIG_MEDIA_TEST_SUPPORT=y + CONFIG_V4L_TEST_DRIVERS=y + CONFIG_VIRTIO_VIDEO=y + CONFIG_GDB_SCRIPTS=y + CONFIG_DRM_VIRTIO_GPU=y + +Host kernel modules: + +:: + + CONFIG_MEDIA_SUPPORT=y + CONFIG_MEDIA_TEST_SUPPORT=y + CONFIG_V4L_TEST_DRIVERS=y + CONFIG_VIDEO_VICODEC=y + +Note: Vicodec has been recently included in the Fedora kernel releases, +but it is not yet set on the default Debian kernel. + +The daemon should be started first (video3 typically is the stateful video): + +:: + + host# vhost-user-video --socket-path=/tmp/video.sock --v4l2-device=/dev/video3 + +The QEMU invocation needs to create a chardev socket the device can +use to communicate as well as share the guests memory over a memfd. + +:: + + host# qemu-system \ + -device vhost-user-video-pci,chardev=video,id=video \ + -chardev socket,path=/tmp//video.sock,id=video \ + -m 4096 \ + -object memory-backend-file,id=mem,size=4G,mem-path=/dev/shm,share=on \ + -numa node,memdev=mem \ + ... + +After booting, the device should be available at /dev/video0: + +:: + + guest# v4l2-ctl -d/dev/video0 --info + Driver Info: + Driver name : virtio-video + Card type : + Bus info : virtio:stateful-decoder + Driver version : 6.1.0 + Capabilities : 0x84204000 + Video Memory-to-Memory Multiplanar + Streaming + Extended Pix Format + Device Capabilities + Device Caps : 0x04204000 + Video Memory-to-Memory Multiplanar + Streaming + Extended Pix Format + +Example v4l2-ctl decode command: + +:: + + guest# wget https://people.linaro.org/~peter.griffin/jelly_640_480-420P.fwht + guest# v4l2-ctl -d0 -x width=640,height=480 -v width=640,height=480,pixelformat=YU12 \ + --stream-mmap --stream-out-mmap --stream-from jelly_640_480-420P.fwht \ + --stream-to out-jelly-640-480.YU12 + +Play the raw decoded video with ffplay or mplayer + +:: + + guest# ffplay -loglevel warning -v info -f rawvideo -pixel_format yuv420p \ + -video_size "640x480" ./out-jelly-640-480.YU12 + guest# mplayer -demuxer rawvideo -rawvideo \ + format=i420:w=640:h=480:fps=25 out-jelly-640-480.YU12 + +Enable v4l2 debug in virtio-video driver + +:: + + # echo 0x1f > /sys/class/video4linux/videoX/dev_debug From patchwork Wed Mar 22 14:21:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Albert Esteve X-Patchwork-Id: 13184158 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84FBBC6FD1C for ; Wed, 22 Mar 2023 14:23:50 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pezLa-0007ps-OY; Wed, 22 Mar 2023 10:21:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLY-0007oT-Jx for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:21:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLV-0007GY-OY for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:21:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679494905; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=84Cr/6MZMABQKFW5vYpViPyJnkJMcWbdHEFemWFRFTg=; b=Zugf4i22tvTZTzqAyGeHoZqXuz/iQBuBLVZHYwP9ok/x6aUiX+QacVKkJyMUbEatEVOi70 hFsR4rWFEaDDmzqF+aZZbgBM/Iv2FdCnGxcdIHIyxFFDB1JxnPUTP1l3hj3QR9PdPAaJaz u81kdr+NxPNn0/xBwNmBlLWrpxw6pWQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-517-OT--XM9WPCiTjdxKUkK2jQ-1; Wed, 22 Mar 2023 10:21:42 -0400 X-MC-Unique: OT--XM9WPCiTjdxKUkK2jQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5582285530E; Wed, 22 Mar 2023 14:21:42 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.226.254]) by smtp.corp.redhat.com (Postfix) with ESMTP id DBEB240C6E67; Wed, 22 Mar 2023 14:21:39 +0000 (UTC) From: Albert Esteve To: qemu-devel@nongnu.org Cc: Thomas Huth , fmartine@redhat.com, Gerd Hoffmann , eballetb@redhat.com, Albert Esteve , alex.bennee@linaro.org, Laurent Vivier , "Michael S. Tsirkin" , sgarzare@redhat.com, =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , peter.griffin@linaro.org, Paolo Bonzini Subject: [PATCH 02/12] MAINTAINERS: Add virtio-video section Date: Wed, 22 Mar 2023 15:21:22 +0100 Message-Id: <20230322142132.22909-3-aesteve@redhat.com> In-Reply-To: <20230322142132.22909-1-aesteve@redhat.com> References: <20230322142132.22909-1-aesteve@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Griffin Add myself as maintainer of the virtio-video files added in this series. Signed-off-by: Peter Griffin Signed-off-by: Albert Esteve --- MAINTAINERS | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 9b56ccdd92..c6cc04c4f7 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2176,6 +2176,14 @@ F: hw/virtio/vhost-user-gpio* F: include/hw/virtio/vhost-user-gpio.h F: tests/qtest/libqos/virtio-gpio.* +virtio-video +M: Albert Esteve +S: Supported +F: hw/display/vhost-user-video.c +F: hw/display/vhost-user-video-pci.c +F: include/hw/virtio/vhost-user-video.h +F: tools/vhost-user-video/* + virtio-crypto M: Gonglei S: Supported From patchwork Wed Mar 22 14:21:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Albert Esteve X-Patchwork-Id: 13184155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7A270C6FD1C for ; Wed, 22 Mar 2023 14:23:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pezLp-00084S-MI; Wed, 22 Mar 2023 10:22:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLl-00081b-1J for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:22:01 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLi-0007LD-Nt for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:22:00 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679494917; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SSUNF/BimE8C399TFeTrg4OO5iVpzjrPmfPv58jwtyI=; b=RjoeUSXtLEcDXO1Chtj05lrFYGslquygpaTycZf9rcfEJjNQb/Gq14/GBH5fF7HikjrNXW cXwUAwQN5FKTt727BH2UJLyK1dlMr+v60euzruEpVvT9sIRCKS0AuZwXZm34Xmn9HYLshf ewPB0jOMhttrFv5QtKIPqxr6ljbSG+0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-296-UumvJf_mPQ2G1sZ5rU8NrQ-1; Wed, 22 Mar 2023 10:21:50 -0400 X-MC-Unique: UumvJf_mPQ2G1sZ5rU8NrQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 90C2A100DEA9; Wed, 22 Mar 2023 14:21:45 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.226.254]) by smtp.corp.redhat.com (Postfix) with ESMTP id A9C2A40C6E68; Wed, 22 Mar 2023 14:21:42 +0000 (UTC) From: Albert Esteve To: qemu-devel@nongnu.org Cc: Thomas Huth , fmartine@redhat.com, Gerd Hoffmann , eballetb@redhat.com, Albert Esteve , alex.bennee@linaro.org, Laurent Vivier , "Michael S. Tsirkin" , sgarzare@redhat.com, =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , peter.griffin@linaro.org, Paolo Bonzini Subject: [PATCH 03/12] vhost-user-video: boiler plate code for vhost-user-video device Date: Wed, 22 Mar 2023 15:21:23 +0100 Message-Id: <20230322142132.22909-4-aesteve@redhat.com> In-Reply-To: <20230322142132.22909-1-aesteve@redhat.com> References: <20230322142132.22909-1-aesteve@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Griffin Signed-off-by: Peter Griffin Signed-off-by: Albert Esteve --- hw/display/Kconfig | 5 + hw/display/meson.build | 3 + hw/display/vhost-user-video.c | 410 +++++++++++++++++++++++++++ include/hw/virtio/vhost-user-video.h | 42 +++ 4 files changed, 460 insertions(+) create mode 100644 hw/display/vhost-user-video.c create mode 100644 include/hw/virtio/vhost-user-video.h diff --git a/hw/display/Kconfig b/hw/display/Kconfig index 7b3da68d1c..9a9dd3e335 100644 --- a/hw/display/Kconfig +++ b/hw/display/Kconfig @@ -118,6 +118,11 @@ config VHOST_USER_VGA default y depends on VIRTIO_VGA && VHOST_USER_GPU +config VHOST_USER_VIDEO + bool + default y + depends on VIRTIO && VHOST_USER + config DPCD bool select AUX diff --git a/hw/display/meson.build b/hw/display/meson.build index 4191694380..5eacf84b1f 100644 --- a/hw/display/meson.build +++ b/hw/display/meson.build @@ -37,6 +37,9 @@ softmmu_ss.add(when: 'CONFIG_MACFB', if_true: files('macfb.c')) softmmu_ss.add(when: 'CONFIG_NEXTCUBE', if_true: files('next-fb.c')) specific_ss.add(when: 'CONFIG_VGA', if_true: files('vga.c')) +specific_ss.add(when: 'CONFIG_VHOST_USER_VIDEO', if_true: files('vhost-user-video.c')) +specific_ss.add(when: ['CONFIG_VHOST_USER_VIDEO', 'CONFIG_VIRTIO_PCI' ], + if_true: files('vhost-user-video-pci.c')) if (config_all_devices.has_key('CONFIG_VGA_CIRRUS') or config_all_devices.has_key('CONFIG_VGA_PCI') or diff --git a/hw/display/vhost-user-video.c b/hw/display/vhost-user-video.c new file mode 100644 index 0000000000..9cc6a717d5 --- /dev/null +++ b/hw/display/vhost-user-video.c @@ -0,0 +1,410 @@ +/* + * Vhost-user VIDEO virtio device + * + * This is the boilerplate for instantiating a vhost-user device + * implementing a virtio-video device. + * + * The virtio video decoder and encoder devices are virtual devices that + * support encoding and decoding respectively. + * + * The actual back-end for this driver is the vhost-user-video daemon. + * The code here just connects up the device in QEMU and allows it to + * be instantiated. + * + * Copyright Red Hat, Inc. 2023 + * Copyright (c) 2021 Linaro Ltd + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "hw/qdev-properties.h" +#include "hw/virtio/virtio-bus.h" +#include "hw/virtio/vhost-user-video.h" +#include "qemu/error-report.h" + +#define MAX_CAPS_LEN 4096 + +static void +vhost_user_video_get_config(VirtIODevice *vdev, uint8_t *config_data) +{ + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + Error *local_err = NULL; + int ret; + + memset(config_data, 0, sizeof(struct virtio_video_config)); + + ret = vhost_dev_get_config(&video->vhost_dev, + config_data, sizeof(struct virtio_video_config), + &local_err); + if (ret) { + error_report_err(local_err); + return; + } +} + +static void vhost_user_video_start(VirtIODevice *vdev) +{ + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + int ret; + int i; + + if (!k->set_guest_notifiers) { + error_report("binding does not support guest notifiers"); + return; + } + + ret = vhost_dev_enable_notifiers(&video->vhost_dev, vdev); + if (ret < 0) { + error_report("Error enabling host notifiers: %d", -ret); + return; + } + + ret = k->set_guest_notifiers(qbus->parent, video->vhost_dev.nvqs, true); + if (ret < 0) { + error_report("Error binding guest notifier: %d", -ret); + goto err_host_notifiers; + } + + video->vhost_dev.acked_features = vdev->guest_features; + + video->vhost_dev.vq_index_end = video->vhost_dev.nvqs; + ret = vhost_dev_start(&video->vhost_dev, vdev, true); + if (ret < 0) { + error_report("Error starting vhost-user-video: %d", -ret); + goto err_guest_notifiers; + } + + /* + * guest_notifier_mask/pending not used yet, so just unmask + * everything here. virtio-pci will do the right thing by + * enabling/disabling irqfd. + */ + for (i = 0; i < video->vhost_dev.nvqs; i++) { + vhost_virtqueue_mask(&video->vhost_dev, vdev, i, false); + } + + return; + +err_guest_notifiers: + k->set_guest_notifiers(qbus->parent, video->vhost_dev.nvqs, false); +err_host_notifiers: + vhost_dev_disable_notifiers(&video->vhost_dev, vdev); +} + +static void vhost_user_video_stop(VirtIODevice *vdev) +{ + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + int ret; + + if (!k->set_guest_notifiers) { + return; + } + + vhost_dev_stop(&video->vhost_dev, vdev, true); + + ret = k->set_guest_notifiers(qbus->parent, video->vhost_dev.nvqs, false); + if (ret < 0) { + error_report("vhost guest notifier cleanup failed: %d", ret); + return; + } + + vhost_dev_disable_notifiers(&video->vhost_dev, vdev); +} + +static void vhost_user_video_set_status(VirtIODevice *vdev, uint8_t status) +{ + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + bool should_start = status & VIRTIO_CONFIG_S_DRIVER_OK; + + if (!vdev->vm_running) { + should_start = false; + } + + if (video->vhost_dev.started == should_start) { + return; + } + + if (should_start) { + vhost_user_video_start(vdev); + } else { + vhost_user_video_stop(vdev); + } +} + +static uint64_t vhost_user_video_get_features(VirtIODevice *vdev, + uint64_t requested_features, + Error **errp) +{ + /* currently only support guest pages */ + virtio_add_feature(&requested_features, + VIRTIO_VIDEO_F_RESOURCE_GUEST_PAGES); + + return requested_features; +} + +static void vhost_user_video_handle_output(VirtIODevice *vdev, VirtQueue *vq) +{ + /* + * Not normally called; it's the daemon that handles the queue; + * however virtio's cleanup path can call this. + */ +} + +static void vhost_user_video_guest_notifier_mask(VirtIODevice *vdev, int idx, + bool mask) +{ + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + + /* + * Add the check for configure interrupt, Use VIRTIO_CONFIG_IRQ_IDX -1 + * as the Marco of configure interrupt's IDX, If this driver does not + * support, the function will return + */ + + if (idx == VIRTIO_CONFIG_IRQ_IDX) { + return; + } + + vhost_virtqueue_mask(&video->vhost_dev, vdev, idx, mask); +} + +static bool vhost_user_video_guest_notifier_pending(VirtIODevice *vdev, int idx) +{ + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + + /* + * Add the check for configure interrupt, Use VIRTIO_CONFIG_IRQ_IDX -1 + * as the Marco of configure interrupt's IDX, If this driver does not + * support, the function will return + */ + + if (idx == VIRTIO_CONFIG_IRQ_IDX) { + return false; + } + + return vhost_virtqueue_pending(&video->vhost_dev, idx); +} + +/* + * Chardev connect/disconnect events + */ + +static int vhost_user_video_handle_config_change(struct vhost_dev *dev) +{ + int ret; + VHostUserVIDEO *video = VHOST_USER_VIDEO(dev->vdev); + Error *local_err = NULL; + + ret = vhost_dev_get_config(dev, (uint8_t *)&video->conf.config, + sizeof(struct virtio_video_config), &local_err); + if (ret < 0) { + error_report("get config space failed"); + return -1; + } + + return 0; +} + +const VhostDevConfigOps video_ops = { + .vhost_dev_config_notifier = vhost_user_video_handle_config_change, +}; + +static int vhost_user_video_connect(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + + if (video->connected) { + return 0; + } + video->connected = true; + + /* restore vhost state */ + if (virtio_device_started(vdev, vdev->status)) { + vhost_user_video_start(vdev); + } + + return 0; +} + +static void vhost_user_video_disconnect(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + + if (!video->connected) { + return; + } + video->connected = false; + + if (video->vhost_dev.started) { + vhost_user_video_stop(vdev); + } + + vhost_dev_cleanup(&video->vhost_dev); +} + +static void vhost_user_video_event(void *opaque, QEMUChrEvent event) +{ + DeviceState *dev = opaque; + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserVIDEO *video = VHOST_USER_VIDEO(vdev); + + switch (event) { + case CHR_EVENT_OPENED: + if (vhost_user_video_connect(dev) < 0) { + qemu_chr_fe_disconnect(&video->conf.chardev); + return; + } + break; + case CHR_EVENT_CLOSED: + vhost_user_video_disconnect(dev); + break; + case CHR_EVENT_BREAK: + case CHR_EVENT_MUX_IN: + case CHR_EVENT_MUX_OUT: + /* Ignore */ + break; + } +} + +static void do_vhost_user_cleanup(VirtIODevice *vdev, VHostUserVIDEO *video) +{ + vhost_user_cleanup(&video->vhost_user); + virtio_delete_queue(video->command_vq); + virtio_delete_queue(video->event_vq); + virtio_cleanup(vdev); + g_free(video->vhost_dev.vqs); + video->vhost_dev.vqs = NULL; +} + + +static void vhost_user_video_device_realize(DeviceState *dev, Error **errp) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserVIDEO *video = VHOST_USER_VIDEO(dev); + int ret; + + if (!video->conf.chardev.chr) { + error_setg(errp, "vhost-user-video: chardev is mandatory"); + return; + } + + if (!vhost_user_init(&video->vhost_user, &video->conf.chardev, errp)) { + return; + } + + /* TODO Implement VIDEO_ENC, currently only support VIDEO_DEC */ + virtio_init(vdev, VIRTIO_ID_VIDEO_DECODER, sizeof(struct virtio_video_config)); + + /* one command queue and one event queue */ + video->vhost_dev.nvqs = 2; + video->vhost_dev.vqs = g_new0(struct vhost_virtqueue, + video->vhost_dev.nvqs); + video->vhost_dev.vq_index = 0; + + vhost_dev_set_config_notifier(&video->vhost_dev, &video_ops); + video->vhost_user.supports_config = true; + + ret = vhost_dev_init(&video->vhost_dev, &video->vhost_user, + VHOST_BACKEND_TYPE_USER, 0, errp); + if (ret < 0) { + error_report("vhost_dev_init failed: %s", strerror(-ret)); + goto vhost_dev_init_failed; + } + + /* One command queue, for sending commands */ + video->command_vq = virtio_add_queue(vdev, 128, + vhost_user_video_handle_output); + + if (!video->command_vq) { + error_setg_errno(errp, -1, "virtio_add_queue() failed"); + goto cmd_q_fail; + } + + /* One event queue, for sending events */ + video->event_vq = virtio_add_queue(vdev, 128, + vhost_user_video_handle_output); + + if (!video->command_vq) { + error_setg_errno(errp, -1, "virtio_add_queue() failed"); + goto event_q_fail; + } + + /* + * At this point the next event we will get is a connection from + * the daemon on the control socket. + */ + + qemu_chr_fe_set_handlers(&video->conf.chardev, NULL, + NULL, vhost_user_video_event, + NULL, (void *)dev, NULL, true); + + return; + +event_q_fail: + virtio_delete_queue(video->event_vq); +cmd_q_fail: + vhost_user_cleanup(&video->vhost_user); +vhost_dev_init_failed: + virtio_cleanup(vdev); + +} + +static void vhost_user_video_device_unrealize(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserVIDEO *video = VHOST_USER_VIDEO(dev); + + /* This will stop vhost backend if appropriate. */ + vhost_user_video_set_status(vdev, 0); + + do_vhost_user_cleanup(vdev, video); +} + +static const VMStateDescription vhost_user_video_vmstate = { + .name = "vhost-user-video", + .unmigratable = 1, +}; + +static Property vhost_user_video_properties[] = { + DEFINE_PROP_CHR("chardev", VHostUserVIDEO, conf.chardev), + DEFINE_PROP_END_OF_LIST(), +}; + +static void vhost_user_video_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + VirtioDeviceClass *vdc = VIRTIO_DEVICE_CLASS(klass); + + device_class_set_props(dc, vhost_user_video_properties); + dc->vmsd = &vhost_user_video_vmstate; + set_bit(DEVICE_CATEGORY_MISC, dc->categories); + vdc->realize = vhost_user_video_device_realize; + vdc->unrealize = vhost_user_video_device_unrealize; + vdc->get_features = vhost_user_video_get_features; + vdc->get_config = vhost_user_video_get_config; + vdc->set_status = vhost_user_video_set_status; + vdc->guest_notifier_mask = vhost_user_video_guest_notifier_mask; + vdc->guest_notifier_pending = vhost_user_video_guest_notifier_pending; +} + +static const TypeInfo vhost_user_video_info = { + .name = TYPE_VHOST_USER_VIDEO, + .parent = TYPE_VIRTIO_DEVICE, + .instance_size = sizeof(VHostUserVIDEO), + .class_init = vhost_user_video_class_init, +}; + +static void vhost_user_video_register_types(void) +{ + type_register_static(&vhost_user_video_info); +} + +type_init(vhost_user_video_register_types) diff --git a/include/hw/virtio/vhost-user-video.h b/include/hw/virtio/vhost-user-video.h new file mode 100644 index 0000000000..f8329c3b36 --- /dev/null +++ b/include/hw/virtio/vhost-user-video.h @@ -0,0 +1,42 @@ +/* + * vhost-user-video virtio device + * + * Copyright (c) 2021 Linaro Ltd + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef _VHOST_USER_VIDEO_H_ +#define _VHOST_USER_VIDEO_H_ + +#include "standard-headers/linux/virtio_ids.h" +#include "standard-headers/linux/virtio_video.h" +#include "hw/virtio/virtio.h" +#include "hw/virtio/vhost.h" +#include "hw/virtio/vhost-user.h" +#include "chardev/char-fe.h" + +#define TYPE_VHOST_USER_VIDEO "vhost-user-video-device" +#define VHOST_USER_VIDEO(obj) \ + OBJECT_CHECK(VHostUserVIDEO, (obj), TYPE_VHOST_USER_VIDEO) + +typedef struct { + CharBackend chardev; + struct virtio_video_config config; +} VHostUserVIDEOConf; + +typedef struct { + /*< private >*/ + VirtIODevice parent; + VHostUserVIDEOConf conf; + struct vhost_virtqueue *vhost_vq; + struct vhost_dev vhost_dev; + VhostUserState vhost_user; + VirtQueue *command_vq; + VirtQueue *event_vq; + bool connected; + /*< public >*/ +} VHostUserVIDEO; + + +#endif /* _VHOST_USER_VIDEO_H_ */ From patchwork Wed Mar 22 14:21:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Albert Esteve X-Patchwork-Id: 13184140 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EF27EC761A6 for ; Wed, 22 Mar 2023 14:22:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pezLl-000817-Fc; Wed, 22 Mar 2023 10:22:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLf-0007yb-Hi for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:21:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLe-0007J5-46 for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:21:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679494913; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sxxtbUbqF+qwyLz81zUF2WTiosA3bmrv/G8FnPLjb/E=; b=gUhEagJGilSaQZr3iorBbf7FtLhOyQ6CVa1zw16tuIL8LqIxd1XahUp9CkeSRoJjDKdgDr kd+jANkly5pGg8eQ16Pxzfg7M5gzRF3L5mUCZZTpUuur7KsuOfaovszUriq9Pf4vmWZesm NqFYjJKjfRbdzzj5mm6WxV78Yvn3iGs= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-512-8NeQSXtGMaePc8jMEBkhRQ-1; Wed, 22 Mar 2023 10:21:48 -0400 X-MC-Unique: 8NeQSXtGMaePc8jMEBkhRQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 62AE81C0759A; Wed, 22 Mar 2023 14:21:48 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.226.254]) by smtp.corp.redhat.com (Postfix) with ESMTP id D774940C6E67; Wed, 22 Mar 2023 14:21:45 +0000 (UTC) From: Albert Esteve To: qemu-devel@nongnu.org Cc: Thomas Huth , fmartine@redhat.com, Gerd Hoffmann , eballetb@redhat.com, Albert Esteve , alex.bennee@linaro.org, Laurent Vivier , "Michael S. Tsirkin" , sgarzare@redhat.com, =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , peter.griffin@linaro.org, Paolo Bonzini Subject: [PATCH 04/12] vhost-user-video: add meson subdir build logic Date: Wed, 22 Mar 2023 15:21:24 +0100 Message-Id: <20230322142132.22909-5-aesteve@redhat.com> In-Reply-To: <20230322142132.22909-1-aesteve@redhat.com> References: <20230322142132.22909-1-aesteve@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Griffin Signed-off-by: Peter Griffin Signed-off-by: Albert Esteve --- tools/meson.build | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tools/meson.build b/tools/meson.build index e69de29bb2..737f7dc140 100644 --- a/tools/meson.build +++ b/tools/meson.build @@ -0,0 +1,7 @@ +have_virtiovideo = (targetos == 'linux' and + have_tools and + have_vhost_user) + +if have_virtiovideo + subdir('vhost-user-video') +endif From patchwork Wed Mar 22 14:21:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Albert Esteve X-Patchwork-Id: 13184145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E874AC6FD1C for ; Wed, 22 Mar 2023 14:23:12 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pezLm-00081c-Qn; Wed, 22 Mar 2023 10:22:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLg-0007ys-2X for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:21:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLd-0007J0-P5 for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:21:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679494913; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O3p4U7iiHs6Fcdufu9lV6f5rSclPKtis3t4ojlW26v8=; b=PlxVvMTvTHAgEAv1FcPB3smRg05vECkCI2o3yg+vIsPKXeClxJ7J/sII1bvoR6dR73Xl7Q kmttGcavK1B3masf0eKL4AKNmwPXJF0Cr/7piLPrsl+bTxeoX6o1B0DmNv+TYC/mlu8Gr5 XI6qn1pTi1CI2rZu02GAh2EB1Z6Kbkc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-649-qW8AoOneMBuEilRqUKO_ew-1; Wed, 22 Mar 2023 10:21:51 -0400 X-MC-Unique: qW8AoOneMBuEilRqUKO_ew-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 26060185A7A3; Wed, 22 Mar 2023 14:21:51 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.226.254]) by smtp.corp.redhat.com (Postfix) with ESMTP id AC69640C6E67; Wed, 22 Mar 2023 14:21:48 +0000 (UTC) From: Albert Esteve To: qemu-devel@nongnu.org Cc: Thomas Huth , fmartine@redhat.com, Gerd Hoffmann , eballetb@redhat.com, Albert Esteve , alex.bennee@linaro.org, Laurent Vivier , "Michael S. Tsirkin" , sgarzare@redhat.com, =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , peter.griffin@linaro.org, Paolo Bonzini Subject: [PATCH 05/12] standard-headers: Add virtio_video.h Date: Wed, 22 Mar 2023 15:21:25 +0100 Message-Id: <20230322142132.22909-6-aesteve@redhat.com> In-Reply-To: <20230322142132.22909-1-aesteve@redhat.com> References: <20230322142132.22909-1-aesteve@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Griffin Signed-off-by: Peter Griffin Message-Id: <20211209145601.331477-6-peter.griffin@linaro.org> --- include/standard-headers/linux/virtio_video.h | 513 ++++++++++++++++++ 1 file changed, 513 insertions(+) create mode 100644 include/standard-headers/linux/virtio_video.h diff --git a/include/standard-headers/linux/virtio_video.h b/include/standard-headers/linux/virtio_video.h new file mode 100644 index 0000000000..6ccd7b1f38 --- /dev/null +++ b/include/standard-headers/linux/virtio_video.h @@ -0,0 +1,513 @@ +/* SPDX-License-Identifier: BSD-3-Clause */ +/* + * Virtio Video Device + * + * This header is BSD licensed so anyone can use the definitions + * to implement compatible drivers/servers: + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of IBM nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL IBM OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * Copyright (C) 2019 OpenSynergy GmbH. + */ + +#ifndef _UAPI_LINUX_VIRTIO_VIDEO_H +#define _UAPI_LINUX_VIRTIO_VIDEO_H + +#include +#include + +/* + * Feature bits + */ + +/* Guest pages can be used for video buffers. */ +#define VIRTIO_VIDEO_F_RESOURCE_GUEST_PAGES 0 +/* + * The host can process buffers even if they are non-contiguous memory such as + * scatter-gather lists. + */ +#define VIRTIO_VIDEO_F_RESOURCE_NON_CONTIG 1 +/* Objects exported by another virtio device can be used for video buffers */ +#define VIRTIO_VIDEO_F_RESOURCE_VIRTIO_OBJECT 2 + +/* + * Image formats + */ + +enum virtio_video_format { + /* Raw formats */ + VIRTIO_VIDEO_FORMAT_RAW_MIN = 1, + VIRTIO_VIDEO_FORMAT_ARGB8888 = VIRTIO_VIDEO_FORMAT_RAW_MIN, + VIRTIO_VIDEO_FORMAT_BGRA8888, + VIRTIO_VIDEO_FORMAT_NV12, /* 12 Y/CbCr 4:2:0 */ + VIRTIO_VIDEO_FORMAT_YUV420, /* 12 YUV 4:2:0 */ + VIRTIO_VIDEO_FORMAT_YVU420, /* 12 YVU 4:2:0 */ + VIRTIO_VIDEO_FORMAT_RAW_MAX = VIRTIO_VIDEO_FORMAT_YVU420, + + /* Coded formats */ + VIRTIO_VIDEO_FORMAT_CODED_MIN = 0x1000, + VIRTIO_VIDEO_FORMAT_MPEG2 = + VIRTIO_VIDEO_FORMAT_CODED_MIN, /* MPEG-2 Part 2 */ + VIRTIO_VIDEO_FORMAT_MPEG4, /* MPEG-4 Part 2 */ + VIRTIO_VIDEO_FORMAT_H264, /* H.264 */ + VIRTIO_VIDEO_FORMAT_HEVC, /* HEVC aka H.265*/ + VIRTIO_VIDEO_FORMAT_VP8, /* VP8 */ + VIRTIO_VIDEO_FORMAT_VP9, /* VP9 */ + VIRTIO_VIDEO_FORMAT_FWHT, /* FWHT used by vicodec */ + VIRTIO_VIDEO_FORMAT_CODED_MAX = VIRTIO_VIDEO_FORMAT_FWHT, +}; + +enum virtio_video_profile { + /* H.264 */ + VIRTIO_VIDEO_PROFILE_H264_MIN = 0x100, + VIRTIO_VIDEO_PROFILE_H264_BASELINE = VIRTIO_VIDEO_PROFILE_H264_MIN, + VIRTIO_VIDEO_PROFILE_H264_MAIN, + VIRTIO_VIDEO_PROFILE_H264_EXTENDED, + VIRTIO_VIDEO_PROFILE_H264_HIGH, + VIRTIO_VIDEO_PROFILE_H264_HIGH10PROFILE, + VIRTIO_VIDEO_PROFILE_H264_HIGH422PROFILE, + VIRTIO_VIDEO_PROFILE_H264_HIGH444PREDICTIVEPROFILE, + VIRTIO_VIDEO_PROFILE_H264_SCALABLEBASELINE, + VIRTIO_VIDEO_PROFILE_H264_SCALABLEHIGH, + VIRTIO_VIDEO_PROFILE_H264_STEREOHIGH, + VIRTIO_VIDEO_PROFILE_H264_MULTIVIEWHIGH, + VIRTIO_VIDEO_PROFILE_H264_MAX = VIRTIO_VIDEO_PROFILE_H264_MULTIVIEWHIGH, + + /* HEVC */ + VIRTIO_VIDEO_PROFILE_HEVC_MIN = 0x200, + VIRTIO_VIDEO_PROFILE_HEVC_MAIN = VIRTIO_VIDEO_PROFILE_HEVC_MIN, + VIRTIO_VIDEO_PROFILE_HEVC_MAIN10, + VIRTIO_VIDEO_PROFILE_HEVC_MAIN_STILL_PICTURE, + VIRTIO_VIDEO_PROFILE_HEVC_MAX = + VIRTIO_VIDEO_PROFILE_HEVC_MAIN_STILL_PICTURE, + + /* VP8 */ + VIRTIO_VIDEO_PROFILE_VP8_MIN = 0x300, + VIRTIO_VIDEO_PROFILE_VP8_PROFILE0 = VIRTIO_VIDEO_PROFILE_VP8_MIN, + VIRTIO_VIDEO_PROFILE_VP8_PROFILE1, + VIRTIO_VIDEO_PROFILE_VP8_PROFILE2, + VIRTIO_VIDEO_PROFILE_VP8_PROFILE3, + VIRTIO_VIDEO_PROFILE_VP8_MAX = VIRTIO_VIDEO_PROFILE_VP8_PROFILE3, + + /* VP9 */ + VIRTIO_VIDEO_PROFILE_VP9_MIN = 0x400, + VIRTIO_VIDEO_PROFILE_VP9_PROFILE0 = VIRTIO_VIDEO_PROFILE_VP9_MIN, + VIRTIO_VIDEO_PROFILE_VP9_PROFILE1, + VIRTIO_VIDEO_PROFILE_VP9_PROFILE2, + VIRTIO_VIDEO_PROFILE_VP9_PROFILE3, + VIRTIO_VIDEO_PROFILE_VP9_MAX = VIRTIO_VIDEO_PROFILE_VP9_PROFILE3, +}; + +enum virtio_video_level { + /* H.264 */ + VIRTIO_VIDEO_LEVEL_H264_MIN = 0x100, + VIRTIO_VIDEO_LEVEL_H264_1_0 = VIRTIO_VIDEO_LEVEL_H264_MIN, + VIRTIO_VIDEO_LEVEL_H264_1_1, + VIRTIO_VIDEO_LEVEL_H264_1_2, + VIRTIO_VIDEO_LEVEL_H264_1_3, + VIRTIO_VIDEO_LEVEL_H264_2_0, + VIRTIO_VIDEO_LEVEL_H264_2_1, + VIRTIO_VIDEO_LEVEL_H264_2_2, + VIRTIO_VIDEO_LEVEL_H264_3_0, + VIRTIO_VIDEO_LEVEL_H264_3_1, + VIRTIO_VIDEO_LEVEL_H264_3_2, + VIRTIO_VIDEO_LEVEL_H264_4_0, + VIRTIO_VIDEO_LEVEL_H264_4_1, + VIRTIO_VIDEO_LEVEL_H264_4_2, + VIRTIO_VIDEO_LEVEL_H264_5_0, + VIRTIO_VIDEO_LEVEL_H264_5_1, + VIRTIO_VIDEO_LEVEL_H264_MAX = VIRTIO_VIDEO_LEVEL_H264_5_1, +}; + +enum virtio_video_bitrate_mode { + VIRTIO_VIDEO_BITRATE_MODE_VBR = 0, + VIRTIO_VIDEO_BITRATE_MODE_CBR = 1, +}; + +/* + * Config + */ + +struct virtio_video_config { + __le32 version; + __le32 max_caps_length; + __le32 max_resp_length; + __u8 device_name[32]; +}; + +/* + * Commands + */ + +enum virtio_video_cmd_type { + /* Command */ + VIRTIO_VIDEO_CMD_QUERY_CAPABILITY = 0x0100, + VIRTIO_VIDEO_CMD_STREAM_CREATE, + VIRTIO_VIDEO_CMD_STREAM_DESTROY, + VIRTIO_VIDEO_CMD_STREAM_DRAIN, + VIRTIO_VIDEO_CMD_RESOURCE_CREATE, + VIRTIO_VIDEO_CMD_RESOURCE_QUEUE, + VIRTIO_VIDEO_CMD_RESOURCE_DESTROY_ALL, + VIRTIO_VIDEO_CMD_QUEUE_CLEAR, + /* GET/SET_PARAMS are being replaced with GET/SET_PARAMS_EXT */ + VIRTIO_VIDEO_CMD_GET_PARAMS__UNUSED, + VIRTIO_VIDEO_CMD_SET_PARAMS__UNUSED, + VIRTIO_VIDEO_CMD_QUERY_CONTROL, + VIRTIO_VIDEO_CMD_GET_CONTROL, + VIRTIO_VIDEO_CMD_SET_CONTROL, + VIRTIO_VIDEO_CMD_GET_PARAMS_EXT, + VIRTIO_VIDEO_CMD_SET_PARAMS_EXT, + + /* Response */ + VIRTIO_VIDEO_RESP_OK_NODATA = 0x0200, + VIRTIO_VIDEO_RESP_OK_QUERY_CAPABILITY, + VIRTIO_VIDEO_RESP_OK_RESOURCE_QUEUE, + VIRTIO_VIDEO_RESP_OK_GET_PARAMS, + VIRTIO_VIDEO_RESP_OK_QUERY_CONTROL, + VIRTIO_VIDEO_RESP_OK_GET_CONTROL, + + VIRTIO_VIDEO_RESP_ERR_INVALID_OPERATION = 0x0300, + VIRTIO_VIDEO_RESP_ERR_OUT_OF_MEMORY, + VIRTIO_VIDEO_RESP_ERR_INVALID_STREAM_ID, + VIRTIO_VIDEO_RESP_ERR_INVALID_RESOURCE_ID, + VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER, + VIRTIO_VIDEO_RESP_ERR_UNSUPPORTED_CONTROL, +}; + +struct virtio_video_cmd_hdr { + __le32 type; /* One of enum virtio_video_cmd_type */ + __le32 stream_id; +}; + +/* VIRTIO_VIDEO_CMD_QUERY_CAPABILITY */ +enum virtio_video_queue_type { + VIRTIO_VIDEO_QUEUE_TYPE_INPUT = 0x100, + VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT, +}; + +struct virtio_video_query_capability { + struct virtio_video_cmd_hdr hdr; + __le32 queue_type; /* One of VIRTIO_VIDEO_QUEUE_TYPE_* types */ + __u8 padding[4]; +}; + +enum virtio_video_planes_layout_flag { + VIRTIO_VIDEO_PLANES_LAYOUT_SINGLE_BUFFER = 1 << 0, + VIRTIO_VIDEO_PLANES_LAYOUT_PER_PLANE = 1 << 1, +}; + +struct virtio_video_format_range { + __le32 min; + __le32 max; + __le32 step; + __u8 padding[4]; +}; + +struct virtio_video_format_frame { + struct virtio_video_format_range width; + struct virtio_video_format_range height; + __le32 num_rates; + __u8 padding[4]; + /* Followed by struct virtio_video_format_range frame_rates[] */ +}; + +struct virtio_video_format_desc { + __le64 mask; + __le32 format; /* One of VIRTIO_VIDEO_FORMAT_* types */ + __le32 planes_layout; /* Bitmask with VIRTIO_VIDEO_PLANES_LAYOUT_* */ + __le32 plane_align; + __le32 num_frames; + /* Followed by struct virtio_video_format_frame frames[] */ +}; + +struct virtio_video_query_capability_resp { + struct virtio_video_cmd_hdr hdr; + __le32 num_descs; + __u8 padding[4]; + /* Followed by struct virtio_video_format_desc descs[] */ +}; + +/* VIRTIO_VIDEO_CMD_STREAM_CREATE */ +enum virtio_video_mem_type { + VIRTIO_VIDEO_MEM_TYPE_GUEST_PAGES, + VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT, +}; + +struct virtio_video_stream_create { + struct virtio_video_cmd_hdr hdr; + __le32 in_mem_type; /* One of VIRTIO_VIDEO_MEM_TYPE_* types */ + __le32 out_mem_type; /* One of VIRTIO_VIDEO_MEM_TYPE_* types */ + __le32 coded_format; /* One of VIRTIO_VIDEO_FORMAT_* types */ + __u8 padding[4]; + __u8 tag[64]; +}; + +/* VIRTIO_VIDEO_CMD_STREAM_DESTROY */ +struct virtio_video_stream_destroy { + struct virtio_video_cmd_hdr hdr; +}; + +/* VIRTIO_VIDEO_CMD_STREAM_DRAIN */ +struct virtio_video_stream_drain { + struct virtio_video_cmd_hdr hdr; +}; + +/* VIRTIO_VIDEO_CMD_RESOURCE_CREATE */ +struct virtio_video_mem_entry { + __le64 addr; + __le32 length; + __u8 padding[4]; +}; + +struct virtio_video_object_entry { + __u8 uuid[16]; +}; + +#define VIRTIO_VIDEO_MAX_PLANES 8 + +struct virtio_video_resource_create { + struct virtio_video_cmd_hdr hdr; + __le32 queue_type; /* One of VIRTIO_VIDEO_QUEUE_TYPE_* types */ + __le32 resource_id; + __le32 planes_layout; + __le32 num_planes; + __le32 plane_offsets[VIRTIO_VIDEO_MAX_PLANES]; + __le32 num_entries[VIRTIO_VIDEO_MAX_PLANES]; + /** + * Followed by either + * - struct virtio_video_mem_entry entries[] + * for VIRTIO_VIDEO_MEM_TYPE_GUEST_PAGES + * - struct virtio_video_object_entry entries[] + * for VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT + */ +}; + +/* VIRTIO_VIDEO_CMD_RESOURCE_QUEUE */ +struct virtio_video_resource_queue { + struct virtio_video_cmd_hdr hdr; + __le32 queue_type; /* One of VIRTIO_VIDEO_QUEUE_TYPE_* types */ + __le32 resource_id; + __le64 timestamp; + __le32 num_data_sizes; + __le32 data_sizes[VIRTIO_VIDEO_MAX_PLANES]; + __u8 padding[4]; +}; + +enum virtio_video_buffer_flag { + VIRTIO_VIDEO_BUFFER_FLAG_ERR = 0x0001, + VIRTIO_VIDEO_BUFFER_FLAG_EOS = 0x0002, + + /* Encoder only */ + VIRTIO_VIDEO_BUFFER_FLAG_IFRAME = 0x0004, + VIRTIO_VIDEO_BUFFER_FLAG_PFRAME = 0x0008, + VIRTIO_VIDEO_BUFFER_FLAG_BFRAME = 0x0010, +}; + +struct virtio_video_resource_queue_resp { + struct virtio_video_cmd_hdr hdr; + __le64 timestamp; + __le32 flags; /* One of VIRTIO_VIDEO_BUFFER_FLAG_* flags */ + __le32 size; /* Encoded size */ +}; + +/* VIRTIO_VIDEO_CMD_RESOURCE_DESTROY_ALL */ +struct virtio_video_resource_destroy_all { + struct virtio_video_cmd_hdr hdr; + __le32 queue_type; /* One of VIRTIO_VIDEO_QUEUE_TYPE_* types */ + __u8 padding[4]; +}; + +/* VIRTIO_VIDEO_CMD_QUEUE_CLEAR */ +struct virtio_video_queue_clear { + struct virtio_video_cmd_hdr hdr; + __le32 queue_type; /* One of VIRTIO_VIDEO_QUEUE_TYPE_* types */ + __u8 padding[4]; +}; + +/* VIRTIO_VIDEO_CMD_GET_PARAMS */ +struct virtio_video_plane_format { + __le32 plane_size; + __le32 stride; +}; + +struct virtio_video_crop { + __le32 left; + __le32 top; + __le32 width; + __le32 height; +}; + +struct virtio_video_params { + __le32 queue_type; /* One of VIRTIO_VIDEO_QUEUE_TYPE_* types */ + __le32 format; /* One of VIRTIO_VIDEO_FORMAT_* types */ + __le32 frame_width; + __le32 frame_height; + __le32 min_buffers; + __le32 max_buffers; + struct virtio_video_crop crop; + __le32 frame_rate; + __le32 num_planes; + struct virtio_video_plane_format plane_formats[VIRTIO_VIDEO_MAX_PLANES]; + __le32 resource_type; /* One of VIRTIO_VIDEO_MEM_TYPE_* types */ + __u8 padding[4]; +}; + +struct virtio_video_get_params { + struct virtio_video_cmd_hdr hdr; + __le32 queue_type; /* One of VIRTIO_VIDEO_QUEUE_TYPE_* types */ + __u8 padding[4]; +}; + +struct virtio_video_get_params_resp { + struct virtio_video_cmd_hdr hdr; + struct virtio_video_params params; +}; + +/* VIRTIO_VIDEO_CMD_SET_PARAMS */ +struct virtio_video_set_params { + struct virtio_video_cmd_hdr hdr; + struct virtio_video_params params; +}; + +/* VIRTIO_VIDEO_CMD_QUERY_CONTROL */ +enum virtio_video_control_type { + VIRTIO_VIDEO_CONTROL_BITRATE = 1, + VIRTIO_VIDEO_CONTROL_PROFILE, + VIRTIO_VIDEO_CONTROL_LEVEL, + VIRTIO_VIDEO_CONTROL_FORCE_KEYFRAME, + VIRTIO_VIDEO_CONTROL_BITRATE_MODE, + VIRTIO_VIDEO_CONTROL_BITRATE_PEAK, + VIRTIO_VIDEO_CONTROL_PREPEND_SPSPPS_TO_IDR, +}; + +struct virtio_video_query_control_profile { + __le32 format; /* One of VIRTIO_VIDEO_FORMAT_* */ + __u8 padding[4]; +}; + +struct virtio_video_query_control_level { + __le32 format; /* One of VIRTIO_VIDEO_FORMAT_* */ + __u8 padding[4]; +}; + +struct virtio_video_query_control { + struct virtio_video_cmd_hdr hdr; + __le32 control; /* One of VIRTIO_VIDEO_CONTROL_* types */ + __u8 padding[4]; + /* + * Followed by a value of struct virtio_video_query_control_* + * in accordance with the value of control. + */ +}; + +struct virtio_video_query_control_resp_profile { + __le32 num; + __u8 padding[4]; + /* Followed by an array le32 profiles[] */ +}; + +struct virtio_video_query_control_resp_level { + __le32 num; + __u8 padding[4]; + /* Followed by an array le32 level[] */ +}; + +struct virtio_video_query_control_resp { + struct virtio_video_cmd_hdr hdr; + /* Followed by one of struct virtio_video_query_control_resp_* */ +}; + +/* VIRTIO_VIDEO_CMD_GET_CONTROL */ +struct virtio_video_get_control { + struct virtio_video_cmd_hdr hdr; + __le32 control; /* One of VIRTIO_VIDEO_CONTROL_* types */ + __u8 padding[4]; +}; + +struct virtio_video_control_val_bitrate { + __le32 bitrate; + __u8 padding[4]; +}; + +struct virtio_video_control_val_bitrate_peak { + __le32 bitrate_peak; + __u8 padding[4]; +}; + +struct virtio_video_control_val_bitrate_mode { + __le32 bitrate_mode; + __u8 padding[4]; +}; + +struct virtio_video_control_val_profile { + __le32 profile; + __u8 padding[4]; +}; + +struct virtio_video_control_val_level { + __le32 level; + __u8 padding[4]; +}; + +struct virtio_video_control_val_prepend_spspps_to_idr { + __le32 prepend_spspps_to_idr; + __u8 padding[4]; +}; + +struct virtio_video_get_control_resp { + struct virtio_video_cmd_hdr hdr; + /* Followed by one of struct virtio_video_control_val_* */ +}; + +/* VIRTIO_VIDEO_CMD_SET_CONTROL */ +struct virtio_video_set_control { + struct virtio_video_cmd_hdr hdr; + __le32 control; /* One of VIRTIO_VIDEO_CONTROL_* types */ + __u8 padding[4]; + /* Followed by one of struct virtio_video_control_val_* */ +}; + +struct virtio_video_set_control_resp { + struct virtio_video_cmd_hdr hdr; +}; + +/* + * Events + */ + +enum virtio_video_event_type { + /* For all devices */ + VIRTIO_VIDEO_EVENT_ERROR = 0x0100, + + /* For decoder only */ + VIRTIO_VIDEO_EVENT_DECODER_RESOLUTION_CHANGED = 0x0200, +}; + +struct virtio_video_event { + __le32 event_type; /* One of VIRTIO_VIDEO_EVENT_* types */ + __le32 stream_id; +}; + +#endif /* _UAPI_LINUX_VIRTIO_VIDEO_H */ From patchwork Wed Mar 22 14:21:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Albert Esteve X-Patchwork-Id: 13184157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 03C4FC6FD1F for ; Wed, 22 Mar 2023 14:23:50 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pezLp-00084K-Lr; Wed, 22 Mar 2023 10:22:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLi-000819-Jv for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:21:59 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLg-0007Jp-1V for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:21:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679494915; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lT1AatYzR6Bv0y6n+UEd84VWzqmG0+0xg1T4GGxISYs=; b=Ghx9Brt6zprjp6eD8u2aIQzw3DSpvF4S449ez2qGp0F4LeV3P1y1nlYCjnsZfCZrShieaQ 0T8NRvvZXfPD6jyuKXNTZ0/xJQPiULGsXpMWtziEc+C0BF8Vfb9tp5n855BqdUwUBjwDMg umWXs3IJXvNMJ4ix+I9QiPLqiAqp8NQ= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-253-GKB3iiILOMye_EludEGPRA-1; Wed, 22 Mar 2023 10:21:54 -0400 X-MC-Unique: GKB3iiILOMye_EludEGPRA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id D85A038173CB; Wed, 22 Mar 2023 14:21:53 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.226.254]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7DAA540C6E67; Wed, 22 Mar 2023 14:21:51 +0000 (UTC) From: Albert Esteve To: qemu-devel@nongnu.org Cc: Thomas Huth , fmartine@redhat.com, Gerd Hoffmann , eballetb@redhat.com, Albert Esteve , alex.bennee@linaro.org, Laurent Vivier , "Michael S. Tsirkin" , sgarzare@redhat.com, =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , peter.griffin@linaro.org, Paolo Bonzini Subject: [PATCH 06/12] hw/display: add vhost-user-video-pci Date: Wed, 22 Mar 2023 15:21:26 +0100 Message-Id: <20230322142132.22909-7-aesteve@redhat.com> In-Reply-To: <20230322142132.22909-1-aesteve@redhat.com> References: <20230322142132.22909-1-aesteve@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Griffin Add boiler plate code for vhost-user-video-pci. Example -device vhost-user-video-pci,chardev=video,id=video -chardev socket,path=video.sock,id=video Signed-off-by: Peter Griffin Message-Id: <20211209145601.331477-8-peter.griffin@linaro.org> --- hw/display/vhost-user-video-pci.c | 82 +++++++++++++++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 hw/display/vhost-user-video-pci.c diff --git a/hw/display/vhost-user-video-pci.c b/hw/display/vhost-user-video-pci.c new file mode 100644 index 0000000000..ceeaad2742 --- /dev/null +++ b/hw/display/vhost-user-video-pci.c @@ -0,0 +1,82 @@ +/* + * Vhost-user VIDEO virtio device PCI glue + * + * Copyright (c) 2021 Linaro Ltd + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "hw/qdev-properties.h" +#include "hw/virtio/vhost-user-video.h" +#include "hw/virtio/virtio-pci.h" + +struct VHostUserVIDEOPCI { + VirtIOPCIProxy parent_obj; + VHostUserVIDEO vdev; +}; + +typedef struct VHostUserVIDEOPCI VHostUserVIDEOPCI; + +#define TYPE_VHOST_USER_VIDEO_PCI "vhost-user-video-pci-base" + +#define VHOST_USER_VIDEO_PCI(obj) \ + OBJECT_CHECK(VHostUserVIDEOPCI, (obj), TYPE_VHOST_USER_VIDEO_PCI) + +static Property vuvideo_pci_properties[] = { + DEFINE_PROP_BIT("ioeventfd", VirtIOPCIProxy, flags, + VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT, true), + DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors, + DEV_NVECTORS_UNSPECIFIED), + DEFINE_PROP_END_OF_LIST(), +}; + +static void vuvideo_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) +{ + VHostUserVIDEOPCI *dev = VHOST_USER_VIDEO_PCI(vpci_dev); + DeviceState *vdev = DEVICE(&dev->vdev); + + if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) { + vpci_dev->nvectors = 1; + } + + qdev_set_parent_bus(vdev, BUS(&vpci_dev->bus), errp); + object_property_set_bool(OBJECT(vdev), "realized", true, errp); +} + +static void vuvideo_pci_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + VirtioPCIClass *k = VIRTIO_PCI_CLASS(klass); + PCIDeviceClass *pcidev_k = PCI_DEVICE_CLASS(klass); + k->realize = vuvideo_pci_realize; + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); + device_class_set_props(dc, vuvideo_pci_properties); + pcidev_k->vendor_id = PCI_VENDOR_ID_REDHAT_QUMRANET; + pcidev_k->device_id = 0; /* Set by virtio-pci based on virtio id */ + pcidev_k->revision = 0x00; + pcidev_k->class_id = PCI_CLASS_STORAGE_OTHER; +} + +static void vuvideo_pci_instance_init(Object *obj) +{ + VHostUserVIDEOPCI *dev = VHOST_USER_VIDEO_PCI(obj); + + virtio_instance_init_common(obj, &dev->vdev, sizeof(dev->vdev), + TYPE_VHOST_USER_VIDEO); +} + +static const VirtioPCIDeviceTypeInfo vuvideo_pci_info = { + .base_name = TYPE_VHOST_USER_VIDEO_PCI, + .non_transitional_name = "vhost-user-video-pci", + .instance_size = sizeof(VHostUserVIDEOPCI), + .instance_init = vuvideo_pci_instance_init, + .class_init = vuvideo_pci_class_init, +}; + +static void vuvideo_pci_register(void) +{ + virtio_pci_types_register(&vuvideo_pci_info); +} + +type_init(vuvideo_pci_register); From patchwork Wed Mar 22 14:21:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Albert Esteve X-Patchwork-Id: 13184139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EF234C6FD1F for ; Wed, 22 Mar 2023 14:22:27 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pezLr-00085S-S3; Wed, 22 Mar 2023 10:22:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLl-00081m-FY for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:22:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLj-0007Lc-Lw for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:22:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679494918; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uL4htXBTQgz6mFt6b4KNN1to06ZNfjPs5qEVJg5uMPk=; b=IJlZeEULiQx3zPVso8g9yvn73qeAx2XyrddOLCUZhm/ndBpuil2u+iyQZyTQS5e6NzH8dr XfkOSKjxV2KVdoI+ml+6z1LgxvvCwMMYw/Ut6ZfKTU5H9NSqSk48ZUH2ve7IxUSj9cunYX vBaNfdL0Xcxpyruerz9fGXfXXYQoZw0= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-654-_Y1_bh-YMF6mIBDqfJxeKA-1; Wed, 22 Mar 2023 10:21:57 -0400 X-MC-Unique: _Y1_bh-YMF6mIBDqfJxeKA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B062A1C0758E; Wed, 22 Mar 2023 14:21:56 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.226.254]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3E8F940C6E67; Wed, 22 Mar 2023 14:21:54 +0000 (UTC) From: Albert Esteve To: qemu-devel@nongnu.org Cc: Thomas Huth , fmartine@redhat.com, Gerd Hoffmann , eballetb@redhat.com, Albert Esteve , alex.bennee@linaro.org, Laurent Vivier , "Michael S. Tsirkin" , sgarzare@redhat.com, =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , peter.griffin@linaro.org, Paolo Bonzini Subject: [PATCH 07/12] vhost-user.json: add video type Date: Wed, 22 Mar 2023 15:21:27 +0100 Message-Id: <20230322142132.22909-8-aesteve@redhat.com> In-Reply-To: <20230322142132.22909-1-aesteve@redhat.com> References: <20230322142132.22909-1-aesteve@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Albert Esteve --- docs/interop/vhost-user.json | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/interop/vhost-user.json b/docs/interop/vhost-user.json index b6ade9e493..8c6c1ba400 100644 --- a/docs/interop/vhost-user.json +++ b/docs/interop/vhost-user.json @@ -31,6 +31,7 @@ # @rpmsg: virtio remote processor messaging # @rproc-serial: virtio remoteproc serial link # @scsi: virtio scsi +# @video: virtio video # @vsock: virtio vsock transport # @fs: virtio fs (since 4.2) # @@ -52,6 +53,7 @@ 'rpmsg', 'rproc-serial', 'scsi', + 'video', 'vsock', 'fs' ] From patchwork Wed Mar 22 14:21:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Albert Esteve X-Patchwork-Id: 13184159 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84FE5C6FD1F for ; Wed, 22 Mar 2023 14:23:53 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pezLx-00087A-AQ; Wed, 22 Mar 2023 10:22:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLv-00086R-Km for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:22:11 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLq-0007NR-T2 for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:22:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679494925; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qD889QUAsGcEF3saeAHr+nacg94gQTDW79B/4XLJ0gA=; b=H+yITwJP8WloMH+Kuw34aqR5HUolRzN8ufB/bmF9DTMxcaCi7DilkFAod+iV8lMtMcLYyG 01ecm4id16n1332LNh5FSzrNolRGjF+VhSdAIG0MJ18bQxo7lMRnyhhSWMLvWoJxUfzCY5 XOj3qqJp5XFXDUcZL0WQMVSFpkaIh9g= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-194-I4h8rjqkMLSJTT4iK92Ftg-1; Wed, 22 Mar 2023 10:22:00 -0400 X-MC-Unique: I4h8rjqkMLSJTT4iK92Ftg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2E55080280D; Wed, 22 Mar 2023 14:22:00 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.226.254]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1D7DF40C6E67; Wed, 22 Mar 2023 14:21:56 +0000 (UTC) From: Albert Esteve To: qemu-devel@nongnu.org Cc: Thomas Huth , fmartine@redhat.com, Gerd Hoffmann , eballetb@redhat.com, Albert Esteve , alex.bennee@linaro.org, Laurent Vivier , "Michael S. Tsirkin" , sgarzare@redhat.com, =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , peter.griffin@linaro.org, Paolo Bonzini Subject: [PATCH 08/12] tools/vhost-user-video: Add initial vhost-user-video vmm Date: Wed, 22 Mar 2023 15:21:28 +0100 Message-Id: <20230322142132.22909-9-aesteve@redhat.com> In-Reply-To: <20230322142132.22909-1-aesteve@redhat.com> References: <20230322142132.22909-1-aesteve@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Griffin This vmm translates from virtio-video v3 protocol and writes to a v4l2 mem2mem stateful decoder/encoder device [1]. v3 was chosen as that is what the virtio-video Linux frontend driver implements. This allows for testing with the v4l2 vicodec test codec [2] module in the Linux kernel, and is intended to also be used with Arm SoCs that implement a v4l2 stateful decoder/encoder drivers. The advantage of developing & testing with vicodec is that is allows quick development on a purely virtual setup with qemu and a host Linux kernel. Also it allows ci systems like lkft, kernelci to easily test the virtio interface. Currently conversion from virtio-video to v4l2 stateless m2m codec driver or VAAPI drivers is consiered out ot scope as is emulation of a decoder device using a something like ffmpeg. Although this could be added in the future. Note some virtio & v4l2 helpers were based off virtio-video Linux frontend driver and yavta utility, both GPL v2. Example host commands sudo modprobe vicodec multiplanar=1 vhost-user-video --v4l2-device=/dev/video3 -v --socket-path=video.sock Run Qemu with -device vhost-user-video-pci,chardev=video,id=video Guest decoder v4l2-ctl -d0 -x width=640,height=480 -v width=640,height=480,pixelformat=YU12 \ --stream-mmap --stream-out-mmap --stream-from jelly_640_480-420P.fwht \ --stream-to out-jelly-640-480.YU12 [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/dev-decoder.html [2] https://lwn.net/Articles/760650/ Signed-off-by: Peter Griffin Signed-off-by: Albert Esteve --- tools/vhost-user-video/50-qemu-video.json.in | 5 + tools/vhost-user-video/meson.build | 10 + tools/vhost-user-video/v4l2_backend.c | 1752 ++++++++++++++++ tools/vhost-user-video/v4l2_backend.h | 104 + tools/vhost-user-video/vhost-user-video.c | 1779 +++++++++++++++++ tools/vhost-user-video/virtio_video_helpers.c | 466 +++++ tools/vhost-user-video/virtio_video_helpers.h | 168 ++ tools/vhost-user-video/vuvideo.h | 43 + 8 files changed, 4327 insertions(+) create mode 100644 tools/vhost-user-video/50-qemu-video.json.in create mode 100644 tools/vhost-user-video/meson.build create mode 100644 tools/vhost-user-video/v4l2_backend.c create mode 100644 tools/vhost-user-video/v4l2_backend.h create mode 100644 tools/vhost-user-video/vhost-user-video.c create mode 100644 tools/vhost-user-video/virtio_video_helpers.c create mode 100644 tools/vhost-user-video/virtio_video_helpers.h create mode 100644 tools/vhost-user-video/vuvideo.h diff --git a/tools/vhost-user-video/50-qemu-video.json.in b/tools/vhost-user-video/50-qemu-video.json.in new file mode 100644 index 0000000000..c333c77812 --- /dev/null +++ b/tools/vhost-user-video/50-qemu-video.json.in @@ -0,0 +1,5 @@ +{ + "description": "QEMU vhost-user-video", + "type": "video", + "binary": "@libexecdir@/vhost-user-video" +} diff --git a/tools/vhost-user-video/meson.build b/tools/vhost-user-video/meson.build new file mode 100644 index 0000000000..bf3b958dc6 --- /dev/null +++ b/tools/vhost-user-video/meson.build @@ -0,0 +1,10 @@ +executable('vhost-user-video', files( + 'vhost-user-video.c', 'v4l2_backend.c', 'virtio_video_helpers.c'), + dependencies: [qemuutil, glib, gio, vhost_user], + install: true, + install_dir: get_option('libexecdir')) + +configure_file(input: '50-qemu-video.json.in', + output: '50-qemu-video.json', + configuration: config_host, + install_dir: qemu_datadir / 'vhost-user') diff --git a/tools/vhost-user-video/v4l2_backend.c b/tools/vhost-user-video/v4l2_backend.c new file mode 100644 index 0000000000..2f5825a733 --- /dev/null +++ b/tools/vhost-user-video/v4l2_backend.c @@ -0,0 +1,1752 @@ +/* + * virtio-video video v4l2 backend + * + * The purpose of this backend is to interface with + * v4l2 stateful encoder and decoder devices in the kernel. + * + * v4l2 stateless devices are NOT supported currently. + * + * Some v4l2 helper functions taken from yatva + * + * Copyright (c) 2023 Red Hat, Inc. + * Copyright (c) 2021 Linaro Ltd + * Copyright (C) 2005-2010 Laurent Pinchart + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" + +#include + +#include +#include + +#include +#include +#include +#include +#include +#include + +#include +#include "virtio_video_helpers.h" +#include "v4l2_backend.h" +#include "standard-headers/linux/virtio_video.h" +#include "vuvideo.h" + +#define V4L2_TYPE_IS_META(type) \ + ((type) == V4L2_BUF_TYPE_META_CAPTURE \ + || (type) == V4L2_BUF_TYPE_META_OUTPUT) + +/* Function prototypes */ +static const struct v4l2_format_info * +v4l2_format_by_fourcc(unsigned int fourcc); +static const char *v4l2_format_name(unsigned int fourcc); +static const char *v4l2_buf_type_name(enum v4l2_buf_type type); +static const char *v4l2_field_name(enum v4l2_field field); + +static int video_enum_frame_intervals(struct v4l2_device *dev, + __u32 pixelformat, + unsigned int width, unsigned int height, + GList **p_vid_fmt_frm_rate_l); + +static int video_enum_frame_sizes(struct v4l2_device *dev, __u32 pixelformat, + GList **p_vid_fmt_frm_l); +static int video_querycap(struct v4l2_device *dev); +static GByteArray *iterate_frame_rate_list(GByteArray *resp, + GList *frm_rate_l); +static GByteArray *iterate_format_frame_list(GByteArray *resp, + GList *fmt_frm_l); +static GByteArray *iterate_format_desc_list(GByteArray *resp, + GList *fmt_desc_l); +static void convert_to_timeval(uint64_t timestamp, struct timeval *t); + +/* v4l2 to str tables & helpers taken from yavta to make prettier logs */ +static struct v4l2_format_info { + const char *name; + unsigned int fourcc; + unsigned char n_planes; +} pixel_formats[] = { + { "RGB332", V4L2_PIX_FMT_RGB332, 1 }, + { "RGB444", V4L2_PIX_FMT_RGB444, 1 }, + { "ARGB444", V4L2_PIX_FMT_ARGB444, 1 }, + { "XRGB444", V4L2_PIX_FMT_XRGB444, 1 }, + { "RGB555", V4L2_PIX_FMT_RGB555, 1 }, + { "ARGB555", V4L2_PIX_FMT_ARGB555, 1 }, + { "XRGB555", V4L2_PIX_FMT_XRGB555, 1 }, + { "RGB565", V4L2_PIX_FMT_RGB565, 1 }, + { "RGB555X", V4L2_PIX_FMT_RGB555X, 1 }, + { "RGB565X", V4L2_PIX_FMT_RGB565X, 1 }, + { "BGR666", V4L2_PIX_FMT_BGR666, 1 }, + { "BGR24", V4L2_PIX_FMT_BGR24, 1 }, + { "RGB24", V4L2_PIX_FMT_RGB24, 1 }, + { "BGR32", V4L2_PIX_FMT_BGR32, 1 }, + { "ABGR32", V4L2_PIX_FMT_ABGR32, 1 }, + { "XBGR32", V4L2_PIX_FMT_XBGR32, 1 }, + { "RGB32", V4L2_PIX_FMT_RGB32, 1 }, + { "ARGB32", V4L2_PIX_FMT_ARGB32, 1 }, + { "XRGB32", V4L2_PIX_FMT_XRGB32, 1 }, + { "HSV24", V4L2_PIX_FMT_HSV24, 1 }, + { "HSV32", V4L2_PIX_FMT_HSV32, 1 }, + { "Y8", V4L2_PIX_FMT_GREY, 1 }, + { "Y10", V4L2_PIX_FMT_Y10, 1 }, + { "Y12", V4L2_PIX_FMT_Y12, 1 }, + { "Y16", V4L2_PIX_FMT_Y16, 1 }, + { "UYVY", V4L2_PIX_FMT_UYVY, 1 }, + { "VYUY", V4L2_PIX_FMT_VYUY, 1 }, + { "YUYV", V4L2_PIX_FMT_YUYV, 1 }, + { "YVYU", V4L2_PIX_FMT_YVYU, 1 }, + { "NV12", V4L2_PIX_FMT_NV12, 1 }, + { "NV12M", V4L2_PIX_FMT_NV12M, 2 }, + { "NV21", V4L2_PIX_FMT_NV21, 1 }, + { "NV21M", V4L2_PIX_FMT_NV21M, 2 }, + { "NV16", V4L2_PIX_FMT_NV16, 1 }, + { "NV16M", V4L2_PIX_FMT_NV16M, 2 }, + { "NV61", V4L2_PIX_FMT_NV61, 1 }, + { "NV61M", V4L2_PIX_FMT_NV61M, 2 }, + { "NV24", V4L2_PIX_FMT_NV24, 1 }, + { "NV42", V4L2_PIX_FMT_NV42, 1 }, + { "YU12", V4L2_PIX_FMT_YVU420, 1}, + { "YUV420M", V4L2_PIX_FMT_YUV420M, 3 }, + { "YUV422M", V4L2_PIX_FMT_YUV422M, 3 }, + { "YUV444M", V4L2_PIX_FMT_YUV444M, 3 }, + { "YVU420M", V4L2_PIX_FMT_YVU420M, 3 }, + { "YVU422M", V4L2_PIX_FMT_YVU422M, 3 }, + { "YVU444M", V4L2_PIX_FMT_YVU444M, 3 }, + { "SBGGR8", V4L2_PIX_FMT_SBGGR8, 1 }, + { "SGBRG8", V4L2_PIX_FMT_SGBRG8, 1 }, + { "SGRBG8", V4L2_PIX_FMT_SGRBG8, 1 }, + { "SRGGB8", V4L2_PIX_FMT_SRGGB8, 1 }, + { "SBGGR10_DPCM8", V4L2_PIX_FMT_SBGGR10DPCM8, 1 }, + { "SGBRG10_DPCM8", V4L2_PIX_FMT_SGBRG10DPCM8, 1 }, + { "SGRBG10_DPCM8", V4L2_PIX_FMT_SGRBG10DPCM8, 1 }, + { "SRGGB10_DPCM8", V4L2_PIX_FMT_SRGGB10DPCM8, 1 }, + { "SBGGR10", V4L2_PIX_FMT_SBGGR10, 1 }, + { "SGBRG10", V4L2_PIX_FMT_SGBRG10, 1 }, + { "SGRBG10", V4L2_PIX_FMT_SGRBG10, 1 }, + { "SRGGB10", V4L2_PIX_FMT_SRGGB10, 1 }, + { "SBGGR10P", V4L2_PIX_FMT_SBGGR10P, 1 }, + { "SGBRG10P", V4L2_PIX_FMT_SGBRG10P, 1 }, + { "SGRBG10P", V4L2_PIX_FMT_SGRBG10P, 1 }, + { "SRGGB10P", V4L2_PIX_FMT_SRGGB10P, 1 }, + { "SBGGR12", V4L2_PIX_FMT_SBGGR12, 1 }, + { "SGBRG12", V4L2_PIX_FMT_SGBRG12, 1 }, + { "SGRBG12", V4L2_PIX_FMT_SGRBG12, 1 }, + { "SRGGB12", V4L2_PIX_FMT_SRGGB12, 1 }, + { "IPU3_SBGGR10", V4L2_PIX_FMT_IPU3_SBGGR10, 1 }, + { "IPU3_SGBRG10", V4L2_PIX_FMT_IPU3_SGBRG10, 1 }, + { "IPU3_SGRBG10", V4L2_PIX_FMT_IPU3_SGRBG10, 1 }, + { "IPU3_SRGGB10", V4L2_PIX_FMT_IPU3_SRGGB10, 1 }, + { "DV", V4L2_PIX_FMT_DV, 1 }, + { "MJPEG", V4L2_PIX_FMT_MJPEG, 1 }, + { "MPEG", V4L2_PIX_FMT_MPEG, 1 }, + { "FWHT", V4L2_PIX_FMT_FWHT, 1 }, +}; + +static const struct v4l2_format_info *v4l2_format_by_fourcc(unsigned int fourcc) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(pixel_formats); ++i) { + if (pixel_formats[i].fourcc == fourcc) { + return &pixel_formats[i]; + } + } + + return NULL; +} + +static const char *v4l2_format_name(unsigned int fourcc) +{ + const struct v4l2_format_info *info; + static char name[5]; + unsigned int i; + + info = v4l2_format_by_fourcc(fourcc); + if (info) { + return info->name; + } + + for (i = 0; i < 4; ++i) { + name[i] = fourcc & 0xff; + fourcc >>= 8; + } + + name[4] = '\0'; + return name; +} + +static struct { + enum v4l2_buf_type type; + bool supported; + const char *name; +} buf_types_array[] = { + { V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE, 1, "Video capture mplanes", }, + { V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE, 1, "Video output mplanes", }, + { V4L2_BUF_TYPE_VIDEO_CAPTURE, 1, "Video capture", }, + { V4L2_BUF_TYPE_VIDEO_OUTPUT, 1, "Video output", }, + { V4L2_BUF_TYPE_VIDEO_OVERLAY, 0, "Video overlay", }, + { V4L2_BUF_TYPE_META_CAPTURE, 0, "Meta-data capture", }, + { V4L2_BUF_TYPE_META_OUTPUT, 0, "Meta-data output", }, +}; + +static const char *v4l2_buf_type_name(enum v4l2_buf_type type) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(buf_types_array); ++i) { + if (buf_types_array[i].type == type) { + return buf_types_array[i].name; + } + } + + if (type & V4L2_BUF_TYPE_PRIVATE) { + return "Private"; + } else { + return "Unknown"; + } +} + +static const struct { + const char *name; + enum v4l2_field field; +} fields[] = { + { "any", V4L2_FIELD_ANY }, + { "none", V4L2_FIELD_NONE }, + { "top", V4L2_FIELD_TOP }, + { "bottom", V4L2_FIELD_BOTTOM }, + { "interlaced", V4L2_FIELD_INTERLACED }, + { "seq-tb", V4L2_FIELD_SEQ_TB }, + { "seq-bt", V4L2_FIELD_SEQ_BT }, + { "alternate", V4L2_FIELD_ALTERNATE }, + { "interlaced-tb", V4L2_FIELD_INTERLACED_TB }, + { "interlaced-bt", V4L2_FIELD_INTERLACED_BT }, +}; + +static const char *v4l2_field_name(enum v4l2_field field) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(fields); ++i) { + if (fields[i].field == field) { + return fields[i].name; + } + } + + return "unknown"; +} + +int v4l2_open(const gchar *devname) +{ + int fd; + + if (!devname) { + return -EINVAL; + } + + fd = open(devname, O_RDWR | O_NONBLOCK); + if (fd < 0) { + g_printerr("Error opening device %s: %s (%d)\n", devname, + g_strerror(errno), errno); + return fd; + } + + g_print("Device %s opened fd(%d).\n", devname, fd); + + return fd; +} + +int v4l2_close(int fd) +{ + int ret; + ret = close(fd); + + if (ret < 0) { + g_printerr("Error closing device: %s (%d)\n", g_strerror(errno), errno); + } + return ret; +} + +static int video_enum_frame_intervals(struct v4l2_device *dev, + __u32 pixelformat, + unsigned int width, unsigned int height, + GList **p_vid_fmt_frm_rate_l) +{ + struct v4l2_frmivalenum ival; + GList *vid_fmt_frm_rate_l = NULL; + struct video_format_frame_rates *fmt_frm_rate; + unsigned int i; + int ret = 0; + + for (i = 0; ; ++i) { + memset(&ival, 0, sizeof ival); + ival.index = i; + ival.pixel_format = pixelformat; + ival.width = width; + ival.height = height; + ret = ioctl(dev->fd, VIDIOC_ENUM_FRAMEINTERVALS, &ival); + if (ret < 0) { + if (errno == EINVAL) /* EINVAL means no more frame intervals */ + ret = 0; + else + g_printerr("VIDIOC_ENUM_FRAMEINTERVALS failed: %s (%d)\n", + g_strerror(errno), errno); + break; + } + + /* driver sanity checks */ + if (i != ival.index) + g_printerr("Warning: driver returned wrong ival index " + "%u.\n", ival.index); + if (pixelformat != ival.pixel_format) + g_printerr("Warning: driver returned wrong ival pixel " + "format %08x.\n", ival.pixel_format); + if (width != ival.width) + g_printerr("Warning: driver returned wrong ival width " + "%u.\n", ival.width); + if (height != ival.height) + g_printerr("Warning: driver returned wrong ival height " + "%u.\n", ival.height); + + if (i != 0) { + g_print(", "); + } + + /* allocate video_format_frame */ + fmt_frm_rate = g_new0(struct video_format_frame_rates, 1); + /* keep a copy of v4l2 frmsizeenum struct */ + memcpy(&fmt_frm_rate->v4l_ival, &ival, + sizeof(struct v4l2_frmivalenum)); + vid_fmt_frm_rate_l = + g_list_append(vid_fmt_frm_rate_l, fmt_frm_rate); + + switch (ival.type) { + case V4L2_FRMIVAL_TYPE_DISCRETE: + g_debug("%u/%u", + ival.discrete.numerator, + ival.discrete.denominator); + + fmt_frm_rate->frame_rates.min = ival.discrete.denominator; + + break; + + case V4L2_FRMIVAL_TYPE_CONTINUOUS: + g_debug("%u/%u - %u/%u", + ival.stepwise.min.numerator, + ival.stepwise.min.denominator, + ival.stepwise.max.numerator, + ival.stepwise.max.denominator); + + fmt_frm_rate->frame_rates.min = ival.stepwise.min.denominator; + fmt_frm_rate->frame_rates.max = ival.stepwise.max.denominator; + fmt_frm_rate->frame_rates.step = 1; + + goto out; + + case V4L2_FRMIVAL_TYPE_STEPWISE: + g_debug("%u/%u - %u/%u (by %u/%u)", + ival.stepwise.min.numerator, + ival.stepwise.min.denominator, + ival.stepwise.max.numerator, + ival.stepwise.max.denominator, + ival.stepwise.step.numerator, + ival.stepwise.step.denominator); + + fmt_frm_rate->frame_rates.min = ival.stepwise.min.denominator; + fmt_frm_rate->frame_rates.max = ival.stepwise.max.denominator; + fmt_frm_rate->frame_rates.step = ival.stepwise.step.denominator; + + goto out; + + default: + break; + } + } + +out: + if (ret == 0) { + g_print("\n%s: Enumerated %d frame intervals\n", __func__ + , g_list_length(vid_fmt_frm_rate_l)); + g_return_val_if_fail(i == g_list_length(vid_fmt_frm_rate_l), -EINVAL); + *p_vid_fmt_frm_rate_l = vid_fmt_frm_rate_l; + } + + return ret; +} + +static void video_frame_size_discrete(struct v4l2_device *dev, + struct video_format_frame *vid_frame, + struct v4l2_frmsizeenum *frame) +{ + g_debug("\tFrame size (D): %ux%u (", frame->discrete.width, + frame->discrete.height); + + vid_frame->frame.width.min = htole32(frame->discrete.width); + vid_frame->frame.width.max = htole32(frame->discrete.width); + vid_frame->frame.height.min = htole32(frame->discrete.height); + vid_frame->frame.height.max = htole32(frame->discrete.height); + + if (video_enum_frame_intervals(dev, frame->pixel_format, + frame->discrete.width, + frame->discrete.height, + &vid_frame->frm_rate_l) < 0) { + g_printerr("%s: video_enum_frame_intervals failed!", __func__); + } + g_debug(")"); +} + +static void video_frame_size_continuous(struct v4l2_device *dev, + struct video_format_frame *vid_frame, + struct v4l2_frmsizeenum *frame) +{ + g_debug("\tFrame size (C): %ux%u - %ux%u (", + frame->stepwise.min_width, + frame->stepwise.min_height, + frame->stepwise.max_width, + frame->stepwise.max_height); + + vid_frame->frame.width.min = htole32(frame->stepwise.min_width); + vid_frame->frame.width.max = htole32(frame->stepwise.max_width); + vid_frame->frame.width.step = htole32(frame->stepwise.step_width); + vid_frame->frame.height.min = htole32(frame->stepwise.min_height); + vid_frame->frame.height.max = htole32(frame->stepwise.max_height); + vid_frame->frame.height.step = htole32(frame->stepwise.step_height); + + /* driver sanity check */ + if (frame->stepwise.step_height != 1 || + frame->stepwise.step_width != 1) { + g_printerr("Warning: invalid step for continuous framesize"); + } + + if (video_enum_frame_intervals(dev, frame->pixel_format, + frame->stepwise.max_width, + frame->stepwise.max_height, + &vid_frame->frm_rate_l) < 0) { + g_printerr("%s: video_enum_frame_intervals failed!\n", __func__); + } + + g_debug(")"); +} + +static void video_frame_size_stepwise(struct v4l2_device *dev, + struct video_format_frame *vid_frame, + struct v4l2_frmsizeenum *frame) +{ + g_debug("\tFrame size (S): %ux%u - %ux%u (by %ux%u) (", + frame->stepwise.min_width, + frame->stepwise.min_height, + frame->stepwise.max_width, + frame->stepwise.max_height, + frame->stepwise.step_width, + frame->stepwise.step_height); + + vid_frame->frame.width.min = htole32(frame->stepwise.min_width); + vid_frame->frame.width.max = htole32(frame->stepwise.max_width); + vid_frame->frame.width.step = htole32(frame->stepwise.step_width); + vid_frame->frame.height.min = htole32(frame->stepwise.min_height); + vid_frame->frame.height.max = htole32(frame->stepwise.max_height); + vid_frame->frame.height.step = htole32(frame->stepwise.step_height); + + if (video_enum_frame_intervals(dev, frame->pixel_format, + frame->stepwise.max_width, + frame->stepwise.max_height, + &vid_frame->frm_rate_l) < 0) { + g_printerr("%s: video_enum_frame_intervals failed!\n", __func__); + } + + g_debug(")"); +} + +static int video_enum_frame_sizes(struct v4l2_device *dev, + __u32 pixelformat, GList **p_vid_fmt_frm_l) +{ + struct v4l2_frmsizeenum frame; + struct video_format_frame *vid_frame = NULL; + GList *vid_fmt_frm_l = NULL; + unsigned int i; + int ret; + + if (!dev) { + return -EINVAL; + } + + for (i = 0; ; ++i) { + memset(&frame, 0, sizeof frame); + frame.index = i; + frame.pixel_format = pixelformat; + ret = ioctl(dev->fd, VIDIOC_ENUM_FRAMESIZES, &frame); + if (ret < 0) { + if (errno == EINVAL) /* EINVAL means no more frame sizes */ + ret = 0; + else + g_printerr("VIDIOC_ENUM_FRAMESIZES failed: %s (%d)\n", + g_strerror(errno), errno); + break; + } + + /* driver sanity checks */ + if (i != frame.index) + g_printerr("Warning: driver returned wrong frame index " + "%u.\n", frame.index); + if (pixelformat != frame.pixel_format) + g_printerr("Warning: driver returned wrong frame pixel " + "format %08x.\n", frame.pixel_format); + + /* allocate video_format_frame */ + vid_frame = g_new0(struct video_format_frame, 1); + /* keep a copy of v4l2 frmsizeenum struct */ + memcpy(&vid_frame->v4l_framesize, &frame, + sizeof(struct v4l2_frmsizeenum)); + vid_fmt_frm_l = g_list_append(vid_fmt_frm_l, vid_frame); + + switch (frame.type) { + case V4L2_FRMSIZE_TYPE_DISCRETE: + video_frame_size_discrete(dev, vid_frame, &frame); + break; + case V4L2_FRMSIZE_TYPE_CONTINUOUS: + video_frame_size_continuous(dev, vid_frame, &frame); + break; + case V4L2_FRMSIZE_TYPE_STEPWISE: + video_frame_size_stepwise(dev, vid_frame, &frame); + break; + default: + break; + } + } + if (ret == 0) { + g_print("%s: Enumerated %d frame sizes and %d frame intervals\n", + __func__, g_list_length(vid_fmt_frm_l), + g_list_length(vid_frame->frm_rate_l)); + + vid_frame->frame.num_rates = + htole32(g_list_length(vid_frame->frm_rate_l)); + + g_return_val_if_fail(i == g_list_length(vid_fmt_frm_l), -EINVAL); + *p_vid_fmt_frm_l = vid_fmt_frm_l; + } + + return ret; +} + +int video_send_decoder_start_cmd(struct v4l2_device *dev) +{ + return v4l2_issue_cmd(dev->fd, /*cmd=*/V4L2_DEC_CMD_START, /*flags=*/0); +} + +static int video_querycap(struct v4l2_device *dev) +{ + struct v4l2_capability cap; + unsigned int caps; + bool has_video; + bool has_meta; + bool has_capture; + bool has_output; + bool has_mplane; + int ret; + + memset(&cap, 0, sizeof cap); + ret = ioctl(dev->fd, VIDIOC_QUERYCAP, &cap); + if (ret < 0) { + return 0; + } + + caps = cap.capabilities & V4L2_CAP_DEVICE_CAPS ? + cap.device_caps : + cap.capabilities; + + has_video = caps & (V4L2_CAP_VIDEO_CAPTURE_MPLANE | + V4L2_CAP_VIDEO_CAPTURE | + V4L2_CAP_VIDEO_OUTPUT_MPLANE | + V4L2_CAP_VIDEO_OUTPUT); + has_meta = caps & (V4L2_CAP_META_CAPTURE | + V4L2_CAP_META_OUTPUT); + has_capture = caps & (V4L2_CAP_VIDEO_CAPTURE_MPLANE | + V4L2_CAP_VIDEO_CAPTURE | + V4L2_CAP_META_CAPTURE); + has_output = caps & (V4L2_CAP_VIDEO_OUTPUT_MPLANE | + V4L2_CAP_VIDEO_OUTPUT | + V4L2_CAP_META_OUTPUT); + has_mplane = caps & (V4L2_CAP_VIDEO_CAPTURE_MPLANE | + V4L2_CAP_VIDEO_OUTPUT_MPLANE | + V4L2_CAP_VIDEO_M2M_MPLANE); + + g_print("Device '%s' on '%s' (driver '%s') " + "supports%s%s%s%s %s mplanes.\n", + cap.card, cap.bus_info, cap.driver, + has_video ? " video," : "", + has_meta ? " meta-data," : "", + has_capture ? " capture," : "", + has_output ? " output," : "", + has_mplane ? "with" : "without"); + + dev->capabilities = caps; + dev->has_mplane = has_mplane; + + return 0; +} + +void v4l2_set_device_type(struct v4l2_device *dev, enum v4l2_buf_type type, + struct v4l2_fmtdesc *fmt_desc) +{ + if (fmt_desc->flags & V4L2_FMT_FLAG_COMPRESSED) { + switch (fmt_desc->pixelformat) { + case V4L2_PIX_FMT_H263: + case V4L2_PIX_FMT_H264: + case V4L2_PIX_FMT_H264_NO_SC: + case V4L2_PIX_FMT_H264_MVC: + case V4L2_PIX_FMT_MPEG1: + case V4L2_PIX_FMT_MPEG2: + case V4L2_PIX_FMT_MPEG4: + case V4L2_PIX_FMT_XVID: + case V4L2_PIX_FMT_VC1_ANNEX_G: + case V4L2_PIX_FMT_VC1_ANNEX_L: + case V4L2_PIX_FMT_VP8: + case V4L2_PIX_FMT_VP9: + case V4L2_PIX_FMT_HEVC: + case V4L2_PIX_FMT_FWHT: + if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT) { + dev->dev_type |= STATEFUL_DECODER; + } + if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { + dev->dev_type |= STATEFUL_DECODER; + } + if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE) { + dev->dev_type |= STATEFUL_ENCODER; + } + if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) { + dev->dev_type |= STATEFUL_ENCODER; + } + break; + case V4L2_PIX_FMT_MPEG2_SLICE: + case V4L2_PIX_FMT_FWHT_STATELESS: + if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT) { + dev->dev_type |= STATELESS_DECODER; + } + if (type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { + dev->dev_type |= STATELESS_DECODER; + } + if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE) { + dev->dev_type |= STATELESS_ENCODER; + } + if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) { + dev->dev_type |= STATELESS_ENCODER; + } + break; + default: + break; + } + } +} + +enum v4l2_buf_type +get_v4l2_buf_type(enum virtio_video_queue_type queue_type, bool has_mplane) +{ + enum v4l2_buf_type buf_type = 0; + + switch (queue_type) { + case VIRTIO_VIDEO_QUEUE_TYPE_INPUT: + buf_type = has_mplane ? V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE + : V4L2_BUF_TYPE_VIDEO_OUTPUT; + break; + case VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT: + buf_type = has_mplane ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE + : V4L2_BUF_TYPE_VIDEO_CAPTURE; + break; + default: + g_warning("%s: Unknown queue_type!", __func__); + break; + } + + g_debug("%s: queue_type(0x%x) has_mplane(%d), buf_type(%s)", + __func__, queue_type, has_mplane, v4l2_buf_type_name(buf_type)); + + return buf_type; +} + +enum v4l2_memory get_v4l2_memory(enum virtio_video_mem_type mem_type) +{ + /* if using GUEST_PAGES queued using USERPTR mechanism */ + enum v4l2_memory memory = V4L2_MEMORY_USERPTR; + if (mem_type == VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT) { + memory = V4L2_MEMORY_DMABUF; + } else { + assert(mem_type == VIRTIO_VIDEO_MEM_TYPE_GUEST_PAGES); + } + g_debug("%s: mem_type(0x%x) memory(0x%x)", __func__, mem_type, memory); + return memory; +} + +enum virtio_video_mem_type +get_queue_mem_type(struct stream *s, + enum virtio_video_queue_type queue_type) +{ + if (queue_type == VIRTIO_VIDEO_QUEUE_TYPE_INPUT) { + return s->vio_stream.in_mem_type; + } else { + return s->vio_stream.out_mem_type; + } +} + +int video_free_buffers(int fd, enum v4l2_buf_type type, enum v4l2_memory memory) +{ + int ret, count = 0; + g_debug("%s: v4l2_buf_type: %s: Issuing REQBUFS 0" + , __func__, v4l2_buf_type_name(type)); + + /* + * Applications can call ioctl VIDIOC_REQBUFS again to change the number + * of buffers. Note that if any buffers are still mapped or exported via + * DMABUF, then ioctl VIDIOC_REQBUFS can only succeed if the + * V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS capability is set. Otherwise ioctl + * VIDIOC_REQBUFS will return the EBUSY error code. If + * V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS is set, then these buffers are + * orphanedand will be freed when they are unmapped or when the exported + * DMABUF fds are closed. A count value of zero frees or orphans all + * buffers, after aborting or finishing any DMA in progress, an implicit + * VIDIOC_STREAMOFF. + */ + + /* TODO support V4L2_BUF_CAP_SUPPORTS_ORPHANED_BUFS */ + ret = v4l2_ioctl_reqbuf(fd, type, memory, &count); + return ret; +} + +int video_resource_create(struct stream *s, + uint32_t queue_type, uint32_t queue_len) +{ + int ret, count; + enum v4l2_buf_type buf_type = get_v4l2_buf_type(queue_type, s->has_mplane); + enum virtio_video_mem_type mem_type = get_queue_mem_type(s, queue_type); + + count = queue_len; + ret = v4l2_ioctl_reqbuf(s->fd, buf_type, get_v4l2_memory(mem_type), &count); + if (ret < 0) { + return ret; + } + + if (V4L2_TYPE_IS_OUTPUT(buf_type)) { + s->output_bufcount = count; + } else if (V4L2_TYPE_IS_CAPTURE(buf_type)) { + s->capture_bufcount = count; + } + + if (count > queue_len) { + g_critical("Unsupported feature: driver initiated more buffers(%d) " + "than requested(%d)", count, queue_len); + return -EINVAL; + } + + return ret; +} + +int v4l2_streamon(struct stream *s, enum v4l2_buf_type type) +{ + int ret = 0; + ret = ioctl(s->fd, VIDIOC_STREAMON, &type); + if (ret < 0) { + g_printerr("VIDIOC_STREAMON failed: fd=(%d) buf type=%s: %s (%d)\n", + s->fd, v4l2_buf_type_name(type), g_strerror(errno), errno); + } else { + g_debug("%s: VIDIOC_STREAMON OK fd=(%d) buf type: %s", + __func__, s->fd, v4l2_buf_type_name(type)); + if (V4L2_TYPE_IS_OUTPUT(type)) { + s->output_streaming = true; + } + if (V4L2_TYPE_IS_CAPTURE(type)) { + s->capture_streaming = true; + } + } + return ret; +} + +int v4l2_streamoff(struct stream *s, enum v4l2_buf_type type) +{ + int ret = 0; + ret = ioctl(s->fd, VIDIOC_STREAMOFF, &type); + if (ret < 0) { + g_printerr("VIDIOC_STREAMOFF failed: fd=(%d) buf type=%s: %s (%d)\n", + s->fd, v4l2_buf_type_name(type), g_strerror(errno), errno); + } else { + g_debug("%s: VIDIOC_STREAMOFF OK buf type: %s", + __func__, v4l2_buf_type_name(type)); + + if (V4L2_TYPE_IS_OUTPUT(type)) { + s->output_streaming = false; + } + if (V4L2_TYPE_IS_CAPTURE(type)) { + s->capture_streaming = false; + } + + /* + * if either queue has STREAMOFF applied, then we enter STOPPED + * Assumes that s->mutex is held by calling function + */ + s->stream_state = STREAM_STOPPED; + g_cond_signal(&s->stream_cond); + } + return ret; +} + +/* activate streaming on both queues */ +int video_streamon(struct stream *s, + struct v4l2_device *dev, enum v4l2_buf_type type) +{ + int ret = 0; + uint32_t id = 0; + + if (!s->subscribed_events) { + /* subscribe for SOURCE_CHANGE event */ + if (dev->sup_dyn_res_switching) { + ret = v4l2_ioctl_subscribe_event( + s->fd, V4L2_EVENT_SOURCE_CHANGE, id); + if (ret < 0) { + g_printerr("V4L2_EVENT_SOURCE_CHANGE failed: %s (%d)\n", + g_strerror(errno), errno); + } + } + /* subscribe for EOS event */ + ret = v4l2_ioctl_subscribe_event(s->fd, V4L2_EVENT_EOS, id); + if (ret < 0) { + g_printerr("V4L2_EVENT_EOS failed: %s (%d)\n", + g_strerror(errno), errno); + } + s->subscribed_events = true; + } + + switch (type) { + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: + case V4L2_BUF_TYPE_VIDEO_OUTPUT: + case V4L2_BUF_TYPE_META_OUTPUT: + if (s->output_streaming == false) { + ret |= v4l2_streamon(s, type); + } + break; + + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: + case V4L2_BUF_TYPE_VIDEO_CAPTURE: + case V4L2_BUF_TYPE_META_CAPTURE: + if (s->capture_streaming == false) { + ret |= v4l2_streamon(s, type); + } + break; + + default: + g_printerr("%s: unknown v4l2 buffer type!", __func__); + ret = EINVAL; + } + + if (s->stream_state != STREAM_DRAINING) { + s->stream_state = STREAM_STREAMING; + g_cond_signal(&s->stream_cond); + } + + return ret; +} + +int video_streamoff(struct stream *s, enum v4l2_buf_type type) +{ + int ret = 0; + bool is_mplane = V4L2_TYPE_IS_MULTIPLANAR(type); + enum v4l2_buf_type type2; + + switch (type) { + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: + case V4L2_BUF_TYPE_VIDEO_OUTPUT: + case V4L2_BUF_TYPE_META_OUTPUT: + if (s->output_streaming == true) { + ret |= v4l2_streamoff(s, type); + } + + if (s->capture_streaming == true) { + type2 = is_mplane ? + V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE : + V4L2_BUF_TYPE_VIDEO_CAPTURE; + ret |= v4l2_streamoff(s, type2); + } + break; + + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: + case V4L2_BUF_TYPE_VIDEO_CAPTURE: + case V4L2_BUF_TYPE_META_CAPTURE: + if (s->capture_streaming == true) { + ret |= v4l2_streamoff(s, type); + } + if (s->output_streaming == true) { + type2 = is_mplane ? + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE : + V4L2_BUF_TYPE_VIDEO_OUTPUT; + ret |= v4l2_streamoff(s, type2); + } + break; + + default: + g_printerr("%s: unknown v4l2 buffer type!", __func__); + ret = EINVAL; + } + + return ret; +} + +int v4l2_queue_buffer(enum v4l2_buf_type type, + enum v4l2_memory memory, + struct virtio_video_resource_queue *qcmd, + struct resource *res, struct stream *s, + struct v4l2_device *dev) +{ + struct v4l2_buffer vbuf; + int ret = 0; + int fd = s->fd; + + memset(&vbuf, 0, sizeof(vbuf)); + vbuf.index = res->v4l2_index; + + vbuf.type = type; + vbuf.memory = memory; + vbuf.field = V4L2_FIELD_NONE; + vbuf.flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + + g_debug("%s: type=%s memory=%d index=%d", __func__, + v4l2_buf_type_name(type), memory, vbuf.index); + + convert_to_timeval(le64toh(qcmd->timestamp), &vbuf.timestamp); + + if (V4L2_TYPE_IS_MULTIPLANAR(type)) { + /* for mplane length field is number of elements in planes array */ + vbuf.length = res->vio_resource.num_planes; + vbuf.m.planes = g_malloc0(sizeof(struct v4l2_plane) + * res->vio_resource.num_planes); + + for (int i = 0; i < vbuf.length; i++) { + vbuf.m.planes[i].m.userptr = (unsigned long)res->iov[i].iov_base; + vbuf.m.planes[i].length = (unsigned long)res->iov[i].iov_len; + } + } else if (res->iov != NULL) { + /* m is a union of userptr, *planes and fd */ + vbuf.m.userptr = (unsigned long)res->iov[0].iov_base; + vbuf.length = res->iov[0].iov_len; + g_debug("%s: iov_base = 0x%p", __func__, res->iov[0].iov_base); + g_debug("%s: iov_len = 0x%lx", __func__, res->iov[0].iov_len); + } + + if (V4L2_TYPE_IS_OUTPUT(type)) { + if (V4L2_TYPE_IS_MULTIPLANAR(type)) { + for (int i = 0; i < vbuf.length; i++) { + vbuf.m.planes[i].bytesused = vbuf.m.planes[i].length; + } + } else { + vbuf.bytesused = vbuf.length; + } + } + + ret = ioctl(fd, VIDIOC_QBUF, &vbuf); + if (ret < 0) { + qcmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + g_printerr("Unable to QBUF: %s (%d).\n", g_strerror(errno), errno); + return ret; + } + + ret = video_streamon(s, dev, type); + if (ret < 0) { + g_printerr("video_streamon failed (%d)", ret); + /* only print error, as video_streamon() does both queues */ + } + + res->queued = true; + if (V4L2_TYPE_IS_MULTIPLANAR(type)) { + g_free(vbuf.m.planes); + } + + g_debug("%s: Queued resource-id(%d) buf_type=%s v4l2_index(%d) " + "virtio_queue(0x%x)", __func__, res->vio_resource.resource_id, + v4l2_buf_type_name(type), res->v4l2_index, + res->vio_resource.queue_type); + + return ret; + +} + +int v4l2_dequeue_buffer(int fd, enum v4l2_buf_type type, + enum v4l2_memory memory, + struct stream *s) +{ + struct v4l2_buffer vbuf; + int ret = 0; + struct resource *r; + struct virtio_video_resource_queue_resp resp; + struct vu_video_ctrl_command *vio_cmd; + + memset(&vbuf, 0, sizeof(vbuf)); + + vbuf.type = type; + vbuf.memory = memory; + + vbuf.field = V4L2_FIELD_NONE; + vbuf.flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + + if (V4L2_TYPE_IS_MULTIPLANAR(type)) { + /* for mplane length field is number of elements in planes array */ + vbuf.length = VIRTIO_VIDEO_MAX_PLANES; + vbuf.m.planes = g_malloc0(sizeof(struct v4l2_plane) + * VIRTIO_VIDEO_MAX_PLANES); + + g_debug("%s: mplane allocating planes array", __func__); + } + + ret = ioctl(fd, VIDIOC_DQBUF, &vbuf); + if (ret < 0) { + g_printerr("Unable to DQBUF: %s (%d).\n", g_strerror(errno), errno); + return -errno; + } + + g_debug("%s: VIDIOC_DQBUF OK index(%d)", __func__, vbuf.index); + + if (V4L2_TYPE_IS_MULTIPLANAR(type)) { + g_free(vbuf.m.planes); + } + + r = find_resource_by_v4l2index(s, type, vbuf.index); + if (!r) { + g_printerr("%s: Can't find resource for dequeued buffer!", __func__); + return -EINVAL; + } + + r->queued = false; + vio_cmd = r->vio_q_cmd; + + resp.flags = 0x0; + resp.hdr.stream_id = r->stream_id; + resp.hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + resp.timestamp = htole64(r->vio_res_q.timestamp); + + /* encoder only */ + resp.size = htole32(vbuf.bytesused); + + if (vbuf.flags & V4L2_BUF_FLAG_LAST && + s->stream_state == STREAM_DRAINING) { + resp.flags |= VIRTIO_VIDEO_BUFFER_FLAG_EOS; + s->stream_state = STREAM_STOPPED; + g_cond_signal(&s->stream_cond); + } + + if (vbuf.flags & V4L2_BUF_FLAG_KEYFRAME) { + resp.flags |= VIRTIO_VIDEO_BUFFER_FLAG_IFRAME; + } + if (vbuf.flags & V4L2_BUF_FLAG_PFRAME) { + resp.flags |= VIRTIO_VIDEO_BUFFER_FLAG_PFRAME; + } + if (vbuf.flags & V4L2_BUF_FLAG_BFRAME) { + resp.flags |= VIRTIO_VIDEO_BUFFER_FLAG_PFRAME; + } + + if (vbuf.flags & V4L2_BUF_FLAG_ERROR) { + resp.flags |= VIRTIO_VIDEO_BUFFER_FLAG_ERR; + g_critical("%s: V4L2_BUF_FLAG_ERROR\n", __func__); + } + + g_debug("%s: Send queue_buffer reply: stream_id=%d type=0x%x " + "flags=0x%x resource_id=%d t=%llx", __func__, + resp.hdr.stream_id, resp.hdr.type, resp.flags, + r->vio_resource.resource_id, resp.timestamp); + + send_ctrl_response(vio_cmd, (uint8_t *) &resp, + sizeof(struct virtio_video_resource_queue_resp)); + + vio_cmd->finished = true; + free_resource_mem(r); + + return ret; +} + +int v4l2_video_get_selection(int fd, enum v4l2_buf_type type, + struct v4l2_selection *sel) +{ + int ret = 0; + + if (!sel) { + return -EINVAL; + } + + memset(sel, 0, sizeof(struct v4l2_selection)); + + sel->type = type; + + if (V4L2_TYPE_IS_CAPTURE(type)) { + sel->target = V4L2_SEL_TGT_COMPOSE; + } else if (V4L2_TYPE_IS_OUTPUT(type)) { + sel->target = V4L2_SEL_TGT_CROP; + } + + ret = ioctl(fd, VIDIOC_G_SELECTION, sel); + if (ret < 0) { + g_printerr("Unable to get selection: %s (%d)\n", + g_strerror(errno), errno); + return ret; + } + + g_debug("%s: VIDIOC_G_SELECTION: fd=(%d) %s: left=(%d) " + "top=(%d) width=(%d) height=(%d)", + __func__, fd, v4l2_buf_type_name(type), sel->r.left, + sel->r.top, sel->r.width, sel->r.height); + + return ret; +} + +int v4l2_video_set_selection(int fd, enum v4l2_buf_type type, + struct v4l2_selection *sel) +{ + int ret = 0; + + if (!sel) { + return -EINVAL; + } + + sel->type = type; + sel->target = V4L2_SEL_TGT_COMPOSE; + /* flags 0 - the driver can adjust the rect size freely */ + sel->flags = 0; + + ret = ioctl(fd, VIDIOC_S_SELECTION, sel); + if (ret < 0) { + g_printerr("Unable to set selection: fd=(%d) left=(%d) top=(%d) " + "width=(%d) height=(%d): %s (%d)\n", + fd, sel->r.left, sel->r.top, sel->r.width, sel->r.height, + g_strerror(errno), errno); + return ret; + } + + g_debug("%s: VIDIOC_S_SELECTION: fd=(%d) left=(%d) " + "top=(%d) width=(%d) height=(%d)", + __func__, fd, sel->r.left, sel->r.top, sel->r.width, sel->r.height); + + return ret; +} + +int v4l2_video_get_param(int fd, enum v4l2_buf_type type, + struct v4l2_streamparm *sparam) +{ + int ret = 0; + + if (!sparam) { + return -EINVAL; + } + + memset(sparam, 0, sizeof(struct v4l2_streamparm)); + sparam->type = type; + + ret = ioctl(fd, VIDIOC_G_PARM, sparam); + if (ret < 0) { + g_printerr("Unable to VIDIOC_G_PARAM: %s (%d)\n", + g_strerror(errno), errno); + return ret; + } + + g_debug("%s: VIDIOC_G_PARM timeperframe (%d/%d)", __func__, + sparam->parm.capture.timeperframe.numerator, + sparam->parm.capture.timeperframe.denominator); + + return ret; +} + +int v4l2_video_get_format(int fd, enum v4l2_buf_type type, + struct v4l2_format *fmt) +{ + unsigned int i; + int ret; + + if (!fmt) { + return -EINVAL; + } + + memset(fmt, 0, sizeof(struct v4l2_format)); + fmt->type = type; + + ret = ioctl(fd, VIDIOC_G_FMT, fmt); + if (ret < 0) { + g_printerr("Unable to get format: %s (%d)\n", + g_strerror(errno), errno); + return ret; + } + + if (V4L2_TYPE_IS_MULTIPLANAR(type)) { + g_print("Video format: %s (%08x) %ux%u field %s, %u planes:\n", + v4l2_format_name(fmt->fmt.pix_mp.pixelformat), + fmt->fmt.pix_mp.pixelformat, + fmt->fmt.pix_mp.width, + fmt->fmt.pix_mp.height, + v4l2_field_name(fmt->fmt.pix_mp.field), + fmt->fmt.pix_mp.num_planes); + + for (i = 0; i < fmt->fmt.pix_mp.num_planes; i++) { + g_print(" * Stride %u, buffer size %u\n", + fmt->fmt.pix_mp.plane_fmt[i].bytesperline, + fmt->fmt.pix_mp.plane_fmt[i].sizeimage); + } + } else if (V4L2_TYPE_IS_META(type)) { + g_print("Meta-data format: %s (%08x) buffer size %u\n", + v4l2_format_name(fmt->fmt.meta.dataformat), + fmt->fmt.meta.dataformat, + fmt->fmt.meta.buffersize); + } else { + g_print("Video format: %s (%08x) %ux%u (stride %u) field %s " + "buffer size %u\n", + v4l2_format_name(fmt->fmt.pix.pixelformat), + fmt->fmt.pix.pixelformat, + fmt->fmt.pix.width, fmt->fmt.pix.height, + fmt->fmt.pix.bytesperline, + v4l2_field_name(fmt->fmt.pix_mp.field), + fmt->fmt.pix.sizeimage); + } + + return 0; +} + +int v4l2_video_set_format(int fd, enum v4l2_buf_type type, + struct virtio_video_params *p) +{ + struct v4l2_format fmt; + int ret = 0; + unsigned int i; + uint32_t pixfmt; + + if (!p) { + return -EINVAL; + } + + memset(&fmt, 0, sizeof fmt); + fmt.type = type; + pixfmt = virtio_video_format_to_v4l2(le32toh(p->format)); + + if (V4L2_TYPE_IS_MULTIPLANAR(type)) { + fmt.fmt.pix_mp.width = le32toh(p->frame_width); + fmt.fmt.pix_mp.height = le32toh(p->frame_height); + fmt.fmt.pix_mp.pixelformat = pixfmt; + /* + * V4L2_FIELD_NONE - matches what Linux frontend driver does in + * virtio_video_format_from_info() + */ + fmt.fmt.pix_mp.field = V4L2_FIELD_NONE; + /*fmt.fmt.pix_mp.num_planes = info->n_planes;*/ + fmt.fmt.pix_mp.num_planes = le32toh(p->num_planes); + fmt.fmt.pix_mp.flags = 0; + + for (i = 0; i < le32toh(p->num_planes); i++) { + fmt.fmt.pix_mp.plane_fmt[i].bytesperline = + le32toh(p->plane_formats[i].stride); + fmt.fmt.pix_mp.plane_fmt[i].sizeimage = + le32toh(p->plane_formats[i].plane_size); + } + } else if (V4L2_TYPE_IS_SINGLEPLANAR(type)) { + fmt.fmt.pix.width = le32toh(p->frame_width); + fmt.fmt.pix.height = le32toh(p->frame_height); + fmt.fmt.pix.pixelformat = pixfmt; + fmt.fmt.pix.field = V4L2_FIELD_NONE; + fmt.fmt.pix.bytesperline = le32toh(p->plane_formats[0].stride); + fmt.fmt.pix.sizeimage = le32toh(p->plane_formats[0].plane_size); + fmt.fmt.pix.priv = V4L2_PIX_FMT_PRIV_MAGIC; + fmt.fmt.pix.flags = 0; + } + + ret = ioctl(fd, VIDIOC_S_FMT, &fmt); + if (ret < 0) { + g_printerr("Unable to set format: %s (%d)\n", + g_strerror(errno), errno); + } + return ret; +} + +int +v4l2_set_pixel_format(int fd, enum v4l2_buf_type buf_type, uint32_t pixelformat) +{ + int ret = 0; + struct v4l2_format cur_fmt; + + g_debug("%s: buf_type=0x%x pixelformat=0x%x", __func__, + buf_type, pixelformat); + + /* get the currently set format */ + ret = v4l2_video_get_format(fd, buf_type, &cur_fmt); + if (ret < 0) { + g_printerr("%s: v4l2_video_get_format() failed\n", __func__); + return ret; + } + + /* keep defaults and set correct pixel format */ + if (V4L2_TYPE_IS_MULTIPLANAR(cur_fmt.type)) { + g_print("%s: Format is mplane\n", __func__); + cur_fmt.fmt.pix_mp.pixelformat = pixelformat; + } else if (V4L2_TYPE_IS_SINGLEPLANAR(cur_fmt.type)) { + g_print("%s: Format is splane\n", __func__); + cur_fmt.fmt.pix.pixelformat = pixelformat; + } + + ret = ioctl(fd, VIDIOC_S_FMT, &cur_fmt); + if (ret < 0) { + g_printerr("Unable to set format: %s (%d)\n", + g_strerror(errno), errno); + } + + return ret; +} + +int video_enum_formats(struct v4l2_device *dev, enum v4l2_buf_type type, + GList **p_fmt_list, bool only_enum_fmt) +{ + struct v4l2_fmtdesc fmt; + struct video_format *vid_fmt = NULL; + GList *fmt_list = NULL; + unsigned int index; + int ret = 0; + + if (!dev) { + return -EINVAL; + } + + for (index = 0; ; ++index) { + memset(&fmt, 0, sizeof fmt); + fmt.index = index; + fmt.type = type; + + ret = ioctl(dev->fd, VIDIOC_ENUM_FMT, &fmt); + if (ret < 0) { + if (errno == EINVAL) { + ret = 0; + } else { + g_printerr("%s: VIDIOC_ENUM_FMT failed %s (%d)\n", + __func__, g_strerror(errno), errno); + } + break; + } + + /* do some driver sanity checks */ + if (index != fmt.index) { + g_warning("v4l2 driver modified index %u\n", fmt.index); + } + if (type != fmt.type) { + g_warning("v4l2 driver modified type %u\n", fmt.type); + } + g_debug("\tFormat %u: %s (%08x)", index, + v4l2_format_name(fmt.pixelformat), fmt.pixelformat); + g_debug("\tType: %s (%u)", v4l2_buf_type_name(fmt.type), + fmt.type); + g_debug("\tName: %.32s", fmt.description); + g_debug("\tFlags: 0x%x", fmt.flags); + + if (fmt.flags & V4L2_FMT_FLAG_DYN_RESOLUTION && + fmt.flags & V4L2_FMT_FLAG_COMPRESSED) { + g_print("dynamic resolution switching supported\n"); + dev->sup_dyn_res_switching = true; + } + + /* test if pixelformat converts to virtio */ + if (!virtio_video_v4l2_format_to_virtio(fmt.pixelformat)) { + g_info("Skipping Format %s (%08x) - no virtio-video equivalent", + v4l2_format_name(fmt.pixelformat), fmt.pixelformat); + continue; + } + + vid_fmt = g_new0(struct video_format, 1); + /* keep a copy of v4l2 struct */ + memcpy(&vid_fmt->fmt, &fmt, sizeof(struct v4l2_fmtdesc)); + /* add it to linked list */ + fmt_list = g_list_append(fmt_list, vid_fmt); + + if (!only_enum_fmt) { + /* pass video_format to enum_frame_sizes */ + ret = video_enum_frame_sizes(dev, fmt.pixelformat, + &vid_fmt->vid_fmt_frm_l); + if (ret < 0) { + g_printerr("video_enum_frame_sizes failed\n"); + } + + /* convert to virtio format */ + v4l2_to_virtio_fmtdesc(dev, vid_fmt, type); + } + + /* determine type of v4l2 device */ + v4l2_set_device_type(dev, type, &fmt); + } + + if (ret == 0) { + g_print("%s: Enumerated %d formats on v4l2 %s queue", + __func__, index, v4l2_buf_type_name(type)); + g_print(" %d formats are representable by virtio-video\n", + g_list_length(fmt_list)); + if (!only_enum_fmt) { + g_print("%s: Enumerated %d frame sizes\n", + __func__, g_list_length(vid_fmt->vid_fmt_frm_l)); + } + + *p_fmt_list = fmt_list; + } + + return ret; +} + +void video_free_frame_intervals(GList *frm_intervals_l) +{ + GList *l; + struct video_format_frame_rates *vid_fmt_frm_rate; + for (l = frm_intervals_l; l != NULL; l = l->next) { + vid_fmt_frm_rate = l->data; + g_free(vid_fmt_frm_rate); + } +} + +void video_free_frame_sizes(GList *frm_sz_l) +{ + GList *l; + struct video_format_frame *vid_frame; + for (l = frm_sz_l; l != NULL; l = l->next) { + vid_frame = l->data; + if (vid_frame->frm_rate_l) { + video_free_frame_intervals(vid_frame->frm_rate_l); + } + g_free(vid_frame); + } +} + +void video_free_formats(GList **fmt_l) +{ + GList *l; + struct video_format *vid_fmt; + + for (l = *fmt_l; l != NULL; l = l->next) { + vid_fmt = l->data; + if (vid_fmt->vid_fmt_frm_l) { + video_free_frame_sizes(vid_fmt->vid_fmt_frm_l); + } + + g_free(vid_fmt); + } +} + + +static GByteArray *iterate_frame_rate_list(GByteArray *resp, GList *frm_rate_l) +{ + struct video_format_frame_rates *vid_fmt_frm_rate; + + /* iterate frame_rate list */ + for (; frm_rate_l != NULL; frm_rate_l = frm_rate_l->next) { + vid_fmt_frm_rate = frm_rate_l->data; + + resp = g_byte_array_append(resp, + (guint8 *) &vid_fmt_frm_rate->frame_rates, + sizeof(struct virtio_video_format_range)); + } + return resp; +} + +static GByteArray *iterate_format_frame_list(GByteArray *resp, GList *fmt_frm_l) +{ + struct video_format_frame *vid_fmt_frm; + GList *frm_rate_l = NULL; + + /* iterate format_frame list */ + for (; fmt_frm_l != NULL; fmt_frm_l = fmt_frm_l->next) { + vid_fmt_frm = fmt_frm_l->data; + + if (!vid_fmt_frm->frm_rate_l) { + vid_fmt_frm->frame.num_rates = htole32(0); + } else { + frm_rate_l = vid_fmt_frm->frm_rate_l; + vid_fmt_frm->frame.num_rates = htole32(g_list_length(frm_rate_l)); + } + + g_debug("%s: num_rates(%d)", __func__, + le32toh(vid_fmt_frm->frame.num_rates)); + + resp = g_byte_array_append(resp, + (guint8 *) &vid_fmt_frm->frame, + sizeof(struct virtio_video_format_frame)); + + if (frm_rate_l) { + resp = iterate_frame_rate_list(resp, frm_rate_l); + } + } + + return resp; +} + +static GByteArray *iterate_format_desc_list(GByteArray *resp, GList *fmt_desc_l) +{ + struct video_format *vid_fmt; + GList *fmt_frm_l = NULL; + + for (; fmt_desc_l != NULL; fmt_desc_l = fmt_desc_l->next) { + vid_fmt = fmt_desc_l->data; + + /* does video_format have a list of format_frame? */ + if (!vid_fmt->vid_fmt_frm_l) { + vid_fmt->desc.num_frames = htole32(0); + } else { + fmt_frm_l = vid_fmt->vid_fmt_frm_l; + vid_fmt->desc.num_frames = htole32(g_list_length(fmt_frm_l)); + } + + g_debug("%s: num_frames(%d)", __func__, + le32toh(vid_fmt->desc.num_frames)); + + resp = g_byte_array_append(resp, + (guint8 *) &vid_fmt->desc, + sizeof(struct virtio_video_format_desc)); + + if (fmt_frm_l) { + resp = iterate_format_frame_list(resp, fmt_frm_l); + } + } + + return resp; +} + +GByteArray *create_query_cap_resp(struct virtio_video_query_capability *qcmd, + GList **fmt_l, GByteArray *resp) +{ + GList *fmt_desc_l; + struct virtio_video_query_capability_resp cap_resp; + + fmt_desc_l = *fmt_l; + + cap_resp.hdr.type = VIRTIO_VIDEO_RESP_OK_QUERY_CAPABILITY; + cap_resp.hdr.stream_id = qcmd->hdr.stream_id; + cap_resp.num_descs = htole32(g_list_length(fmt_desc_l)); + + assert(le32toh(cap_resp.num_descs) < MAX_FMT_DESCS); + + resp = g_byte_array_append(resp, (guint8 *) &cap_resp, sizeof(cap_resp)); + resp = iterate_format_desc_list(resp, fmt_desc_l); + + return resp; +} + +/* timestamp in nsecs */ +void convert_to_timeval(uint64_t timestamp, struct timeval *t) +{ + uint64_t f_nsecs; + + uint64_t nsecs; + + /* convert to seconds */ + t->tv_sec = timestamp / 1000000000; + + /* deal with fraction of a second */ + f_nsecs = t->tv_sec * 1000000000; + t->tv_usec = (timestamp - f_nsecs) / 1000; + + /* sanity check above conversion */ + nsecs = t->tv_sec * 1000000000; + nsecs += (t->tv_usec * 1000); + + if (timestamp != nsecs) { + g_critical("%s: timestamp != nsecs", __func__); + } +} + +void v4l2_backend_free(struct v4l2_device *dev) +{ + if (dev && dev->opened) { + close(dev->fd); + } + g_free(dev); +} + +struct v4l2_device *v4l2_backend_init(const gchar *devname) +{ + struct v4l2_device *dev; + int ret = 0; + GList *vid_output_fmt_l = NULL; + GList *vid_capture_fmt_l = NULL; + enum v4l2_buf_type buf_type; + + if (!devname) { + return NULL; + } + + dev = g_malloc0(sizeof(struct v4l2_device)); + + /* open the device */ + dev->fd = v4l2_open(devname); + if (dev->fd < 0) { + g_printerr("v4l2_open() failed!\n"); + goto err; + } + + dev->opened = 1; + dev->devname = devname; + + ret = video_querycap(dev); + + buf_type = dev->has_mplane ? V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE + : V4L2_BUF_TYPE_VIDEO_OUTPUT; + + /* enumerate coded formats on OUTPUT */ + ret = video_enum_formats(dev, buf_type, + &vid_output_fmt_l, true); + if (ret < 0) { + g_printerr("video_enum_formats() failed OUTPUT\n"); + goto err; + } + + buf_type = dev->has_mplane ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE + : V4L2_BUF_TYPE_VIDEO_CAPTURE; + + /* enumerate coded formats on CAPTURE */ + ret = video_enum_formats(dev, buf_type, + &vid_capture_fmt_l, true); + if (ret < 0) { + g_printerr("video_enum_formats() failed CAPTURE\n"); + goto err2; + } + + if (dev->dev_type & STATEFUL_ENCODER) + g_print("%s: %s is a stateful encoder (0x%x)!\n", __func__, + devname, dev->dev_type); + + if (dev->dev_type & STATEFUL_DECODER) + g_print("%s: %s is a stateful decoder (0x%x)!\n", __func__, + devname, dev->dev_type); + + video_free_formats(&vid_output_fmt_l); + video_free_formats(&vid_capture_fmt_l); + + if (!(dev->dev_type & STATEFUL_ENCODER || + dev->dev_type & STATEFUL_DECODER)) { + g_printerr("v4l2 device not supported! v4l2 backend only supports " + "stateful codec devices currently(%d)!\n", dev->dev_type); + goto err3; + } + + g_debug("%s: success!\n", __func__); + return dev; + +err3: + video_free_formats(&vid_capture_fmt_l); +err2: + video_free_formats(&vid_output_fmt_l); +err: + v4l2_backend_free(dev); + return NULL; +} + +int v4l2_ioctl_query_control(int fd, uint32_t control, int32_t *value) +{ + int ret = 0; + struct v4l2_queryctrl ctrl; + + g_debug("%s:%d", __func__, __LINE__); + + ctrl.id = control; + + ret = ioctl(fd, VIDIOC_QUERYCTRL, &ctrl); + if (ret < 0) { + g_printerr("Unable to query control: %s (%d)\n", + g_strerror(errno), errno); + return ret; + } + + *value = ctrl.type; + g_debug("%s: ctrl=0x%x type=0x%x", __func__, control, *value); + + return ret; +} + +int v4l2_ioctl_get_control(int fd , uint32_t control, int32_t *value) +{ + int ret = 0; + struct v4l2_control ctrl; + + g_debug("%s:%d", __func__, __LINE__); + + ctrl.id = control; + + ret = ioctl(fd, VIDIOC_G_CTRL, &ctrl); + if (ret < 0) { + g_printerr("Unable to get control: %s (%d)\n", + g_strerror(errno), errno); + return ret; + } + + *value = ctrl.value; + g_debug("%s: ctrl=0x%x value=0x%x", __func__, control, *value); + + return ret; +} + +int v4l2_ioctl_reqbuf(int fd, enum v4l2_buf_type type, + enum v4l2_memory memory, int *count) +{ + int ret; + struct v4l2_requestbuffers reqbuf; + + memset(&reqbuf, 0, sizeof(reqbuf)); + reqbuf.type = type; + reqbuf.memory = memory; + reqbuf.count = *count; + + ret = ioctl(fd, VIDIOC_REQBUFS, &reqbuf); + if (ret == -1) { + if (errno == EBUSY) { + g_critical("%s: EBUSY: buffers for %s still mapped or exported!\n", + __func__, v4l2_buf_type_name(type)); + } else { + g_printerr("VIDIOC_REQBUFS failed: %s (%d)\n", + g_strerror(errno), errno); + } + return ret; + } + + *count = reqbuf.count; + + g_debug("%s: VIDIOC_REQBUFS capabilities(0x%x) granted(%d)", + __func__, reqbuf.capabilities, reqbuf.count); + + return ret; +} + +int v4l2_ioctl_subscribe_event(int fd, uint32_t event_type, uint32_t id) +{ + int ret = 0; + struct v4l2_event_subscription sub; + + memset(&sub, 0, sizeof(sub)); + sub.type = event_type; + sub.id = 0; + + if (event_type == V4L2_EVENT_SOURCE_CHANGE) { + sub.id = id; + } + + ret = ioctl(fd, VIDIOC_SUBSCRIBE_EVENT, &sub); + if (ret < 0) { + g_printerr("VIDIOC_SUBSCRIBE_EVENT failed: %s (%d)\n", + g_strerror(errno), errno); + return ret; + } + + g_debug("%s event(0x%x) OK", __func__, event_type); + + return ret; +} + +int v4l2_issue_cmd(int fd, uint32_t cmd, uint32_t flags) +{ + int ret = 0; + struct v4l2_decoder_cmd decoder_cmd; + memset(&decoder_cmd, 0, sizeof(struct v4l2_decoder_cmd)); + + decoder_cmd.cmd = cmd; + decoder_cmd.flags = flags; + /* Normal speed */ + decoder_cmd.start.speed = 1000; + + ret = ioctl(fd, VIDIOC_DECODER_CMD, &decoder_cmd); + if (ret < 0) { + g_printerr("VIDIOC_DECODER_CMD(%d) failed fd=(%d): %s: (%d)\n", + cmd, fd, g_strerror(errno), errno); + return ret; + } + + g_debug("%s: VIDIOC_DECODER_CMD(%d) fd(%d) OK\n", __func__, cmd, fd); + + return ret; +} diff --git a/tools/vhost-user-video/v4l2_backend.h b/tools/vhost-user-video/v4l2_backend.h new file mode 100644 index 0000000000..ef9a5f0337 --- /dev/null +++ b/tools/vhost-user-video/v4l2_backend.h @@ -0,0 +1,104 @@ +/* + * Virtio vhost-user VIDEO Device + * + * Copyright Red Hat, Inc. 2023 + * Copyright Linaro 2021 + * + * Authors: + * Peter Griffin + * Albert Esteve + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef V4L2_BACKEND_H +#define V4L2_BACKEND_H + +#include "standard-headers/linux/virtio_video.h" +#include "virtio_video_helpers.h" + +#define MAX_CAPS_LEN 4096 +#define MAX_FMT_DESCS 64 + +#define STATEFUL_ENCODER (1 << 0) +#define STATEFUL_DECODER (1 << 1) +#define STATELESS_ENCODER (1 << 2) +#define STATELESS_DECODER (1 << 3) + +#define V4L2_TYPE_IS_SINGLEPLANAR(type) \ + ((type) == V4L2_BUF_TYPE_VIDEO_CAPTURE \ + || (type) == V4L2_BUF_TYPE_VIDEO_OUTPUT) + +/* Function protoypes */ +GByteArray *create_query_cap_resp(struct virtio_video_query_capability *qcmd, + GList **fmt_l, GByteArray *querycapresp); + +/* video to v4l2 interactions */ +int video_resource_create(struct stream *s, + uint32_t queue_type, uint32_t queue_len); +int video_send_decoder_start_cmd(struct v4l2_device *dev); +void video_free_frame_intervals(GList *frm_intervals_l); +void video_free_frame_sizes(GList *frm_sz_l); +int video_enum_formats(struct v4l2_device *dev, enum v4l2_buf_type type, + GList **p_fmt_list, bool only_enum_fmt); +void video_free_formats(GList **fmt_l); +int video_streamon(struct stream *s, struct v4l2_device *dev, + enum v4l2_buf_type type); +int video_streamoff(struct stream *s, enum v4l2_buf_type type); +int video_free_buffers(int fd, enum v4l2_buf_type type, + enum v4l2_memory memory); + +struct v4l2_device *v4l2_backend_init(const gchar *devname); +void v4l2_backend_free(struct v4l2_device *dev); + +/* v4l2 wrappers */ +enum v4l2_buf_type +get_v4l2_buf_type(enum virtio_video_queue_type queue_type, bool has_mplane); +enum v4l2_memory get_v4l2_memory(enum virtio_video_mem_type mem_type); +enum virtio_video_mem_type +get_queue_mem_type(struct stream *s, + enum virtio_video_queue_type queue_type); + +void v4l2_set_device_type(struct v4l2_device *dev, enum v4l2_buf_type type, + struct v4l2_fmtdesc *fmt_desc); +int v4l2_video_get_format(int fd, enum v4l2_buf_type type, + struct v4l2_format *fmt); +int v4l2_video_set_format(int fd, enum v4l2_buf_type type, + struct virtio_video_params *p); +int v4l2_set_pixel_format(int fd, enum v4l2_buf_type buf_type, + uint32_t pixelformat); + +int v4l2_queue_buffer(enum v4l2_buf_type type, + enum v4l2_memory memory, + struct virtio_video_resource_queue *qcmd, + struct resource *res, struct stream *s, + struct v4l2_device *dev); +int v4l2_dequeue_buffer(int fd, enum v4l2_buf_type type, + enum v4l2_memory memory, + struct stream *s); + +int v4l2_video_get_param(int fd, enum v4l2_buf_type type, + struct v4l2_streamparm *param); + +int v4l2_video_get_selection(int fd, enum v4l2_buf_type type, + struct v4l2_selection *sel); +int v4l2_video_set_selection(int fd, enum v4l2_buf_type type, + struct v4l2_selection *sel); + +int v4l2_streamon(struct stream *s, enum v4l2_buf_type type); +int v4l2_streamoff(struct stream *s, enum v4l2_buf_type type); + +int v4l2_open(const gchar *devname); +int v4l2_close(int fd); + +/* ioctl wrappers */ +int v4l2_ioctl_query_control(int fd, uint32_t control, int32_t *value); +int v4l2_ioctl_get_control(int fd, uint32_t control, int32_t *value); +int v4l2_ioctl_reqbuf(int fd, enum v4l2_buf_type type, + enum v4l2_memory memory, int *count); +int v4l2_ioctl_subscribe_event(int fd, uint32_t event_type, uint32_t id); + +int v4l2_issue_cmd(int fd, uint32_t cmd, uint32_t flags); + +#endif diff --git a/tools/vhost-user-video/vhost-user-video.c b/tools/vhost-user-video/vhost-user-video.c new file mode 100644 index 0000000000..cee81c69c3 --- /dev/null +++ b/tools/vhost-user-video/vhost-user-video.c @@ -0,0 +1,1779 @@ +/* + * VIRTIO Video Emulation via vhost-user + * + * Copyright (c) 2023 Red Hat, Inc. + * Copyright (c) 2021 Linaro Ltd + * + * Authors: + * Peter Griffin + * Albert Esteve + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#define G_LOG_DOMAIN "vhost-user-video" +#define G_LOG_USE_STRUCTURED 1 + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "libvhost-user-glib.h" +#include "libvhost-user.h" +#include "standard-headers/linux/virtio_video.h" + +#include "qemu/compiler.h" +#include "qemu/iov.h" + +#include "vuvideo.h" +#include "v4l2_backend.h" +#include "virtio_video_helpers.h" + +#ifndef container_of +#define container_of(ptr, type, member) ({ \ + const typeof(((type *) 0)->member) * __mptr = (ptr); \ + (type *) ((char *) __mptr - offsetof(type, member)); }) +#endif + +static gchar *socket_path; +static gchar *v4l2_path; +static gint socket_fd = -1; +static gboolean print_cap; +static gboolean verbose; +static gboolean debug; + +static GOptionEntry options[] = { + { "socket-path", 0, 0, G_OPTION_ARG_FILENAME, &socket_path, + "Location of vhost-user Unix domain socket, " + "incompatible with --fd", "PATH" }, + { "v4l2-device", 0, 0, G_OPTION_ARG_FILENAME, &v4l2_path, + "Location of v4l2 device node", "PATH" }, + { "fd", 0, 0, G_OPTION_ARG_INT, &socket_fd, + "Specify the fd of the backend, " + "incompatible with --socket-path", "FD" }, + { "print-capabilities", 0, 0, G_OPTION_ARG_NONE, &print_cap, + "Output to stdout the backend capabilities " + "in JSON format and exit", NULL}, + { "verbose", 'v', 0, G_OPTION_ARG_NONE, &verbose, + "Be more verbose in output", NULL}, + { "debug", 0, 0, G_OPTION_ARG_NONE, &debug, + "Include debug output", NULL}, + { NULL } +}; + +enum { + VHOST_USER_VIDEO_MAX_QUEUES = 2, +}; + +static const char * +vv_cmd_to_string(int cmd) +{ +#define CMD(cmd) [cmd] = #cmd + static const char *vg_cmd_str[] = { + /* Command */ + CMD(VIRTIO_VIDEO_CMD_QUERY_CAPABILITY), + CMD(VIRTIO_VIDEO_CMD_STREAM_CREATE), + CMD(VIRTIO_VIDEO_CMD_STREAM_DESTROY), + CMD(VIRTIO_VIDEO_CMD_STREAM_DRAIN), + CMD(VIRTIO_VIDEO_CMD_RESOURCE_CREATE), + CMD(VIRTIO_VIDEO_CMD_RESOURCE_QUEUE), + CMD(VIRTIO_VIDEO_CMD_RESOURCE_DESTROY_ALL), + CMD(VIRTIO_VIDEO_CMD_QUEUE_CLEAR), + CMD(VIRTIO_VIDEO_CMD_QUERY_CONTROL), + CMD(VIRTIO_VIDEO_CMD_GET_CONTROL), + CMD(VIRTIO_VIDEO_CMD_SET_CONTROL), + CMD(VIRTIO_VIDEO_CMD_GET_PARAMS_EXT), + CMD(VIRTIO_VIDEO_CMD_SET_PARAMS_EXT), + }; +#undef CMD + + if (cmd >= 0 && cmd < G_N_ELEMENTS(vg_cmd_str)) { + return vg_cmd_str[cmd]; + } else { + return "unknown"; + } +} + +static void video_panic(VuDev *dev, const char *msg) +{ + g_critical("%s\n", msg); + exit(EXIT_FAILURE); +} + +static uint64_t video_get_features(VuDev *dev) +{ + g_info("%s: replying", __func__); + return 0; +} + +static void video_set_features(VuDev *dev, uint64_t features) +{ + if (features) { + g_autoptr(GString) s = g_string_new("Requested un-handled feature"); + g_string_append_printf(s, " 0x%" PRIx64 "", features); + g_info("%s: %s", __func__, s->str); + } +} + +/* + * The configuration of the device is static and set when we start the + * daemon. + */ +static int +video_get_config(VuDev *dev, uint8_t *config, uint32_t len) +{ + VuVideo *v = container_of(dev, VuVideo, dev.parent); + + g_return_val_if_fail(len <= sizeof(struct virtio_video_config), -1); + v->virtio_config.version = 0; + v->virtio_config.max_caps_length = MAX_CAPS_LEN; + v->virtio_config.max_resp_length = MAX_CAPS_LEN; + + memcpy(config, &v->virtio_config, len); + + g_debug("%s: config.max_caps_length = %d", __func__ + , ((struct virtio_video_config *)config)->max_caps_length); + g_debug("%s: config.max_resp_length = %d", __func__ + , ((struct virtio_video_config *)config)->max_resp_length); + + return 0; +} + +static int +video_set_config(VuDev *dev, const uint8_t *data, + uint32_t offset, uint32_t size, + uint32_t flags) +{ + g_debug("%s: ", __func__); + /* + * set_config is required to set the F_CONFIG feature, + * but we can just ignore the call + */ + return 0; +} + +/* + * Handlers for individual control messages + */ + +static void +handle_set_params_cmd(struct VuVideo *v, struct vu_video_ctrl_command *vio_cmd) +{ + int ret = 0; + enum v4l2_buf_type buf_type; + struct virtio_video_set_params *cmd = + (struct virtio_video_set_params *) vio_cmd->cmd_buf; + struct stream *s; + + g_debug("%s: type(0x%x) resource_type(%d) stream_id(%d) %s ", + __func__, cmd->hdr.type, + le32toh(cmd->params.resource_type), cmd->hdr.stream_id, + vio_queue_name(le32toh(cmd->params.queue_type))); + g_debug("%s: format=0x%x frame_width(%d) frame_height(%d)", + __func__, le32toh(cmd->params.format), + le32toh(cmd->params.frame_width), + le32toh(cmd->params.frame_height)); + g_debug("%s: min_buffers(%d) max_buffers(%d)", __func__, + le32toh(cmd->params.min_buffers), le32toh(cmd->params.max_buffers)); + g_debug("%s: frame_rate(%d) num_planes(%d)", __func__, + le32toh(cmd->params.frame_rate), le32toh(cmd->params.num_planes)); + g_debug("%s: crop top=%d, left=%d, width=%d, height=%d", __func__, + le32toh(cmd->params.crop.left), le32toh(cmd->params.crop.top), + le32toh(cmd->params.crop.width), le32toh(cmd->params.crop.height)); + + s = find_stream(v, cmd->hdr.stream_id); + if (!s) { + g_critical("%s: stream_id(%d) not found", __func__, cmd->hdr.stream_id); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + return; + } + + g_mutex_lock(&s->mutex); + + buf_type = get_v4l2_buf_type(le32toh(cmd->params.queue_type), + s->has_mplane); + + ret = v4l2_video_set_format(s->fd, buf_type, &cmd->params); + if (ret < 0) { + g_error("%s: v4l2_video_set_format() failed", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + + if (V4L2_TYPE_IS_CAPTURE(buf_type)) { + /* decoder supports composing on CAPTURE */ + struct v4l2_selection sel; + memset(&sel, 0, sizeof(struct v4l2_selection)); + + sel.r.left = le32toh(cmd->params.crop.left); + sel.r.top = le32toh(cmd->params.crop.top); + sel.r.width = le32toh(cmd->params.crop.width); + sel.r.height = le32toh(cmd->params.crop.height); + + ret = v4l2_video_set_selection(s->fd, buf_type, &sel); + if (ret < 0) { + g_printerr("%s: v4l2_video_set_selection failed: %s (%d).\n" + , __func__, g_strerror(errno), errno); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + } + + cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + +out_unlock: + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); + g_mutex_unlock(&s->mutex); + return; +} + +static void +handle_get_params_cmd(struct VuVideo *v, struct vu_video_ctrl_command *vio_cmd) +{ + int ret; + struct v4l2_format fmt; + struct v4l2_selection sel; + enum v4l2_buf_type buf_type; + struct virtio_video_get_params *cmd = + (struct virtio_video_get_params *) vio_cmd->cmd_buf; + struct virtio_video_get_params_resp getparams_reply; + struct stream *s; + + g_debug("%s: type(0x%x) stream_id(%d) %s", __func__, + cmd->hdr.type, cmd->hdr.stream_id, + vio_queue_name(le32toh(cmd->queue_type))); + + s = find_stream(v, cmd->hdr.stream_id); + if (!s) { + g_critical("%s: stream_id(%d) not found\n" + , __func__, cmd->hdr.stream_id); + getparams_reply.hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + return; + } + + g_mutex_lock(&s->mutex); + + getparams_reply.hdr.stream_id = cmd->hdr.stream_id; + getparams_reply.params.queue_type = cmd->queue_type; + + buf_type = get_v4l2_buf_type(cmd->queue_type, s->has_mplane); + + ret = v4l2_video_get_format(s->fd, buf_type, &fmt); + if (ret < 0) { + g_printerr("v4l2_video_get_format failed\n"); + getparams_reply.hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + + if (V4L2_TYPE_IS_CAPTURE(buf_type)) { + ret = v4l2_video_get_selection(s->fd, buf_type, &sel); + if (ret < 0) { + g_printerr("v4l2_video_get_selection failed\n"); + getparams_reply.hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + } + + /* convert from v4l2 to virtio */ + v4l2_to_virtio_video_params(v->v4l2_dev, &fmt, &sel, + &getparams_reply); + + getparams_reply.hdr.type = VIRTIO_VIDEO_RESP_OK_GET_PARAMS; + +out_unlock: + vio_cmd->finished = true; + send_ctrl_response(vio_cmd, (uint8_t *)&getparams_reply, + sizeof(struct virtio_video_get_params_resp)); + g_mutex_unlock(&s->mutex); +} + +struct stream *find_stream(struct VuVideo *v, uint32_t stream_id) +{ + GList *l; + struct stream *s; + + for (l = v->streams; l != NULL; l = l->next) { + s = (struct stream *)l->data; + if (s->stream_id == stream_id) { + return s; + } + } + + return NULL; +} + +int add_resource(struct stream *s, struct resource *r, uint32_t queue_type) +{ + + if (!s || !r) { + return -EINVAL; + } + + switch (queue_type) { + case VIRTIO_VIDEO_QUEUE_TYPE_INPUT: + s->inputq_resources = g_list_append(s->inputq_resources, r); + break; + + case VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT: + s->outputq_resources = g_list_append(s->outputq_resources, r); + break; + default: + return -EINVAL; + } + + return 0; +} + +void free_resource_mem(struct resource *r) +{ + + /* + * Frees the memory allocated for resource_queue_cmd + * not the memory allocated in resource_create + */ + + if (r->vio_q_cmd) { + g_free(r->vio_q_cmd->cmd_buf); + r->vio_q_cmd->cmd_buf = NULL; + free(r->vio_q_cmd); + r->vio_q_cmd = NULL; + } +} + +void remove_all_resources(struct stream *s, uint32_t queue_type) +{ + GList **resource_list; + struct resource *r; + + /* assumes stream mutex is held by caller */ + + switch (queue_type) { + case VIRTIO_VIDEO_QUEUE_TYPE_INPUT: + resource_list = &s->inputq_resources; + break; + case VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT: + resource_list = &s->outputq_resources; + break; + default: + g_critical("%s: Invalid virtio queue!", __func__); + return; + } + + g_debug("%s: resource_list has %d elements", __func__ + , g_list_length(*resource_list)); + + GList *l = *resource_list; + while (l != NULL) { + GList *next = l->next; + r = (struct resource *)l->data; + if (r) { + g_debug("%s: Removing resource_id(%d) resource=%p" + , __func__, r->vio_resource.resource_id, r); + + /* + * Assumes that either QUEUE_CLEAR or normal dequeuing + * of buffers will have freed resource_queue cmd memory + */ + + /* free resource memory allocated in resource_create() */ + g_free(r->iov); + g_free(r); + *resource_list = g_list_delete_link(*resource_list, l); + } + l = next; + } +} + +struct resource *find_resource(struct stream *s, uint32_t resource_id, + uint32_t queue_type) +{ + GList *l; + struct resource *r; + + switch (queue_type) { + case VIRTIO_VIDEO_QUEUE_TYPE_INPUT: + l = s->inputq_resources; + break; + + case VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT: + l = s->outputq_resources; + break; + default: + g_error("%s: Invalid queue type!", __func__); + return NULL; + } + + for (; l != NULL; l = l->next) { + r = (struct resource *)l->data; + if (r->vio_resource.resource_id == resource_id) { + return r; + } + } + + return NULL; +} + +struct resource *find_resource_by_v4l2index(struct stream *s, + enum v4l2_buf_type buf_type, + uint32_t v4l2_index) +{ + GList *l; + struct resource *r; + + switch (buf_type) { + case V4L2_BUF_TYPE_VIDEO_CAPTURE: + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: + l = s->outputq_resources; + break; + + case V4L2_BUF_TYPE_VIDEO_OUTPUT: + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: + l = s->inputq_resources; + break; + + default: + g_error("Unsupported buffer type\n"); + return NULL; + } + + for (; l != NULL; l = l->next) { + r = (struct resource *)l->data; + if (r->v4l2_index == v4l2_index) { + g_debug("%s: found Resource=%p streamid(%d) resourceid(%d) " + "numplanes(%d) planes_layout(0x%x) vio_q_cmd=%p", __func__, + r, r->stream_id, r->vio_resource.resource_id, + r->vio_resource.num_planes, r->vio_resource.planes_layout, + r->vio_q_cmd); + return r; + } + } + return NULL; +} + +#define EVENT_WQ_IDX 1 + +static void *stream_worker_thread(gpointer data) +{ + int ret; + struct stream *s = data; + VuVideo *v = s->video; + VugDev *vugdev = &v->dev; + VuDev *vudev = &vugdev->parent; + VuVirtq *vq = vu_get_queue(vudev, EVENT_WQ_IDX); + VuVirtqElement *elem; + size_t len; + + struct v4l2_event ev; + struct virtio_video_event vio_event; + + /* select vars */ + fd_set efds, rfds, wfds; + bool have_event, have_read, have_write; + enum v4l2_buf_type buf_type; + + fcntl(s->fd, F_SETFL, fcntl(s->fd, F_GETFL) | O_NONBLOCK); + + while (true) { + int res; + + g_mutex_lock(&s->mutex); + + g_debug("Stream: id %d state %d", s->stream_id, s->stream_state); + /* wait for STREAMING or DESTROYING state */ + while (s->stream_state != STREAM_DESTROYING && + s->stream_state != STREAM_STREAMING && + s->stream_state != STREAM_DRAINING) + g_cond_wait(&s->stream_cond, &s->mutex); + + if (s->stream_state == STREAM_DESTROYING) { + g_debug("stream worker thread exiting!"); + s->stream_state = STREAM_DESTROYED; + g_cond_signal(&s->stream_cond); + g_mutex_unlock(&s->mutex); + g_thread_exit(0); + } + + g_mutex_unlock(&s->mutex); + + FD_ZERO(&efds); + FD_SET(s->fd, &efds); + FD_ZERO(&rfds); + FD_SET(s->fd, &rfds); + FD_ZERO(&wfds); + FD_SET(s->fd, &wfds); + + struct timeval tv = { 0 , 500000 }; + res = select(s->fd + 1, &rfds, &wfds, &efds, &tv); + if (res < 0) { + g_printerr("%s:%d - select() failed: %s (%d)\n", + __func__, __LINE__, g_strerror(errno), errno); + break; + } + + if (res == 0) { + g_debug("%s:%d - select() timeout", __func__, __LINE__); + continue; + } + + have_event = FD_ISSET(s->fd, &efds); + have_read = FD_ISSET(s->fd, &rfds); + have_write = FD_ISSET(s->fd, &wfds); + /* read is capture queue, write is output queue */ + + g_debug("%s:%d have_event=%d, have_write=%d, have_read=%d\n", + __func__, __LINE__, FD_ISSET(s->fd, &efds), + FD_ISSET(s->fd, &wfds), FD_ISSET(s->fd, &rfds)); + + g_mutex_lock(&s->mutex); + + if (have_event) { + g_debug("%s: have_event!", __func__); + res = ioctl(s->fd, VIDIOC_DQEVENT, &ev); + if (res < 0) { + g_printerr("%s:%d - VIDIOC_DQEVENT failed: %s (%d)\n", + __func__, __LINE__, g_strerror(errno), errno); + break; + } + v4l2_to_virtio_event(&ev, &vio_event); + + /* get event workqueue */ + elem = vu_queue_pop(vudev, vq, sizeof(struct VuVirtqElement)); + if (!elem) { + g_debug("%s:%d\n", __func__, __LINE__); + break; + } + + len = iov_from_buf_full(elem->in_sg, + elem->in_num, 0, (void *) &vio_event, + sizeof(struct virtio_video_event)); + + if (vio_event.event_type) { + vu_queue_push(vudev, vq, elem, len); + vu_queue_notify(vudev, vq); + } + } + + if (have_read && s->capture_streaming) { + /* TODO assumes decoder */ + buf_type = s->has_mplane ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE + : V4L2_BUF_TYPE_VIDEO_CAPTURE; + + enum virtio_video_mem_type mem_type = + get_queue_mem_type(s, VIRTIO_VIDEO_QUEUE_TYPE_INPUT); + enum v4l2_memory memory = get_v4l2_memory(mem_type); + + ret = v4l2_dequeue_buffer(s->fd, buf_type, memory, s); + if (ret < 0) { + g_info("%s: v4l2_dequeue_buffer() failed CAPTURE ret(%d)", + __func__, ret); + + if (ret == -EPIPE) { + g_debug("Dequeued last buffer, stop streaming."); + /* dequeued last buf, so stop streaming */ + v4l2_streamoff(s, buf_type); + } + } + } + + if (have_write && s->output_streaming) { + buf_type = s->has_mplane ? V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE + : V4L2_BUF_TYPE_VIDEO_OUTPUT; + + enum virtio_video_mem_type mem_type = + get_queue_mem_type(s, VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT); + enum v4l2_memory memory = get_v4l2_memory(mem_type); + + ret = v4l2_dequeue_buffer(s->fd, buf_type, memory, s); + if (ret < 0) { + g_info("%s: v4l2_dequeue_buffer() failed OUTPUT ret(%d)", + __func__, ret); + } + } + + g_mutex_unlock(&s->mutex); + } + + return NULL; +} + +void handle_queue_clear_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd) +{ + struct virtio_video_queue_clear *cmd = + (struct virtio_video_queue_clear *)vio_cmd->cmd_buf; + int ret = 0; + struct stream *s; + uint32_t stream_id = le32toh(cmd->hdr.stream_id); + enum virtio_video_queue_type queue_type = le32toh(cmd->queue_type); + GList *res_list = NULL; + + g_debug("%s: stream_id(%d) %s\n", __func__, stream_id, + vio_queue_name(queue_type)); + + if (!v || !cmd) { + return; + } + + s = find_stream(v, stream_id); + if (!s) { + g_critical("%s: stream_id(%d) not found", __func__, stream_id); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + return; + } + + g_mutex_lock(&s->mutex); + + enum v4l2_buf_type buf_type = + get_v4l2_buf_type(le32toh(cmd->queue_type), s->has_mplane); + + /* + * QUEUE_CLEAR behaviour from virtio-video spec + * Return already queued buffers back from the input or the output queue + * of the device. The device SHOULD return all of the buffers from the + * respective queue as soon as possible without pushing the buffers through + * the processing pipeline. + * + * From v4l2 PoV we issue a VIDIOC_STREAMOFF on the queue which will abort + * or finish any DMA in progress, unlocks any user pointer buffers locked + * in physical memory, and it removes all buffers from the incoming and + * outgoing queues. + */ + + /* issue streamoff */ + ret = v4l2_streamoff(s, buf_type); + if (ret < 0) { + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + + /* iterate the queues resources list - and send a reply to each one */ + + /* + * If the processing was stopped due to VIRTIO_VIDEO_CMD_QUEUE_CLEAR, + * the device MUST respond with VIRTIO_VIDEO_RESP_OK_NODATA as a response + * type and VIRTIO_- VIDEO_BUFFER_FLAG_ERR in flags. + */ + + res_list = get_resource_list(s, queue_type); + g_list_foreach(res_list, (GFunc)send_qclear_res_reply, s); + cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + +out_unlock: + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); + g_mutex_unlock(&s->mutex); +} + +GList *get_resource_list(struct stream *s, uint32_t queue_type) +{ + switch (queue_type) { + case VIRTIO_VIDEO_QUEUE_TYPE_INPUT: + return s->inputq_resources; + break; + + case VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT: + return s->outputq_resources; + break; + default: + g_critical("%s: Unknown queue type!", __func__); + return NULL; + } +} + +void send_ctrl_response(struct vu_video_ctrl_command *vio_cmd, + uint8_t *resp, size_t resp_len) +{ + size_t len; + + virtio_video_ctrl_hdr_htole((struct virtio_video_cmd_hdr *)resp); + + /* send virtio_video_resource_queue_resp */ + len = iov_from_buf_full(vio_cmd->elem.in_sg, + vio_cmd->elem.in_num, 0, resp, resp_len); + + if (len != resp_len) { + g_critical("%s: response size incorrect %zu vs %zu", + __func__, len, resp_len); + } + + vu_queue_push(vio_cmd->dev, vio_cmd->vq, &vio_cmd->elem, len); + vu_queue_notify(vio_cmd->dev, vio_cmd->vq); + + if (vio_cmd->finished) { + g_free(vio_cmd->cmd_buf); + free(vio_cmd); + } +} + +void send_ctrl_response_nodata(struct vu_video_ctrl_command *vio_cmd) +{ + send_ctrl_response(vio_cmd, vio_cmd->cmd_buf, + sizeof(struct virtio_video_cmd_hdr)); +} + +void send_qclear_res_reply(gpointer data, gpointer user_data) +{ + struct resource *r = data; + if (!r->queued) { + /* + * only need to send replies for buffers that are + * inflight + */ + return; + } + + struct vu_video_ctrl_command *vio_cmd = r->vio_q_cmd; + struct virtio_video_queue_clear *cmd = + (struct virtio_video_queue_clear *) vio_cmd->cmd_buf; + struct virtio_video_resource_queue_resp resp; + + + + resp.hdr.stream_id = cmd->hdr.stream_id; + resp.hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + resp.flags = htole32(VIRTIO_VIDEO_BUFFER_FLAG_ERR); + resp.timestamp = htole64(r->vio_res_q.timestamp); + + g_debug("%s: stream_id=%d type=0x%x flags=0x%x resource_id=%d t=%llx" + , __func__, resp.hdr.stream_id, resp.hdr.type, resp.flags, + r->vio_resource.resource_id, resp.timestamp); + + vio_cmd->finished = true; + send_ctrl_response(vio_cmd, (uint8_t *) &resp, + sizeof(struct virtio_video_resource_queue_resp)); +} + +static int +handle_resource_create_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd) +{ + int ret = 0, i; + uint32_t total_entries = 0; + uint32_t stream_id; + struct virtio_video_resource_create *cmd = + (struct virtio_video_resource_create *)vio_cmd->cmd_buf; + struct resource *res; + struct virtio_video_resource_create *r; + struct stream *s; + enum virtio_video_mem_type mem_type; + + stream_id = cmd->hdr.stream_id; + + s = find_stream(v, stream_id); + if (!s) { + g_critical("%s: stream_id(%d) not found", __func__, stream_id); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + return ret; + } + + g_mutex_lock(&s->mutex); + + if (le32toh(cmd->resource_id) == 0) { + g_critical("%s: resource id 0 is not allowed", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + + /* check resource id doesn't already exist */ + res = find_resource(s, le32toh(cmd->resource_id), le32toh(cmd->queue_type)); + if (res) { + g_critical("%s: resource_id:%d already exists" + , __func__, le32toh(cmd->resource_id)); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_RESOURCE_ID; + goto out_unlock; + } else { + res = g_new0(struct resource, 1); + res->vio_resource.resource_id = le32toh(cmd->resource_id); + res->vio_resource.queue_type = le32toh(cmd->queue_type); + res->vio_resource.planes_layout = le32toh(cmd->planes_layout); + + res->vio_resource.num_planes = le32toh(cmd->num_planes); + r = &res->vio_resource; + + switch (le32toh(cmd->queue_type)) { + case VIRTIO_VIDEO_QUEUE_TYPE_INPUT: + res->v4l2_index = g_list_length(s->inputq_resources); + break; + case VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT: + res->v4l2_index = g_list_length(s->outputq_resources); + break; + default: + g_critical("%s: invalid queue_type(%s) resource_id(%d)", + __func__, vio_queue_name(r->queue_type), + le32toh(cmd->resource_id)); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_RESOURCE_ID; + goto out_unlock; + } + + ret = add_resource(s, res, le32toh(cmd->queue_type)); + if (ret) { + g_critical("%s: resource_add id:%d failed", + __func__, le32toh(cmd->resource_id)); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_RESOURCE_ID; + goto out_unlock; + } + + g_debug("%s: resource=%p streamid(%d) resourceid(%d) numplanes(%d) " + "planes_layout(0x%x) %s", + __func__, res, res->stream_id, r->resource_id, r->num_planes, + r->planes_layout, vio_queue_name(r->queue_type)); + } + + if (r->planes_layout & VIRTIO_VIDEO_PLANES_LAYOUT_PER_PLANE) { + g_debug("%s: streamid(%d) resourceid(%d) planes_layout(0x%x)", + __func__, res->stream_id, r->resource_id, r->planes_layout); + + for (i = 0; i < r->num_planes; i++) { + total_entries += le32toh(cmd->num_entries[i]); + g_debug("%s: streamid(%d) resourceid(%d) num_entries[%d]=%d", + __func__, res->stream_id, r->resource_id, + i, le32toh(cmd->num_entries[i])); + } + } else { + total_entries = 1; + } + + /* + * virtio_video_resource_create is followed by either + * - struct virtio_video_mem_entry entries[] + * for VIRTIO_VIDEO_MEM_TYPE_GUEST_PAGES + * - struct virtio_video_object_entry entries[] + * for VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT + */ + + mem_type = get_queue_mem_type(s, r->queue_type); + + switch (mem_type) { + case VIRTIO_VIDEO_MEM_TYPE_GUEST_PAGES: + { + struct virtio_video_mem_entry *ent; + ent = (void *)cmd + sizeof(struct virtio_video_resource_create); + + res->iov = g_malloc0(sizeof(struct iovec) * total_entries); + for (i = 0; i < total_entries; i++) { + uint64_t len = le32toh(ent[i].length); + g_debug("%s: ent[%d] addr=0x%lx", + __func__, i, le64toh(ent[i].addr)); + + res->iov[i].iov_len = le32toh(ent[i].length); + res->iov[i].iov_base = + vu_gpa_to_va(&v->dev.parent, &len, le64toh(ent[i].addr)); + g_debug("%s: [%d] iov_len = 0x%lx", __func__ + , i, res->iov[i].iov_len); + g_debug("%s: [%d] iov_base = 0x%p", __func__ + , i, res->iov[i].iov_base); + } + res->iov_count = total_entries; + } break; + case VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT: + g_critical("%s: VIRTIO_OBJECT not implemented!", __func__); + /* TODO implement VIRTIO_OBJECT support */ + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + + cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + +out_unlock: + /* send response */ + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); + g_mutex_unlock(&s->mutex); + return ret; +} + +static int +handle_resource_queue_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd) +{ + struct virtio_video_resource_queue *cmd = + (struct virtio_video_resource_queue *)vio_cmd->cmd_buf; + struct resource *res; + struct stream *s; + uint32_t stream_id; + int ret = 0; + + g_debug("%s: type(0x%x) %s resource_id(%d)", __func__, + cmd->hdr.type, vio_queue_name(le32toh(cmd->queue_type)), + le32toh(cmd->resource_id)); + g_debug("%s: num_data_sizes = %d", __func__, le32toh(cmd->num_data_sizes)); + g_debug("%s: data_sizes[0] = %d", __func__, le32toh(cmd->data_sizes[0])); + + stream_id = cmd->hdr.stream_id; + + s = find_stream(v, stream_id); + if (!s) { + g_critical("%s: stream_id(%d) not found", __func__, stream_id); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + return ret; + } + + g_mutex_lock(&s->mutex); + + if (cmd->resource_id == 0) { + g_critical("%s: resource id 0 is not allowed", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_RESOURCE_ID; + goto out_unlock; + } + + if (le32toh(cmd->queue_type) == VIRTIO_VIDEO_QUEUE_TYPE_INPUT) { + if (g_list_length(s->inputq_resources) && !s->output_bufcount) { + ret = video_resource_create(s, le32toh(cmd->queue_type), + g_list_length(s->inputq_resources)); + if (ret < 0) { + g_critical("%s: output buffer allocation failed", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + } + } else { + if (g_list_length(s->outputq_resources) && !s->capture_bufcount) { + ret = video_resource_create(s, le32toh(cmd->queue_type), + g_list_length(s->outputq_resources)); + if (ret < 0) { + g_critical("%s: capture buffer allocation failed", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + } + } + + /* get resource object */ + res = find_resource(s, le32toh(cmd->resource_id), le32toh(cmd->queue_type)); + if (!res) { + g_critical("%s: resource_id:%d does not exist!" + , __func__, le32toh(cmd->resource_id)); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_RESOURCE_ID; + goto out_unlock; + } + + res->vio_res_q.timestamp = le64toh(cmd->timestamp); + res->vio_res_q.num_data_sizes = le32toh(cmd->num_data_sizes); + res->vio_res_q.queue_type = le32toh(cmd->queue_type); + res->vio_q_cmd = vio_cmd; + + g_debug("%s: res=%p res->vio_q_cmd=0x%p", __func__, res, res->vio_q_cmd); + + enum v4l2_buf_type buf_type = get_v4l2_buf_type( + cmd->queue_type, s->has_mplane); + enum virtio_video_mem_type mem_type = + get_queue_mem_type(s, cmd->queue_type); + enum v4l2_memory memory = get_v4l2_memory(mem_type); + + ret = v4l2_queue_buffer(buf_type, memory, cmd, res, s, v->v4l2_dev); + if (ret < 0) { + g_critical("%s: v4l2_queue_buffer failed", __func__); + /* virtio error set by v4l2_queue_buffer */ + goto out_unlock; + } + + /* + * let the stream worker thread do the dequeueing of output and + * capture queue buffers and send the resource_queue replies + */ + + g_mutex_unlock(&s->mutex); + return ret; + +out_unlock: + /* send response */ + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); + g_mutex_unlock(&s->mutex); + return ret; +} + + +static void +handle_resource_destroy_all_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd) +{ + struct virtio_video_resource_destroy_all *cmd = + (struct virtio_video_resource_destroy_all *)vio_cmd->cmd_buf; + enum v4l2_buf_type buf_type; + struct stream *s; + int ret = 0; + + g_debug("%s: type(0x%x) %s stream_id(%d)", __func__, + cmd->hdr.type, vio_queue_name(le32toh(cmd->queue_type)), + cmd->hdr.stream_id); + + s = find_stream(v, cmd->hdr.stream_id); + if (!s) { + g_critical("%s: stream_id(%d) not found", __func__, cmd->hdr.stream_id); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out; + } + + g_mutex_lock(&s->mutex); + + buf_type = get_v4l2_buf_type(le32toh(cmd->queue_type), s->has_mplane); + enum virtio_video_mem_type mem_type = + get_queue_mem_type(s, cmd->queue_type); + + ret = video_free_buffers(s->fd, buf_type, get_v4l2_memory(mem_type)); + if (ret) { + g_critical("%s: video_free_buffers() failed", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out; + } + + remove_all_resources(s, le32toh(cmd->queue_type)); + + /* free resource objects from queue list */ + cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + +out: + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); + g_mutex_unlock(&s->mutex); +} + +static void +handle_stream_create_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd) +{ + int ret = 0; + struct stream *s; + uint32_t req_stream_id; + uint32_t coded_format; + + struct virtio_video_stream_create *cmd = + (struct virtio_video_stream_create *)vio_cmd->cmd_buf; + + g_debug("%s: type(0x%x) stream_id(%d) in_mem_type(0x%x) " + "out_mem_type(0x%x) coded_format(0x%x)", + __func__, cmd->hdr.type, cmd->hdr.stream_id, + le32toh(cmd->in_mem_type), le32toh(cmd->out_mem_type), + le32toh(cmd->coded_format)); + + req_stream_id = cmd->hdr.stream_id; + coded_format = le32toh(cmd->coded_format); + + if ((le32toh(cmd->in_mem_type) == VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT) || + (le32toh(cmd->out_mem_type) == VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT)) { + /* TODO implement VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT */ + g_printerr("%s: MEM_TYPE_VIRTIO_OBJECT not supported yet", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + } else if (find_stream(v, req_stream_id)) { + g_debug("%s: Stream ID in use - ", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_STREAM_ID; + } else { + s = g_new0(struct stream, 1); + /* copy but bswap */ + s->vio_stream.in_mem_type = le32toh(cmd->in_mem_type); + s->vio_stream.out_mem_type = le32toh(cmd->out_mem_type); + s->vio_stream.coded_format = le32toh(cmd->coded_format); + strncpy((char *)&s->vio_stream.tag, (char *)cmd->tag, + sizeof(cmd->tag) - 1); + s->vio_stream.tag[sizeof(cmd->tag) - 1] = 0; + s->stream_id = req_stream_id; + s->video = v; + s->stream_state = STREAM_STOPPED; + s->has_mplane = v->v4l2_dev->has_mplane; + g_mutex_init(&s->mutex); + g_cond_init(&s->stream_cond); + v->streams = g_list_append(v->streams, s); + + cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + + /* set the requested coded format */ + ret = v4l2_stream_create(v->v4l2_dev, coded_format, s); + if (ret < 0) { + g_printerr("%s: v4l2_stream_create() failed", __func__); + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + + v->streams = g_list_remove(v->streams, s); + g_free(s); + } + + /* + * create thread to handle + * - dequeing buffers from output & capture queues + * - sending resource replies for buffers + * - handling EOS and dynamic-resoltion events + */ + s->worker_thread = g_thread_new("vio-video stream worker", + stream_worker_thread, s); + } + + /* send response */ + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); +} + +static void +handle_stream_drain_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd) +{ + int ret; + struct stream *s; + uint32_t stream_id; + struct virtio_video_stream_drain *cmd = + (struct virtio_video_stream_drain *)vio_cmd->cmd_buf; + + stream_id = cmd->hdr.stream_id; + + g_debug("%s: stream_id(%d)", __func__, stream_id); + + s = find_stream(v, stream_id); + if (!s) { + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_STREAM_ID; + return; + } + + g_debug("%s: Found stream=0x%p", __func__, s); + + g_mutex_lock(&s->mutex); + + ret = v4l2_issue_cmd(s->fd, V4L2_DEC_CMD_STOP, 0); + if (ret < 0) { + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; + goto out_unlock; + } + s->stream_state = STREAM_DRAINING; + g_cond_signal(&s->stream_cond); + + cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + +out_unlock: + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); + g_mutex_unlock(&s->mutex); +} + +static void +handle_stream_destroy_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd) +{ + struct stream *s; + uint32_t stream_id; + struct virtio_video_stream_destroy *cmd = + (struct virtio_video_stream_destroy *)vio_cmd->cmd_buf; + enum v4l2_buf_type buftype; + + if (!v || !vio_cmd) { + return; + } + + stream_id = cmd->hdr.stream_id; + + g_debug("%s: stream_id=(%d)", __func__, stream_id); + + s = find_stream(v, stream_id); + + if (!s) { + cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_STREAM_ID; + return; + } + + if (s) { + g_debug("%s: Found stream=0x%p", __func__, s); + + g_mutex_lock(&s->mutex); + /* TODO assumes decoder */ + buftype = s->has_mplane ? V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE + : V4L2_BUF_TYPE_VIDEO_OUTPUT; + + video_streamoff(s, buftype); + + /* signal worker thread */ + s->stream_state = STREAM_DESTROYING; + g_cond_signal(&s->stream_cond); + g_mutex_unlock(&s->mutex); + + /* wait for DESTROYED state */ + g_mutex_lock(&s->mutex); + while (s->stream_state != STREAM_DESTROYED) { + g_cond_wait(&s->stream_cond, &s->mutex); + } + + /* stream worker thread now exited */ + + enum virtio_video_mem_type mem_type = \ + get_queue_mem_type(s, VIRTIO_VIDEO_QUEUE_TYPE_INPUT); + /* deallocate the buffers */ + video_free_buffers(s->fd, buftype, get_v4l2_memory(mem_type)); + remove_all_resources(s, VIRTIO_VIDEO_QUEUE_TYPE_INPUT); + + buftype = s->has_mplane ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE : + V4L2_BUF_TYPE_VIDEO_CAPTURE; + + mem_type = get_queue_mem_type(s, VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT); + video_free_buffers(s->fd, buftype, get_v4l2_memory(mem_type)); + remove_all_resources(s, VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT); + + g_cond_clear(&s->stream_cond); + + v4l2_close(s->fd); + + v->streams = g_list_remove(v->streams, (gconstpointer) s); + cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; + } + + /* send response */ + vio_cmd->finished = true; + send_ctrl_response_nodata(vio_cmd); + g_mutex_unlock(&s->mutex); + g_mutex_clear(&s->mutex); + g_free(s); + + return; +} + +struct virtio_video_get_control_resp_level { + struct virtio_video_cmd_hdr hdr; + struct virtio_video_control_val_level level; +}; + +struct virtio_video_get_control_resp_profile { + struct virtio_video_cmd_hdr hdr; + struct virtio_video_control_val_profile profile; +}; + +struct virtio_video_get_control_resp_bitrate { + struct virtio_video_cmd_hdr hdr; + struct virtio_video_control_val_bitrate bitrate; +}; + +static int +handle_query_control_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *cmd) +{ + int ret; + uint32_t v4l2_control; + int32_t value; + + struct virtio_video_query_control_resp ctl_resp; + + struct virtio_video_query_control *qcmd = + (struct virtio_video_query_control *)cmd->cmd_buf; + + g_debug("%s: type(0x%x) stream_id(%d) control(0x%x)", __func__, + qcmd->hdr.type, qcmd->hdr.stream_id, le32toh(qcmd->control)); + + v4l2_control = virtio_video_control_to_v4l2(le32toh(qcmd->control)); + if (!v4l2_control) { + goto out_err_unlock; + } + + + ctl_resp.hdr.stream_id = qcmd->hdr.stream_id; + ctl_resp.hdr.type = VIRTIO_VIDEO_RESP_OK_QUERY_CONTROL; + + switch (le32toh(qcmd->control)) { + case VIRTIO_VIDEO_CONTROL_PROFILE: + struct virtio_video_query_control_resp_profile *ctl_resp_profile; + ctl_resp_profile = (void *)&ctl_resp + + sizeof(struct virtio_video_query_control_resp_profile); + + g_debug("%s: VIRTIO_VIDEO_CONTROL_PROFILE", __func__); + + ret = v4l2_ioctl_query_control(v->v4l2_dev->fd, v4l2_control, &value); + if (ret < 0) { + g_printerr("v4l2_ioctl_query_control() failed\n"); + goto out_err_unlock; + } + + ctl_resp_profile->num = htole32(value); + + cmd->finished = true; + send_ctrl_response(cmd, (uint8_t *)&ctl_resp_profile, + sizeof(ctl_resp_profile)); + + break; + + case VIRTIO_VIDEO_CONTROL_LEVEL: + struct virtio_video_query_control_resp_level *ctl_resp_level; + ctl_resp_level = (void *)&ctl_resp + + sizeof(struct virtio_video_query_control_resp_level); + + g_debug("%s: VIRTIO_VIDEO_CONTROL_LEVEL", __func__); + + ret = v4l2_ioctl_query_control(v->v4l2_dev->fd, v4l2_control, &value); + if (ret < 0) { + g_printerr("v4l2_ioctl_query_control() failed\n"); + goto out_err_unlock; + } + + ctl_resp_level->num = htole32(value); + + cmd->finished = true; + send_ctrl_response(cmd, (uint8_t *)&ctl_resp_level, + sizeof(ctl_resp_profile)); + break; + + default: + g_critical("Unknown control requested!"); + goto out_err_unlock; + break; + } + + return 0; + +out_err_unlock: + ctl_resp.hdr.stream_id = qcmd->hdr.stream_id; + ctl_resp.hdr.type = VIRTIO_VIDEO_RESP_ERR_UNSUPPORTED_CONTROL; + cmd->finished = true; + send_ctrl_response(cmd, (uint8_t *)&ctl_resp, + sizeof(struct virtio_video_query_control_resp)); + return -EINVAL; +} + +static int +handle_get_control_cmd(struct VuVideo *v, struct vu_video_ctrl_command *vio_cmd) +{ + int ret; + uint32_t v4l2_control; + int32_t value; + + struct virtio_video_get_control_resp ctl_resp_err; + struct virtio_video_get_control_resp_level ctl_resp_level; + struct virtio_video_get_control_resp_profile ctl_resp_profile; + struct virtio_video_get_control_resp_bitrate ctl_resp_bitrate; + + struct stream *s; + + struct virtio_video_get_control *cmd = + (struct virtio_video_get_control *)vio_cmd->cmd_buf; + + g_debug("%s: type(0x%x) stream_id(%d) control(0x%x)", __func__, + cmd->hdr.type, cmd->hdr.stream_id, le32toh(cmd->control)); + + s = find_stream(v, cmd->hdr.stream_id); + if (!s) { + g_critical("%s: stream_id(%d) not found", __func__, cmd->hdr.stream_id); + return -EINVAL; + } + + g_mutex_lock(&s->mutex); + + v4l2_control = virtio_video_control_to_v4l2(le32toh(cmd->control)); + if (!v4l2_control) { + goto out_err_unlock; + } + + + switch (le32toh(cmd->control)) { + case VIRTIO_VIDEO_CONTROL_BITRATE: + g_debug("%s: VIRTIO_VIDEO_CONTROL_BITRATE", __func__); + + ctl_resp_bitrate.hdr.stream_id = cmd->hdr.stream_id; + ctl_resp_bitrate.hdr.type = VIRTIO_VIDEO_RESP_OK_GET_PARAMS; + + if (v->v4l2_dev->dev_type == STATEFUL_ENCODER) { + ret = v4l2_ioctl_get_control(s->fd, v4l2_control, &value); + if (ret < 0) { + g_printerr("v4l2_ioctl_get_control() failed\n"); + goto out_err_unlock; + } + ctl_resp_bitrate.bitrate.bitrate = htole32(value); + + } else { + g_debug("%s: CONTROL_BITRATE unsupported for decoders!", __func__); + goto out_err_unlock; + } + + vio_cmd->finished = true; + send_ctrl_response( + vio_cmd, (uint8_t *)&ctl_resp_bitrate, + sizeof(struct virtio_video_get_control_resp_bitrate)); + break; + + case VIRTIO_VIDEO_CONTROL_PROFILE: + g_debug("%s: VIRTIO_VIDEO_CONTROL_PROFILE", __func__); + + ctl_resp_profile.hdr.stream_id = cmd->hdr.stream_id; + ctl_resp_profile.hdr.type = VIRTIO_VIDEO_RESP_OK_GET_PARAMS; + + ret = v4l2_ioctl_get_control(s->fd, v4l2_control, &value); + if (ret < 0) { + g_printerr("v4l2_ioctl_get_control() failed\n"); + goto out_err_unlock; + } + + ctl_resp_profile.profile.profile = htole32(value); + + vio_cmd->finished = true; + send_ctrl_response( + vio_cmd, (uint8_t *)&ctl_resp_profile, + sizeof(struct virtio_video_get_control_resp_profile)); + + /* + * TODO need to determine "in use" codec to (h264/vp8/vp9) to map to + * v4l2 control for PROFILE? + */ + + break; + + case VIRTIO_VIDEO_CONTROL_LEVEL: + g_debug("%s: VIRTIO_VIDEO_CONTROL_LEVEL", __func__); + + ctl_resp_level.hdr.stream_id = cmd->hdr.stream_id; + ctl_resp_level.hdr.type = VIRTIO_VIDEO_RESP_OK_GET_PARAMS; + + ret = v4l2_ioctl_get_control(s->fd, v4l2_control, &value); + if (ret < 0) { + g_printerr("v4l2_ioctl_get_control() failed\n"); + goto out_err_unlock; + } + + ctl_resp_level.level.level = htole32(value); + + vio_cmd->finished = true; + send_ctrl_response(vio_cmd, (uint8_t *)&ctl_resp_level, + sizeof(struct virtio_video_get_control_resp_level)); + break; + + case VIRTIO_VIDEO_CONTROL_BITRATE_MODE: + case VIRTIO_VIDEO_CONTROL_BITRATE_PEAK: + case VIRTIO_VIDEO_CONTROL_PREPEND_SPSPPS_TO_IDR: + g_info("Unsupported control requested"); + goto out_err_unlock; + break; + + default: + g_critical("Unknown control requested!"); + goto out_err_unlock; + break; + } + + g_mutex_unlock(&s->mutex); + return 0; + +out_err_unlock: + ctl_resp_err.hdr.stream_id = cmd->hdr.stream_id; + ctl_resp_err.hdr.type = VIRTIO_VIDEO_RESP_ERR_UNSUPPORTED_CONTROL; + vio_cmd->finished = true; + send_ctrl_response(vio_cmd, (uint8_t *)&ctl_resp_err, + sizeof(struct virtio_video_get_control_resp)); + g_mutex_unlock(&s->mutex); + return -EINVAL; +} + +static int +handle_query_capability_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *cmd) +{ + GList *fmt_l; + int ret; + enum v4l2_buf_type buf_type; + struct virtio_video_query_capability *qcmd = + (struct virtio_video_query_capability *)cmd->cmd_buf; + GByteArray *querycapresp; + + /* hdr bswapped already */ + g_debug("%s: type(0x%x) stream_id(%d) %s", __func__, + qcmd->hdr.type, qcmd->hdr.stream_id, + vio_queue_name(le32toh(qcmd->queue_type))); + + buf_type = get_v4l2_buf_type(le32toh(qcmd->queue_type), + v->v4l2_dev->has_mplane); + + /* enumerate formats */ + ret = video_enum_formats(v->v4l2_dev, buf_type, &fmt_l, false); + if (ret < 0) { + g_printerr("video_enum_formats failed"); + return ret; + } + + querycapresp = g_byte_array_new(); + querycapresp = create_query_cap_resp(qcmd, &fmt_l, querycapresp); + cmd->finished = true; + send_ctrl_response(cmd, querycapresp->data, querycapresp->len); + + video_free_formats(&fmt_l); + g_byte_array_free(querycapresp, true); + + return 0; +} + +static void +vv_process_cmd(VuVideo *video, struct vu_video_ctrl_command *cmd) +{ + switch (cmd->cmd_hdr->type) { + case VIRTIO_VIDEO_CMD_QUERY_CAPABILITY: + handle_query_capability_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_STREAM_CREATE: + handle_stream_create_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_STREAM_DESTROY: + handle_stream_destroy_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_STREAM_DRAIN: + handle_stream_drain_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_RESOURCE_CREATE: + handle_resource_create_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_RESOURCE_QUEUE: + handle_resource_queue_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_RESOURCE_DESTROY_ALL: + handle_resource_destroy_all_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_QUEUE_CLEAR: + handle_queue_clear_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_GET_PARAMS_EXT: + handle_get_params_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_SET_PARAMS_EXT: + handle_set_params_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_QUERY_CONTROL: + handle_query_control_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_GET_CONTROL: + handle_get_control_cmd(video, cmd); + break; + case VIRTIO_VIDEO_CMD_SET_CONTROL: + g_error("**** VIRTIO_VIDEO_CMD_SET_CONTROL unimplemented!"); + break; + default: + g_error("Unknown VIRTIO_VIDEO command!"); + break; + } +} + +/* for v3 virtio-video spec currently */ +static void +video_handle_ctrl(VuDev *dev, int qidx) +{ + VuVirtq *vq = vu_get_queue(dev, qidx); + VuVideo *video = container_of(dev, VuVideo, dev.parent); + size_t cmd_len, len, offset = 0; + + struct vu_video_ctrl_command *cmd; + + for (;;) { + + cmd = vu_queue_pop(dev, vq, sizeof(struct vu_video_ctrl_command)); + if (!cmd) { + break; + } + + cmd->vq = vq; + cmd->error = 0; + cmd->finished = false; + cmd->dev = dev; + + cmd_len = iov_size(cmd->elem.out_sg, cmd->elem.out_num); + cmd->cmd_buf = g_malloc0(cmd_len); + len = iov_to_buf_full(cmd->elem.out_sg, cmd->elem.out_num, + offset, cmd->cmd_buf, cmd_len); + + if (len != cmd_len) { + g_warning("%s: command size incorrect %zu vs %zu\n", + __func__, len, cmd_len); + } + + /* header is first on every cmd struct */ + cmd->cmd_hdr = (struct virtio_video_cmd_hdr *) cmd->cmd_buf; + /* bswap header */ + virtio_video_ctrl_hdr_letoh(cmd->cmd_hdr); + g_debug("Received %s cmd", vv_cmd_to_string(cmd->cmd_hdr->type)); + vv_process_cmd(video, cmd); + } +} + +static void +video_queue_set_started(VuDev *dev, int qidx, bool started) +{ + VuVirtq *vq = vu_get_queue(dev, qidx); + + g_debug("queue started %d:%d\n", qidx, started); + + switch (qidx) { + case 0: + vu_set_queue_handler(dev, vq, started ? video_handle_ctrl : NULL); + break; + default: + break; + } +} + +/* + * video_process_msg: process messages of vhost-user interface + * + * Any that are not handled here are processed by the libvhost library + * itself. + */ +static int video_process_msg(VuDev *dev, VhostUserMsg *msg, int *do_reply) +{ + VuVideo *r = container_of(dev, VuVideo, dev.parent); + + g_debug("%s: msg %d", __func__, msg->request); + + switch (msg->request) { + case VHOST_USER_NONE: + g_main_loop_quit(r->loop); + return 1; + default: + return 0; + } +} + +static const VuDevIface vuiface = { + .set_features = video_set_features, + .get_features = video_get_features, + .queue_set_started = video_queue_set_started, + .process_msg = video_process_msg, + .get_config = video_get_config, + .set_config = video_set_config, +}; + +static void video_destroy(VuVideo *v) +{ + vug_deinit(&v->dev); + if (socket_path) { + unlink(socket_path); + } + + v4l2_backend_free(v->v4l2_dev); +} + +/* Print vhost-user.json backend program capabilities */ +static void print_capabilities(void) +{ + printf("{\n"); + printf(" \"type\": \"misc\"\n"); + printf("}\n"); +} + +static gboolean hangup(gpointer user_data) +{ + GMainLoop *loop = (GMainLoop *) user_data; + g_info("%s: caught hangup/quit signal, quitting main loop", __func__); + g_main_loop_quit(loop); + return true; +} + +int main(int argc, char *argv[]) +{ + GError *error = NULL; + GOptionContext *context; + g_autoptr(GSocket) socket = NULL; + VuVideo video = { }; + + context = g_option_context_new("vhost-user emulation of video device"); + g_option_context_add_main_entries(context, options, "vhost-user-video"); + if (!g_option_context_parse(context, &argc, &argv, &error)) { + g_printerr("option parsing failed: %s\n", error->message); + exit(1); + } + + g_option_context_free(context); + + if (print_cap) { + print_capabilities(); + exit(0); + } + + if (!socket_path && socket_fd < 0) { + g_printerr("Please specify either --fd or --socket-path\n"); + exit(EXIT_FAILURE); + } + + if (verbose || debug) { + g_log_set_handler(NULL, G_LOG_LEVEL_MASK, g_log_default_handler, NULL); + if (debug) { + g_setenv("G_MESSAGES_DEBUG", "all", true); + } + } else { + g_log_set_handler(NULL, + G_LOG_LEVEL_WARNING | G_LOG_LEVEL_CRITICAL + | G_LOG_LEVEL_ERROR, + g_log_default_handler, NULL); + } + + /* + * Open the v4l2 device and enumerate supported formats. + * Use this to determine whether it is a stateful encoder/decoder. + */ + if (!v4l2_path || !g_file_test(v4l2_path, G_FILE_TEST_EXISTS)) { + g_printerr("Please specify a valid --v4l2-device\n"); + exit(EXIT_FAILURE); + } else { + video.v4l2_dev = v4l2_backend_init(v4l2_path); + if (!video.v4l2_dev) { + g_printerr("v4l2 backend init failed!\n"); + exit(EXIT_FAILURE); + } + } + + /* + * Now create a vhost-user socket that we will receive messages + * on. Once we have our handler set up we can enter the glib main + * loop. + */ + if (socket_path) { + g_autoptr(GSocketAddress) addr = g_unix_socket_address_new(socket_path); + g_autoptr(GSocket) bind_socket = + g_socket_new(G_SOCKET_FAMILY_UNIX, G_SOCKET_TYPE_STREAM, + G_SOCKET_PROTOCOL_DEFAULT, &error); + + if (!g_socket_bind(bind_socket, addr, false, &error)) { + g_printerr("Failed to bind to socket at %s (%s).\n", + socket_path, error->message); + exit(EXIT_FAILURE); + } + if (!g_socket_listen(bind_socket, &error)) { + g_printerr("Failed to listen on socket %s (%s).\n", + socket_path, error->message); + } + g_message("awaiting connection to %s", socket_path); + socket = g_socket_accept(bind_socket, NULL, &error); + if (!socket) { + g_printerr("Failed to accept on socket %s (%s).\n", + socket_path, error->message); + } + } else { + socket = g_socket_new_from_fd(socket_fd, &error); + if (!socket) { + g_printerr("Failed to connect to FD %d (%s).\n", + socket_fd, error->message); + exit(EXIT_FAILURE); + } + } + + /* + * Create the main loop first so all the various sources can be + * added. As well as catching signals we need to ensure vug_init + * can add it's GSource watches. + */ + + video.loop = g_main_loop_new(NULL, FALSE); + /* catch exit signals */ + g_unix_signal_add(SIGHUP, hangup, video.loop); + g_unix_signal_add(SIGINT, hangup, video.loop); + + if (!vug_init(&video.dev, VHOST_USER_VIDEO_MAX_QUEUES, + g_socket_get_fd(socket), + video_panic, &vuiface)) { + g_printerr("Failed to initialize libvhost-user-glib.\n"); + exit(EXIT_FAILURE); + } + + g_message("entering main loop, awaiting messages"); + g_main_loop_run(video.loop); + g_message("finished main loop, cleaning up"); + + g_main_loop_unref(video.loop); + video_destroy(&video); +} diff --git a/tools/vhost-user-video/virtio_video_helpers.c b/tools/vhost-user-video/virtio_video_helpers.c new file mode 100644 index 0000000000..9553707274 --- /dev/null +++ b/tools/vhost-user-video/virtio_video_helpers.c @@ -0,0 +1,466 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * virtio-video helpers + * + * Copyright Red Hat, Inc. 2023 + * Copyright Linaro 2021 + * Copyright 2019 OpenSynergy GmbH. + * + * Authors: + * Peter Griffin + * Albert Esteve + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include +#include +#include +#include +#include + +#include +#include + +#include "standard-headers/linux/virtio_video.h" +#include +#include "v4l2_backend.h" +#include "virtio_video_helpers.h" + +struct virtio_video_convert_table { + uint32_t virtio_value; + uint32_t v4l2_value; +}; + +#define ARRAY_SIZE(a) (sizeof(a) / sizeof((a)[0])) + +static struct virtio_video_convert_table level_table[] = { + { VIRTIO_VIDEO_LEVEL_H264_1_0, V4L2_MPEG_VIDEO_H264_LEVEL_1_0 }, + { VIRTIO_VIDEO_LEVEL_H264_1_1, V4L2_MPEG_VIDEO_H264_LEVEL_1_1 }, + { VIRTIO_VIDEO_LEVEL_H264_1_2, V4L2_MPEG_VIDEO_H264_LEVEL_1_2 }, + { VIRTIO_VIDEO_LEVEL_H264_1_3, V4L2_MPEG_VIDEO_H264_LEVEL_1_3 }, + { VIRTIO_VIDEO_LEVEL_H264_2_0, V4L2_MPEG_VIDEO_H264_LEVEL_2_0 }, + { VIRTIO_VIDEO_LEVEL_H264_2_1, V4L2_MPEG_VIDEO_H264_LEVEL_2_1 }, + { VIRTIO_VIDEO_LEVEL_H264_2_2, V4L2_MPEG_VIDEO_H264_LEVEL_2_2 }, + { VIRTIO_VIDEO_LEVEL_H264_3_0, V4L2_MPEG_VIDEO_H264_LEVEL_3_0 }, + { VIRTIO_VIDEO_LEVEL_H264_3_1, V4L2_MPEG_VIDEO_H264_LEVEL_3_1 }, + { VIRTIO_VIDEO_LEVEL_H264_3_2, V4L2_MPEG_VIDEO_H264_LEVEL_3_2 }, + { VIRTIO_VIDEO_LEVEL_H264_4_0, V4L2_MPEG_VIDEO_H264_LEVEL_4_0 }, + { VIRTIO_VIDEO_LEVEL_H264_4_1, V4L2_MPEG_VIDEO_H264_LEVEL_4_1 }, + { VIRTIO_VIDEO_LEVEL_H264_4_2, V4L2_MPEG_VIDEO_H264_LEVEL_4_2 }, + { VIRTIO_VIDEO_LEVEL_H264_5_0, V4L2_MPEG_VIDEO_H264_LEVEL_5_0 }, + { VIRTIO_VIDEO_LEVEL_H264_5_1, V4L2_MPEG_VIDEO_H264_LEVEL_5_1 }, + { 0 }, +}; + +uint32_t virtio_video_level_to_v4l2(uint32_t level) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(level_table); idx++) { + if (level_table[idx].virtio_value == level) { + return level_table[idx].v4l2_value; + } + } + + return 0; +} + +uint32_t virtio_video_v4l2_level_to_virtio(uint32_t v4l2_level) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(level_table); idx++) { + if (level_table[idx].v4l2_value == v4l2_level) { + return level_table[idx].virtio_value; + } + } + + return 0; +} + +static struct virtio_video_convert_table profile_table[] = { + { VIRTIO_VIDEO_PROFILE_H264_BASELINE, + V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE }, + { VIRTIO_VIDEO_PROFILE_H264_MAIN, V4L2_MPEG_VIDEO_H264_PROFILE_MAIN }, + { VIRTIO_VIDEO_PROFILE_H264_EXTENDED, + V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED }, + { VIRTIO_VIDEO_PROFILE_H264_HIGH, V4L2_MPEG_VIDEO_H264_PROFILE_HIGH }, + { VIRTIO_VIDEO_PROFILE_H264_HIGH10PROFILE, + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_10 }, + { VIRTIO_VIDEO_PROFILE_H264_HIGH422PROFILE, + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_422}, + { VIRTIO_VIDEO_PROFILE_H264_HIGH444PREDICTIVEPROFILE, + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH_444_PREDICTIVE }, + { VIRTIO_VIDEO_PROFILE_H264_SCALABLEBASELINE, + V4L2_MPEG_VIDEO_H264_PROFILE_SCALABLE_BASELINE }, + { VIRTIO_VIDEO_PROFILE_H264_SCALABLEHIGH, + V4L2_MPEG_VIDEO_H264_PROFILE_SCALABLE_HIGH }, + { VIRTIO_VIDEO_PROFILE_H264_STEREOHIGH, + V4L2_MPEG_VIDEO_H264_PROFILE_STEREO_HIGH }, + { VIRTIO_VIDEO_PROFILE_H264_MULTIVIEWHIGH, + V4L2_MPEG_VIDEO_H264_PROFILE_MULTIVIEW_HIGH }, + { 0 }, +}; + +uint32_t virtio_video_profile_to_v4l2(uint32_t profile) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(profile_table); idx++) { + if (profile_table[idx].virtio_value == profile) { + return profile_table[idx].v4l2_value; + } + } + + return 0; +} + +uint32_t virtio_video_v4l2_profile_to_virtio(uint32_t v4l2_profile) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(profile_table); idx++) { + if (profile_table[idx].v4l2_value == v4l2_profile) { + return profile_table[idx].virtio_value; + } + } + + return 0; +} + + +static struct virtio_video_convert_table format_table[] = { + { VIRTIO_VIDEO_FORMAT_ARGB8888, V4L2_PIX_FMT_ARGB32 }, + { VIRTIO_VIDEO_FORMAT_BGRA8888, V4L2_PIX_FMT_ABGR32 }, + { VIRTIO_VIDEO_FORMAT_NV12, V4L2_PIX_FMT_NV12 }, + { VIRTIO_VIDEO_FORMAT_YUV420, V4L2_PIX_FMT_YUV420 }, + { VIRTIO_VIDEO_FORMAT_YVU420, V4L2_PIX_FMT_YVU420 }, + { VIRTIO_VIDEO_FORMAT_MPEG2, V4L2_PIX_FMT_MPEG2 }, + { VIRTIO_VIDEO_FORMAT_MPEG4, V4L2_PIX_FMT_MPEG4 }, + { VIRTIO_VIDEO_FORMAT_H264, V4L2_PIX_FMT_H264 }, + { VIRTIO_VIDEO_FORMAT_HEVC, V4L2_PIX_FMT_HEVC }, + { VIRTIO_VIDEO_FORMAT_VP8, V4L2_PIX_FMT_VP8 }, + { VIRTIO_VIDEO_FORMAT_VP9, V4L2_PIX_FMT_VP9 }, + { VIRTIO_VIDEO_FORMAT_FWHT, V4L2_PIX_FMT_FWHT }, + { 0 }, +}; + +uint32_t virtio_video_format_to_v4l2(uint32_t format) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(format_table); idx++) { + if (format_table[idx].virtio_value == format) { + return format_table[idx].v4l2_value; + } + } + + return 0; +} + +uint32_t virtio_video_v4l2_format_to_virtio(uint32_t v4l2_format) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(format_table); idx++) { + if (format_table[idx].v4l2_value == v4l2_format) { + return format_table[idx].virtio_value; + } + } + + return 0; +} + +/* + * TODO FIXME PROFILE and LEVEL seem wrong here as tied to H264 codec. + * V4L2_CID_MPEG_VIDEO_VP9_PROFILE + * e.g. https://elixir.bootlin.com/linux/v5.12.1/source/ + * include/uapi/linux/v4l2-controls.h#L669 + */ +static struct virtio_video_convert_table control_table[] = { + { VIRTIO_VIDEO_CONTROL_BITRATE, V4L2_CID_MPEG_VIDEO_BITRATE }, + { VIRTIO_VIDEO_CONTROL_BITRATE_PEAK, V4L2_CID_MPEG_VIDEO_BITRATE_PEAK }, + { VIRTIO_VIDEO_CONTROL_BITRATE_MODE, V4L2_CID_MPEG_VIDEO_BITRATE_MODE }, + { VIRTIO_VIDEO_CONTROL_PROFILE, V4L2_CID_MPEG_VIDEO_H264_PROFILE }, + { VIRTIO_VIDEO_CONTROL_LEVEL, V4L2_CID_MPEG_VIDEO_H264_LEVEL }, + { VIRTIO_VIDEO_CONTROL_FORCE_KEYFRAME, + V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME }, + { VIRTIO_VIDEO_CONTROL_PREPEND_SPSPPS_TO_IDR, + V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR }, + { 0 }, +}; + +uint32_t virtio_video_control_to_v4l2(uint32_t control) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(control_table); idx++) { + if (control_table[idx].virtio_value == control) { + return control_table[idx].v4l2_value; + } + } + + return 0; +} + +uint32_t virtio_video_v4l2_control_to_virtio(uint32_t v4l2_control) +{ + size_t idx; + + for (idx = 0; idx < ARRAY_SIZE(control_table); idx++) { + if (control_table[idx].v4l2_value == v4l2_control) { + return control_table[idx].virtio_value; + } + } + + return 0; +} + +/* new helper functions (not from Linux frontend driver) */ + +const char *vio_queue_name(enum virtio_video_queue_type queue) +{ + if (queue == VIRTIO_VIDEO_QUEUE_TYPE_INPUT) { + return "Queue Input"; + } + if (queue == VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT) { + return "Queue Output"; + } + + return "Queue type unknown"; +} + + +__le64 virtio_fmtdesc_generate_mask(GList **p_list) +{ + uint64_t mask = 0; + unsigned int bit = 0; + GList *l; + + for (l = *p_list; l != NULL; l = l->next) { + mask |= (1 << bit); + bit++; + } + g_debug("%s: mask(0x%lx)\n", __func__, mask); + + return mask; +} + +/* vio_codedformat endian swapped by upper level */ + +int v4l2_stream_create(struct v4l2_device *dev, uint32_t vio_codedformat, + struct stream *s) +{ + enum v4l2_buf_type buf_type; + uint32_t v4l2_pixformat; + int ret; + + /* buf type for coded format depends on device type */ + if (dev->dev_type & STATEFUL_DECODER) { + buf_type = dev->has_mplane ? V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE + : V4L2_BUF_TYPE_VIDEO_OUTPUT; + + } else if (dev->dev_type & STATEFUL_ENCODER) { + buf_type = dev->has_mplane ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE : + V4L2_BUF_TYPE_VIDEO_CAPTURE; + } else { + g_critical("Unknown device type %d!", dev->dev_type); + return -EINVAL; + } + + s->fd = v4l2_open(dev->devname); + if (s->fd < 0) { + g_printerr("Error opening device %s: %s (%d).\n", dev->devname, + g_strerror(errno), errno); + } + + v4l2_pixformat = virtio_video_format_to_v4l2(vio_codedformat); + if (v4l2_pixformat == 0) { + g_error("%s: virtio to v4l2 format translation failed!", __func__); + ret = -EINVAL; + return ret; + } + + /* set the requested coded format */ + ret = v4l2_set_pixel_format(s->fd, buf_type, v4l2_pixformat); + if (ret < 0) { + g_printerr("%s: v4l2_video_set_pixel_format() failed", __func__); + } + + return ret; +} + +void v4l2_to_virtio_fmtdesc(struct v4l2_device *dev, + struct video_format *vid_fmt, + enum v4l2_buf_type type) +{ + struct v4l2_fmtdesc *v4l2_fmtdsc = &vid_fmt->fmt; + struct virtio_video_format_desc *virtio_fmtdesc = &vid_fmt->desc; + enum v4l2_buf_type buftype = V4L2_BUF_TYPE_PRIVATE; + int ret; + + if (!vid_fmt) { + return; + } + + virtio_fmtdesc->format = + htole32(virtio_video_v4l2_format_to_virtio(v4l2_fmtdsc->pixelformat)); + + /* + * To generate the mask we need to check the FORMAT is already set. + * before we enumerate the other queue to generate the mask + */ + + ret = v4l2_set_pixel_format(dev->fd, type, vid_fmt->fmt.pixelformat); + if (ret < 0) { + g_printerr("%s: v4l2_video_get_format() failed\n", __func__); + } + + /* enumerate formats on the other queue now the format is set */ + GList *vid_fmts_l = NULL; + + if (V4L2_TYPE_IS_OUTPUT(type)) { + buftype = V4L2_TYPE_IS_MULTIPLANAR(type) ? + V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE : + V4L2_BUF_TYPE_VIDEO_CAPTURE; + } + + if (V4L2_TYPE_IS_CAPTURE(type)) { + buftype = V4L2_TYPE_IS_MULTIPLANAR(type) ? + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE : + V4L2_BUF_TYPE_VIDEO_OUTPUT; + } + + ret = video_enum_formats(dev, buftype, &vid_fmts_l, true); + + /* + * generate the capability mask. bitset represents the supported + * combinations of input and output formats. + */ + + virtio_fmtdesc->mask = htole64(virtio_fmtdesc_generate_mask(&vid_fmts_l)); + + virtio_fmtdesc->planes_layout = + htole32(VIRTIO_VIDEO_PLANES_LAYOUT_SINGLE_BUFFER); + + /* TODO need to set plane_align */ + if ((!v4l2_fmtdsc->flags & V4L2_FMT_FLAG_COMPRESSED) && + (le32toh(virtio_fmtdesc->planes_layout) & + VIRTIO_VIDEO_PLANES_LAYOUT_SINGLE_BUFFER)) { + g_critical("%s: TODO need to set plane_align field", __func__); + } + + virtio_fmtdesc->num_frames = htole32(g_list_length(vid_fmt->vid_fmt_frm_l)); + + video_free_formats(&vid_fmts_l); +} + +void v4l2_to_virtio_video_params(struct v4l2_device *dev, + struct v4l2_format *fmt, + struct v4l2_selection *sel, + struct virtio_video_get_params_resp *resp) +{ + struct virtio_video_params *vid_params = &resp->params; + int i; + + /* min/max_buffers default (taken from crosvm) */ + vid_params->min_buffers = htole32(1); + vid_params->max_buffers = htole32(32); + + if (V4L2_TYPE_IS_MULTIPLANAR(fmt->type)) { + vid_params->format = + virtio_video_v4l2_format_to_virtio(fmt->fmt.pix.pixelformat); + vid_params->frame_width = htole32(fmt->fmt.pix_mp.width); + vid_params->frame_height = htole32(fmt->fmt.pix_mp.height); + + vid_params->num_planes = htole32(fmt->fmt.pix_mp.num_planes); + + for (i = 0; i < fmt->fmt.pix_mp.num_planes; i++) { + vid_params->plane_formats[i].stride = \ + htole32(fmt->fmt.pix_mp.plane_fmt[i].bytesperline); + + vid_params->plane_formats[i].plane_size = \ + htole32(fmt->fmt.pix_mp.plane_fmt[i].sizeimage); + + g_debug(" ** Stride %u, buffer size %u\n", + fmt->fmt.pix_mp.plane_fmt[i].bytesperline, + fmt->fmt.pix_mp.plane_fmt[i].sizeimage); + } + } else if (V4L2_TYPE_IS_SINGLEPLANAR(fmt->type)) { + vid_params->format = + virtio_video_v4l2_format_to_virtio(fmt->fmt.pix.pixelformat); + vid_params->frame_width = htole32(fmt->fmt.pix.width); + vid_params->frame_height = htole32(fmt->fmt.pix.height); + vid_params->num_planes = htole32(1); + + vid_params->plane_formats[0].stride = \ + htole32(fmt->fmt.pix.bytesperline); + + vid_params->plane_formats[0].plane_size = \ + htole32(fmt->fmt.pix.sizeimage); + } + + /* cropping rectangle */ + if (V4L2_TYPE_IS_CAPTURE(fmt->type)) { + vid_params->crop.left = htole32(sel->r.left); + vid_params->crop.top = htole32(sel->r.top); + vid_params->crop.width = htole32(sel->r.width); + vid_params->crop.height = htole32(sel->r.height); + + g_debug("%s: crop: left=(%d) top=(%d) width=(%d) height=(%d)", + __func__, sel->r.left, sel->r.top, sel->r.width, sel->r.height); + } + + /* TODO frame_rate field for encoder */ +} + +void v4l2_to_virtio_event(struct v4l2_event *ev, + struct virtio_video_event *vio_ev) +{ + g_debug("%s: %ld.%06ld: event %u, pending %u", + __func__, ev->timestamp.tv_sec, ev->timestamp.tv_nsec / 1000, + ev->sequence, ev->pending); + /* Reset event type */ + vio_ev->event_type = 0x0; + switch (ev->type) { + case V4L2_EVENT_VSYNC: + g_debug("vsync\n"); + break; + case V4L2_EVENT_EOS: + g_debug("eos\n"); + break; + case V4L2_EVENT_CTRL: + g_debug("eos\n"); + break; + case V4L2_EVENT_FRAME_SYNC: + g_debug("frame_sync %d\n", ev->u.frame_sync.frame_sequence); + break; + case V4L2_EVENT_SOURCE_CHANGE: + g_debug("source_change!: pad/input=%d changes: %x\n" + , ev->id, ev->u.src_change.changes); + + vio_ev->event_type = + htole32(VIRTIO_VIDEO_EVENT_DECODER_RESOLUTION_CHANGED); + /* TODO need proper mapping from v4l2 streamid to virtio streamid */ + vio_ev->stream_id = htole32(ev->id) + 1; + break; + case V4L2_EVENT_MOTION_DET: + if (ev->u.motion_det.flags & V4L2_EVENT_MD_FL_HAVE_FRAME_SEQ) { + g_debug("motion_det frame %d, regions 0x%x\n", + ev->u.motion_det.frame_sequence, + ev->u.motion_det.region_mask); + } else { + g_debug("motion_det regions 0x%x\n", ev->u.motion_det.region_mask); + } + break; + default: + if (ev->type >= V4L2_EVENT_PRIVATE_START) { + g_debug("unknown private event (%08x)\n", ev->type); + } else { + g_debug("unknown event (%08x)\n", ev->type); + } + break; + } +} diff --git a/tools/vhost-user-video/virtio_video_helpers.h b/tools/vhost-user-video/virtio_video_helpers.h new file mode 100644 index 0000000000..8f35ccc4b5 --- /dev/null +++ b/tools/vhost-user-video/virtio_video_helpers.h @@ -0,0 +1,168 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * virtio-video helpers + * + * Copyright Red Hat, Inc. 2023 + * Copyright Linaro 2021 + * + * Authors: + * Peter Griffin + * Albert Esteve + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef VIRTIO_VIDEO_HELPERS_H +#define VIRTIO_VIDEO_HELPERS_H + +#include +#include "standard-headers/linux/virtio_video.h" +#include +#include "libvhost-user-glib.h" +#include "libvhost-user.h" + +/* + * Structure to track internal state of VIDEO Device + */ + +typedef struct VuVideo { + VugDev dev; + struct virtio_video_config virtio_config; + GMainLoop *loop; + struct v4l2_device *v4l2_dev; + GList *streams; +} VuVideo; + +struct v4l2_device { + const gchar *devname; + unsigned int dev_type; + unsigned int capabilities; + int fd; + int epollfd; + int opened; + bool has_mplane; + bool sup_dyn_res_switching; +}; + +struct vu_video_ctrl_command { + VuVirtqElement elem; + VuVirtq *vq; + VuDev *dev; + struct virtio_video_cmd_hdr *cmd_hdr; + uint32_t error; + bool finished; + uint8_t *cmd_buf; +}; + + +/* + * Structure to track internal state of a Stream + */ + +struct stream { + struct virtio_video_stream_create vio_stream; + uint32_t stream_id; + GList *inputq_resources; + GList *outputq_resources; + VuVideo *video; + GThread *worker_thread; + uint32_t stream_state; + GMutex mutex; + GCond stream_cond; + bool output_streaming; + bool capture_streaming; + bool subscribed_events; + bool has_mplane; + int fd; + uint32_t output_bufcount; + uint32_t capture_bufcount; +}; + +#define STREAM_STOPPED 1 +#define STREAM_STREAMING 2 +#define STREAM_DRAINING 3 +#define STREAM_DESTROYING 4 +#define STREAM_DESTROYED 5 + +/* Structure to track resources */ + +struct resource { + uint32_t stream_id; + struct virtio_video_resource_create vio_resource; + struct virtio_video_resource_queue vio_res_q; + struct iovec *iov; + uint32_t iov_count; + uint32_t v4l2_index; + enum v4l2_buf_type type; + struct vu_video_ctrl_command *vio_q_cmd; + bool queued; +}; + +struct video_format_frame_rates { + struct virtio_video_format_range frame_rates; + struct v4l2_frmivalenum v4l_ival; +}; + +struct video_format_frame { + struct virtio_video_format_frame frame; + struct v4l2_frmsizeenum v4l_framesize; + GList *frm_rate_l; +}; + +struct video_format { + struct v4l2_fmtdesc fmt; + struct virtio_video_format_desc desc; + GList *vid_fmt_frm_l; +}; + +/* function prototypes */ +int v4l2_stream_create(struct v4l2_device *dev, + uint32_t vio_codedformat, struct stream *s); +void v4l2_to_virtio_video_params(struct v4l2_device *dev, + struct v4l2_format *fmt, + struct v4l2_selection *sel, + struct virtio_video_get_params_resp *resp); + +void v4l2_to_virtio_fmtdesc(struct v4l2_device *dev, + struct video_format *vid_fmt, + enum v4l2_buf_type type); + +void v4l2_to_virtio_event(struct v4l2_event *ev, + struct virtio_video_event *vio_ev); + +struct resource *find_resource_by_v4l2index(struct stream *s, + enum v4l2_buf_type buf_type, + uint32_t v4l2_index); +/* + * The following conversion helpers and tables taken from Linux + * frontend driver from opensynergy + */ + +uint32_t virtio_video_level_to_v4l2(uint32_t level); +uint32_t virtio_video_v4l2_level_to_virtio(uint32_t v4l2_level); +uint32_t virtio_video_profile_to_v4l2(uint32_t profile); +uint32_t virtio_video_v4l2_profile_to_virtio(uint32_t v4l2_profile); +uint32_t virtio_video_format_to_v4l2(uint32_t format); +uint32_t virtio_video_v4l2_format_to_virtio(uint32_t v4l2_format); +uint32_t virtio_video_control_to_v4l2(uint32_t control); +uint32_t virtio_video_v4l2_control_to_virtio(uint32_t v4l2_control); +__le64 virtio_fmtdesc_generate_mask(GList **p_list); + +/* Helpers for logging */ +const char *vio_queue_name(enum virtio_video_queue_type queue); + +static inline void +virtio_video_ctrl_hdr_letoh(struct virtio_video_cmd_hdr *hdr) +{ + hdr->type = le32toh(hdr->type); + hdr->stream_id = le32toh(hdr->stream_id); +} + +static inline void +virtio_video_ctrl_hdr_htole(struct virtio_video_cmd_hdr *hdr) +{ + hdr->type = htole32(hdr->type); + hdr->stream_id = htole32(hdr->stream_id); +} +#endif diff --git a/tools/vhost-user-video/vuvideo.h b/tools/vhost-user-video/vuvideo.h new file mode 100644 index 0000000000..d853c69682 --- /dev/null +++ b/tools/vhost-user-video/vuvideo.h @@ -0,0 +1,43 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * vhost-user-video header + * + * Copyright Red Hat, Inc. 2023 + * Copyright Linaro 2021 + * + * Authors: + * Peter Griffin + * Albert Esteve + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef VUVIDEO_H +#define VUVIDEO_H + +#include "virtio_video_helpers.h" +#include "v4l2_backend.h" +#include "vuvideo.h" + +GList *get_resource_list(struct stream *s, uint32_t queue_type); +void send_qclear_res_reply(gpointer data, gpointer user_data); + +struct stream *find_stream(struct VuVideo *v, uint32_t stream_id); +int add_resource(struct stream *s, struct resource *r, uint32_t queue_type); +int remove_resource(struct stream *s, struct resource *r, uint32_t queue_type); +struct resource *find_resource(struct stream *s, uint32_t resource_id, + uint32_t queue_type); + +void send_ctrl_response(struct vu_video_ctrl_command *vio_cmd, + uint8_t *resp, size_t resp_len); + +void send_ctrl_response_nodata(struct vu_video_ctrl_command *vio_cmd); + +void free_resource_mem(struct resource *r); +void remove_all_resources(struct stream *s, uint32_t queue_type); + +void handle_queue_clear_cmd(struct VuVideo *v, + struct vu_video_ctrl_command *vio_cmd); + +#endif From patchwork Wed Mar 22 14:21:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Albert Esteve X-Patchwork-Id: 13184156 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4A7A6C6FD1F for ; Wed, 22 Mar 2023 14:23:33 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pezLv-00086Q-LP; Wed, 22 Mar 2023 10:22:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLt-000860-V5 for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:22:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLs-0007OF-1t for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:22:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679494927; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Cy45JThKfhw+pOczCa5UvFFxft7EmjKUU3fwffIkfGE=; b=bticcc1Vmq9LoUHbO+yj5ZSYm2JdeklOMbjFt8JMf3Id0Z0Kh0UPpOiXKp12XfDstkRPep ia6K0viuGfZ3HXpmjezdmAJAQcg9LIbYzVjcBRaFD4xwkqPlKdUFrTqkuLh15wAo9pM4U4 NIcfGXbawd+8xPpEDZWKPbRuQosmqp8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-137-Fn0hgK4AOB-qQDuVRWAhCg-1; Wed, 22 Mar 2023 10:22:04 -0400 X-MC-Unique: Fn0hgK4AOB-qQDuVRWAhCg-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 22483858F09; Wed, 22 Mar 2023 14:22:03 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.226.254]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8064540C6E67; Wed, 22 Mar 2023 14:22:00 +0000 (UTC) From: Albert Esteve To: qemu-devel@nongnu.org Cc: Thomas Huth , fmartine@redhat.com, Gerd Hoffmann , eballetb@redhat.com, Albert Esteve , alex.bennee@linaro.org, Laurent Vivier , "Michael S. Tsirkin" , sgarzare@redhat.com, =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , peter.griffin@linaro.org, Paolo Bonzini Subject: [PATCH 09/12] tests/qtest: add virtio-video test Date: Wed, 22 Mar 2023 15:21:29 +0100 Message-Id: <20230322142132.22909-10-aesteve@redhat.com> In-Reply-To: <20230322142132.22909-1-aesteve@redhat.com> References: <20230322142132.22909-1-aesteve@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add some minimal testing of the vhost-user-video backend similar to what is tested for other vhost-user devices. For now it only stresses the backend instantiation and some message passing through the control socket. Signed-off-by: Albert Esteve --- tests/qtest/libqos/meson.build | 1 + tests/qtest/libqos/virtio-video.c | 179 ++++++++++++++++++++++++++++++ tests/qtest/libqos/virtio-video.h | 39 +++++++ tests/qtest/vhost-user-test.c | 32 ++++++ 4 files changed, 251 insertions(+) create mode 100644 tests/qtest/libqos/virtio-video.c create mode 100644 tests/qtest/libqos/virtio-video.h diff --git a/tests/qtest/libqos/meson.build b/tests/qtest/libqos/meson.build index cc209a8de5..055bd47928 100644 --- a/tests/qtest/libqos/meson.build +++ b/tests/qtest/libqos/meson.build @@ -46,6 +46,7 @@ libqos_srcs = files( 'virtio-serial.c', 'virtio-iommu.c', 'virtio-gpio.c', + 'virtio-video.c', 'generic-pcihost.c', # qgraph machines: diff --git a/tests/qtest/libqos/virtio-video.c b/tests/qtest/libqos/virtio-video.c new file mode 100644 index 0000000000..f61c34753d --- /dev/null +++ b/tests/qtest/libqos/virtio-video.c @@ -0,0 +1,179 @@ +/* + * libqos virtio-video driver + * + * Copyright (c) 2023 Red Hat Inc. + * + * Authors: + * Albert Esteve + * (based on virtio-gpio.c) + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "standard-headers/linux/virtio_config.h" +#include "../libqtest.h" +#include "qemu/module.h" +#include "qgraph.h" +#include "virtio-video.h" + +#define NUM_VQUEUES 2 +#define PCI_SLOT 0x04 +#define PCI_FN 0x00 + +static QGuestAllocator *alloc; + +static void virtio_video_cleanup(QVhostUserVideo *video) +{ + QVirtioDevice *vdev = video->vdev; + int i; + + for (i = 0; i < NUM_VQUEUES; i++) { + qvirtqueue_cleanup(vdev->bus, video->queues[i], alloc); + } + g_free(video->queues); +} + +/* + * This handles the VirtIO setup from the point of view of the driver + * frontend and therefore doesn't present any vhost specific features + * and in fact masks off the re-used bit. + */ +static void virtio_video_setup(QVhostUserVideo *video) +{ + QVirtioDevice *vdev = video->vdev; + uint64_t features; + int i; + + features = qvirtio_get_features(vdev); + features &= ~QVIRTIO_F_BAD_FEATURE; + qvirtio_set_features(vdev, features); + + video->queues = g_new(QVirtQueue *, NUM_VQUEUES); + for (i = 0; i < NUM_VQUEUES; i++) { + video->queues[i] = qvirtqueue_setup(vdev, alloc, i); + } + qvirtio_set_driver_ok(vdev); +} + +static void *qvirtio_video_get_driver(QVhostUserVideo *v_video, + const char *interface) +{ + if (!g_strcmp0(interface, "vhost-user-video")) { + return v_video; + } + if (!g_strcmp0(interface, "virtio")) { + return v_video->vdev; + } + + g_assert_not_reached(); +} + +static void *qvirtio_video_device_get_driver(void *object, + const char *interface) +{ + QVhostUserVideoDevice *v_video = object; + return qvirtio_video_get_driver(&v_video->video, interface); +} + +/* virtio-video (mmio) */ +static void qvirtio_video_device_destructor(QOSGraphObject *obj) +{ + QVhostUserVideoDevice *video_dev = (QVhostUserVideoDevice *) obj; + virtio_video_cleanup(&video_dev->video); +} + +static void qvirtio_video_device_start_hw(QOSGraphObject *obj) +{ + QVhostUserVideoDevice *video_dev = (QVhostUserVideoDevice *) obj; + virtio_video_setup(&video_dev->video); +} + +static void *virtio_video_device_create(void *virtio_dev, + QGuestAllocator *t_alloc, + void *addr) +{ + QVhostUserVideoDevice *virtio_device = g_new0(QVhostUserVideoDevice, 1); + QVhostUserVideo *interface = &virtio_device->video; + + interface->vdev = virtio_dev; + alloc = t_alloc; + + virtio_device->obj.get_driver = qvirtio_video_device_get_driver; + virtio_device->obj.start_hw = qvirtio_video_device_start_hw; + virtio_device->obj.destructor = qvirtio_video_device_destructor; + + return &virtio_device->obj; +} + +/* virtio-video-pci */ +static void qvirtio_video_pci_destructor(QOSGraphObject *obj) +{ + QVhostUserVideoPCI *video_pci = (QVhostUserVideoPCI *) obj; + QOSGraphObject *pci_vobj = &video_pci->pci_vdev.obj; + + virtio_video_cleanup(&video_pci->video); + qvirtio_pci_destructor(pci_vobj); +} + +static void qvirtio_video_pci_start_hw(QOSGraphObject *obj) +{ + QVhostUserVideoPCI *video_pci = (QVhostUserVideoPCI *) obj; + QOSGraphObject *pci_vobj = &video_pci->pci_vdev.obj; + + qvirtio_pci_start_hw(pci_vobj); + virtio_video_setup(&video_pci->video); +} + +static void *qvirtio_video_pci_get_driver(void *object, const char *interface) +{ + QVhostUserVideoPCI *v_video = object; + + if (!g_strcmp0(interface, "pci-device")) { + return v_video->pci_vdev.pdev; + } + return qvirtio_video_get_driver(&v_video->video, interface); +} + +static void *virtio_video_pci_create(void *pci_bus, QGuestAllocator *t_alloc, + void *addr) +{ + QVhostUserVideoPCI *virtio_spci = g_new0(QVhostUserVideoPCI, 1); + QVhostUserVideo *interface = &virtio_spci->video; + QOSGraphObject *obj = &virtio_spci->pci_vdev.obj; + + virtio_pci_init(&virtio_spci->pci_vdev, pci_bus, addr); + interface->vdev = &virtio_spci->pci_vdev.vdev; + alloc = t_alloc; + + obj->get_driver = qvirtio_video_pci_get_driver; + obj->start_hw = qvirtio_video_pci_start_hw; + obj->destructor = qvirtio_video_pci_destructor; + + return obj; +} + +static void virtio_video_register_nodes(void) +{ + QPCIAddress addr = { + .devfn = QPCI_DEVFN(PCI_SLOT, PCI_FN), + }; + + QOSGraphEdgeOptions edge_opts = { + .extra_device_opts = "id=video0,chardev=chr-vhost-user-test", + }; + + /* vhost-user-video-device */ + qos_node_create_driver("vhost-user-video-device", + virtio_video_device_create); + qos_node_consumes("vhost-user-video-device", "virtio-bus", &edge_opts); + qos_node_produces("vhost-user-video-device", "vhost-user-video"); + + /* vhost-user-video-pci */ + add_qpci_address(&edge_opts, &addr); + qos_node_create_driver("vhost-user-video-pci", virtio_video_pci_create); + qos_node_consumes("vhost-user-video-pci", "pci-bus", &edge_opts); + qos_node_produces("vhost-user-video-pci", "vhost-user-video"); +} + +libqos_init(virtio_video_register_nodes); diff --git a/tests/qtest/libqos/virtio-video.h b/tests/qtest/libqos/virtio-video.h new file mode 100644 index 0000000000..6ce42f5edb --- /dev/null +++ b/tests/qtest/libqos/virtio-video.h @@ -0,0 +1,39 @@ +/* + * libqos virtio-video definitions + * + * Copyright (c) 2023 Red Hat Inc. + * + * Authors: + * Albert Esteve + * (based on virtio-gpio.h) + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef TESTS_LIBQOS_VIRTIO_VIDEO_H +#define TESTS_LIBQOS_VIRTIO_VIDEO_H + +#include "qgraph.h" +#include "virtio.h" +#include "virtio-pci.h" + +typedef struct QVhostUserVideo QVhostUserVideo; +typedef struct QVhostUserVideoPCI QVhostUserVideoPCI; +typedef struct QVhostUserVideoDevice QVhostUserVideoDevice; + +struct QVhostUserVideo { + QVirtioDevice *vdev; + QVirtQueue **queues; +}; + +struct QVhostUserVideoPCI { + QVirtioPCIDevice pci_vdev; + QVhostUserVideo video; +}; + +struct QVhostUserVideoDevice { + QOSGraphObject obj; + QVhostUserVideo video; +}; + +#endif /* TESTS_LIBQOS_VIRTIO_VIDEO_H */ diff --git a/tests/qtest/vhost-user-test.c b/tests/qtest/vhost-user-test.c index bf9f7c4248..27d5c4042e 100644 --- a/tests/qtest/vhost-user-test.c +++ b/tests/qtest/vhost-user-test.c @@ -33,6 +33,7 @@ #include "standard-headers/linux/virtio_ids.h" #include "standard-headers/linux/virtio_net.h" #include "standard-headers/linux/virtio_gpio.h" +#include "standard-headers/linux/virtio_video.h" #ifdef CONFIG_LINUX #include @@ -145,6 +146,7 @@ enum { enum { VHOST_USER_NET, VHOST_USER_GPIO, + VHOST_USER_VIDEO, }; typedef struct TestServer { @@ -1156,3 +1158,33 @@ static void register_vhost_gpio_test(void) "vhost-user-gpio", test_read_guest_mem, &opts); } libqos_init(register_vhost_gpio_test); + +static uint64_t vu_video_get_features(TestServer *s) +{ + return 0x1ULL << VHOST_USER_F_PROTOCOL_FEATURES; +} + +static struct vhost_user_ops g_vu_video_ops = { + .type = VHOST_USER_VIDEO, + + .append_opts = append_vhost_gpio_opts, + + .get_features = vu_video_get_features, + .set_features = vu_net_set_features, + .get_protocol_features = vu_gpio_get_protocol_features, +}; + +static void register_vhost_video_test(void) +{ + QOSGraphTestOptions opts = { + .before = vhost_user_test_setup, + .subprocess = true, + .arg = &g_vu_video_ops, + }; + + qemu_add_opts(&qemu_chardev_opts); + + qos_add_test("read-guest-mem/memfile", + "vhost-user-video", test_read_guest_mem, &opts); +} +libqos_init(register_vhost_video_test); From patchwork Wed Mar 22 14:21:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Albert Esteve X-Patchwork-Id: 13184143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 808A2C6FD1F for ; Wed, 22 Mar 2023 14:23:05 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pezLz-0008B2-Lz; Wed, 22 Mar 2023 10:22:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLx-00087B-Ct for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:22:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLv-0007PV-RU for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:22:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679494931; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E5wvpndhRCwTda5Q3tQ0X6OyRA7fdM9iBx5DMXb0RVI=; b=c3qcJpCfSoRyY1Ekid6FoLz85/4HJKcDhJXeEKlHf6LfVbJQglya65xZfltr/C2+ebQQZ3 5RiAMS5in1f/nNPT6cyEKQGAqB/Ku3ZqSsexB2deQqWrgEGhY1mValivpBRJBaDR67hPo6 747CISgrhj1/6EarTvvN0R4on2J7Fu0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-168-j9tEj7ACMXS-bYXCPL6B2w-1; Wed, 22 Mar 2023 10:22:06 -0400 X-MC-Unique: j9tEj7ACMXS-bYXCPL6B2w-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 879FB185A792; Wed, 22 Mar 2023 14:22:06 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.226.254]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7900740C6E67; Wed, 22 Mar 2023 14:22:03 +0000 (UTC) From: Albert Esteve To: qemu-devel@nongnu.org Cc: Thomas Huth , fmartine@redhat.com, Gerd Hoffmann , eballetb@redhat.com, Albert Esteve , alex.bennee@linaro.org, Laurent Vivier , "Michael S. Tsirkin" , sgarzare@redhat.com, =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , peter.griffin@linaro.org, Paolo Bonzini Subject: [PATCH 10/12] vhost-user-video: add dev_type to CLI Date: Wed, 22 Mar 2023 15:21:30 +0100 Message-Id: <20230322142132.22909-11-aesteve@redhat.com> In-Reply-To: <20230322142132.22909-1-aesteve@redhat.com> References: <20230322142132.22909-1-aesteve@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add dev_type to the command line interface. This way we can select dev_type=encoder or dev_type=decoder and assign the virtio ID appropiately. If ommited, default virtio ID used is decoder's ID. Example: -device vhost-user-video-pci,chardev=video,dev_type=encoder,id=video Signed-off-by: Albert Esteve --- docs/system/devices/vhost-user-video.rst | 2 +- hw/display/vhost-user-video.c | 13 +++++++++++-- include/hw/virtio/vhost-user-video.h | 1 + 3 files changed, 13 insertions(+), 3 deletions(-) diff --git a/docs/system/devices/vhost-user-video.rst b/docs/system/devices/vhost-user-video.rst index ff0a8fe5c7..d428a773e2 100644 --- a/docs/system/devices/vhost-user-video.rst +++ b/docs/system/devices/vhost-user-video.rst @@ -72,7 +72,7 @@ use to communicate as well as share the guests memory over a memfd. :: host# qemu-system \ - -device vhost-user-video-pci,chardev=video,id=video \ + -device vhost-user-video-pci,chardev=video,dev_type=decoder,id=video \ -chardev socket,path=/tmp//video.sock,id=video \ -m 4096 \ -object memory-backend-file,id=mem,size=4G,mem-path=/dev/shm,share=on \ diff --git a/hw/display/vhost-user-video.c b/hw/display/vhost-user-video.c index 9cc6a717d5..469e9e7c89 100644 --- a/hw/display/vhost-user-video.c +++ b/hw/display/vhost-user-video.c @@ -300,8 +300,16 @@ static void vhost_user_video_device_realize(DeviceState *dev, Error **errp) return; } - /* TODO Implement VIDEO_ENC, currently only support VIDEO_DEC */ - virtio_init(vdev, VIRTIO_ID_VIDEO_DECODER, sizeof(struct virtio_video_config)); + if (video->conf.type == NULL || !strcmp(video->conf.type, "decoder")) { + virtio_init(vdev, VIRTIO_ID_VIDEO_DECODER, + sizeof(struct virtio_video_config)); + } else if (!strcmp(video->conf.type, "encoder")) { + virtio_init(vdev, VIRTIO_ID_VIDEO_ENCODER, + sizeof(struct virtio_video_config)); + } else { + error_report("invalid type received: %s", video->conf.type); + goto vhost_dev_init_failed; + } /* one command queue and one event queue */ video->vhost_dev.nvqs = 2; @@ -375,6 +383,7 @@ static const VMStateDescription vhost_user_video_vmstate = { static Property vhost_user_video_properties[] = { DEFINE_PROP_CHR("chardev", VHostUserVIDEO, conf.chardev), + DEFINE_PROP_STRING("dev_type", VHostUserVIDEO, conf.type), DEFINE_PROP_END_OF_LIST(), }; diff --git a/include/hw/virtio/vhost-user-video.h b/include/hw/virtio/vhost-user-video.h index f8329c3b36..3fe00b79a6 100644 --- a/include/hw/virtio/vhost-user-video.h +++ b/include/hw/virtio/vhost-user-video.h @@ -22,6 +22,7 @@ typedef struct { CharBackend chardev; + char *type; struct virtio_video_config config; } VHostUserVIDEOConf; From patchwork Wed Mar 22 14:21:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Albert Esteve X-Patchwork-Id: 13184141 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 17227C77B61 for ; Wed, 22 Mar 2023 14:22:28 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pezM0-0008CD-HU; Wed, 22 Mar 2023 10:22:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLy-00089h-VM for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:22:14 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLw-0007Pg-WE for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:22:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679494932; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2qfQPFMXuKd9XuujS5PGKZgJ3H66W4t4FFJ46QJTk7Y=; b=ait4S1FhVjczL+hnRo78JW7NHJQQd6p5Ueibybc0KSfZ4kKmjplF75WnlK+H46/D5OWEKN jSxKAi7cN6vrHnqJmo/RpdxRmnEnCt4xZqPmC6iYnje2AEMUxtUlO8Utw7SKVzPC3HmoMh bjhedPrLmhVSp1Lgzo5bxS7uihlzHQA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-655-cYJcQXB3OvOYhHN1IxdqkQ-1; Wed, 22 Mar 2023 10:22:09 -0400 X-MC-Unique: cYJcQXB3OvOYhHN1IxdqkQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 54AB4101A54F; Wed, 22 Mar 2023 14:22:09 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.226.254]) by smtp.corp.redhat.com (Postfix) with ESMTP id D4BDE40C6E67; Wed, 22 Mar 2023 14:22:06 +0000 (UTC) From: Albert Esteve To: qemu-devel@nongnu.org Cc: Thomas Huth , fmartine@redhat.com, Gerd Hoffmann , eballetb@redhat.com, Albert Esteve , alex.bennee@linaro.org, Laurent Vivier , "Michael S. Tsirkin" , sgarzare@redhat.com, =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , peter.griffin@linaro.org, Paolo Bonzini Subject: [PATCH 11/12] vhost-user-video-udmabuf: add udmabuf helpers Date: Wed, 22 Mar 2023 15:21:31 +0100 Message-Id: <20230322142132.22909-12-aesteve@redhat.com> In-Reply-To: <20230322142132.22909-1-aesteve@redhat.com> References: <20230322142132.22909-1-aesteve@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.129.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add helper for handling user-created DMA buffers through the udmabuf device. Buffers are stored in a hash table and associated to an UUID, so that they can be looked up later from the backend. Signed-off-by: Albert Esteve --- tools/vhost-user-video/meson.build | 2 +- tools/vhost-user-video/virtio_video_helpers.h | 32 +++- tools/vhost-user-video/virtio_video_udmabuf.c | 180 ++++++++++++++++++ tools/vhost-user-video/vuvideo.h | 10 +- 4 files changed, 221 insertions(+), 3 deletions(-) create mode 100644 tools/vhost-user-video/virtio_video_udmabuf.c diff --git a/tools/vhost-user-video/meson.build b/tools/vhost-user-video/meson.build index bf3b958dc6..87c70091e6 100644 --- a/tools/vhost-user-video/meson.build +++ b/tools/vhost-user-video/meson.build @@ -1,5 +1,5 @@ executable('vhost-user-video', files( - 'vhost-user-video.c', 'v4l2_backend.c', 'virtio_video_helpers.c'), + 'vhost-user-video.c', 'v4l2_backend.c', 'virtio_video_helpers.c', 'virtio_video_udmabuf.c'), dependencies: [qemuutil, glib, gio, vhost_user], install: true, install_dir: get_option('libexecdir')) diff --git a/tools/vhost-user-video/virtio_video_helpers.h b/tools/vhost-user-video/virtio_video_helpers.h index 8f35ccc4b5..171c74fc00 100644 --- a/tools/vhost-user-video/virtio_video_helpers.h +++ b/tools/vhost-user-video/virtio_video_helpers.h @@ -21,17 +21,37 @@ #include #include "libvhost-user-glib.h" #include "libvhost-user.h" +#include "qemu/uuid.h" +#include "qemu/queue.h" /* * Structure to track internal state of VIDEO Device */ +struct resource; +struct VuVideoDMABuf; + +struct vuvbm_device { + bool opened; + int fd; + + bool (*alloc_bm)(struct VuVideoDMABuf *buf); + void (*free_bm)(struct VuVideoDMABuf *buf); + int (*get_fd)(struct VuVideoDMABuf *buf); + bool (*map_bm)(struct VuVideoDMABuf *buf); + void (*unmap_bm)(struct VuVideoDMABuf *buf); + void (*device_destroy)(struct vuvbm_device *dev); + + GHashTable *resource_uuids; +}; + typedef struct VuVideo { VugDev dev; struct virtio_video_config virtio_config; GMainLoop *loop; struct v4l2_device *v4l2_dev; GList *streams; + struct vuvbm_device *bm_dev; } VuVideo; struct v4l2_device { @@ -56,10 +76,18 @@ struct vu_video_ctrl_command { }; +typedef struct VuVideoDMABuf { + struct vuvbm_device *dev; + int memfd; + int dmafd; + + void *start; + size_t length; +} VuVideoDMABuf; + /* * Structure to track internal state of a Stream */ - struct stream { struct virtio_video_stream_create vio_stream; uint32_t stream_id; @@ -89,11 +117,13 @@ struct stream { struct resource { uint32_t stream_id; + QemuUUID uuid; struct virtio_video_resource_create vio_resource; struct virtio_video_resource_queue vio_res_q; struct iovec *iov; uint32_t iov_count; uint32_t v4l2_index; + struct VuVideoDMABuf *buf; enum v4l2_buf_type type; struct vu_video_ctrl_command *vio_q_cmd; bool queued; diff --git a/tools/vhost-user-video/virtio_video_udmabuf.c b/tools/vhost-user-video/virtio_video_udmabuf.c new file mode 100644 index 0000000000..4fda1b5a78 --- /dev/null +++ b/tools/vhost-user-video/virtio_video_udmabuf.c @@ -0,0 +1,180 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* + * Virtio Video Device + * + * Copyright Red Hat, Inc. 2023 + * + * Authors: + * Albert Esteve + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#include "qemu/osdep.h" +#include +#include +#include "linux/udmabuf.h" + +#include "vuvideo.h" + +static size_t +udmabuf_get_size(struct VuVideoDMABuf *buf) +{ + return ROUND_UP(buf->length, qemu_real_host_page_size()); +} + +static bool +udmabuf_alloc_bm(struct VuVideoDMABuf *buf) +{ + int ret; + + buf->memfd = memfd_create("udmabuf-video-bm", MFD_ALLOW_SEALING); + if (buf->memfd < 0) { + g_printerr("%s: memfd_create failed.", __func__); + return false; + } + + ret = ftruncate(buf->memfd, udmabuf_get_size(buf)); + if (ret < 0) { + g_printerr("%s: ftruncate failed.", __func__); + close(buf->memfd); + return false; + } + + ret = fcntl(buf->memfd, F_ADD_SEALS, F_SEAL_SHRINK); + if (ret < 0) { + g_printerr("%s: fcntl failed.", __func__); + close(buf->memfd); + return false; + } + + return true; +} + +static void +udmabuf_free_bm(struct VuVideoDMABuf *buf) +{ + close(buf->memfd); +} + +static bool +udmabuf_map_bm(struct VuVideoDMABuf *buf) +{ + g_debug("Map the buffer memory."); + buf->start = mmap(NULL, udmabuf_get_size(buf), + PROT_READ | PROT_WRITE, MAP_SHARED, buf->memfd, 0); + if (buf->start == MAP_FAILED) { + return false; + } + + return true; +} + +static void +udmabuf_unmap_bm(struct VuVideoDMABuf *buf) +{ + g_debug("Unmap the buffer memory."); + munmap(buf->start, udmabuf_get_size(buf)); +} + +static int +udmabuf_get_fd(struct VuVideoDMABuf *buf) +{ + if (buf->dmafd > 0) { + return buf->dmafd; + } + + struct udmabuf_create create = { + .memfd = buf->memfd, + .offset = 0, + .size = udmabuf_get_size(buf), + }; + + buf->dmafd = ioctl(buf->dev->fd, UDMABUF_CREATE, &create); + if (buf->dmafd < 0) { + g_printerr("%s: UDMABUF_CREATE failed.", __func__); + } + + return buf->dmafd; +} + +static void +udmabuf_device_destroy(struct vuvbm_device *dev) +{ + close(dev->fd); +} + +static bool +vuvbm_buffer_map(struct VuVideoDMABuf *buf) +{ + struct vuvbm_device *dev = buf->dev; + + return dev->map_bm(buf); +} + +bool vuvbm_buffer_create(struct vuvbm_device *dev, + struct VuVideoDMABuf *buffer, uint32_t len) +{ + g_debug("Creating buffer length(%d)", len); + buffer->dev = dev; + buffer->length = len; + if (!dev->alloc_bm(buffer)) { + g_warning("alloc_bm failed"); + return false; + } + + if (!vuvbm_buffer_map(buffer)) { + g_warning("map_bm failed"); + goto err; + } + + return true; + +err: + buffer->dev->free_bm(buffer); + return false; +} + +void vuvbm_init_device(struct vuvbm_device *dev) +{ + if (!dev->opened && g_file_test("/dev/udmabuf", G_FILE_TEST_EXISTS)) { + dev->fd = open("/dev/udmabuf", O_RDWR); + if (dev->fd >= 0) { + g_debug("Using experimental udmabuf backend"); + dev->alloc_bm = udmabuf_alloc_bm; + dev->free_bm = udmabuf_free_bm; + dev->get_fd = udmabuf_get_fd; + dev->map_bm = udmabuf_map_bm; + dev->unmap_bm = udmabuf_unmap_bm; + dev->device_destroy = udmabuf_device_destroy; + dev->resource_uuids = g_hash_table_new_full( + NULL, NULL, NULL, g_free); + dev->opened = true; + } + } + g_debug("Using udmabuf backend"); +} + +struct VuVideoDMABuf *vuvbm_lookup(struct vuvbm_device *dev, QemuUUID uuid) +{ + g_debug("Lookup for buffer uuid(%s)", qemu_uuid_unparse_strdup(&uuid)); + return g_hash_table_lookup(dev->resource_uuids, &uuid); +} + +void vuvbm_buffer_destroy(struct VuVideoDMABuf *buffer) +{ + struct vuvbm_device *dev = buffer->dev; + + dev->unmap_bm(buffer); + dev->free_bm(buffer); +} + +void vuvbm_device_destroy(struct vuvbm_device *dev) +{ + if (!dev->opened) { + return; + } + + dev->device_destroy(dev); +} diff --git a/tools/vhost-user-video/vuvideo.h b/tools/vhost-user-video/vuvideo.h index d853c69682..b198b71fad 100644 --- a/tools/vhost-user-video/vuvideo.h +++ b/tools/vhost-user-video/vuvideo.h @@ -18,7 +18,6 @@ #include "virtio_video_helpers.h" #include "v4l2_backend.h" -#include "vuvideo.h" GList *get_resource_list(struct stream *s, uint32_t queue_type); void send_qclear_res_reply(gpointer data, gpointer user_data); @@ -40,4 +39,13 @@ void remove_all_resources(struct stream *s, uint32_t queue_type); void handle_queue_clear_cmd(struct VuVideo *v, struct vu_video_ctrl_command *vio_cmd); +/* virtio_video_udmabuf.c */ +bool vuvbm_buffer_create(struct vuvbm_device *dev, + struct VuVideoDMABuf *buffer, + uint32_t len); +void vuvbm_init_device(struct vuvbm_device *dev); +struct VuVideoDMABuf *vuvbm_lookup(struct vuvbm_device *dev, QemuUUID uuid); +void vuvbm_buffer_destroy(struct VuVideoDMABuf *buffer); +void vuvbm_device_destroy(struct vuvbm_device *dev); + #endif From patchwork Wed Mar 22 14:21:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Albert Esteve X-Patchwork-Id: 13184144 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 217A9C6FD1F for ; Wed, 22 Mar 2023 14:23:08 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pezM2-0008Dx-6M; Wed, 22 Mar 2023 10:22:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezM0-0008By-8g for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:22:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pezLy-0007QL-DP for qemu-devel@nongnu.org; Wed, 22 Mar 2023 10:22:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679494933; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3ovx8Bzs5alJDuN+08tHto3LmtzPDrLlRmmuw63sCws=; b=LbFJfOtFr51SkBpHIRLXNQ/ogxhOrC41sbImK8Ps9F67S+1N2nWcJSkjsOVT+Tg9MwS8kh 61dexYS57jX2P39qd3Tq1aP8E4c54cWXzhUAF1kItHNn5/MlsoyYv3iLKjoQi6xWPNVPaM 1qpLGA/KoisxIkTzzsrONn+3bW/rZDc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-416-7PEo0BQbPIGORP0xyJpnuA-1; Wed, 22 Mar 2023 10:22:12 -0400 X-MC-Unique: 7PEo0BQbPIGORP0xyJpnuA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 4D053185A7A2; Wed, 22 Mar 2023 14:22:12 +0000 (UTC) Received: from fedora.redhat.com (unknown [10.45.226.254]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9DF8940C6E67; Wed, 22 Mar 2023 14:22:09 +0000 (UTC) From: Albert Esteve To: qemu-devel@nongnu.org Cc: Thomas Huth , fmartine@redhat.com, Gerd Hoffmann , eballetb@redhat.com, Albert Esteve , alex.bennee@linaro.org, Laurent Vivier , "Michael S. Tsirkin" , sgarzare@redhat.com, =?utf-8?q?Marc-Andr=C3=A9_Lureau?= , peter.griffin@linaro.org, Paolo Bonzini Subject: [PATCH 12/12] Add support for v4l2_memory_dmabuf Date: Wed, 22 Mar 2023 15:21:32 +0100 Message-Id: <20230322142132.22909-13-aesteve@redhat.com> In-Reply-To: <20230322142132.22909-1-aesteve@redhat.com> References: <20230322142132.22909-1-aesteve@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 Received-SPF: pass client-ip=170.10.133.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Support VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT feature, by importing DMA buffers using the v4l2 API. In this patch, we do not import the buffer from any other virtio device, but just create a DMA buffer through udmabuf, and import it. However, in preparation for the case where we actually share buffers between different devices, we store the UUID and keep buffers associated to them in a hash table. Signed-off-by: Albert Esteve --- tools/vhost-user-video/v4l2_backend.c | 118 +++++++++++++++++++--- tools/vhost-user-video/v4l2_backend.h | 9 +- tools/vhost-user-video/vhost-user-video.c | 34 ++++--- 3 files changed, 131 insertions(+), 30 deletions(-) diff --git a/tools/vhost-user-video/v4l2_backend.c b/tools/vhost-user-video/v4l2_backend.c index 2f5825a733..e7c03a1c70 100644 --- a/tools/vhost-user-video/v4l2_backend.c +++ b/tools/vhost-user-video/v4l2_backend.c @@ -888,11 +888,55 @@ int video_streamoff(struct stream *s, enum v4l2_buf_type type) return ret; } +int v4l2_dmabuf_lookup_mplane(struct vuvbm_device *dev, + struct resource *res, + unsigned int iov_cnt) +{ + struct VuVideoDMABuf *buf = vuvbm_lookup(dev, res->uuid); + if (!buf) { + g_debug("Buffer not found. Creating."); + res->buf = g_malloc0(sizeof(struct VuVideoDMABuf) * iov_cnt); + for (int i = 0; i < iov_cnt; i++) { + if (!vuvbm_buffer_create(dev, &res->buf[i], res->iov[i].iov_len)) { + return -1; + } + } + g_debug("Inserting buffer into the table."); + g_hash_table_insert(dev->resource_uuids, res->buf, &res->uuid); + } else { + res->buf = buf; + g_debug("Buffer found."); + } + + return 0; +} + +int v4l2_dmabuf_lookup(struct vuvbm_device *dev, + struct resource *res, + unsigned int iov_len) +{ + struct VuVideoDMABuf *buf = vuvbm_lookup(dev, res->uuid); + if (!buf) { + g_debug("Buffer not found. Creating."); + res->buf = g_new0(struct VuVideoDMABuf, 1); + if (!vuvbm_buffer_create(dev, res->buf, iov_len)) { + return -1; + } + g_debug("Inserting buffer into the table."); + g_hash_table_insert(dev->resource_uuids, res->buf, &res->uuid); + } else { + res->buf = buf; + g_debug("Buffer found."); + } + + return 0; +} + int v4l2_queue_buffer(enum v4l2_buf_type type, enum v4l2_memory memory, struct virtio_video_resource_queue *qcmd, struct resource *res, struct stream *s, - struct v4l2_device *dev) + struct v4l2_device *dev, struct vuvbm_device *bm_dev) { struct v4l2_buffer vbuf; int ret = 0; @@ -911,22 +955,64 @@ int v4l2_queue_buffer(enum v4l2_buf_type type, convert_to_timeval(le64toh(qcmd->timestamp), &vbuf.timestamp); - if (V4L2_TYPE_IS_MULTIPLANAR(type)) { - /* for mplane length field is number of elements in planes array */ - vbuf.length = res->vio_resource.num_planes; - vbuf.m.planes = g_malloc0(sizeof(struct v4l2_plane) - * res->vio_resource.num_planes); + if (memory == V4L2_MEMORY_USERPTR) { + if (V4L2_TYPE_IS_MULTIPLANAR(type)) { + /* for mplane length field is number of elements in planes array */ + vbuf.length = res->vio_resource.num_planes; + vbuf.m.planes = g_malloc0(sizeof(struct v4l2_plane) + * res->vio_resource.num_planes); - for (int i = 0; i < vbuf.length; i++) { - vbuf.m.planes[i].m.userptr = (unsigned long)res->iov[i].iov_base; - vbuf.m.planes[i].length = (unsigned long)res->iov[i].iov_len; + for (int i = 0; i < vbuf.length; i++) { + vbuf.m.planes[i].m.userptr = + (unsigned long)res->iov[i].iov_base; + vbuf.m.planes[i].length = (unsigned long)res->iov[i].iov_len; + } + } else { + assert(res->iov != NULL); + vbuf.m.userptr = (unsigned long)res->iov[0].iov_base; + vbuf.length = res->iov[0].iov_len; + g_debug("%s: iov_base = 0x%p", __func__, res->iov[0].iov_base); + g_debug("%s: iov_len = 0x%lx", __func__, res->iov[0].iov_len); + } + } else { + assert(memory == V4L2_MEMORY_DMABUF); + if (V4L2_TYPE_IS_MULTIPLANAR(type)) { + vbuf.length = res->vio_resource.num_planes; + vbuf.m.planes = g_malloc0(sizeof(struct v4l2_plane) * vbuf.length); + if (!res->buf) { + ret = v4l2_dmabuf_lookup_mplane(bm_dev, res, vbuf.length); + if (ret < 0) { + g_warning("Buffer create failed."); + return ret; + } + } + + for (int i = 0; i < vbuf.length; i++) { + vbuf.m.planes[i].m.fd = res->buf->dev->get_fd(&res->buf[i]); + vbuf.m.planes[i].length = (unsigned long)res->iov[i].iov_len; + /* Copy virtio shared memory contents to DMA buffer */ + memcpy(res->buf[i].start, + res->iov[i].iov_base, res->iov[i].iov_len); + } + } else { + if (!res->buf) { + ret = ioctl(fd, VIDIOC_QUERYBUF, &vbuf); + if (ret == -EINVAL) { + g_printerr("VIDIOC_QUERYBUF failed: %s (%d)\n", + g_strerror(errno), errno); + return ret; + } + + ret = v4l2_dmabuf_lookup(bm_dev, res, vbuf.length); + if (ret < 0) { + g_warning("Buffer create failed."); + return ret; + } + } + vbuf.m.fd = res->buf->dev->get_fd(res->buf); + /* Copy virtio shared memory contents to DMA buffer */ + memcpy(res->buf->start, res->iov[0].iov_base, res->iov[0].iov_len); } - } else if (res->iov != NULL) { - /* m is a union of userptr, *planes and fd */ - vbuf.m.userptr = (unsigned long)res->iov[0].iov_base; - vbuf.length = res->iov[0].iov_len; - g_debug("%s: iov_base = 0x%p", __func__, res->iov[0].iov_base); - g_debug("%s: iov_len = 0x%lx", __func__, res->iov[0].iov_len); } if (V4L2_TYPE_IS_OUTPUT(type)) { @@ -979,7 +1065,7 @@ int v4l2_dequeue_buffer(int fd, enum v4l2_buf_type type, memset(&vbuf, 0, sizeof(vbuf)); vbuf.type = type; - vbuf.memory = memory; + vbuf.memory = memory; vbuf.field = V4L2_FIELD_NONE; vbuf.flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; diff --git a/tools/vhost-user-video/v4l2_backend.h b/tools/vhost-user-video/v4l2_backend.h index ef9a5f0337..05409e03d4 100644 --- a/tools/vhost-user-video/v4l2_backend.h +++ b/tools/vhost-user-video/v4l2_backend.h @@ -69,11 +69,18 @@ int v4l2_video_set_format(int fd, enum v4l2_buf_type type, int v4l2_set_pixel_format(int fd, enum v4l2_buf_type buf_type, uint32_t pixelformat); +int v4l2_dmabuf_lookup_mplane(struct vuvbm_device *dev, + struct resource *res, + unsigned int iov_cnt); +int v4l2_dmabuf_lookup(struct vuvbm_device *dev, + struct resource *res, + unsigned int iov_len); int v4l2_queue_buffer(enum v4l2_buf_type type, enum v4l2_memory memory, struct virtio_video_resource_queue *qcmd, struct resource *res, struct stream *s, - struct v4l2_device *dev); + struct v4l2_device *dev, + struct vuvbm_device *bm_dev); int v4l2_dequeue_buffer(int fd, enum v4l2_buf_type type, enum v4l2_memory memory, struct stream *s); diff --git a/tools/vhost-user-video/vhost-user-video.c b/tools/vhost-user-video/vhost-user-video.c index cee81c69c3..14c8f42c52 100644 --- a/tools/vhost-user-video/vhost-user-video.c +++ b/tools/vhost-user-video/vhost-user-video.c @@ -391,6 +391,9 @@ void remove_all_resources(struct stream *s, uint32_t queue_type) /* free resource memory allocated in resource_create() */ g_free(r->iov); + if (r->buf != NULL) { + vuvbm_buffer_destroy(r->buf); + } g_free(r); *resource_list = g_list_delete_link(*resource_list, l); } @@ -880,10 +883,16 @@ handle_resource_create_cmd(struct VuVideo *v, res->iov_count = total_entries; } break; case VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT: - g_critical("%s: VIRTIO_OBJECT not implemented!", __func__); - /* TODO implement VIRTIO_OBJECT support */ - cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; - goto out_unlock; + { + struct virtio_video_object_entry *ent; + ent = (void *)cmd + sizeof(struct virtio_video_resource_create); + + memcpy(&res->uuid, ent->uuid, sizeof(ent->uuid)); + g_debug("%s: create resource uuid(%s)", + __func__, qemu_uuid_unparse_strdup(&res->uuid)); + + vuvbm_init_device(v->bm_dev); + } break; } cmd->hdr.type = VIRTIO_VIDEO_RESP_OK_NODATA; @@ -974,7 +983,7 @@ handle_resource_queue_cmd(struct VuVideo *v, get_queue_mem_type(s, cmd->queue_type); enum v4l2_memory memory = get_v4l2_memory(mem_type); - ret = v4l2_queue_buffer(buf_type, memory, cmd, res, s, v->v4l2_dev); + ret = v4l2_queue_buffer(buf_type, memory, cmd, res, s, v->v4l2_dev, v->bm_dev); if (ret < 0) { g_critical("%s: v4l2_queue_buffer failed", __func__); /* virtio error set by v4l2_queue_buffer */ @@ -1064,12 +1073,7 @@ handle_stream_create_cmd(struct VuVideo *v, req_stream_id = cmd->hdr.stream_id; coded_format = le32toh(cmd->coded_format); - if ((le32toh(cmd->in_mem_type) == VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT) || - (le32toh(cmd->out_mem_type) == VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT)) { - /* TODO implement VIRTIO_VIDEO_MEM_TYPE_VIRTIO_OBJECT */ - g_printerr("%s: MEM_TYPE_VIRTIO_OBJECT not supported yet", __func__); - cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_PARAMETER; - } else if (find_stream(v, req_stream_id)) { + if (find_stream(v, req_stream_id)) { g_debug("%s: Stream ID in use - ", __func__); cmd->hdr.type = VIRTIO_VIDEO_RESP_ERR_INVALID_STREAM_ID; } else { @@ -1212,7 +1216,6 @@ handle_stream_destroy_cmd(struct VuVideo *v, buftype = s->has_mplane ? V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE : V4L2_BUF_TYPE_VIDEO_CAPTURE; - mem_type = get_queue_mem_type(s, VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT); video_free_buffers(s->fd, buftype, get_v4l2_memory(mem_type)); remove_all_resources(s, VIRTIO_VIDEO_QUEUE_TYPE_OUTPUT); @@ -1644,7 +1647,7 @@ static void video_destroy(VuVideo *v) if (socket_path) { unlink(socket_path); } - + vuvbm_device_destroy(v->bm_dev); v4l2_backend_free(v->v4l2_dev); } @@ -1717,6 +1720,11 @@ int main(int argc, char *argv[]) } } + /* + * Create a new Buffer Memory device to handle DMA buffers. + */ + video.bm_dev = g_new0(struct vuvbm_device, 1); + /* * Now create a vhost-user socket that we will receive messages * on. Once we have our handler set up we can enter the glib main