| Message ID | 20210213101512.3275069-1-mperttunen@nvidia.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | NVIDIA Tegra NVDEC support | expand |
Hi, On 13/02/2021 11:15, Mikko Perttunen wrote: > Hi all, > > with the release of documentation headers for Tegra multimedia engines > (NVDEC, NVENC, NVJPG) [1], I have started working on the corresponding > implementations. Here's the first one, NVDEC. > > The kernel driver is a simple Falcon boot driver based on the VIC > driver. Some code sharing should be considered there in the future. > The userspace driver to accompany this is a bit more complicated - > I have expanded vaapi-tegra-driver[2] to support MPEG2 decoding. > It should be noted that the implementation is still very clunky > and has poor performance, but it's a start. Funny how all this tries to avoid all the DRM, remoteproc, V4L2-M2M stateless & co all the other vendors tries to make usage of... Neil > > This series is based on top of the "Host1x/TegraDRM UAPI" series. > For testing, appropriate firmware should be obtained from a > Linux for Tegra distribution for now; the GPU should also be > enabled in the device tree. > > Series was tested on Tegra186. > > Thanks! > > Mikko > > [1] https://github.com/NVIDIA/open-gpu-doc/tree/master/classes/video > [2] https://github.com/cyndis/vaapi-tegra-driver > > Mikko Perttunen (3): > dt-bindings: Add YAML bindings for Host1x and NVDEC > arm64: tegra: Add NVDEC to Tegra186 device tree > drm/tegra: Add NVDEC driver > > .../gpu/host1x/nvidia,tegra20-host1x.yaml | 129 +++++ > .../gpu/host1x/nvidia,tegra210-nvdec.yaml | 90 ++++ > MAINTAINERS | 1 + > arch/arm64/boot/dts/nvidia/tegra186.dtsi | 15 + > drivers/gpu/drm/tegra/Makefile | 3 +- > drivers/gpu/drm/tegra/drm.c | 4 + > drivers/gpu/drm/tegra/drm.h | 1 + > drivers/gpu/drm/tegra/nvdec.c | 497 ++++++++++++++++++ > drivers/gpu/host1x/dev.c | 12 + > include/linux/host1x.h | 1 + > 10 files changed, 752 insertions(+), 1 deletion(-) > create mode 100644 Documentation/devicetree/bindings/gpu/host1x/nvidia,tegra20-host1x.yaml > create mode 100644 Documentation/devicetree/bindings/gpu/host1x/nvidia,tegra210-nvdec.yaml > create mode 100644 drivers/gpu/drm/tegra/nvdec.c >
On Mon, Feb 15, 2021 at 10:10:26AM +0100, Neil Armstrong wrote: > Hi, > > On 13/02/2021 11:15, Mikko Perttunen wrote: > > Hi all, > > > > with the release of documentation headers for Tegra multimedia engines > > (NVDEC, NVENC, NVJPG) [1], I have started working on the corresponding > > implementations. Here's the first one, NVDEC. > > > > The kernel driver is a simple Falcon boot driver based on the VIC > > driver. Some code sharing should be considered there in the future. > > The userspace driver to accompany this is a bit more complicated - > > I have expanded vaapi-tegra-driver[2] to support MPEG2 decoding. > > It should be noted that the implementation is still very clunky > > and has poor performance, but it's a start. > > Funny how all this tries to avoid all the DRM, remoteproc, V4L2-M2M stateless & co > all the other vendors tries to make usage of... Care to elaborate why you think this is trying to avoid anything? Mikko pointed you at the documentation for these engines, provided a link to an open-source (albeit work in progress) userspace driver and posts an extension to an existing DRM driver to add the required kernel functionality. That's a standard approach for submitting this kind of driver. Thierry
Hi Thierry, On 15/02/2021 10:50, Thierry Reding wrote: > On Mon, Feb 15, 2021 at 10:10:26AM +0100, Neil Armstrong wrote: >> Hi, >> >> On 13/02/2021 11:15, Mikko Perttunen wrote: >>> Hi all, >>> >>> with the release of documentation headers for Tegra multimedia engines >>> (NVDEC, NVENC, NVJPG) [1], I have started working on the corresponding >>> implementations. Here's the first one, NVDEC. >>> >>> The kernel driver is a simple Falcon boot driver based on the VIC >>> driver. Some code sharing should be considered there in the future. >>> The userspace driver to accompany this is a bit more complicated - >>> I have expanded vaapi-tegra-driver[2] to support MPEG2 decoding. >>> It should be noted that the implementation is still very clunky >>> and has poor performance, but it's a start. >> >> Funny how all this tries to avoid all the DRM, remoteproc, V4L2-M2M stateless & co >> all the other vendors tries to make usage of... > > Care to elaborate why you think this is trying to avoid anything? Mikko > pointed you at the documentation for these engines, provided a link to > an open-source (albeit work in progress) userspace driver and posts an > extension to an existing DRM driver to add the required kernel > functionality. That's a standard approach for submitting this kind of > driver. Thanks for the reply, I didn't look extensively at all the documents & userspace libraries, but I wonder why this couldn't fit in the V4L2-M2M approach and avoid having userspace drivers and specific libraries to handle this. Neil > > Thierry >
On Mon, Feb 15, 2021 at 11:21:27AM +0100, Neil Armstrong wrote: > Hi Thierry, > > > On 15/02/2021 10:50, Thierry Reding wrote: > > On Mon, Feb 15, 2021 at 10:10:26AM +0100, Neil Armstrong wrote: > >> Hi, > >> > >> On 13/02/2021 11:15, Mikko Perttunen wrote: > >>> Hi all, > >>> > >>> with the release of documentation headers for Tegra multimedia engines > >>> (NVDEC, NVENC, NVJPG) [1], I have started working on the corresponding > >>> implementations. Here's the first one, NVDEC. > >>> > >>> The kernel driver is a simple Falcon boot driver based on the VIC > >>> driver. Some code sharing should be considered there in the future. > >>> The userspace driver to accompany this is a bit more complicated - > >>> I have expanded vaapi-tegra-driver[2] to support MPEG2 decoding. > >>> It should be noted that the implementation is still very clunky > >>> and has poor performance, but it's a start. > >> > >> Funny how all this tries to avoid all the DRM, remoteproc, V4L2-M2M stateless & co > >> all the other vendors tries to make usage of... > > > > Care to elaborate why you think this is trying to avoid anything? Mikko > > pointed you at the documentation for these engines, provided a link to > > an open-source (albeit work in progress) userspace driver and posts an > > extension to an existing DRM driver to add the required kernel > > functionality. That's a standard approach for submitting this kind of > > driver. > > Thanks for the reply, I didn't look extensively at all the documents & userspace > libraries, but I wonder why this couldn't fit in the V4L2-M2M approach and avoid > having userspace drivers and specific libraries to handle this. Ah, I see. Without going too much into the details, the reason for this is that the multimedia engines on Tegra use host1x for command submission. host1x is roughly a command stream parser with built-in capability for synchronization and (in newer generations) process isolation, etc. This same engine is used for things like 2D and 3D acceleration on older chips and there are other hardware blocks that use it, such as the video image compositor (used for some post-processing tasks). The GPU can also interoperate with host1x for synchronization with these multimedia engines. The userspace interfaces for 2D and 3D have existed for a long time, and the fundamental programming sequences are largely the same, so we chose to use the same interface for simplicity rather that duplicating most of this into the kernel. Constructing these command streams can be fairly complicated and a number of extra data structures are needed for each command. Putting all of that into userspace reduces the potential for bugs and crashes in the kernel that may take down the whole system. As for the userspace drivers argument, this isn't adding anything new but rather provides a driver for the existing VAAPI library that's already widely supported in multimedia applications. Thierry