| Message ID | 20231105165521.3592037-1-jonas@kwiboo.se (mailing list archive) |
|---|---|
| Headers | show |
| Series | media: rkvdec: Add H.264 High 10 and 4:2:2 profile support | expand |
Hi Jonas, thanks for this work. Le dimanche 05 novembre 2023 à 16:54 +0000, Jonas Karlman a écrit : > This is a revival of a 3 year old series [1] now that NV15/NV20/NV30 support > for display driver have landed in mainline tree. > > This series adds H.264 High 10 and 4:2:2 profile support to the Rockchip > Video Decoder driver. > > Patch 1 adds helpers for calculating plane bytesperline and sizeimage. > Patch 2 adds two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV. > > Patch 3 change to use bytesperline and buffer height to configure strides. > Patch 4 change to use values from SPS/PPS control to configure the HW. > Patch 5 remove an unnecessary call to validate sps at streaming start. > > Patch 6-10 refactor code to support filtering of CAPUTRE formats based > on the image format returned from a get_image_fmt ops. > > Patch 11 adds final bits to support H.264 High 10 and 4:2:2 profiles. > > Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328): > > v4l2-compliance 1.24.1, 64 bits, 64-bit time_t > ... > Total for rkvdec device /dev/video1: 46, Succeeded: 46, Failed: 0, Warnings: 0 > > Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request > ... > Ran 65/69 tests successfully I added GStreamer support for these formats and could confirm this results with GStreamer too. https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5612 For the set: Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com> > > Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-V4L2-request > ... > Ran 127/135 tests successfully I'm getting 129 with GStreamer, which matches also with FFMpeg software decoder. > > Before this series: > > Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request > ... > Ran 44/69 tests successfully I had 43 with GStreamer, but then I notice you fixed something in fluster. The good news this is now upstream, thanks for spotting. https://github.com/fluendo/fluster/pull/148 > > Changes in v4: > - Fix failed v4l2-compliance tests related to CAPTURE queue > - Rework CAPTURE format filter anv validate to use an image format > - Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5] > Link to v3: https://lore.kernel.org/linux-media/20231029183427.1781554-1-jonas@kwiboo.se/ > > Changes in v3: > - Drop merged patches > - Use bpp and bpp_div instead of prior misuse of block_w/block_h > - New patch to use values from SPS/PPS control to configure the HW > - New patch to remove an unnecessary call to validate sps at streaming start > - Reworked pixel format validation > Link to v2: https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/ > > Changes in v2: > - Collect r-b tags > - SPS pic width and height in mbs validation moved to rkvdec_try_ctrl > - New patch to not override output buffer sizeimage > - Reworked pixel format validation > - Only align decoded buffer instead of changing frmsize step_width > Link to v1: https://lore.kernel.org/linux-media/20200701215616.30874-1-jonas@kwiboo.se/ > > Following commits adds support for NV15/NV20/NV30 to VOP driver: > 728c15b4b5f3 ("drm/fourcc: Add NV20 and NV30 YUV formats") > d4b384228562 ("drm/rockchip: vop: Add NV15, NV20 and NV30 support") > > To fully runtime test this series you may need above drm commits and ffmpeg > patches from [2], this series and drm patches is also available at [3]. > > [1] https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/ > [2] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-n6.1-dev/ > [3] https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v4/ > [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865 > [5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf > > Regards, > Jonas > > Alex Bee (1): > media: rkvdec: h264: Don't hardcode SPS/PPS parameters > > Jonas Karlman (10): > media: v4l2-common: Add helpers to calculate bytesperline and > sizeimage > media: v4l2: Add NV15 and NV20 pixel formats > media: rkvdec: h264: Use bytesperline and buffer height as virstride > media: rkvdec: h264: Remove SPS validation at streaming start > media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper > media: rkvdec: Move rkvdec_reset_decoded_fmt helper > media: rkvdec: Extract decoded format enumeration into helper > media: rkvdec: Add image format concept > media: rkvdec: Add get_image_fmt ops > media: rkvdec: h264: Support High 10 and 4:2:2 profiles > > .../media/v4l/pixfmt-yuv-planar.rst | 128 +++++++++++ > drivers/media/v4l2-core/v4l2-common.c | 80 +++---- > drivers/media/v4l2-core/v4l2-ioctl.c | 2 + > drivers/staging/media/rkvdec/rkvdec-h264.c | 83 +++---- > drivers/staging/media/rkvdec/rkvdec.c | 217 +++++++++++++----- > drivers/staging/media/rkvdec/rkvdec.h | 18 +- > include/uapi/linux/videodev2.h | 2 + > 7 files changed, 396 insertions(+), 134 deletions(-) >
Hi Nicolas, On 2023-11-07 22:43, Nicolas Dufresne wrote: > Hi Jonas, > > thanks for this work. > > Le dimanche 05 novembre 2023 à 16:54 +0000, Jonas Karlman a écrit : >> This is a revival of a 3 year old series [1] now that NV15/NV20/NV30 support >> for display driver have landed in mainline tree. >> >> This series adds H.264 High 10 and 4:2:2 profile support to the Rockchip >> Video Decoder driver. >> >> Patch 1 adds helpers for calculating plane bytesperline and sizeimage. >> Patch 2 adds two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV. >> >> Patch 3 change to use bytesperline and buffer height to configure strides. >> Patch 4 change to use values from SPS/PPS control to configure the HW. >> Patch 5 remove an unnecessary call to validate sps at streaming start. >> >> Patch 6-10 refactor code to support filtering of CAPUTRE formats based >> on the image format returned from a get_image_fmt ops. >> >> Patch 11 adds final bits to support H.264 High 10 and 4:2:2 profiles. >> >> Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328): >> >> v4l2-compliance 1.24.1, 64 bits, 64-bit time_t >> ... >> Total for rkvdec device /dev/video1: 46, Succeeded: 46, Failed: 0, Warnings: 0 >> >> Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request >> ... >> Ran 65/69 tests successfully > > I added GStreamer support for these formats and could confirm this > results with GStreamer too. > > https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5612 Nice! > > > For the set: > Tested-by: Nicolas Dufresne <nicolas.dufresne@collabora.com> > >> >> Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-V4L2-request >> ... >> Ran 127/135 tests successfully > > I'm getting 129 with GStreamer, which matches also with FFMpeg software > decoder. Great, I was able to track down a long term ref idx issue in the ffmpeg v4l2-request implementation and can now also get to 129 :-) > >> >> Before this series: >> >> Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request >> ... >> Ran 44/69 tests successfully > > I had 43 with GStreamer, but then I notice you fixed something in > fluster. The good news this is now upstream, thanks for spotting. > > https://github.com/fluendo/fluster/pull/148 Thanks for sending the PR! Regards, Jonas > >> >> Changes in v4: >> - Fix failed v4l2-compliance tests related to CAPTURE queue >> - Rework CAPTURE format filter anv validate to use an image format >> - Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5] >> Link to v3: https://lore.kernel.org/linux-media/20231029183427.1781554-1-jonas@kwiboo.se/ >> >> Changes in v3: >> - Drop merged patches >> - Use bpp and bpp_div instead of prior misuse of block_w/block_h >> - New patch to use values from SPS/PPS control to configure the HW >> - New patch to remove an unnecessary call to validate sps at streaming start >> - Reworked pixel format validation >> Link to v2: https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/ >> >> Changes in v2: >> - Collect r-b tags >> - SPS pic width and height in mbs validation moved to rkvdec_try_ctrl >> - New patch to not override output buffer sizeimage >> - Reworked pixel format validation >> - Only align decoded buffer instead of changing frmsize step_width >> Link to v1: https://lore.kernel.org/linux-media/20200701215616.30874-1-jonas@kwiboo.se/ >> >> Following commits adds support for NV15/NV20/NV30 to VOP driver: >> 728c15b4b5f3 ("drm/fourcc: Add NV20 and NV30 YUV formats") >> d4b384228562 ("drm/rockchip: vop: Add NV15, NV20 and NV30 support") >> >> To fully runtime test this series you may need above drm commits and ffmpeg >> patches from [2], this series and drm patches is also available at [3]. >> >> [1] https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/ >> [2] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-n6.1-dev/ >> [3] https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v4/ >> [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865 >> [5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf >> >> Regards, >> Jonas >> >> Alex Bee (1): >> media: rkvdec: h264: Don't hardcode SPS/PPS parameters >> >> Jonas Karlman (10): >> media: v4l2-common: Add helpers to calculate bytesperline and >> sizeimage >> media: v4l2: Add NV15 and NV20 pixel formats >> media: rkvdec: h264: Use bytesperline and buffer height as virstride >> media: rkvdec: h264: Remove SPS validation at streaming start >> media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper >> media: rkvdec: Move rkvdec_reset_decoded_fmt helper >> media: rkvdec: Extract decoded format enumeration into helper >> media: rkvdec: Add image format concept >> media: rkvdec: Add get_image_fmt ops >> media: rkvdec: h264: Support High 10 and 4:2:2 profiles >> >> .../media/v4l/pixfmt-yuv-planar.rst | 128 +++++++++++ >> drivers/media/v4l2-core/v4l2-common.c | 80 +++---- >> drivers/media/v4l2-core/v4l2-ioctl.c | 2 + >> drivers/staging/media/rkvdec/rkvdec-h264.c | 83 +++---- >> drivers/staging/media/rkvdec/rkvdec.c | 217 +++++++++++++----- >> drivers/staging/media/rkvdec/rkvdec.h | 18 +- >> include/uapi/linux/videodev2.h | 2 + >> 7 files changed, 396 insertions(+), 134 deletions(-) >> >
Hi Jonas, Happy new year! On Sun, 2023-11-05 at 16:54 +0000, Jonas Karlman wrote: > This is a revival of a 3 year old series [1] now that NV15/NV20/NV30 support > for display driver have landed in mainline tree. > > This series adds H.264 High 10 and 4:2:2 profile support to the Rockchip > Video Decoder driver. > > Patch 1 adds helpers for calculating plane bytesperline and sizeimage. > Patch 2 adds two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV. > > Patch 3 change to use bytesperline and buffer height to configure strides. > Patch 4 change to use values from SPS/PPS control to configure the HW. > Patch 5 remove an unnecessary call to validate sps at streaming start. > > Patch 6-10 refactor code to support filtering of CAPUTRE formats based > on the image format returned from a get_image_fmt ops. > > Patch 11 adds final bits to support H.264 High 10 and 4:2:2 profiles. I send my Tested-by to v3 of this series, I also tested with v4 so for future series please add: Tested-by: Christopher Obbard <chris.obbard@collabora.com> > > Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328): > > v4l2-compliance 1.24.1, 64 bits, 64-bit time_t > ... > Total for rkvdec device /dev/video1: 46, Succeeded: 46, Failed: 0, > Warnings: 0 > > Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request > ... > Ran 65/69 tests successfully > > Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-V4L2-request > ... > Ran 127/135 tests successfully > > Before this series: > > Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request > ... > Ran 44/69 tests successfully > > Changes in v4: > - Fix failed v4l2-compliance tests related to CAPTURE queue > - Rework CAPTURE format filter anv validate to use an image format > - Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5] > Link to v3: > https://lore.kernel.org/linux-media/20231029183427.1781554-1-jonas@kwiboo.se/ > > Changes in v3: > - Drop merged patches > - Use bpp and bpp_div instead of prior misuse of block_w/block_h > - New patch to use values from SPS/PPS control to configure the HW > - New patch to remove an unnecessary call to validate sps at streaming start > - Reworked pixel format validation > Link to v2: > https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/ > > Changes in v2: > - Collect r-b tags > - SPS pic width and height in mbs validation moved to rkvdec_try_ctrl > - New patch to not override output buffer sizeimage > - Reworked pixel format validation > - Only align decoded buffer instead of changing frmsize step_width > Link to v1: > https://lore.kernel.org/linux-media/20200701215616.30874-1-jonas@kwiboo.se/ > > Following commits adds support for NV15/NV20/NV30 to VOP driver: > 728c15b4b5f3 ("drm/fourcc: Add NV20 and NV30 YUV formats") > d4b384228562 ("drm/rockchip: vop: Add NV15, NV20 and NV30 support") > > To fully runtime test this series you may need above drm commits and ffmpeg > patches from [2], this series and drm patches is also available at [3]. > > [1] > https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/ > [2] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-n6.1-dev/ > [3] > https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v4/ > [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865 > [5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf > > Regards, > Jonas > > Alex Bee (1): > media: rkvdec: h264: Don't hardcode SPS/PPS parameters > > Jonas Karlman (10): > media: v4l2-common: Add helpers to calculate bytesperline and > sizeimage > media: v4l2: Add NV15 and NV20 pixel formats > media: rkvdec: h264: Use bytesperline and buffer height as virstride > media: rkvdec: h264: Remove SPS validation at streaming start > media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper > media: rkvdec: Move rkvdec_reset_decoded_fmt helper > media: rkvdec: Extract decoded format enumeration into helper > media: rkvdec: Add image format concept > media: rkvdec: Add get_image_fmt ops > media: rkvdec: h264: Support High 10 and 4:2:2 profiles > > .../media/v4l/pixfmt-yuv-planar.rst | 128 +++++++++++ > drivers/media/v4l2-core/v4l2-common.c | 80 +++---- > drivers/media/v4l2-core/v4l2-ioctl.c | 2 + > drivers/staging/media/rkvdec/rkvdec-h264.c | 83 +++---- > drivers/staging/media/rkvdec/rkvdec.c | 217 +++++++++++++----- > drivers/staging/media/rkvdec/rkvdec.h | 18 +- > include/uapi/linux/videodev2.h | 2 + > 7 files changed, 396 insertions(+), 134 deletions(-) >
On Sunday, 5 November 2023 17:54:59 CEST Jonas Karlman wrote: > This is a revival of a 3 year old series [1] now that NV15/NV20/NV30 support > for display driver have landed in mainline tree. > > This series adds H.264 High 10 and 4:2:2 profile support to the Rockchip > Video Decoder driver. > > Patch 1 adds helpers for calculating plane bytesperline and sizeimage. > Patch 2 adds two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV. > > Patch 3 change to use bytesperline and buffer height to configure strides. > Patch 4 change to use values from SPS/PPS control to configure the HW. > Patch 5 remove an unnecessary call to validate sps at streaming start. > > Patch 6-10 refactor code to support filtering of CAPUTRE formats based > on the image format returned from a get_image_fmt ops. > > Patch 11 adds final bits to support H.264 High 10 and 4:2:2 profiles. > > Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328): > > v4l2-compliance 1.24.1, 64 bits, 64-bit time_t > ... > Total for rkvdec device /dev/video1: 46, Succeeded: 46, Failed: 0, > Warnings: 0 > > Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request > ... > Ran 65/69 tests successfully > > Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-V4L2-request > ... > Ran 127/135 tests successfully > > Before this series: > > Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request > ... > Ran 44/69 tests successfully > > ... > > Following commits adds support for NV15/NV20/NV30 to VOP driver: > 728c15b4b5f3 ("drm/fourcc: Add NV20 and NV30 YUV formats") > d4b384228562 ("drm/rockchip: vop: Add NV15, NV20 and NV30 support") > > To fully runtime test this series you may need above drm commits and ffmpeg > patches from [2], this series and drm patches is also available at [3]. > > [1] > https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/ > [2] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-n6.1-dev/ [3] > https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v4/ > [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865 [5] > https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf Reviving this old thread now that rkvdec2 'stuff' emerged. I have (actually) done quite some tests with this (and "media: rkvdec: Add HEVC backend" patch set) and they have been part of my kernel builds ever since. I _think_, but don't know, that is relevant for Andy's question: On zondag 16 juni 2024 08:58:20 CEST Andy Yan <andyshrk@163.com> wrote: > How can I test these patches? Do they require any additional userspace > patches? I have the same question and I think you'd need this and the HEVC patch set and then also patch FFmpeg and then it should enable HW acceleration. So my question boils down to: with the rkvdec2 patch set, should V4L2-requests now also work with rkvdec, so not just Hantro anymore? BTW: the libdrm commits have been merged upstream quite some time ago, so if you have a recent version of that, you don't need to patch that. If you use FFmpeg 7.0, then Jonas has a branch for that too (haven't tried it yet though). FWIW: my test results were a bit mixed. I didn't post them before as I don't fully/really understand this 'video stuff', and I didn't want you all to suffer from what was likely a PEBKAC issue. On my PineTab2 (rk3566) I had some h.264 videos HW accelerated, but not all. My guess is that it's related to the resolution. 1920x1080 worked, while it didn't work with a 1280x640 video. The video still played, just not HW accelerated. IOW: improvements in some and otherwise it was just rendered by the CPU (I think), just like before. On my Rock64 I got a pink tint with all videos, like described here: https://github.com/mpv-player/mpv/issues/12968 IIUC, that's actually a problem in the lima driver? Cheers, Diederik
Hi, Le dimanche 16 juin 2024 à 11:47 +0200, Diederik de Haas a écrit : > On Sunday, 5 November 2023 17:54:59 CEST Jonas Karlman wrote: > > This is a revival of a 3 year old series [1] now that NV15/NV20/NV30 support > > for display driver have landed in mainline tree. > > > > This series adds H.264 High 10 and 4:2:2 profile support to the Rockchip > > Video Decoder driver. > > > > Patch 1 adds helpers for calculating plane bytesperline and sizeimage. > > Patch 2 adds two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV. > > > > Patch 3 change to use bytesperline and buffer height to configure strides. > > Patch 4 change to use values from SPS/PPS control to configure the HW. > > Patch 5 remove an unnecessary call to validate sps at streaming start. > > > > Patch 6-10 refactor code to support filtering of CAPUTRE formats based > > on the image format returned from a get_image_fmt ops. > > > > Patch 11 adds final bits to support H.264 High 10 and 4:2:2 profiles. > > > > Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328): > > > > v4l2-compliance 1.24.1, 64 bits, 64-bit time_t > > ... > > Total for rkvdec device /dev/video1: 46, Succeeded: 46, Failed: 0, > > Warnings: 0 > > > > Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request > > ... > > Ran 65/69 tests successfully > > > > Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-V4L2-request > > ... > > Ran 127/135 tests successfully > > > > Before this series: > > > > Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request > > ... > > Ran 44/69 tests successfully > > > > ... > > > > Following commits adds support for NV15/NV20/NV30 to VOP driver: > > 728c15b4b5f3 ("drm/fourcc: Add NV20 and NV30 YUV formats") > > d4b384228562 ("drm/rockchip: vop: Add NV15, NV20 and NV30 support") > > > > To fully runtime test this series you may need above drm commits and ffmpeg > > patches from [2], this series and drm patches is also available at [3]. > > > > [1] > > https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/ > > [2] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-n6.1-dev/ [3] > > https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v4/ > > [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865 [5] > > https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf > > Reviving this old thread now that rkvdec2 'stuff' emerged. > I have (actually) done quite some tests with this (and "media: rkvdec: Add > HEVC backend" patch set) and they have been part of my kernel builds ever > since. > I _think_, but don't know, that is relevant for Andy's question: > > On zondag 16 juni 2024 08:58:20 CEST Andy Yan <andyshrk@163.com> wrote: > > How can I test these patches? Do they require any additional userspace > > patches? > > I have the same question and I think you'd need this and the HEVC patch set > and then also patch FFmpeg and then it should enable HW acceleration. > So my question boils down to: with the rkvdec2 patch set, should V4L2-requests > now also work with rkvdec, so not just Hantro anymore? FFmpeg changes are still downstream, and different people (even within LibreELEC) seems to have slightly different version or alteration. It would be really nice if this work could move upstream FFMpeg so that we can be more sure what what "working with FFmpeg v4l2-requests" means. Meanwhile, support in upstream GStreamer is stable on Hantro G2 and Mediatek VCODEC. In theory, it works fine with RKVDEC, and it will certainly work with RKVDEC2 when we get to write the HEVC support. > > BTW: the libdrm commits have been merged upstream quite some time ago, so if > you have a recent version of that, you don't need to patch that. > If you use FFmpeg 7.0, then Jonas has a branch for that too (haven't tried it > yet though). > > FWIW: my test results were a bit mixed. I didn't post them before as I don't > fully/really understand this 'video stuff', and I didn't want you all to suffer > from what was likely a PEBKAC issue. > > On my PineTab2 (rk3566) I had some h.264 videos HW accelerated, but not all. > My guess is that it's related to the resolution. 1920x1080 worked, while it > didn't work with a 1280x640 video. The video still played, just not HW > accelerated. IOW: improvements in some and otherwise it was just rendered by > the CPU (I think), just like before. This is because all rk35XX have two hardware video decoders for H.264. This is not to be be confused with rkvdec which is gone. It has a modified Hantro G1 core (limited to 1080p60) and rkvdec2 core (driver in progress). I don't think Rockchip really expected the first one to be ever used, but upstream has been pushy and its now enabled upstream. That has a side effect, which is that userspace will have to work harder on these platform to pick the right HW for the task. > > On my Rock64 I got a pink tint with all videos, like described here: > https://github.com/mpv-player/mpv/issues/12968 > IIUC, that's actually a problem in the lima driver? Its not clear from the bug report. This visual artefact has been seen with wayland compositors lately (notably weston). Notably, this can happen if you try and import NV12 with mesa (panfrost and lima included) but forcing a TEXTURE_2D target instead of external target. Normally this should be rejected by mesa, but is accidentally not, and cause miss-render. Nicolas > > Cheers, > Diederik
This is a revival of a 3 year old series [1] now that NV15/NV20/NV30 support for display driver have landed in mainline tree. This series adds H.264 High 10 and 4:2:2 profile support to the Rockchip Video Decoder driver. Patch 1 adds helpers for calculating plane bytesperline and sizeimage. Patch 2 adds two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV. Patch 3 change to use bytesperline and buffer height to configure strides. Patch 4 change to use values from SPS/PPS control to configure the HW. Patch 5 remove an unnecessary call to validate sps at streaming start. Patch 6-10 refactor code to support filtering of CAPUTRE formats based on the image format returned from a get_image_fmt ops. Patch 11 adds final bits to support H.264 High 10 and 4:2:2 profiles. Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328): v4l2-compliance 1.24.1, 64 bits, 64-bit time_t ... Total for rkvdec device /dev/video1: 46, Succeeded: 46, Failed: 0, Warnings: 0 Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request ... Ran 65/69 tests successfully Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-V4L2-request ... Ran 127/135 tests successfully Before this series: Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request ... Ran 44/69 tests successfully Changes in v4: - Fix failed v4l2-compliance tests related to CAPTURE queue - Rework CAPTURE format filter anv validate to use an image format - Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5] Link to v3: https://lore.kernel.org/linux-media/20231029183427.1781554-1-jonas@kwiboo.se/ Changes in v3: - Drop merged patches - Use bpp and bpp_div instead of prior misuse of block_w/block_h - New patch to use values from SPS/PPS control to configure the HW - New patch to remove an unnecessary call to validate sps at streaming start - Reworked pixel format validation Link to v2: https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/ Changes in v2: - Collect r-b tags - SPS pic width and height in mbs validation moved to rkvdec_try_ctrl - New patch to not override output buffer sizeimage - Reworked pixel format validation - Only align decoded buffer instead of changing frmsize step_width Link to v1: https://lore.kernel.org/linux-media/20200701215616.30874-1-jonas@kwiboo.se/ Following commits adds support for NV15/NV20/NV30 to VOP driver: 728c15b4b5f3 ("drm/fourcc: Add NV20 and NV30 YUV formats") d4b384228562 ("drm/rockchip: vop: Add NV15, NV20 and NV30 support") To fully runtime test this series you may need above drm commits and ffmpeg patches from [2], this series and drm patches is also available at [3]. [1] https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/ [2] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-n6.1-dev/ [3] https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v4/ [4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865 [5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf Regards, Jonas Alex Bee (1): media: rkvdec: h264: Don't hardcode SPS/PPS parameters Jonas Karlman (10): media: v4l2-common: Add helpers to calculate bytesperline and sizeimage media: v4l2: Add NV15 and NV20 pixel formats media: rkvdec: h264: Use bytesperline and buffer height as virstride media: rkvdec: h264: Remove SPS validation at streaming start media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper media: rkvdec: Move rkvdec_reset_decoded_fmt helper media: rkvdec: Extract decoded format enumeration into helper media: rkvdec: Add image format concept media: rkvdec: Add get_image_fmt ops media: rkvdec: h264: Support High 10 and 4:2:2 profiles .../media/v4l/pixfmt-yuv-planar.rst | 128 +++++++++++ drivers/media/v4l2-core/v4l2-common.c | 80 +++---- drivers/media/v4l2-core/v4l2-ioctl.c | 2 + drivers/staging/media/rkvdec/rkvdec-h264.c | 83 +++---- drivers/staging/media/rkvdec/rkvdec.c | 217 +++++++++++++----- drivers/staging/media/rkvdec/rkvdec.h | 18 +- include/uapi/linux/videodev2.h | 2 + 7 files changed, 396 insertions(+), 134 deletions(-)