| Message ID | 20210427111526.1772293-1-acourbot@chromium.org (mailing list archive) |
|---|---|
| Headers | show |
| Series | media: mtk-vcodec: support for MT8183 decoder | expand |
On 27/04/2021 13:15, Alexandre Courbot wrote: > This series adds support for the stateless API into mtk-vcodec, by first > separating the stateful ops into their own source file, and introducing > a new set of ops suitable for stateless decoding. As such, support for > stateful decoders should remain completely unaffected. > > This series has been tested with both MT8183 and MT8173. Decoding was > working for both chips, and in the case of MT8173 no regression has been > noticed. > > Patches 1-9 add MT8183 support to the decoder using the stateless API. > MT8183 only support H.264 acceleration. > > Patches 10-15 are follow-ups that further improve compliance for the > decoder and encoder, by fixing support for commands on both. Patch 11 > also makes sure that supported H.264 profiles are exported on MT8173. For a v5 I would recommend that - where possible - these 'improve compliance' patches are moved to the beginning of the series. That way they can be picked up quickly without having to wait for the whole series to be accepted. Regards, Hans > > Changes since v3: > * Stop checking that controls are set for every request. > * Add V4L2_CID_STATELESS_H264_START_CODE control. > * Stop mapping OUTPUT buffers and getting the NAL type from them, use the > nal_ref_idc field instead. > * Make V4L2_CID_MIN_BUFFERS_FOR_CAPTURE control stateful-only. > * Set vb2_buffer's field to V4L2_FIELD_NONE in buffer validation hook. > > Changes since v2: > * Add follow-up patches fixing support for START/STOP commands for the > encoder, and stateful decoder. > > Alexandre Courbot (8): > media: mtk-vcodec: vdec: handle firmware version field > media: mtk-vcodec: support version 2 of decoder firmware ABI > media: add Mediatek's MM21 format > dt-bindings: media: document mediatek,mt8183-vcodec-dec > media: mtk-vcodec: vdec: use helpers in VIDIOC_(TRY_)DECODER_CMD > media: mtk-vcodec: vdec: clamp OUTPUT resolution to hardware limits > media: mtk-vcodec: make flush buffer reusable by encoder > media: mtk-vcodec: venc: support START and STOP commands > > Hirokazu Honda (1): > media: mtk-vcodec: vdec: Support H264 profile control > > Hsin-Yi Wang (1): > media: mtk-vcodec: venc: make sure buffer exists in list before > removing > > Yunfei Dong (5): > media: mtk-vcodec: vdec: move stateful ops into their own file > media: mtk-vcodec: vdec: support stateless API > media: mtk-vcodec: vdec: support stateless H.264 decoding > media: mtk-vcodec: vdec: add media device if using stateless api > media: mtk-vcodec: enable MT8183 decoder > > .../bindings/media/mediatek-vcodec.txt | 1 + > .../media/v4l/pixfmt-reserved.rst | 7 + > drivers/media/platform/Kconfig | 2 + > drivers/media/platform/mtk-vcodec/Makefile | 3 + > .../platform/mtk-vcodec/mtk_vcodec_dec.c | 818 +++--------------- > .../platform/mtk-vcodec/mtk_vcodec_dec.h | 28 +- > .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 66 +- > .../mtk-vcodec/mtk_vcodec_dec_stateful.c | 667 ++++++++++++++ > .../mtk-vcodec/mtk_vcodec_dec_stateless.c | 370 ++++++++ > .../platform/mtk-vcodec/mtk_vcodec_drv.h | 58 +- > .../platform/mtk-vcodec/mtk_vcodec_enc.c | 135 ++- > .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 4 + > .../mtk-vcodec/vdec/vdec_h264_req_if.c | 780 +++++++++++++++++ > .../media/platform/mtk-vcodec/vdec_drv_if.c | 3 + > .../media/platform/mtk-vcodec/vdec_drv_if.h | 1 + > .../media/platform/mtk-vcodec/vdec_ipi_msg.h | 23 +- > .../media/platform/mtk-vcodec/vdec_vpu_if.c | 43 +- > .../media/platform/mtk-vcodec/vdec_vpu_if.h | 5 + > drivers/media/v4l2-core/v4l2-ioctl.c | 1 + > include/uapi/linux/videodev2.h | 1 + > 20 files changed, 2293 insertions(+), 723 deletions(-) > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c > create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c > > -- > 2.31.1.498.g6c1eba8ee3d-goog >
Hi, On 27.04.21 13:15, Alexandre Courbot wrote: > This series adds support for the stateless API into mtk-vcodec, by first > separating the stateful ops into their own source file, and introducing > a new set of ops suitable for stateless decoding. As such, support for > stateful decoders should remain completely unaffected. > > This series has been tested with both MT8183 and MT8173. Decoding was > working for both chips, and in the case of MT8173 no regression has been > noticed. I am trying to test the decoder with this patchset using v4l2-ctl on mt8173. I am trying to decode an h264 file into V4L2_PIX_FMT_MT21C format. I am not able to do it. It seems that the driver expects each buffer to start with a nal starting code, and the v4l2-ctl command just read the files into buffers without any parsing. Can you share the command and the files you used for testing? Thank you, Dafna > > Patches 1-9 add MT8183 support to the decoder using the stateless API. > MT8183 only support H.264 acceleration. > > Patches 10-15 are follow-ups that further improve compliance for the > decoder and encoder, by fixing support for commands on both. Patch 11 > also makes sure that supported H.264 profiles are exported on MT8173. > > Changes since v3: > * Stop checking that controls are set for every request. > * Add V4L2_CID_STATELESS_H264_START_CODE control. > * Stop mapping OUTPUT buffers and getting the NAL type from them, use the > nal_ref_idc field instead. > * Make V4L2_CID_MIN_BUFFERS_FOR_CAPTURE control stateful-only. > * Set vb2_buffer's field to V4L2_FIELD_NONE in buffer validation hook. > > Changes since v2: > * Add follow-up patches fixing support for START/STOP commands for the > encoder, and stateful decoder. > > Alexandre Courbot (8): > media: mtk-vcodec: vdec: handle firmware version field > media: mtk-vcodec: support version 2 of decoder firmware ABI > media: add Mediatek's MM21 format > dt-bindings: media: document mediatek,mt8183-vcodec-dec > media: mtk-vcodec: vdec: use helpers in VIDIOC_(TRY_)DECODER_CMD > media: mtk-vcodec: vdec: clamp OUTPUT resolution to hardware limits > media: mtk-vcodec: make flush buffer reusable by encoder > media: mtk-vcodec: venc: support START and STOP commands > > Hirokazu Honda (1): > media: mtk-vcodec: vdec: Support H264 profile control > > Hsin-Yi Wang (1): > media: mtk-vcodec: venc: make sure buffer exists in list before > removing > > Yunfei Dong (5): > media: mtk-vcodec: vdec: move stateful ops into their own file > media: mtk-vcodec: vdec: support stateless API > media: mtk-vcodec: vdec: support stateless H.264 decoding > media: mtk-vcodec: vdec: add media device if using stateless api > media: mtk-vcodec: enable MT8183 decoder > > .../bindings/media/mediatek-vcodec.txt | 1 + > .../media/v4l/pixfmt-reserved.rst | 7 + > drivers/media/platform/Kconfig | 2 + > drivers/media/platform/mtk-vcodec/Makefile | 3 + > .../platform/mtk-vcodec/mtk_vcodec_dec.c | 818 +++--------------- > .../platform/mtk-vcodec/mtk_vcodec_dec.h | 28 +- > .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 66 +- > .../mtk-vcodec/mtk_vcodec_dec_stateful.c | 667 ++++++++++++++ > .../mtk-vcodec/mtk_vcodec_dec_stateless.c | 370 ++++++++ > .../platform/mtk-vcodec/mtk_vcodec_drv.h | 58 +- > .../platform/mtk-vcodec/mtk_vcodec_enc.c | 135 ++- > .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 4 + > .../mtk-vcodec/vdec/vdec_h264_req_if.c | 780 +++++++++++++++++ > .../media/platform/mtk-vcodec/vdec_drv_if.c | 3 + > .../media/platform/mtk-vcodec/vdec_drv_if.h | 1 + > .../media/platform/mtk-vcodec/vdec_ipi_msg.h | 23 +- > .../media/platform/mtk-vcodec/vdec_vpu_if.c | 43 +- > .../media/platform/mtk-vcodec/vdec_vpu_if.h | 5 + > drivers/media/v4l2-core/v4l2-ioctl.c | 1 + > include/uapi/linux/videodev2.h | 1 + > 20 files changed, 2293 insertions(+), 723 deletions(-) > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c > create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c > create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c > > -- > 2.31.1.498.g6c1eba8ee3d-goog >
On Thu, Apr 29, 2021 at 4:35 PM Hans Verkuil <hverkuil-cisco@xs4all.nl> wrote: > > On 27/04/2021 13:15, Alexandre Courbot wrote: > > This series adds support for the stateless API into mtk-vcodec, by first > > separating the stateful ops into their own source file, and introducing > > a new set of ops suitable for stateless decoding. As such, support for > > stateful decoders should remain completely unaffected. > > > > This series has been tested with both MT8183 and MT8173. Decoding was > > working for both chips, and in the case of MT8173 no regression has been > > noticed. > > > > Patches 1-9 add MT8183 support to the decoder using the stateless API. > > MT8183 only support H.264 acceleration. > > > > Patches 10-15 are follow-ups that further improve compliance for the > > decoder and encoder, by fixing support for commands on both. Patch 11 > > also makes sure that supported H.264 profiles are exported on MT8173. > > For a v5 I would recommend that - where possible - these 'improve compliance' > patches are moved to the beginning of the series. That way they can be picked > up quickly without having to wait for the whole series to be accepted. Makes sense, the current order reflects the chronology these patches have been written, but I agree that improving compliance should be merged first. Let me try to reorder things a bit. Cheers, Alex.
Hi Dafna, On Thu, Apr 29, 2021 at 9:07 PM Dafna Hirschfeld <dafna.hirschfeld@collabora.com> wrote: > > Hi, > > On 27.04.21 13:15, Alexandre Courbot wrote: > > This series adds support for the stateless API into mtk-vcodec, by first > > separating the stateful ops into their own source file, and introducing > > a new set of ops suitable for stateless decoding. As such, support for > > stateful decoders should remain completely unaffected. > > > > This series has been tested with both MT8183 and MT8173. Decoding was > > working for both chips, and in the case of MT8173 no regression has been > > noticed. > > I am trying to test the decoder with this patchset using v4l2-ctl on mt8173. > > I am trying to decode an h264 file into V4L2_PIX_FMT_MT21C format. > I am not able to do it. It seems that the driver expects each buffer to start > with a nal starting code, and the v4l2-ctl command just read the files into > buffers without any parsing. > > Can you share the command and the files you used for testing? I have been using the Chromium test suite (aka video_decode_accelerator_tests). I had doubts after reading your email but I tested the series again using this tool and confirmed it was working. Unfortunately this test is not easy to build, but maybe if you have a Chromium environment ready you can run it. mtk-vcodec does expect the input buffers to be split by NAL units (as per the spec [1] IIUC), so that would explain why v4l2-ctl failed (I assume that it also fails without this series?). Maybe ffmpeg can also be used to exercice this driver with properly split NAL units, but I am not familiar with its use with V4L2. I'm also not sure it would be happy about the MT21C format that the driver outputs, or that it could pick the MDP to convert it to something readable... Cheers, Alex. [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/pixfmt-compressed.html#compressed-formats
On 13.05.21 10:21, Alexandre Courbot wrote: > Hi Dafna, > > On Thu, Apr 29, 2021 at 9:07 PM Dafna Hirschfeld > <dafna.hirschfeld@collabora.com> wrote: >> >> Hi, >> >> On 27.04.21 13:15, Alexandre Courbot wrote: >>> This series adds support for the stateless API into mtk-vcodec, by first >>> separating the stateful ops into their own source file, and introducing >>> a new set of ops suitable for stateless decoding. As such, support for >>> stateful decoders should remain completely unaffected. >>> >>> This series has been tested with both MT8183 and MT8173. Decoding was >>> working for both chips, and in the case of MT8173 no regression has been >>> noticed. >> >> I am trying to test the decoder with this patchset using v4l2-ctl on mt8173. >> >> I am trying to decode an h264 file into V4L2_PIX_FMT_MT21C format. >> I am not able to do it. It seems that the driver expects each buffer to start >> with a nal starting code, and the v4l2-ctl command just read the files into >> buffers without any parsing. >> >> Can you share the command and the files you used for testing? > > I have been using the Chromium test suite (aka > video_decode_accelerator_tests). I had doubts after reading your email > but I tested the series again using this tool and confirmed it was > working. Hi, I have a chromeos userspace that I compiled. I use it to test the codec drivers on latest 5.14. Could you share the exact command you use for your tests? I used the command: tast -verbose run -build=false 10.42.0.175 video.DecodeAccel*h264* With that commands, all the tests that ends with 'VD' fail , the other tests pass. The docs in [1] says that the command video_decode_accelerator_tests does not require the "full Chrome browser stack", does that mean that it can be compiled and run on userspace other than chromeos? I could not find the instructions of how to compile and install that command. Could you instruct me to it? The mt8173 doc [2] says that the supported video decoders are "H.264, H.265 / HEVC, MPEG-1/2/4" But running 'v4l2-ctl --list-formats-out -d0' shows: ioctl: VIDIOC_ENUM_FMT Type: Video Output Multiplanar [0]: 'H264' (H.264, compressed) [1]: 'VP80' (VP8, compressed) [2]: 'VP90' (VP9, compressed) So the source code shows support for vp8, vp9 which is not stated in the official doc. Do you know the explanation for that difference? Do you know if elm device should support vp8, vp9? [1] https://chromium.googlesource.com/chromium/src/+/HEAD/docs/media/gpu/video_decoder_test_usage.md [2] https://www.mediatek.com/products/tablets/mt8173 Thanks, Dafna > > Unfortunately this test is not easy to build, but maybe if you have a > Chromium environment ready you can run it. mtk-vcodec does expect the > input buffers to be split by NAL units (as per the spec [1] IIUC), so > that would explain why v4l2-ctl failed (I assume that it also fails > without this series?). > > Maybe ffmpeg can also be used to exercice this driver with properly > split NAL units, but I am not familiar with its use with V4L2. I'm > also not sure it would be happy about the MT21C format that the driver > outputs, or that it could pick the MDP to convert it to something > readable... > > Cheers, > Alex. > > [1] https://www.kernel.org/doc/html/latest/userspace-api/media/v4l/pixfmt-compressed.html#compressed-formats >
This series adds support for the stateless API into mtk-vcodec, by first separating the stateful ops into their own source file, and introducing a new set of ops suitable for stateless decoding. As such, support for stateful decoders should remain completely unaffected. This series has been tested with both MT8183 and MT8173. Decoding was working for both chips, and in the case of MT8173 no regression has been noticed. Patches 1-9 add MT8183 support to the decoder using the stateless API. MT8183 only support H.264 acceleration. Patches 10-15 are follow-ups that further improve compliance for the decoder and encoder, by fixing support for commands on both. Patch 11 also makes sure that supported H.264 profiles are exported on MT8173. Changes since v3: * Stop checking that controls are set for every request. * Add V4L2_CID_STATELESS_H264_START_CODE control. * Stop mapping OUTPUT buffers and getting the NAL type from them, use the nal_ref_idc field instead. * Make V4L2_CID_MIN_BUFFERS_FOR_CAPTURE control stateful-only. * Set vb2_buffer's field to V4L2_FIELD_NONE in buffer validation hook. Changes since v2: * Add follow-up patches fixing support for START/STOP commands for the encoder, and stateful decoder. Alexandre Courbot (8): media: mtk-vcodec: vdec: handle firmware version field media: mtk-vcodec: support version 2 of decoder firmware ABI media: add Mediatek's MM21 format dt-bindings: media: document mediatek,mt8183-vcodec-dec media: mtk-vcodec: vdec: use helpers in VIDIOC_(TRY_)DECODER_CMD media: mtk-vcodec: vdec: clamp OUTPUT resolution to hardware limits media: mtk-vcodec: make flush buffer reusable by encoder media: mtk-vcodec: venc: support START and STOP commands Hirokazu Honda (1): media: mtk-vcodec: vdec: Support H264 profile control Hsin-Yi Wang (1): media: mtk-vcodec: venc: make sure buffer exists in list before removing Yunfei Dong (5): media: mtk-vcodec: vdec: move stateful ops into their own file media: mtk-vcodec: vdec: support stateless API media: mtk-vcodec: vdec: support stateless H.264 decoding media: mtk-vcodec: vdec: add media device if using stateless api media: mtk-vcodec: enable MT8183 decoder .../bindings/media/mediatek-vcodec.txt | 1 + .../media/v4l/pixfmt-reserved.rst | 7 + drivers/media/platform/Kconfig | 2 + drivers/media/platform/mtk-vcodec/Makefile | 3 + .../platform/mtk-vcodec/mtk_vcodec_dec.c | 818 +++--------------- .../platform/mtk-vcodec/mtk_vcodec_dec.h | 28 +- .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c | 66 +- .../mtk-vcodec/mtk_vcodec_dec_stateful.c | 667 ++++++++++++++ .../mtk-vcodec/mtk_vcodec_dec_stateless.c | 370 ++++++++ .../platform/mtk-vcodec/mtk_vcodec_drv.h | 58 +- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 135 ++- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 4 + .../mtk-vcodec/vdec/vdec_h264_req_if.c | 780 +++++++++++++++++ .../media/platform/mtk-vcodec/vdec_drv_if.c | 3 + .../media/platform/mtk-vcodec/vdec_drv_if.h | 1 + .../media/platform/mtk-vcodec/vdec_ipi_msg.h | 23 +- .../media/platform/mtk-vcodec/vdec_vpu_if.c | 43 +- .../media/platform/mtk-vcodec/vdec_vpu_if.h | 5 + drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/uapi/linux/videodev2.h | 1 + 20 files changed, 2293 insertions(+), 723 deletions(-) create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c -- 2.31.1.498.g6c1eba8ee3d-goog