| Message ID | 20220117120615.21687-1-irui.wang@mediatek.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Enable two H264 encoder cores on MT8195 | expand |
Dear all maintainers, Gently ping. Could you help to review this series of patches? I would be very grateful for any of your comments. Thanks Best Regards On Mon, 2022-01-17 at 20:06 +0800, Irui Wang wrote: > MT8195 has two H264 encoder cores, they have their own power-domains, > clocks, interrupts, register base. The two H264 encoder cores can > work > together to achieve higher performance, it's a core mode called > frame-racing, one core has 4K@30fps performance, two cores can > achieve > 4K@60fps. > The two encoder core encoding process looks like this: > > VENC Core0: frm#0....frm#2....frm#4.... > VENC Core1: ..frm#1....frm#3....frm#5.... > > This series of patches are used to enable the two H264 encoder cores, > encoding process will be changed: > As-Is: Synchronous > V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder > IRQ --> > encoding done with result --> job_finish > V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder > IRQ --> > encoding done with result --> job_finish > ... > > To-Be: Asynchronous > V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish > ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish > (venc core0 may encode done here, done the encoding result to client) > V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish. > > There is no "wait encoder IRQ" synchronous call during frame-racing > mode > encoding process, it can full use the two encoder cores to achieve > higher > performance. > > --- > This series patches dependent on: > [1]: the latest linux stage tree: > https://git.linuxtv.org/media_stage.git > > mtk decoder patches > [2]: > https://patchwork.linuxtv.org/project/linux-media/list/?series=7105 > [3]: > https://patchwork.linuxtv.org/project/linux-media/list/?series=7131 > > new yaml included files > [4]: > https://patchwork.kernel.org/project/linux-mediatek/list/?series=551641 > [5]: > https://patchwork.kernel.org/project/linux-mediatek/list/?series=580579 > > --- > --- > changes compared with v1: > - of_platform_populate was used in place of the component framework. > - new yaml file for venc cores. > - some modifications for patch v1's review comments. > --- > > Irui Wang (10): > media: mtk-vcodec: Use core type to indicate h264 and vp8 enc > media: mtk-vcodec: export encoder functions > dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for > mt8195 > media: mtk-vcodec: Enable venc dual core usage > media: mtk-vcodec: mtk-vcodec: Rewrite venc power manage interface > media: mtk-vcodec: Add venc power on/off interface > media: mtk-vcodec: Rewrite venc clock interface > media: mtk-vcodec: Add more extra processing for dual-core mode > media: mtk-vcodec: Add dual core mode encode process > media: mtk-vcodec: Done encode result to client > > .../media/mediatek,vcodec-encoder-core.yaml | 214 > +++++++++++++++++ > drivers/media/platform/mtk-vcodec/Makefile | 4 +- > .../platform/mtk-vcodec/mtk_vcodec_drv.h | 44 +++- > .../platform/mtk-vcodec/mtk_vcodec_enc.c | 109 ++++++--- > .../platform/mtk-vcodec/mtk_vcodec_enc.h | 7 +- > .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 187 +++++++++++++++ > .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++ > .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 118 ++++++---- > .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 187 +++++++++++++-- > .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 11 +- > .../platform/mtk-vcodec/mtk_vcodec_util.c | 19 ++ > .../platform/mtk-vcodec/mtk_vcodec_util.h | 5 + > .../platform/mtk-vcodec/venc/venc_h264_if.c | 216 +++++++++++++++- > -- > .../platform/mtk-vcodec/venc/venc_vp8_if.c | 3 +- > .../media/platform/mtk-vcodec/venc_drv_if.c | 79 +++++-- > .../media/platform/mtk-vcodec/venc_drv_if.h | 7 + > .../media/platform/mtk-vcodec/venc_vpu_if.c | 10 +- > .../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +- > 18 files changed, 1097 insertions(+), 162 deletions(-) > create mode 100644 > Documentation/devicetree/bindings/media/mediatek,vcodec-encoder- > core.yaml > create mode 100644 drivers/media/platform/mtk- > vcodec/mtk_vcodec_enc_core.c > create mode 100644 drivers/media/platform/mtk- > vcodec/mtk_vcodec_enc_core.h >
MT8195 has two H264 encoder cores, they have their own power-domains, clocks, interrupts, register base. The two H264 encoder cores can work together to achieve higher performance, it's a core mode called frame-racing, one core has 4K@30fps performance, two cores can achieve 4K@60fps. The two encoder core encoding process looks like this: VENC Core0: frm#0....frm#2....frm#4.... VENC Core1: ..frm#1....frm#3....frm#5.... This series of patches are used to enable the two H264 encoder cores, encoding process will be changed: As-Is: Synchronous V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ --> encoding done with result --> job_finish V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ --> encoding done with result --> job_finish ... To-Be: Asynchronous V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish (venc core0 may encode done here, done the encoding result to client) V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish. There is no "wait encoder IRQ" synchronous call during frame-racing mode encoding process, it can full use the two encoder cores to achieve higher performance. --- This series patches dependent on: [1]: the latest linux stage tree: https://git.linuxtv.org/media_stage.git mtk decoder patches [2]: https://patchwork.linuxtv.org/project/linux-media/list/?series=7105 [3]: https://patchwork.linuxtv.org/project/linux-media/list/?series=7131 new yaml included files [4]: https://patchwork.kernel.org/project/linux-mediatek/list/?series=551641 [5]: https://patchwork.kernel.org/project/linux-mediatek/list/?series=580579 --- --- changes compared with v1: - of_platform_populate was used in place of the component framework. - new yaml file for venc cores. - some modifications for patch v1's review comments. --- Irui Wang (10): media: mtk-vcodec: Use core type to indicate h264 and vp8 enc media: mtk-vcodec: export encoder functions dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195 media: mtk-vcodec: Enable venc dual core usage media: mtk-vcodec: mtk-vcodec: Rewrite venc power manage interface media: mtk-vcodec: Add venc power on/off interface media: mtk-vcodec: Rewrite venc clock interface media: mtk-vcodec: Add more extra processing for dual-core mode media: mtk-vcodec: Add dual core mode encode process media: mtk-vcodec: Done encode result to client .../media/mediatek,vcodec-encoder-core.yaml | 214 +++++++++++++++++ drivers/media/platform/mtk-vcodec/Makefile | 4 +- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 44 +++- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 109 ++++++--- .../platform/mtk-vcodec/mtk_vcodec_enc.h | 7 +- .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 187 +++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++ .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 118 ++++++---- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 187 +++++++++++++-- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 11 +- .../platform/mtk-vcodec/mtk_vcodec_util.c | 19 ++ .../platform/mtk-vcodec/mtk_vcodec_util.h | 5 + .../platform/mtk-vcodec/venc/venc_h264_if.c | 216 +++++++++++++++--- .../platform/mtk-vcodec/venc/venc_vp8_if.c | 3 +- .../media/platform/mtk-vcodec/venc_drv_if.c | 79 +++++-- .../media/platform/mtk-vcodec/venc_drv_if.h | 7 + .../media/platform/mtk-vcodec/venc_vpu_if.c | 10 +- .../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +- 18 files changed, 1097 insertions(+), 162 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h