| Message ID | 20250401073527.1626-1-ming.qian@oss.nxp.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | [v5,1/2] media: amphion: Reduce decoding latency for HEVC decoder | expand |
Hey Ming, thanks for the patches, unfortunatly our testing pipeline isn't happy with them yet. See below ... On 01.04.2025 15:35, ming.qian@oss.nxp.com wrote: >From: Ming Qian <ming.qian@oss.nxp.com> > >The amphion decoder firmware supports a low latency flush mode for the >HEVC format since v1.9.0. This feature, which is enabled when the >display delay is set to 0, can help to reduce the decoding latency by >appending some padding data to every frame. > >Signed-off-by: Ming Qian <ming.qian@oss.nxp.com> >Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com> >--- >v5 >- Apply FIELD_PREP() and FIELD_GET() in CHECK_VERSION >v4 >- Add CHECK_VERSION macro >v3 >- Improve commit message as recommended >v2 >- Improve commit message >- Add firmware version check > > drivers/media/platform/amphion/vpu_malone.c | 24 ++++++++++++++++++--- > 1 file changed, 21 insertions(+), 3 deletions(-) > >diff --git a/drivers/media/platform/amphion/vpu_malone.c b/drivers/media/platform/amphion/vpu_malone.c >index 5c6b2a841b6f..b6e4996c2d91 100644 >--- a/drivers/media/platform/amphion/vpu_malone.c >+++ b/drivers/media/platform/amphion/vpu_malone.c >@@ -68,6 +68,12 @@ > > #define MALONE_DEC_FMT_RV_MASK BIT(21) > >+#define MALONE_VERSION_MASK 0xFFFFF >+#define MALONE_VERSION(maj, min, inc) \ >+ (FIELD_PREP(0xF0000, maj) | FIELD_PREP(0xFF00, min) | FIELD_PREP(0xFF, inc)) drivers/media/platform/amphion/vpu_malone.c:675:52: error: call to undeclared function 'FIELD_GET'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] 675 | if (params->codec_format == V4L2_PIX_FMT_HEVC && !CHECK_VERSION(iface, 1, 9)) | ^ drivers/media/platform/amphion/vpu_malone.c:79:4: note: expanded from macro 'CHECK_VERSION' 79 | (FIELD_GET(MALONE_VERSION_MASK, (iface)->fw_version) >= MALONE_VERSION(maj, min, 0)) | ^ drivers/media/platform/amphion/vpu_malone.c:675:52: error: call to undeclared function 'FIELD_PREP'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] drivers/media/platform/amphion/vpu_malone.c:79:59: note: expanded from macro 'CHECK_VERSION' 79 | (FIELD_GET(MALONE_VERSION_MASK, (iface)->fw_version) >= MALONE_VERSION(maj, min, 0)) | ^ drivers/media/platform/amphion/vpu_malone.c:77:4: note: expanded from macro 'MALONE_VERSION' 77 | (FIELD_PREP(0xF0000, maj) | FIELD_PREP(0xFF00, min) | FIELD_PREP(0xFF, inc)) | ^ 2 errors generated. https://linux-media.pages.freedesktop.org/-/users/sebastianfricke/-/jobs/73725346/artifacts/report.htm Regards, Sebastian >+#define CHECK_VERSION(iface, maj, min) \ >+ (FIELD_GET(MALONE_VERSION_MASK, (iface)->fw_version) >= MALONE_VERSION(maj, min, 0)) >+ > enum vpu_malone_stream_input_mode { > INVALID_MODE = 0, > FRAME_LVL, >@@ -332,6 +338,8 @@ struct vpu_dec_ctrl { > u32 buf_addr[VID_API_NUM_STREAMS]; > }; > >+static const struct malone_padding_scode *get_padding_scode(u32 type, u32 fmt); >+ > u32 vpu_malone_get_data_size(void) > { > return sizeof(struct vpu_dec_ctrl); >@@ -654,9 +662,15 @@ static int vpu_malone_set_params(struct vpu_shared_addr *shared, > hc->jpg[instance].jpg_mjpeg_interlaced = 0; > } > >- hc->codec_param[instance].disp_imm = params->display_delay_enable ? 1 : 0; >- if (malone_format != MALONE_FMT_AVC) >+ if (params->display_delay_enable && >+ get_padding_scode(SCODE_PADDING_BUFFLUSH, params->codec_format)) >+ hc->codec_param[instance].disp_imm = 1; >+ else > hc->codec_param[instance].disp_imm = 0; >+ >+ if (params->codec_format == V4L2_PIX_FMT_HEVC && !CHECK_VERSION(iface, 1, 9)) >+ hc->codec_param[instance].disp_imm = 0; >+ > hc->codec_param[instance].dbglog_enable = 0; > iface->dbglog_desc.level = 0; > >@@ -1024,6 +1038,7 @@ static const struct malone_padding_scode padding_scodes[] = { > {SCODE_PADDING_EOS, V4L2_PIX_FMT_JPEG, {0x0, 0x0}}, > {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_H264, {0x15010000, 0x0}}, > {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_H264_MVC, {0x15010000, 0x0}}, >+ {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_HEVC, {0x3e010000, 0x20}}, > }; > > static const struct malone_padding_scode padding_scode_dft = {0x0, 0x0}; >@@ -1058,8 +1073,11 @@ static int vpu_malone_add_padding_scode(struct vpu_buffer *stream_buffer, > int ret; > > ps = get_padding_scode(scode_type, pixelformat); >- if (!ps) >+ if (!ps) { >+ if (scode_type == SCODE_PADDING_BUFFLUSH) >+ return 0; > return -EINVAL; >+ } > > wptr = readl(&str_buf->wptr); > if (wptr < stream_buffer->phys || wptr > stream_buffer->phys + stream_buffer->length) >-- >2.43.0-rc1 > Sebastian Fricke Consultant Software Engineer Collabora Ltd Platinum Building, St John's Innovation Park, Cambridge CB4 0DS, UK Registered in England & Wales no 5513718.
Hi Sebastian, On 2025/4/1 16:53, Sebastian Fricke wrote: > Hey Ming, > > thanks for the patches, unfortunatly our testing pipeline isn't happy > with them yet. > > See below ... > > On 01.04.2025 15:35, ming.qian@oss.nxp.com wrote: >> From: Ming Qian <ming.qian@oss.nxp.com> >> >> The amphion decoder firmware supports a low latency flush mode for the >> HEVC format since v1.9.0. This feature, which is enabled when the >> display delay is set to 0, can help to reduce the decoding latency by >> appending some padding data to every frame. >> >> Signed-off-by: Ming Qian <ming.qian@oss.nxp.com> >> Reviewed-by: Nicolas Dufresne <nicolas.dufresne@collabora.com> >> --- >> v5 >> - Apply FIELD_PREP() and FIELD_GET() in CHECK_VERSION >> v4 >> - Add CHECK_VERSION macro >> v3 >> - Improve commit message as recommended >> v2 >> - Improve commit message >> - Add firmware version check >> >> drivers/media/platform/amphion/vpu_malone.c | 24 ++++++++++++++++++--- >> 1 file changed, 21 insertions(+), 3 deletions(-) >> >> diff --git a/drivers/media/platform/amphion/vpu_malone.c >> b/drivers/media/platform/amphion/vpu_malone.c >> index 5c6b2a841b6f..b6e4996c2d91 100644 >> --- a/drivers/media/platform/amphion/vpu_malone.c >> +++ b/drivers/media/platform/amphion/vpu_malone.c >> @@ -68,6 +68,12 @@ >> >> #define MALONE_DEC_FMT_RV_MASK BIT(21) >> >> +#define MALONE_VERSION_MASK 0xFFFFF >> +#define MALONE_VERSION(maj, min, inc) \ >> + (FIELD_PREP(0xF0000, maj) | FIELD_PREP(0xFF00, min) | >> FIELD_PREP(0xFF, inc)) > > drivers/media/platform/amphion/vpu_malone.c:675:52: error: call to > undeclared function 'FIELD_GET'; ISO C99 and later do not support > implicit function declarations [-Wimplicit-function-declaration] > 675 | if (params->codec_format == V4L2_PIX_FMT_HEVC && > !CHECK_VERSION(iface, 1, 9)) > | ^ > drivers/media/platform/amphion/vpu_malone.c:79:4: note: expanded from > macro 'CHECK_VERSION' > 79 | (FIELD_GET(MALONE_VERSION_MASK, > (iface)->fw_version) >= MALONE_VERSION(maj, min, 0)) > | ^ > drivers/media/platform/amphion/vpu_malone.c:675:52: error: call to > undeclared function 'FIELD_PREP'; ISO C99 and later do not support > implicit function declarations [-Wimplicit-function-declaration] > drivers/media/platform/amphion/vpu_malone.c:79:59: note: expanded from > macro 'CHECK_VERSION' > 79 | (FIELD_GET(MALONE_VERSION_MASK, > (iface)->fw_version) >= MALONE_VERSION(maj, min, 0)) > > | ^ > drivers/media/platform/amphion/vpu_malone.c:77:4: note: expanded from > macro 'MALONE_VERSION' > 77 | (FIELD_PREP(0xF0000, maj) | FIELD_PREP(0xFF00, > min) | FIELD_PREP(0xFF, inc)) > | ^ > 2 errors generated. > > https://linux-media.pages.freedesktop.org/-/users/sebastianfricke/-/jobs/73725346/artifacts/report.htm > I'm sorry I missed include the header file. Regards, Ming > Regards, > Sebastian > >> +#define CHECK_VERSION(iface, maj, min) \ >> + (FIELD_GET(MALONE_VERSION_MASK, (iface)->fw_version) >= >> MALONE_VERSION(maj, min, 0)) >> + >> enum vpu_malone_stream_input_mode { >> INVALID_MODE = 0, >> FRAME_LVL, >> @@ -332,6 +338,8 @@ struct vpu_dec_ctrl { >> u32 buf_addr[VID_API_NUM_STREAMS]; >> }; >> >> +static const struct malone_padding_scode *get_padding_scode(u32 type, >> u32 fmt); >> + >> u32 vpu_malone_get_data_size(void) >> { >> return sizeof(struct vpu_dec_ctrl); >> @@ -654,9 +662,15 @@ static int vpu_malone_set_params(struct >> vpu_shared_addr *shared, >> hc->jpg[instance].jpg_mjpeg_interlaced = 0; >> } >> >> - hc->codec_param[instance].disp_imm = params->display_delay_enable >> ? 1 : 0; >> - if (malone_format != MALONE_FMT_AVC) >> + if (params->display_delay_enable && >> + get_padding_scode(SCODE_PADDING_BUFFLUSH, params->codec_format)) >> + hc->codec_param[instance].disp_imm = 1; >> + else >> hc->codec_param[instance].disp_imm = 0; >> + >> + if (params->codec_format == V4L2_PIX_FMT_HEVC && >> !CHECK_VERSION(iface, 1, 9)) >> + hc->codec_param[instance].disp_imm = 0; >> + >> hc->codec_param[instance].dbglog_enable = 0; >> iface->dbglog_desc.level = 0; >> >> @@ -1024,6 +1038,7 @@ static const struct malone_padding_scode >> padding_scodes[] = { >> {SCODE_PADDING_EOS, V4L2_PIX_FMT_JPEG, {0x0, 0x0}}, >> {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_H264, {0x15010000, >> 0x0}}, >> {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_H264_MVC, {0x15010000, >> 0x0}}, >> + {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_HEVC, {0x3e010000, >> 0x20}}, >> }; >> >> static const struct malone_padding_scode padding_scode_dft = {0x0, 0x0}; >> @@ -1058,8 +1073,11 @@ static int vpu_malone_add_padding_scode(struct >> vpu_buffer *stream_buffer, >> int ret; >> >> ps = get_padding_scode(scode_type, pixelformat); >> - if (!ps) >> + if (!ps) { >> + if (scode_type == SCODE_PADDING_BUFFLUSH) >> + return 0; >> return -EINVAL; >> + } >> >> wptr = readl(&str_buf->wptr); >> if (wptr < stream_buffer->phys || wptr > stream_buffer->phys + >> stream_buffer->length) >> -- >> 2.43.0-rc1 >> > Sebastian Fricke > Consultant Software Engineer > > Collabora Ltd > Platinum Building, St John's Innovation Park, Cambridge CB4 0DS, UK > Registered in England & Wales no 5513718.
diff --git a/drivers/media/platform/amphion/vpu_malone.c b/drivers/media/platform/amphion/vpu_malone.c index 5c6b2a841b6f..b6e4996c2d91 100644 --- a/drivers/media/platform/amphion/vpu_malone.c +++ b/drivers/media/platform/amphion/vpu_malone.c @@ -68,6 +68,12 @@ #define MALONE_DEC_FMT_RV_MASK BIT(21) +#define MALONE_VERSION_MASK 0xFFFFF +#define MALONE_VERSION(maj, min, inc) \ + (FIELD_PREP(0xF0000, maj) | FIELD_PREP(0xFF00, min) | FIELD_PREP(0xFF, inc)) +#define CHECK_VERSION(iface, maj, min) \ + (FIELD_GET(MALONE_VERSION_MASK, (iface)->fw_version) >= MALONE_VERSION(maj, min, 0)) + enum vpu_malone_stream_input_mode { INVALID_MODE = 0, FRAME_LVL, @@ -332,6 +338,8 @@ struct vpu_dec_ctrl { u32 buf_addr[VID_API_NUM_STREAMS]; }; +static const struct malone_padding_scode *get_padding_scode(u32 type, u32 fmt); + u32 vpu_malone_get_data_size(void) { return sizeof(struct vpu_dec_ctrl); @@ -654,9 +662,15 @@ static int vpu_malone_set_params(struct vpu_shared_addr *shared, hc->jpg[instance].jpg_mjpeg_interlaced = 0; } - hc->codec_param[instance].disp_imm = params->display_delay_enable ? 1 : 0; - if (malone_format != MALONE_FMT_AVC) + if (params->display_delay_enable && + get_padding_scode(SCODE_PADDING_BUFFLUSH, params->codec_format)) + hc->codec_param[instance].disp_imm = 1; + else hc->codec_param[instance].disp_imm = 0; + + if (params->codec_format == V4L2_PIX_FMT_HEVC && !CHECK_VERSION(iface, 1, 9)) + hc->codec_param[instance].disp_imm = 0; + hc->codec_param[instance].dbglog_enable = 0; iface->dbglog_desc.level = 0; @@ -1024,6 +1038,7 @@ static const struct malone_padding_scode padding_scodes[] = { {SCODE_PADDING_EOS, V4L2_PIX_FMT_JPEG, {0x0, 0x0}}, {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_H264, {0x15010000, 0x0}}, {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_H264_MVC, {0x15010000, 0x0}}, + {SCODE_PADDING_BUFFLUSH, V4L2_PIX_FMT_HEVC, {0x3e010000, 0x20}}, }; static const struct malone_padding_scode padding_scode_dft = {0x0, 0x0}; @@ -1058,8 +1073,11 @@ static int vpu_malone_add_padding_scode(struct vpu_buffer *stream_buffer, int ret; ps = get_padding_scode(scode_type, pixelformat); - if (!ps) + if (!ps) { + if (scode_type == SCODE_PADDING_BUFFLUSH) + return 0; return -EINVAL; + } wptr = readl(&str_buf->wptr); if (wptr < stream_buffer->phys || wptr > stream_buffer->phys + stream_buffer->length)