From patchwork Fri May 13 20:29:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849474 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A2CDAC433EF for ; Fri, 13 May 2022 20:41:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wppvh+ogOf9HRyZYeSB6DIXxnRDPWjmIHcdb0srAsdg=; b=EU4relJm/1s3Qg YUItyageJmEm775oyt/4TMQyaN7i763XxQd9+Kr8XC2P7hLG1+GzfbGDSGn+luC/Di0ZBWgBdb0Yb zrige4Ip+THAmi0nnCrnX/0ZoDJNH6fxoLlCeS1lj1cl7Rs1Xf2//QKLPq21m0gCkBT2+G/PTyVNT yGoTx93jCOSdwpCFajcX/00oEG68wMm0jYBagzTcC2kNGJzNHOE/1Vr9fEEWyRyOiCjPNPcoZ/J77 xUEaV7nNdjEEACeJaB0ZBMRZ/PJGmDJramD9qvk67fkRlszMlWdHaeRBktSoqH8GahMHdE96CKvts drRK0pTOKFXsBYBuWPGQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc5i-00HZz6-7N; Fri, 13 May 2022 20:40:50 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbut-00HV12-Ld; Fri, 13 May 2022 20:29:44 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 9909C1F46481 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473774; bh=Sp4NALpVvxuZpnUtnlafMWQU3cvt1hsOH3okI6KMqBg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=D6FJR6emnD7kC2ErYClSk16G0/d8/pH7wK/5sIy3AcRhPGwm6F6WKjeUgfXW8zafY 6jJaYWqKXJEKhpES/ROj7He010Iz+oKUvONEN7cJhea7VUR0NX9dCzbOkA0+/jrKml 1F8NJSG8IQ2ov7aSS67jNEuUYE4mD8YkrvqIIumUhK7UV7uDZbfZa1vVit5tmpU0zH rM9vs/rOWmFrvGMRPH5qGg7DQeYScGRDLZW3N8rAIS//VYIZEkT1GH7fJCbYku6e4A EwqdaoY3SwgypOfpA3vwKt0t5xx4XqkiRSW8ZRUaKHtw2bKaDZ7kMO/OMdchbSG5Q1 97zhwuf388xZg== From: Nicolas Dufresne To: Tiffany Lin , Andrew-CT Chen , Mauro Carvalho Chehab , Dmitry Osipenko , Thierry Reding , Jonathan Hunter , Ezequiel Garcia , Philipp Zabel , Greg Kroah-Hartman , Heiko Stuebner , Matthias Brugger Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Dmitry Osipenko , Hans Verkuil , linux-kernel@vger.kernel.org, linux-tegra@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org Subject: [PATCH v5 01/20] media: h264: Use v4l2_h264_reference for reflist Date: Fri, 13 May 2022 16:29:03 -0400 Message-Id: <20220513202922.13846-2-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_132940_139074_9B9EB795 X-CRM114-Status: GOOD ( 21.45 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org In preparation for adding field decoding support, convert the byte arrays for reflist into array of struct v4l2_h264_reference. That struct will allow us to mark which field of the reference picture is being referenced. Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia Tested-by: Dmitry Osipenko Signed-off-by: Hans Verkuil [hverkuil: top_field_order_cnt -> pic_order_count] --- .../vcodec/vdec/vdec_h264_req_common.c | 21 +++- .../vcodec/vdec/vdec_h264_req_common.h | 11 ++- .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 15 ++- .../vcodec/vdec/vdec_h264_req_multi_if.c | 27 +++-- .../media/platform/nvidia/tegra-vde/h264.c | 19 ++-- drivers/media/v4l2-core/v4l2-h264.c | 33 ++++--- .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++---- drivers/staging/media/hantro/hantro_hw.h | 6 +- .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++---------- drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +-- include/media/v4l2-h264.h | 19 ++-- 11 files changed, 167 insertions(+), 132 deletions(-) diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_common.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_common.c index 3c75a7b4e845..b6931729e4ea 100644 --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_common.c +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_common.c @@ -12,11 +12,24 @@ #define GET_MTK_VDEC_PARAM(param) \ { dst_param->param = src_param->param; } -/* - * The firmware expects unused reflist entries to have the value 0x20. - */ -void mtk_vdec_h264_fixup_ref_list(u8 *ref_list, size_t num_valid) +void mtk_vdec_h264_get_ref_list(u8 *ref_list, + const struct v4l2_h264_reference * v4l2_ref_list, + int num_valid) { + u32 i; + + /* + * TODO The firmware does not support field decoding. Future + * implementation must use v4l2_ref_list[i].fields to obtain + * the reference field parity. + */ + + for (i = 0; i < num_valid; i++) + ref_list[i] = v4l2_ref_list[i].index; + + /* + * The firmware expects unused reflist entries to have the value 0x20. + */ memset(&ref_list[num_valid], 0x20, 32 - num_valid); } diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_common.h b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_common.h index 0113f380b491..416277c70af3 100644 --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_common.h +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_common.h @@ -164,12 +164,15 @@ struct h264_fb { }; /** - * mtk_vdec_h264_fixup_ref_list - fixup unused reference to 0x20. + * mtk_vdec_h264_get_ref_list - translate V4L2 reference list * - * @ref_list: reference picture list - * @num_valid: used reference number + * @ref_list: Mediatek reference picture list + * @v4l2_ref_list: V4L2 reference picture list + * @num_valid: used reference number */ -void mtk_vdec_h264_fixup_ref_list(u8 *ref_list, size_t num_valid); +void mtk_vdec_h264_get_ref_list(u8 *ref_list, + const struct v4l2_h264_reference * v4l2_ref_list, + int num_valid); /** * mtk_vdec_h264_get_ctrl_ptr - get each CID contrl address. diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c index b055ceea481d..4bc05ab5afea 100644 --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c @@ -102,6 +102,9 @@ static int get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst) const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix; struct mtk_h264_dec_slice_param *slice_param = &inst->h264_slice_param; struct v4l2_h264_reflist_builder reflist_builder; + struct v4l2_h264_reference v4l2_p0_reflist[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference v4l2_b0_reflist[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference v4l2_b1_reflist[V4L2_H264_REF_LIST_LEN]; u8 *p0_reflist = slice_param->decode_params.ref_pic_list_p0; u8 *b0_reflist = slice_param->decode_params.ref_pic_list_b0; u8 *b1_reflist = slice_param->decode_params.ref_pic_list_b1; @@ -137,12 +140,14 @@ static int get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst) /* Build the reference lists */ v4l2_h264_init_reflist_builder(&reflist_builder, dec_params, sps, inst->dpb); - v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist); - v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist); + v4l2_h264_build_p_ref_list(&reflist_builder, v4l2_p0_reflist); + v4l2_h264_build_b_ref_lists(&reflist_builder, v4l2_b0_reflist, + v4l2_b1_reflist); + /* Adapt the built lists to the firmware's expectations */ - mtk_vdec_h264_fixup_ref_list(p0_reflist, reflist_builder.num_valid); - mtk_vdec_h264_fixup_ref_list(b0_reflist, reflist_builder.num_valid); - mtk_vdec_h264_fixup_ref_list(b1_reflist, reflist_builder.num_valid); + mtk_vdec_h264_get_ref_list(p0_reflist, v4l2_p0_reflist, reflist_builder.num_valid); + mtk_vdec_h264_get_ref_list(b0_reflist, v4l2_b0_reflist, reflist_builder.num_valid); + mtk_vdec_h264_get_ref_list(b1_reflist, v4l2_b1_reflist, reflist_builder.num_valid); memcpy(&inst->vsi_ctx.h264_slice_params, slice_param, sizeof(inst->vsi_ctx.h264_slice_params)); diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_multi_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_multi_if.c index 1d9e753cf894..784d01f8bd50 100644 --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_multi_if.c +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_multi_if.c @@ -222,6 +222,9 @@ static int get_vdec_sig_decode_parameters(struct vdec_h264_slice_inst *inst) const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix; struct vdec_h264_slice_lat_dec_param *slice_param = &inst->h264_slice_param; struct v4l2_h264_reflist_builder reflist_builder; + struct v4l2_h264_reference v4l2_p0_reflist[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference v4l2_b0_reflist[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference v4l2_b1_reflist[V4L2_H264_REF_LIST_LEN]; u8 *p0_reflist = slice_param->decode_params.ref_pic_list_p0; u8 *b0_reflist = slice_param->decode_params.ref_pic_list_b0; u8 *b1_reflist = slice_param->decode_params.ref_pic_list_b1; @@ -256,13 +259,14 @@ static int get_vdec_sig_decode_parameters(struct vdec_h264_slice_inst *inst) /* Build the reference lists */ v4l2_h264_init_reflist_builder(&reflist_builder, dec_params, sps, inst->dpb); - v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist); + v4l2_h264_build_p_ref_list(&reflist_builder, v4l2_p0_reflist); + v4l2_h264_build_b_ref_lists(&reflist_builder, v4l2_b0_reflist, v4l2_b1_reflist); - v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist); /* Adapt the built lists to the firmware's expectations */ - mtk_vdec_h264_fixup_ref_list(p0_reflist, reflist_builder.num_valid); - mtk_vdec_h264_fixup_ref_list(b0_reflist, reflist_builder.num_valid); - mtk_vdec_h264_fixup_ref_list(b1_reflist, reflist_builder.num_valid); + mtk_vdec_h264_get_ref_list(p0_reflist, v4l2_p0_reflist, reflist_builder.num_valid); + mtk_vdec_h264_get_ref_list(b0_reflist, v4l2_b0_reflist, reflist_builder.num_valid); + mtk_vdec_h264_get_ref_list(b1_reflist, v4l2_b1_reflist, reflist_builder.num_valid); + memcpy(&inst->vsi_ctx.h264_slice_params, slice_param, sizeof(inst->vsi_ctx.h264_slice_params)); @@ -276,6 +280,9 @@ static void vdec_h264_slice_fill_decode_reflist(struct vdec_h264_slice_inst *ins struct v4l2_ctrl_h264_decode_params *dec_params = &share_info->dec_params; struct v4l2_ctrl_h264_sps *sps = &share_info->sps; struct v4l2_h264_reflist_builder reflist_builder; + struct v4l2_h264_reference v4l2_p0_reflist[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference v4l2_b0_reflist[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference v4l2_b1_reflist[V4L2_H264_REF_LIST_LEN]; u8 *p0_reflist = slice_param->decode_params.ref_pic_list_p0; u8 *b0_reflist = slice_param->decode_params.ref_pic_list_b0; u8 *b1_reflist = slice_param->decode_params.ref_pic_list_b1; @@ -291,13 +298,13 @@ static void vdec_h264_slice_fill_decode_reflist(struct vdec_h264_slice_inst *ins /* Build the reference lists */ v4l2_h264_init_reflist_builder(&reflist_builder, dec_params, sps, inst->dpb); - v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist); - v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist); + v4l2_h264_build_p_ref_list(&reflist_builder, v4l2_p0_reflist); + v4l2_h264_build_b_ref_lists(&reflist_builder, v4l2_b0_reflist, v4l2_b1_reflist); /* Adapt the built lists to the firmware's expectations */ - mtk_vdec_h264_fixup_ref_list(p0_reflist, reflist_builder.num_valid); - mtk_vdec_h264_fixup_ref_list(b0_reflist, reflist_builder.num_valid); - mtk_vdec_h264_fixup_ref_list(b1_reflist, reflist_builder.num_valid); + mtk_vdec_h264_get_ref_list(p0_reflist, v4l2_p0_reflist, reflist_builder.num_valid); + mtk_vdec_h264_get_ref_list(b0_reflist, v4l2_b0_reflist, reflist_builder.num_valid); + mtk_vdec_h264_get_ref_list(b1_reflist, v4l2_b1_reflist, reflist_builder.num_valid); } static int vdec_h264_slice_alloc_mv_buf(struct vdec_h264_slice_inst *inst, diff --git a/drivers/media/platform/nvidia/tegra-vde/h264.c b/drivers/media/platform/nvidia/tegra-vde/h264.c index d8e5534e80c8..4fb0aaad16d6 100644 --- a/drivers/media/platform/nvidia/tegra-vde/h264.c +++ b/drivers/media/platform/nvidia/tegra-vde/h264.c @@ -45,9 +45,9 @@ struct tegra_vde_h264_decoder_ctx { }; struct h264_reflists { - u8 p[V4L2_H264_NUM_DPB_ENTRIES]; - u8 b0[V4L2_H264_NUM_DPB_ENTRIES]; - u8 b1[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference p[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference b0[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference b1[V4L2_H264_NUM_DPB_ENTRIES]; }; static int tegra_vde_wait_mbe(struct tegra_vde *vde) @@ -765,10 +765,10 @@ static int tegra_vde_h264_setup_frames(struct tegra_ctx *ctx, struct tegra_m2m_buffer *tb = vb_to_tegra_buf(&dst->vb2_buf); struct tegra_ctx_h264 *h = &ctx->h264; struct v4l2_h264_reflist_builder b; + struct v4l2_h264_reference *dpb_id; struct h264_reflists reflists; struct vb2_buffer *ref; unsigned int i; - u8 *dpb_id; int err; /* @@ -811,14 +811,16 @@ static int tegra_vde_h264_setup_frames(struct tegra_ctx *ctx, } for (i = 0; i < b.num_valid; i++) { - ref = get_ref_buf(ctx, dst, dpb_id[i]); + int dpb_idx = dpb_id[i].index; - err = tegra_vde_h264_setup_frame(ctx, h264, &b, ref, dpb_id[i], + ref = get_ref_buf(ctx, dst, dpb_idx); + + err = tegra_vde_h264_setup_frame(ctx, h264, &b, ref, dpb_idx, h264->dpb_frames_nb++); if (err) return err; - if (b.refs[dpb_id[i]].pic_order_count < b.cur_pic_order_count) + if (b.refs[dpb_idx].pic_order_count < b.cur_pic_order_count) h264->dpb_ref_frames_with_earlier_poc_nb++; } @@ -880,6 +882,9 @@ static int tegra_vde_h264_setup_context(struct tegra_ctx *ctx, if (h->pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE) return -EOPNOTSUPP; + if (h->decode_params->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) + return -EOPNOTSUPP; + if (h->sps->profile_idc == 66) h264->baseline_profile = 1; diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index ac47519a9fbe..afbfcf78efe4 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, pic_order_count = dpb[i].top_field_order_cnt; b->refs[i].pic_order_count = pic_order_count; - b->unordered_reflist[b->num_valid] = i; + b->unordered_reflist[b->num_valid].index = i; b->num_valid++; } for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++) - b->unordered_reflist[i] = i; + b->unordered_reflist[i].index = i; } EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder); @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb, const struct v4l2_h264_reflist_builder *builder = data; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb, s32 poca, pocb; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, s32 poca, pocb; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16-bytes array used to store the P reference list. Each entry - * is an index in the DPB + * @reflist: 16 sized array used to store the P reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in * section '8.2.4 Decoding process for reference picture lists construction' @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, */ void v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, - u8 *reflist) + struct v4l2_h264_reference *reflist) { memcpy(reflist, builder->unordered_reflist, sizeof(builder->unordered_reflist[0]) * builder->num_valid); @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry - * is an index in the DPB - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry - * is an index in the DPB + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * is a v4l2_h264_reference structure + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described * in section '8.2.4 Decoding process for reference picture lists construction' @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); */ void v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, - u8 *b0_reflist, u8 *b1_reflist) + struct v4l2_h264_reference *b0_reflist, + struct v4l2_h264_reference *b1_reflist) { memcpy(b0_reflist, builder->unordered_reflist, sizeof(builder->unordered_reflist[0]) * builder->num_valid); diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c index f49dbfb8a843..9de7f05eff2a 100644 --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf) static void set_ref(struct hantro_ctx *ctx) { - const u8 *b0_reflist, *b1_reflist, *p_reflist; + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist; struct hantro_dev *vpu = ctx->dev; int reg_num; u32 reg; @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx) */ reg_num = 0; for (i = 0; i < 15; i += 3) { - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]); + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index); vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++)); } @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx) * of forward and backward reference picture lists and first 4 entries * of P forward picture list. */ - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) | - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]); + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) | + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index); vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC); /* @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx) */ reg_num = 0; for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) { - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) | - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) | - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) | - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) | - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) | - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]); + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index); vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++)); } diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 04844bbcbd36..3608e463290e 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls { * @b1: B1 reflist */ struct hantro_h264_dec_reflists { - u8 p[HANTRO_H264_DPB_SIZE]; - u8 b0[HANTRO_H264_DPB_SIZE]; - u8 b1[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE]; }; /** diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c index 64a6330475eb..46c1a83bcc4e 100644 --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf) static void set_ref(struct hantro_ctx *ctx) { - const u8 *b0_reflist, *b1_reflist, *p_reflist; + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist; struct hantro_dev *vpu = ctx->dev; u32 reg; int i; @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx) b1_reflist = ctx->h264_dec.reflists.b1; p_reflist = ctx->h264_dec.reflists.p; - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) | - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) | - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) | - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) | - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) | - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]); + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) | + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) | + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) | + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) | + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) | + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74)); - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) | - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) | - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) | - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) | - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) | - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]); + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) | + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) | + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) | + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) | + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) | + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75)); reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) | @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx) VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14)); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83)); - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) | - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) | - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) | - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) | - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) | - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]); + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) | + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) | + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) | + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) | + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) | + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100)); - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) | - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) | - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) | - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) | - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) | - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]); + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) | + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) | + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) | + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) | + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) | + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101)); - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) | - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) | - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) | - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]); + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) | + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) | + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) | + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102)); - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) | - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) | - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) | - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) | - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) | - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]); + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) | + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) | + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) | + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) | + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) | + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103)); - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) | - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) | - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) | - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) | - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) | - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]); + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) | + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) | + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) | + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) | + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) | + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104)); - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) | - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) | - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) | - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]); + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) | + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) | + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) | + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105)); - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) | - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) | - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) | - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]); + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) | + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) | + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) | + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106)); reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm); diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 951e19231da2..3c7f3d87fab4 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl { #define RKVDEC_H264_DPB_SIZE 16 struct rkvdec_h264_reflists { - u8 p[RKVDEC_H264_DPB_SIZE]; - u8 b0[RKVDEC_H264_DPB_SIZE]; - u8 b1[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE]; u8 num_valid; }; @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, switch (j) { case 0: - idx = h264_ctx->reflists.p[i]; + idx = h264_ctx->reflists.p[i].index; break; case 1: - idx = h264_ctx->reflists.b0[i]; + idx = h264_ctx->reflists.b0[i].index; break; case 2: - idx = h264_ctx->reflists.b1[i]; + idx = h264_ctx->reflists.b1[i].index; break; } diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index 4b1c71c935e0..ef9a894e3c32 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder { u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; s32 cur_pic_order_count; - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; u8 num_valid; }; @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry - * is an index in the DPB - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry - * is an index in the DPB + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * is a v4l2_h264_reference structure + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described * in section '8.2.4 Decoding process for reference picture lists construction' @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, */ void v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, - u8 *b0_reflist, u8 *b1_reflist); + struct v4l2_h264_reference *b0_reflist, + struct v4l2_h264_reference *b1_reflist); /** * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16-bytes array used to store the P reference list. Each entry - * is an index in the DPB + * @reflist: 16 sized array used to store the P reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in * section '8.2.4 Decoding process for reference picture lists construction' @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, */ void v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, - u8 *reflist); + struct v4l2_h264_reference *reflist); #endif /* _MEDIA_V4L2_H264_H */ From patchwork Fri May 13 20:29:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849470 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 45B9EC4332F for ; Fri, 13 May 2022 20:39:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=G98+hKaaW9NBxeSv/bU1YxoPvDuVN5vHl6xtDErofZ0=; b=2YhS/0AdddeHDE tSQcGwhfzoWOd5YxC95ihMb7V4NpdiCciJ3WfAuBK7YqGJ4yuVkvUVtcM+VbqyJGHGTe2FiTmMgxQ gYoYq/sEquq/jD4qD9+SETn61DKjszKA7LPGny/ZNxMMpPEBiTPxg1qh8PgIQ5XF+9KI135lD1+U3 PPsy3lH4xd+6wPstxxv63Q1V2GxZGdIaVpmgnc6173/Mf1uPKs/odTvK17I1VrZMbQQtoC8nKccEE 7riWIi/ZxhXzQWJU2xyH9sPftqy3X88mxkQtufHfeHbdpTDdWbe2oOwtkrIq2r3ywpm8Uu6YyvZfy uiQ5ALFydlqnnjZumMPA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc4d-00HZQh-Tv; Fri, 13 May 2022 20:39:43 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbut-00HV20-L8 for linux-rockchip@lists.infradead.org; Fri, 13 May 2022 20:29:42 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id E2CB61F46482 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473776; bh=molSu9Ndn1sYFzHom/iyL7R+4m1ZjEvEB6PLGf0nq+8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XOdq8tM1BWHoSrbopxlJpYNQJLRnrt9+F1t6Ue7JNDdbHugNTNnnmt8UvKOoidhLg 9LyA6wsJIDxPd2WS2eMJyL+GCF4cS8ZXMdR2SD8notkAZAFeqnMqung4764rPIqENN bL1xKHErTi3FVJFbyqMfMTje5xGo1kX7bGljs0hlxnXyv2PPw/zqqO6nFqcU4R+OYL Mkxax0HOK0L9Irt/+y1NzuIOQqC+jJEARQnkW03ixcnvPcdS79MqRS8X2GGtnFdFr/ iBWSUAH+3MRtfe2z1kgCvuqkBqka0aS7N9ekeAEjFRAAbriHenaTEAkOtpqlQQLTR1 ghjrHw9+dN7sA== From: Nicolas Dufresne To: Mauro Carvalho Chehab , Ezequiel Garcia , Philipp Zabel , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , Hans Verkuil , linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v5 02/20] media: h264: Increase reference lists size to 32 Date: Fri, 13 May 2022 16:29:04 -0400 Message-Id: <20220513202922.13846-3-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_132940_088116_8EA39B44 X-CRM114-Status: GOOD ( 11.95 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org This is to accommodate support for field decoding, which splits the top and the bottom references into the reference list. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia Signed-off-by: Hans Verkuil --- drivers/media/v4l2-core/v4l2-h264.c | 6 +++--- drivers/staging/media/hantro/hantro_hw.h | 6 +++--- drivers/staging/media/rkvdec/rkvdec-h264.c | 6 +++--- include/media/v4l2-h264.h | 8 ++++---- 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index afbfcf78efe4..4b46b36526c0 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -212,7 +212,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16 sized array used to store the P reference list. Each entry + * @reflist: 32 sized array used to store the P reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in @@ -235,9 +235,9 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry * is a v4l2_h264_reference structure - * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 3608e463290e..0b5b9da86c43 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls { * @b1: B1 reflist */ struct hantro_h264_dec_reflists { - struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE]; - struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE]; - struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; }; /** diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 3c7f3d87fab4..dff89732ddd0 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl { #define RKVDEC_H264_DPB_SIZE 16 struct rkvdec_h264_reflists { - struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE]; - struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE]; - struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index ef9a894e3c32..e282fb16ac58 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder { u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; s32 cur_pic_order_count; - struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; @@ -51,9 +51,9 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry * is a v4l2_h264_reference structure - * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described @@ -70,7 +70,7 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16 sized array used to store the P reference list. Each entry + * @reflist: 32 sized array used to store the P reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in From patchwork Fri May 13 20:29:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849471 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 87177C433F5 for ; Fri, 13 May 2022 20:41:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=s7v8+xN25nQb1tKDiW3y467DIwh9HhvGPjEKSIcNKfA=; b=krfcu3vsCfFRPM v6Gg4rFr9zCk8cjc2ysCKf5gheNZbSGdIPZ1E5+CzRpD+X/1uNFgITSkaMN4hzZJlaCSelmyzV04O 74lurz38hwDdytTeRacKoCEbe14uh0Ts52R/p/QQNT5v2oKKYgjkK0PjAMYegBVhVprYlwh99ypXe OnzZe7Pk4xK0AsaYELruFByVulRkNRyejPjKUc+C68Xaau4u4XI5PEep1ToqmUfXEffK6pjocRjD6 iIQfAxCXD3IIDQ/yWXkpNe7FZqD1vEaOkM/XYXE0SiZj6pxT21M2J3zDVM8IeG4Cyyo0k31MLxNmp 24BZ0a0EJSqJZix0IGaQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc5g-00HZyZ-NF; Fri, 13 May 2022 20:40:48 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbv1-00HV9T-B1 for linux-rockchip@lists.infradead.org; Fri, 13 May 2022 20:29:50 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id A58E51F46488 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473786; bh=1LAjnriWsrClpu2BgpAHhARrkeNr44Bjsa5xlaJTDIY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S9GVx/CinlhOPZTJhlPPqc9iQ3pJ3w1oZzG7vPL/qiObuNyYBcK7jqoDSiHOmgEMv X4tKhkxSloNyDOnCwVSQcb2c47QQAMKo3K3PTTc/2Vqn/gCZty4Aj0dUkPP9P27YFB 4Lymgjfmjflj16uU7bzI1HCoCi5SWlpMaFsmExGaCwd8sRANSke7ugB521OvZcdKi8 +6K95qbX7G6Msxfan7hcIuxdSf/sD4JEw/5MO8sRcTkOPUaiYZ3IdV8MOOWhcFFQBT IsOp/nSI/alhfJu0uxQI1v2zK0gORVX4IqlRn1A0p0ZgFUYUX2wiixlUI0UEG7DMsE 6u9EGtMgdFemg== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman , Hans Verkuil , Boris Brezillon Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , Mauro Carvalho Chehab , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v5 08/20] media: rkvdec: Stop overclocking the decoder Date: Fri, 13 May 2022 16:29:10 -0400 Message-Id: <20220513202922.13846-9-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_132947_568523_ED425396 X-CRM114-Status: GOOD ( 10.83 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org While this overclock hack seems to work on some implementations (some ChromeBooks, RockPi4) it also causes instability on other implementations (notably LibreComputer Renegade, but there were more reports in the LibreELEC project, where this has been removed). While performance is indeed affected (tested with GStreamer), 4K playback still works as long as you don't operate in lock step and keep at least 1 frame ahead of time in the decode queue. After discussion with ChromeOS members, it would seem that their implementation indeed used to synchronously decode each frame, so this hack was simply compensating for their code being less efficient. In my opinion, this hack should not have been included upstream. Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver") Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia Signed-off-by: Hans Verkuil --- drivers/staging/media/rkvdec/rkvdec.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index c0cf3488f970..2df8cf4883e2 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -1027,12 +1027,6 @@ static int rkvdec_probe(struct platform_device *pdev) if (ret) return ret; - /* - * Bump ACLK to max. possible freq. (500 MHz) to improve performance - * When 4k video playback. - */ - clk_set_rate(rkvdec->clocks[0].clk, 500 * 1000 * 1000); - rkvdec->regs = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(rkvdec->regs)) return PTR_ERR(rkvdec->regs); From patchwork Fri May 13 20:29:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CAF1AC433F5 for ; Fri, 13 May 2022 20:41:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=h8tgv2VpCKO3zrudlsem92uK/ecrCQuTESR+sAfnGV4=; b=n3uclnQuOWPpdr mG5sVA2RSKChVGcP/tc92SAVfn1k5UOT5fabhhhnpzjfSPKm0run3frJEjD87tdYkFFBxnBWNCBON bytUEVymaghlpOfcEd0WM00Fkc6lTxaYGVMHggyMYrJnhU49+dIXmpPKMIgsHxG8nCQPKZ3BYQYR4 zdHCAPCIpbYaAbWog0TiEshOFotcBKmCCtpu0pcfGr0fy5KknF/sDHR1Yn5FPCMm8colMAB1ferMW z6eKNlWfMddc/VnoY2VrnVr/8jCKDCbPqHkwdQOQ2S8TlhUQfZadJwtajHZJ6pXVdyOLCqG2CDf4A TOEjRRc3i3cjbA3jnZeA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc6Y-00HaWs-Lq; Fri, 13 May 2022 20:41:42 +0000 Received: from desiato.infradead.org ([90.155.92.199]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbvd-00HVcb-0O for linux-rockchip@bombadil.infradead.org; Fri, 13 May 2022 20:30:25 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=zDcJfwUBt1+Cr76K75MtMYNiQoQ0/pYjz/+UDdBw/TM=; b=aG+sEL6FdjqRz4U9BWN6vfBMBS VAsefdq8Q9Twd5TOSWU7eN5xF46cLUoa9wqU0X0+D/zPYnTmI8je0gkbiGJnCW4Jl7dedxgi/fd/m 6PyXqlTWYNP2AgiWD1s/0l/IMVp88h7OHERa1TP5UOyTfTUMWwRfsgzdteXtU8i9Uf1LM6orvwOOv B2kUA5/eFNjE248VbctQVu/xZ7bt8rVMSfb7B/cHvp2uHcxg1VLzLBxzieo8LD1UAqNctxil3kkbo QdeODqGJ+ROVpHpsgJLo/sikZnbeXpO1fy/Ks9nQRyiYrhLpgVsLIPzp14mS9YYO/4ayWP6uEyucM O5OC8/Vw==; Received: from bhuna.collabora.co.uk ([46.235.227.227]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbv6-0006W6-0H for linux-rockchip@lists.infradead.org; Fri, 13 May 2022 20:30:14 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id B18A31F46481 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473788; bh=6VfvBHSE6A+IDfkNG2Ci8I/HmRwNsgBMdcwXJyxIJpc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Z7qCUdIwoDGpQAAqigEm1APjRXaASTGnbn5nOFjTBCdFuJz8mlySsv2OIM9VhI2ED wj9K4t4Ts12HDTTUVDQNy26TP2jtl4kIwsCC5pE+V3Z6RjnaXFutyUv7HjO0iIhpj+ Dd8ku4PfAunrnTznMqnLhUjzbrqEocEE6gDMTpUPEx4O5oYWyzFlTP4v5Rwwao6v57 9Mx/yBKY/nS5D7VoBYfJCukfy3n9EiBm90VdGG/i8FDOgevmcMDJfZc5dpL626gg8k gj5hAzmxO86pJO7zRDf+2rER/vzoG3vzB7tLtMcM/tllpBgXv3eLTU8X9MmxgonOwQ 3Agta1u7fLoDA== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman , Hans Verkuil , Boris Brezillon Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , Mauro Carvalho Chehab , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v5 09/20] media: rkvdec: h264: Fix dpb_valid implementation Date: Fri, 13 May 2022 16:29:11 -0400 Message-Id: <20220513202922.13846-10-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_212956_226088_EA74777B X-CRM114-Status: GOOD ( 17.38 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org The ref builder only provided references that are marked as valid in the dpb. Thus the current implementation of dpb_valid would always set the flag to 1. This is not representing missing frames (this is called 'non-existing' pictures in the spec). In some context, these non-existing pictures still need to occupy a slot in the reference list according to the spec. Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver") Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia Signed-off-by: Hans Verkuil --- drivers/staging/media/rkvdec/rkvdec-h264.c | 33 ++++++++++++++++------ 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index dff89732ddd0..bcde37d72244 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -112,6 +112,7 @@ struct rkvdec_h264_run { const struct v4l2_ctrl_h264_sps *sps; const struct v4l2_ctrl_h264_pps *pps; const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix; + int ref_buf_idx[V4L2_H264_NUM_DPB_ENTRIES]; }; struct rkvdec_h264_ctx { @@ -725,6 +726,26 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, } } +static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx, + struct rkvdec_h264_run *run) +{ + const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params; + u32 i; + + for (i = 0; i < ARRAY_SIZE(dec_params->dpb); i++) { + struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx; + const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb; + struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; + int buf_idx = -1; + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) + buf_idx = vb2_find_timestamp(cap_q, + dpb[i].reference_ts, 0); + + run->ref_buf_idx[i] = buf_idx; + } +} + static void assemble_hw_rps(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run) { @@ -762,7 +783,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, for (j = 0; j < RKVDEC_NUM_REFLIST; j++) { for (i = 0; i < h264_ctx->reflists.num_valid; i++) { - u8 dpb_valid = 0; + bool dpb_valid = run->ref_buf_idx[i] >= 0; u8 idx = 0; switch (j) { @@ -779,8 +800,6 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, if (idx >= ARRAY_SIZE(dec_params->dpb)) continue; - dpb_valid = !!(dpb[idx].flags & - V4L2_H264_DPB_ENTRY_FLAG_ACTIVE); set_ps_field(hw_rps, DPB_INFO(i, j), idx | dpb_valid << 4); @@ -859,13 +878,8 @@ get_ref_buf(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run, unsigned int dpb_idx) { struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx; - const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb; struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; - int buf_idx = -1; - - if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) - buf_idx = vb2_find_timestamp(cap_q, - dpb[dpb_idx].reference_ts, 0); + int buf_idx = run->ref_buf_idx[dpb_idx]; /* * If a DPB entry is unused or invalid, address of current destination @@ -1102,6 +1116,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) assemble_hw_scaling_list(ctx, &run); assemble_hw_pps(ctx, &run); + lookup_ref_buf_idx(ctx, &run); assemble_hw_rps(ctx, &run); config_registers(ctx, &run); From patchwork Fri May 13 20:29:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849472 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2F18AC433F5 for ; Fri, 13 May 2022 20:41:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=O9IlfpK+tCUs3dKLJrTauZz2rf7XjZK/1fXdNHGy8QI=; b=mEGn4Y4Q7YvUgz dfi5qqV5x2MJbRJ5Hs+cAuMaOEcdfqnSQ8i01/2hODCIr4hByOyceB7FIMwGTpIbClmdv3yTcmLgR dmxEsJKINIey0+r+onvOpo0t2oa3WJGHxomIkTvvjcfLmuIk5FxMTMxT4++ze6U/7cUsE4YHuKCy9 TeW5/UejTdAItUhJm2Q5zBRQKKNh8Vw0wUm1SYJmQ0ytH/2/O7GGYuRckqcR2SvrUhpTiAwEcBG+Y V3+X6IH9LNCKsMbKpb6V8Dhx7b4FSezz8e7cBls6Dnr08N6hJ1Hzwh39etHdj0iTLm23eH4TDUxAf RZOFSA59kH/A1CP7ncVA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc5l-00Ha0J-CS; Fri, 13 May 2022 20:40:53 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbv5-00HVCE-Hb for linux-rockchip@lists.infradead.org; Fri, 13 May 2022 20:29:53 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id B4FE31F46489 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473790; bh=lIxlnphkLCaVBUiFzEyVrDZH6uoUDfUFIWUVGXdyJKU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VqNNO9dpmcXgsqSoG08E/e4ANf6VIO0SS4tqU/u1KEPXwTpJERV+/ZKV39worJ/Ik EhhzeGw9ektnLCaFKZZoVh/ov4dW1HN3Pl/mjsaVhtSZb8u7j0l1YQnXBJmEAuL+YO qK96ZpUc1cFJOR2fjrOiqBMi/JuYcjQQ5Mc97EZYhIM5kGujK0nsMRGGAXwnf/8IJa Dy5jYKWOlG057qgtsqTConC6avwGnh2A+0nb+gzMwMZczHRgUyQT6MpcVCB3f4Rf46 YroY+kNaa97PiU91F/GlX/lNJdDVEyQpTpUKwDhXeLuMC3dPe3zDCBwFaMvywATF98 mCB9fD+/Q2cDQ== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman , Hans Verkuil , Boris Brezillon Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, Ezequiel Garcia , Mauro Carvalho Chehab , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v5 10/20] media: rkvdec: h264: Fix bit depth wrap in pps packet Date: Fri, 13 May 2022 16:29:12 -0400 Message-Id: <20220513202922.13846-11-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_132951_813070_B89806C6 X-CRM114-Status: GOOD ( 11.29 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman The luma and chroma bit depth fields in the pps packet are 3 bits wide. 8 is wrongly added to the bit depth values written to these 3 bit fields. Because only the 3 LSB are written, the hardware was configured correctly. Correct this by not adding 8 to the luma and chroma bit depth value. Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver") Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia Signed-off-by: Hans Verkuil --- drivers/staging/media/rkvdec/rkvdec-h264.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index bcde37d72244..8d44a884a52e 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -662,8 +662,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, WRITE_PPS(0xff, PROFILE_IDC); WRITE_PPS(1, CONSTRAINT_SET3_FLAG); WRITE_PPS(sps->chroma_format_idc, CHROMA_FORMAT_IDC); - WRITE_PPS(sps->bit_depth_luma_minus8 + 8, BIT_DEPTH_LUMA); - WRITE_PPS(sps->bit_depth_chroma_minus8 + 8, BIT_DEPTH_CHROMA); + WRITE_PPS(sps->bit_depth_luma_minus8, BIT_DEPTH_LUMA); + WRITE_PPS(sps->bit_depth_chroma_minus8, BIT_DEPTH_CHROMA); WRITE_PPS(0, QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG); WRITE_PPS(sps->log2_max_frame_num_minus4, LOG2_MAX_FRAME_NUM_MINUS4); WRITE_PPS(sps->max_num_ref_frames, MAX_NUM_REF_FRAMES); From patchwork Fri May 13 20:29:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849473 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E1887C433FE for ; Fri, 13 May 2022 20:41:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=VHZU6uipqM0goaK9cw8369Rkl+/GnDqTdzYajyf5z1s=; b=Fw2ulFifzXfiGV 8O1Js4gPw6js8fYvQTaTvapgzfPiuVgivbOFoN4kKzwC2VhkgUC1+JkwJDKd2KYND8l/eH72nBuwI 9KaJVawcw8xyb/nsPQ0p66MD6Kc7Cbud694lCwMAidAxuFgg/dMrNqu8vFLoXECNIrMFfDOxFZEGo Q7jyHjnsQjfMsO3DrDExC419bBKcphBx4E4ztNJ+GJLdL3a9//cvk6dmk2fQS8dMarKgUcOBpcThj 0vBhjp077Y8+miGT3VnyFczYXZwTaVTniaoAT/uGv+PNpP1UF6f0UTgvgiJxtH+MnWkuh8uyPrT5R qhda1AlsGjKpvY4wCp2w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc5n-00Ha2B-PY; Fri, 13 May 2022 20:40:55 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbv7-00HVE9-80 for linux-rockchip@lists.infradead.org; Fri, 13 May 2022 20:29:56 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 098C41F46484 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473792; bh=/HUm+2tIimTRRFVj71OaFyqc2Gk4HBundroiAuGI1EY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=iZUI2QrUdNrjgYRKH8Evuj3VNdhydoESdxJ0hiTMRmlNUijZoRZfLwt6Bb++CbrZC GNNaE3CEmKeSdOajaCYNS6ECqplCV/2Jms6LFKRgXtoCW6TktozZiDkZqXkbnKlsUp CY5boNhczvZfmK2jkzLEuvp7+NutQRnY17z6jTj9Gy9zrhxrxxk1nDopkA/RxJFA5d P3CFHHQQj8/Lrn5zYVTk/aLO7zj9QGBgM2WZLrVAxY9O5WAphypJrPjHMx0Ik4ZjHT uLGxlWw+GiCWDF+DjkMR6FU5sfWbQdtxvLpObdewkrX3T/v0I4P9vNl1YFS8+a0zOj o6AYUdJCO739g== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Hans Verkuil , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v5 11/20] media: rkvdec: Move H264 SPS validation in rkvdec-h264 Date: Fri, 13 May 2022 16:29:13 -0400 Message-Id: <20220513202922.13846-12-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_132953_535829_DAD2A6E0 X-CRM114-Status: GOOD ( 16.57 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org No functional change, this moves H264 specific validation into the H264 specific code. This is in preparation of improving this validation and reusing it when VIDIOC_STREAMON is called. Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia Signed-off-by: Hans Verkuil --- drivers/staging/media/rkvdec/rkvdec-h264.c | 23 ++++++++++++++++++++++ drivers/staging/media/rkvdec/rkvdec.c | 23 ++++++---------------- drivers/staging/media/rkvdec/rkvdec.h | 1 + 3 files changed, 30 insertions(+), 17 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 8d44a884a52e..0dcbcb1bac80 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -1137,9 +1137,32 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) return 0; } +static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl) +{ + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { + const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps; + /* + * TODO: The hardware supports 10-bit and 4:2:2 profiles, + * but it's currently broken in the driver. + * Reject them for now, until it's fixed. + */ + if (sps->chroma_format_idc > 1) + /* Only 4:0:0 and 4:2:0 are supported */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) + /* Luma and chroma bit depth mismatch */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != 0) + /* Only 8-bit is supported */ + return -EINVAL; + } + return 0; +} + const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = { .adjust_fmt = rkvdec_h264_adjust_fmt, .start = rkvdec_h264_start, .stop = rkvdec_h264_stop, .run = rkvdec_h264_run, + .try_ctrl = rkvdec_h264_try_ctrl, }; diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 2df8cf4883e2..e3d44d5b35f3 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -29,23 +29,12 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) { - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps; - /* - * TODO: The hardware supports 10-bit and 4:2:2 profiles, - * but it's currently broken in the driver. - * Reject them for now, until it's fixed. - */ - if (sps->chroma_format_idc > 1) - /* Only 4:0:0 and 4:2:0 are supported */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) - /* Luma and chroma bit depth mismatch */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != 0) - /* Only 8-bit is supported */ - return -EINVAL; - } + struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl); + const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc; + + if (desc->ops->try_ctrl) + return desc->ops->try_ctrl(ctx, ctrl); + return 0; } diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h index 2f4ea1786b93..9df0fba799a4 100644 --- a/drivers/staging/media/rkvdec/rkvdec.h +++ b/drivers/staging/media/rkvdec/rkvdec.h @@ -72,6 +72,7 @@ struct rkvdec_coded_fmt_ops { void (*done)(struct rkvdec_ctx *ctx, struct vb2_v4l2_buffer *src_buf, struct vb2_v4l2_buffer *dst_buf, enum vb2_buffer_state result); + int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl); }; struct rkvdec_coded_fmt_desc { From patchwork Fri May 13 20:29:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849475 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ABE9EC433EF for ; Fri, 13 May 2022 20:41:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=SrJKbDkxPOdxrHhU3K4tibC+5Zxzxh0+VLttqrYnkv8=; b=wQCTuJhBKhacAH vQyvop06SuAuO70S6hXcof02JXmIhHuUQIKy7cOtGZGeJo0picBs0PttTFfkJnymwmCJvZcOB8C10 KKfVnTq8VcKzeMUBTUIG1s/qX+KW8AeAP2qZv/TAdOi/DZk65i7JRTdC+oTPLWKfTzYE7gzW5gTry 1W+JgEaxk90SP3kB1PKUk5Rp5aBa499SbuRl+YCI1y1kaz7N1sgcjv9HED/bdVeo4oRbp0PQS5d7z ljfmf5nmYCkVXK/j2H+4EM1Hv85hEx6XNTtVRU3Eshf95Bcm4rHE0bHlWRuQELrfLGjpkeqTLI+Yb zBuQyBZUVl2l6j+RDlHA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc5q-00Ha47-4I; Fri, 13 May 2022 20:40:58 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbv9-00HVGS-4r for linux-rockchip@lists.infradead.org; Fri, 13 May 2022 20:29:59 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id B57C31F4648B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473794; bh=OwoOPzI0oWG8z/kV0ov2jyP52XLRugG702kjNP9bzdg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Q7eq3aSvGlkLfDeGPaYMdLh30T2S2rDV0GycIPESgZWoNx5zSVVAcpf2H/KF5doRZ gmwkc0ZTK8Mf2+h1tNvV0xc4fQLYuu2oX8+gNkudSv2vZR27p5q31MZaqybHuXKApr wxCGuaqrfVVPGZhOHr4C3yr+UtghJvZ+WmiwO1pfZGsM04frRUMrpqsRlZ1Nxbw7pv sCKW+/G6lyeO09+SbiMwnghbESL0ML40Kq3q+G5vAibBTxeEOKDOl4hzoIileCKI6T 5CnUEuUiWi8/FJT0NleaEvpNYyzQ+TIO6O4Xw6Qeqo261C+OUMOTNO0VG6p9pckk+0 XtXSf0MBGIenw== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, Hans Verkuil , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v5 12/20] media: rkvdec: h264: Validate and use pic width and height in mbs Date: Fri, 13 May 2022 16:29:14 -0400 Message-Id: <20220513202922.13846-13-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_132955_542104_C9C712B5 X-CRM114-Status: GOOD ( 21.81 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman The width and height in macroblocks is currently configured based on OUTPUT buffer resolution, this works for frame pictures but can cause issues for field pictures. When frame_mbs_only_flag is 0 the height in mbs should be height of the field instead of height of frame. Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1 against OUTPUT buffer resolution and use these values to configure HW. The validation is happening in both try_ctrt() and start() since it is otherwise possible to trick the driver during initialization by changing the OUTPUT format after having set a valid control. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia Signed-off-by: Hans Verkuil [hverkuil: when -> When (first word in a comment block)] --- drivers/staging/media/rkvdec/rkvdec-h264.c | 81 +++++++++++++++++----- 1 file changed, 62 insertions(+), 19 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 0dcbcb1bac80..fb41e2fd8359 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -672,8 +672,17 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO), DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG); - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS); - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS); + + /* + * Use the SPS values since they are already in macroblocks + * dimensions, height can be field height (halved) if + * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set and also it allows + * decoding smaller images into larger allocation which can be used + * to implementing SVC spatial layer support. + */ + WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS); + WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS); + WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY), FRAME_MBS_ONLY_FLAG); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD), @@ -1035,13 +1044,61 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx, return 0; } +static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx, + const struct v4l2_ctrl_h264_sps *sps) +{ + unsigned int width, height; + + /* + * TODO: The hardware supports 10-bit and 4:2:2 profiles, + * but it's currently broken in the driver. + * Reject them for now, until it's fixed. + */ + if (sps->chroma_format_idc > 1) + /* Only 4:0:0 and 4:2:0 are supported */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) + /* Luma and chroma bit depth mismatch */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != 0) + /* Only 8-bit is supported */ + return -EINVAL; + + width = (sps->pic_width_in_mbs_minus1 + 1) * 16; + height = (sps->pic_height_in_map_units_minus1 + 1) * 16; + + /* + * When frame_mbs_only_flag is not set, this is field height, + * which is half the final height (see (7-18) in the + * specification) + */ + if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY)) + height *= 2; + + if (width > ctx->coded_fmt.fmt.pix_mp.width || + height > ctx->coded_fmt.fmt.pix_mp.height) + return -EINVAL; + + return 0; +} + static int rkvdec_h264_start(struct rkvdec_ctx *ctx) { struct rkvdec_dev *rkvdec = ctx->dev; struct rkvdec_h264_priv_tbl *priv_tbl; struct rkvdec_h264_ctx *h264_ctx; + struct v4l2_ctrl *ctrl; int ret; + ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl, + V4L2_CID_STATELESS_H264_SPS); + if (!ctrl) + return -EINVAL; + + ret = rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps); + if (ret) + return ret; + h264_ctx = kzalloc(sizeof(*h264_ctx), GFP_KERNEL); if (!h264_ctx) return -ENOMEM; @@ -1139,23 +1196,9 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl) { - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps; - /* - * TODO: The hardware supports 10-bit and 4:2:2 profiles, - * but it's currently broken in the driver. - * Reject them for now, until it's fixed. - */ - if (sps->chroma_format_idc > 1) - /* Only 4:0:0 and 4:2:0 are supported */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) - /* Luma and chroma bit depth mismatch */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != 0) - /* Only 8-bit is supported */ - return -EINVAL; - } + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) + return rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps); + return 0; } From patchwork Fri May 13 20:29:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849482 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A98B6C433F5 for ; Fri, 13 May 2022 20:42:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=DybS3AP25z1JE6q3vWvnDawBptocQBcgU1AO0avp3Fw=; b=DL2/Zf5rcSd5OK Hv4aKKcN/G1RvIxzUbaSJhK1nYiS5XI85qglX9fFzoKuBlaHfkwN+GmW/JbFdVwH0IZouII4PBIYR 33j/kyA5uFLlCMosjrstkLbooUZkLJb0a1LIEYgHo1hLm5JQwPvplAnJSMYfbJ4Pddrir+84A6m0I nqr3eJFEXsYWneUzs6NI5knJR/FYmKN/GwwpzBYnBScAJGMrQw5JHCD/5W9DVg6IOgKf3Qeum+3Vn FhK7HWLi2aVSgE3KhL7hqqDptAViwnDDEQz8RoJzeHhJeOg0+o/qJJXOnaz6iwwvNyWRP3CqBJ28S 1iFriCPCW7vvXE0R/tng==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc6d-00Hab1-JO; Fri, 13 May 2022 20:41:47 +0000 Received: from desiato.infradead.org ([90.155.92.199]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbvj-00HVg3-Ea for linux-rockchip@bombadil.infradead.org; Fri, 13 May 2022 20:30:31 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=/AD9bW2iPnSe60c8xsO4dLOqYL4ulPeQkV4zFTrqJ9U=; b=DTdctgNIww2TRDwiHFaLVM2Dx2 /uKjnOGUZyo/7oWIxzbehxXgK0uBaRz5gNGusK47lPzCMALBlQv470RUfB9f+wSpP1yFcuJ5b2i9d 6zIqrHQvqjpCdb7NmbzNCHmSI+zOpe5N7xFDXu+1CYTdKfu5e+a/YxX0oW0eR+n46QB2c2IxYyIj7 0ZogmltWOsdCX4QHi6CiTGHhyqKzfXj8hczNAL4hE6sJu63Ne0teLFzIC2SiJc5hukFmEQVE1oYKI EiBwYIvz19zVynWnq0PXWW4tLJrGP2ZgENsic79aC8SzUJyKr5LgvcrtWWVZmlVxAJJEN05TGciq/ XAoD649A==; Received: from bhuna.collabora.co.uk ([46.235.227.227]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbvS-0006Wr-9L for linux-rockchip@lists.infradead.org; Fri, 13 May 2022 20:30:20 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 7FC571F46483 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473795; bh=HEETDTFaV8Jr4FyaHgjigxD+fRrzRsYiKM7mEOGoNmg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DRcfnh3oM2e+iY5OVgfBPViyB1R8HzRN7EGwmsv0HiWeiKn8/prlTyovriPq6aAGi YelkJEHGWZzlu8MFXOSCTG+7/BkAjSe4JxuTh7oqVNvOG+WguOdn7SRa+eNFe9loh7 C9TlKiLRKEjVZgR7D6mqqyxBOQOlWBcZ49QucyW8xAet+9K4cCJfTmgKWC26E32uoN S7QrSxgu3duuuaGPweLKsq9EJqjEKT/g0BY98q5IKNH75Ka2UPQ///34Z222qALPg2 HhyODvB2NtFiJ/ratDO2d5/qXSl8JFIWZXZjgahYXcnvlEqD6mpgFmnfh4pPCmVEYB kQnmBKFOn7gHw== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, Ezequiel Garcia , Sebastian Fricke , Hans Verkuil , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v5 13/20] media: rkvdec: h264: Fix reference frame_num wrap for second field Date: Fri, 13 May 2022 16:29:15 -0400 Message-Id: <20220513202922.13846-14-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_213018_370229_A0B0B755 X-CRM114-Status: GOOD ( 11.01 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman When decoding the second field in a complementary field pair the second field is sharing the same frame_num with the first field. Currently the frame_num for the first field is wrapped when it matches the field being decoded, this caused issues decoding the second field in a complementary field pair. Fix this by using inclusive comparison: 'less than or equal'. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia Reviewed-by: Sebastian Fricke Signed-off-by: Hans Verkuil --- drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index fb41e2fd8359..57821ee3b213 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -782,7 +782,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, continue; if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM || - dpb[i].frame_num < dec_params->frame_num) { + dpb[i].frame_num <= dec_params->frame_num) { p[i] = dpb[i].frame_num; continue; } From patchwork Fri May 13 20:29:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849484 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ECA3DC433F5 for ; Fri, 13 May 2022 20:42:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=njE8TUDInD8/Q58GKHbp1RQJ+TEEQvFpW5wzEbw3Stw=; b=z4zqVlFcXJZcxC 9yeShYxyKbIqVkfLTpDUvG7Qv2HCI9c/qZjCeoYYoUL0AtR2Ul0BanUnxsK/uX5HGpMAXTjlR7XHJ nUsfTZToAhI2MG5kIVcMEAVW/MgMFs59U2WPm+2w8SSH7neshTrMizKReYYa0sW/NTCs8AXE06hJP 8NIrJoFgo8RP5kq8c7Xe+lEEBgl7DikDOasRD24tehx7Yd0ABEi5bwuzGLzRbwObZu919fLFJxX5F 3dXNAZgTQ934DYzGy/grHk31C4Gm2ZNsmbgT1ubktgyLygJ72dwsq4i1IBcbMc3cqBFLXtu0GKwt+ wK5rbTJCsD3FBMGVXGYw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc6u-00HaoG-Jm; Fri, 13 May 2022 20:42:04 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc3q-00HZ4Z-IV for linux-rockchip@lists.infradead.org; Fri, 13 May 2022 20:38:56 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 7A9CA1F4648D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473797; bh=LjTJxs5qEzAa3nkEOUZdNzT48i0a0uaCnC2plhBS5zs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MB455OrcIQrH7tKLVa186WBFXhoBTk3j9/bJ1hrcjrvcHSQHqA6EwiPs6GGOEeZob V+DfjddNdMqFmU6rmQcEchWJby9cDdPUJw5SO5SK7U4MPMqB+ogTgml+zkvfr5MA5k YnXZZ8gBqlpFsIqS7qI00Pjf6/uxa6dumwVNRymz3MbwhW5lbd2g+gYi/vx+Am9+MK uN5lrYQF8qQZh6VtfcvqpDya5hiUzDHoZIzXamNFkDTW0nPZKejdEG68SzWa41nhOu AiY0YAfkmVcARIYspGSBjLbIBXFNP6GvdnTMOtLOX5aF19SAu4hVk8oXtgvkSzKF0j dGav5ndnQobcQ== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, Sebastian Fricke , Hans Verkuil , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v5 14/20] media: rkvdec: Ensure decoded resolution fit coded resolution Date: Fri, 13 May 2022 16:29:16 -0400 Message-Id: <20220513202922.13846-15-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_133854_830789_C53198EB X-CRM114-Status: UNSURE ( 8.40 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman Ensure decoded CAPTURE buffer resolution is larger or equal to the coded OUTPUT buffer resolution. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Signed-off-by: Hans Verkuil --- drivers/staging/media/rkvdec/rkvdec.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index e3d44d5b35f3..2bc4b1a40989 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -257,6 +257,8 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv, pix_mp->pixelformat = coded_desc->decoded_fmts[0]; /* Always apply the frmsize constraint of the coded end. */ + pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width); + pix_mp->height = max(pix_mp->height, ctx->coded_fmt.fmt.pix_mp.height); v4l2_apply_frmsize_constraints(&pix_mp->width, &pix_mp->height, &coded_desc->frmsize); From patchwork Fri May 13 20:29:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C6A1FC433FE for ; Fri, 13 May 2022 20:41:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=DqPNh0NcblucLNvbMPEURlXrI9Mv9zyLY9jK7Bqmzvw=; b=HS+aiz6DumKNQ7 T0vHBgvOP8c9snSVwSA4+K9WiBGlOb563REWEBf7uOuq5q4GmP0Sgb6yxqx5leVc5+A1Na8wLh6jW k74NZFoeZ6Etlp8JgXTzKyWZzB4U2Qe0LGWJAWEqQFhWQxDLryuJ8E4w95zBcusKTIRsuaqE3LHfL 9Ug7Wy0e4695sqHC6TmzOlYgbfGGst38vRktbao+k7l2nx0AzxcTOKpZjhoAiBwcVs5nAz4y3nOUs ZgkFx11G2MbvcU/NtRRXzhTDV3mEbWNQkdUTEJfb5zG5W3eWYCTNgn2qoLK0bvaCnGqIglwm4df/V atIzHBoH4Rr8GA+ra1+w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc5s-00Ha5a-I6; Fri, 13 May 2022 20:41:00 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbvF-00HVKT-1U for linux-rockchip@lists.infradead.org; Fri, 13 May 2022 20:30:04 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 6A8191F46486 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473799; bh=wcSMBW1PE3loIEvEqZw8GdQcBUBFBq3QMi5QVRrm7q4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AAOJUJfjinwTVzoF16VNlMVZuBL8JyDZ0qxQC6pArhTPuqkSsPn0aXGTzmMGfZL2K QUiKSncCODxMRfUPt0gEEdHyIrK30fHPQguskY09sj87Z+XVulPghWVqL+Edq1MbIF CRFI0+VB6Go7UJQtbDiISYF4awLHMZ+pp86sYdS8ELc14yUv3pqWcMTPbRDfahDrUd ix+Cdy3F7MqtVuXoOENTn9KOSmFM+TSOKRV1IfAdXSfZkMPaG2lf+UG/eVWiysHpqr uKdAZaCrxCBcKZNO68HN9tXNwYmbdpFsftd2nJ7DzDWJ9abiL8sllogJ5PxT3lZgG+ 0A1TbgDPOKiUA== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , Hans Verkuil , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v5 15/20] media: rkvdec-h264: Add field decoding support Date: Fri, 13 May 2022 16:29:17 -0400 Message-Id: <20220513202922.13846-16-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_133001_438557_52FA66AD X-CRM114-Status: GOOD ( 17.94 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org This makes use of the new feature in the reference builder to program up to 32 references when doing field decoding. It also signals the parity (top or bottom) of the field to the hardware. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Signed-off-by: Hans Verkuil --- drivers/staging/media/rkvdec/rkvdec-h264.c | 48 ++++++++++------------ 1 file changed, 21 insertions(+), 27 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 57821ee3b213..2992fb87cf72 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -97,13 +97,10 @@ struct rkvdec_h264_priv_tbl { u8 err_info[RKV_ERROR_INFO_SIZE]; }; -#define RKVDEC_H264_DPB_SIZE 16 - struct rkvdec_h264_reflists { struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; - u8 num_valid; }; struct rkvdec_h264_run { @@ -747,23 +744,26 @@ static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx, struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; int buf_idx = -1; - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) { buf_idx = vb2_find_timestamp(cap_q, dpb[i].reference_ts, 0); + if (buf_idx < 0) + pr_debug("No buffer for reference_ts %llu", + dpb[i].reference_ts); + } run->ref_buf_idx[i] = buf_idx; } } static void assemble_hw_rps(struct rkvdec_ctx *ctx, + struct v4l2_h264_reflist_builder *builder, struct rkvdec_h264_run *run) { const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params; const struct v4l2_h264_dpb_entry *dpb = dec_params->dpb; struct rkvdec_h264_ctx *h264_ctx = ctx->priv; - const struct v4l2_ctrl_h264_sps *sps = run->sps; struct rkvdec_h264_priv_tbl *priv_tbl = h264_ctx->priv_tbl.cpu; - u32 max_frame_num = 1 << (sps->log2_max_frame_num_minus4 + 4); u32 *hw_rps = priv_tbl->rps; u32 i, j; @@ -781,37 +781,36 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) continue; - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM || - dpb[i].frame_num <= dec_params->frame_num) { - p[i] = dpb[i].frame_num; - continue; - } - - p[i] = dpb[i].frame_num - max_frame_num; + p[i] = builder->refs[i].frame_num; } for (j = 0; j < RKVDEC_NUM_REFLIST; j++) { - for (i = 0; i < h264_ctx->reflists.num_valid; i++) { - bool dpb_valid = run->ref_buf_idx[i] >= 0; - u8 idx = 0; + for (i = 0; i < builder->num_valid; i++) { + struct v4l2_h264_reference *ref; + bool dpb_valid; + bool bottom; switch (j) { case 0: - idx = h264_ctx->reflists.p[i].index; + ref = &h264_ctx->reflists.p[i]; break; case 1: - idx = h264_ctx->reflists.b0[i].index; + ref = &h264_ctx->reflists.b0[i]; break; case 2: - idx = h264_ctx->reflists.b1[i].index; + ref = &h264_ctx->reflists.b1[i]; break; } - if (idx >= ARRAY_SIZE(dec_params->dpb)) + if (WARN_ON(ref->index >= ARRAY_SIZE(dec_params->dpb))) continue; + dpb_valid = run->ref_buf_idx[ref->index] >= 0; + bottom = ref->fields == V4L2_H264_BOTTOM_FIELD_REF; + set_ps_field(hw_rps, DPB_INFO(i, j), - idx | dpb_valid << 4); + ref->index | dpb_valid << 4); + set_ps_field(hw_rps, BOTTOM_FLAG(i, j), bottom); } } } @@ -999,10 +998,6 @@ static void config_registers(struct rkvdec_ctx *ctx, rkvdec->regs + RKVDEC_REG_H264_BASE_REFER15); } - /* - * Since support frame mode only - * top_field_order_cnt is the same as bottom_field_order_cnt - */ reg = RKVDEC_CUR_POC(dec_params->top_field_order_cnt); writel_relaxed(reg, rkvdec->regs + RKVDEC_REG_CUR_POC0); @@ -1166,7 +1161,6 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) /* Build the P/B{0,1} ref lists. */ v4l2_h264_init_reflist_builder(&reflist_builder, run.decode_params, run.sps, run.decode_params->dpb); - h264_ctx->reflists.num_valid = reflist_builder.num_valid; v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p); v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0, h264_ctx->reflists.b1); @@ -1174,7 +1168,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) assemble_hw_scaling_list(ctx, &run); assemble_hw_pps(ctx, &run); lookup_ref_buf_idx(ctx, &run); - assemble_hw_rps(ctx, &run); + assemble_hw_rps(ctx, &reflist_builder, &run); config_registers(ctx, &run); rkvdec_run_postamble(ctx, &run.base); From patchwork Fri May 13 20:29:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849476 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3D2EAC433F5 for ; Fri, 13 May 2022 20:41:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=O7AAgXKjQVqklx3SqmWrWlmzXlCZ1awa6FyynxMBI08=; b=odEg4YTp/irGzS mcbENPeWd9rXss1ok71jzZd1CK4Qa3rG8Swz4A4LzYb1OkgHOs0I1MU0ka+n2m18IhuqSXQDKmP+G TyljglT24fzd3fmGHW9+rWEHMXF1fxvr2eSZOTf+hU0TT6fWqXjO8CZwbditGwMaMWOFYOeW/mtg4 eA9TDp5w40QTDlECcPXpzw2hxT/b0Q7dZmapF/6hjMMoBvBRVI14wvKLeUgQmlRBc6xts/26dZyeS 0JyYZQg+cvVC6qyatuY0FkD11VZVpxQuMwI8Cc1iR0KnNTmw+USGW88CwLN3sMu5cQrJOXJdP3cAA mcwio8hQqr8tQmmRVBUg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc5v-00Ha6y-17; Fri, 13 May 2022 20:41:03 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbvG-00HVLu-QZ for linux-rockchip@lists.infradead.org; Fri, 13 May 2022 20:30:04 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 556061F46480 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473801; bh=97qH9MAwt4IfrnTZaZQXT3dbPZPKSF1LMKpA5X+Ai/k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fAOU94QvsZuULWUCMD5y91dOEGbHjkpuyz4xFLajyvtMqF2+bqnHjaVjUQ6TmsEF1 PbOCu26qy7SRC13hkZwUCHq0+1mreVQyQHIJxwGvgvJBrtHtV+Qyw0VbLlyygkNTNh zDkqYi2Rq3Mj9NCP0QZA6tOS7YUK1EMBpXk7DXhwK2BrcDJ88oQbNadoZcMnG7WE+2 OCmqL1SknN71gqGccgItegAPjfq2HJRlL7LM6HeNOOjlDv7nsVa7ljQTaF3uI1Rcbm Nye7rk42/XaiF6/9mjjdVW63YpqNH7OKF7FC54UBpzBZnokZq0eivHSGmqFbxBETnL HdZ7HEA+rXMOw== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , Hans Verkuil , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v5 16/20] media: rkvdec: Enable capture buffer holding for H264 Date: Fri, 13 May 2022 16:29:18 -0400 Message-Id: <20220513202922.13846-17-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_133003_070265_B47F509D X-CRM114-Status: GOOD ( 11.74 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org In order to support interlaced video decoding, the driver must allow holding the capture buffer so that the second field can be decoded into it. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Signed-off-by: Hans Verkuil --- drivers/staging/media/rkvdec/rkvdec.c | 4 ++++ drivers/staging/media/rkvdec/rkvdec.h | 1 + 2 files changed, 5 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 2bc4b1a40989..7bab7586918c 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -127,6 +127,7 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = { .ops = &rkvdec_h264_fmt_ops, .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts), .decoded_fmts = rkvdec_h264_vp9_decoded_fmts, + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, }, { .fourcc = V4L2_PIX_FMT_VP9_FRAME, @@ -385,6 +386,9 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv, cap_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; cap_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization; + /* Enable format specific queue features */ + vq->subsystem_flags |= desc->subsystem_flags; + return 0; } diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h index 9df0fba799a4..633335ebb9c4 100644 --- a/drivers/staging/media/rkvdec/rkvdec.h +++ b/drivers/staging/media/rkvdec/rkvdec.h @@ -82,6 +82,7 @@ struct rkvdec_coded_fmt_desc { const struct rkvdec_coded_fmt_ops *ops; unsigned int num_decoded_fmts; const u32 *decoded_fmts; + u32 subsystem_flags; }; struct rkvdec_dev { From patchwork Fri May 13 20:29:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C25BEC433F5 for ; Fri, 13 May 2022 20:42:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JbfvuBbImBwHh39Fbzx+5mlKJxpknWULltvrb0HJCDA=; b=s7iIFRLrAXPY2q Tb8YTOXlG82zGgjKBkaLiZxZ+HvlatuL+Zex8BGpIhPNHiE4hQYQkh+oxbRdHqkCjI8DYIZjqTYH2 +vx41jLs9SITjseVnesLGO1JCRcYEPWrD/Ic16iCrmVO45CbjjjCHHtSVPRycwf6jF9yGiML47Tsn QZhwJaav/r4S+OiYU/dPMZKE5IeOfuyy9NwymXJfKIw0ZaFNQv2HWslzJpu8vgvse4gYv5/yWGjCk k+Q6wPtGDlG1LaLY9epfoDmNeHmmZoDpHfCr7sd9LdevPqXi3CkSeH4UmVroozbSe9HcCjit+z+N7 nT0nPxsshXG32iE0Mt2w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc6q-00Hal4-38; Fri, 13 May 2022 20:42:00 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc3q-00HZ4Y-IT for linux-rockchip@lists.infradead.org; Fri, 13 May 2022 20:38:56 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 20D1E1F46491 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473803; bh=alVT6vQf62mKVLIbN1ZQcw7iCvLKw2erGbicRcgWoFw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=f7/OJULvIcqwg6BCOeOzihRiWBBA1N2k0JlV3VV3gUA558pAkZH8uMs+dxR8TK/VU xTGC9GVIWPX0ODnx0xEGeIkXJLKjC2A/97uQDSuJugCBBppKZ7h836g8QkzCynZyvP OZPhu2DMF2q7J/6QVvRsEBR/nOeCZW4D08LnKQDSLbHG4ZwjV/c+VFxTz1a/+y/B0v YsbXrHTOvLjBgHEclouJ/bKS2men49VFmZgjpDh8ACltoaV9xfN6ZKxfohlar+2BCs VjDItssViDl5W3DrGQpLkg8iBaoGzcoLiuWELrzejuOww1qVCOr/nvqsmK1JMfQEhy y3xub2XUcvCEA== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , Hans Verkuil , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v5 17/20] media: hantro: Stop using H.264 parameter pic_num Date: Fri, 13 May 2022 16:29:19 -0400 Message-Id: <20220513202922.13846-18-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_133854_796317_E6D18572 X-CRM114-Status: UNSURE ( 8.60 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org The hardware expects FrameNumWrap or long_term_frame_idx. Picture numbers are per field, and are mostly used during the memory management process, which is done in userland. This fixes two ITU conformance tests: - MR6_BT_B - MR8_BT_B Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Signed-off-by: Hans Verkuil --- drivers/staging/media/hantro/hantro_h264.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 0b4d2491be3b..228629fb3cdf 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -354,8 +354,6 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) return 0; - if (dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) - return dpb->pic_num; return dpb->frame_num; } From patchwork Fri May 13 20:29:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849478 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 08C9FC433EF for ; Fri, 13 May 2022 20:41:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=KCIkti282G2Q0TdlxEKT4CDukgu8HdsVG+nozX8ASTo=; b=IrwxkFN+nsXO9S RDSSkRQDKRZru7VWyP368Xbanox4pcmNHAOpQm0BIrFZIfoNc6myHRpCWdg5RWNVSgSUn0Kzsk6qQ /ITdCynG+4S914ZVMWFqtkf5tRY7DglyvU4EqcpZXoNaxrngeJL4dxFSGdqhB++JhNOhjpULYOeFU qWq6QWV+GX99oM6FsOImDMJ8zpYRDAFzh3FAQxenSxNXooA+ukOLWaCVlBe6C0zNcI01G45flFpUk aCyK4IjlCCMtNX2bdc20oRxbnhMItoNIG9QsD0H3n8RNGxmYPLb8/0Qs/tGFijIu1CCG6+4b7g2cQ 5BpJaJBLaVyrz6hM4jBw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc67-00HaG9-O4; Fri, 13 May 2022 20:41:16 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbvK-00HVP7-Nq for linux-rockchip@lists.infradead.org; Fri, 13 May 2022 20:30:11 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 047D41F4648F DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473805; bh=v1JZAts5kOJok04nYryeXLmZ+bVQm+vS+8ZHy3WqOLI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kJL4NSPR0h0kk5EdUKp4jVtaHAKbXWhSkRC4UrsM0mrxjvkff2gcKuJGbZxai5zgM yS/DrK5x3WIMebABA23xBsP8US+Gk5zkkWCSoHzqeSReNtdwKKMF8HF8VyVjW+/K6Y 1+GMqBnkLl1QpBQRs/oXwfD4sXzvnZ9oFax4e9cGZi3p3Tlwb48U5V8Lvq0GXGiQPc zAJr77j9ThTH2+ZcwHlFbFntQ9C0mIk4DfBGh9NIt1XxsoPL81tvgQ9MRLx2rQyPSD +A6tzvVcsU641c20bXdX5fzZ613S5icZKp1pmYO8sOvtgtmChaYJMFIAmaSVMJcqp7 EnsoVvLhu4yHA== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, Hans Verkuil , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v5 18/20] media: hantro: h264: Make dpb entry management more robust Date: Fri, 13 May 2022 16:29:20 -0400 Message-Id: <20220513202922.13846-19-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_133007_025403_7F2C5635 X-CRM114-Status: GOOD ( 12.41 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman The driver maintains stable slot locations for reference pictures. This change makes the code more robust by using the reference_ts as key and by marking all entries invalid right from the start. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Signed-off-by: Hans Verkuil --- drivers/staging/media/hantro/hantro_h264.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 228629fb3cdf..7377fc26f780 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -258,8 +258,7 @@ static void prepare_table(struct hantro_ctx *ctx) static bool dpb_entry_match(const struct v4l2_h264_dpb_entry *a, const struct v4l2_h264_dpb_entry *b) { - return a->top_field_order_cnt == b->top_field_order_cnt && - a->bottom_field_order_cnt == b->bottom_field_order_cnt; + return a->reference_ts == b->reference_ts; } static void update_dpb(struct hantro_ctx *ctx) @@ -273,13 +272,13 @@ static void update_dpb(struct hantro_ctx *ctx) /* Disable all entries by default. */ for (i = 0; i < ARRAY_SIZE(ctx->h264_dec.dpb); i++) - ctx->h264_dec.dpb[i].flags &= ~V4L2_H264_DPB_ENTRY_FLAG_ACTIVE; + ctx->h264_dec.dpb[i].flags = 0; /* Try to match new DPB entries with existing ones by their POCs. */ for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) { const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i]; - if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) + if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) continue; /* @@ -290,8 +289,7 @@ static void update_dpb(struct hantro_ctx *ctx) struct v4l2_h264_dpb_entry *cdpb; cdpb = &ctx->h264_dec.dpb[j]; - if (cdpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE || - !dpb_entry_match(cdpb, ndpb)) + if (!dpb_entry_match(cdpb, ndpb)) continue; *cdpb = *ndpb; From patchwork Fri May 13 20:29:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BA48BC433F5 for ; Fri, 13 May 2022 20:41:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=SFouK7fUF061JkQ89mBMOJFzdZBrK/75p2/7vTPoGAU=; b=OxTUvrqL+3cZYR rpVG0XqzA7v99E23S+7E1A+e34ziZAXYC0GsKznSnH/diDFEDs/GmyEI49aSwoBMUUVFBUzH3D+8N CLkr3tekSx09CyIRi7moAgi+qxsidiHdxObyHj6/asG3XJ174dfbbBdxehUxi+lT87ameWShQtiwh SttbhS51ifzr/OKRTAMFkYRBONRkF8WDs7yG/PODjh4PT3tp7dpwDy4rzt7kJd7dVOEwabkiSz+mZ x1t9Wf39/raJ9jhM7KyDqmdqhuZ8lZt9yT8RJJNEclbdByzwuMNhah1CuY3qiXqQ953eTIm5De491 tz2/gMYAI4tXnsdXiFNA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc6H-00HaLl-Gp; Fri, 13 May 2022 20:41:25 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbvM-00HVRD-NI for linux-rockchip@lists.infradead.org; Fri, 13 May 2022 20:30:13 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 153881F46487 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473807; bh=Qsy9WCWxktbVBKy9C7ak7cTtluRSu2/yoKiiFbx2xF8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DEXSS5t9vW0a8BxQea8RFLoepwd1+Zj1u8hf5g5eG62BHhQ+W9MsMucOgTnn71SIo EBo4VuPdciQi09u7N7zJ2/m5qswQEXl2erJX4pc3zL+VzzGmUuWWcbMv5Z3iSBsOf2 U1rWvBBwjEmK1CfpiBjxP25Jx8LnLPTWqsPEBGK2VZ7j/Uw0aB9FMyRlOlqtX879Qh 7o/tfNN2P5jBb/h1qwAPWXgBU1/tbrd0fWViMlr0taeC2YDa9SlP6il71OH3V/0/0t nC8S7OCWnJHX6xfm45nGpkaHYrgV3b7JxzCARmOvT3tFAoDJXZlCvbRFL4df06QU4l KB4988InjDr2w== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Jonas Karlman , Hans Verkuil , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v5 19/20] media: hantro: Add H.264 field decoding support Date: Fri, 13 May 2022 16:29:21 -0400 Message-Id: <20220513202922.13846-20-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_133009_162665_DA08C6BA X-CRM114-Status: GOOD ( 26.63 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org This adds the required code to support field decoding. While most of the code is derived from Rockchip and VSI reference code, the reduction of the reference list to 16 entries was found by trial and errors. The list consists of all the references with the opposite field parity. The strategy is to deduplicate the reference picture that points to the same storage (same index). The choice of opposite parity has been made to keep the other field of the current field pair in the list. This method may not be robust if a field was lost. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Signed-off-by: Hans Verkuil [hverkuil: fix typos in the comment before deduplicate_reflist()] [hverkuil: document new cur_poc field] --- drivers/staging/media/hantro/hantro_h264.c | 122 ++++++++++++++++++--- drivers/staging/media/hantro/hantro_hw.h | 2 + 2 files changed, 110 insertions(+), 14 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 7377fc26f780..4e9a0ecf5c13 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -22,6 +22,12 @@ #define POC_BUFFER_SIZE 34 #define SCALING_LIST_SIZE (6 * 16 + 2 * 64) +/* + * For valid and long term reference marking, index are reversed, so bit 31 + * indicates the status of the picture 0. + */ +#define REF_BIT(i) BIT(32 - 1 - (i)) + /* Data structure describing auxiliary buffer format. */ struct hantro_h264_dec_priv_tbl { u32 cabac_table[CABAC_INIT_BUFFER_SIZE]; @@ -227,6 +233,7 @@ static void prepare_table(struct hantro_ctx *ctx) { const struct hantro_h264_dec_ctrls *ctrls = &ctx->h264_dec.ctrls; const struct v4l2_ctrl_h264_decode_params *dec_param = ctrls->decode; + const struct v4l2_ctrl_h264_sps *sps = ctrls->sps; struct hantro_h264_dec_priv_tbl *tbl = ctx->h264_dec.priv.cpu; const struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb; u32 dpb_longterm = 0; @@ -237,20 +244,45 @@ static void prepare_table(struct hantro_ctx *ctx) tbl->poc[i * 2] = dpb[i].top_field_order_cnt; tbl->poc[i * 2 + 1] = dpb[i].bottom_field_order_cnt; + if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) + continue; + /* * Set up bit maps of valid and long term DPBs. - * NOTE: The bits are reversed, i.e. MSb is DPB 0. + * NOTE: The bits are reversed, i.e. MSb is DPB 0. For frame + * decoding, bit 31 to 15 are used, while for field decoding, + * all bits are used, with bit 31 being a top field, 30 a bottom + * field and so on. */ - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) - dpb_valid |= BIT(HANTRO_H264_DPB_SIZE - 1 - i); - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) - dpb_longterm |= BIT(HANTRO_H264_DPB_SIZE - 1 - i); + if (dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) { + if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF) + dpb_valid |= REF_BIT(i * 2); + + if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF) + dpb_valid |= REF_BIT(i * 2 + 1); + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) { + dpb_longterm |= REF_BIT(i * 2); + dpb_longterm |= REF_BIT(i * 2 + 1); + } + } else { + dpb_valid |= REF_BIT(i); + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) + dpb_longterm |= REF_BIT(i); + } + } + ctx->h264_dec.dpb_valid = dpb_valid; + ctx->h264_dec.dpb_longterm = dpb_longterm; + + if ((dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) || + !(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)) { + tbl->poc[32] = ctx->h264_dec.cur_poc; + tbl->poc[33] = 0; + } else { + tbl->poc[32] = dec_param->top_field_order_cnt; + tbl->poc[33] = dec_param->bottom_field_order_cnt; } - ctx->h264_dec.dpb_valid = dpb_valid << 16; - ctx->h264_dec.dpb_longterm = dpb_longterm << 16; - - tbl->poc[32] = dec_param->top_field_order_cnt; - tbl->poc[33] = dec_param->bottom_field_order_cnt; assemble_scaling_list(ctx); } @@ -326,6 +358,8 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx, { struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb; dma_addr_t dma_addr = 0; + s32 cur_poc = ctx->h264_dec.cur_poc; + u32 flags; if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) dma_addr = hantro_get_ref(ctx, dpb[dpb_idx].reference_ts); @@ -343,7 +377,12 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx, dma_addr = hantro_get_dec_buf_addr(ctx, buf); } - return dma_addr; + flags = dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_FIELD ? 0x2 : 0; + flags |= abs(dpb[dpb_idx].top_field_order_cnt - cur_poc) < + abs(dpb[dpb_idx].bottom_field_order_cnt - cur_poc) ? + 0x1 : 0; + + return dma_addr | flags; } u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) @@ -355,6 +394,47 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) return dpb->frame_num; } +/* + * Removes all references with the same parity as the current picture from the + * reference list. The remaining list will have references with the opposite + * parity. This is effectively a deduplication of references since each buffer + * stores two fields. For this reason, each buffer is found twice in the + * reference list. + * + * This technique has been chosen through trial and error. This simple approach + * resulted in the highest conformance score. Note that this method may suffer + * worse quality in the case an opposite reference frame has been lost. If this + * becomes a problem in the future, it should be possible to add a preprocessing + * to identify un-paired fields and avoid removing them. + */ +static void deduplicate_reflist(struct v4l2_h264_reflist_builder *b, + struct v4l2_h264_reference *reflist) +{ + int write_idx = 0; + int i; + + if (b->cur_pic_fields == V4L2_H264_FRAME_REF) { + write_idx = b->num_valid; + goto done; + } + + for (i = 0; i < b->num_valid; i++) { + if (!(b->cur_pic_fields == reflist[i].fields)) { + reflist[write_idx++] = reflist[i]; + continue; + } + } + +done: + /* Should not happen unless we have a bug in the reflist builder. */ + if (WARN_ON(write_idx > 16)) + write_idx = 16; + + /* Clear the remaining, some streams fails otherwise */ + for (; write_idx < 16; write_idx++) + reflist[write_idx].index = 15; +} + int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx) { struct hantro_h264_dec_hw_ctx *h264_ctx = &ctx->h264_dec; @@ -386,15 +466,29 @@ int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx) /* Update the DPB with new refs. */ update_dpb(ctx); - /* Prepare data in memory. */ - prepare_table(ctx); - /* Build the P/B{0,1} ref lists. */ v4l2_h264_init_reflist_builder(&reflist_builder, ctrls->decode, ctrls->sps, ctx->h264_dec.dpb); + h264_ctx->cur_poc = reflist_builder.cur_pic_order_count; + + /* Prepare data in memory. */ + prepare_table(ctx); + v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p); v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0, h264_ctx->reflists.b1); + + /* + * Reduce ref lists to at most 16 entries, Hantro hardware will deduce + * the actual picture lists in field through the dpb_valid, + * dpb_longterm bitmap along with the current frame parity. + */ + if (reflist_builder.cur_pic_fields != V4L2_H264_FRAME_REF) { + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.p); + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b0); + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b1); + } + return 0; } diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 0b5b9da86c43..52a960f6fa4a 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -83,6 +83,7 @@ struct hantro_h264_dec_reflists { * @ctrls: V4L2 controls attached to a run * @dpb_longterm: DPB long-term * @dpb_valid: DPB valid + * @cur_poc: Current picture order count */ struct hantro_h264_dec_hw_ctx { struct hantro_aux_buf priv; @@ -91,6 +92,7 @@ struct hantro_h264_dec_hw_ctx { struct hantro_h264_dec_ctrls ctrls; u32 dpb_longterm; u32 dpb_valid; + s32 cur_poc; }; /** From patchwork Fri May 13 20:29:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12849480 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C5B2CC433EF for ; Fri, 13 May 2022 20:41:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Pt+XJ3Zlr3WsSVQrwiKeZbDPwHMvZd+fYiPzWB7Uxls=; b=nlQRBZzlVeINyC EyvtqTPU59vuR2ffQcY34UVBGy9N7dSPjdqbUJ8OLmohyHmsWJZzLLHBWbWO151P5okeF0IwAsJyL ygKu0IAWa+nTgXrM7iMsGmjmxiiHpSCcBELa6l08qZpdeoYJ+vp4yOcc8HQNMdIPneDx3jM6R0S29 7NaUas3qou0/SKXPINIVyZydvqkZo44JNA29JYzExIn+ldUU2qrGeWM2Rwph79rFWmBpEvVrghfu+ nAAtAsEKuI1V13ytJQu6VIXPMqCs/+zxwQQtYetor+vRDEs+YivVpQ1hSA8wqWCXTE7Y5MR/IOAGg csZPXVcMXltyrKquw3sQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1npc6Q-00HaQr-Vn; Fri, 13 May 2022 20:41:35 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1npbvO-00HVSk-Bh for linux-rockchip@lists.infradead.org; Fri, 13 May 2022 20:30:12 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id E733F1F46488 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1652473809; bh=OIhbnD+GYKGESjhoWfLHkSeEKZZ1mESH1BJd09/VI6M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=bGgZSkmlNeEq1zqsBAQ6J/QOV0+EU2s34r7sIdcuq7QFsqawbnepKngMCh61jgJhb N4QzmDr5jRBoOubx6Jot6ecXLzlAHuRNlkxCLgae9AB+779PwnWuU14zbXr7HhLxnV 7UVERLCsKPqD9oJ9j6+525Q0QScfBxvmXp7S512qpp0xna7bFwb1jMCHjGeo5FtB+6 8dxVa9nBvkioolmbl9Xavcft1MohKl9Ieg2cSKWVt6zmRbuBZahPNa/hm7R8B6EynC NAk2Wr4eqyUb5MjnAyINxTahI9gwHZ5/I20f9sHKDUs8Hq026Va/QJ1pYd+OB1lyY2 cRYhzAVQ8w1Pg== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , Hans Verkuil , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v5 20/20] media: hantro: Enable HOLD_CAPTURE_BUF for H.264 Date: Fri, 13 May 2022 16:29:22 -0400 Message-Id: <20220513202922.13846-21-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.3 In-Reply-To: <20220513202922.13846-1-nicolas.dufresne@collabora.com> References: <20220513202922.13846-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220513_133010_614825_1F32603C X-CRM114-Status: GOOD ( 10.23 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org This is needed to optimize field decoding. Each field will be decoded into the same capture buffer. To be able to queue multiple buffers, we need to be able to ask the driver to hold the capture buffer. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Signed-off-by: Hans Verkuil --- drivers/staging/media/hantro/hantro_v4l2.c | 25 ++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c index ed458866257a..22ad182ee972 100644 --- a/drivers/staging/media/hantro/hantro_v4l2.c +++ b/drivers/staging/media/hantro/hantro_v4l2.c @@ -408,6 +408,30 @@ hantro_update_requires_request(struct hantro_ctx *ctx, u32 fourcc) } } +static void +hantro_update_requires_hold_capture_buf(struct hantro_ctx *ctx, u32 fourcc) +{ + struct vb2_queue *vq; + + vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + + switch (fourcc) { + case V4L2_PIX_FMT_JPEG: + case V4L2_PIX_FMT_MPEG2_SLICE: + case V4L2_PIX_FMT_VP8_FRAME: + case V4L2_PIX_FMT_HEVC_SLICE: + case V4L2_PIX_FMT_VP9_FRAME: + vq->subsystem_flags &= ~(VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF); + break; + case V4L2_PIX_FMT_H264_SLICE: + vq->subsystem_flags |= VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF; + break; + default: + break; + } +} + static int hantro_set_fmt_out(struct hantro_ctx *ctx, struct v4l2_pix_format_mplane *pix_mp) { @@ -471,6 +495,7 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx, ctx->dst_fmt.quantization = pix_mp->quantization; hantro_update_requires_request(ctx, pix_mp->pixelformat); + hantro_update_requires_hold_capture_buf(ctx, pix_mp->pixelformat); vpu_debug(0, "OUTPUT codec mode: %d\n", ctx->vpu_src_fmt->codec_mode); vpu_debug(0, "fmt - w: %d, h: %d\n",