From patchwork Tue Apr 26 12:57:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827098 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0C756C433EF for ; Tue, 26 Apr 2022 13:07:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0psb3ZykwSnwXnvVbrTgq9zs0CIFL0o8ntF7h3OkPNg=; b=QRhwt4Y8gth155 Fp7Aqc2BCgiZtHfbG0+8uJBQTZWlq8qDmX3hX8iIkLXqNeuTMySyXETHSuZO2QD1IYPuaGuv1Seov v7ELweDLXiq0//+J3c3kb7OztcRs/ZuJCPf0G/cP9mQJNWtBfDyDyvtnA83PigTOV3/37O88brP5B vByPvQV075x5m5Uq5zENJZejG3ag6mZM/iW1iZSYWrHmT/xqz2IpzOWV2jtmrLZtw7MJP2nCzawuE 274AEn8o/4ehW/rUy3iEO3G/3Zgb0qPKV6o9tb5IeUX4Bbw9bpM3xSuf4/icS+rdE9wBNnWsvpwkl kcRJiUTLH0SFOhQDtORA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKul-00ETPK-VW; Tue, 26 Apr 2022 13:07:36 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKlg-00EOzZ-DB; Tue, 26 Apr 2022 12:58:17 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 861891F4398D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977888; bh=6spd5zDDXQhVBNuvmuT3DFsMJlwSvquZ1zvEha0LOEI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TqWzgHDY9Ak5+G5HgVpoyVMfBXHYr75AcBGNmAOztGJoSAIQxxLMMNBvAa0VdbZg7 PwqrtSiuXgNOa0qKhke4XVyj0B+XN9fbeDhKoNd5+VuJj9dQTU3gl+YaP6iyUpszT6 gb1Mx1iQmQV1aZ7jiv8kT3p0kLqwCcgB8TWzltYbHYUWHBFvOd8ey+ziDLB8B3JJpC Zv0r99IMt0ZZ60+ovHe9BkZHnrReClurTJg3EMiBwlukxp7U//ZJQ22UNEWA1AIRjd gbJ+i5e4jv0OKEmg5nnVZBv5iseBAwAUDGRB/IA0GDDrd/KuIUal1HIikQP48oBeFq vgbDSyV8+okqw== From: Nicolas Dufresne To: Tiffany Lin , Andrew-CT Chen , Mauro Carvalho Chehab , Dmitry Osipenko , Thierry Reding , Jonathan Hunter , Ezequiel Garcia , Philipp Zabel , Greg Kroah-Hartman , Heiko Stuebner , Matthias Brugger Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-tegra@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org Subject: [PATCH v4 05/24] media: h264: Use v4l2_h264_reference for reflist Date: Tue, 26 Apr 2022 08:57:31 -0400 Message-Id: <20220426125751.108293-6-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055812_785398_9C4CB01C X-CRM114-Status: GOOD ( 21.62 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org In preparation for adding field decoding support, convert the byte arrays for reflist into array of struct v4l2_h264_reference. That struct will allow us to mark which field of the reference picture is being referenced. Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia Tested-by: Dmitry Osipenko --- .../mediatek/vcodec/vdec/vdec_h264_req_if.c | 36 +++++-- .../media/platform/nvidia/tegra-vde/h264.c | 19 ++-- drivers/media/v4l2-core/v4l2-h264.c | 33 ++++--- .../staging/media/hantro/hantro_g1_h264_dec.c | 38 +++---- drivers/staging/media/hantro/hantro_hw.h | 6 +- .../media/hantro/rockchip_vpu2_hw_h264_dec.c | 98 +++++++++---------- drivers/staging/media/rkvdec/rkvdec-h264.c | 12 +-- include/media/v4l2-h264.h | 19 ++-- 8 files changed, 143 insertions(+), 118 deletions(-) diff --git a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c index 43542de11e9c..8e59cf5c05a0 100644 --- a/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c +++ b/drivers/media/platform/mediatek/vcodec/vdec/vdec_h264_req_if.c @@ -439,11 +439,24 @@ static void update_dpb(const struct v4l2_ctrl_h264_decode_params *dec_param, } } -/* - * The firmware expects unused reflist entries to have the value 0x20. - */ -static void fixup_ref_list(u8 *ref_list, size_t num_valid) +static void get_ref_list(u8 *ref_list, + const struct v4l2_h264_reference *v4l2_ref_list, + int num_valid) { + u32 i; + + /* + * TODO The firmware does not support field decoding. Future + * implementation must use v4l2_ref_list[i].fields to obtain + * the reference field parity. + */ + + for (i = 0; i < num_valid; i++) + ref_list[i] = v4l2_ref_list[i].index; + + /* + * The firmware expects unused reflist entries to have the value 0x20. + */ memset(&ref_list[num_valid], 0x20, 32 - num_valid); } @@ -459,6 +472,9 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst) get_ctrl_ptr(inst->ctx, V4L2_CID_STATELESS_H264_SCALING_MATRIX); struct mtk_h264_dec_slice_param *slice_param = &inst->h264_slice_param; struct v4l2_h264_reflist_builder reflist_builder; + struct v4l2_h264_reference v4l2_p0_reflist[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference v4l2_b0_reflist[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference v4l2_b1_reflist[V4L2_H264_REF_LIST_LEN]; u8 *p0_reflist = slice_param->decode_params.ref_pic_list_p0; u8 *b0_reflist = slice_param->decode_params.ref_pic_list_b0; u8 *b1_reflist = slice_param->decode_params.ref_pic_list_b1; @@ -475,12 +491,14 @@ static void get_vdec_decode_parameters(struct vdec_h264_slice_inst *inst) /* Build the reference lists */ v4l2_h264_init_reflist_builder(&reflist_builder, dec_params, sps, inst->dpb); - v4l2_h264_build_p_ref_list(&reflist_builder, p0_reflist); - v4l2_h264_build_b_ref_lists(&reflist_builder, b0_reflist, b1_reflist); + v4l2_h264_build_p_ref_list(&reflist_builder, v4l2_p0_reflist); + v4l2_h264_build_b_ref_lists(&reflist_builder, v4l2_b0_reflist, + v4l2_b1_reflist); + /* Adapt the built lists to the firmware's expectations */ - fixup_ref_list(p0_reflist, reflist_builder.num_valid); - fixup_ref_list(b0_reflist, reflist_builder.num_valid); - fixup_ref_list(b1_reflist, reflist_builder.num_valid); + get_ref_list(p0_reflist, v4l2_p0_reflist, reflist_builder.num_valid); + get_ref_list(b0_reflist, v4l2_b0_reflist, reflist_builder.num_valid); + get_ref_list(b1_reflist, v4l2_b1_reflist, reflist_builder.num_valid); memcpy(&inst->vsi_ctx.h264_slice_params, slice_param, sizeof(inst->vsi_ctx.h264_slice_params)); diff --git a/drivers/media/platform/nvidia/tegra-vde/h264.c b/drivers/media/platform/nvidia/tegra-vde/h264.c index d8e5534e80c8..acc1b97f06ad 100644 --- a/drivers/media/platform/nvidia/tegra-vde/h264.c +++ b/drivers/media/platform/nvidia/tegra-vde/h264.c @@ -45,9 +45,9 @@ struct tegra_vde_h264_decoder_ctx { }; struct h264_reflists { - u8 p[V4L2_H264_NUM_DPB_ENTRIES]; - u8 b0[V4L2_H264_NUM_DPB_ENTRIES]; - u8 b1[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference p[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference b0[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference b1[V4L2_H264_NUM_DPB_ENTRIES]; }; static int tegra_vde_wait_mbe(struct tegra_vde *vde) @@ -768,7 +768,7 @@ static int tegra_vde_h264_setup_frames(struct tegra_ctx *ctx, struct h264_reflists reflists; struct vb2_buffer *ref; unsigned int i; - u8 *dpb_id; + struct v4l2_h264_reference *dpb_id; int err; /* @@ -811,14 +811,16 @@ static int tegra_vde_h264_setup_frames(struct tegra_ctx *ctx, } for (i = 0; i < b.num_valid; i++) { - ref = get_ref_buf(ctx, dst, dpb_id[i]); + int dpb_idx = dpb_id[i].index; - err = tegra_vde_h264_setup_frame(ctx, h264, &b, ref, dpb_id[i], + ref = get_ref_buf(ctx, dst, dpb_idx); + + err = tegra_vde_h264_setup_frame(ctx, h264, &b, ref, dpb_idx, h264->dpb_frames_nb++); if (err) return err; - if (b.refs[dpb_id[i]].pic_order_count < b.cur_pic_order_count) + if (b.refs[dpb_idx].top_field_order_cnt < b.cur_pic_order_count) h264->dpb_ref_frames_with_earlier_poc_nb++; } @@ -880,6 +882,9 @@ static int tegra_vde_h264_setup_context(struct tegra_ctx *ctx, if (h->pps->flags & V4L2_H264_PPS_FLAG_ENTROPY_CODING_MODE) return -EOPNOTSUPP; + if (h->decode_params->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) + return -EOPNOTSUPP; + if (h->sps->profile_idc == 66) h264->baseline_profile = 1; diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index ac47519a9fbe..afbfcf78efe4 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -75,12 +75,12 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, pic_order_count = dpb[i].top_field_order_cnt; b->refs[i].pic_order_count = pic_order_count; - b->unordered_reflist[b->num_valid] = i; + b->unordered_reflist[b->num_valid].index = i; b->num_valid++; } for (i = b->num_valid; i < ARRAY_SIZE(b->unordered_reflist); i++) - b->unordered_reflist[i] = i; + b->unordered_reflist[i].index = i; } EXPORT_SYMBOL_GPL(v4l2_h264_init_reflist_builder); @@ -90,8 +90,8 @@ static int v4l2_h264_p_ref_list_cmp(const void *ptra, const void *ptrb, const struct v4l2_h264_reflist_builder *builder = data; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -125,8 +125,8 @@ static int v4l2_h264_b0_ref_list_cmp(const void *ptra, const void *ptrb, s32 poca, pocb; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -170,8 +170,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, s32 poca, pocb; u8 idxa, idxb; - idxa = *((u8 *)ptra); - idxb = *((u8 *)ptrb); + idxa = ((struct v4l2_h264_reference *)ptra)->index; + idxb = ((struct v4l2_h264_reference *)ptrb)->index; if (WARN_ON(idxa >= V4L2_H264_NUM_DPB_ENTRIES || idxb >= V4L2_H264_NUM_DPB_ENTRIES)) @@ -212,8 +212,8 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16-bytes array used to store the P reference list. Each entry - * is an index in the DPB + * @reflist: 16 sized array used to store the P reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in * section '8.2.4 Decoding process for reference picture lists construction' @@ -222,7 +222,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, */ void v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, - u8 *reflist) + struct v4l2_h264_reference *reflist) { memcpy(reflist, builder->unordered_reflist, sizeof(builder->unordered_reflist[0]) * builder->num_valid); @@ -235,10 +235,10 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry - * is an index in the DPB - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry - * is an index in the DPB + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * is a v4l2_h264_reference structure + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described * in section '8.2.4 Decoding process for reference picture lists construction' @@ -247,7 +247,8 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); */ void v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, - u8 *b0_reflist, u8 *b1_reflist) + struct v4l2_h264_reference *b0_reflist, + struct v4l2_h264_reference *b1_reflist) { memcpy(b0_reflist, builder->unordered_reflist, sizeof(builder->unordered_reflist[0]) * builder->num_valid); diff --git a/drivers/staging/media/hantro/hantro_g1_h264_dec.c b/drivers/staging/media/hantro/hantro_g1_h264_dec.c index f49dbfb8a843..9de7f05eff2a 100644 --- a/drivers/staging/media/hantro/hantro_g1_h264_dec.c +++ b/drivers/staging/media/hantro/hantro_g1_h264_dec.c @@ -126,7 +126,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf) static void set_ref(struct hantro_ctx *ctx) { - const u8 *b0_reflist, *b1_reflist, *p_reflist; + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist; struct hantro_dev *vpu = ctx->dev; int reg_num; u32 reg; @@ -157,12 +157,12 @@ static void set_ref(struct hantro_ctx *ctx) */ reg_num = 0; for (i = 0; i < 15; i += 3) { - reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1]) | - G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2]); + reg = G1_REG_BD_REF_PIC_BINIT_RLIST_F0(b0_reflist[i].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_F1(b0_reflist[i + 1].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_F2(b0_reflist[i + 2].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B0(b1_reflist[i].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B1(b1_reflist[i + 1].index) | + G1_REG_BD_REF_PIC_BINIT_RLIST_B2(b1_reflist[i + 2].index); vdpu_write_relaxed(vpu, reg, G1_REG_BD_REF_PIC(reg_num++)); } @@ -171,12 +171,12 @@ static void set_ref(struct hantro_ctx *ctx) * of forward and backward reference picture lists and first 4 entries * of P forward picture list. */ - reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15]) | - G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2]) | - G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3]); + reg = G1_REG_BD_P_REF_PIC_BINIT_RLIST_F15(b0_reflist[15].index) | + G1_REG_BD_P_REF_PIC_BINIT_RLIST_B15(b1_reflist[15].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F0(p_reflist[0].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F1(p_reflist[1].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F2(p_reflist[2].index) | + G1_REG_BD_P_REF_PIC_PINIT_RLIST_F3(p_reflist[3].index); vdpu_write_relaxed(vpu, reg, G1_REG_BD_P_REF_PIC); /* @@ -185,12 +185,12 @@ static void set_ref(struct hantro_ctx *ctx) */ reg_num = 0; for (i = 4; i < HANTRO_H264_DPB_SIZE; i += 6) { - reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i]) | - G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1]) | - G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2]) | - G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3]) | - G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4]) | - G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5]); + reg = G1_REG_FWD_PIC_PINIT_RLIST_F0(p_reflist[i].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F1(p_reflist[i + 1].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F2(p_reflist[i + 2].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F3(p_reflist[i + 3].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F4(p_reflist[i + 4].index) | + G1_REG_FWD_PIC_PINIT_RLIST_F5(p_reflist[i + 5].index); vdpu_write_relaxed(vpu, reg, G1_REG_FWD_PIC(reg_num++)); } diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index ed018e293ba0..2bc6b8f088f5 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls { * @b1: B1 reflist */ struct hantro_h264_dec_reflists { - u8 p[HANTRO_H264_DPB_SIZE]; - u8 b0[HANTRO_H264_DPB_SIZE]; - u8 b1[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE]; }; /** diff --git a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c index 64a6330475eb..46c1a83bcc4e 100644 --- a/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c +++ b/drivers/staging/media/hantro/rockchip_vpu2_hw_h264_dec.c @@ -298,7 +298,7 @@ static void set_params(struct hantro_ctx *ctx, struct vb2_v4l2_buffer *src_buf) static void set_ref(struct hantro_ctx *ctx) { - const u8 *b0_reflist, *b1_reflist, *p_reflist; + const struct v4l2_h264_reference *b0_reflist, *b1_reflist, *p_reflist; struct hantro_dev *vpu = ctx->dev; u32 reg; int i; @@ -307,20 +307,20 @@ static void set_ref(struct hantro_ctx *ctx) b1_reflist = ctx->h264_dec.reflists.b1; p_reflist = ctx->h264_dec.reflists.p; - reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9]) | - VDPU_REG_PINIT_RLIST_F8(p_reflist[8]) | - VDPU_REG_PINIT_RLIST_F7(p_reflist[7]) | - VDPU_REG_PINIT_RLIST_F6(p_reflist[6]) | - VDPU_REG_PINIT_RLIST_F5(p_reflist[5]) | - VDPU_REG_PINIT_RLIST_F4(p_reflist[4]); + reg = VDPU_REG_PINIT_RLIST_F9(p_reflist[9].index) | + VDPU_REG_PINIT_RLIST_F8(p_reflist[8].index) | + VDPU_REG_PINIT_RLIST_F7(p_reflist[7].index) | + VDPU_REG_PINIT_RLIST_F6(p_reflist[6].index) | + VDPU_REG_PINIT_RLIST_F5(p_reflist[5].index) | + VDPU_REG_PINIT_RLIST_F4(p_reflist[4].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(74)); - reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15]) | - VDPU_REG_PINIT_RLIST_F14(p_reflist[14]) | - VDPU_REG_PINIT_RLIST_F13(p_reflist[13]) | - VDPU_REG_PINIT_RLIST_F12(p_reflist[12]) | - VDPU_REG_PINIT_RLIST_F11(p_reflist[11]) | - VDPU_REG_PINIT_RLIST_F10(p_reflist[10]); + reg = VDPU_REG_PINIT_RLIST_F15(p_reflist[15].index) | + VDPU_REG_PINIT_RLIST_F14(p_reflist[14].index) | + VDPU_REG_PINIT_RLIST_F13(p_reflist[13].index) | + VDPU_REG_PINIT_RLIST_F12(p_reflist[12].index) | + VDPU_REG_PINIT_RLIST_F11(p_reflist[11].index) | + VDPU_REG_PINIT_RLIST_F10(p_reflist[10].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(75)); reg = VDPU_REG_REFER1_NBR(hantro_h264_get_ref_nbr(ctx, 1)) | @@ -355,54 +355,54 @@ static void set_ref(struct hantro_ctx *ctx) VDPU_REG_REFER14_NBR(hantro_h264_get_ref_nbr(ctx, 14)); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(83)); - reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5]) | - VDPU_REG_BINIT_RLIST_F4(b0_reflist[4]) | - VDPU_REG_BINIT_RLIST_F3(b0_reflist[3]) | - VDPU_REG_BINIT_RLIST_F2(b0_reflist[2]) | - VDPU_REG_BINIT_RLIST_F1(b0_reflist[1]) | - VDPU_REG_BINIT_RLIST_F0(b0_reflist[0]); + reg = VDPU_REG_BINIT_RLIST_F5(b0_reflist[5].index) | + VDPU_REG_BINIT_RLIST_F4(b0_reflist[4].index) | + VDPU_REG_BINIT_RLIST_F3(b0_reflist[3].index) | + VDPU_REG_BINIT_RLIST_F2(b0_reflist[2].index) | + VDPU_REG_BINIT_RLIST_F1(b0_reflist[1].index) | + VDPU_REG_BINIT_RLIST_F0(b0_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(100)); - reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11]) | - VDPU_REG_BINIT_RLIST_F10(b0_reflist[10]) | - VDPU_REG_BINIT_RLIST_F9(b0_reflist[9]) | - VDPU_REG_BINIT_RLIST_F8(b0_reflist[8]) | - VDPU_REG_BINIT_RLIST_F7(b0_reflist[7]) | - VDPU_REG_BINIT_RLIST_F6(b0_reflist[6]); + reg = VDPU_REG_BINIT_RLIST_F11(b0_reflist[11].index) | + VDPU_REG_BINIT_RLIST_F10(b0_reflist[10].index) | + VDPU_REG_BINIT_RLIST_F9(b0_reflist[9].index) | + VDPU_REG_BINIT_RLIST_F8(b0_reflist[8].index) | + VDPU_REG_BINIT_RLIST_F7(b0_reflist[7].index) | + VDPU_REG_BINIT_RLIST_F6(b0_reflist[6].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(101)); - reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15]) | - VDPU_REG_BINIT_RLIST_F14(b0_reflist[14]) | - VDPU_REG_BINIT_RLIST_F13(b0_reflist[13]) | - VDPU_REG_BINIT_RLIST_F12(b0_reflist[12]); + reg = VDPU_REG_BINIT_RLIST_F15(b0_reflist[15].index) | + VDPU_REG_BINIT_RLIST_F14(b0_reflist[14].index) | + VDPU_REG_BINIT_RLIST_F13(b0_reflist[13].index) | + VDPU_REG_BINIT_RLIST_F12(b0_reflist[12].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(102)); - reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5]) | - VDPU_REG_BINIT_RLIST_B4(b1_reflist[4]) | - VDPU_REG_BINIT_RLIST_B3(b1_reflist[3]) | - VDPU_REG_BINIT_RLIST_B2(b1_reflist[2]) | - VDPU_REG_BINIT_RLIST_B1(b1_reflist[1]) | - VDPU_REG_BINIT_RLIST_B0(b1_reflist[0]); + reg = VDPU_REG_BINIT_RLIST_B5(b1_reflist[5].index) | + VDPU_REG_BINIT_RLIST_B4(b1_reflist[4].index) | + VDPU_REG_BINIT_RLIST_B3(b1_reflist[3].index) | + VDPU_REG_BINIT_RLIST_B2(b1_reflist[2].index) | + VDPU_REG_BINIT_RLIST_B1(b1_reflist[1].index) | + VDPU_REG_BINIT_RLIST_B0(b1_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(103)); - reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11]) | - VDPU_REG_BINIT_RLIST_B10(b1_reflist[10]) | - VDPU_REG_BINIT_RLIST_B9(b1_reflist[9]) | - VDPU_REG_BINIT_RLIST_B8(b1_reflist[8]) | - VDPU_REG_BINIT_RLIST_B7(b1_reflist[7]) | - VDPU_REG_BINIT_RLIST_B6(b1_reflist[6]); + reg = VDPU_REG_BINIT_RLIST_B11(b1_reflist[11].index) | + VDPU_REG_BINIT_RLIST_B10(b1_reflist[10].index) | + VDPU_REG_BINIT_RLIST_B9(b1_reflist[9].index) | + VDPU_REG_BINIT_RLIST_B8(b1_reflist[8].index) | + VDPU_REG_BINIT_RLIST_B7(b1_reflist[7].index) | + VDPU_REG_BINIT_RLIST_B6(b1_reflist[6].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(104)); - reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15]) | - VDPU_REG_BINIT_RLIST_B14(b1_reflist[14]) | - VDPU_REG_BINIT_RLIST_B13(b1_reflist[13]) | - VDPU_REG_BINIT_RLIST_B12(b1_reflist[12]); + reg = VDPU_REG_BINIT_RLIST_B15(b1_reflist[15].index) | + VDPU_REG_BINIT_RLIST_B14(b1_reflist[14].index) | + VDPU_REG_BINIT_RLIST_B13(b1_reflist[13].index) | + VDPU_REG_BINIT_RLIST_B12(b1_reflist[12].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(105)); - reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3]) | - VDPU_REG_PINIT_RLIST_F2(p_reflist[2]) | - VDPU_REG_PINIT_RLIST_F1(p_reflist[1]) | - VDPU_REG_PINIT_RLIST_F0(p_reflist[0]); + reg = VDPU_REG_PINIT_RLIST_F3(p_reflist[3].index) | + VDPU_REG_PINIT_RLIST_F2(p_reflist[2].index) | + VDPU_REG_PINIT_RLIST_F1(p_reflist[1].index) | + VDPU_REG_PINIT_RLIST_F0(p_reflist[0].index); vdpu_write_relaxed(vpu, reg, VDPU_SWREG(106)); reg = VDPU_REG_REFER_LTERM_E(ctx->h264_dec.dpb_longterm); diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 951e19231da2..3c7f3d87fab4 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl { #define RKVDEC_H264_DPB_SIZE 16 struct rkvdec_h264_reflists { - u8 p[RKVDEC_H264_DPB_SIZE]; - u8 b0[RKVDEC_H264_DPB_SIZE]; - u8 b1[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE]; u8 num_valid; }; @@ -767,13 +767,13 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, switch (j) { case 0: - idx = h264_ctx->reflists.p[i]; + idx = h264_ctx->reflists.p[i].index; break; case 1: - idx = h264_ctx->reflists.b0[i]; + idx = h264_ctx->reflists.b0[i].index; break; case 2: - idx = h264_ctx->reflists.b1[i]; + idx = h264_ctx->reflists.b1[i].index; break; } diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index 4b1c71c935e0..ef9a894e3c32 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder { u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; s32 cur_pic_order_count; - u8 unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; u8 num_valid; }; @@ -51,10 +51,10 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16-bytes array used to store the B0 reference list. Each entry - * is an index in the DPB - * @b1_reflist: 16-bytes array used to store the B1 reference list. Each entry - * is an index in the DPB + * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * is a v4l2_h264_reference structure + * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described * in section '8.2.4 Decoding process for reference picture lists construction' @@ -63,14 +63,15 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, */ void v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, - u8 *b0_reflist, u8 *b1_reflist); + struct v4l2_h264_reference *b0_reflist, + struct v4l2_h264_reference *b1_reflist); /** * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16-bytes array used to store the P reference list. Each entry - * is an index in the DPB + * @reflist: 16 sized array used to store the P reference list. Each entry + * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in * section '8.2.4 Decoding process for reference picture lists construction' @@ -79,6 +80,6 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, */ void v4l2_h264_build_p_ref_list(const struct v4l2_h264_reflist_builder *builder, - u8 *reflist); + struct v4l2_h264_reference *reflist); #endif /* _MEDIA_V4L2_H264_H */ From patchwork Tue Apr 26 12:57:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827096 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 159DDC433F5 for ; Tue, 26 Apr 2022 13:05:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=VMSfzZ4HQ83kDwxV7jiUD5iY7yyfY5TMqQjlPgElKYQ=; b=dH5rdDley3AEIY 9KLhMU6tHk416umSxlnoZwH28uea7TLqOYmkl9+c4aKAMD+JUxT771q/0UqegNVvu+7zsSyZv5ReO VMGZjLm+vM1ZHhRCcoI2fMdsEEemag3skCF2IfBj2dVzQGcsl/uWrQw1OBQ1OsdHv26it7xpJfLCS h88ydfLxVjDXUT0uW/hffgOWZadTkoC5Oji7e+/1oBqX7PPLzB9reRocdDZkFiq0ZbKrTK8uILeVl fS7znEQGd0LSG6P0XLRXMjwEmCRbhkmL0sbOtl3ieN3eur82zxtte/aOQXWW/+7bvwZubB17H12wa BQKllrBfZKgj2GrXgz5Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKt6-00ESYO-8x; Tue, 26 Apr 2022 13:05:52 +0000 Received: from bhuna.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e3e3]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKli-00EP0U-6q for linux-rockchip@lists.infradead.org; Tue, 26 Apr 2022 12:58:17 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 91AFA1F43982 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977890; bh=vuftnHCEWwJdNLAqyrxNwOkREV8boF39Sd5EZvT7GwM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dtPFBWG6icCFN7/tfl63OERpTPpFK1cwwNlfKMgDXOOU9TLmeK1S/ki6qAOudgkTV vQ2HDDwuRi9QvOTEoVjdKTyyvMlLCTzWETNVUfZB5rozmycH1v6s9SXzgsLJti/+Vd AK4+rkEH7j8lmAzHwSOxR0PIN8tPMNKJ25ammNbGrlEOgwO/0nj7vB5jFMqXRPQyLM VgeWaOv2r6UjnNc0fMvCugMQWzlhhP1RGoBWRE/4MzM41pieC0mg0DnwNP4Fbb4Znw SmfWBxVj7f1jycI4SblMk54J7k8C+qRbZQhhRnlpASecDA42/sGOlYVt4djFaJ9J/L 5sNqDy13qx5eg== From: Nicolas Dufresne To: Mauro Carvalho Chehab , Ezequiel Garcia , Philipp Zabel , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev Subject: [PATCH v4 06/24] media: h264: Increase reference lists size to 32 Date: Tue, 26 Apr 2022 08:57:32 -0400 Message-Id: <20220426125751.108293-7-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055814_612732_AC6A81C2 X-CRM114-Status: GOOD ( 12.21 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org This is to accommodate support for field decoding, which splits the top and the bottom references into the reference list. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/media/v4l2-core/v4l2-h264.c | 6 +++--- drivers/staging/media/hantro/hantro_hw.h | 6 +++--- drivers/staging/media/rkvdec/rkvdec-h264.c | 6 +++--- include/media/v4l2-h264.h | 8 ++++---- 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/media/v4l2-core/v4l2-h264.c b/drivers/media/v4l2-core/v4l2-h264.c index afbfcf78efe4..4b46b36526c0 100644 --- a/drivers/media/v4l2-core/v4l2-h264.c +++ b/drivers/media/v4l2-core/v4l2-h264.c @@ -212,7 +212,7 @@ static int v4l2_h264_b1_ref_list_cmp(const void *ptra, const void *ptrb, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16 sized array used to store the P reference list. Each entry + * @reflist: 32 sized array used to store the P reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in @@ -235,9 +235,9 @@ EXPORT_SYMBOL_GPL(v4l2_h264_build_p_ref_list); * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry * is a v4l2_h264_reference structure - * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 2bc6b8f088f5..292aaaabaf24 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -69,9 +69,9 @@ struct hantro_h264_dec_ctrls { * @b1: B1 reflist */ struct hantro_h264_dec_reflists { - struct v4l2_h264_reference p[HANTRO_H264_DPB_SIZE]; - struct v4l2_h264_reference b0[HANTRO_H264_DPB_SIZE]; - struct v4l2_h264_reference b1[HANTRO_H264_DPB_SIZE]; + struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; }; /** diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 3c7f3d87fab4..dff89732ddd0 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -100,9 +100,9 @@ struct rkvdec_h264_priv_tbl { #define RKVDEC_H264_DPB_SIZE 16 struct rkvdec_h264_reflists { - struct v4l2_h264_reference p[RKVDEC_H264_DPB_SIZE]; - struct v4l2_h264_reference b0[RKVDEC_H264_DPB_SIZE]; - struct v4l2_h264_reference b1[RKVDEC_H264_DPB_SIZE]; + struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index ef9a894e3c32..e282fb16ac58 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -37,7 +37,7 @@ struct v4l2_h264_reflist_builder { u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; s32 cur_pic_order_count; - struct v4l2_h264_reference unordered_reflist[V4L2_H264_NUM_DPB_ENTRIES]; + struct v4l2_h264_reference unordered_reflist[V4L2_H264_REF_LIST_LEN]; u8 num_valid; }; @@ -51,9 +51,9 @@ v4l2_h264_init_reflist_builder(struct v4l2_h264_reflist_builder *b, * v4l2_h264_build_b_ref_lists() - Build the B0/B1 reference lists * * @builder: reference list builder context - * @b0_reflist: 16 sized array used to store the B0 reference list. Each entry + * @b0_reflist: 32 sized array used to store the B0 reference list. Each entry * is a v4l2_h264_reference structure - * @b1_reflist: 16 sized array used to store the B1 reference list. Each entry + * @b1_reflist: 32 sized array used to store the B1 reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the B0/B1 reference lists. This procedure is described @@ -70,7 +70,7 @@ v4l2_h264_build_b_ref_lists(const struct v4l2_h264_reflist_builder *builder, * v4l2_h264_build_p_ref_list() - Build the P reference list * * @builder: reference list builder context - * @reflist: 16 sized array used to store the P reference list. Each entry + * @reflist: 32 sized array used to store the P reference list. Each entry * is a v4l2_h264_reference structure * * This functions builds the P reference lists. This procedure is describe in From patchwork Tue Apr 26 12:57:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827097 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 20703C433F5 for ; Tue, 26 Apr 2022 13:07:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=17kMiniVAIUkWuiZWkSkT55gDkLlNHex6EPE4Nd1iCg=; b=KauBJ60lRXPLgr 0jfIp/WzlJG4g/b9aLCFiqIjgofhTN7tZJXPHmNLhYtzvmxE42Ra48af3DaouDzQcl/oHytGzrXxC CbsmKFUzhRit3pRFRyPH1aW3x3LCozuBvUsQlqVAYZpZzmIKxy842lW6Ce0x4SaaLfm8X+yXQTs+n uu35W8bhdIonTJDonUdlHgNvITi87UDKZUWO92a2pmbcz5KkUx/YVEe0cNoWwnUq+ew9ZiGOWpTHJ pwaxZ3bsK2LvymtASAhiTY+pyRZwnsH8+koZwFBgDFJ29TMEhQ7JFFuU1G5WW/hFwYLLI7I8OsOaS NvT92rBwzxXZ+vWc+YQA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKuk-00ETOf-8s; Tue, 26 Apr 2022 13:07:34 +0000 Received: from bhuna.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e3e3]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKlp-00EP6S-0q for linux-rockchip@lists.infradead.org; Tue, 26 Apr 2022 12:58:24 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 61F8F1F43992 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977899; bh=gu3hWa9l9RwIqYfPgXJTq2hXyJdb/X3UYi37Vl+bXOc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=A9PPhaws+0UJKFGVd1hph8rKqPW0f16roDicNJFfCdGnt2qPN466cQ22WLaQfg4U5 +Jxnnc1YkNht2e50RoYOjFvb/wNAKG1eu+UGyCbR/EhBVNTOiBmGazcu+7DO0/G5ql WnI0BtliPH3K5k0AyymLQZNm8otEZaMpTYQsCZo5HAI7ZBydI7UbBYsjEizKhU0c4c JTiYZEDPyYpzkw2cpXNWb1S/C+OrHDHktRpzO2Or1om6MilkX5eHtIzxOxPweZ9ioC WIyhM23PqBPJS0uf9kApXCI2runbW5D6iZ2Uyj/LxM4r2W/haPC6ILBcRTnprAIRMr u0V8WQkrdXY/Q== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman , Hans Verkuil , Boris Brezillon Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , Mauro Carvalho Chehab , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 12/24] media: rkvdec: Stop overclocking the decoder Date: Tue, 26 Apr 2022 08:57:38 -0400 Message-Id: <20220426125751.108293-13-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055821_241011_4636EC3A X-CRM114-Status: GOOD ( 10.90 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org While this overclock hack seems to work on some implementations (some ChromeBooks, RockPi4) it also causes instability on other implementations (notably LibreComputer Renegade, but there were more reports in the LibreELEC project, where this has been removed). While performance is indeed affected (tested with GStreamer), 4K playback still works as long as you don't operate in lock step and keep at least 1 frame ahead of time in the decode queue. After discussion with ChromeOS members, it would seem that their implementation indeed used to synchronously decode each frame, so this hack was simply compensating for their code being less efficient. In my opinion, this hack should not have been included upstream. Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver") Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index c0cf3488f970..2df8cf4883e2 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -1027,12 +1027,6 @@ static int rkvdec_probe(struct platform_device *pdev) if (ret) return ret; - /* - * Bump ACLK to max. possible freq. (500 MHz) to improve performance - * When 4k video playback. - */ - clk_set_rate(rkvdec->clocks[0].clk, 500 * 1000 * 1000); - rkvdec->regs = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(rkvdec->regs)) return PTR_ERR(rkvdec->regs); From patchwork Tue Apr 26 12:57:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827099 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 75D42C433F5 for ; Tue, 26 Apr 2022 13:07:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=zc4yfmXX3hqlkZtx48WALALNxnp0n3T7mSv+rKRJ1W8=; b=dG4SWNQmvEBXMn zeryW/Y2wcIXS0080hkd6EpMlwFm4IbT7pKH+BhDyb2a0vQP3X+J7MwhCDalbwho8IrlpYE4/0BKn 17HIwOH6rrSXFuMs760qSEw1pznwq4r2be+tvKuc3U4e/p5okEuBcJXf4WLuzt1kwOaoIHCPZH+F5 Jhp++SZJBKZnDhwUg6wME0VqRkGgNbKfU85bzPXVYjw9SMAsvCij3fWqwCtUqEwxRs7YCqs6jKCan BDmxB+US01l5aYRoXBYkH07XkbFZjB4AuO12vD7vry/UNTy96z5jP/U8errBdo6ZGMypPRWjUmkE1 ZRZJwPut6nXS8+qLOOiQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKuo-00ETQe-6Z; Tue, 26 Apr 2022 13:07:38 +0000 Received: from bhuna.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e3e3]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKlr-00EP8f-9j for linux-rockchip@lists.infradead.org; Tue, 26 Apr 2022 12:58:26 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 70C1F1F4398D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977901; bh=qqGKTdK+BoJvGXzu5rDFgeoUxEBT3Njqv5q7sTj6RAI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UWJo24Y3WFgGCpdd4zZo3nChuheimOm8+x1xB7rfBR4tcpQbgBd5XeyIgafDQP77Z zswC30ioFQ2xggDqfWxE28q2BI4HwrlPX9L4ZTJdEzT5UmiLmjLKTlOtSmseXz6l/K aabCPa3lyhyFNCAZHmeyenUAwEkbp5/ktJR1A7NlS1IpGmxxtfFTyyyM85grd9qBOD q06Wul8cQjnx8TVI8wWUD350iIcI//IzY5t1vwZcKAGkgffWqsyxuv+GsFjBRDoVcn 2YtZat4CUTXCYVEmQP0idVgjZX/eEyp3qMCxNZSNtS2iKyoYw0HpXqnQo/NtblBbtm btlZS7Eq962BA== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman , Boris Brezillon , Hans Verkuil Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , Mauro Carvalho Chehab , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 13/24] media: rkvdec: h264: Fix dpb_valid implementation Date: Tue, 26 Apr 2022 08:57:39 -0400 Message-Id: <20220426125751.108293-14-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055823_558908_DFE1D3E3 X-CRM114-Status: GOOD ( 17.73 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org The ref builder only provided references that are marked as valid in the dpb. Thus the current implementation of dpb_valid would always set the flag to 1. This is not representing missing frames (this is called 'non-existing' pictures in the spec). In some context, these non-existing pictures still need to occupy a slot in the reference list according to the spec. Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver") Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 33 ++++++++++++++++------ 1 file changed, 24 insertions(+), 9 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index dff89732ddd0..bcde37d72244 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -112,6 +112,7 @@ struct rkvdec_h264_run { const struct v4l2_ctrl_h264_sps *sps; const struct v4l2_ctrl_h264_pps *pps; const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix; + int ref_buf_idx[V4L2_H264_NUM_DPB_ENTRIES]; }; struct rkvdec_h264_ctx { @@ -725,6 +726,26 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, } } +static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx, + struct rkvdec_h264_run *run) +{ + const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params; + u32 i; + + for (i = 0; i < ARRAY_SIZE(dec_params->dpb); i++) { + struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx; + const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb; + struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; + int buf_idx = -1; + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) + buf_idx = vb2_find_timestamp(cap_q, + dpb[i].reference_ts, 0); + + run->ref_buf_idx[i] = buf_idx; + } +} + static void assemble_hw_rps(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run) { @@ -762,7 +783,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, for (j = 0; j < RKVDEC_NUM_REFLIST; j++) { for (i = 0; i < h264_ctx->reflists.num_valid; i++) { - u8 dpb_valid = 0; + bool dpb_valid = run->ref_buf_idx[i] >= 0; u8 idx = 0; switch (j) { @@ -779,8 +800,6 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, if (idx >= ARRAY_SIZE(dec_params->dpb)) continue; - dpb_valid = !!(dpb[idx].flags & - V4L2_H264_DPB_ENTRY_FLAG_ACTIVE); set_ps_field(hw_rps, DPB_INFO(i, j), idx | dpb_valid << 4); @@ -859,13 +878,8 @@ get_ref_buf(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run, unsigned int dpb_idx) { struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx; - const struct v4l2_h264_dpb_entry *dpb = run->decode_params->dpb; struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; - int buf_idx = -1; - - if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) - buf_idx = vb2_find_timestamp(cap_q, - dpb[dpb_idx].reference_ts, 0); + int buf_idx = run->ref_buf_idx[dpb_idx]; /* * If a DPB entry is unused or invalid, address of current destination @@ -1102,6 +1116,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) assemble_hw_scaling_list(ctx, &run); assemble_hw_pps(ctx, &run); + lookup_ref_buf_idx(ctx, &run); assemble_hw_rps(ctx, &run); config_registers(ctx, &run); From patchwork Tue Apr 26 12:57:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827100 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9F6C3C433EF for ; Tue, 26 Apr 2022 13:07:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=S5lZbopukreAxcM8xGXSCitvly1dJA3e5jSJvE/7w+U=; b=WDDXAouBLMieUl ba0yxZDCiFvdv7Qa2liB81eK0iY0twQ3id8i1FEXrUtrGzG9Z3xz5mOGqwf7106XBzfhSKUVJbM2k Nty4I4FlmL8oPTGT7a5C+kiTqX/M25cPoM+aRzurVvcfVnAEXjBLIThaaL0Iri1iW6dZKahE7sdt6 swknle41ihP9APQVJSC7AUYexKdxy8aIvwyyhgoYNsi3p9GKY5kLBsbj1TXlrLyvmyH+9SAzBW/kE ceARXNOy7MUSq5eKBWZyM2TBHxcUaEbonq/kWrb0eYJSYVuuzS/OCjGpeJtcj8er0idf38qnPmvwA lbyGGsQnfEqV/5EKNP1w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKur-00ETSx-TG; Tue, 26 Apr 2022 13:07:41 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKlt-00EPAa-8a for linux-rockchip@lists.infradead.org; Tue, 26 Apr 2022 12:58:30 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 762DA1F43982 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977904; bh=FmxP3Of2pm4gkbqbbaer5qaBCfIvVbPhhE5DeBELz4c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hBcUdsG1vo3Ibv1T3PQzLkVRTO0BNO/MVGkjQjQwI+Cq1k4fW3YqyXc+hxVDtQf/o MJuP+asy+HuX8zNfZEw/5amOnVG+lj3zPobSzVPtm7WuekupyjirBzpIGRu0Cq98GL unLEetrzkGph3I6wU4USp8dMV6FRCHAqo9gjaEOKEuSHlDMgn9f17RAdUro6jJ1Wqu uPEsQMUgSnXfrEjpk5ebf2GLQCIv418eOVG79hBN5E7Zv/6+6Asg/D3t2WXaURMCbn 8JaCNhHnsPh6AgM3V5WfTkzw1p2TWjl9L9muaea9yujNEXRbpLnSD2BpZQNgkLwFT1 bsPyAFMV8nknw== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman , Hans Verkuil , Boris Brezillon Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, Ezequiel Garcia , Mauro Carvalho Chehab , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 14/24] media: rkvdec: h264: Fix bit depth wrap in pps packet Date: Tue, 26 Apr 2022 08:57:40 -0400 Message-Id: <20220426125751.108293-15-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055825_579604_30C29B32 X-CRM114-Status: GOOD ( 11.55 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman The luma and chroma bit depth fields in the pps packet are 3 bits wide. 8 is wrongly added to the bit depth values written to these 3 bit fields. Because only the 3 LSB are written, the hardware was configured correctly. Correct this by not adding 8 to the luma and chroma bit depth value. Fixes: cd33c830448ba ("media: rkvdec: Add the rkvdec driver") Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index bcde37d72244..8d44a884a52e 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -662,8 +662,8 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, WRITE_PPS(0xff, PROFILE_IDC); WRITE_PPS(1, CONSTRAINT_SET3_FLAG); WRITE_PPS(sps->chroma_format_idc, CHROMA_FORMAT_IDC); - WRITE_PPS(sps->bit_depth_luma_minus8 + 8, BIT_DEPTH_LUMA); - WRITE_PPS(sps->bit_depth_chroma_minus8 + 8, BIT_DEPTH_CHROMA); + WRITE_PPS(sps->bit_depth_luma_minus8, BIT_DEPTH_LUMA); + WRITE_PPS(sps->bit_depth_chroma_minus8, BIT_DEPTH_CHROMA); WRITE_PPS(0, QPPRIME_Y_ZERO_TRANSFORM_BYPASS_FLAG); WRITE_PPS(sps->log2_max_frame_num_minus4, LOG2_MAX_FRAME_NUM_MINUS4); WRITE_PPS(sps->max_num_ref_frames, MAX_NUM_REF_FRAMES); From patchwork Tue Apr 26 12:57:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827101 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 97734C433FE for ; Tue, 26 Apr 2022 13:07:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=HTiAJQY/Bm9Z1wWJ94BLBct5EaQl/lu6ZxtziRi/glE=; b=dG7MQG+Mfxs4tB bBXIKmygKd1gEWFZeD12WOyZwFwHScoRrYLJDEip004pXbYSj9VMwYuBqSSmVtQB3hbIZXDpxRJhc EeuYVLF+BBvE7lJV9QCeELoQIiJ/5qqt/YeYvoEu9dIpD6ApMSjLEm2Ai4Ia2mDFN05iuBdWDRBaJ BknuJnXZa3KHCPc/aaMxMT2BeriHk5nX6y7SbKlxAJsMkfYYiSuKMiY3Tmn1g55nxJqBlQapRHm6U AFrhMogSo92LN80Tx0n5MG51F+oU+RWt58Zz85Pm5y1/Mu+GO0aPVA4Uu13sy29ZaCxItv5kTXni4 Ty6W7CrVRAn3v0lJ0wcQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKuv-00ETV4-5h; Tue, 26 Apr 2022 13:07:45 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKlv-00EPBb-1c for linux-rockchip@lists.infradead.org; Tue, 26 Apr 2022 12:58:31 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id C07D41F43992 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977905; bh=Zdyv7lj5Zwk1trozSl5/ToeVKlMaPGRdeC6c5HT1zS8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Od0WtrC9/3w+VoxRpi3zn10sJ0Br+Gyxb8aoVslheETwwKNg/iGAv2aeShXCkitp/ Mu4Hk9C93KBfpJpPhx+gFja3MN2vX+NB0+0n75rpbpgc71IN++FRPYYTja1zrDaY/n 3kUPquAsCscegcf3N7IYdBYQloUa83CIBpVs5ZxT7WEvn2yM/NBLYZKjrNrE7pWiTJ ZHOiflQlrvYlhRoSitzL1RDhm9rQ++Mj2oFRrdWePsCXJcFPMU9H15LspCuU6jmBLa vpwvQbkapiOUjG2v5o0kcEh5tZLdyUH6uhfMZ5GWU0SHZQBPMBsPa4/rAIxwk0kvQo xpd9x7iSfaPsw== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 15/24] media: rkvdec: Move H264 SPS validation in rkvdec-h264 Date: Tue, 26 Apr 2022 08:57:41 -0400 Message-Id: <20220426125751.108293-16-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055827_349101_594D65C7 X-CRM114-Status: GOOD ( 16.73 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org No functional change, this moves H264 specific validation into the H264 specific code. This is in preparation of improving this validation and reusing it when VIDIOC_STREAMON is called. Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 23 ++++++++++++++++++++++ drivers/staging/media/rkvdec/rkvdec.c | 23 ++++++---------------- drivers/staging/media/rkvdec/rkvdec.h | 1 + 3 files changed, 30 insertions(+), 17 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 8d44a884a52e..0dcbcb1bac80 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -1137,9 +1137,32 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) return 0; } +static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl) +{ + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { + const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps; + /* + * TODO: The hardware supports 10-bit and 4:2:2 profiles, + * but it's currently broken in the driver. + * Reject them for now, until it's fixed. + */ + if (sps->chroma_format_idc > 1) + /* Only 4:0:0 and 4:2:0 are supported */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) + /* Luma and chroma bit depth mismatch */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != 0) + /* Only 8-bit is supported */ + return -EINVAL; + } + return 0; +} + const struct rkvdec_coded_fmt_ops rkvdec_h264_fmt_ops = { .adjust_fmt = rkvdec_h264_adjust_fmt, .start = rkvdec_h264_start, .stop = rkvdec_h264_stop, .run = rkvdec_h264_run, + .try_ctrl = rkvdec_h264_try_ctrl, }; diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 2df8cf4883e2..e3d44d5b35f3 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -29,23 +29,12 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) { - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps; - /* - * TODO: The hardware supports 10-bit and 4:2:2 profiles, - * but it's currently broken in the driver. - * Reject them for now, until it's fixed. - */ - if (sps->chroma_format_idc > 1) - /* Only 4:0:0 and 4:2:0 are supported */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) - /* Luma and chroma bit depth mismatch */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != 0) - /* Only 8-bit is supported */ - return -EINVAL; - } + struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl); + const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc; + + if (desc->ops->try_ctrl) + return desc->ops->try_ctrl(ctx, ctrl); + return 0; } diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h index 2f4ea1786b93..9df0fba799a4 100644 --- a/drivers/staging/media/rkvdec/rkvdec.h +++ b/drivers/staging/media/rkvdec/rkvdec.h @@ -72,6 +72,7 @@ struct rkvdec_coded_fmt_ops { void (*done)(struct rkvdec_ctx *ctx, struct vb2_v4l2_buffer *src_buf, struct vb2_v4l2_buffer *dst_buf, enum vb2_buffer_state result); + int (*try_ctrl)(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl); }; struct rkvdec_coded_fmt_desc { From patchwork Tue Apr 26 12:57:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827102 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EF8BBC433F5 for ; Tue, 26 Apr 2022 13:07:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=H2gfJ/evrAugqn7UMYm21zAWmqSF0+aVKhXyFOfRbo4=; b=QfZEtqFLpELeqe qW511NPri7S0bphGqXNGjJHcF/DA358s6b1Qdt6rjjj5HPiju+AwfTlkM6kZYmINq6UrJmFifxsnx Xhzwc7P50DPKt7tkwXu/dfHnNnzjOfT9Kh7qcnn03sS/q0BhvNOjX3BZqMd+rj9lycSghmOPkejQm 9g/xE0M0QtEIkCDPMG5mxE3CNbVwwHee2gs+W2f6ba/SMIxFYkUH69+mIzzSis6CYBednwS1E8poV YgAOIz/7WtjpJn218amzThvbw5mvLUR3l4DOXbLnLta5xjQQzu+rFIarRczSo3W4V0kcU+YwRfKGh pctcI7c6MkO2bes9BnkQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKuy-00ETXo-Vb; Tue, 26 Apr 2022 13:07:49 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKlw-00EPCb-St for linux-rockchip@lists.infradead.org; Tue, 26 Apr 2022 12:58:32 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 67CE51F4398D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977907; bh=0J+46psvSTuyS1wejcCrG3atoNuCwZAAyNkbTjDMFsU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aeiO2ufbvhFNYb7zVWgxzT+dtqF3tl7RyjAsaf8KVsW60NFm4QkISfL9a12zsEYle jA5BgytAjoyMdKIVIBVxxaJCwyvW+Doi+iLHXdGa8ZOsDMwxLXq5Dw27QRO9EyRnCu i6IPmh8a6W26TEeWalrLgOoVyFvNBD6Na9c+tWGE90XED+IfZGudpMYlKko4q1dVuw TwKI2T194dTmEohVKjaaRS5uwYzF/3zpslBwroYj1IByMcEFDoidltpPOkXEV2wFgH FqpjjuNIEWv4xLOOb2EcmI0gKiO7+Gg23oICPHN2qO5sv8vIjpnkJ5WaHUmnn9qt0j poaAaDreu9YVQ== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 16/24] media: rkvdec: h264: Validate and use pic width and height in mbs Date: Tue, 26 Apr 2022 08:57:42 -0400 Message-Id: <20220426125751.108293-17-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055829_297511_FB89AE9E X-CRM114-Status: GOOD ( 22.10 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman The width and height in macroblocks is currently configured based on OUTPUT buffer resolution, this works for frame pictures but can cause issues for field pictures. When frame_mbs_only_flag is 0 the height in mbs should be height of the field instead of height of frame. Validate pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1 against OUTPUT buffer resolution and use these values to configure HW. The validation is happening in both try_ctrt() and start() since it is otherwise possible to trick the driver during initialization by changing the OUTPUT format after having set a valid control. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia --- drivers/staging/media/rkvdec/rkvdec-h264.c | 81 +++++++++++++++++----- 1 file changed, 62 insertions(+), 19 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 0dcbcb1bac80..d5ac0285f36f 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -672,8 +672,17 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_DELTA_PIC_ORDER_ALWAYS_ZERO), DELTA_PIC_ORDER_ALWAYS_ZERO_FLAG); - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.width, 16), PIC_WIDTH_IN_MBS); - WRITE_PPS(DIV_ROUND_UP(ctx->coded_fmt.fmt.pix_mp.height, 16), PIC_HEIGHT_IN_MBS); + + /* + * Use the SPS values since they are already in macroblocks + * dimensions, height can be field height (halved) if + * V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY is not set and also it allows + * decoding smaller images into larger allocation which can be used + * to implementing SVC spatial layer support. + */ + WRITE_PPS(sps->pic_width_in_mbs_minus1 + 1, PIC_WIDTH_IN_MBS); + WRITE_PPS(sps->pic_height_in_map_units_minus1 + 1, PIC_HEIGHT_IN_MBS); + WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY), FRAME_MBS_ONLY_FLAG); WRITE_PPS(!!(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD), @@ -1035,13 +1044,61 @@ static int rkvdec_h264_adjust_fmt(struct rkvdec_ctx *ctx, return 0; } +static int rkvdec_h264_validate_sps(struct rkvdec_ctx *ctx, + const struct v4l2_ctrl_h264_sps *sps) +{ + unsigned int width, height; + + /* + * TODO: The hardware supports 10-bit and 4:2:2 profiles, + * but it's currently broken in the driver. + * Reject them for now, until it's fixed. + */ + if (sps->chroma_format_idc > 1) + /* Only 4:0:0 and 4:2:0 are supported */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) + /* Luma and chroma bit depth mismatch */ + return -EINVAL; + if (sps->bit_depth_luma_minus8 != 0) + /* Only 8-bit is supported */ + return -EINVAL; + + width = (sps->pic_width_in_mbs_minus1 + 1) * 16; + height = (sps->pic_height_in_map_units_minus1 + 1) * 16; + + /* + * when frame_mbs_only_flag is not set, this is field height, + * which is half the final height (see (7-18) in the + * specification) + */ + if (!(sps->flags & V4L2_H264_SPS_FLAG_FRAME_MBS_ONLY)) + height *= 2; + + if (width > ctx->coded_fmt.fmt.pix_mp.width || + height > ctx->coded_fmt.fmt.pix_mp.height) + return -EINVAL; + + return 0; +} + static int rkvdec_h264_start(struct rkvdec_ctx *ctx) { struct rkvdec_dev *rkvdec = ctx->dev; struct rkvdec_h264_priv_tbl *priv_tbl; struct rkvdec_h264_ctx *h264_ctx; + struct v4l2_ctrl *ctrl; int ret; + ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl, + V4L2_CID_STATELESS_H264_SPS); + if (!ctrl) + return -EINVAL; + + ret = rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps); + if (ret) + return ret; + h264_ctx = kzalloc(sizeof(*h264_ctx), GFP_KERNEL); if (!h264_ctx) return -ENOMEM; @@ -1139,23 +1196,9 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) static int rkvdec_h264_try_ctrl(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl) { - if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) { - const struct v4l2_ctrl_h264_sps *sps = ctrl->p_new.p_h264_sps; - /* - * TODO: The hardware supports 10-bit and 4:2:2 profiles, - * but it's currently broken in the driver. - * Reject them for now, until it's fixed. - */ - if (sps->chroma_format_idc > 1) - /* Only 4:0:0 and 4:2:0 are supported */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != sps->bit_depth_chroma_minus8) - /* Luma and chroma bit depth mismatch */ - return -EINVAL; - if (sps->bit_depth_luma_minus8 != 0) - /* Only 8-bit is supported */ - return -EINVAL; - } + if (ctrl->id == V4L2_CID_STATELESS_H264_SPS) + return rkvdec_h264_validate_sps(ctx, ctrl->p_new.p_h264_sps); + return 0; } From patchwork Tue Apr 26 12:57:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827103 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9126FC433F5 for ; Tue, 26 Apr 2022 13:07:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=zhM73RXKmY7uKcrnJocAKgoKYGtSPu6ulMKdYjI3nEQ=; b=3EvYkqeGfdkPqQ xIOrWjd/lAtKaGEeiXqk/RsLzOdE6kKPvZ2atbnqsRzlZsd7JzrLyttHgxvuAvlCoNePXeXyzkQow KQ1wiZJOTo8aqjYsRgVTAUOX40uIYfDBL6ioIeWXsmOkujYk7RriLk5mrX6KK97uOTnPCJtpHi6W2 Dm6SDt85XZ3Wqizj1KAvTzVXvcD4MMGFaOZHUtrS2ajNLTauQlTdkNajnAmiNj22nqHv0c4IXSET8 8C8WIF4BDYZwFgU5xG7xIyGB1gELVgPD1fjAEbwF6wEXsZRLSCilTW7JtLX9hlnANKmT4qT/q++uo 3NdeQO8z5DHsQhe9ljOQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKv1-00ETaN-I0; Tue, 26 Apr 2022 13:07:51 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKlz-00EPEP-1S for linux-rockchip@lists.infradead.org; Tue, 26 Apr 2022 12:58:36 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 1C6751F43996 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977909; bh=Obrg+kliljhQBVbkjHH9eqyMF1KK1PscTQElBfjHqLA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=M/Su31vKdsLjJViMlDV+oZDM2lWGxhUOrM0xafZj4XX9GRff9fgOXPSG4OMAc1QAs B5qSVl8w3PC38ceehO0XjfkGqsCHvh4k940A679PTb+LHLTOx3rrilS9Sp2Exzrkoq ij85Gh4X8ovJXGy0mCiByNyHOjNK0/xSni3J/R+PkETo4Hy0Yqi1agSZBU8bzvqRff gBE+vPFCxcz2wg3bZN0kte+59S6ty4xTRoc04YP6VwCQSuRwFbDEKWby7uO2+0SfCE OLgllG/q6s3WF7qFY103nMu47ObqYHVano4sby24zZQCO/SzPVbi64lxCNU86vSSC9 Hm2TfZqLLoO/w== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, Ezequiel Garcia , Sebastian Fricke , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 17/24] media: rkvdec: h264: Fix reference frame_num wrap for second field Date: Tue, 26 Apr 2022 08:57:43 -0400 Message-Id: <20220426125751.108293-18-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055831_275523_4C29BA22 X-CRM114-Status: GOOD ( 11.27 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman When decoding the second field in a complementary field pair the second field is sharing the same frame_num with the first field. Currently the frame_num for the first field is wrapped when it matches the field being decoded, this caused issues decoding the second field in a complementary field pair. Fix this by using inclusive comparison: 'less than or equal'. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Ezequiel Garcia Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec-h264.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index d5ac0285f36f..7011b66c1aab 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -782,7 +782,7 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, continue; if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM || - dpb[i].frame_num < dec_params->frame_num) { + dpb[i].frame_num <= dec_params->frame_num) { p[i] = dpb[i].frame_num; continue; } From patchwork Tue Apr 26 12:57:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827104 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 03339C433F5 for ; Tue, 26 Apr 2022 13:07:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GHZ5f+fUUVNEJZM3gonoS/rsoBSs6dTXw4VF2XoHKwM=; b=ud+eoYqMiehHUG j1bNOPQW5osrARueKmNNl0X1GAaydFtnAYjYrN6iomyj5tI9L7Gg6JepClcdnQTwzax+NapXVcuTT CGAX6WkR3TnRLwJN2uQsl4yhaxrHeyVGcohcT67JuxvFSCvWtuUHDH0j4F1BGu9+gIDQJwA06uelr c5t83EB+L6tHMvBr4MLciB4zMag8auiiv0JdxBIIkn8Y5xYjoCtxDhx8vWhs8DFU2/LjDDbT3YuX6 xPYJAu8bQhRlVy5V3JqO4oX4cXEMmezHncqzx0hGgXIlsX51GpARr9E26IGim5iLTdyS4KbrG7kdF gijUmze6+MuCVEEPmX3g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKv4-00ETd5-D4; Tue, 26 Apr 2022 13:07:54 +0000 Received: from bhuna.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e3e3]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKm0-00EPG0-Iv for linux-rockchip@lists.infradead.org; Tue, 26 Apr 2022 12:58:36 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id F359F1F43999 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977911; bh=mlmVguJcq18q2C5FsZYpROZPrzKGkqiR80YOa10U0rQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=R4DhZD/ZO/ZdN80EgBWl3CS/bbfNmq0mGNkaXtVKCew0b+SnrGkmV0GdkFy2zQZzj /J1QimWglAQnyxDyECg5tgducpujBvM7Jgn9I+pVNQ17xK0KLQKLq1NyZ2LjHJZizs 0IxLhE91L+gsPFTfB4BrsOoanpy/oB9V+TZ76QqbIX2qZ2FeSc3Hikz3TUBtukHukW gGactiPcwUwSMCxq6Fg2Ryr+iijk6a/spxFCBeAF9sv0CLB+igDPaGCO+51/9V0shg NIOuXwDbLBdLSIN6xt60BecfrAGk4xZHH0HggzrLiGP4//WTfbNVeNSYTmRwhrpE0b GDecRSL1kCfpQ== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, Sebastian Fricke , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 18/24] media: rkvdec: Ensure decoded resolution fit coded resolution Date: Tue, 26 Apr 2022 08:57:44 -0400 Message-Id: <20220426125751.108293-19-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055832_978977_DB0C3BC2 X-CRM114-Status: UNSURE ( 8.55 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman Ensure decoded CAPTURE buffer resolution is larger or equal to the coded OUTPUT buffer resolution. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index e3d44d5b35f3..2bc4b1a40989 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -257,6 +257,8 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv, pix_mp->pixelformat = coded_desc->decoded_fmts[0]; /* Always apply the frmsize constraint of the coded end. */ + pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width); + pix_mp->height = max(pix_mp->height, ctx->coded_fmt.fmt.pix_mp.height); v4l2_apply_frmsize_constraints(&pix_mp->width, &pix_mp->height, &coded_desc->frmsize); From patchwork Tue Apr 26 12:57:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8AE37C433FE for ; Tue, 26 Apr 2022 13:08:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gT9TIlLT3alRBg4TKGQ4JMMlLetMFI+GkCggiWla1wE=; b=RuATSYOPGscLsk ipiWKKvAjmtk4NbvuBiEvuRxF6DdIdssf5JSbmMQHQvSxmiM73TqQL1a7DsbqteRA8SPU2bJYn80a +74Cxp6ZalOsAinIrWi7ELSeIJtBsOaL02MfBTfywLPXckzclbRSs1HTBbMXT9c+DDKDe0WM+P3A4 BZ+kg4KEeOHT8mazT+YTYnZvsaEI6bdX2vMPe59hB1UduKqrW8czg8xUQR6i4am7JtzW30FnBuaWS DeNbnONGF50MDykEn6AUhkDpfSVXCNtfsM9V9LBNPPYWQX+nbtS3ZVkXNDONmB8uPTqmpJOaAi/jT xKqPWn/JQgNZ7QqwwnUw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKv8-00ETgE-EA; Tue, 26 Apr 2022 13:07:58 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKm2-00EPHG-Cz for linux-rockchip@lists.infradead.org; Tue, 26 Apr 2022 12:58:41 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id CDA921F4399D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977913; bh=dZLY5A5SJIWQWpJ/gOaUMK4w2rzl/JljG8yfzL7NyHg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=U+8pjgSzvqCzEEZreu22RuqA2NH9VJNkmR7y6PYxSdjioGUS+WP7LoQIgWgysclSJ shE+gSMTTqRSucz2lQni/PQQ02CQ1yyNWaZ/l2o1Q65kIIBrvruTKcauIJXLFHPYDf v82GLbYC5ca7DfVCj6HJKy6LZDYCsfYP14D7UjjH90s9EA7P/mGEXh9ZKNItEzCo5W hGMVC/BaX6c/bZY+n+PAPKk52I0TOZExvDIm4J+3Ujl/JvFvYn6eqJsyUVrNszZ4h9 GHW2APLlSJaOQBGfLZuURGPj8w7+UPzUXiXri1VJoc7WZSTBFvB22VS7xpUJO/hFsi W8GPnwn2/JBXg== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 19/24] media: rkvdec-h264: Add field decoding support Date: Tue, 26 Apr 2022 08:57:45 -0400 Message-Id: <20220426125751.108293-20-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055834_827419_77134BB2 X-CRM114-Status: GOOD ( 18.14 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org This makes use of the new feature in the reference builder to program up to 32 references when doing field decoding. It also signals the parity (top or bottom) of the field to the hardware. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec-h264.c | 48 ++++++++++------------ 1 file changed, 21 insertions(+), 27 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 7011b66c1aab..fbe23d598720 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -97,13 +97,10 @@ struct rkvdec_h264_priv_tbl { u8 err_info[RKV_ERROR_INFO_SIZE]; }; -#define RKVDEC_H264_DPB_SIZE 16 - struct rkvdec_h264_reflists { struct v4l2_h264_reference p[V4L2_H264_REF_LIST_LEN]; struct v4l2_h264_reference b0[V4L2_H264_REF_LIST_LEN]; struct v4l2_h264_reference b1[V4L2_H264_REF_LIST_LEN]; - u8 num_valid; }; struct rkvdec_h264_run { @@ -747,23 +744,26 @@ static void lookup_ref_buf_idx(struct rkvdec_ctx *ctx, struct vb2_queue *cap_q = &m2m_ctx->cap_q_ctx.q; int buf_idx = -1; - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) { buf_idx = vb2_find_timestamp(cap_q, dpb[i].reference_ts, 0); + if (buf_idx < 0) + pr_debug("No buffer for reference_ts %llu", + dpb[i].reference_ts); + } run->ref_buf_idx[i] = buf_idx; } } static void assemble_hw_rps(struct rkvdec_ctx *ctx, + struct v4l2_h264_reflist_builder *builder, struct rkvdec_h264_run *run) { const struct v4l2_ctrl_h264_decode_params *dec_params = run->decode_params; const struct v4l2_h264_dpb_entry *dpb = dec_params->dpb; struct rkvdec_h264_ctx *h264_ctx = ctx->priv; - const struct v4l2_ctrl_h264_sps *sps = run->sps; struct rkvdec_h264_priv_tbl *priv_tbl = h264_ctx->priv_tbl.cpu; - u32 max_frame_num = 1 << (sps->log2_max_frame_num_minus4 + 4); u32 *hw_rps = priv_tbl->rps; u32 i, j; @@ -781,37 +781,36 @@ static void assemble_hw_rps(struct rkvdec_ctx *ctx, if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) continue; - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM || - dpb[i].frame_num <= dec_params->frame_num) { - p[i] = dpb[i].frame_num; - continue; - } - - p[i] = dpb[i].frame_num - max_frame_num; + p[i] = builder->refs[i].frame_num; } for (j = 0; j < RKVDEC_NUM_REFLIST; j++) { - for (i = 0; i < h264_ctx->reflists.num_valid; i++) { - bool dpb_valid = run->ref_buf_idx[i] >= 0; - u8 idx = 0; + for (i = 0; i < builder->num_valid; i++) { + struct v4l2_h264_reference *ref; + bool dpb_valid; + bool bottom; switch (j) { case 0: - idx = h264_ctx->reflists.p[i].index; + ref = &h264_ctx->reflists.p[i]; break; case 1: - idx = h264_ctx->reflists.b0[i].index; + ref = &h264_ctx->reflists.b0[i]; break; case 2: - idx = h264_ctx->reflists.b1[i].index; + ref = &h264_ctx->reflists.b1[i]; break; } - if (idx >= ARRAY_SIZE(dec_params->dpb)) + if (WARN_ON(ref->index >= ARRAY_SIZE(dec_params->dpb))) continue; + dpb_valid = run->ref_buf_idx[ref->index] >= 0; + bottom = ref->fields == V4L2_H264_BOTTOM_FIELD_REF; + set_ps_field(hw_rps, DPB_INFO(i, j), - idx | dpb_valid << 4); + ref->index | dpb_valid << 4); + set_ps_field(hw_rps, BOTTOM_FLAG(i, j), bottom); } } } @@ -999,10 +998,6 @@ static void config_registers(struct rkvdec_ctx *ctx, rkvdec->regs + RKVDEC_REG_H264_BASE_REFER15); } - /* - * Since support frame mode only - * top_field_order_cnt is the same as bottom_field_order_cnt - */ reg = RKVDEC_CUR_POC(dec_params->top_field_order_cnt); writel_relaxed(reg, rkvdec->regs + RKVDEC_REG_CUR_POC0); @@ -1166,7 +1161,6 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) /* Build the P/B{0,1} ref lists. */ v4l2_h264_init_reflist_builder(&reflist_builder, run.decode_params, run.sps, run.decode_params->dpb); - h264_ctx->reflists.num_valid = reflist_builder.num_valid; v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p); v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0, h264_ctx->reflists.b1); @@ -1174,7 +1168,7 @@ static int rkvdec_h264_run(struct rkvdec_ctx *ctx) assemble_hw_scaling_list(ctx, &run); assemble_hw_pps(ctx, &run); lookup_ref_buf_idx(ctx, &run); - assemble_hw_rps(ctx, &run); + assemble_hw_rps(ctx, &reflist_builder, &run); config_registers(ctx, &run); rkvdec_run_postamble(ctx, &run.base); From patchwork Tue Apr 26 12:57:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827106 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D12D9C433EF for ; Tue, 26 Apr 2022 13:08:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=YQw5/4VCCvtNKTqT0MH1c1tOK9oTh2kKI2pVgCdk04Q=; b=Wh9NAZ0AjpZP5A ng07UmGJ37R/q9bSFOcqwQQKUZ7tJWK06bWwuN7IUgUXtP/4ZdjfvCnrBBfxTGW3Tvk+9ZsCrUFr4 qYx9sqOD2YySjbOyiv3ShXAjHzdX1kwmGA+8skGakytto1WEB/ln80V4wzP2q8dma/TWssAuY/FJr AvsrON0UAzMrblhAguwiHentnDu9fkZ2lT4Ser+TESrlyuCpiaFAy/FEdXFo58VKav2m4qHlg2nps q6jrK6SgAVBLCFmxFAweyCcW5rKervn1zRq/c8yi2Sa5nlnDG1ePnZuP/hZAuv2y5RCIcBanI78Dz vLoqnkHtso5TYqR+VGWQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKvG-00ETlc-08; Tue, 26 Apr 2022 13:08:06 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKm4-00EPJK-55 for linux-rockchip@lists.infradead.org; Tue, 26 Apr 2022 12:58:42 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 9A1961F4398D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977914; bh=MTyE9QSy2RUy2J+DqczMQkzQ2U7CksfjNGFIrOd9SV8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=J9X8SraGQdLyW+/JGUWxPCHGxezEXLlPh5u0O88RM7vcGfKUZ1NvV8FhDg9p3TbfV KJqToHGy8OAloKMz37PAL5AAe2VNPEHwmLhGX1wqvA/fzDZrsPArcgOmPca5aeyllJ yW+FyC4MMSs1gDzXPzSdUVfwT77sU4oFCs0WXV5EwbguRS4Ui8Pi6Kj7y/J1o9chV7 msxtO1hRbODKHyxxlAdKBGrKxvEB39UoulTByyp54H/9METlccF8u1d7hHlsRTjJ2B o5oXWPrhNtvvt2WbIcpxXa+m/j4mFD4P/OlEDp0QagtTCgp7DrlMWjtEPOpKAJz0KA jV/SE88oopK7w== From: Nicolas Dufresne To: Ezequiel Garcia , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 20/24] media: rkvdec: Enable capture buffer holding for H264 Date: Tue, 26 Apr 2022 08:57:46 -0400 Message-Id: <20220426125751.108293-21-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055836_381336_47F3F371 X-CRM114-Status: GOOD ( 11.88 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org In order to support interlaced video decoding, the driver must allow holding the capture buffer so that the second field can be decoded into it. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 4 ++++ drivers/staging/media/rkvdec/rkvdec.h | 1 + 2 files changed, 5 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 2bc4b1a40989..7bab7586918c 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -127,6 +127,7 @@ static const struct rkvdec_coded_fmt_desc rkvdec_coded_fmts[] = { .ops = &rkvdec_h264_fmt_ops, .num_decoded_fmts = ARRAY_SIZE(rkvdec_h264_vp9_decoded_fmts), .decoded_fmts = rkvdec_h264_vp9_decoded_fmts, + .subsystem_flags = VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF, }, { .fourcc = V4L2_PIX_FMT_VP9_FRAME, @@ -385,6 +386,9 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv, cap_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; cap_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization; + /* Enable format specific queue features */ + vq->subsystem_flags |= desc->subsystem_flags; + return 0; } diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h index 9df0fba799a4..633335ebb9c4 100644 --- a/drivers/staging/media/rkvdec/rkvdec.h +++ b/drivers/staging/media/rkvdec/rkvdec.h @@ -82,6 +82,7 @@ struct rkvdec_coded_fmt_desc { const struct rkvdec_coded_fmt_ops *ops; unsigned int num_decoded_fmts; const u32 *decoded_fmts; + u32 subsystem_flags; }; struct rkvdec_dev { From patchwork Tue Apr 26 12:57:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827107 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 81FACC433EF for ; Tue, 26 Apr 2022 13:08:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JqGPZFUtPTIgrt3qpOCihFL5P1KRK+O0dbzkpDNSbsw=; b=Ck6UpiGSj5JkBK LPIqOIVRyjs9bNN4N3Uug1/rAPxZwDXUiXE3DjyZwQQI/VEVEQqVUD74GuWzkMpdhP87EqTzpf/4S XAvpqxNosH69jZdIWRHXXpT8G6uNoDj8swz14rM8i7EGahMZ75fvHReBJ3o3J7s6+1Krpe2bh5162 tShbODs+845RDvOT3CtBBZVbIefVA0EFlfB+LOpBCN4PgpFIyTnPaj8LImIOlg9p4VeOM0O1aFpip umb3ojMi4sDuKCpSQM6Po6/K/nBiP2wOWEGfUpdKgHmW73+OVgAPtGREmZ7L3qSrIEnDIsWhNvraB fLxA+YvtItZvc3IWE3tw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKvS-00ETvQ-Eg; Tue, 26 Apr 2022 13:08:18 +0000 Received: from bhuna.collabora.co.uk ([2a00:1098:0:82:1000:25:2eeb:e3e3]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKm5-00EPKY-SI for linux-rockchip@lists.infradead.org; Tue, 26 Apr 2022 12:58:43 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 498941F43992 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977916; bh=rqMU1/6TJ46L9lH5r2+bP74ysDX3ywT+LO6k1I+k3Bg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NN38d68dmHTqhw4X0vJpCFONROr///7o0gkQVxpcwDpQj8Jz3KwUV+lRZ6Gi7dbRj gbmNVtVshNHarIw1F26EsLUgxGPzqHidqyagAW0FfA06AVfEwlcjDMef4Te3Kx/UDO eh5Xc4y3fhdtPBW2jrxfjzpBJWAwoJcc8lSliupb+DA0kRPI9sYJMvmxUycl4sQJkl ZnWY/7oUZxfN7qUeJYl5eBXwuZbozC0bufbyGQt6TaU8dAkNVTZ57fYuuL6gUsULQT GpPwN9luugkjVokP+4YYhj0OYebaC7hMtmTrcfPw+qmIGzhtpShgaq34Xea3VHQEbN hARecf+vWDilw== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 21/24] media: hantro: Stop using H.264 parameter pic_num Date: Tue, 26 Apr 2022 08:57:47 -0400 Message-Id: <20220426125751.108293-22-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055838_116798_0D3A70FE X-CRM114-Status: UNSURE ( 9.05 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org The hardware expects FrameNumWrap or long_term_frame_idx. Picture numbers are per field, and are mostly used during the memory management process, which is done in userland. This fixes two ITU conformance tests: - MR6_BT_B - MR8_BT_B Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/hantro/hantro_h264.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 0b4d2491be3b..228629fb3cdf 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -354,8 +354,6 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) return 0; - if (dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) - return dpb->pic_num; return dpb->frame_num; } From patchwork Tue Apr 26 12:57:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 00893C433EF for ; Tue, 26 Apr 2022 13:08:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=fiByLGfoKqBnur1KkCUlqe9zDHU+LppUZaG7F7jSSIU=; b=IXO0EmVVmw09C3 Nn6zqFNSnAtflM187jahKeeCTdzRmYRfZLhnfaVnpI3MSVXycePM9glwR2benioS+1wOJmaIiOWgl ZxcInJl2Icas7k+oQQGD6JnVikDLaum3vEsYcg9bMIZPE/z87xogKDZLfAYyZwMYOmTVxp8eCm6+6 HiYZzp+E+qceKRaxuhlzNM22uuDACcfOrkKRzrwrhLc3gdwuCibM0bxq0GNKmsvZcnlhik27st5D5 REQHaGpYnjkMpoE6qw5qs/ImhlfjZX1J9vx0iZ2OCa8px51z+Q4tWDBCSrTpj8DDddDF73Zy6OUmX ceFDMf2ej0hAehMpbvkQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKvZ-00EU19-MF; Tue, 26 Apr 2022 13:08:25 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKm7-00EPLu-LM for linux-rockchip@lists.infradead.org; Tue, 26 Apr 2022 12:58:46 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 1B85C1F43999 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977918; bh=XEgENTYjCMT2Anghn2qsNUIMLssrmzXSWUAtGOk2EIY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WPDkJ3gJEAFrvwPz4kOJCVGDXM69w7LquNYCOmkDxy8hKiZhuHv8vwhLVFR32zL4a GZFenh67Yc8514goV0s8OFXXMF1zLMrphjl9JlgC4cqg4k9HbcsCmO+q93q27VylJ9 MUPV1OyWv4X/5Ny9Xyn3EL9f13FZhfpDaMqdSGa69D7uGqlbGb0WWOIeZnaOihf6fN GDHh0QjC6IsvTHlCUySvLox9sKEr2wUuqQq1jIGLQhgWnpuipLrCZXeNqCiAyCg+Ya 5yae5ut0kQN5j2mtL/wsZwKIzGRe7n6VgZuF/iNEkf5tmJ+G8/1O5A40i7E+yy1Jcu ZqsQp8Dvl7YzA== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, Jonas Karlman , linux-media@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 22/24] media: hantro: h264: Make dpb entry management more robust Date: Tue, 26 Apr 2022 08:57:48 -0400 Message-Id: <20220426125751.108293-23-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055839_927857_47FFEFCC X-CRM114-Status: GOOD ( 12.68 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org From: Jonas Karlman The driver maintains stable slot locations for reference pictures. This change makes the code more robust by using the reference_ts as key and by marking all entries invalid right from the start. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne --- drivers/staging/media/hantro/hantro_h264.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 228629fb3cdf..7377fc26f780 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -258,8 +258,7 @@ static void prepare_table(struct hantro_ctx *ctx) static bool dpb_entry_match(const struct v4l2_h264_dpb_entry *a, const struct v4l2_h264_dpb_entry *b) { - return a->top_field_order_cnt == b->top_field_order_cnt && - a->bottom_field_order_cnt == b->bottom_field_order_cnt; + return a->reference_ts == b->reference_ts; } static void update_dpb(struct hantro_ctx *ctx) @@ -273,13 +272,13 @@ static void update_dpb(struct hantro_ctx *ctx) /* Disable all entries by default. */ for (i = 0; i < ARRAY_SIZE(ctx->h264_dec.dpb); i++) - ctx->h264_dec.dpb[i].flags &= ~V4L2_H264_DPB_ENTRY_FLAG_ACTIVE; + ctx->h264_dec.dpb[i].flags = 0; /* Try to match new DPB entries with existing ones by their POCs. */ for (i = 0; i < ARRAY_SIZE(dec_param->dpb); i++) { const struct v4l2_h264_dpb_entry *ndpb = &dec_param->dpb[i]; - if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) + if (!(ndpb->flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) continue; /* @@ -290,8 +289,7 @@ static void update_dpb(struct hantro_ctx *ctx) struct v4l2_h264_dpb_entry *cdpb; cdpb = &ctx->h264_dec.dpb[j]; - if (cdpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE || - !dpb_entry_match(cdpb, ndpb)) + if (!dpb_entry_match(cdpb, ndpb)) continue; *cdpb = *ndpb; From patchwork Tue Apr 26 12:57:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 838CEC433F5 for ; Tue, 26 Apr 2022 13:08:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kl/QOYAO8yhyefO/QSED589pC+Blez31UX76tPxvUmE=; b=RZU4VXBSe/mlFN UmWKv5leEYCLkvGRwXXWc21TLl3SoIj4FvPakPfBGjWSZWBdkN9Sp43XyKRgQJmn4SvV4Aos7Abn3 V3QF5ZqJlMxjBKCphvGw8i0c65q3ej4rvDH8I+JaVllP/o/quKw9fa0Dkon8ld6wJJIvqw7kqC6ZD vJ6QXfcgw6YksDuj7eEBmTwCezvxEAldEzAtbaNPCi7rSzbsiwd9KDXNaV9t5tb1bKUd8PuaRrZnw Gm9a8obfkTLn1Ctu3xR4lKVQNH7pqCOMF8q0Q/HwXYQy9mgnrz2QOy6ts4JP4jVAn+sQ+CbNmkHUq IXW+B61GrwL7ZDKKxHsg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKvk-00EUAk-BW; Tue, 26 Apr 2022 13:08:36 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKmE-00EPNX-Fs for linux-rockchip@lists.infradead.org; Tue, 26 Apr 2022 12:58:49 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id 165331F4399E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977920; bh=pVl8tjC66altjb15PP079+w75QgIf0TKWek+rq3jsWY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LXCJPO6QrlKwvHosSiRMLPaOQ9AFK/j3CKMJ3oDEVEAuOIEpoHuepVI48IN9MMswv xQhVumfQMc6WzxJTwsz0XL7QuZgAEl4ffOOVljx6BxBdEu6SoM/vmhX0glj8wuCHcM b2bCStl/OHiGD6TXCi8Ha7QwKYfbG+TtvxS6ThO0s/n69E9zWAva+WRXD7nrbWjInU 6DQUWRFY/y8WwWdzvQejVOJxkIgAVqiNqHb4Su69fVwwfmiynRJa5o0QFQ05roxmp5 +2EnYp78w9c9HyuUZOexRLse1c+lpgtGGllcbGVE60qYZfsscOljJu5uxsXdh6lcpP QIFWv5CiLJM1g== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Jonas Karlman , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 23/24] media: hantro: Add H.264 field decoding support Date: Tue, 26 Apr 2022 08:57:49 -0400 Message-Id: <20220426125751.108293-24-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055846_897256_20883E57 X-CRM114-Status: GOOD ( 26.47 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org This adds the required code to support field decoding. While most of the code is derived from Rockchip and VSI reference code, the reduction of the reference list to 16 entries was found by trial and errors. The list consists of all the references with the opposite field parity. The strategy is to deduplicate the reference picture that points to the same storage (same index). The choice of opposite parity has been made to keep the other field of the current field pair in the list. This method may not be robust if a field was lost. Signed-off-by: Jonas Karlman Signed-off-by: Nicolas Dufresne --- drivers/staging/media/hantro/hantro_h264.c | 122 ++++++++++++++++++--- drivers/staging/media/hantro/hantro_hw.h | 1 + 2 files changed, 109 insertions(+), 14 deletions(-) diff --git a/drivers/staging/media/hantro/hantro_h264.c b/drivers/staging/media/hantro/hantro_h264.c index 7377fc26f780..7502dddb324c 100644 --- a/drivers/staging/media/hantro/hantro_h264.c +++ b/drivers/staging/media/hantro/hantro_h264.c @@ -22,6 +22,12 @@ #define POC_BUFFER_SIZE 34 #define SCALING_LIST_SIZE (6 * 16 + 2 * 64) +/* + * For valid and long term reference marking, index are reversed, so bit 31 + * indicates the status of the picture 0. + */ +#define REF_BIT(i) BIT(32 - 1 - (i)) + /* Data structure describing auxiliary buffer format. */ struct hantro_h264_dec_priv_tbl { u32 cabac_table[CABAC_INIT_BUFFER_SIZE]; @@ -227,6 +233,7 @@ static void prepare_table(struct hantro_ctx *ctx) { const struct hantro_h264_dec_ctrls *ctrls = &ctx->h264_dec.ctrls; const struct v4l2_ctrl_h264_decode_params *dec_param = ctrls->decode; + const struct v4l2_ctrl_h264_sps *sps = ctrls->sps; struct hantro_h264_dec_priv_tbl *tbl = ctx->h264_dec.priv.cpu; const struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb; u32 dpb_longterm = 0; @@ -237,20 +244,45 @@ static void prepare_table(struct hantro_ctx *ctx) tbl->poc[i * 2] = dpb[i].top_field_order_cnt; tbl->poc[i * 2 + 1] = dpb[i].bottom_field_order_cnt; + if (!(dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_VALID)) + continue; + /* * Set up bit maps of valid and long term DPBs. - * NOTE: The bits are reversed, i.e. MSb is DPB 0. + * NOTE: The bits are reversed, i.e. MSb is DPB 0. For frame + * decoding, bit 31 to 15 are used, while for field decoding, + * all bits are used, with bit 31 being a top field, 30 a bottom + * field and so on. */ - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) - dpb_valid |= BIT(HANTRO_H264_DPB_SIZE - 1 - i); - if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) - dpb_longterm |= BIT(HANTRO_H264_DPB_SIZE - 1 - i); + if (dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) { + if (dpb[i].fields & V4L2_H264_TOP_FIELD_REF) + dpb_valid |= REF_BIT(i * 2); + + if (dpb[i].fields & V4L2_H264_BOTTOM_FIELD_REF) + dpb_valid |= REF_BIT(i * 2 + 1); + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) { + dpb_longterm |= REF_BIT(i * 2); + dpb_longterm |= REF_BIT(i * 2 + 1); + } + } else { + dpb_valid |= REF_BIT(i); + + if (dpb[i].flags & V4L2_H264_DPB_ENTRY_FLAG_LONG_TERM) + dpb_longterm |= REF_BIT(i); + } + } + ctx->h264_dec.dpb_valid = dpb_valid; + ctx->h264_dec.dpb_longterm = dpb_longterm; + + if ((dec_param->flags & V4L2_H264_DECODE_PARAM_FLAG_FIELD_PIC) || + !(sps->flags & V4L2_H264_SPS_FLAG_MB_ADAPTIVE_FRAME_FIELD)) { + tbl->poc[32] = ctx->h264_dec.cur_poc; + tbl->poc[33] = 0; + } else { + tbl->poc[32] = dec_param->top_field_order_cnt; + tbl->poc[33] = dec_param->bottom_field_order_cnt; } - ctx->h264_dec.dpb_valid = dpb_valid << 16; - ctx->h264_dec.dpb_longterm = dpb_longterm << 16; - - tbl->poc[32] = dec_param->top_field_order_cnt; - tbl->poc[33] = dec_param->bottom_field_order_cnt; assemble_scaling_list(ctx); } @@ -326,6 +358,8 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx, { struct v4l2_h264_dpb_entry *dpb = ctx->h264_dec.dpb; dma_addr_t dma_addr = 0; + s32 cur_poc = ctx->h264_dec.cur_poc; + u32 flags; if (dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE) dma_addr = hantro_get_ref(ctx, dpb[dpb_idx].reference_ts); @@ -343,7 +377,12 @@ dma_addr_t hantro_h264_get_ref_buf(struct hantro_ctx *ctx, dma_addr = hantro_get_dec_buf_addr(ctx, buf); } - return dma_addr; + flags = dpb[dpb_idx].flags & V4L2_H264_DPB_ENTRY_FLAG_FIELD ? 0x2 : 0; + flags |= abs(dpb[dpb_idx].top_field_order_cnt - cur_poc) < + abs(dpb[dpb_idx].bottom_field_order_cnt - cur_poc) ? + 0x1 : 0; + + return dma_addr | flags; } u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) @@ -355,6 +394,47 @@ u16 hantro_h264_get_ref_nbr(struct hantro_ctx *ctx, unsigned int dpb_idx) return dpb->frame_num; } +/* + * Removes all references with he same parity as current picture from the + * reference list. The remaining list will have references with the opposite + * parity. This is effectively a deduplication of references since each buffer + * stores two fields. For this eason, each buffer are found twice in the + * reference list. + * + * This technique has been chosen through trial and error. This simple approach + * resulted in the highest conformance score. Note that this method may suffer + * worse quality in the case an opposite reference frame has been lost. If this + * becomes a problem in the future, it should be possible to add a preprocessing + * to identify un-paired fields and avoid removing them. + */ +static void deduplicate_reflist(struct v4l2_h264_reflist_builder *b, + struct v4l2_h264_reference *reflist) +{ + int write_idx = 0; + int i; + + if (b->cur_pic_fields == V4L2_H264_FRAME_REF) { + write_idx = b->num_valid; + goto done; + } + + for (i = 0; i < b->num_valid; i++) { + if (!(b->cur_pic_fields == reflist[i].fields)) { + reflist[write_idx++] = reflist[i]; + continue; + } + } + +done: + /* Should not happen unless we have a bug in the reflist builder. */ + if (WARN_ON(write_idx > 16)) + write_idx = 16; + + /* Clear the remaining, some streams fails otherwise */ + for (; write_idx < 16; write_idx++) + reflist[write_idx].index = 15; +} + int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx) { struct hantro_h264_dec_hw_ctx *h264_ctx = &ctx->h264_dec; @@ -386,15 +466,29 @@ int hantro_h264_dec_prepare_run(struct hantro_ctx *ctx) /* Update the DPB with new refs. */ update_dpb(ctx); - /* Prepare data in memory. */ - prepare_table(ctx); - /* Build the P/B{0,1} ref lists. */ v4l2_h264_init_reflist_builder(&reflist_builder, ctrls->decode, ctrls->sps, ctx->h264_dec.dpb); + h264_ctx->cur_poc = reflist_builder.cur_pic_order_count; + + /* Prepare data in memory. */ + prepare_table(ctx); + v4l2_h264_build_p_ref_list(&reflist_builder, h264_ctx->reflists.p); v4l2_h264_build_b_ref_lists(&reflist_builder, h264_ctx->reflists.b0, h264_ctx->reflists.b1); + + /* + * Reduce ref lists to at most 16 entries, Hantro hardware will deduce + * the actual picture lists in field through the dpb_valid, + * dpb_longterm bitmap along with the current frame parity. + */ + if (reflist_builder.cur_pic_fields != V4L2_H264_FRAME_REF) { + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.p); + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b0); + deduplicate_reflist(&reflist_builder, h264_ctx->reflists.b1); + } + return 0; } diff --git a/drivers/staging/media/hantro/hantro_hw.h b/drivers/staging/media/hantro/hantro_hw.h index 292aaaabaf24..fd869369fb97 100644 --- a/drivers/staging/media/hantro/hantro_hw.h +++ b/drivers/staging/media/hantro/hantro_hw.h @@ -91,6 +91,7 @@ struct hantro_h264_dec_hw_ctx { struct hantro_h264_dec_ctrls ctrls; u32 dpb_longterm; u32 dpb_valid; + s32 cur_poc; }; /** From patchwork Tue Apr 26 12:57:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 12827118 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E66C7C433F5 for ; Tue, 26 Apr 2022 13:08:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=R99d1t88PfHO1MavJ+lsYLpOc6iWSPg0dQY7Illl360=; b=kLNr3hU5atrcWD esEm7HzecJvi9pjpZaKzhPKweosyf493FQtBMFDAdIhZLCKwf9F0Venhy91TaSUSVPWcTsrHe9IBB 5jyhb3BjgxwPUUdDcuya3OYiYCtPxusjgYW+FPfZjHSr5fRdTGf6k8s8s0xp/QQo+9jv7rGLlSxjp u3JQcaWBtAnHZ5SvyeLf1bvA7EMoAcjWYDNZQCPPG8ikBHxJYDdpuNMdx9iuejucb2ryT6nysoX0r dkcfO25r4/0rEakoZzhNPXTMTuOmjmpwuhcUZZnh7StNqT21+LZ1jSaFyB3VUtGZTPzmI6mT7pmUc Zz5RflXfudtET+9Q/+7w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKvf-00EU52-48; Tue, 26 Apr 2022 13:08:31 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1njKmF-00EPRV-9K for linux-rockchip@lists.infradead.org; Tue, 26 Apr 2022 12:58:48 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nicolas) with ESMTPSA id DCF2E1F439A0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1650977922; bh=OT93d1YFbCsPl7b/q8Q9Ng7KBwuAX7su/L0Pt2JnS6E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cBZN0vuWa0CQYiFTtznCmJ69cnWWpBJjUsOTglusEeiqSVayUvm4THtei/qrnz+XU sxKfjpwT+fa/HTO2lmpV7wL724s3r+9mMdahR1qJH2P8t1iXck7NuGbaxaJly5vKqm +CnjtVkTT3iaegGldibqCE7Vq/khasZDDfEUwlJxfHfFV8bcR5uN90Z9I3HPpzsrwr H5DMc5649WTvQ9vq9OrnHgzL20sr2Z3Y9i6F2AjFq/IVQ6dGlbdqjLCg5i3ua4vQp1 94j5HRwXA2ySzWD6Zqm1P5Cu2nEMfyBeoAfYWvwy7OQ8HM7PJJY/Mr1lI+iWwHPo40 nEC1lF3RULBxw== From: Nicolas Dufresne To: Ezequiel Garcia , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman Cc: nicolas@ndufresne.ca, linux-media@vger.kernel.org, Sebastian Fricke , linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, linux-kernel@vger.kernel.org Subject: [PATCH v4 24/24] media: hantro: Enable HOLD_CAPTURE_BUF for H.264 Date: Tue, 26 Apr 2022 08:57:50 -0400 Message-Id: <20220426125751.108293-25-nicolas.dufresne@collabora.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220426125751.108293-1-nicolas.dufresne@collabora.com> References: <20220426125751.108293-1-nicolas.dufresne@collabora.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220426_055847_523148_CF04E5C3 X-CRM114-Status: GOOD ( 10.36 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+linux-rockchip=archiver.kernel.org@lists.infradead.org This is needed to optimize field decoding. Each field will be decoded into the same capture buffer. To be able to queue multiple buffers, we need to be able to ask the driver to hold the capture buffer. Signed-off-by: Nicolas Dufresne Reviewed-by: Sebastian Fricke --- drivers/staging/media/hantro/hantro_v4l2.c | 25 ++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/drivers/staging/media/hantro/hantro_v4l2.c b/drivers/staging/media/hantro/hantro_v4l2.c index 67148ba346f5..50d636678ff3 100644 --- a/drivers/staging/media/hantro/hantro_v4l2.c +++ b/drivers/staging/media/hantro/hantro_v4l2.c @@ -409,6 +409,30 @@ hantro_update_requires_request(struct hantro_ctx *ctx, u32 fourcc) } } +static void +hantro_update_requires_hold_capture_buf(struct hantro_ctx *ctx, u32 fourcc) +{ + struct vb2_queue *vq; + + vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + + switch (fourcc) { + case V4L2_PIX_FMT_JPEG: + case V4L2_PIX_FMT_MPEG2_SLICE: + case V4L2_PIX_FMT_VP8_FRAME: + case V4L2_PIX_FMT_HEVC_SLICE: + case V4L2_PIX_FMT_VP9_FRAME: + vq->subsystem_flags &= ~(VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF); + break; + case V4L2_PIX_FMT_H264_SLICE: + vq->subsystem_flags |= VB2_V4L2_FL_SUPPORTS_M2M_HOLD_CAPTURE_BUF; + break; + default: + break; + } +} + static int hantro_set_fmt_out(struct hantro_ctx *ctx, struct v4l2_pix_format_mplane *pix_mp) { @@ -472,6 +496,7 @@ static int hantro_set_fmt_out(struct hantro_ctx *ctx, ctx->dst_fmt.quantization = pix_mp->quantization; hantro_update_requires_request(ctx, pix_mp->pixelformat); + hantro_update_requires_hold_capture_buf(ctx, pix_mp->pixelformat); vpu_debug(0, "OUTPUT codec mode: %d\n", ctx->vpu_src_fmt->codec_mode); vpu_debug(0, "fmt - w: %d, h: %d\n",