From patchwork Mon Jun 3 11:09:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Brezillon X-Patchwork-Id: 10972759 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7990514E5 for ; Mon, 3 Jun 2019 11:09:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6A6C32874E for ; Mon, 3 Jun 2019 11:09:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5DF462877E; Mon, 3 Jun 2019 11:09:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E6E982878F for ; Mon, 3 Jun 2019 11:09:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727728AbfFCLJw (ORCPT ); Mon, 3 Jun 2019 07:09:52 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:60828 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727345AbfFCLJw (ORCPT ); Mon, 3 Jun 2019 07:09:52 -0400 Received: from localhost.localdomain (unknown [IPv6:2a01:e0a:2c:6930:5cf4:84a1:2763:fe0d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: bbrezillon) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 8E411285156; Mon, 3 Jun 2019 12:09:50 +0100 (BST) From: Boris Brezillon To: Mauro Carvalho Chehab , Hans Verkuil , Laurent Pinchart , Sakari Ailus , linux-media@vger.kernel.org Cc: Tomasz Figa , Nicolas Dufresne , kernel@collabora.com, Paul Kocialkowski , Ezequiel Garcia , Jonas Karlman , Jernej Skrabec , Alexandre Courbot , Thierry Reding , Boris Brezillon Subject: [PATCH RFC 1/6] media: uapi: h264: Clarify our expectations regarding NAL header format Date: Mon, 3 Jun 2019 13:09:41 +0200 Message-Id: <20190603110946.4952-2-boris.brezillon@collabora.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190603110946.4952-1-boris.brezillon@collabora.com> References: <20190603110946.4952-1-boris.brezillon@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Looks like some stateless decoders expect slices to be prefixed with ANNEX B start codes (they most likely do some kind of bitstream parsing and/or need that to delimit slices when doing per frame decoding). Since skipping those start codes for dummy stateless decoders (those expecting all params to be passed through controls) should be pretty easy, let's mandate that all slices be prepended with ANNEX B start codes. If we ever need to support AVC headers, we can add a new menu control to select the type of NAL header to use. Signed-off-by: Boris Brezillon --- Note: we might want to add a nal_header_size field to allow drivers to quickly find where the actual payload start. That'd be particularly useful here since ANNEX B start codes can be of arbitrary size (2+(0x00 bytes) + 1(0x01 byte)). The other option would be to enforce the number of leading 0x00 bytes (a minimum of 2 is required). --- Documentation/media/uapi/v4l/ext-ctrls-codec.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst index d6ea2ffd65c5..82547d5de250 100644 --- a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst +++ b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst @@ -1726,6 +1726,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - :ref:`h264`, section 7.4.3 "Slice Header Semantics". For further documentation, refer to the above specification, unless there is an explicit comment stating otherwise. + All slices should be prepended with an ANNEX B start code. .. note:: From patchwork Mon Jun 3 11:09:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Brezillon X-Patchwork-Id: 10972761 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 03AFF6C5 for ; Mon, 3 Jun 2019 11:09:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E83D22879E for ; Mon, 3 Jun 2019 11:09:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DCBD1287AE; Mon, 3 Jun 2019 11:09:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 46440283C8 for ; Mon, 3 Jun 2019 11:09:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727861AbfFCLJx (ORCPT ); Mon, 3 Jun 2019 07:09:53 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:60850 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727682AbfFCLJx (ORCPT ); Mon, 3 Jun 2019 07:09:53 -0400 Received: from localhost.localdomain (unknown [IPv6:2a01:e0a:2c:6930:5cf4:84a1:2763:fe0d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: bbrezillon) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 2646A285157; Mon, 3 Jun 2019 12:09:51 +0100 (BST) From: Boris Brezillon To: Mauro Carvalho Chehab , Hans Verkuil , Laurent Pinchart , Sakari Ailus , linux-media@vger.kernel.org Cc: Tomasz Figa , Nicolas Dufresne , kernel@collabora.com, Paul Kocialkowski , Ezequiel Garcia , Jonas Karlman , Jernej Skrabec , Alexandre Courbot , Thierry Reding , Boris Brezillon Subject: [PATCH RFC 2/6] media: uapi: h264: Add the concept of decoding mode Date: Mon, 3 Jun 2019 13:09:42 +0200 Message-Id: <20190603110946.4952-3-boris.brezillon@collabora.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190603110946.4952-1-boris.brezillon@collabora.com> References: <20190603110946.4952-1-boris.brezillon@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Some stateless decoders don't support per-slice decoding (or at least not in a way that would make them efficient or easy to use). Let's expose a menu to control and expose the supported decoding modes. Drivers are allowed to support only one decoding but they can support both too. Signed-off-by: Boris Brezillon --- .../media/uapi/v4l/ext-ctrls-codec.rst | 42 ++++++++++++++++++- drivers/media/v4l2-core/v4l2-ctrls.c | 9 ++++ include/media/h264-ctrls.h | 13 ++++++ 3 files changed, 63 insertions(+), 1 deletion(-) diff --git a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst index 82547d5de250..188f625acb7c 100644 --- a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst +++ b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst @@ -1748,6 +1748,14 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - * - __u32 - ``size`` - + * - __u32 + - ``start_byte_offset`` + - Where the slice payload starts in the output buffer. Useful when + operating in per frame decoding mode and decoding multi-slice content. + In this case, the output buffer will contain more than one slice and + some codecs need to know where each slice starts. Note that this + offsets points to the beginning of the slice which is supposed to + contain an ANNEX B start code * - __u32 - ``header_bit_size`` - @@ -1931,7 +1939,10 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - - * - __u16 - ``num_slices`` - - Number of slices needed to decode the current frame + - Number of slices needed to decode the current frame/field. When + operating in per-slice decoding mode (see + :c:type:`v4l2_mpeg_video_h264_decoding_mode`), this field + should always be set to one * - __u16 - ``nal_ref_idc`` - NAL reference ID value coming from the NAL Unit header @@ -2022,6 +2033,35 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - - 0x00000004 - The DPB entry is a long term reference frame +``V4L2_CID_MPEG_VIDEO_H264_DECODING_MODE (enum)`` + Specifies the decoding mode to use. Currently exposes per slice and per + frame decoding but new modes might be added later on. + + .. note:: + + This menu control is not yet part of the public kernel API and + it is expected to change. + +.. c:type:: v4l2_mpeg_video_h264_decoding_mode + +.. cssclass:: longtable + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - ``V4L2_MPEG_VIDEO_H264_DECODING_PER_SLICE`` + - 0 + - The decoding is done per slice. v4l2_ctrl_h264_decode_params->num_slices + must be set to 1 and the output buffer should contain only one slice. + * - ``V4L2_MPEG_VIDEO_H264_DECODING_PER_FRAME`` + - 1 + - The decoding is done per frame. v4l2_ctrl_h264_decode_params->num_slices + can be > 1. When that happens, the output buffer should contain all + slices needed to decode a frame/field, each slice being prefixed by an + Annex B NAL header/start-code. + .. _v4l2-mpeg-mpeg2: ``V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS (struct)`` diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c index 1217d38ea394..72bb3c8882f5 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls.c +++ b/drivers/media/v4l2-core/v4l2-ctrls.c @@ -406,6 +406,11 @@ const char * const *v4l2_ctrl_get_menu(u32 id) "Explicit", NULL, }; + static const char * const h264_decoding_mode[] = { + "Per Slice", + "Per Frame", + NULL, + }; static const char * const mpeg_mpeg2_level[] = { "Low", "Main", @@ -637,6 +642,8 @@ const char * const *v4l2_ctrl_get_menu(u32 id) return h264_fp_arrangement_type; case V4L2_CID_MPEG_VIDEO_H264_FMO_MAP_TYPE: return h264_fmo_map_type; + case V4L2_CID_MPEG_VIDEO_H264_DECODING_MODE: + return h264_decoding_mode; case V4L2_CID_MPEG_VIDEO_MPEG2_LEVEL: return mpeg_mpeg2_level; case V4L2_CID_MPEG_VIDEO_MPEG2_PROFILE: @@ -856,6 +863,7 @@ const char *v4l2_ctrl_get_name(u32 id) case V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX: return "H264 Scaling Matrix"; case V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS: return "H264 Slice Parameters"; case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS: return "H264 Decode Parameters"; + case V4L2_CID_MPEG_VIDEO_H264_DECODING_MODE: return "H264 Decoding Mode"; case V4L2_CID_MPEG_VIDEO_MPEG2_LEVEL: return "MPEG2 Level"; case V4L2_CID_MPEG_VIDEO_MPEG2_PROFILE: return "MPEG2 Profile"; case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value"; @@ -1224,6 +1232,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, case V4L2_CID_MPEG_VIDEO_H264_VUI_SAR_IDC: case V4L2_CID_MPEG_VIDEO_H264_SEI_FP_ARRANGEMENT_TYPE: case V4L2_CID_MPEG_VIDEO_H264_FMO_MAP_TYPE: + case V4L2_CID_MPEG_VIDEO_H264_DECODING_MODE: case V4L2_CID_MPEG_VIDEO_MPEG2_LEVEL: case V4L2_CID_MPEG_VIDEO_MPEG2_PROFILE: case V4L2_CID_MPEG_VIDEO_MPEG4_LEVEL: diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h index e1404d78d6ff..26de2243f6f5 100644 --- a/include/media/h264-ctrls.h +++ b/include/media/h264-ctrls.h @@ -26,6 +26,7 @@ #define V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (V4L2_CID_MPEG_BASE+1002) #define V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS (V4L2_CID_MPEG_BASE+1003) #define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (V4L2_CID_MPEG_BASE+1004) +#define V4L2_CID_MPEG_VIDEO_H264_DECODING_MODE (V4L2_CID_MPEG_BASE+1005) /* enum v4l2_ctrl_type type values */ #define V4L2_CTRL_TYPE_H264_SPS 0x0110 @@ -33,6 +34,12 @@ #define V4L2_CTRL_TYPE_H264_SCALING_MATRIX 0x0112 #define V4L2_CTRL_TYPE_H264_SLICE_PARAMS 0x0113 #define V4L2_CTRL_TYPE_H264_DECODE_PARAMS 0x0114 +#define V4L2_CTRL_TYPE_H264_DECODING_MODE 0x0115 + +enum v4l2_mpeg_video_h264_decoding_mode { + V4L2_MPEG_VIDEO_H264_DECODING_PER_SLICE, + V4L2_MPEG_VIDEO_H264_DECODING_PER_FRAME, +}; #define V4L2_H264_SPS_CONSTRAINT_SET0_FLAG 0x01 #define V4L2_H264_SPS_CONSTRAINT_SET1_FLAG 0x02 @@ -111,6 +118,8 @@ struct v4l2_h264_pred_weight_table { struct v4l2_h264_weight_factors weight_factors[2]; }; +#define V4L2_H264_MAX_SLICES_PER_FRAME 16 + #define V4L2_H264_SLICE_TYPE_P 0 #define V4L2_H264_SLICE_TYPE_B 1 #define V4L2_H264_SLICE_TYPE_I 2 @@ -125,6 +134,10 @@ struct v4l2_h264_pred_weight_table { struct v4l2_ctrl_h264_slice_params { /* Size in bytes, including header */ __u32 size; + + /* Where the slice starts in the output buffer (expressed in bytes). */ + __u32 start_byte_offset; + /* Offset in bits to slice_data() from the beginning of this slice. */ __u32 header_bit_size; From patchwork Mon Jun 3 11:09:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Brezillon X-Patchwork-Id: 10972763 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 435BB1398 for ; Mon, 3 Jun 2019 11:09:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 346642879E for ; Mon, 3 Jun 2019 11:09:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 28CAC287AE; Mon, 3 Jun 2019 11:09:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD2852877E for ; Mon, 3 Jun 2019 11:09:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727608AbfFCLJy (ORCPT ); Mon, 3 Jun 2019 07:09:54 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:60872 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727345AbfFCLJx (ORCPT ); Mon, 3 Jun 2019 07:09:53 -0400 Received: from localhost.localdomain (unknown [IPv6:2a01:e0a:2c:6930:5cf4:84a1:2763:fe0d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: bbrezillon) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id B60F7285161; Mon, 3 Jun 2019 12:09:51 +0100 (BST) From: Boris Brezillon To: Mauro Carvalho Chehab , Hans Verkuil , Laurent Pinchart , Sakari Ailus , linux-media@vger.kernel.org Cc: Tomasz Figa , Nicolas Dufresne , kernel@collabora.com, Paul Kocialkowski , Ezequiel Garcia , Jonas Karlman , Jernej Skrabec , Alexandre Courbot , Thierry Reding , Boris Brezillon Subject: [PATCH RFC 3/6] media: uapi: h264: Get rid of the p0/b0/b1 ref-lists Date: Mon, 3 Jun 2019 13:09:43 +0200 Message-Id: <20190603110946.4952-4-boris.brezillon@collabora.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190603110946.4952-1-boris.brezillon@collabora.com> References: <20190603110946.4952-1-boris.brezillon@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Those lists can be extracted from the dpb, let's simplify userspace life and build that list kernel-side (generic helpers will be provided for drivers that need this list). Signed-off-by: Boris Brezillon --- Documentation/media/uapi/v4l/ext-ctrls-codec.rst | 9 --------- 1 file changed, 9 deletions(-) diff --git a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst index 188f625acb7c..f3edb18ac19a 100644 --- a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst +++ b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst @@ -1946,15 +1946,6 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - * - __u16 - ``nal_ref_idc`` - NAL reference ID value coming from the NAL Unit header - * - __u8 - - ``ref_pic_list_p0[32]`` - - Backward reference list used by P-frames in the original bitstream order - * - __u8 - - ``ref_pic_list_b0[32]`` - - Backward reference list used by B-frames in the original bitstream order - * - __u8 - - ``ref_pic_list_b1[32]`` - - Forward reference list used by B-frames in the original bitstream order * - __s32 - ``top_field_order_cnt`` - Picture Order Count for the coded top field From patchwork Mon Jun 3 11:09:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Brezillon X-Patchwork-Id: 10972765 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 86A7F14E5 for ; Mon, 3 Jun 2019 11:09:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 75B072877E for ; Mon, 3 Jun 2019 11:09:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 69975287AE; Mon, 3 Jun 2019 11:09:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ED09B1FFCA for ; Mon, 3 Jun 2019 11:09:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727867AbfFCLJz (ORCPT ); Mon, 3 Jun 2019 07:09:55 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:60894 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727760AbfFCLJy (ORCPT ); Mon, 3 Jun 2019 07:09:54 -0400 Received: from localhost.localdomain (unknown [IPv6:2a01:e0a:2c:6930:5cf4:84a1:2763:fe0d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: bbrezillon) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 506DF285165; Mon, 3 Jun 2019 12:09:52 +0100 (BST) From: Boris Brezillon To: Mauro Carvalho Chehab , Hans Verkuil , Laurent Pinchart , Sakari Ailus , linux-media@vger.kernel.org Cc: Tomasz Figa , Nicolas Dufresne , kernel@collabora.com, Paul Kocialkowski , Ezequiel Garcia , Jonas Karlman , Jernej Skrabec , Alexandre Courbot , Thierry Reding , Boris Brezillon Subject: [PATCH RFC 4/6] media: cedrus: Prepare things to support !compound controls Date: Mon, 3 Jun 2019 13:09:44 +0200 Message-Id: <20190603110946.4952-5-boris.brezillon@collabora.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190603110946.4952-1-boris.brezillon@collabora.com> References: <20190603110946.4952-1-boris.brezillon@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We are about to add a menu control, so let's make the code more generic to support other control types. Signed-off-by: Boris Brezillon --- drivers/staging/media/sunxi/cedrus/cedrus.c | 47 ++++++++++++--------- drivers/staging/media/sunxi/cedrus/cedrus.h | 3 +- 2 files changed, 29 insertions(+), 21 deletions(-) diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c index 370937edfc14..378032fe71f9 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c @@ -29,44 +29,44 @@ static const struct cedrus_control cedrus_controls[] = { { - .id = V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS, - .elem_size = sizeof(struct v4l2_ctrl_mpeg2_slice_params), + .cfg.id = V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS, + .cfg.elem_size = sizeof(struct v4l2_ctrl_mpeg2_slice_params), .codec = CEDRUS_CODEC_MPEG2, .required = true, }, { - .id = V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION, - .elem_size = sizeof(struct v4l2_ctrl_mpeg2_quantization), + .cfg.id = V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION, + .cfg.elem_size = sizeof(struct v4l2_ctrl_mpeg2_quantization), .codec = CEDRUS_CODEC_MPEG2, .required = false, }, { - .id = V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS, - .elem_size = sizeof(struct v4l2_ctrl_h264_decode_params), + .cfg.id = V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS, + .cfg.elem_size = sizeof(struct v4l2_ctrl_h264_decode_params), .codec = CEDRUS_CODEC_H264, .required = true, }, { - .id = V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS, - .elem_size = sizeof(struct v4l2_ctrl_h264_slice_params), + .cfg.id = V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS, + .cfg.elem_size = sizeof(struct v4l2_ctrl_h264_slice_params), .codec = CEDRUS_CODEC_H264, .required = true, }, { - .id = V4L2_CID_MPEG_VIDEO_H264_SPS, - .elem_size = sizeof(struct v4l2_ctrl_h264_sps), + .cfg.id = V4L2_CID_MPEG_VIDEO_H264_SPS, + .cfg.elem_size = sizeof(struct v4l2_ctrl_h264_sps), .codec = CEDRUS_CODEC_H264, .required = true, }, { - .id = V4L2_CID_MPEG_VIDEO_H264_PPS, - .elem_size = sizeof(struct v4l2_ctrl_h264_pps), + .cfg.id = V4L2_CID_MPEG_VIDEO_H264_PPS, + .cfg.elem_size = sizeof(struct v4l2_ctrl_h264_pps), .codec = CEDRUS_CODEC_H264, .required = true, }, { - .id = V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX, - .elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix), + .cfg.id = V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX, + .cfg.elem_size = sizeof(struct v4l2_ctrl_h264_scaling_matrix), .codec = CEDRUS_CODEC_H264, .required = true, }, @@ -106,12 +106,21 @@ static int cedrus_init_ctrls(struct cedrus_dev *dev, struct cedrus_ctx *ctx) return -ENOMEM; for (i = 0; i < CEDRUS_CONTROLS_COUNT; i++) { - struct v4l2_ctrl_config cfg = {}; + const struct v4l2_ctrl_config *cfg = &cedrus_controls[i].cfg; - cfg.elem_size = cedrus_controls[i].elem_size; - cfg.id = cedrus_controls[i].id; + if (cfg->elem_size) + ctrl = v4l2_ctrl_new_custom(hdl, cfg, NULL); + else if (cfg->type == V4L2_CTRL_TYPE_MENU || + cfg->type == V4L2_CTRL_TYPE_INTEGER_MENU) + ctrl = v4l2_ctrl_new_std_menu(hdl, NULL, + cfg->id, cfg->max, + cfg->menu_skip_mask, + cfg->def); + else + ctrl = v4l2_ctrl_new_std(hdl, NULL, cfg->id, cfg->min, + cfg->max, cfg->step, + cfg->def); - ctrl = v4l2_ctrl_new_custom(hdl, &cfg, NULL); if (hdl->error) { v4l2_err(&dev->v4l2_dev, "Failed to create new custom control\n"); @@ -178,7 +187,7 @@ static int cedrus_request_validate(struct media_request *req) continue; ctrl_test = v4l2_ctrl_request_hdl_ctrl_find(hdl, - cedrus_controls[i].id); + cedrus_controls[i].cfg.id); if (!ctrl_test) { v4l2_info(&ctx->dev->v4l2_dev, "Missing required codec control\n"); diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h b/drivers/staging/media/sunxi/cedrus/cedrus.h index 3f476d0fd981..69c037724d93 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus.h +++ b/drivers/staging/media/sunxi/cedrus/cedrus.h @@ -49,8 +49,7 @@ enum cedrus_h264_pic_type { }; struct cedrus_control { - u32 id; - u32 elem_size; + struct v4l2_ctrl_config cfg; enum cedrus_codec codec; unsigned char required:1; }; From patchwork Mon Jun 3 11:09:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Brezillon X-Patchwork-Id: 10972769 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 435001398 for ; Mon, 3 Jun 2019 11:09:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3293C1FFCA for ; Mon, 3 Jun 2019 11:09:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 26AC5287AE; Mon, 3 Jun 2019 11:09:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CC4881FFCA for ; Mon, 3 Jun 2019 11:09:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727872AbfFCLJz (ORCPT ); Mon, 3 Jun 2019 07:09:55 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:60904 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727862AbfFCLJz (ORCPT ); Mon, 3 Jun 2019 07:09:55 -0400 Received: from localhost.localdomain (unknown [IPv6:2a01:e0a:2c:6930:5cf4:84a1:2763:fe0d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: bbrezillon) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id DC28D285168; Mon, 3 Jun 2019 12:09:52 +0100 (BST) From: Boris Brezillon To: Mauro Carvalho Chehab , Hans Verkuil , Laurent Pinchart , Sakari Ailus , linux-media@vger.kernel.org Cc: Tomasz Figa , Nicolas Dufresne , kernel@collabora.com, Paul Kocialkowski , Ezequiel Garcia , Jonas Karlman , Jernej Skrabec , Alexandre Courbot , Thierry Reding , Boris Brezillon Subject: [PATCH RFC 5/6] media: cedrus: Make the slice_params array size limitation more explicit Date: Mon, 3 Jun 2019 13:09:45 +0200 Message-Id: <20190603110946.4952-6-boris.brezillon@collabora.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190603110946.4952-1-boris.brezillon@collabora.com> References: <20190603110946.4952-1-boris.brezillon@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The driver only supports per-slice decoding, and in that mode decode_params->num_slices must be set to 1 and the slice_params array should contain only one element. The current code already had this limitation but it made it look like the slice_params control was a single struct while, according to the spec, it's actually an array. Make it more explicit by setting dims[0] and adding a comment explaining why we have this limitation. Signed-off-by: Boris Brezillon --- drivers/staging/media/sunxi/cedrus/cedrus.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c index 378032fe71f9..3661c6a04864 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c @@ -49,6 +49,12 @@ static const struct cedrus_control cedrus_controls[] = { { .cfg.id = V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS, .cfg.elem_size = sizeof(struct v4l2_ctrl_h264_slice_params), + /* + * This driver does not support per-frame decoding (yet?). + * Allow only on per-slice decoding operation, which implies + * that only 1 slice param is passed per decoding operation. + */ + .cfg.dims[0] = 1, .codec = CEDRUS_CODEC_H264, .required = true, }, From patchwork Mon Jun 3 11:09:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Boris Brezillon X-Patchwork-Id: 10972767 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E39456C5 for ; Mon, 3 Jun 2019 11:09:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D47312874E for ; Mon, 3 Jun 2019 11:09:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C5C742877E; Mon, 3 Jun 2019 11:09:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 729E72874E for ; Mon, 3 Jun 2019 11:09:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727869AbfFCLJz (ORCPT ); Mon, 3 Jun 2019 07:09:55 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:60850 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727345AbfFCLJz (ORCPT ); Mon, 3 Jun 2019 07:09:55 -0400 Received: from localhost.localdomain (unknown [IPv6:2a01:e0a:2c:6930:5cf4:84a1:2763:fe0d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: bbrezillon) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 8D7EC284773; Mon, 3 Jun 2019 12:09:53 +0100 (BST) From: Boris Brezillon To: Mauro Carvalho Chehab , Hans Verkuil , Laurent Pinchart , Sakari Ailus , linux-media@vger.kernel.org Cc: Tomasz Figa , Nicolas Dufresne , kernel@collabora.com, Paul Kocialkowski , Ezequiel Garcia , Jonas Karlman , Jernej Skrabec , Alexandre Courbot , Thierry Reding , Boris Brezillon Subject: [PATCH RFC 6/6] media: cedrus: Add the H264_DECODING_MODE control Date: Mon, 3 Jun 2019 13:09:46 +0200 Message-Id: <20190603110946.4952-7-boris.brezillon@collabora.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190603110946.4952-1-boris.brezillon@collabora.com> References: <20190603110946.4952-1-boris.brezillon@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The H264 uAPI has been modified to expose 2 operation modes: per slice and per frame decoding. The cedrus driver only supports per-slice decoding for now. Signed-off-by: Boris Brezillon --- drivers/staging/media/sunxi/cedrus/cedrus.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c index 3661c6a04864..fe6e6a12409b 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c @@ -76,6 +76,19 @@ static const struct cedrus_control cedrus_controls[] = { .codec = CEDRUS_CODEC_H264, .required = true, }, + { + .cfg.id = V4L2_CID_MPEG_VIDEO_H264_DECODING_MODE, + .cfg.type = V4L2_CTRL_TYPE_MENU, + /* + * Only per-slice decoding is supported for now. Note that we + * don't need to extract this control value since only one + * value is allowed: V4L2_MPEG_VIDEO_H264_DECODING_PER_SLICE. + */ + .cfg.max = V4L2_MPEG_VIDEO_H264_DECODING_PER_SLICE, + .cfg.def = V4L2_MPEG_VIDEO_H264_DECODING_PER_SLICE, + .codec = CEDRUS_CODEC_H264, + .required = true, + }, }; #define CEDRUS_CONTROLS_COUNT ARRAY_SIZE(cedrus_controls)