From patchwork Tue Feb 5 20:24:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 10798365 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A0D6313B4 for ; Tue, 5 Feb 2019 20:24:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 91D832ADC1 for ; Tue, 5 Feb 2019 20:24:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 858742AEA3; Tue, 5 Feb 2019 20:24:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0D4B62ADC1 for ; Tue, 5 Feb 2019 20:24:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:To :From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=46CXUUMmttL/fD7G3KTBUIGyicrLKjL+oLdw8ZSB9Uw=; b=cglrhRvJt/w00o D2jhaErQcsL7t1rarzX8xN9O2NjU93SdDdo2xyqBENHVuXpH8IQ4avchOqURmpXti9rq5ABH48bT7 i4Twb7PpmlA6jqesWL2fKBex0uAT6AYkvhjBrsVVFJG3tUZ+UVG2kBFLXCjWPd6BXlrYFWl4qZHKT qbBy9lFsLHpABcbn18SpQVcqpcwpOM1MZBsMbOsT5lx7dHi4SWB1v3gIrlK4Ys9Es0wYeeAgw9i+k ho+tvLIbCvE5RtLh3sB4C0al+uGuePx6yuvLUshqh97IMyJMyiELp1x5jzPGdqZ5ieqvw9feH+lmz JRkDbIjLa/uSeWBQhQRQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1gr7Gy-00060w-V0; Tue, 05 Feb 2019 20:24:48 +0000 Received: from bhuna.collabora.co.uk ([46.235.227.227]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1gr7Gv-0005x2-Et for linux-rockchip@lists.infradead.org; Tue, 05 Feb 2019 20:24:47 +0000 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id 6AA6B28020E From: Ezequiel Garcia To: linux-media@vger.kernel.org Subject: [PATCH v1 00/10] Add MPEG-2 decoding to Rockchip VPU Date: Tue, 5 Feb 2019 17:24:07 -0300 Message-Id: <20190205202417.16555-1-ezequiel@collabora.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190205_122445_763766_C73A38AF X-CRM114-Status: GOOD ( 12.07 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Nicolas Dufresne , Heiko Stuebner , Jonas Karlman , Tomasz Figa , linux-rockchip@lists.infradead.org, Hans Verkuil , kernel@collabora.com, Ezequiel Garcia Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+patchwork-linux-rockchip=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP This series introduces the decoding infrastructure that will be used to add support for other codecs such as VP8, VP9 and H.264. There are a number of v4l2-compliance issues, which I will be addressing shortly. For those wanting to take a peep, I've pasted the v4l2-compliance issues here: http://ix.io/1AfJ. Minor issues aside, "release early" they say, so here it is! About the JPEG bounce buffer, we can probably get rid of it, but for now, it's fine to keep it as it won't affect video decoding. About the media controller topology change, which is no doubt the nastiest change in this series, it's important to mention that we need multiple video interface nodes: one video node for encoding, and one video node for decoding. Since the VPU is a single engine, it needs a single mem2mem device to serialize the codec jobs thru it, taking advantage of the whole mem2mem infrastructure. This works perfectly well, but requires a somewhat involved topology. For the encoder the graph goes like this: ┌────────────────────────────────┐ │ rockchip,rk3399-vpu-enc-source │ │ /dev/video0 │ └────────────────────────────────┘ ┃ ┃ ▼ ┌────────────────────────────────┐ │ rockchip,rk3399-vpu-enc-proc │ └────────────────────────────────┘ ┃ ┃ ▼ ┌────────────────────────────────┐ │ rockchip,rk3399-vpu-enc-sink │ │ /dev/video0 │ └────────────────────────────────┘ For the decoder the graph goes like this: ┌────────────────────────────────┐ │ rockchip,rk3399-vpu-dec-source │ │ /dev/video1 │ └────────────────────────────────┘ ┃ ┃ ▼ ┌────────────────────────────────┐ │ rockchip,rk3399-vpu-dec-proc │ └────────────────────────────────┘ ┃ ┃ ▼ ┌────────────────────────────────┐ │ rockchip,rk3399-vpu-dec-sink │ │ /dev/video1 │ └────────────────────────────────┘ Both are tied to the same media device, i.e. /dev/media0. Does this comply with the media controller specification? The pixel format helpers deserve a discussion of its own. Note that these helpers will be useful for probably most V4L drivers. See for instance, this recent bug caused by bad math in vivid's sizeimage calculations: https://patchwork.kernel.org/patch/10796201/ Finally, I have to thank Jonas Karlman, who did the initial MPEG-2 decoding work and also for getting mpv+ffmpeg to work using the Request API. This driver can be tested using mpv+ffmpeg for the video decoding side, and the Panfrost mesa driver for rendering. I should be posting instructions to set all of this up, and also will be submitting the support for H264, VP8 and VP9, hopefully very soon. Ezequiel Garcia (9): media: Introduce helpers to fill pixel format structs rockchip/vpu: Use pixel format helpers rockchip/vpu: Use v4l2_m2m_buf_copy_data rockchip/vpu: Cleanup macroblock alignment rockchip/vpu: Cleanup JPEG bounce buffer management rockchip/vpu: Open-code media controller register rockchip/vpu: Support the Request API rockchip/vpu: Add decoder boilerplate rockchip/vpu: Add support for non-standard controls Jonas Karlman (1): rockchip/vpu: Add support for MPEG-2 decoding drivers/media/v4l2-core/Makefile | 2 +- drivers/media/v4l2-core/v4l2-common.c | 71 +++ drivers/media/v4l2-core/v4l2-fourcc.c | 109 ++++ drivers/staging/media/rockchip/vpu/Makefile | 5 +- .../media/rockchip/vpu/rk3288_vpu_hw.c | 4 +- .../rockchip/vpu/rk3288_vpu_hw_jpeg_enc.c | 4 +- .../media/rockchip/vpu/rk3399_vpu_hw.c | 61 +- .../rockchip/vpu/rk3399_vpu_hw_jpeg_enc.c | 10 +- .../rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c | 263 +++++++++ .../staging/media/rockchip/vpu/rockchip_vpu.h | 115 +++- .../media/rockchip/vpu/rockchip_vpu_common.h | 10 + .../media/rockchip/vpu/rockchip_vpu_dec.c | 557 ++++++++++++++++++ .../media/rockchip/vpu/rockchip_vpu_drv.c | 426 ++++++++++++-- .../media/rockchip/vpu/rockchip_vpu_enc.c | 152 ++--- .../media/rockchip/vpu/rockchip_vpu_hw.h | 42 ++ .../media/rockchip/vpu/rockchip_vpu_jpeg.c | 25 + .../media/rockchip/vpu/rockchip_vpu_mpeg2.c | 61 ++ include/media/v4l2-common.h | 5 + include/media/v4l2-fourcc.h | 53 ++ 19 files changed, 1803 insertions(+), 172 deletions(-) create mode 100644 drivers/media/v4l2-core/v4l2-fourcc.c create mode 100644 drivers/staging/media/rockchip/vpu/rk3399_vpu_hw_mpeg2_dec.c create mode 100644 drivers/staging/media/rockchip/vpu/rockchip_vpu_dec.c create mode 100644 drivers/staging/media/rockchip/vpu/rockchip_vpu_mpeg2.c create mode 100644 include/media/v4l2-fourcc.h