From patchwork Thu Mar 7 10:03:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Randy Li X-Patchwork-Id: 10842541 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 61C111575 for ; Thu, 7 Mar 2019 10:03:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 275F128789 for ; Thu, 7 Mar 2019 10:03:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 17F052E199; Thu, 7 Mar 2019 10:03:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.7 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED,RCVD_IN_SORBS_WEB autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id AC1402E194 for ; Thu, 7 Mar 2019 10:03:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:To :From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=qOCdHpSkgQWdOESRXuezxE1hnnntHTOxB9yFNS0ulGk=; b=uvn5cJS6nv3HTv s/3C7VEr/biPFj/nGxl3nj1b+KuNQzJpfUXtUrWVGZgpHEJ3tEDCK+QnkEPqckQBTrUK+yTZAP5oZ mQr2vtg/VmhfG/JgGOimk+CMnt8FPENBQyMpT5KDHWIcsVb7ruq1LdIn1zR6MQdAUOMHxsPakeNdq I7gJm+IQX+4OVyH0xEUEnGTQZYQjDmAH5yrgOpVNanGdiQm+n8ZiPHkTlIGk9XRodpUz4YK0TwC90 XX2+XdAGh+A9Wd7IIrcnw7BmTNX+ho7/7k7psuxt5GOgTuHAURpJCv0tNurO3AK9W2pUS4t5gLWhF Msgpgl5CFUIBKp2NH6FA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1h1psV-0004FW-3y; Thu, 07 Mar 2019 10:03:51 +0000 Received: from regular1.263xmail.com ([211.150.99.136]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1h1psK-00046j-C0; Thu, 07 Mar 2019 10:03:43 +0000 Received: from randy.li?rock-chips.com (unknown [192.168.167.139]) by regular1.263xmail.com (Postfix) with ESMTP id 4166C48E; Thu, 7 Mar 2019 18:03:27 +0800 (CST) X-263anti-spam: KSV:0; X-MAIL-GRAY: 0 X-MAIL-DELIVERY: 1 X-KSVirus-check: 0 X-ABS-CHECKED: 4 Received: from randy-pc.lan (unknown [58.22.7.114]) by smtp.263.net (postfix) whith ESMTP id P17008T140071111993088S1551952999924990_; Thu, 07 Mar 2019 18:03:27 +0800 (CST) X-IP-DOMAINF: 1 X-UNIQUE-TAG: X-RL-SENDER: randy.li@rock-chips.com X-SENDER: randy.li@rock-chips.com X-LOGIN-NAME: randy.li@rock-chips.com X-FST-TO: linux-media@vger.kernel.org X-SENDER-IP: 58.22.7.114 X-ATTACHMENT-NUM: 0 X-DNS-TYPE: 0 From: Randy Li To: linux-media@vger.kernel.org Subject: [PATCH v2 0/6] [WIP]: rockchip mpp for v4l2 video deocder Date: Thu, 7 Mar 2019 18:03:10 +0800 Message-Id: <20190307100316.925-1-randy.li@rock-chips.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190307_020340_846288_AEF5C252 X-CRM114-Status: GOOD ( 13.36 ) X-BeenThere: linux-rockchip@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Upstream kernel work for Rockchip platforms List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: paul.kocialkowski@bootlin.com, maxime.ripard@bootlin.com, joro@8bytes.org, ayaka@soulik.info, Randy Li , linux-kernel@vger.kernel.org, jernej.skrabec@gmail.com, nicolas@ndufresne.ca, hverkuil@xs4all.nl, linux-rockchip@lists.infradead.org, posciak@chromium.org, thomas.petazzoni@bootlin.com, groeck@chromium.org, mchehab@kernel.org, ezequiel@collabora.com, linux-arm-kernel@lists.infradead.org Sender: "Linux-rockchip" Errors-To: linux-rockchip-bounces+patchwork-linux-rockchip=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Although I really want to push those work after I added more codec supports, but I found it is more urge to do those in v4l2 core framework and userspace. I would use this driver to present the current problems, write down a summary here and I would reply to those threads later to push forward. 1. Slice construction is a bad idea. I think I have said my reason in the IRC and mail before, vp9 is always good example. And it would request the driver to update QP table/CABAC table every slice. I would make something to describe a buffer with some addtional meta data. But the current request API limit a buffer with associated with an request buffer, which prevent sharing some sequence data, but it still can solve some problems. 2. Advantage DMA memory control. I think I need to do some work at v4l2 core. 2.1 The DMA address of each planes. I have sent a mail before talked about why multiple planes is necessary for the rockchip platform. And it maybe required by the other platforms. 2.2 IOMMU resume The most effective way to restore the decoder from critical error is doing a restting by reset controller. Which would leading its slave IOMMU reset at the same time. Then none of those v4l2 buffers are mapping in the IOMMU. 3. H.264 and HEVC header I still think those structure have some not necessary fileds in dpb or reference part, which I don't think hardware decoder would care about that or can be predict from the other information. I would join to talk later. 4. The work flow of V4L2 I need a method to prepare the register set before the device acutally begin the transaction. Which is necessary for those high frame rate usecase. Also it is useful for those device would share some hardware resources with the other device and it can save more power. I think I need to do some work at v4l2 core. Randy Li (6): arm64: dts: rockchip: add power domain to iommu staging: video: rockchip: add v4l2 decoder [TEST]: rockchip: mpp: support qptable staging: video: rockchip: add video codec arm64: dts: rockchip: boost clocks for rk3328 arm64: dts: rockchip: add video codec for rk3328 .../arm64/boot/dts/rockchip/rk3328-rock64.dts | 32 + arch/arm64/boot/dts/rockchip/rk3328.dtsi | 116 +- arch/arm64/boot/dts/rockchip/rk3399.dtsi | 2 + drivers/staging/Kconfig | 2 + drivers/staging/Makefile | 1 + drivers/staging/rockchip-mpp/Kconfig | 34 + drivers/staging/rockchip-mpp/Makefile | 10 + drivers/staging/rockchip-mpp/mpp_debug.h | 87 ++ drivers/staging/rockchip-mpp/mpp_dev_common.c | 1367 +++++++++++++++++ drivers/staging/rockchip-mpp/mpp_dev_common.h | 212 +++ drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c | 997 ++++++++++++ drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c | 619 ++++++++ drivers/staging/rockchip-mpp/mpp_service.c | 197 +++ drivers/staging/rockchip-mpp/mpp_service.h | 38 + drivers/staging/rockchip-mpp/rkvdec/hal.h | 63 + drivers/staging/rockchip-mpp/rkvdec/hevc.c | 166 ++ drivers/staging/rockchip-mpp/rkvdec/regs.h | 608 ++++++++ drivers/staging/rockchip-mpp/vdpu2/hal.h | 52 + drivers/staging/rockchip-mpp/vdpu2/mpeg2.c | 277 ++++ drivers/staging/rockchip-mpp/vdpu2/regs.h | 699 +++++++++ 20 files changed, 5575 insertions(+), 4 deletions(-) create mode 100644 drivers/staging/rockchip-mpp/Kconfig create mode 100644 drivers/staging/rockchip-mpp/Makefile create mode 100644 drivers/staging/rockchip-mpp/mpp_debug.h create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_common.c create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_common.h create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_rkvdec.c create mode 100644 drivers/staging/rockchip-mpp/mpp_dev_vdpu2.c create mode 100644 drivers/staging/rockchip-mpp/mpp_service.c create mode 100644 drivers/staging/rockchip-mpp/mpp_service.h create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hal.h create mode 100644 drivers/staging/rockchip-mpp/rkvdec/hevc.c create mode 100644 drivers/staging/rockchip-mpp/rkvdec/regs.h create mode 100644 drivers/staging/rockchip-mpp/vdpu2/hal.h create mode 100644 drivers/staging/rockchip-mpp/vdpu2/mpeg2.c create mode 100644 drivers/staging/rockchip-mpp/vdpu2/regs.h