From patchwork Fri Feb 11 14:27:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Stein X-Patchwork-Id: 12743493 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 66E9EC433EF for ; Fri, 11 Feb 2022 14:29:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=chSMdeEnl2WtCU//dXTT8+QG/r/jiYQ4CJV3QyJyRn0=; b=0044jsXt6NxBu1 OcCf9lKyNsBzU0/EOMTAYBR8WRd0ORtn9PKFi0XpUUDNI9dOkoUuR5GRJ1bXXKiuF7CTsEa/+o3PG RIgdUYM5Fa/abPOZK2Oo7BRZF7xAe0hrjOSUTjuKJjEWSR8Ey7EkYIlSUSXhT6qph+zq9xk2iPbaE JZwxt2YyVjb6sxAOkDTGB2VeNK7VhuElQdCnBelmB4d7PPOAbIU2EO6YNdI8qZk1FzfpioUv+ZaNt t36Y9JHtMmymfTrxvtliLfipjpgYlXY6JEu6dgXYaf1QCB16iBj4sdbi1ajfqz5ilf9liYBMlK0Zw Egf0+BsXmTax76ss2pjg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nIWuA-007YOZ-75; Fri, 11 Feb 2022 14:28:10 +0000 Received: from mx1.tq-group.com ([93.104.207.81]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nIWu6-007YNE-2u for linux-arm-kernel@lists.infradead.org; Fri, 11 Feb 2022 14:28:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tq-group.com; i=@tq-group.com; q=dns/txt; s=key1; t=1644589686; x=1676125686; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=MKDoPVkuHIq+33Y64fqol9qvZyUA4rLWQUVSaAFy+xE=; b=cDmdfULfDbflXL1OqIehLxibQ63Gk7dcoadKTmdFXmDKGGZ6v2fEcens fZTCXKxWG0ICAEnTuj5P4jA9Bn6zJgrPDSoIbs587UOKtPeSm1/A63JZ2 0Km8wOdgqi/5kv0beaN4kAcaN/V6qT6G5h6vyiJvQ64y3mC3vEGZ3tIQW hasiZK9s/4caXO4OxjBNpG0/U4cC1ahCmalEoO73zpaIhYL9dlP96Ncd8 quNhtO8RzLJE9kkUk8wznJQuYvdthFJ80pAk2x76NGF1rWaeiNDY5RNNA CjQ8zYmDjfyiRI+GClXxVAI2Sc62QrY4QUPC0gimB0iW3J4potiEpDtYD g==; X-IronPort-AV: E=Sophos;i="5.88,361,1635199200"; d="scan'208";a="22042278" Received: from unknown (HELO tq-pgp-pr1.tq-net.de) ([192.168.6.15]) by mx1-pgp.tq-group.com with ESMTP; 11 Feb 2022 15:28:00 +0100 Received: from mx1.tq-group.com ([192.168.6.7]) by tq-pgp-pr1.tq-net.de (PGP Universal service); Fri, 11 Feb 2022 15:28:01 +0100 X-PGP-Universal: processed; by tq-pgp-pr1.tq-net.de on Fri, 11 Feb 2022 15:28:01 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tq-group.com; i=@tq-group.com; q=dns/txt; s=key1; t=1644589680; x=1676125680; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=MKDoPVkuHIq+33Y64fqol9qvZyUA4rLWQUVSaAFy+xE=; b=j5i1aRLqHFqcv+Blo3WMTwucR7ouBBn58aYXhsbzkLhQsi//OSbYFGQa Yfb9ZCPAvrv/ey+kje9tqXrA56Bn/xaMyomb/lqFZ5ZekmOviVVEmVN8J zPDX+dfg303jXiY5yvJd/0aiocJn5f2tt4BhR/ehCFBYBgqtSzhpJ3vqE 5toA44umYFt+30z+3AeZFg8ll/63GaHRG760OKkGt3W12uYd9MEqnIp4M m6A4+ldO7oeyYsTeQtWvrW5t0egkEGePnfijIKdE/NJf0QPxY482eyhGs mI/z1e3iukYwyEcfDbz1qkpCrq1Z/oUWia4/S0SWxzs+A6NDxdlItjZ8K g==; X-IronPort-AV: E=Sophos;i="5.88,361,1635199200"; d="scan'208";a="22042276" Received: from vtuxmail01.tq-net.de ([10.115.0.20]) by mx1.tq-group.com with ESMTP; 11 Feb 2022 15:28:00 +0100 Received: from steina-w.tq-net.de (unknown [10.123.49.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by vtuxmail01.tq-net.de (Postfix) with ESMTPSA id 32BC9280065; Fri, 11 Feb 2022 15:28:00 +0100 (CET) From: Alexander Stein To: Steve Longerbeam , Philipp Zabel , Mauro Carvalho Chehab , Greg Kroah-Hartman , Shawn Guo , Sascha Hauer , Pengutronix Kernel Team , Fabio Estevam , NXP Linux Team , Rui Miguel Silva , Laurent Pinchart , Dorota Czaplejewicz Cc: Alexander Stein , linux-media@vger.kernel.org, linux-staging@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 0/9] imx7/imx8mm media / csi patches Date: Fri, 11 Feb 2022 15:27:43 +0100 Message-Id: <20220211142752.779952-1-alexander.stein@ew.tq-group.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220211_062806_525357_86CD7F3B X-CRM114-Status: GOOD ( 18.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hey everyone, thanks for the feedback in v1. Changes in v2: * Switch back from bool to enum for distinguishing imx5/6 and imx7/8 * Added missing colorspace settings in PATCH 5 * Added PATCH 8 for hardening media-imx code Below is theoriginal message slightly modified to match the v2 changes. This is a set of patch for imx7 media drivers based on next-20220210. With this set I was able to capture video frames from a MIPI CSI-2 camera in my TQMa8MQML + MBA8MX hardware. The actual camera used is a Vision Components 'VC MIPI IMX327 C' camera. IMX327 is compatible to IMX290. Patch 9 shows the DT overlay I'm using, not suitable for merging. Please ignore the FPGA part, this is mainly for power supply and GPIO reset line. This is currently a custom driver I'm working on, but I do not want to focus on in this series. Please note I tested this only on this imx8 platform. First thanks to Dorota for the patchset at [1] (patches 1-4) which is necessary to capture correct images. Starting from patch 5 there are small fixes which allows me to configure my media device. Device configuration: ``` media-ctl -l "'imx290 2-001a':0->'csis-32e30000.mipi-csi':0 [1]" media-ctl -V "'imx290 2-001a':0 [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw]" media-ctl -V "'csi':0 [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw]" v4l2-ctl -d0 --set-fmt-video width=1920,height=1080,pixelformat='RG10',field=none media-ctl -p ``` The media-ctl topology is: ``` # media-ctl -p Media controller API version 5.17.0 Media device information ------------------------ driver imx7-csi model imx-media serial bus info platform:32e20000.csi hw revision 0x0 driver version 5.17.0 Device topology - entity 1: csi (2 pads, 2 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev0 pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range] <- "csis-32e30000.mipi-csi":1 [ENABLED,IMMUTABLE] pad1: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range] -> "csi capture":0 [ENABLED,IMMUTABLE] - entity 4: csi capture (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video0 pad0: Sink <- "csi":1 [ENABLED,IMMUTABLE] - entity 10: csis-32e30000.mipi-csi (2 pads, 2 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev1 pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:709 ycbcr:601 quantization:lim-range] <- "imx290 2-001a":0 [ENABLED] pad1: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:709 ycbcr:601 quantization:lim-range] -> "csi":0 [ENABLED,IMMUTABLE] - entity 15: imx290 2-001a (1 pad, 1 link) type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev2 pad0: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw] -> "csis-32e30000.mipi-csi":0 [ENABLED] ``` Note: MIPI CSI settle times are not calculated correctly right now and need a manual overwrite: echo 13 > /sys/kernel/debug/32e30000.mipi-csi/ths_settle echo 2 > /sys/kernel/debug/32e30000.mipi-csi/tclk_settle I ignored the settings for xfer, ycbcr and quantization for now. I do neither know how they will affect me nor what it should be. Also I did not focus on v4l2-compliance tool, this is a further task as well. IMHO imx7-mipi-csis.c should also create an immutable link to the camera sensor. Regards, Alexander [1] https://patchwork.linuxtv.org/project/linux-media/patch/20211104113631.206899-2-dorota.czaplejewicz@puri.sm/ Alexander Stein (5): media: imx: imx7_mipi_csis: store colorspace in set_fmt as well media: imx: imx7_media-csi: Add support for additional Bayer patterns media: imx: utils: Add more Bayer formats media: imx: utils: initialize local variable [DNI] arm64: dts: tqma8mqml: add IMX327 MIPI-CSI overlay Dorota Czaplejewicz (4): media: imx: Store the type of hardware implementation media: imx: Forward type of hardware implementation media: imx: Use dedicated format handler for i.MX7/8 media: imx: Fail conversion if pixel format not supported arch/arm64/boot/dts/freescale/Makefile | 4 + .../imx8mm-tqma8mqml-mba8mx-imx327.dts | 95 ++++++++++++ drivers/staging/media/imx/imx-ic-prpencvf.c | 3 +- drivers/staging/media/imx/imx-media-capture.c | 20 ++- drivers/staging/media/imx/imx-media-csi.c | 3 +- drivers/staging/media/imx/imx-media-utils.c | 136 +++++++++++++++++- drivers/staging/media/imx/imx-media.h | 11 +- drivers/staging/media/imx/imx7-media-csi.c | 15 +- drivers/staging/media/imx/imx7-mipi-csis.c | 4 + 9 files changed, 275 insertions(+), 16 deletions(-) create mode 100644 arch/arm64/boot/dts/freescale/imx8mm-tqma8mqml-mba8mx-imx327.dts