From patchwork Wed Feb 14 14:19:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 13556594 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8E4EDC48BC1 for ; Wed, 14 Feb 2024 14:19:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xHtARYcdAq3+1IgiP1bKKZB3SlYtiDv1V9Rku30mQ2k=; b=2ryzMUWbPYOhN3 IBW27IDE9genYCFG0DDeFCIsqtEuKfw3nhUBZb3N4N4Z1b/VrAqmHAc9uUVUxwNv5w9UkS5RkMRar ejoPhfPH1HY1fUTUUiGwKm3zhPjeyPAtj4zDW5NwT2w1dLr31PkB2Ud2j7TiGp+Mb3YkVAvUC2K+7 My2jJmi+NTMKLdsuWW6SWF1+bQCvsflUb/SxanEUqnwv5RPGzo/uswjQSbugD8vMA9Feut0AuOt09 RZwmF8gmQjZn7OTBA0s3pIqqpDmB1NcotrHAzdA/OsvN27+IT8AwO3k9uBYGqdHLNxayLTiHlYb7z ZQL9kOseA1C5qUO1qroQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1raG6t-0000000D8wx-12uH; Wed, 14 Feb 2024 14:19:39 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1raG6o-0000000D8s5-2gkc for linux-arm-kernel@lists.infradead.org; Wed, 14 Feb 2024 14:19:36 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F0797673; Wed, 14 Feb 2024 15:19:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1707920365; bh=sqiorC07iNCnAOOr+9nkt4xVwJBGRC9DMZNV8p/gebA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jYGm01eWsXLr1N8KtnS7q5ilFrfLFTjWXT0DDSAtPs5P9gL4vLhYiCcrM9GWdfkO+ 0NdjoKzcTRCBJz7xFjrxSTuCejCjX09/LZEwp9MemNEQzHz5o/DraL2gxgm1OOrATo EuKraLCEUxI4JBIaUC8qBiaptLPYclJGX5ol3lVQ= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, Daniel Scally Subject: [PATCH v2 1/5] media: uapi: Add MEDIA_BUS_FMT_RGB202020_1X60 format code Date: Wed, 14 Feb 2024 14:19:02 +0000 Message-Id: <20240214141906.245685-2-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240214141906.245685-1-dan.scally@ideasonboard.com> References: <20240214141906.245685-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240214_061934_871920_A9371E5B X-CRM114-Status: GOOD ( 10.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The Mali-C55 ISP by ARM requires 20-bits per colour channel input on the bus. Add a new media bus format code to represent it. Acked-by: Nayden Kanchev Signed-off-by: Daniel Scally --- Changes in v2: - none .../media/v4l/subdev-formats.rst | 168 ++++++++++++++++++ include/uapi/linux/media-bus-format.h | 3 +- 2 files changed, 170 insertions(+), 1 deletion(-) diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst index eb3cd20b0cf2..269b61926855 100644 --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst @@ -2223,6 +2223,174 @@ The following table list existing packed 48bit wide RGB formats. \endgroup +The following table list existing packed 60bit wide RGB formats. + +.. tabularcolumns:: |p{4.0cm}|p{0.7cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}|p{0.22cm}| + +.. _v4l2-mbus-pixelcode-rgb-60: + +.. raw:: latex + + \begingroup + \tiny + \setlength{\tabcolsep}{2pt} + +.. flat-table:: 60bit RGB formats + :header-rows: 3 + :stub-columns: 0 + :widths: 36 7 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 + + * - Identifier + - Code + - + - :cspan:`31` Data organization + * - + - + - Bit + - + - + - + - + - 59 + - 58 + - 57 + - 56 + - 55 + - 54 + - 53 + - 52 + - 51 + - 50 + - 49 + - 48 + - 47 + - 46 + - 45 + - 44 + - 43 + - 42 + - 41 + - 40 + - 39 + - 38 + - 37 + - 36 + - 35 + - 34 + - 33 + - 32 + * - + - + - + - 31 + - 30 + - 29 + - 28 + - 27 + - 26 + - 25 + - 24 + - 23 + - 22 + - 21 + - 20 + - 19 + - 18 + - 17 + - 16 + - 15 + - 14 + - 13 + - 12 + - 11 + - 10 + - 9 + - 8 + - 7 + - 6 + - 5 + - 4 + - 3 + - 2 + - 1 + - 0 + * .. _MEDIA-BUS-FMT-RGB202020-1X60: + + - MEDIA_BUS_FMT_RGB202020_1X60 + - 0x1026 + - + - + - + - + - + - r\ :sub:`19` + - r\ :sub:`18` + - r\ :sub:`17` + - r\ :sub:`16` + - r\ :sub:`15` + - r\ :sub:`14` + - r\ :sub:`13` + - r\ :sub:`12` + - r\ :sub:`11` + - r\ :sub:`10` + - r\ :sub:`8` + - r\ :sub:`8` + - r\ :sub:`7` + - r\ :sub:`6` + - r\ :sub:`5` + - r\ :sub:`4` + - r\ :sub:`3` + - r\ :sub:`2` + - r\ :sub:`1` + - r\ :sub:`0` + - g\ :sub:`19` + - g\ :sub:`18` + - g\ :sub:`17` + - g\ :sub:`16` + - g\ :sub:`15` + - g\ :sub:`14` + - g\ :sub:`13` + - g\ :sub:`12` + * - + - + - + - g\ :sub:`11` + - g\ :sub:`10` + - g\ :sub:`9` + - g\ :sub:`8` + - g\ :sub:`7` + - g\ :sub:`6` + - g\ :sub:`5` + - g\ :sub:`4` + - g\ :sub:`3` + - g\ :sub:`2` + - g\ :sub:`1` + - g\ :sub:`0` + - b\ :sub:`19` + - b\ :sub:`18` + - b\ :sub:`17` + - b\ :sub:`16` + - b\ :sub:`15` + - b\ :sub:`14` + - b\ :sub:`13` + - b\ :sub:`12` + - b\ :sub:`11` + - b\ :sub:`10` + - b\ :sub:`9` + - b\ :sub:`8` + - b\ :sub:`7` + - b\ :sub:`6` + - b\ :sub:`5` + - b\ :sub:`4` + - b\ :sub:`3` + - b\ :sub:`2` + - b\ :sub:`1` + - b\ :sub:`0` + +.. raw:: latex + + \endgroup + On LVDS buses, usually each sample is transferred serialized in seven time slots per pixel clock, on three (18-bit) or four (24-bit) differential data pairs at the same time. The remaining bits are used diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h index f05f747e444d..8633818cc33c 100644 --- a/include/uapi/linux/media-bus-format.h +++ b/include/uapi/linux/media-bus-format.h @@ -34,7 +34,7 @@ #define MEDIA_BUS_FMT_FIXED 0x0001 -/* RGB - next is 0x1026 */ +/* RGB - next is 0x1027 */ #define MEDIA_BUS_FMT_RGB444_1X12 0x1016 #define MEDIA_BUS_FMT_RGB444_2X8_PADHI_BE 0x1001 #define MEDIA_BUS_FMT_RGB444_2X8_PADHI_LE 0x1002 @@ -72,6 +72,7 @@ #define MEDIA_BUS_FMT_RGB888_1X36_CPADLO 0x1021 #define MEDIA_BUS_FMT_RGB121212_1X36 0x1019 #define MEDIA_BUS_FMT_RGB161616_1X48 0x101a +#define MEDIA_BUS_FMT_RGB202020_1X60 0x1026 /* YUV (including grey) - next is 0x202f */ #define MEDIA_BUS_FMT_Y8_1X8 0x2001 From patchwork Wed Feb 14 14:19:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 13556593 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8113DC48BC1 for ; Wed, 14 Feb 2024 14:19:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=WjH5ualkz/xiaSlrY3UIxyy/vY6CwiXC9vrBmdrlxzM=; b=BWkbNs+PvnEklC QT4PqcIXbylzLKPk2EbFSx8+Nv/1sjvXzkUKwGNN6HzlX3tW6MBbhe3To/KdK706ydFG+TKSM5HLt xRCYmMhm3hIM5GI/+ZsvgqVi7ICRShC0O9dIj8wN4Df/lAaiJooaAdD9daWTa+vzkk3W5aoEvpQM9 r3y4W46j9P5VQXLSWDLgC7MVsM1UdZEvD62B7QJHbm950zv2GFf4IHvQtennGh074gSEk2LvlykJL O2pZiKfHHU6JDoEBwPnMmTYhjO0xRgf/bz9TcHUl31NVTH6xpW9zijZPo0hcJfu0lKUTbnmNp/hMw wlm1UBvlT2NB8WaemlNw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1raG6r-0000000D8w4-46W8; Wed, 14 Feb 2024 14:19:37 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1raG6o-0000000D8s8-2Yly for linux-arm-kernel@lists.infradead.org; Wed, 14 Feb 2024 14:19:36 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A46F6CD1; Wed, 14 Feb 2024 15:19:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1707920366; bh=8kXr1VzKI8Qi87mLB25VO8hgs/KMxjE9hzUkS/GEHDo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=C3TMBThMd0IwPji9JKG4nH/s33MjGx0GddTzj3PWUFp2oXV5ZAKb55FDZkzCK5T86 cLVB2c0kcYuH+CAMWLWNOn8fEfa9vH2nHoqteACDoPbZ9SZbsYW4dKyG4PQcdXkQTW Ea/qumibOLxPGMSPLViO8T/NnTlk/bkSjBgxIkbE= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, Daniel Scally Subject: [PATCH v2 2/5] dt-bindings: media: Add bindings for ARM mali-c55 Date: Wed, 14 Feb 2024 14:19:03 +0000 Message-Id: <20240214141906.245685-3-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240214141906.245685-1-dan.scally@ideasonboard.com> References: <20240214141906.245685-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240214_061934_821242_72DF4A2D X-CRM114-Status: GOOD ( 12.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add the yaml binding for ARM's Mali-C55 Image Signal Processor. Acked-by: Nayden Kanchev Signed-off-by: Daniel Scally --- Changes in v2: - Added clocks information - Fixed the warnings raised by Rob .../bindings/media/arm,mali-c55.yaml | 77 +++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/arm,mali-c55.yaml diff --git a/Documentation/devicetree/bindings/media/arm,mali-c55.yaml b/Documentation/devicetree/bindings/media/arm,mali-c55.yaml new file mode 100644 index 000000000000..30038cfec3a4 --- /dev/null +++ b/Documentation/devicetree/bindings/media/arm,mali-c55.yaml @@ -0,0 +1,77 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/media/arm,mali-c55.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: ARM Mali-C55 Image Signal Processor + +maintainers: + - Daniel Scally + - Jacopo Mondi + +properties: + compatible: + const: arm,mali-c55 + + reg: + maxItems: 1 + + interrupts: + maxItems: 1 + + clocks: + items: + - description: ISP video clock + - description: ISP AXI clock + - description: ISP AHB-lite clock + + clock-names: + items: + - const: vclk + - const: aclk + - const: hclk + + ports: + $ref: /schemas/graph.yaml#/properties/ports + + properties: + port@0: + $ref: /schemas/graph.yaml#/properties/port + + properties: + endpoint: + $ref: /schemas/graph.yaml#/properties/endpoint + +required: + - compatible + - reg + - interrupts + - clocks + - clock-names + - ports + +additionalProperties: false + +examples: + - | + mali_c55: isp@400000 { + compatible = "arm,mali-c55"; + reg = <0x400000 0x200000>; + clocks = <&clk 0>, <&clk 1>, <&clk 2>; + clock-names = "vclk", "aclk", "hclk"; + interrupts = <0>; + + ports { + #address-cells = <1>; + #size-cells = <0>; + + port@0 { + reg = <0>; + isp_in: endpoint { + remote-endpoint = <&mipi_out>; + }; + }; + }; + }; +... From patchwork Wed Feb 14 14:19:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 13556597 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9CFB1C48BC3 for ; Wed, 14 Feb 2024 14:20:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=YS3TktmxWUYhwTVbIBPb2Folicccs2M07f+deVsa2N4=; b=pR8sMNPpm9hJ0G 3KFOw81BQoFLSZMyy71XqXC/jkOotY/uZcw08ey61mXlXheCTQ9zkV/Ah2VahtF2k/Gk32PhfIkGD NfetrbIULHFmbaZnFlNHn55n69TJqp1YqmPXbAoUT69PNU2Y6Zz+2WT14gl6eo5pfeF/5p8KaoTEc df3Lou0/QfzeQZNWQkgK0i678mFCDRYnvgaRR1/fHpaLQuK5SLez2jWJJGTVAjKgf0vNF00N/8nvw z+jmIVYZUth19VtmqzkJ0jPR6NXgsrbWfddVFDjviMfLCuHFw84P0pFnf/nzE6LHusAQqOn5RDGW2 shz3RBoSIwVdRH2bt4Kg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1raG73-0000000D92d-3K5L; Wed, 14 Feb 2024 14:19:49 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1raG6p-0000000D8sS-0JMX for linux-arm-kernel@lists.infradead.org; Wed, 14 Feb 2024 14:19:37 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 28CCCFA6; Wed, 14 Feb 2024 15:19:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1707920367; bh=yKDAbJPSf6bX6f3GS+eu+37ZxEXfUtZWeHHBBhMDcBY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=m0ve/P1b5QitoJLkpHqLjRm6NXxdqSHOaO3jMZUiZLJ9/6XjXf7mS2nZMzw+Z03sQ lyLHsAnRCIuC3YjkZDUzBmi+sp7D3Lb9ZUYHRlrexm1Qm0mLE0yPG5shT6VgWdE5uC 8UdTccKVHoEj2a2hMxiQsWg3egZzP3Cb5lPhuD30= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, Daniel Scally Subject: [PATCH v2 4/5] media: Documentation: Add Mali-C55 ISP Documentation Date: Wed, 14 Feb 2024 14:19:05 +0000 Message-Id: <20240214141906.245685-5-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240214141906.245685-1-dan.scally@ideasonboard.com> References: <20240214141906.245685-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240214_061935_471553_46F1AD8F X-CRM114-Status: GOOD ( 32.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a documentation page for the mali-c55 driver, which gives a brief overview of the hardware and explains how to use the driver's capture devices and the crop/scaler functions. Acked-by: Nayden Kanchev Signed-off-by: Daniel Scally --- Changes in v2: - none .../admin-guide/media/mali-c55-graph.dot | 19 ++ Documentation/admin-guide/media/mali-c55.rst | 318 ++++++++++++++++++ .../admin-guide/media/v4l-drivers.rst | 1 + 3 files changed, 338 insertions(+) create mode 100644 Documentation/admin-guide/media/mali-c55-graph.dot create mode 100644 Documentation/admin-guide/media/mali-c55.rst diff --git a/Documentation/admin-guide/media/mali-c55-graph.dot b/Documentation/admin-guide/media/mali-c55-graph.dot new file mode 100644 index 000000000000..0775ba42bf4c --- /dev/null +++ b/Documentation/admin-guide/media/mali-c55-graph.dot @@ -0,0 +1,19 @@ +digraph board { + rankdir=TB + n00000001 [label="{{} | mali-c55 tpg\n/dev/v4l-subdev0 | { 0}}", shape=Mrecord, style=filled, fillcolor=green] + n00000001:port0 -> n00000003:port0 [style=dashed] + n00000003 [label="{{ 0} | mali-c55 isp\n/dev/v4l-subdev1 | { 1 | 2}}", shape=Mrecord, style=filled, fillcolor=green] + n00000003:port1 -> n00000007:port0 [style=bold] + n00000003:port2 -> n00000007:port2 [style=bold] + n00000003:port1 -> n0000000b:port0 [style=bold] + n00000007 [label="{{ 0 | 2} | mali-c55 resizer fr\n/dev/v4l-subdev2 | { 1}}", shape=Mrecord, style=filled, fillcolor=green] + n00000007:port1 -> n0000000e [style=bold] + n0000000b [label="{{ 0} | mali-c55 resizer ds\n/dev/v4l-subdev3 | { 1}}", shape=Mrecord, style=filled, fillcolor=green] + n0000000b:port1 -> n00000012 [style=bold] + n0000000e [label="mali-c55 fr\n/dev/video0", shape=box, style=filled, fillcolor=yellow] + n00000012 [label="mali-c55 ds\n/dev/video1", shape=box, style=filled, fillcolor=yellow] + n00000022 [label="{{ 0} | csi2-rx\n/dev/v4l-subdev4 | { 1}}", shape=Mrecord, style=filled, fillcolor=green] + n00000022:port1 -> n00000003:port0 + n00000027 [label="{{} | imx415 1-001a\n/dev/v4l-subdev5 | { 0}}", shape=Mrecord, style=filled, fillcolor=green] + n00000027:port0 -> n00000022:port0 [style=bold] +} \ No newline at end of file diff --git a/Documentation/admin-guide/media/mali-c55.rst b/Documentation/admin-guide/media/mali-c55.rst new file mode 100644 index 000000000000..83f630c3bd9d --- /dev/null +++ b/Documentation/admin-guide/media/mali-c55.rst @@ -0,0 +1,318 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================================== +ARM Mali-C55 Image Signal Processor driver +========================================== + +Introduction +============ + +This file documents the driver for ARM's Mali-C55 Image Signal Processor. The +driver is located under drivers/media/platform/arm/mali-c55. + +The Mali-C55 ISP receives data in either raw Bayer format or RGB/YUV format from +sensors through either a parallel interface or a memory bus before processing it +and outputting it through an internal DMA engine. Two output pipelines are +possible (though one may not be fitted, depending on the implementation). These +are referred to as "Full resolution" and "Downscale", but the naming is historic +and both pipes are capable of cropping/scaling operations. An integrated test +pattern generator can be used to drive the ISP and produce image data in the +absence of a connected camera sensor. The driver module is named mali_c55, and +is enabled through the CONFIG_VIDEO_MALI_C55 config option. + +The driver implements V4L2, Media Controller and V4L2 Subdevice interfaces and +expects camera sensors connected to the ISP to have V4L2 subdevice interfaces. + +Mali-C55 ISP hardware +===================== + +A high level functional view of the Mali-C55 ISP is presented below. The ISP +takes input from either a live source or through a DMA engine for memory input, +depending on the SoC integration.:: + + +---------+ +----------+ +--------+ + | Sensor |--->| CSI-2 Rx | "Full Resolution" | DMA | + +---------+ +----------+ |\ Output +--->| Writer | + | | \ | +--------+ + | | \ +----------+ +------+---> Streaming I/O + +------------+ +------->| | | | | + | | | |-->| Mali-C55 |--+ + | DMA Reader |--------------->| | | ISP | | + | | | / | | | +---> Streaming I/O + +------------+ | / +----------+ | | + |/ +------+ + | +--------+ + +--->| DMA | + "Downscaled" | Writer | + Output +--------+ + +Media Controller Topology +========================= + +An example of the ISP's topology (as implemented in a system with an IMX415 +camera sensor and generic CSI-2 receiver) is below: + + +.. kernel-figure:: mali-c55-graph.dot + :alt: mali-c55-graph.dot + :align: center + +The driver has 4 V4L2 subdevices: + +- `mali_c55 isp`: Responsible for configuring input crop and color space + conversion +- `mali_c55 tpg`: The test pattern generator, emulating a camera sensor. +- `mali_c55 resizer fr`: The Full-Resolution pipe resizer +- `mali_c55 resizer ds`: The Downscale pipe resizer + +The driver has 2 V4L2 video devices: + +- `mali-c55 fr`: The full-resolution pipe's capture device +- `mali-c55 ds`: The downscale pipe's capture device + +Idiosyncrasies +-------------- + +**mali-c55 isp** +The `mali-c55 isp` subdevice has a single sink pad to which all sources of data +should be connected. The active source is selected by enabling the appropriate +media link and disabling all others. The ISP has two source pads, reflecting the +different paths through which it can internally route data. Tap points within +the ISP allow users to divert data to avoid processing by some or all of the +hardware's processing steps. The diagram below is intended only to highlight how +the bypassing works and is not a true reflection of those processing steps; for +a high-level functional block diagram see ARM's developer page for the +ISP [3]_:: + + +--------------------------------------------------------------+ + | Possible Internal ISP Data Routes | + | +------------+ +----------+ +------------+ | + +---+ | | | | | Colour | +---+ + | 0 |--+-->| Processing |->| Demosaic |->| Space |--->| 1 | + +---+ | | | | | | Conversion | +---+ + | | +------------+ +----------+ +------------+ | + | | +---+ + | +---------------------------------------------------| 2 | + | +---+ + | | + +--------------------------------------------------------------+ + + +.. flat-table:: + :header-rows: 1 + + * - Pad + - Direction + - Purpose + + * - 0 + - sink + - Data input, connected to the TPG and camera sensors + + * - 1 + - source + - RGB/YUV data, connected to the FR and DS V4L2 subdevices + + * - 2 + - source + - RAW bayer data, connected to the FR V4L2 subdevices + +**mali-c55 resizer fr** +The `mali-c55 resizer fr` subdevice has two _sink_ pads to reflect the different +insertion points in the hardware (either RAW or demosaiced data): + +.. flat-table:: + :header-rows: 1 + + * - Pad + - Direction + - Purpose + + * - 0 + - sink + - Data input connected to the ISP's demosaiced stream. + + * - 1 + - source + - Data output connected to the capture video device + + * - 2 + - sink + - Data input connected to the ISP's raw data stream + +The data source in use is selected through the routing API; two routes each of a +single stream are available: + +.. flat-table:: + :header-rows: 1 + + * - Sink Pad + - Source Pad + - Purpose + + * - 0 + - 1 + - Demosaiced data route + + * - 2 + - 1 + - Raw data route + + +If the demosaiced route is active then the FR pipe is only capable of output +in RGB/YUV formats. If the raw route is active then the output reflects the +input (which may be either Bayer or RGB/YUV data). + +Using the driver to capture video +================================= + +Using the media controller APIs we can configure the input source and ISP to +capture images in a variety of formats. In the examples below, configuring the +media graph is done with the v4l-utils [1]_ package's media-ctl utility. +Capturing the images is done with yavta [2]_. + +Configuring the input source +---------------------------- + +The first step is to set the input source that we wish by enabling the correct +media link. Using the example topology above, we can select the TPG as follows: + +.. code-block:: none + + media-ctl -l "'lte-csi2-rx':1->'mali-c55 isp':0[0]" + media-ctl -l "'mali-c55 tpg':0->'mali-c55 isp':0[1]" + +Capturing bayer data from the source and processing to RGB/YUV +-------------------------------------------------------------- +To capture 1920x1080 bayer data from the source and push it through the ISP's +full processing pipeline, we configure the data formats appropriately on the +source, ISP and resizer subdevices and set the FR resizer's routing to select +processed data. The media bus format on the resizer's source pad will be either +RGB121212_1X36 or YUV10_1X30, depending on whether you want to capture RGB or +YUV. The ISP's debayering block outputs RGB data natively, setting the source +pad format to YUV10_1X30 enables the colour space conversion block. + +In this example we target RGB565 output, so select RGB121212_1X36 as the resizer +source pad's format: + +.. code-block:: none + + # Set formats on the TPG and ISP + media-ctl -V "'mali-c55 tpg':0[fmt:SRGGB16_1X16/1920x1080]" + media-ctl -V "'mali-c55 isp':0[fmt:SRGGB16_1X16/1920x1080]" + media-ctl -V "'mali-c55 isp':1[fmt:SRGGB16_1X16/1920x1080]" + + # Set routing on the FR resizer + media-ctl -R "'mali-c55 resizer fr'[0/0->1/0[1],2/0->1/0[0]]" + + # Set format on the resizer, must be done AFTER the routing. + media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/1920x1080]" + +The downscale output can also be used to stream data at the same time. In this +case only processed data can be captured and so no routing need be set: + +.. code-block:: none + + # Set format on the resizer + media-ctl -V "'mali-c55 resizer ds':1[fmt:RGB121212_1X36/1920x1080]" + +Following which images can be captured from both the FR and DS output's video +devices (simultaneously, if desired): + +.. code-block:: none + + yavta -f RGB565 -s 1920x1080 -c10 /dev/video0 + yavta -f RGB565 -s 1920x1080 -c10 /dev/video1 + +Cropping the image +~~~~~~~~~~~~~~~~~~ + +Both the full resolution and downscale pipes can crop to a minimum resolution of +640x480. To crop the image simply configure the resizer's sink pad's crop and +compose rectangles and set the format on the video device: + +.. code-block:: none + + media-ctl -V "'mali-c55 resizer fr':0[fmt:RGB121212_1X36/1920x1080 crop:(480,270)/640x480 compose:(0,0)/640x480]" + media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/640x480]" + yavta -f RGB565 -s 640x480 -c10 /dev/video0 + +Downscaling the image +~~~~~~~~~~~~~~~~~~~~~ + +Both the full resolution and downscale pipes can downscale the image by up to 8x +provided the minimum 640x480 resolution is adhered to. For the best image result +the scaling ratio for each dimension should be the same. To configure scaling we +use the compose rectangle on the resizer's sink pad: + +.. code-block:: none + + media-ctl -V "'mali-c55 resizer fr':0[fmt:RGB121212_1X36/1920x1080 crop:(0,0)/1920x1080 compose:(0,0)/640x480]" + media-ctl -V "'mali-c55 resizer fr':1[fmt:RGB121212_1X36/640x480]" + yavta -f RGB565 -s 640x480 -c10 /dev/video0 + +Capturing images in YUV formats +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If we need to output YUV data rather than RGB the color space conversion block +needs to be active, which is achieved by setting MEDIA_BUS_FMT_YUV10_1X30 on the +resizer's source pad. We can then configure a capture format like NV12 (here in +its multi-planar variant) + +.. code-block:: none + + media-ctl -V "'mali-c55 resizer fr':1[fmt:YUV10_1X30/1920x1080]" + yavta -f NV12M -s 1920x1080 -c10 /dev/video0 + +Capturing RGB data from the source and processing it with the resizers +---------------------------------------------------------------------- + +The Mali-C55 ISP can work with sensors capable of outputting RGB data. In this +case although none of the image quality blocks would be used it can still +crop/scale the data in the usual way. + +To achieve this, the ISP's sink pad's format is set to +MEDIA_BUS_FMT_RGB202020_1X60 - this reflects the format that data must be in to +work with the ISP. Converting the camera sensor's output to that format is the +responsibility of external hardware. + +In this example we ask the test pattern generator to give us RGB data instead of +bayer. + +.. code-block:: none + + media-ctl -V "'mali-c55 tpg':0[fmt:RGB202020_1X60/1920x1080]" + media-ctl -V "'mali-c55 isp':0[fmt:RGB202020_1X60/1920x1080]" + +Cropping or scaling the data can be done in exactly the same way as outlined +earlier. + +Capturing raw data from the source and outputting it unmodified +----------------------------------------------------------------- + +The ISP can additionally capture raw data from the source and output it on the +full resolution pipe only, completely unmodified. In this case the downscale +pipe can still process the data normally and be used at the same time. + +To configure raw bypass the FR resizer's subdevice's routing table needs to be +configured, followed by formats in the appropriate places: + +.. code-block:: none + + # We need to configure the routing table for the resizer to use the bypass + # path along with set formats on the resizer's bypass sink pad. Doing this + # necessitates a single media-ctl command, as multiple calls to the program + # reset the routing table. + media-ctl -R "'mali-c55 resizer fr'[0/0->1/0[0],2/0->1/0[1]]"\ + -V "'mali-c55 isp':0[fmt:RGB202020_1X60/1920x1080],"\ + "'mali-c55 resizer fr':2[fmt:RGB202020_1X60/1920x1080],"\ + "'mali-c55 resizer fr':1[fmt:RGB202020_1X60/1920x1080]" + + # Set format on the video device and stream + yavta -f RGB565 -s 1920x1080 -c10 /dev/video0 + +References +========== +.. [1] https://git.linuxtv.org/v4l-utils.git/ +.. [2] https://git.ideasonboard.org/yavta.git +.. [3] https://developer.arm.com/Processors/Mali-C55 diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst index f4bb2605f07e..af033c892808 100644 --- a/Documentation/admin-guide/media/v4l-drivers.rst +++ b/Documentation/admin-guide/media/v4l-drivers.rst @@ -17,6 +17,7 @@ Video4Linux (V4L) driver-specific documentation imx7 ipu3 ivtv + mali-c55 mgb4 omap3isp omap4_camera From patchwork Wed Feb 14 14:19:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Scally X-Patchwork-Id: 13556596 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2FAAFC48BC3 for ; Wed, 14 Feb 2024 14:20:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=dxax3zeQ43ZIi5Lu+RvxUdIpvwqSc2ylmw82sPPE40A=; b=n2PwCFKVeCB82c O1sBaTImF6yyQMwezAVrdQkLQYz/957FhGf69CEPJJP3xv4GtLPsgF469iGHfIfCMNBvKtdQ86oFl kZ0LmAasZmAvs024nQkq/7UVaSENCe3EzV2CkYY0UC/eKjXM5V2QpJyhYWp80lMwYmk0hv4s6Z/+p 77W8v4lkipujX+JxWCrKAA67nXq1Du+kjFPBdSmKK4WAElzOo19TrEGEK2C5NifAvHfUSb+tFmILO j4azxPMULbheRH4OR5D/lAM6M0azTbcqueN80iuW6z5ZVO7OLO/tBxrSFwvd0+ylYlIA6U0oxsxBu UvEUfyqTZWzRtW5v+YtA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1raG74-0000000D93M-3Edt; Wed, 14 Feb 2024 14:19:50 +0000 Received: from perceval.ideasonboard.com ([213.167.242.64]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1raG6r-0000000D8uv-1uci for linux-arm-kernel@lists.infradead.org; Wed, 14 Feb 2024 14:19:38 +0000 Received: from mail.ideasonboard.com (cpc141996-chfd3-2-0-cust928.12-3.cable.virginm.net [86.13.91.161]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BC80215B5; Wed, 14 Feb 2024 15:19:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1707920368; bh=Sy4YAaBmxWzOZQ7mY2+7+54bBBpkqPLkoAUSPW/kIZ4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kfCgS4iWni5jJ4d8nRNVGy/H5hL84AljmPOkJ6Hn+dTLM/y9duxl4GAG9RCg6HWPd kelwqNJTZNk156IRfl8JlhTDzRxN1J9F7pfRKteBIEwV2oAeD9FebXLCuqamz/rI/P ATAWlRpE22n8Wkog6XqquTDf8H0alCzEGQVdA9oE= From: Daniel Scally To: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: jacopo.mondi@ideasonboard.com, nayden.kanchev@arm.com, robh+dt@kernel.org, mchehab@kernel.org, krzysztof.kozlowski+dt@linaro.org, conor+dt@kernel.org, jerome.forissier@linaro.org, kieran.bingham@ideasonboard.com, laurent.pinchart@ideasonboard.com, Daniel Scally Subject: [PATCH v2 5/5] MAINTAINERS: Add entry for mali-c55 driver Date: Wed, 14 Feb 2024 14:19:06 +0000 Message-Id: <20240214141906.245685-6-dan.scally@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240214141906.245685-1-dan.scally@ideasonboard.com> References: <20240214141906.245685-1-dan.scally@ideasonboard.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240214_061937_733652_E989BE15 X-CRM114-Status: UNSURE ( 9.44 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a MAINTAINERS entry for the mali-c55 driver and its associated documentation. Signed-off-by: Daniel Scally --- Changes in v2: - none MAINTAINERS | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 8d1052fa6a69..bec50bfb09ec 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1668,6 +1668,16 @@ F: Documentation/gpu/panfrost.rst F: drivers/gpu/drm/panfrost/ F: include/uapi/drm/panfrost_drm.h +ARM MALI-C55 ISP DRIVER +M: Daniel Scally +M: Jacopo Mondi +L: linux-media@vger.kernel.org +S: Maintained +T: git git://linuxtv.org/media_tree.git +F: Documentation/admin-guide/media/mali-c55.rst +F: Documentation/devicetree/bindings/media/arm,mali-c55.yaml +F: drivers/media/platform/arm/mali-c55/ + ARM MALI-DP DRM DRIVER M: Liviu Dudau S: Supported