From patchwork Thu Jan 11 06:55:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bingbu Cao X-Patchwork-Id: 13516822 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 294C35672 for ; Thu, 11 Jan 2024 06:47:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="CB7h3Dbc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704955626; x=1736491626; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9P1Nu6asS2B7HUvvY5KCXJwm6dJesUD9xHYIGYRExuM=; b=CB7h3DbcA61kvytFv/9NBrr5FKgTk6CBQINPPjpfKf2pJyJCqHFpUZtK J6aQUfzCo4RxrCNwoTSKCSSlanmMeNmLZT0vyjHEfbNhH5q2jMxpON968 4604PLGt9HwGrf73I/A79xUdg8vsGzD8JeGZFyz4ll4m6TmZQP4M9n0fm vC0EnK9DJ4kqhB8JcmkPVIOy+1W0PSfoWJhbpMFAPeWqM3KgOStQorjYF PpJKsFwMAl88LlB+JDiQo6RVu4LXLhPpuHdbLFHn1kF4TXAsIjU6MoVtu gxJkm0jax1xD7GPz4vO2NrAa8ycEHS7dra9OraFtFVwYsYFYjgIkh597Z A==; X-IronPort-AV: E=McAfee;i="6600,9927,10949"; a="397629216" X-IronPort-AV: E=Sophos;i="6.04,185,1695711600"; d="scan'208";a="397629216" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Jan 2024 22:47:05 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10949"; a="775515692" X-IronPort-AV: E=Sophos;i="6.04,185,1695711600"; d="scan'208";a="775515692" Received: from icg-kernel3.bj.intel.com ([172.16.126.107]) by orsmga007.jf.intel.com with ESMTP; 10 Jan 2024 22:47:01 -0800 From: bingbu.cao@intel.com To: linux-media@vger.kernel.org, sakari.ailus@linux.intel.com, laurent.pinchart@ideasonboard.com, andriy.shevchenko@linux.intel.com, hdegoede@redhat.com Cc: ilpo.jarvinen@linux.intel.com, claus.stovgaard@gmail.com, tomi.valkeinen@ideasonboard.com, tfiga@chromium.org, senozhatsky@chromium.org, andreaskleist@gmail.com, bingbu.cao@intel.com, bingbu.cao@linux.intel.com, tian.shu.qiu@intel.com, hongju.wang@intel.com Subject: [PATCH v3 14/17] Documentation: add Intel IPU6 ISYS driver admin-guide doc Date: Thu, 11 Jan 2024 14:55:28 +0800 Message-ID: <20240111065531.2418836-15-bingbu.cao@intel.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20240111065531.2418836-1-bingbu.cao@intel.com> References: <20240111065531.2418836-1-bingbu.cao@intel.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Bingbu Cao This document mainly describe the functionality of IPU6 and IPU6 isys driver, and gives an example that how user can do imaging capture with tools. Signed-off-by: Bingbu Cao --- Documentation/admin-guide/media/ipu6-isys.rst | 158 ++++++++++++++++ .../admin-guide/media/ipu6_isys_graph.svg | 174 ++++++++++++++++++ .../admin-guide/media/v4l-drivers.rst | 1 + 3 files changed, 333 insertions(+) create mode 100644 Documentation/admin-guide/media/ipu6-isys.rst create mode 100644 Documentation/admin-guide/media/ipu6_isys_graph.svg diff --git a/Documentation/admin-guide/media/ipu6-isys.rst b/Documentation/admin-guide/media/ipu6-isys.rst new file mode 100644 index 000000000000..5e78ab88c649 --- /dev/null +++ b/Documentation/admin-guide/media/ipu6-isys.rst @@ -0,0 +1,158 @@ +.. SPDX-License-Identifier: GPL-2.0 + +.. include:: + +======================================================== +Intel Image Processing Unit 6 (IPU6) Input System driver +======================================================== + +Copyright |copy| 2023 Intel Corporation + +Introduction +============ + +This file documents the Intel IPU6 (6th generation Image Processing Unit) +Input System (MIPI CSI2 receiver) drivers located under +drivers/media/pci/intel/ipu6. + +The Intel IPU6 can be found in certain Intel Chipsets but not in all SKUs: + +* TigerLake +* JasperLake +* AlderLake +* RaptorLake +* MeteorLake + +Intel IPU6 is made up of two components - Input System (ISYS) and Processing +System (PSYS). + +The Input System mainly works as MIPI CSI2 receiver which receives and +processes the imaging data from the sensors and outputs the frames to memory. + +There are 2 driver modules - intel_ipu6 and intel_ipu6_isys. intel_ipu6 is an +IPU6 common driver which does PCI configuration, firmware loading and parsing, +firmware authentication, DMA mapping and IPU-MMU (internal Memory mapping Unit) +configuration. intel_ipu6_isys implements V4L2, Media Controller and V4L2 +sub-device interfaces. The IPU6 ISYS driver supports camera sensors connected +to the IPU6 ISYS through V4L2 sub-device sensor drivers. + +.. Note:: See Documentation/driver-api/media/drivers/ipu6.rst for more + information about the IPU6 hardware. + + +Input system driver +=================== + +The input System driver mainly configures CSI2 DPHY, constructs the firmware +stream configuration, sends commands to firmware, gets response from hardware +and firmware and then returns buffers to user. +The ISYS is represented as several V4L2 sub-devices - 'Intel IPU6 CSI2 $port', +which provide V4L2 subdev interfaces to the user space, there are also several +video nodes for each CSI-2 stream capture - 'Intel IPU6 ISYS capture $num' which +provide interface to user to set formats, queue buffers and streaming. + +.. kernel-figure:: ipu6_isys_graph.svg + :alt: ipu6 isys media graph with multiple streams support + + ipu6 isys media graph with multiple streams support + +Capturing frames by IPU6 ISYS +----------------------------- + +IPU6 ISYS is used to capture frames from the camera sensors connected to the +CSI2 ports. The supported input formats of ISYS are listed in table below: + +.. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}| + +.. flat-table:: + :header-rows: 1 + + * - IPU6 ISYS supported input formats + + * - RGB565, RGB888 + + * - UYVY8, YUYV8 + + * - RAW8, RAW10, RAW12 + +.. _ipu6_isys_capture_examples: + +Examples +~~~~~~~~ +Here is an example of IPU6 ISYS raw capture on Dell XPS 9315 laptop. On this +machine, ov01a10 sensor is connected to IPU ISYS CSI2 port 2, which can +generate images at sBGGR10 with resolution 1280x800. + +Using the media controller APIs, we can configure ov01a10 sensor by +media-ctl [#f1]_ and yavta [#f2]_ to transmit frames to IPU6 ISYS. + +.. code-block:: none + + # Example 1 capture frame from ov01a10 camera sensor + # This example assumes /dev/media0 as the IPU ISYS media device + export MDEV=/dev/media0 + + # Establish the link for the media devices using media-ctl + media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]" + + # Set the format for the media devices + media-ctl -d $MDEV -V "ov01a10:0 [fmt:SBGGR10/1280x800]" + media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:0 [fmt:SBGGR10/1280x800]" + media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:1 [fmt:SBGGR10/1280x800]" + +Once the media pipeline is configured, desired sensor specific settings +(such as exposure and gain settings) can be set, using the yavta tool. + +e.g + +.. code-block:: none + + # and that ov01a10 sensor is connected to i2c bus 3 with address 0x36 + export SDEV=$(media-ctl -d $MDEV -e "ov01a10 3-0036") + + yavta -w 0x009e0903 400 $SDEV + yavta -w 0x009e0913 1000 $SDEV + yavta -w 0x009e0911 2000 $SDEV + +Once the desired sensor settings are set, frame captures can be done as below. + +e.g + +.. code-block:: none + + yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --file=/tmp/frame-#.bin \ + -f SBGGR10 $(media-ctl -d $MDEV -e "Intel IPU6 ISYS Capture 0") + +With the above command, 10 frames are captured at 1280x800 resolution with +sBGGR10 format. The captured frames are available as /tmp/frame-#.bin files. + +Here is another example of IPU6 ISYS RAW and metadata capture from camera +sensor ov2740 on Lenovo X1 Yoga laptop. + +.. code-block:: none + + media-ctl -l "\"ov2740 14-0036\":0 -> \"Intel IPU6 CSI2 1\":0[1]" + media-ctl -l "\"Intel IPU6 CSI2 1\":1 -> \"Intel IPU6 ISYS Capture 0\":0[5]" + media-ctl -l "\"Intel IPU6 CSI2 1\":2 -> \"Intel IPU6 ISYS Capture 1\":0[5]" + + # set routing + media-ctl -v -R "\"Intel IPU6 CSI2 1\" [0/0->1/0[1],0/1->2/1[1]]" + + media-ctl -v "\"Intel IPU6 CSI2 1\":0/0 [fmt:SGRBG10/1932x1092]" + media-ctl -v "\"Intel IPU6 CSI2 1\":0/1 [fmt:GENERIC_8/97x1]" + media-ctl -v "\"Intel IPU6 CSI2 1\":1/0 [fmt:SGRBG10/1932x1092]" + media-ctl -v "\"Intel IPU6 CSI2 1\":2/1 [fmt:GENERIC_8/97x1]" + + CAPTURE_DEV=$(media-ctl -e "Intel IPU6 ISYS Capture 0") + ./yavta --data-prefix -c100 -n5 -I -s1932x1092 --file=/tmp/frame-#.bin \ + -f SGRBG10 ${CAPTURE_DEV} + + CAPTURE_META=$(media-ctl -e "Intel IPU6 ISYS Capture 1") + ./yavta --data-prefix -c100 -n5 -I -s97x1 -B meta-capture \ + --file=/tmp/meta-#.bin -f GENERIC_8 ${CAPTURE_META} + +References +========== + +.. [#f1] https://git.ideasonboard.org/?p=media-ctl.git;a=summary +.. [#f2] https://git.ideasonboard.org/yavta.git diff --git a/Documentation/admin-guide/media/ipu6_isys_graph.svg b/Documentation/admin-guide/media/ipu6_isys_graph.svg new file mode 100644 index 000000000000..707747c75280 --- /dev/null +++ b/Documentation/admin-guide/media/ipu6_isys_graph.svg @@ -0,0 +1,174 @@ + + + + + + +board + + +n00000001 + +Intel IPU6 ISYS Capture 0 +/dev/video0 + + +n00000002 + +Intel IPU6 ISYS Capture 1 +/dev/video1 + + +n00000003 + +Intel IPU6 ISYS Capture 2 +/dev/video2 + + +n00000004 + +Intel IPU6 ISYS Capture 3 +/dev/video3 + + +n0000007d + +0 + +Intel IPU6 CSI2 0 +/dev/v4l-subdev0 + +1 + +2 + +3 + +4 + +5 + +6 + +7 + +8 + + +n0000007d:port1->n00000001 + + + + +n00000087 + +0 + +Intel IPU6 CSI2 1 +/dev/v4l-subdev1 + +1 + +2 + +3 + +4 + +5 + +6 + +7 + +8 + + +n00000087:port1->n00000002 + + + + +n00000091 + +0 + +Intel IPU6 CSI2 2 +/dev/v4l-subdev2 + +1 + +2 + +3 + +4 + +5 + +6 + +7 + +8 + + +n00000091:port1->n00000003 + + + + +n0000009b + +0 + +Intel IPU6 CSI2 3 +/dev/v4l-subdev3 + +1 + +2 + +3 + +4 + +5 + +6 + +7 + +8 + + +n0000009b:port1->n00000004 + + + + +n00000865 + + + +ov01a10 3-0036 +/dev/v4l-subdev4 + +0 + + +n00000865:port0->n00000091:port0 + + + + + + + + + + + diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst index 61283d67ceef..50bdef2d1762 100644 --- a/Documentation/admin-guide/media/v4l-drivers.rst +++ b/Documentation/admin-guide/media/v4l-drivers.rst @@ -16,6 +16,7 @@ Video4Linux (V4L) driver-specific documentation imx imx7 ipu3 + ipu6-isys ivtv mgb4 omap3isp