| Message ID | 20221118093931.1284465-1-paul.elder@ideasonboard.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | media: rkisp1: Add support for i.MX8MP | expand |
Hi Adam, On Wed, Feb 15, 2023 at 07:57:53AM -0600, Adam Ford wrote: > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote: > > > This series depends on v3 of "dt-bindings: media: Add macros for video > > interface bus types" [1]. > > > > This series extends the rkisp1 driver to support the ISP found in the > > NXP i.MX8MP SoC. > > I'm going to spend some time testing this over the weekend. Is there a V4 > pending, or should I just test whatever is in Laurent's repo? I've updated all the v6.2-based branches on https://gitlab.com/ideasonboard/nxp/linux.git (and renamed them to v6.2/*). Beside a rebase, the v6.2/isp branch contains (at the bottom) 6 additional patches that I've previously posted to the linux-media mailing list (feel free to review them ;-)). My only concern with this series is with patch "media: rkisp1: Add match data for i.MX8MP ISP", and in particular with the following hunk: enum rkisp1_cif_isp_version { RKISP1_V10 = 10, RKISP1_V11, RKISP1_V12, RKISP1_V13, + IMX8MP_V10, }; It's not a very nice versioning scheme :-S I'll see if I can find something better, but regardless of that, I'll post v4 with the goal of merging it in v6.4. > I have an IMX219 camera with 4-lane support and an i.MX8M Plus kit from > Beacon, and I want to test the RGGB bayer conversion to see how well it > works. > > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1") > > and in the NXP i.MX8MP have the same origin, and have slightly diverged > > over time as they are now independently developed (afaik) by Rockchip > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano", > > and is close enough to the RK3399 ISP that it can easily be supported by > > the same driver. > > Is there a reason the driver cannot be renamed to a more generic name than > rkisp1 if the Rockchip and VeriSilicon had similar origins? Having the > name Rockchip referenced from an NXP i.MX8M Plus seems odd to me. The common roots of the IP core predate both Rockchip and VeriSilicon. Those two implementations have now diverged (as with all forks), so either name would be wrong in some cases :-S > > The last two patches add support for UYVY output format, which can be > > implemented on the ISP version in the i.MX8MP but not in the one in the > > RK3399. > > > > This version of the series specifically has been tested on a Polyhex > > Debix model A with an imx219 (Raspberry Pi cam v2). > > > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/ > > > > Laurent Pinchart (3): > > dt-bindings: media: rkisp1: Add i.MX8MP ISP example > > media: rkisp1: Add and use rkisp1_has_feature() macro > > media: rkisp1: Configure gasket on i.MX8MP > > > > Paul Elder (11): > > dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible > > media: rkisp1: Add match data for i.MX8MP ISP > > media: rkisp1: Add and set registers for crop for i.MX8MP > > media: rkisp1: Add and set registers for output size config on i.MX8MP > > media: rkisp1: Add i.MX8MP-specific registers for MI and resizer > > media: rkisp1: Shift DMA buffer addresses on i.MX8MP > > media: rkisp1: Add register definitions for the test pattern generator > > media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP > > media: rkisp1: Support devices without self path > > media: rkisp1: Add YC swap capability > > media: rkisp1: Add UYVY as an output format > > > > .../bindings/media/rockchip-isp1.yaml | 79 ++++++++++- > > .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++--- > > .../platform/rockchip/rkisp1/rkisp1-common.h | 32 +++++ > > .../platform/rockchip/rkisp1/rkisp1-debug.c | 14 +- > > .../platform/rockchip/rkisp1/rkisp1-dev.c | 67 +++++++-- > > .../platform/rockchip/rkisp1/rkisp1-isp.c | 128 +++++++++++++++++- > > .../platform/rockchip/rkisp1/rkisp1-regs.h | 90 ++++++++++++ > > .../platform/rockchip/rkisp1/rkisp1-resizer.c | 35 ++++- > > include/uapi/linux/rkisp1-config.h | 2 + > > 9 files changed, 509 insertions(+), 40 deletions(-)
On Wed, Feb 15, 2023 at 5:55 PM Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote: > > Hi Adam, > > On Wed, Feb 15, 2023 at 07:57:53AM -0600, Adam Ford wrote: > > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote: > > > > > This series depends on v3 of "dt-bindings: media: Add macros for video > > > interface bus types" [1]. > > > > > > This series extends the rkisp1 driver to support the ISP found in the > > > NXP i.MX8MP SoC. > > > > I'm going to spend some time testing this over the weekend. Is there a V4 > > pending, or should I just test whatever is in Laurent's repo? > > I've updated all the v6.2-based branches on > https://gitlab.com/ideasonboard/nxp/linux.git (and renamed them to > v6.2/*). Beside a rebase, the v6.2/isp branch contains (at the bottom) 6 > additional patches that I've previously posted to the linux-media > mailing list (feel free to review them ;-)). I grabbed your v6.2 series, and applied some updates to enable an imx219 camera and routed it through the ISP and configured the camera to SRGGB10_1X10/640x480 and had the ISP convert to YUYV8_2X8/640x480 and it captured just fine. With that, I think you can add Tested-by: Adam Ford <aford173@gmail.com> #imx8mp-beacon I haven't experimented with the resizer yet,but I did have some questions on the AWB. The AWB appears to be available on the 8MP per the TRM, and I see reference to AWB in the driver, but when I query the subdev via yavta, I didn't see anything obvious on how to enable it. My pipeline (attached) shows klisp1_params as video2 and rkisp1_stats as video1. I attempted to query both without much success. root@beacon-imx8mp-kit:~# yavta -l /dev/video2 Device /dev/video2 opened. Device `rkisp1_params' on `platform:rkisp1' (driver 'rkisp1') supports meta-data, output, without mplanes. unable to query control 0xc0000000: Inappropriate ioctl for device (25). Meta-data format: RK1P (50314b52) buffer size 3048 root@beacon-imx8mp-kit:~# yavta -l /dev/video1 Device /dev/video1 opened. Device `rkisp1_stats' on `platform:rkisp1' (driver 'rkisp1') supports meta-data, capture, without mplanes. unable to query control 0xc0000000: Inappropriate ioctl for device (25). Meta-data format: RK1S (53314b52) buffer size 260 root@beacon-imx8mp-kit:~# Is there documentation somewhere on where to test the AWB? This is of particular interest to me, because the RGGB format of the camera comes across with a green tint. I am able to remove this green-ness on a different platform using some AWB on the ARM, but I'd rather do it in hardware if possible. Thanks adam > > My only concern with this series is with patch "media: rkisp1: Add match > data for i.MX8MP ISP", and in particular with the following hunk: > > enum rkisp1_cif_isp_version { > RKISP1_V10 = 10, > RKISP1_V11, > RKISP1_V12, > RKISP1_V13, > + IMX8MP_V10, > }; > > It's not a very nice versioning scheme :-S I'll see if I can find > something better, but regardless of that, I'll post v4 with the goal of > merging it in v6.4. > > > I have an IMX219 camera with 4-lane support and an i.MX8M Plus kit from > > Beacon, and I want to test the RGGB bayer conversion to see how well it > > works. > > > > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1") > > > and in the NXP i.MX8MP have the same origin, and have slightly diverged > > > over time as they are now independently developed (afaik) by Rockchip > > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano", > > > and is close enough to the RK3399 ISP that it can easily be supported by > > > the same driver. > > > > Is there a reason the driver cannot be renamed to a more generic name than > > rkisp1 if the Rockchip and VeriSilicon had similar origins? Having the > > name Rockchip referenced from an NXP i.MX8M Plus seems odd to me. > > The common roots of the IP core predate both Rockchip and VeriSilicon. > Those two implementations have now diverged (as with all forks), so > either name would be wrong in some cases :-S > > > > The last two patches add support for UYVY output format, which can be > > > implemented on the ISP version in the i.MX8MP but not in the one in the > > > RK3399. > > > > > > This version of the series specifically has been tested on a Polyhex > > > Debix model A with an imx219 (Raspberry Pi cam v2). > > > > > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/ > > > > > > Laurent Pinchart (3): > > > dt-bindings: media: rkisp1: Add i.MX8MP ISP example > > > media: rkisp1: Add and use rkisp1_has_feature() macro > > > media: rkisp1: Configure gasket on i.MX8MP > > > > > > Paul Elder (11): > > > dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible > > > media: rkisp1: Add match data for i.MX8MP ISP > > > media: rkisp1: Add and set registers for crop for i.MX8MP > > > media: rkisp1: Add and set registers for output size config on i.MX8MP > > > media: rkisp1: Add i.MX8MP-specific registers for MI and resizer > > > media: rkisp1: Shift DMA buffer addresses on i.MX8MP > > > media: rkisp1: Add register definitions for the test pattern generator > > > media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP > > > media: rkisp1: Support devices without self path > > > media: rkisp1: Add YC swap capability > > > media: rkisp1: Add UYVY as an output format > > > > > > .../bindings/media/rockchip-isp1.yaml | 79 ++++++++++- > > > .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++--- > > > .../platform/rockchip/rkisp1/rkisp1-common.h | 32 +++++ > > > .../platform/rockchip/rkisp1/rkisp1-debug.c | 14 +- > > > .../platform/rockchip/rkisp1/rkisp1-dev.c | 67 +++++++-- > > > .../platform/rockchip/rkisp1/rkisp1-isp.c | 128 +++++++++++++++++- > > > .../platform/rockchip/rkisp1/rkisp1-regs.h | 90 ++++++++++++ > > > .../platform/rockchip/rkisp1/rkisp1-resizer.c | 35 ++++- > > > include/uapi/linux/rkisp1-config.h | 2 + > > > 9 files changed, 509 insertions(+), 40 deletions(-) > > -- > Regards, > > Laurent Pinchart
On Fri, Nov 18, 2022 at 3:44 AM Paul Elder <paul.elder@ideasonboard.com> wrote: > > This series depends on v3 of "dt-bindings: media: Add macros for video > interface bus types" [1]. > > This series extends the rkisp1 driver to support the ISP found in the > NXP i.MX8MP SoC. > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1") > and in the NXP i.MX8MP have the same origin, and have slightly diverged > over time as they are now independently developed (afaik) by Rockchip > and VeriSilicon. The latter is marketed under the name "ISP8000Nano", > and is close enough to the RK3399 ISP that it can easily be supported by > the same driver. > > The last two patches add support for UYVY output format, which can be > implemented on the ISP version in the i.MX8MP but not in the one in the > RK3399. > > This version of the series specifically has been tested on a Polyhex > Debix model A with an imx219 (Raspberry Pi cam v2). > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/ > > Laurent Pinchart (3): > dt-bindings: media: rkisp1: Add i.MX8MP ISP example > media: rkisp1: Add and use rkisp1_has_feature() macro > media: rkisp1: Configure gasket on i.MX8MP > > Paul Elder (11): > dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible > media: rkisp1: Add match data for i.MX8MP ISP > media: rkisp1: Add and set registers for crop for i.MX8MP > media: rkisp1: Add and set registers for output size config on i.MX8MP > media: rkisp1: Add i.MX8MP-specific registers for MI and resizer > media: rkisp1: Shift DMA buffer addresses on i.MX8MP > media: rkisp1: Add register definitions for the test pattern generator > media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP > media: rkisp1: Support devices without self path > media: rkisp1: Add YC swap capability > media: rkisp1: Add UYVY as an output format > Paul / Laurent, I noticed an unexpected behaviour on the imx8mp. If I setup my pipeline for 640x480, it works just fine using an imx219 camera configured for SRGGB10_1X10. However, when I try to configure the pipeline to use the same camera at 1920x1080 (no resizing), the ISP source keeps defaulting to 640x480 Media device information ------------------------ driver rkisp1 model rkisp1 serial bus info platform:rkisp1 hw revision 0xe driver version 6.2.0 Device topology - entity 1: rkisp1_isp (4 pads, 4 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev0 pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480] <- "csis-32e40000.csi":1 [ENABLED] pad1: Sink [fmt:unknown/0x0 field:none] <- "rkisp1_params":0 [ENABLED,IMMUTABLE] pad2: Source [fmt:YUYV8_2X8/640x480 field:none colorspace:raw xfer:none ycbcr:601 quantization:lim-range crop.bounds:(0,0)/640x480 crop:(0,0)/640x480] -> "rkisp1_resizer_mainpath":0 [ENABLED] pad3: Source [fmt:unknown/0x0 field:none] -> "rkisp1_stats":0 [ENABLED,IMMUTABLE] - entity 6: rkisp1_resizer_mainpath (2 pads, 2 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev1 pad0: Sink [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480] <- "rkisp1_isp":2 [ENABLED] pad1: Source [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range] -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE] - entity 9: rkisp1_mainpath (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video0 pad0: Sink <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE] - entity 13: rkisp1_stats (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video1 pad0: Sink <- "rkisp1_isp":3 [ENABLED,IMMUTABLE] - entity 17: rkisp1_params (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video2 pad0: Source -> "rkisp1_isp":1 [ENABLED,IMMUTABLE] - entity 29: csis-32e40000.csi (2 pads, 2 links) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev2 pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] <- "imx219 1-0010":0 [ENABLED] pad1: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] -> "rkisp1_isp":0 [ENABLED] - entity 34: imx219 1-0010 (1 pad, 1 link) type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev3 pad0: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(8,8)/3280x2464 crop:(688,700)/1920x1080] -> "csis-32e40000.csi":0 [ENABLED] It's at this point that everything except the ISP source is 1920x1080. When I try to set the ISP sink to 1080, it ends up being 640x480 and the resizer sink is also changed to 640x480 root@beacon-imx8mp-kit:~# media-ctl -v -V "'rkisp1_isp':2 [fmt:YUYV8_2X8/1920x1080 field:none]" Opening media device /dev/media0 Enumerating entities looking up device: 81:3 looking up device: 81:4 looking up device: 81:0 looking up device: 81:1 looking up device: 81:2 looking up device: 81:5 looking up device: 81:6 Found 7 entities Enumerating pads and links Setting up format YUYV8_2X8 1920x1080 on pad rkisp1_isp/2 Format set: YUYV8_2X8 640x480 Setting up format YUYV8_2X8 640x480 on pad rkisp1_resizer_mainpath/0 Format set: YUYV8_2X8 640x480 It's my understanding that the ISP should be able to handle 1920x1080, and the resizer sink should match the ISP source. With the pipeline improperly setup, the capture fails. adam > .../bindings/media/rockchip-isp1.yaml | 79 ++++++++++- > .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++--- > .../platform/rockchip/rkisp1/rkisp1-common.h | 32 +++++ > .../platform/rockchip/rkisp1/rkisp1-debug.c | 14 +- > .../platform/rockchip/rkisp1/rkisp1-dev.c | 67 +++++++-- > .../platform/rockchip/rkisp1/rkisp1-isp.c | 128 +++++++++++++++++- > .../platform/rockchip/rkisp1/rkisp1-regs.h | 90 ++++++++++++ > .../platform/rockchip/rkisp1/rkisp1-resizer.c | 35 ++++- > include/uapi/linux/rkisp1-config.h | 2 + > 9 files changed, 509 insertions(+), 40 deletions(-) > > -- > 2.35.1 >
Hi Adam sorry to jump up without being involved in the conversation On Sat, Feb 18, 2023 at 10:14:08AM -0600, Adam Ford wrote: > On Wed, Feb 15, 2023 at 5:55 PM Laurent Pinchart > <laurent.pinchart@ideasonboard.com> wrote: > > > > Hi Adam, > > > > On Wed, Feb 15, 2023 at 07:57:53AM -0600, Adam Ford wrote: > > > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote: > > > > > > > This series depends on v3 of "dt-bindings: media: Add macros for video > > > > interface bus types" [1]. > > > > > > > > This series extends the rkisp1 driver to support the ISP found in the > > > > NXP i.MX8MP SoC. > > > > > > I'm going to spend some time testing this over the weekend. Is there a V4 > > > pending, or should I just test whatever is in Laurent's repo? > > > > I've updated all the v6.2-based branches on > > https://gitlab.com/ideasonboard/nxp/linux.git (and renamed them to > > v6.2/*). Beside a rebase, the v6.2/isp branch contains (at the bottom) 6 > > additional patches that I've previously posted to the linux-media > > mailing list (feel free to review them ;-)). > > I grabbed your v6.2 series, and applied some updates to enable an > imx219 camera and routed it through the ISP and configured the camera > to SRGGB10_1X10/640x480 and had the ISP convert to YUYV8_2X8/640x480 > and it captured just fine. > > With that, I think you can add > > Tested-by: Adam Ford <aford173@gmail.com> #imx8mp-beacon > > I haven't experimented with the resizer yet,but I did have some > questions on the AWB. The AWB appears to be available on the 8MP per > the TRM, and I see reference to AWB in the driver, but when I query > the subdev via yavta, I didn't see anything obvious on how to enable > it. My pipeline (attached) shows klisp1_params as video2 and > rkisp1_stats as video1. I attempted to query both without much > success. As you might be aware there's no magic button to "turn AWB on". The ISP enables the implementation of AWB algorithms that consumes the statistics the ISP produces on the raw images it is fed with and allows to program the color gains to realize colors balancing. The implementation of such algorithms doesn't live in the driver but rather in a separate component usually running in user space. With libcamera we're creating a userspace camera stack where it is possible to implement such algorithms, and the i.MX8MP is fairly well supported by the "rkisp1" component. https://git.libcamera.org/libcamera/libcamera.git/tree/src/libcamera/pipeline/rkisp1 https://git.libcamera.org/libcamera/libcamera.git/tree/src/ipa/rkisp1/algorithms There's probably one single patch still out of tree in libcamera to flip the switch and enable i.MX8MP support through the RkISP1 component. If you're willing to give it a spin let me know and I can try support you in testing it. > > root@beacon-imx8mp-kit:~# yavta -l /dev/video2 > Device /dev/video2 opened. > Device `rkisp1_params' on `platform:rkisp1' (driver 'rkisp1') supports > meta-data, output, without mplanes. > unable to query control 0xc0000000: Inappropriate ioctl for device (25). > Meta-data format: RK1P (50314b52) buffer size 3048 > root@beacon-imx8mp-kit:~# yavta -l /dev/video1 > Device /dev/video1 opened. > Device `rkisp1_stats' on `platform:rkisp1' (driver 'rkisp1') supports > meta-data, capture, without mplanes. > unable to query control 0xc0000000: Inappropriate ioctl for device (25). > Meta-data format: RK1S (53314b52) buffer size 260 > root@beacon-imx8mp-kit:~# > > Is there documentation somewhere on where to test the AWB? This is of > particular interest to me, because the RGGB format of the camera comes > across with a green tint. I am able to remove this green-ness on a > different platform using some AWB on the ARM, but I'd rather do it in > hardware if possible. > > Thanks > > adam > > > > > > My only concern with this series is with patch "media: rkisp1: Add match > > data for i.MX8MP ISP", and in particular with the following hunk: > > > > enum rkisp1_cif_isp_version { > > RKISP1_V10 = 10, > > RKISP1_V11, > > RKISP1_V12, > > RKISP1_V13, > > + IMX8MP_V10, > > }; > > > > It's not a very nice versioning scheme :-S I'll see if I can find > > something better, but regardless of that, I'll post v4 with the goal of > > merging it in v6.4. > > > > > I have an IMX219 camera with 4-lane support and an i.MX8M Plus kit from > > > Beacon, and I want to test the RGGB bayer conversion to see how well it > > > works. > > > > > > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1") > > > > and in the NXP i.MX8MP have the same origin, and have slightly diverged > > > > over time as they are now independently developed (afaik) by Rockchip > > > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano", > > > > and is close enough to the RK3399 ISP that it can easily be supported by > > > > the same driver. > > > > > > Is there a reason the driver cannot be renamed to a more generic name than > > > rkisp1 if the Rockchip and VeriSilicon had similar origins? Having the > > > name Rockchip referenced from an NXP i.MX8M Plus seems odd to me. > > > > The common roots of the IP core predate both Rockchip and VeriSilicon. > > Those two implementations have now diverged (as with all forks), so > > either name would be wrong in some cases :-S > > > > > > The last two patches add support for UYVY output format, which can be > > > > implemented on the ISP version in the i.MX8MP but not in the one in the > > > > RK3399. > > > > > > > > This version of the series specifically has been tested on a Polyhex > > > > Debix model A with an imx219 (Raspberry Pi cam v2). > > > > > > > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/ > > > > > > > > Laurent Pinchart (3): > > > > dt-bindings: media: rkisp1: Add i.MX8MP ISP example > > > > media: rkisp1: Add and use rkisp1_has_feature() macro > > > > media: rkisp1: Configure gasket on i.MX8MP > > > > > > > > Paul Elder (11): > > > > dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible > > > > media: rkisp1: Add match data for i.MX8MP ISP > > > > media: rkisp1: Add and set registers for crop for i.MX8MP > > > > media: rkisp1: Add and set registers for output size config on i.MX8MP > > > > media: rkisp1: Add i.MX8MP-specific registers for MI and resizer > > > > media: rkisp1: Shift DMA buffer addresses on i.MX8MP > > > > media: rkisp1: Add register definitions for the test pattern generator > > > > media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP > > > > media: rkisp1: Support devices without self path > > > > media: rkisp1: Add YC swap capability > > > > media: rkisp1: Add UYVY as an output format > > > > > > > > .../bindings/media/rockchip-isp1.yaml | 79 ++++++++++- > > > > .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++--- > > > > .../platform/rockchip/rkisp1/rkisp1-common.h | 32 +++++ > > > > .../platform/rockchip/rkisp1/rkisp1-debug.c | 14 +- > > > > .../platform/rockchip/rkisp1/rkisp1-dev.c | 67 +++++++-- > > > > .../platform/rockchip/rkisp1/rkisp1-isp.c | 128 +++++++++++++++++- > > > > .../platform/rockchip/rkisp1/rkisp1-regs.h | 90 ++++++++++++ > > > > .../platform/rockchip/rkisp1/rkisp1-resizer.c | 35 ++++- > > > > include/uapi/linux/rkisp1-config.h | 2 + > > > > 9 files changed, 509 insertions(+), 40 deletions(-) > > > > -- > > Regards, > > > > Laurent Pinchart
Hi Adam On Wed, Feb 22, 2023 at 05:39:30PM -0600, Adam Ford wrote: > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder <paul.elder@ideasonboard.com> wrote: > > > > This series depends on v3 of "dt-bindings: media: Add macros for video > > interface bus types" [1]. > > > > This series extends the rkisp1 driver to support the ISP found in the > > NXP i.MX8MP SoC. > > > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1") > > and in the NXP i.MX8MP have the same origin, and have slightly diverged > > over time as they are now independently developed (afaik) by Rockchip > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano", > > and is close enough to the RK3399 ISP that it can easily be supported by > > the same driver. > > > > The last two patches add support for UYVY output format, which can be > > implemented on the ISP version in the i.MX8MP but not in the one in the > > RK3399. > > > > This version of the series specifically has been tested on a Polyhex > > Debix model A with an imx219 (Raspberry Pi cam v2). > > > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/ > > > > Laurent Pinchart (3): > > dt-bindings: media: rkisp1: Add i.MX8MP ISP example > > media: rkisp1: Add and use rkisp1_has_feature() macro > > media: rkisp1: Configure gasket on i.MX8MP > > > > Paul Elder (11): > > dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible > > media: rkisp1: Add match data for i.MX8MP ISP > > media: rkisp1: Add and set registers for crop for i.MX8MP > > media: rkisp1: Add and set registers for output size config on i.MX8MP > > media: rkisp1: Add i.MX8MP-specific registers for MI and resizer > > media: rkisp1: Shift DMA buffer addresses on i.MX8MP > > media: rkisp1: Add register definitions for the test pattern generator > > media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP > > media: rkisp1: Support devices without self path > > media: rkisp1: Add YC swap capability > > media: rkisp1: Add UYVY as an output format > > > > Paul / Laurent, > > I noticed an unexpected behaviour on the imx8mp. > > If I setup my pipeline for 640x480, it works just fine using an imx219 > camera configured for SRGGB10_1X10. > > However, when I try to configure the pipeline to use the same camera > at 1920x1080 (no resizing), the ISP source keeps defaulting to 640x480 > > Media device information > ------------------------ > driver rkisp1 > model rkisp1 > serial > bus info platform:rkisp1 > hw revision 0xe > driver version 6.2.0 > > Device topology > - entity 1: rkisp1_isp (4 pads, 4 links) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev0 > pad0: Sink > [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none > ycbcr:601 quantization:full-range > crop.bounds:(0,0)/1920x1080 > crop:(0,0)/640x480] > <- "csis-32e40000.csi":1 [ENABLED] > pad1: Sink > [fmt:unknown/0x0 field:none] > <- "rkisp1_params":0 [ENABLED,IMMUTABLE] > pad2: Source > [fmt:YUYV8_2X8/640x480 field:none colorspace:raw xfer:none ycbcr:601 > quantization:lim-range > crop.bounds:(0,0)/640x480 > crop:(0,0)/640x480] > -> "rkisp1_resizer_mainpath":0 [ENABLED] > pad3: Source > [fmt:unknown/0x0 field:none] > -> "rkisp1_stats":0 [ENABLED,IMMUTABLE] > > - entity 6: rkisp1_resizer_mainpath (2 pads, 2 links) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev1 > pad0: Sink > [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb > ycbcr:601 quantization:lim-range > crop.bounds:(0,0)/1920x1080 > crop:(0,0)/640x480] > <- "rkisp1_isp":2 [ENABLED] > pad1: Source > [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb > ycbcr:601 quantization:lim-range] > -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE] > > - entity 9: rkisp1_mainpath (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video0 > pad0: Sink > <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE] > > - entity 13: rkisp1_stats (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video1 > pad0: Sink > <- "rkisp1_isp":3 [ENABLED,IMMUTABLE] > > - entity 17: rkisp1_params (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video2 > pad0: Source > -> "rkisp1_isp":1 [ENABLED,IMMUTABLE] > > - entity 29: csis-32e40000.csi (2 pads, 2 links) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev2 > pad0: Sink > [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb > ycbcr:601 quantization:full-range] > <- "imx219 1-0010":0 [ENABLED] > pad1: Source > [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb > ycbcr:601 quantization:full-range] > -> "rkisp1_isp":0 [ENABLED] > > - entity 34: imx219 1-0010 (1 pad, 1 link) > type V4L2 subdev subtype Sensor flags 0 > device node name /dev/v4l-subdev3 > pad0: Source > [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb > ycbcr:601 quantization:full-range > crop.bounds:(8,8)/3280x2464 > crop:(688,700)/1920x1080] > -> "csis-32e40000.csi":0 [ENABLED] Not sure if it's my email reader but I'm sorry I can't read this. Could you use an online paste service next time if lines are long and gets wrapped in this way ? > > It's at this point that everything except the ISP source is 1920x1080. > > When I try to set the ISP sink to 1080, it ends up being 640x480 and > the resizer sink is also changed to 640x480 > > root@beacon-imx8mp-kit:~# media-ctl -v -V "'rkisp1_isp':2 > [fmt:YUYV8_2X8/1920x1080 field:none]" > Opening media device /dev/media0 > Enumerating entities > looking up device: 81:3 > looking up device: 81:4 > looking up device: 81:0 > looking up device: 81:1 > looking up device: 81:2 > looking up device: 81:5 > looking up device: 81:6 > Found 7 entities > Enumerating pads and links > Setting up format YUYV8_2X8 1920x1080 on pad rkisp1_isp/2 > Format set: YUYV8_2X8 640x480 > Setting up format YUYV8_2X8 640x480 on pad rkisp1_resizer_mainpath/0 > Format set: YUYV8_2X8 640x480 > I'm on https://gitlab.com/ideasonboard/nxp/linux/-/tree/v6.2/merge with an additional patch to enable the ISP on the board I'm testing with I've tested capturing YUYV 1920x1080 with the following script ------------------------------------------------------------------------------ #!/bin/bash -x media-ctl -r # Link entities media-ctl -l '"ar0521 1-0036":0 -> "csis-32e40000.csi":0'[1] media-ctl -l '"csis-32e40000.csi":1 -> "rkisp1_isp":0'[1] media-ctl -l '"rkisp1_isp":2 -> "rkisp1_resizer_mainpath":0'[1] # Setup format media-ctl -v -V '"ar0521 1-0036":0 [fmt:SGRBG8_1X8/2592x1944]' media-ctl -v -V '"csis-32e40000.csi":1 [fmt:SGRBG8_1X8/2592x1944]' media-ctl -v -V '"rkisp1_isp":2 [fmt:YUYV8_2X8/2592x1944]' media-ctl -v -V '"rkisp1_resizer_mainpath":1 [fmt:YUYV8_2X8/1920x1080]' yavta -f YUYV -s 1920x1080 -c10 /dev/video0 --file=/tmp/frame-#.bin ------------------------------------------------------------------------------ You will have to adjust it for imx219 Unless you tweak the sensor's exposure and gains manually do not expect nice images. You would need an auto-exposure and gain routine like the one implemented in libcamera. > > It's my understanding that the ISP should be able to handle 1920x1080, > and the resizer sink should match the ISP source. > > With the pipeline improperly setup, the capture fails. > > adam > > > > .../bindings/media/rockchip-isp1.yaml | 79 ++++++++++- > > .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++--- > > .../platform/rockchip/rkisp1/rkisp1-common.h | 32 +++++ > > .../platform/rockchip/rkisp1/rkisp1-debug.c | 14 +- > > .../platform/rockchip/rkisp1/rkisp1-dev.c | 67 +++++++-- > > .../platform/rockchip/rkisp1/rkisp1-isp.c | 128 +++++++++++++++++- > > .../platform/rockchip/rkisp1/rkisp1-regs.h | 90 ++++++++++++ > > .../platform/rockchip/rkisp1/rkisp1-resizer.c | 35 ++++- > > include/uapi/linux/rkisp1-config.h | 2 + > > 9 files changed, 509 insertions(+), 40 deletions(-) > > > > -- > > 2.35.1 > >
Hi Adam, On Wed, Feb 22, 2023 at 05:39:30PM -0600, Adam Ford wrote: > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote: > > > > This series depends on v3 of "dt-bindings: media: Add macros for video > > interface bus types" [1]. > > > > This series extends the rkisp1 driver to support the ISP found in the > > NXP i.MX8MP SoC. > > > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1") > > and in the NXP i.MX8MP have the same origin, and have slightly diverged > > over time as they are now independently developed (afaik) by Rockchip > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano", > > and is close enough to the RK3399 ISP that it can easily be supported by > > the same driver. > > > > The last two patches add support for UYVY output format, which can be > > implemented on the ISP version in the i.MX8MP but not in the one in the > > RK3399. > > > > This version of the series specifically has been tested on a Polyhex > > Debix model A with an imx219 (Raspberry Pi cam v2). > > > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/ > > > > Laurent Pinchart (3): > > dt-bindings: media: rkisp1: Add i.MX8MP ISP example > > media: rkisp1: Add and use rkisp1_has_feature() macro > > media: rkisp1: Configure gasket on i.MX8MP > > > > Paul Elder (11): > > dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible > > media: rkisp1: Add match data for i.MX8MP ISP > > media: rkisp1: Add and set registers for crop for i.MX8MP > > media: rkisp1: Add and set registers for output size config on i.MX8MP > > media: rkisp1: Add i.MX8MP-specific registers for MI and resizer > > media: rkisp1: Shift DMA buffer addresses on i.MX8MP > > media: rkisp1: Add register definitions for the test pattern generator > > media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP > > media: rkisp1: Support devices without self path > > media: rkisp1: Add YC swap capability > > media: rkisp1: Add UYVY as an output format > > Paul / Laurent, > > I noticed an unexpected behaviour on the imx8mp. > > If I setup my pipeline for 640x480, it works just fine using an imx219 > camera configured for SRGGB10_1X10. > > However, when I try to configure the pipeline to use the same camera > at 1920x1080 (no resizing), the ISP source keeps defaulting to 640x480 > > Media device information > ------------------------ > driver rkisp1 > model rkisp1 > serial > bus info platform:rkisp1 > hw revision 0xe > driver version 6.2.0 > > Device topology > - entity 1: rkisp1_isp (4 pads, 4 links) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev0 > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480] You're cropping the image to 640x480 here. You need to set the crop rectangle to 1920x1080. As Jacopo mentioned, I wouldn't recommend exercising the ISP directly. Not only do you need to setup the pipeline, but you would also need to implement all the imaging algorithms in userspace. libcamera will do all this for you. > <- "csis-32e40000.csi":1 [ENABLED] > pad1: Sink [fmt:unknown/0x0 field:none] > <- "rkisp1_params":0 [ENABLED,IMMUTABLE] > pad2: Source [fmt:YUYV8_2X8/640x480 field:none colorspace:raw xfer:none ycbcr:601 quantization:lim-range crop.bounds:(0,0)/640x480 crop:(0,0)/640x480] > -> "rkisp1_resizer_mainpath":0 [ENABLED] > pad3: Source [fmt:unknown/0x0 field:none] > -> "rkisp1_stats":0 [ENABLED,IMMUTABLE] > > - entity 6: rkisp1_resizer_mainpath (2 pads, 2 links) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev1 > pad0: Sink [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480] > <- "rkisp1_isp":2 [ENABLED] > pad1: Source [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range] > -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE] > > - entity 9: rkisp1_mainpath (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video0 > pad0: Sink > <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE] > > - entity 13: rkisp1_stats (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video1 > pad0: Sink > <- "rkisp1_isp":3 [ENABLED,IMMUTABLE] > > - entity 17: rkisp1_params (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video2 > pad0: Source > -> "rkisp1_isp":1 [ENABLED,IMMUTABLE] > > - entity 29: csis-32e40000.csi (2 pads, 2 links) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev2 > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] > <- "imx219 1-0010":0 [ENABLED] > pad1: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] > -> "rkisp1_isp":0 [ENABLED] > > - entity 34: imx219 1-0010 (1 pad, 1 link) > type V4L2 subdev subtype Sensor flags 0 > device node name /dev/v4l-subdev3 > pad0: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(8,8)/3280x2464 crop:(688,700)/1920x1080] > -> "csis-32e40000.csi":0 [ENABLED] > > It's at this point that everything except the ISP source is 1920x1080. > > When I try to set the ISP sink to 1080, it ends up being 640x480 and > the resizer sink is also changed to 640x480 > > root@beacon-imx8mp-kit:~# media-ctl -v -V "'rkisp1_isp':2 > [fmt:YUYV8_2X8/1920x1080 field:none]" > Opening media device /dev/media0 > Enumerating entities > looking up device: 81:3 > looking up device: 81:4 > looking up device: 81:0 > looking up device: 81:1 > looking up device: 81:2 > looking up device: 81:5 > looking up device: 81:6 > Found 7 entities > Enumerating pads and links > Setting up format YUYV8_2X8 1920x1080 on pad rkisp1_isp/2 > Format set: YUYV8_2X8 640x480 > Setting up format YUYV8_2X8 640x480 on pad rkisp1_resizer_mainpath/0 > Format set: YUYV8_2X8 640x480 > > > It's my understanding that the ISP should be able to handle 1920x1080, > and the resizer sink should match the ISP source. > > With the pipeline improperly setup, the capture fails. > > > .../bindings/media/rockchip-isp1.yaml | 79 ++++++++++- > > .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++--- > > .../platform/rockchip/rkisp1/rkisp1-common.h | 32 +++++ > > .../platform/rockchip/rkisp1/rkisp1-debug.c | 14 +- > > .../platform/rockchip/rkisp1/rkisp1-dev.c | 67 +++++++-- > > .../platform/rockchip/rkisp1/rkisp1-isp.c | 128 +++++++++++++++++- > > .../platform/rockchip/rkisp1/rkisp1-regs.h | 90 ++++++++++++ > > .../platform/rockchip/rkisp1/rkisp1-resizer.c | 35 ++++- > > include/uapi/linux/rkisp1-config.h | 2 + > > 9 files changed, 509 insertions(+), 40 deletions(-)
On Thu, Feb 23, 2023 at 8:26 AM Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote: > > Hi Adam, > > On Wed, Feb 22, 2023 at 05:39:30PM -0600, Adam Ford wrote: > > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote: > > > > > > This series depends on v3 of "dt-bindings: media: Add macros for video > > > interface bus types" [1]. > > > > > > This series extends the rkisp1 driver to support the ISP found in the > > > NXP i.MX8MP SoC. > > > > > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1") > > > and in the NXP i.MX8MP have the same origin, and have slightly diverged > > > over time as they are now independently developed (afaik) by Rockchip > > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano", > > > and is close enough to the RK3399 ISP that it can easily be supported by > > > the same driver. > > > > > > The last two patches add support for UYVY output format, which can be > > > implemented on the ISP version in the i.MX8MP but not in the one in the > > > RK3399. > > > > > > This version of the series specifically has been tested on a Polyhex > > > Debix model A with an imx219 (Raspberry Pi cam v2). > > > > > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/ > > > > > > Laurent Pinchart (3): > > > dt-bindings: media: rkisp1: Add i.MX8MP ISP example > > > media: rkisp1: Add and use rkisp1_has_feature() macro > > > media: rkisp1: Configure gasket on i.MX8MP > > > > > > Paul Elder (11): > > > dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible > > > media: rkisp1: Add match data for i.MX8MP ISP > > > media: rkisp1: Add and set registers for crop for i.MX8MP > > > media: rkisp1: Add and set registers for output size config on i.MX8MP > > > media: rkisp1: Add i.MX8MP-specific registers for MI and resizer > > > media: rkisp1: Shift DMA buffer addresses on i.MX8MP > > > media: rkisp1: Add register definitions for the test pattern generator > > > media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP > > > media: rkisp1: Support devices without self path > > > media: rkisp1: Add YC swap capability > > > media: rkisp1: Add UYVY as an output format > > > > Paul / Laurent, > > > > I noticed an unexpected behaviour on the imx8mp. > > > > If I setup my pipeline for 640x480, it works just fine using an imx219 > > camera configured for SRGGB10_1X10. > > > > However, when I try to configure the pipeline to use the same camera > > at 1920x1080 (no resizing), the ISP source keeps defaulting to 640x480 > > > > Media device information > > ------------------------ > > driver rkisp1 > > model rkisp1 > > serial > > bus info platform:rkisp1 > > hw revision 0xe > > driver version 6.2.0 > > > > Device topology > > - entity 1: rkisp1_isp (4 pads, 4 links) > > type V4L2 subdev subtype Unknown flags 0 > > device node name /dev/v4l-subdev0 > > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480] > > You're cropping the image to 640x480 here. You need to set the crop > rectangle to 1920x1080. > > As Jacopo mentioned, I wouldn't recommend exercising the ISP directly. > Not only do you need to setup the pipeline, but you would also need to > implement all the imaging algorithms in userspace. libcamera will do all > this for you. I'll give that a try. My current employer has a v4l2src requirement, but I can likely make an argument to switch to libcamera. I didn't catch the cropping part. Thanks for that. adam > > > <- "csis-32e40000.csi":1 [ENABLED] > > pad1: Sink [fmt:unknown/0x0 field:none] > > <- "rkisp1_params":0 [ENABLED,IMMUTABLE] > > pad2: Source [fmt:YUYV8_2X8/640x480 field:none colorspace:raw xfer:none ycbcr:601 quantization:lim-range crop.bounds:(0,0)/640x480 crop:(0,0)/640x480] > > -> "rkisp1_resizer_mainpath":0 [ENABLED] > > pad3: Source [fmt:unknown/0x0 field:none] > > -> "rkisp1_stats":0 [ENABLED,IMMUTABLE] > > > > - entity 6: rkisp1_resizer_mainpath (2 pads, 2 links) > > type V4L2 subdev subtype Unknown flags 0 > > device node name /dev/v4l-subdev1 > > pad0: Sink [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480] > > <- "rkisp1_isp":2 [ENABLED] > > pad1: Source [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range] > > -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE] > > > > - entity 9: rkisp1_mainpath (1 pad, 1 link) > > type Node subtype V4L flags 0 > > device node name /dev/video0 > > pad0: Sink > > <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE] > > > > - entity 13: rkisp1_stats (1 pad, 1 link) > > type Node subtype V4L flags 0 > > device node name /dev/video1 > > pad0: Sink > > <- "rkisp1_isp":3 [ENABLED,IMMUTABLE] > > > > - entity 17: rkisp1_params (1 pad, 1 link) > > type Node subtype V4L flags 0 > > device node name /dev/video2 > > pad0: Source > > -> "rkisp1_isp":1 [ENABLED,IMMUTABLE] > > > > - entity 29: csis-32e40000.csi (2 pads, 2 links) > > type V4L2 subdev subtype Unknown flags 0 > > device node name /dev/v4l-subdev2 > > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] > > <- "imx219 1-0010":0 [ENABLED] > > pad1: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] > > -> "rkisp1_isp":0 [ENABLED] > > > > - entity 34: imx219 1-0010 (1 pad, 1 link) > > type V4L2 subdev subtype Sensor flags 0 > > device node name /dev/v4l-subdev3 > > pad0: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(8,8)/3280x2464 crop:(688,700)/1920x1080] > > -> "csis-32e40000.csi":0 [ENABLED] > > > > It's at this point that everything except the ISP source is 1920x1080. > > > > When I try to set the ISP sink to 1080, it ends up being 640x480 and > > the resizer sink is also changed to 640x480 > > > > root@beacon-imx8mp-kit:~# media-ctl -v -V "'rkisp1_isp':2 > > [fmt:YUYV8_2X8/1920x1080 field:none]" > > Opening media device /dev/media0 > > Enumerating entities > > looking up device: 81:3 > > looking up device: 81:4 > > looking up device: 81:0 > > looking up device: 81:1 > > looking up device: 81:2 > > looking up device: 81:5 > > looking up device: 81:6 > > Found 7 entities > > Enumerating pads and links > > Setting up format YUYV8_2X8 1920x1080 on pad rkisp1_isp/2 > > Format set: YUYV8_2X8 640x480 > > Setting up format YUYV8_2X8 640x480 on pad rkisp1_resizer_mainpath/0 > > Format set: YUYV8_2X8 640x480 > > > > > > It's my understanding that the ISP should be able to handle 1920x1080, > > and the resizer sink should match the ISP source. > > > > With the pipeline improperly setup, the capture fails. > > > > > .../bindings/media/rockchip-isp1.yaml | 79 ++++++++++- > > > .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++--- > > > .../platform/rockchip/rkisp1/rkisp1-common.h | 32 +++++ > > > .../platform/rockchip/rkisp1/rkisp1-debug.c | 14 +- > > > .../platform/rockchip/rkisp1/rkisp1-dev.c | 67 +++++++-- > > > .../platform/rockchip/rkisp1/rkisp1-isp.c | 128 +++++++++++++++++- > > > .../platform/rockchip/rkisp1/rkisp1-regs.h | 90 ++++++++++++ > > > .../platform/rockchip/rkisp1/rkisp1-resizer.c | 35 ++++- > > > include/uapi/linux/rkisp1-config.h | 2 + > > > 9 files changed, 509 insertions(+), 40 deletions(-) > > -- > Regards, > > Laurent Pinchart
Hi Adam, On Thu, Feb 23, 2023 at 10:10:28AM -0600, Adam Ford wrote: > On Thu, Feb 23, 2023 at 8:26 AM Laurent Pinchart wrote: > > On Wed, Feb 22, 2023 at 05:39:30PM -0600, Adam Ford wrote: > > > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote: > > > > > > > > This series depends on v3 of "dt-bindings: media: Add macros for video > > > > interface bus types" [1]. > > > > > > > > This series extends the rkisp1 driver to support the ISP found in the > > > > NXP i.MX8MP SoC. > > > > > > > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1") > > > > and in the NXP i.MX8MP have the same origin, and have slightly diverged > > > > over time as they are now independently developed (afaik) by Rockchip > > > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano", > > > > and is close enough to the RK3399 ISP that it can easily be supported by > > > > the same driver. > > > > > > > > The last two patches add support for UYVY output format, which can be > > > > implemented on the ISP version in the i.MX8MP but not in the one in the > > > > RK3399. > > > > > > > > This version of the series specifically has been tested on a Polyhex > > > > Debix model A with an imx219 (Raspberry Pi cam v2). > > > > > > > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/ > > > > > > > > Laurent Pinchart (3): > > > > dt-bindings: media: rkisp1: Add i.MX8MP ISP example > > > > media: rkisp1: Add and use rkisp1_has_feature() macro > > > > media: rkisp1: Configure gasket on i.MX8MP > > > > > > > > Paul Elder (11): > > > > dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible > > > > media: rkisp1: Add match data for i.MX8MP ISP > > > > media: rkisp1: Add and set registers for crop for i.MX8MP > > > > media: rkisp1: Add and set registers for output size config on i.MX8MP > > > > media: rkisp1: Add i.MX8MP-specific registers for MI and resizer > > > > media: rkisp1: Shift DMA buffer addresses on i.MX8MP > > > > media: rkisp1: Add register definitions for the test pattern generator > > > > media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP > > > > media: rkisp1: Support devices without self path > > > > media: rkisp1: Add YC swap capability > > > > media: rkisp1: Add UYVY as an output format > > > > > > Paul / Laurent, > > > > > > I noticed an unexpected behaviour on the imx8mp. > > > > > > If I setup my pipeline for 640x480, it works just fine using an imx219 > > > camera configured for SRGGB10_1X10. > > > > > > However, when I try to configure the pipeline to use the same camera > > > at 1920x1080 (no resizing), the ISP source keeps defaulting to 640x480 > > > > > > Media device information > > > ------------------------ > > > driver rkisp1 > > > model rkisp1 > > > serial > > > bus info platform:rkisp1 > > > hw revision 0xe > > > driver version 6.2.0 > > > > > > Device topology > > > - entity 1: rkisp1_isp (4 pads, 4 links) > > > type V4L2 subdev subtype Unknown flags 0 > > > device node name /dev/v4l-subdev0 > > > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480] > > > > You're cropping the image to 640x480 here. You need to set the crop > > rectangle to 1920x1080. > > > > As Jacopo mentioned, I wouldn't recommend exercising the ISP directly. > > Not only do you need to setup the pipeline, but you would also need to > > implement all the imaging algorithms in userspace. libcamera will do all > > this for you. > > I'll give that a try. My current employer has a v4l2src requirement, > but I can likely make an argument to switch to libcamera. I didn't > catch the cropping part. Thanks for that. libcamera has a GStreamer source element called libcamerasrc which is aimed to be a drop-in replacement for v4l2src. Some features may be missing, let us know and we can discuss how to add them. > > > <- "csis-32e40000.csi":1 [ENABLED] > > > pad1: Sink [fmt:unknown/0x0 field:none] > > > <- "rkisp1_params":0 [ENABLED,IMMUTABLE] > > > pad2: Source [fmt:YUYV8_2X8/640x480 field:none colorspace:raw xfer:none ycbcr:601 quantization:lim-range crop.bounds:(0,0)/640x480 crop:(0,0)/640x480] > > > -> "rkisp1_resizer_mainpath":0 [ENABLED] > > > pad3: Source [fmt:unknown/0x0 field:none] > > > -> "rkisp1_stats":0 [ENABLED,IMMUTABLE] > > > > > > - entity 6: rkisp1_resizer_mainpath (2 pads, 2 links) > > > type V4L2 subdev subtype Unknown flags 0 > > > device node name /dev/v4l-subdev1 > > > pad0: Sink [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480] > > > <- "rkisp1_isp":2 [ENABLED] > > > pad1: Source [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range] > > > -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE] > > > > > > - entity 9: rkisp1_mainpath (1 pad, 1 link) > > > type Node subtype V4L flags 0 > > > device node name /dev/video0 > > > pad0: Sink > > > <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE] > > > > > > - entity 13: rkisp1_stats (1 pad, 1 link) > > > type Node subtype V4L flags 0 > > > device node name /dev/video1 > > > pad0: Sink > > > <- "rkisp1_isp":3 [ENABLED,IMMUTABLE] > > > > > > - entity 17: rkisp1_params (1 pad, 1 link) > > > type Node subtype V4L flags 0 > > > device node name /dev/video2 > > > pad0: Source > > > -> "rkisp1_isp":1 [ENABLED,IMMUTABLE] > > > > > > - entity 29: csis-32e40000.csi (2 pads, 2 links) > > > type V4L2 subdev subtype Unknown flags 0 > > > device node name /dev/v4l-subdev2 > > > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] > > > <- "imx219 1-0010":0 [ENABLED] > > > pad1: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] > > > -> "rkisp1_isp":0 [ENABLED] > > > > > > - entity 34: imx219 1-0010 (1 pad, 1 link) > > > type V4L2 subdev subtype Sensor flags 0 > > > device node name /dev/v4l-subdev3 > > > pad0: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(8,8)/3280x2464 crop:(688,700)/1920x1080] > > > -> "csis-32e40000.csi":0 [ENABLED] > > > > > > It's at this point that everything except the ISP source is 1920x1080. > > > > > > When I try to set the ISP sink to 1080, it ends up being 640x480 and > > > the resizer sink is also changed to 640x480 > > > > > > root@beacon-imx8mp-kit:~# media-ctl -v -V "'rkisp1_isp':2 > > > [fmt:YUYV8_2X8/1920x1080 field:none]" > > > Opening media device /dev/media0 > > > Enumerating entities > > > looking up device: 81:3 > > > looking up device: 81:4 > > > looking up device: 81:0 > > > looking up device: 81:1 > > > looking up device: 81:2 > > > looking up device: 81:5 > > > looking up device: 81:6 > > > Found 7 entities > > > Enumerating pads and links > > > Setting up format YUYV8_2X8 1920x1080 on pad rkisp1_isp/2 > > > Format set: YUYV8_2X8 640x480 > > > Setting up format YUYV8_2X8 640x480 on pad rkisp1_resizer_mainpath/0 > > > Format set: YUYV8_2X8 640x480 > > > > > > > > > It's my understanding that the ISP should be able to handle 1920x1080, > > > and the resizer sink should match the ISP source. > > > > > > With the pipeline improperly setup, the capture fails. > > > > > > > .../bindings/media/rockchip-isp1.yaml | 79 ++++++++++- > > > > .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++--- > > > > .../platform/rockchip/rkisp1/rkisp1-common.h | 32 +++++ > > > > .../platform/rockchip/rkisp1/rkisp1-debug.c | 14 +- > > > > .../platform/rockchip/rkisp1/rkisp1-dev.c | 67 +++++++-- > > > > .../platform/rockchip/rkisp1/rkisp1-isp.c | 128 +++++++++++++++++- > > > > .../platform/rockchip/rkisp1/rkisp1-regs.h | 90 ++++++++++++ > > > > .../platform/rockchip/rkisp1/rkisp1-resizer.c | 35 ++++- > > > > include/uapi/linux/rkisp1-config.h | 2 + > > > > 9 files changed, 509 insertions(+), 40 deletions(-)
Hi Adam, Le jeudi 23 février 2023 à 10:10 -0600, Adam Ford a écrit : > On Thu, Feb 23, 2023 at 8:26 AM Laurent Pinchart > <laurent.pinchart@ideasonboard.com> wrote: > > > > Hi Adam, > > > > On Wed, Feb 22, 2023 at 05:39:30PM -0600, Adam Ford wrote: > > > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote: > > > > > > > > This series depends on v3 of "dt-bindings: media: Add macros for video > > > > interface bus types" [1]. > > > > > > > > This series extends the rkisp1 driver to support the ISP found in the > > > > NXP i.MX8MP SoC. > > > > > > > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1") > > > > and in the NXP i.MX8MP have the same origin, and have slightly diverged > > > > over time as they are now independently developed (afaik) by Rockchip > > > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano", > > > > and is close enough to the RK3399 ISP that it can easily be supported by > > > > the same driver. > > > > > > > > The last two patches add support for UYVY output format, which can be > > > > implemented on the ISP version in the i.MX8MP but not in the one in the > > > > RK3399. > > > > > > > > This version of the series specifically has been tested on a Polyhex > > > > Debix model A with an imx219 (Raspberry Pi cam v2). > > > > > > > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/ > > > > > > > > Laurent Pinchart (3): > > > > dt-bindings: media: rkisp1: Add i.MX8MP ISP example > > > > media: rkisp1: Add and use rkisp1_has_feature() macro > > > > media: rkisp1: Configure gasket on i.MX8MP > > > > > > > > Paul Elder (11): > > > > dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible > > > > media: rkisp1: Add match data for i.MX8MP ISP > > > > media: rkisp1: Add and set registers for crop for i.MX8MP > > > > media: rkisp1: Add and set registers for output size config on i.MX8MP > > > > media: rkisp1: Add i.MX8MP-specific registers for MI and resizer > > > > media: rkisp1: Shift DMA buffer addresses on i.MX8MP > > > > media: rkisp1: Add register definitions for the test pattern generator > > > > media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP > > > > media: rkisp1: Support devices without self path > > > > media: rkisp1: Add YC swap capability > > > > media: rkisp1: Add UYVY as an output format > > > > > > Paul / Laurent, > > > > > > I noticed an unexpected behaviour on the imx8mp. > > > > > > If I setup my pipeline for 640x480, it works just fine using an imx219 > > > camera configured for SRGGB10_1X10. > > > > > > However, when I try to configure the pipeline to use the same camera > > > at 1920x1080 (no resizing), the ISP source keeps defaulting to 640x480 > > > > > > Media device information > > > ------------------------ > > > driver rkisp1 > > > model rkisp1 > > > serial > > > bus info platform:rkisp1 > > > hw revision 0xe > > > driver version 6.2.0 > > > > > > Device topology > > > - entity 1: rkisp1_isp (4 pads, 4 links) > > > type V4L2 subdev subtype Unknown flags 0 > > > device node name /dev/v4l-subdev0 > > > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480] > > > > You're cropping the image to 640x480 here. You need to set the crop > > rectangle to 1920x1080. > > > > As Jacopo mentioned, I wouldn't recommend exercising the ISP directly. > > Not only do you need to setup the pipeline, but you would also need to > > implement all the imaging algorithms in userspace. libcamera will do all > > this for you. > > I'll give that a try. My current employer has a v4l2src requirement, > but I can likely make an argument to switch to libcamera. I didn't > catch the cropping part. Thanks for that. I'd hope you can transparently replace v4l2src with libcamerasrc, the plugins currently lives inside the libcamera project. If not, I'd really like to know why. We can work together on adding missing controls (this is something I'm starting on soon). regards, Nicolas > > adam > > > > > <- "csis-32e40000.csi":1 [ENABLED] > > > pad1: Sink [fmt:unknown/0x0 field:none] > > > <- "rkisp1_params":0 [ENABLED,IMMUTABLE] > > > pad2: Source [fmt:YUYV8_2X8/640x480 field:none colorspace:raw xfer:none ycbcr:601 quantization:lim-range crop.bounds:(0,0)/640x480 crop:(0,0)/640x480] > > > -> "rkisp1_resizer_mainpath":0 [ENABLED] > > > pad3: Source [fmt:unknown/0x0 field:none] > > > -> "rkisp1_stats":0 [ENABLED,IMMUTABLE] > > > > > > - entity 6: rkisp1_resizer_mainpath (2 pads, 2 links) > > > type V4L2 subdev subtype Unknown flags 0 > > > device node name /dev/v4l-subdev1 > > > pad0: Sink [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480] > > > <- "rkisp1_isp":2 [ENABLED] > > > pad1: Source [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range] > > > -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE] > > > > > > - entity 9: rkisp1_mainpath (1 pad, 1 link) > > > type Node subtype V4L flags 0 > > > device node name /dev/video0 > > > pad0: Sink > > > <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE] > > > > > > - entity 13: rkisp1_stats (1 pad, 1 link) > > > type Node subtype V4L flags 0 > > > device node name /dev/video1 > > > pad0: Sink > > > <- "rkisp1_isp":3 [ENABLED,IMMUTABLE] > > > > > > - entity 17: rkisp1_params (1 pad, 1 link) > > > type Node subtype V4L flags 0 > > > device node name /dev/video2 > > > pad0: Source > > > -> "rkisp1_isp":1 [ENABLED,IMMUTABLE] > > > > > > - entity 29: csis-32e40000.csi (2 pads, 2 links) > > > type V4L2 subdev subtype Unknown flags 0 > > > device node name /dev/v4l-subdev2 > > > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] > > > <- "imx219 1-0010":0 [ENABLED] > > > pad1: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] > > > -> "rkisp1_isp":0 [ENABLED] > > > > > > - entity 34: imx219 1-0010 (1 pad, 1 link) > > > type V4L2 subdev subtype Sensor flags 0 > > > device node name /dev/v4l-subdev3 > > > pad0: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(8,8)/3280x2464 crop:(688,700)/1920x1080] > > > -> "csis-32e40000.csi":0 [ENABLED] > > > > > > It's at this point that everything except the ISP source is 1920x1080. > > > > > > When I try to set the ISP sink to 1080, it ends up being 640x480 and > > > the resizer sink is also changed to 640x480 > > > > > > root@beacon-imx8mp-kit:~# media-ctl -v -V "'rkisp1_isp':2 > > > [fmt:YUYV8_2X8/1920x1080 field:none]" > > > Opening media device /dev/media0 > > > Enumerating entities > > > looking up device: 81:3 > > > looking up device: 81:4 > > > looking up device: 81:0 > > > looking up device: 81:1 > > > looking up device: 81:2 > > > looking up device: 81:5 > > > looking up device: 81:6 > > > Found 7 entities > > > Enumerating pads and links > > > Setting up format YUYV8_2X8 1920x1080 on pad rkisp1_isp/2 > > > Format set: YUYV8_2X8 640x480 > > > Setting up format YUYV8_2X8 640x480 on pad rkisp1_resizer_mainpath/0 > > > Format set: YUYV8_2X8 640x480 > > > > > > > > > It's my understanding that the ISP should be able to handle 1920x1080, > > > and the resizer sink should match the ISP source. > > > > > > With the pipeline improperly setup, the capture fails. > > > > > > > .../bindings/media/rockchip-isp1.yaml | 79 ++++++++++- > > > > .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++--- > > > > .../platform/rockchip/rkisp1/rkisp1-common.h | 32 +++++ > > > > .../platform/rockchip/rkisp1/rkisp1-debug.c | 14 +- > > > > .../platform/rockchip/rkisp1/rkisp1-dev.c | 67 +++++++-- > > > > .../platform/rockchip/rkisp1/rkisp1-isp.c | 128 +++++++++++++++++- > > > > .../platform/rockchip/rkisp1/rkisp1-regs.h | 90 ++++++++++++ > > > > .../platform/rockchip/rkisp1/rkisp1-resizer.c | 35 ++++- > > > > include/uapi/linux/rkisp1-config.h | 2 + > > > > 9 files changed, 509 insertions(+), 40 deletions(-) > > > > -- > > Regards, > > > > Laurent Pinchart
On Fri, Feb 24, 2023 at 12:24 PM Nicolas Dufresne <nicolas@ndufresne.ca> wrote: > > Hi Adam, > > Le jeudi 23 février 2023 à 10:10 -0600, Adam Ford a écrit : > > On Thu, Feb 23, 2023 at 8:26 AM Laurent Pinchart > > <laurent.pinchart@ideasonboard.com> wrote: > > > > > > Hi Adam, > > > > > > On Wed, Feb 22, 2023 at 05:39:30PM -0600, Adam Ford wrote: > > > > On Fri, Nov 18, 2022 at 3:44 AM Paul Elder wrote: > > > > > > > > > > This series depends on v3 of "dt-bindings: media: Add macros for video > > > > > interface bus types" [1]. > > > > > > > > > > This series extends the rkisp1 driver to support the ISP found in the > > > > > NXP i.MX8MP SoC. > > > > > > > > > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1") > > > > > and in the NXP i.MX8MP have the same origin, and have slightly diverged > > > > > over time as they are now independently developed (afaik) by Rockchip > > > > > and VeriSilicon. The latter is marketed under the name "ISP8000Nano", > > > > > and is close enough to the RK3399 ISP that it can easily be supported by > > > > > the same driver. > > > > > > > > > > The last two patches add support for UYVY output format, which can be > > > > > implemented on the ISP version in the i.MX8MP but not in the one in the > > > > > RK3399. > > > > > > > > > > This version of the series specifically has been tested on a Polyhex > > > > > Debix model A with an imx219 (Raspberry Pi cam v2). > > > > > > > > > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/ > > > > > > > > > > Laurent Pinchart (3): > > > > > dt-bindings: media: rkisp1: Add i.MX8MP ISP example > > > > > media: rkisp1: Add and use rkisp1_has_feature() macro > > > > > media: rkisp1: Configure gasket on i.MX8MP > > > > > > > > > > Paul Elder (11): > > > > > dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible > > > > > media: rkisp1: Add match data for i.MX8MP ISP > > > > > media: rkisp1: Add and set registers for crop for i.MX8MP > > > > > media: rkisp1: Add and set registers for output size config on i.MX8MP > > > > > media: rkisp1: Add i.MX8MP-specific registers for MI and resizer > > > > > media: rkisp1: Shift DMA buffer addresses on i.MX8MP > > > > > media: rkisp1: Add register definitions for the test pattern generator > > > > > media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP > > > > > media: rkisp1: Support devices without self path > > > > > media: rkisp1: Add YC swap capability > > > > > media: rkisp1: Add UYVY as an output format > > > > > > > > Paul / Laurent, > > > > > > > > I noticed an unexpected behaviour on the imx8mp. > > > > > > > > If I setup my pipeline for 640x480, it works just fine using an imx219 > > > > camera configured for SRGGB10_1X10. > > > > > > > > However, when I try to configure the pipeline to use the same camera > > > > at 1920x1080 (no resizing), the ISP source keeps defaulting to 640x480 > > > > > > > > Media device information > > > > ------------------------ > > > > driver rkisp1 > > > > model rkisp1 > > > > serial > > > > bus info platform:rkisp1 > > > > hw revision 0xe > > > > driver version 6.2.0 > > > > > > > > Device topology > > > > - entity 1: rkisp1_isp (4 pads, 4 links) > > > > type V4L2 subdev subtype Unknown flags 0 > > > > device node name /dev/v4l-subdev0 > > > > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480] > > > > > > You're cropping the image to 640x480 here. You need to set the crop > > > rectangle to 1920x1080. > > > > > > As Jacopo mentioned, I wouldn't recommend exercising the ISP directly. > > > Not only do you need to setup the pipeline, but you would also need to > > > implement all the imaging algorithms in userspace. libcamera will do all > > > this for you. > > > > I'll give that a try. My current employer has a v4l2src requirement, > > but I can likely make an argument to switch to libcamera. I didn't > > catch the cropping part. Thanks for that. > > I'd hope you can transparently replace v4l2src with libcamerasrc, the plugins > currently lives inside the libcamera project. If not, I'd really like to know > why. We can work together on adding missing controls (this is something I'm > starting on soon). I plan to give it a try. From what I've read it appears to be the right thing to do. I just need to carve out some time to get it installed. I mostly wanted to check out a camera adapter board my company made, test some updates I pushed for the imx219 on a second platform, and get more familiar with the ISP on the 8MP. I'll open a separate thread if I have questions on the cameralib. Thanks for all the feedback. I look forward to seeing this driver merged. adam > > regards, > Nicolas > > > > > adam > > > > > > > <- "csis-32e40000.csi":1 [ENABLED] > > > > pad1: Sink [fmt:unknown/0x0 field:none] > > > > <- "rkisp1_params":0 [ENABLED,IMMUTABLE] > > > > pad2: Source [fmt:YUYV8_2X8/640x480 field:none colorspace:raw xfer:none ycbcr:601 quantization:lim-range crop.bounds:(0,0)/640x480 crop:(0,0)/640x480] > > > > -> "rkisp1_resizer_mainpath":0 [ENABLED] > > > > pad3: Source [fmt:unknown/0x0 field:none] > > > > -> "rkisp1_stats":0 [ENABLED,IMMUTABLE] > > > > > > > > - entity 6: rkisp1_resizer_mainpath (2 pads, 2 links) > > > > type V4L2 subdev subtype Unknown flags 0 > > > > device node name /dev/v4l-subdev1 > > > > pad0: Sink [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/640x480] > > > > <- "rkisp1_isp":2 [ENABLED] > > > > pad1: Source [fmt:YUYV8_2X8/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range] > > > > -> "rkisp1_mainpath":0 [ENABLED,IMMUTABLE] > > > > > > > > - entity 9: rkisp1_mainpath (1 pad, 1 link) > > > > type Node subtype V4L flags 0 > > > > device node name /dev/video0 > > > > pad0: Sink > > > > <- "rkisp1_resizer_mainpath":1 [ENABLED,IMMUTABLE] > > > > > > > > - entity 13: rkisp1_stats (1 pad, 1 link) > > > > type Node subtype V4L flags 0 > > > > device node name /dev/video1 > > > > pad0: Sink > > > > <- "rkisp1_isp":3 [ENABLED,IMMUTABLE] > > > > > > > > - entity 17: rkisp1_params (1 pad, 1 link) > > > > type Node subtype V4L flags 0 > > > > device node name /dev/video2 > > > > pad0: Source > > > > -> "rkisp1_isp":1 [ENABLED,IMMUTABLE] > > > > > > > > - entity 29: csis-32e40000.csi (2 pads, 2 links) > > > > type V4L2 subdev subtype Unknown flags 0 > > > > device node name /dev/v4l-subdev2 > > > > pad0: Sink [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] > > > > <- "imx219 1-0010":0 [ENABLED] > > > > pad1: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range] > > > > -> "rkisp1_isp":0 [ENABLED] > > > > > > > > - entity 34: imx219 1-0010 (1 pad, 1 link) > > > > type V4L2 subdev subtype Sensor flags 0 > > > > device node name /dev/v4l-subdev3 > > > > pad0: Source [fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(8,8)/3280x2464 crop:(688,700)/1920x1080] > > > > -> "csis-32e40000.csi":0 [ENABLED] > > > > > > > > It's at this point that everything except the ISP source is 1920x1080. > > > > > > > > When I try to set the ISP sink to 1080, it ends up being 640x480 and > > > > the resizer sink is also changed to 640x480 > > > > > > > > root@beacon-imx8mp-kit:~# media-ctl -v -V "'rkisp1_isp':2 > > > > [fmt:YUYV8_2X8/1920x1080 field:none]" > > > > Opening media device /dev/media0 > > > > Enumerating entities > > > > looking up device: 81:3 > > > > looking up device: 81:4 > > > > looking up device: 81:0 > > > > looking up device: 81:1 > > > > looking up device: 81:2 > > > > looking up device: 81:5 > > > > looking up device: 81:6 > > > > Found 7 entities > > > > Enumerating pads and links > > > > Setting up format YUYV8_2X8 1920x1080 on pad rkisp1_isp/2 > > > > Format set: YUYV8_2X8 640x480 > > > > Setting up format YUYV8_2X8 640x480 on pad rkisp1_resizer_mainpath/0 > > > > Format set: YUYV8_2X8 640x480 > > > > > > > > > > > > It's my understanding that the ISP should be able to handle 1920x1080, > > > > and the resizer sink should match the ISP source. > > > > > > > > With the pipeline improperly setup, the capture fails. > > > > > > > > > .../bindings/media/rockchip-isp1.yaml | 79 ++++++++++- > > > > > .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++--- > > > > > .../platform/rockchip/rkisp1/rkisp1-common.h | 32 +++++ > > > > > .../platform/rockchip/rkisp1/rkisp1-debug.c | 14 +- > > > > > .../platform/rockchip/rkisp1/rkisp1-dev.c | 67 +++++++-- > > > > > .../platform/rockchip/rkisp1/rkisp1-isp.c | 128 +++++++++++++++++- > > > > > .../platform/rockchip/rkisp1/rkisp1-regs.h | 90 ++++++++++++ > > > > > .../platform/rockchip/rkisp1/rkisp1-resizer.c | 35 ++++- > > > > > include/uapi/linux/rkisp1-config.h | 2 + > > > > > 9 files changed, 509 insertions(+), 40 deletions(-) > > > > > > -- > > > Regards, > > > > > > Laurent Pinchart >
Hello Paul, On Fri, Nov 18, 2022 at 06:39:17PM +0900, Paul Elder wrote: > This series depends on v3 of "dt-bindings: media: Add macros for video > interface bus types" [1]. > > This series extends the rkisp1 driver to support the ISP found in the > NXP i.MX8MP SoC. > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1") > and in the NXP i.MX8MP have the same origin, and have slightly diverged > over time as they are now independently developed (afaik) by Rockchip > and VeriSilicon. The latter is marketed under the name "ISP8000Nano", > and is close enough to the RK3399 ISP that it can easily be supported by > the same driver. > > The last two patches add support for UYVY output format, which can be > implemented on the ISP version in the i.MX8MP but not in the one in the > RK3399. > > This version of the series specifically has been tested on a Polyhex > Debix model A with an imx219 (Raspberry Pi cam v2). > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/ I tested your series on imx274 on imx8mp-evk csi0. All looks good on my side. Thanks for your work! Tested-by: Tommaso Merciai <tomm.merciai@gmail.com> Regards, Tommaso > > Laurent Pinchart (3): > dt-bindings: media: rkisp1: Add i.MX8MP ISP example > media: rkisp1: Add and use rkisp1_has_feature() macro > media: rkisp1: Configure gasket on i.MX8MP > > Paul Elder (11): > dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible > media: rkisp1: Add match data for i.MX8MP ISP > media: rkisp1: Add and set registers for crop for i.MX8MP > media: rkisp1: Add and set registers for output size config on i.MX8MP > media: rkisp1: Add i.MX8MP-specific registers for MI and resizer > media: rkisp1: Shift DMA buffer addresses on i.MX8MP > media: rkisp1: Add register definitions for the test pattern generator > media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP > media: rkisp1: Support devices without self path > media: rkisp1: Add YC swap capability > media: rkisp1: Add UYVY as an output format > > .../bindings/media/rockchip-isp1.yaml | 79 ++++++++++- > .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++--- > .../platform/rockchip/rkisp1/rkisp1-common.h | 32 +++++ > .../platform/rockchip/rkisp1/rkisp1-debug.c | 14 +- > .../platform/rockchip/rkisp1/rkisp1-dev.c | 67 +++++++-- > .../platform/rockchip/rkisp1/rkisp1-isp.c | 128 +++++++++++++++++- > .../platform/rockchip/rkisp1/rkisp1-regs.h | 90 ++++++++++++ > .../platform/rockchip/rkisp1/rkisp1-resizer.c | 35 ++++- > include/uapi/linux/rkisp1-config.h | 2 + > 9 files changed, 509 insertions(+), 40 deletions(-) > > -- > 2.35.1 >
Hi Paul, On 18/11/2022 10:39, Paul Elder wrote: > This series depends on v3 of "dt-bindings: media: Add macros for video > interface bus types" [1]. > > This series extends the rkisp1 driver to support the ISP found in the > NXP i.MX8MP SoC. > > The ISP IP cores in the Rockchip RK3399 (known as the "Rockchip ISP1") > and in the NXP i.MX8MP have the same origin, and have slightly diverged > over time as they are now independently developed (afaik) by Rockchip > and VeriSilicon. The latter is marketed under the name "ISP8000Nano", > and is close enough to the RK3399 ISP that it can easily be supported by > the same driver. > > The last two patches add support for UYVY output format, which can be > implemented on the ISP version in the i.MX8MP but not in the one in the > RK3399. > > This version of the series specifically has been tested on a Polyhex > Debix model A with an imx219 (Raspberry Pi cam v2). There were comments for the first few patches, but I haven't seen a v4. I'm marking this series as 'Changes Requested' in patchwork, just so you know. Regards, Hans > > [1] https://lore.kernel.org/linux-media/20220615221410.27459-2-laurent.pinchart@ideasonboard.com/ > > Laurent Pinchart (3): > dt-bindings: media: rkisp1: Add i.MX8MP ISP example > media: rkisp1: Add and use rkisp1_has_feature() macro > media: rkisp1: Configure gasket on i.MX8MP > > Paul Elder (11): > dt-bindings: media: rkisp1: Add i.MX8MP ISP to compatible > media: rkisp1: Add match data for i.MX8MP ISP > media: rkisp1: Add and set registers for crop for i.MX8MP > media: rkisp1: Add and set registers for output size config on i.MX8MP > media: rkisp1: Add i.MX8MP-specific registers for MI and resizer > media: rkisp1: Shift DMA buffer addresses on i.MX8MP > media: rkisp1: Add register definitions for the test pattern generator > media: rkisp1: Fix RSZ_CTRL bits for i.MX8MP > media: rkisp1: Support devices without self path > media: rkisp1: Add YC swap capability > media: rkisp1: Add UYVY as an output format > > .../bindings/media/rockchip-isp1.yaml | 79 ++++++++++- > .../platform/rockchip/rkisp1/rkisp1-capture.c | 102 +++++++++++--- > .../platform/rockchip/rkisp1/rkisp1-common.h | 32 +++++ > .../platform/rockchip/rkisp1/rkisp1-debug.c | 14 +- > .../platform/rockchip/rkisp1/rkisp1-dev.c | 67 +++++++-- > .../platform/rockchip/rkisp1/rkisp1-isp.c | 128 +++++++++++++++++- > .../platform/rockchip/rkisp1/rkisp1-regs.h | 90 ++++++++++++ > .../platform/rockchip/rkisp1/rkisp1-resizer.c | 35 ++++- > include/uapi/linux/rkisp1-config.h | 2 + > 9 files changed, 509 insertions(+), 40 deletions(-) >