mbox series

[v5,00/16] Add Arm Mali-C55 Image Signal Processor Driver

Message ID 20240529152858.183799-1-dan.scally@ideasonboard.com (mailing list archive)
Headers show
Series Add Arm Mali-C55 Image Signal Processor Driver | expand

Message

Dan Scally May 29, 2024, 3:28 p.m. UTC
Hello all

This patchset introduces a driver for Arm's Mali-C55 Image Signal Processor.
The driver uses the V4L2 / media controller API and implements both of the ISP's
capture pipelines allowing a range of output formats plus downscaling and
cropping. The capture pipelines are named "Full resolution" and "Downscale" and
so abbreviated FR and DS throughout the driver.

The driver exposes 4 V4L2 subdevices:

- mali-c55 isp: input data formatting
- mali-c55 tpg: test pattern generator (modeled as a camera sensor entity)
- mali-c55 resizer fr: downscale / crop and format setting for the FR pipe
- mali-c55 resizer ds: downscale / crop and format setting for the DS pipe

Along with 4 V4L2 Video devices:

- mali-c55 fr: Capture device for the full resolution pipe
- mali-c55 ds: Capture device for the downscale pipe
- mali-c55 3a stats: Capture device for statistics to support 3A algorithms
- mali-c55 3a params: Output device for parameter buffers to configure the ISP

Support is implemented in the parameters video device code for many of the ISP'S
hardware blocks, but not yet all of them. The buffer format is (as far as I am
aware anyway) a novel design that we intend to be extensible so that support for
the C55's remaining hardware blocks can be added later.

Patches 1, 4, 5, 6 and 7 have already had 4 versions on the mailing list...I
decided to post the additional work on the driver as extra patches rather than
merge them all into the existing series as it's already a lot of code to review
and I hoped that that might make it a little easier...if I'm wrong and that's
not liked I can just squash them into a much smaller series.

Thanks
Dan

Daniel Scally (15):
  media: uapi: Add MEDIA_BUS_FMT_RGB202020_1X60 format code
  media: uapi: Add 20-bit bayer formats
  media: v4l2-common: Add RAW16 format info
  dt-bindings: media: Add bindings for ARM mali-c55
  media: mali-c55: Add Mali-C55 ISP driver
  media: Documentation: Add Mali-C55 ISP Documentation
  MAINTAINERS: Add entry for mali-c55 driver
  media: Add MALI_C55_3A_STATS meta format
  media: uapi: Add 3a stats buffer for mali-c55
  media: platform: Add mali-c55 3a stats devnode
  media: platform: Fill stats buffer on ISP_START
  Documentation: mali-c55: Add Statistics documentation
  media: uapi: Add parameters structs to mali-c55-config.h
  media: platform: Add mali-c55 parameters video node
  Documentation: mali-c55: Document the mali-c55 parameter setting

Jacopo Mondi (1):
  media: mali-c55: Add image formats for Mali-C55 parameters buffer

 .../admin-guide/media/mali-c55-graph.dot      |  19 +
 Documentation/admin-guide/media/mali-c55.rst  | 406 ++++++++
 .../admin-guide/media/v4l-drivers.rst         |   1 +
 .../bindings/media/arm,mali-c55.yaml          |  66 ++
 .../userspace-api/media/v4l/meta-formats.rst  |   1 +
 .../media/v4l/metafmt-arm-mali-c55.rst        |  87 ++
 .../media/v4l/subdev-formats.rst              | 268 +++++
 MAINTAINERS                                   |  10 +
 drivers/media/platform/Kconfig                |   1 +
 drivers/media/platform/Makefile               |   1 +
 drivers/media/platform/arm/Kconfig            |   5 +
 drivers/media/platform/arm/Makefile           |   2 +
 drivers/media/platform/arm/mali-c55/Kconfig   |  18 +
 drivers/media/platform/arm/mali-c55/Makefile  |  11 +
 .../platform/arm/mali-c55/mali-c55-capture.c  | 951 ++++++++++++++++++
 .../platform/arm/mali-c55/mali-c55-common.h   | 312 ++++++
 .../platform/arm/mali-c55/mali-c55-core.c     | 825 +++++++++++++++
 .../platform/arm/mali-c55/mali-c55-isp.c      | 626 ++++++++++++
 .../platform/arm/mali-c55/mali-c55-params.c   | 615 +++++++++++
 .../arm/mali-c55/mali-c55-registers.h         | 365 +++++++
 .../arm/mali-c55/mali-c55-resizer-coefs.h     | 382 +++++++
 .../platform/arm/mali-c55/mali-c55-resizer.c  | 779 ++++++++++++++
 .../platform/arm/mali-c55/mali-c55-stats.c    | 350 +++++++
 .../platform/arm/mali-c55/mali-c55-tpg.c      | 402 ++++++++
 drivers/media/v4l2-core/v4l2-common.c         |   4 +
 drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
 include/uapi/linux/media-bus-format.h         |   9 +-
 .../uapi/linux/media/arm/mali-c55-config.h    | 851 ++++++++++++++++
 include/uapi/linux/videodev2.h                |   3 +
 29 files changed, 7370 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/admin-guide/media/mali-c55-graph.dot
 create mode 100644 Documentation/admin-guide/media/mali-c55.rst
 create mode 100644 Documentation/devicetree/bindings/media/arm,mali-c55.yaml
 create mode 100644 Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
 create mode 100644 drivers/media/platform/arm/Kconfig
 create mode 100644 drivers/media/platform/arm/Makefile
 create mode 100644 drivers/media/platform/arm/mali-c55/Kconfig
 create mode 100644 drivers/media/platform/arm/mali-c55/Makefile
 create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-capture.c
 create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-common.h
 create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-core.c
 create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-isp.c
 create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-params.c
 create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-registers.h
 create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-resizer-coefs.h
 create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
 create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-stats.c
 create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
 create mode 100644 include/uapi/linux/media/arm/mali-c55-config.h

Comments

Laurent Pinchart May 29, 2024, 11:27 p.m. UTC | #1
Hi Dan,

Thank you for the patches.

On Wed, May 29, 2024 at 04:28:42PM +0100, Daniel Scally wrote:
> Hello all
> 
> This patchset introduces a driver for Arm's Mali-C55 Image Signal Processor.
> The driver uses the V4L2 / media controller API and implements both of the ISP's
> capture pipelines allowing a range of output formats plus downscaling and
> cropping. The capture pipelines are named "Full resolution" and "Downscale" and
> so abbreviated FR and DS throughout the driver.
> 
> The driver exposes 4 V4L2 subdevices:
> 
> - mali-c55 isp: input data formatting
> - mali-c55 tpg: test pattern generator (modeled as a camera sensor entity)
> - mali-c55 resizer fr: downscale / crop and format setting for the FR pipe
> - mali-c55 resizer ds: downscale / crop and format setting for the DS pipe
> 
> Along with 4 V4L2 Video devices:
> 
> - mali-c55 fr: Capture device for the full resolution pipe
> - mali-c55 ds: Capture device for the downscale pipe
> - mali-c55 3a stats: Capture device for statistics to support 3A algorithms
> - mali-c55 3a params: Output device for parameter buffers to configure the ISP
> 
> Support is implemented in the parameters video device code for many of the ISP'S
> hardware blocks, but not yet all of them. The buffer format is (as far as I am
> aware anyway) a novel design that we intend to be extensible so that support for
> the C55's remaining hardware blocks can be added later.
> 
> Patches 1, 4, 5, 6 and 7 have already had 4 versions on the mailing list...I
> decided to post the additional work on the driver as extra patches rather than
> merge them all into the existing series as it's already a lot of code to review
> and I hoped that that might make it a little easier...if I'm wrong and that's
> not liked I can just squash them into a much smaller series.

Could you please include the v4l2-compliance report in v6 ? Make sure to
use the very latest version of v4l-utils, compiled from the master
branch.

> Daniel Scally (15):
>   media: uapi: Add MEDIA_BUS_FMT_RGB202020_1X60 format code
>   media: uapi: Add 20-bit bayer formats
>   media: v4l2-common: Add RAW16 format info
>   dt-bindings: media: Add bindings for ARM mali-c55
>   media: mali-c55: Add Mali-C55 ISP driver
>   media: Documentation: Add Mali-C55 ISP Documentation
>   MAINTAINERS: Add entry for mali-c55 driver
>   media: Add MALI_C55_3A_STATS meta format
>   media: uapi: Add 3a stats buffer for mali-c55
>   media: platform: Add mali-c55 3a stats devnode
>   media: platform: Fill stats buffer on ISP_START
>   Documentation: mali-c55: Add Statistics documentation
>   media: uapi: Add parameters structs to mali-c55-config.h
>   media: platform: Add mali-c55 parameters video node
>   Documentation: mali-c55: Document the mali-c55 parameter setting
> 
> Jacopo Mondi (1):
>   media: mali-c55: Add image formats for Mali-C55 parameters buffer
> 
>  .../admin-guide/media/mali-c55-graph.dot      |  19 +
>  Documentation/admin-guide/media/mali-c55.rst  | 406 ++++++++
>  .../admin-guide/media/v4l-drivers.rst         |   1 +
>  .../bindings/media/arm,mali-c55.yaml          |  66 ++
>  .../userspace-api/media/v4l/meta-formats.rst  |   1 +
>  .../media/v4l/metafmt-arm-mali-c55.rst        |  87 ++
>  .../media/v4l/subdev-formats.rst              | 268 +++++
>  MAINTAINERS                                   |  10 +
>  drivers/media/platform/Kconfig                |   1 +
>  drivers/media/platform/Makefile               |   1 +
>  drivers/media/platform/arm/Kconfig            |   5 +
>  drivers/media/platform/arm/Makefile           |   2 +
>  drivers/media/platform/arm/mali-c55/Kconfig   |  18 +
>  drivers/media/platform/arm/mali-c55/Makefile  |  11 +
>  .../platform/arm/mali-c55/mali-c55-capture.c  | 951 ++++++++++++++++++
>  .../platform/arm/mali-c55/mali-c55-common.h   | 312 ++++++
>  .../platform/arm/mali-c55/mali-c55-core.c     | 825 +++++++++++++++
>  .../platform/arm/mali-c55/mali-c55-isp.c      | 626 ++++++++++++
>  .../platform/arm/mali-c55/mali-c55-params.c   | 615 +++++++++++
>  .../arm/mali-c55/mali-c55-registers.h         | 365 +++++++
>  .../arm/mali-c55/mali-c55-resizer-coefs.h     | 382 +++++++
>  .../platform/arm/mali-c55/mali-c55-resizer.c  | 779 ++++++++++++++
>  .../platform/arm/mali-c55/mali-c55-stats.c    | 350 +++++++
>  .../platform/arm/mali-c55/mali-c55-tpg.c      | 402 ++++++++
>  drivers/media/v4l2-core/v4l2-common.c         |   4 +
>  drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
>  include/uapi/linux/media-bus-format.h         |   9 +-
>  .../uapi/linux/media/arm/mali-c55-config.h    | 851 ++++++++++++++++
>  include/uapi/linux/videodev2.h                |   3 +
>  29 files changed, 7370 insertions(+), 2 deletions(-)
>  create mode 100644 Documentation/admin-guide/media/mali-c55-graph.dot
>  create mode 100644 Documentation/admin-guide/media/mali-c55.rst
>  create mode 100644 Documentation/devicetree/bindings/media/arm,mali-c55.yaml
>  create mode 100644 Documentation/userspace-api/media/v4l/metafmt-arm-mali-c55.rst
>  create mode 100644 drivers/media/platform/arm/Kconfig
>  create mode 100644 drivers/media/platform/arm/Makefile
>  create mode 100644 drivers/media/platform/arm/mali-c55/Kconfig
>  create mode 100644 drivers/media/platform/arm/mali-c55/Makefile
>  create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-capture.c
>  create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-common.h
>  create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-core.c
>  create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-isp.c
>  create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-params.c
>  create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-registers.h
>  create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-resizer-coefs.h
>  create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-resizer.c
>  create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-stats.c
>  create mode 100644 drivers/media/platform/arm/mali-c55/mali-c55-tpg.c
>  create mode 100644 include/uapi/linux/media/arm/mali-c55-config.h