| Message ID | 20240416193319.778192-8-sakari.ailus@linux.intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Generic line based metadata support, internal pads | expand |
Hi Sakari, Thank you for the patch. On Tue, Apr 16, 2024 at 10:32:40PM +0300, Sakari Ailus wrote: > Many camera sensors, among other devices, transmit embedded data and image > data for each CSI-2 frame. This embedded data typically contains register > configuration of the sensor that has been used to capture the image data > of the same frame. > > The embedded data is received by the CSI-2 receiver and has the same > properties as the image data, including that it is line based: it has > width, height and bytesperline (stride). > > Add these fields to struct v4l2_meta_format and document them. > > Also add V4L2_FMT_FLAG_META_LINE_BASED to tell a given format is > line-based i.e. these fields of struct v4l2_meta_format are valid for it. > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com> > --- > .../userspace-api/media/v4l/dev-meta.rst | 21 +++++++++++++++++++ > .../media/v4l/vidioc-enum-fmt.rst | 7 +++++++ > .../media/videodev2.h.rst.exceptions | 1 + > drivers/media/v4l2-core/v4l2-ioctl.c | 5 +++-- > include/uapi/linux/videodev2.h | 10 +++++++++ > 5 files changed, 42 insertions(+), 2 deletions(-) > > diff --git a/Documentation/userspace-api/media/v4l/dev-meta.rst b/Documentation/userspace-api/media/v4l/dev-meta.rst > index 0e7e1ee1471a..5eee9ab60395 100644 > --- a/Documentation/userspace-api/media/v4l/dev-meta.rst > +++ b/Documentation/userspace-api/media/v4l/dev-meta.rst > @@ -47,6 +47,12 @@ member of the ``fmt`` union as needed per the desired operation. Both drivers > and applications must set the remainder of the :c:type:`v4l2_format` structure > to 0. > > +Devices that capture metadata by line have the struct v4l2_fmtdesc > +``V4L2_FMT_FLAG_META_LINE_BASED`` flag set for :c:func:`VIDIOC_ENUM_FMT`. Such > +devices can typically also :ref:`capture image data <capture>`. This primarily > +involves devices that receive the data from a different devices such as a camera > +sensor. > + > .. c:type:: v4l2_meta_format > > .. tabularcolumns:: |p{1.4cm}|p{2.4cm}|p{13.5cm}| > @@ -65,3 +71,18 @@ to 0. > - ``buffersize`` > - Maximum buffer size in bytes required for data. The value is set by the > driver. > + * - __u32 > + - ``width`` > + - Width of a line of metadata in Data Units. Valid when > + :c:type`v4l2_fmtdesc` flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, > + otherwise zero. See :c:func:`VIDIOC_ENUM_FMT`. > + * - __u32 > + - ``height`` > + - Number of rows of metadata. Valid when :c:type`v4l2_fmtdesc` flag > + ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, otherwise zero. See > + :c:func:`VIDIOC_ENUM_FMT`. > + * - __u32 > + - ``bytesperline`` > + - Offset in bytes between the beginning of two consecutive lines. Valid > + when :c:type`v4l2_fmtdesc` flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is > + set, otherwise zero. See :c:func:`VIDIOC_ENUM_FMT`. > diff --git a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst > index 000c154b0f98..a439be1b15d1 100644 > --- a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst > +++ b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst > @@ -227,6 +227,13 @@ the ``mbus_code`` field is handled differently: > The application can ask to configure the quantization of the capture > device when calling the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl with > :ref:`V4L2_PIX_FMT_FLAG_SET_CSC <v4l2-pix-fmt-flag-set-csc>` set. > + * - ``V4L2_FMT_FLAG_META_LINE_BASED`` > + - 0x0200 > + - The metadata format is line-based. In this case the ``width``, > + ``height`` and ``bytesperline`` fields of :c:type:`v4l2_meta_format` are > + valid. The buffer consists of ``height`` lines, each having ``width`` > + Data Units of data and offset (in bytes) between the beginning of each s/and offset/, and the offset/ > + two consecutive lines is ``bytesperline``. > > Return Value > ============ > diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions b/Documentation/userspace-api/media/videodev2.h.rst.exceptions > index 3e58aac4ef0b..bdc628e8c1d6 100644 > --- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions > +++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions > @@ -215,6 +215,7 @@ replace define V4L2_FMT_FLAG_CSC_XFER_FUNC fmtdesc-flags > replace define V4L2_FMT_FLAG_CSC_YCBCR_ENC fmtdesc-flags > replace define V4L2_FMT_FLAG_CSC_HSV_ENC fmtdesc-flags > replace define V4L2_FMT_FLAG_CSC_QUANTIZATION fmtdesc-flags > +replace define V4L2_FMT_FLAG_META_LINE_BASED fmtdesc-flags > > # V4L2 timecode types > replace define V4L2_TC_TYPE_24FPS timecode-type > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c > index ae2dca7f2817..2cfc9106857a 100644 > --- a/drivers/media/v4l2-core/v4l2-ioctl.c > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c > @@ -343,8 +343,9 @@ static void v4l_print_format(const void *arg, bool write_only) > case V4L2_BUF_TYPE_META_OUTPUT: > meta = &p->fmt.meta; > pixelformat = meta->dataformat; > - pr_cont(", dataformat=%p4cc, buffersize=%u\n", > - &pixelformat, meta->buffersize); > + pr_cont(", dataformat=%p4cc, buffersize=%u, width=%u, height=%u, bytesperline=%u\n", > + &pixelformat, meta->buffersize, meta->width, > + meta->height, meta->bytesperline); > break; > } > } > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > index c7cf20b5da67..37112dfebd0c 100644 > --- a/include/uapi/linux/videodev2.h > +++ b/include/uapi/linux/videodev2.h > @@ -877,6 +877,7 @@ struct v4l2_fmtdesc { > #define V4L2_FMT_FLAG_CSC_YCBCR_ENC 0x0080 > #define V4L2_FMT_FLAG_CSC_HSV_ENC V4L2_FMT_FLAG_CSC_YCBCR_ENC > #define V4L2_FMT_FLAG_CSC_QUANTIZATION 0x0100 > +#define V4L2_FMT_FLAG_META_LINE_BASED 0x0200 Could the V4L2_FMT_FLAG_META_LINE_BASED flag be set by the V4L2 core for line-based metadata formats, instead of requiring drivers to set it ? It would ensure consistency. > > /* Frame Size and frame rate enumeration */ > /* > @@ -2424,10 +2425,19 @@ struct v4l2_sdr_format { > * struct v4l2_meta_format - metadata format definition > * @dataformat: little endian four character code (fourcc) > * @buffersize: maximum size in bytes required for data > + * @width: number of data units of data per line (valid for line > + * based formats only, see format documentation) > + * @height: number of lines of data per buffer (valid for line based > + * formats only) > + * @bytesperline: offset between the beginnings of two adjacent lines in > + * bytes (valid for line based formats only) > */ > struct v4l2_meta_format { > __u32 dataformat; > __u32 buffersize; > + __u32 width; > + __u32 height; > + __u32 bytesperline; > } __attribute__ ((packed)); > > /**
Hi Laurent, On Fri, Apr 19, 2024 at 07:30:09PM +0300, Laurent Pinchart wrote: > Hi Sakari, > > Thank you for the patch. Thanks for the review. > > On Tue, Apr 16, 2024 at 10:32:40PM +0300, Sakari Ailus wrote: > > Many camera sensors, among other devices, transmit embedded data and image > > data for each CSI-2 frame. This embedded data typically contains register > > configuration of the sensor that has been used to capture the image data > > of the same frame. > > > > The embedded data is received by the CSI-2 receiver and has the same > > properties as the image data, including that it is line based: it has > > width, height and bytesperline (stride). > > > > Add these fields to struct v4l2_meta_format and document them. > > > > Also add V4L2_FMT_FLAG_META_LINE_BASED to tell a given format is > > line-based i.e. these fields of struct v4l2_meta_format are valid for it. > > > > Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com> > > --- > > .../userspace-api/media/v4l/dev-meta.rst | 21 +++++++++++++++++++ > > .../media/v4l/vidioc-enum-fmt.rst | 7 +++++++ > > .../media/videodev2.h.rst.exceptions | 1 + > > drivers/media/v4l2-core/v4l2-ioctl.c | 5 +++-- > > include/uapi/linux/videodev2.h | 10 +++++++++ > > 5 files changed, 42 insertions(+), 2 deletions(-) > > > > diff --git a/Documentation/userspace-api/media/v4l/dev-meta.rst b/Documentation/userspace-api/media/v4l/dev-meta.rst > > index 0e7e1ee1471a..5eee9ab60395 100644 > > --- a/Documentation/userspace-api/media/v4l/dev-meta.rst > > +++ b/Documentation/userspace-api/media/v4l/dev-meta.rst > > @@ -47,6 +47,12 @@ member of the ``fmt`` union as needed per the desired operation. Both drivers > > and applications must set the remainder of the :c:type:`v4l2_format` structure > > to 0. > > > > +Devices that capture metadata by line have the struct v4l2_fmtdesc > > +``V4L2_FMT_FLAG_META_LINE_BASED`` flag set for :c:func:`VIDIOC_ENUM_FMT`. Such > > +devices can typically also :ref:`capture image data <capture>`. This primarily > > +involves devices that receive the data from a different devices such as a camera > > +sensor. > > + > > .. c:type:: v4l2_meta_format > > > > .. tabularcolumns:: |p{1.4cm}|p{2.4cm}|p{13.5cm}| > > @@ -65,3 +71,18 @@ to 0. > > - ``buffersize`` > > - Maximum buffer size in bytes required for data. The value is set by the > > driver. > > + * - __u32 > > + - ``width`` > > + - Width of a line of metadata in Data Units. Valid when > > + :c:type`v4l2_fmtdesc` flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, > > + otherwise zero. See :c:func:`VIDIOC_ENUM_FMT`. > > + * - __u32 > > + - ``height`` > > + - Number of rows of metadata. Valid when :c:type`v4l2_fmtdesc` flag > > + ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, otherwise zero. See > > + :c:func:`VIDIOC_ENUM_FMT`. > > + * - __u32 > > + - ``bytesperline`` > > + - Offset in bytes between the beginning of two consecutive lines. Valid > > + when :c:type`v4l2_fmtdesc` flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is > > + set, otherwise zero. See :c:func:`VIDIOC_ENUM_FMT`. > > diff --git a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst > > index 000c154b0f98..a439be1b15d1 100644 > > --- a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst > > +++ b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst > > @@ -227,6 +227,13 @@ the ``mbus_code`` field is handled differently: > > The application can ask to configure the quantization of the capture > > device when calling the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl with > > :ref:`V4L2_PIX_FMT_FLAG_SET_CSC <v4l2-pix-fmt-flag-set-csc>` set. > > + * - ``V4L2_FMT_FLAG_META_LINE_BASED`` > > + - 0x0200 > > + - The metadata format is line-based. In this case the ``width``, > > + ``height`` and ``bytesperline`` fields of :c:type:`v4l2_meta_format` are > > + valid. The buffer consists of ``height`` lines, each having ``width`` > > + Data Units of data and offset (in bytes) between the beginning of each > > s/and offset/, and the offset/ Sounds good. > > > + two consecutive lines is ``bytesperline``. > > > > Return Value > > ============ > > diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions b/Documentation/userspace-api/media/videodev2.h.rst.exceptions > > index 3e58aac4ef0b..bdc628e8c1d6 100644 > > --- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions > > +++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions > > @@ -215,6 +215,7 @@ replace define V4L2_FMT_FLAG_CSC_XFER_FUNC fmtdesc-flags > > replace define V4L2_FMT_FLAG_CSC_YCBCR_ENC fmtdesc-flags > > replace define V4L2_FMT_FLAG_CSC_HSV_ENC fmtdesc-flags > > replace define V4L2_FMT_FLAG_CSC_QUANTIZATION fmtdesc-flags > > +replace define V4L2_FMT_FLAG_META_LINE_BASED fmtdesc-flags > > > > # V4L2 timecode types > > replace define V4L2_TC_TYPE_24FPS timecode-type > > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c > > index ae2dca7f2817..2cfc9106857a 100644 > > --- a/drivers/media/v4l2-core/v4l2-ioctl.c > > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c > > @@ -343,8 +343,9 @@ static void v4l_print_format(const void *arg, bool write_only) > > case V4L2_BUF_TYPE_META_OUTPUT: > > meta = &p->fmt.meta; > > pixelformat = meta->dataformat; > > - pr_cont(", dataformat=%p4cc, buffersize=%u\n", > > - &pixelformat, meta->buffersize); > > + pr_cont(", dataformat=%p4cc, buffersize=%u, width=%u, height=%u, bytesperline=%u\n", > > + &pixelformat, meta->buffersize, meta->width, > > + meta->height, meta->bytesperline); > > break; > > } > > } > > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h > > index c7cf20b5da67..37112dfebd0c 100644 > > --- a/include/uapi/linux/videodev2.h > > +++ b/include/uapi/linux/videodev2.h > > @@ -877,6 +877,7 @@ struct v4l2_fmtdesc { > > #define V4L2_FMT_FLAG_CSC_YCBCR_ENC 0x0080 > > #define V4L2_FMT_FLAG_CSC_HSV_ENC V4L2_FMT_FLAG_CSC_YCBCR_ENC > > #define V4L2_FMT_FLAG_CSC_QUANTIZATION 0x0100 > > +#define V4L2_FMT_FLAG_META_LINE_BASED 0x0200 > > Could the V4L2_FMT_FLAG_META_LINE_BASED flag be set by the V4L2 core for > line-based metadata formats, instead of requiring drivers to set it ? It > would ensure consistency. This would essentially need to be set after driver callbacks dealing with a format-related IOCTL has returned, drivers don't need this information as such. I can add a separate patch for it. > > > > > /* Frame Size and frame rate enumeration */ > > /* > > @@ -2424,10 +2425,19 @@ struct v4l2_sdr_format { > > * struct v4l2_meta_format - metadata format definition > > * @dataformat: little endian four character code (fourcc) > > * @buffersize: maximum size in bytes required for data > > + * @width: number of data units of data per line (valid for line > > + * based formats only, see format documentation) > > + * @height: number of lines of data per buffer (valid for line based > > + * formats only) > > + * @bytesperline: offset between the beginnings of two adjacent lines in > > + * bytes (valid for line based formats only) > > */ > > struct v4l2_meta_format { > > __u32 dataformat; > > __u32 buffersize; > > + __u32 width; > > + __u32 height; > > + __u32 bytesperline; > > } __attribute__ ((packed)); > > > > /** >
diff --git a/Documentation/userspace-api/media/v4l/dev-meta.rst b/Documentation/userspace-api/media/v4l/dev-meta.rst index 0e7e1ee1471a..5eee9ab60395 100644 --- a/Documentation/userspace-api/media/v4l/dev-meta.rst +++ b/Documentation/userspace-api/media/v4l/dev-meta.rst @@ -47,6 +47,12 @@ member of the ``fmt`` union as needed per the desired operation. Both drivers and applications must set the remainder of the :c:type:`v4l2_format` structure to 0. +Devices that capture metadata by line have the struct v4l2_fmtdesc +``V4L2_FMT_FLAG_META_LINE_BASED`` flag set for :c:func:`VIDIOC_ENUM_FMT`. Such +devices can typically also :ref:`capture image data <capture>`. This primarily +involves devices that receive the data from a different devices such as a camera +sensor. + .. c:type:: v4l2_meta_format .. tabularcolumns:: |p{1.4cm}|p{2.4cm}|p{13.5cm}| @@ -65,3 +71,18 @@ to 0. - ``buffersize`` - Maximum buffer size in bytes required for data. The value is set by the driver. + * - __u32 + - ``width`` + - Width of a line of metadata in Data Units. Valid when + :c:type`v4l2_fmtdesc` flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, + otherwise zero. See :c:func:`VIDIOC_ENUM_FMT`. + * - __u32 + - ``height`` + - Number of rows of metadata. Valid when :c:type`v4l2_fmtdesc` flag + ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, otherwise zero. See + :c:func:`VIDIOC_ENUM_FMT`. + * - __u32 + - ``bytesperline`` + - Offset in bytes between the beginning of two consecutive lines. Valid + when :c:type`v4l2_fmtdesc` flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is + set, otherwise zero. See :c:func:`VIDIOC_ENUM_FMT`. diff --git a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst index 000c154b0f98..a439be1b15d1 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst @@ -227,6 +227,13 @@ the ``mbus_code`` field is handled differently: The application can ask to configure the quantization of the capture device when calling the :ref:`VIDIOC_S_FMT <VIDIOC_G_FMT>` ioctl with :ref:`V4L2_PIX_FMT_FLAG_SET_CSC <v4l2-pix-fmt-flag-set-csc>` set. + * - ``V4L2_FMT_FLAG_META_LINE_BASED`` + - 0x0200 + - The metadata format is line-based. In this case the ``width``, + ``height`` and ``bytesperline`` fields of :c:type:`v4l2_meta_format` are + valid. The buffer consists of ``height`` lines, each having ``width`` + Data Units of data and offset (in bytes) between the beginning of each + two consecutive lines is ``bytesperline``. Return Value ============ diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions b/Documentation/userspace-api/media/videodev2.h.rst.exceptions index 3e58aac4ef0b..bdc628e8c1d6 100644 --- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions +++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions @@ -215,6 +215,7 @@ replace define V4L2_FMT_FLAG_CSC_XFER_FUNC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_YCBCR_ENC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_HSV_ENC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_QUANTIZATION fmtdesc-flags +replace define V4L2_FMT_FLAG_META_LINE_BASED fmtdesc-flags # V4L2 timecode types replace define V4L2_TC_TYPE_24FPS timecode-type diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index ae2dca7f2817..2cfc9106857a 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -343,8 +343,9 @@ static void v4l_print_format(const void *arg, bool write_only) case V4L2_BUF_TYPE_META_OUTPUT: meta = &p->fmt.meta; pixelformat = meta->dataformat; - pr_cont(", dataformat=%p4cc, buffersize=%u\n", - &pixelformat, meta->buffersize); + pr_cont(", dataformat=%p4cc, buffersize=%u, width=%u, height=%u, bytesperline=%u\n", + &pixelformat, meta->buffersize, meta->width, + meta->height, meta->bytesperline); break; } } diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index c7cf20b5da67..37112dfebd0c 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -877,6 +877,7 @@ struct v4l2_fmtdesc { #define V4L2_FMT_FLAG_CSC_YCBCR_ENC 0x0080 #define V4L2_FMT_FLAG_CSC_HSV_ENC V4L2_FMT_FLAG_CSC_YCBCR_ENC #define V4L2_FMT_FLAG_CSC_QUANTIZATION 0x0100 +#define V4L2_FMT_FLAG_META_LINE_BASED 0x0200 /* Frame Size and frame rate enumeration */ /* @@ -2424,10 +2425,19 @@ struct v4l2_sdr_format { * struct v4l2_meta_format - metadata format definition * @dataformat: little endian four character code (fourcc) * @buffersize: maximum size in bytes required for data + * @width: number of data units of data per line (valid for line + * based formats only, see format documentation) + * @height: number of lines of data per buffer (valid for line based + * formats only) + * @bytesperline: offset between the beginnings of two adjacent lines in + * bytes (valid for line based formats only) */ struct v4l2_meta_format { __u32 dataformat; __u32 buffersize; + __u32 width; + __u32 height; + __u32 bytesperline; } __attribute__ ((packed)); /**
Many camera sensors, among other devices, transmit embedded data and image data for each CSI-2 frame. This embedded data typically contains register configuration of the sensor that has been used to capture the image data of the same frame. The embedded data is received by the CSI-2 receiver and has the same properties as the image data, including that it is line based: it has width, height and bytesperline (stride). Add these fields to struct v4l2_meta_format and document them. Also add V4L2_FMT_FLAG_META_LINE_BASED to tell a given format is line-based i.e. these fields of struct v4l2_meta_format are valid for it. Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com> --- .../userspace-api/media/v4l/dev-meta.rst | 21 +++++++++++++++++++ .../media/v4l/vidioc-enum-fmt.rst | 7 +++++++ .../media/videodev2.h.rst.exceptions | 1 + drivers/media/v4l2-core/v4l2-ioctl.c | 5 +++-- include/uapi/linux/videodev2.h | 10 +++++++++ 5 files changed, 42 insertions(+), 2 deletions(-)