| Message ID | 20211006110207.256325-1-dorota.czaplejewicz@puri.sm (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | media: imx: Round line size to 4 bytes | expand |
Hi Dorota, On Wed, 2021-10-06 at 13:05 +0200, Dorota Czaplejewicz wrote: > Section 13.7.6.13 "CSI Image Parameter Register" of the > i.MX 8M Quad Applications Processors Reference Manual > states that the line size should be divisible by 8 bytes. > However, the hardware also accepts sizes divisible by 4 bytes. > > This patch accepts line sizes divisible 4-bytes in non-planar mode. Thank you, this makes it much clearer. I see two issues with this, though, one small and one a bit bigger: First, I'd be wary of disregarding the reference manual - unless we know better, and then it should be well documented in the code. It might be that the 8-byte alignment requirement stems from the fact that the FIFO operates in double-word units, which might cause the CSI to write over the end of the buffer if the line width is odd (in 32-bit words). Or maybe it's just that the FBUF_STRIDE conflicts with this, I'm unclear on whether that is only given in units of dwords (although the driver currently doesn't support this anyway). I wonder: if you use 4-byte aligned width and odd height, does the CSI write over the end of the buffer? > Signed-off-by: Dorota Czaplejewicz <dorota.czaplejewicz@puri.sm> > --- > > Hello, > > my previous patch identified something that was not a problem, > so I'm sending a different one. > > This has been tested on the Librem 5. > > Cheers, > Dorota > > drivers/staging/media/imx/imx-media-utils.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c > index 5128915a5d6f..a303003929e3 100644 > --- a/drivers/staging/media/imx/imx-media-utils.c > +++ b/drivers/staging/media/imx/imx-media-utils.c > @@ -545,13 +545,13 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix, > } > > /* Round up width for minimum burst size */ > - width = round_up(mbus->width, 8); > + width = round_up(mbus->width, 4); > > /* Round up stride for IDMAC line start address alignment */ > if (cc->planar) > stride = round_up(width, 16); > else > - stride = round_up((width * cc->bpp) >> 3, 8); > + stride = round_up((width * cc->bpp) >> 3, 4); Second, even if this works fine on the i.MX7/8M CSI, the alignment is still required for the i.MX5/6 IPU, for which this code and the comments were written. So we need a way to differentiate the two cases here. regards Philipp
Hello, On Fri, 08 Oct 2021 14:19:41 +0200 Philipp Zabel <p.zabel@pengutronix.de> wrote: > Hi Dorota, > > On Wed, 2021-10-06 at 13:05 +0200, Dorota Czaplejewicz wrote: > > Section 13.7.6.13 "CSI Image Parameter Register" of the > > i.MX 8M Quad Applications Processors Reference Manual > > states that the line size should be divisible by 8 bytes. > > However, the hardware also accepts sizes divisible by 4 bytes. > > > > This patch accepts line sizes divisible 4-bytes in non-planar mode. > > Thank you, this makes it much clearer. I see two issues with this, > though, one small and one a bit bigger: > > First, I'd be wary of disregarding the reference manual - unless we know > better, and then it should be well documented in the code. It might be > that the 8-byte alignment requirement stems from the fact that the FIFO > operates in double-word units, which might cause the CSI to write over > the end of the buffer if the line width is odd (in 32-bit words). > Or maybe it's just that the FBUF_STRIDE conflicts with this, I'm unclear > on whether that is only given in units of dwords (although the driver > currently doesn't support this anyway). > > I wonder: if you use 4-byte aligned width and odd height, does the CSI > write over the end of the buffer? I tested this case, and found a glitch which suggests the last 4 bytes are ignored: https://source.puri.sm/Librem5/linux-next/uploads/cfb59e3832431aaa3a69549455502568/image.png That would be taken care of rounding up towards a number decided at runtime, like: divisor = 8 >> (mbus->height % 2); > > > Signed-off-by: Dorota Czaplejewicz <dorota.czaplejewicz@puri.sm> > > --- > > > > Hello, > > > > my previous patch identified something that was not a problem, > > so I'm sending a different one. > > > > This has been tested on the Librem 5. > > > > Cheers, > > Dorota > > > > drivers/staging/media/imx/imx-media-utils.c | 4 ++-- > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c > > index 5128915a5d6f..a303003929e3 100644 > > --- a/drivers/staging/media/imx/imx-media-utils.c > > +++ b/drivers/staging/media/imx/imx-media-utils.c > > @@ -545,13 +545,13 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix, > > } > > > > /* Round up width for minimum burst size */ > > - width = round_up(mbus->width, 8); > > + width = round_up(mbus->width, 4); > > > > /* Round up stride for IDMAC line start address alignment */ > > if (cc->planar) > > stride = round_up(width, 16); > > else > > - stride = round_up((width * cc->bpp) >> 3, 8); > > + stride = round_up((width * cc->bpp) >> 3, 4); > > Second, even if this works fine on the i.MX7/8M CSI, the alignment is > still required for the i.MX5/6 IPU, for which this code and the comments > were written. So we need a way to differentiate the two cases here. > > regards > Philipp How best to go about this? I can see in the file imx-media-capture.c that there the video device lives in struct capture_priv.vdev.vfd. Would that be the right place to query about the underlying hardware? Then the following functions would gain a new "small_divisor" parameter: - imx_media_mbus_fmt_to_pix_fmt (a GPL symbol) - imx_media_mbus_fmt_to_ipu_image (a GPL symbol) - __capture_try_fmt Those would have to extract the device type from struct capture_priv: - __capture_legacy_try_fmt - capture_try_fmt_vid_cap - capture_s_fmt_vid_cap - capture_init_format Regards, Dorota
Hi Dorota, On Wed, 2021-10-13 at 11:26 +0200, Dorota Czaplejewicz wrote: > On Fri, 08 Oct 2021 14:19:41 +0200 Philipp Zabel <p.zabel@pengutronix.de> wrote: [...] > > I wonder: if you use 4-byte aligned width and odd height, does the CSI > > write over the end of the buffer? > > I tested this case, and found a glitch which suggests the last 4 bytes are ignored: > > https://source.puri.sm/Librem5/linux-next/uploads/cfb59e3832431aaa3a69549455502568/image.png Thank you for testing, so it appears that at least without FBUF_STRIDE the only requirement is that the whole image size must be a multiple of 8 bytes. > That would be taken care of rounding up towards a number decided at runtime, like: > > divisor = 8 >> (mbus->height % 2); Which would then cause the CSI to write past the end of the buffer? I'd rather make sure that either the number of lines is even or the width is a multiple of 8 bytes. > > > Signed-off-by: Dorota Czaplejewicz <dorota.czaplejewicz@puri.sm> > > > --- > > > > > > Hello, > > > > > > my previous patch identified something that was not a problem, > > > so I'm sending a different one. > > > > > > This has been tested on the Librem 5. > > > > > > Cheers, > > > Dorota > > > > > > drivers/staging/media/imx/imx-media-utils.c | 4 ++-- > > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > > > diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c > > > index 5128915a5d6f..a303003929e3 100644 > > > --- a/drivers/staging/media/imx/imx-media-utils.c > > > +++ b/drivers/staging/media/imx/imx-media-utils.c > > > @@ -545,13 +545,13 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix, > > > } > > > > > > /* Round up width for minimum burst size */ > > > - width = round_up(mbus->width, 8); > > > + width = round_up(mbus->width, 4); > > > > > > /* Round up stride for IDMAC line start address alignment */ > > > if (cc->planar) > > > stride = round_up(width, 16); > > > else > > > - stride = round_up((width * cc->bpp) >> 3, 8); > > > + stride = round_up((width * cc->bpp) >> 3, 4); > > > > Second, even if this works fine on the i.MX7/8M CSI, the alignment is > > still required for the i.MX5/6 IPU, for which this code and the comments > > were written. So we need a way to differentiate the two cases here. > > > > regards > > Philipp > > How best to go about this? I can see in the file imx-media-capture.c > that there the video device lives in struct capture_priv.vdev.vfd. > Would that be the right place to query about the underlying hardware? > > Then the following functions would gain a new "small_divisor" parameter: > - imx_media_mbus_fmt_to_pix_fmt (a GPL symbol) > - imx_media_mbus_fmt_to_ipu_image (a GPL symbol) > - __capture_try_fmt That sounds like it would work around the current code when it (at least part of imx_media_mbus_fmt_to_pix_fmt()) should be split between i.MX5/6 and i.MX7/8 implementations. For example rounding up the stride is not useful on i.MX7/8, it just doesn't currently hurt because imx7-media-csi is not using bytesperline to set up FBUF_STRIDE. And certainly the comments don't apply. imx_media_mbus_fmt_to_ipu_image() is unused and should probably be dropped, same as imx_media_ipu_image_to_mbus_fmt(). > Those would have to extract the device type from struct capture_priv: > - __capture_legacy_try_fmt > - capture_try_fmt_vid_cap > - capture_s_fmt_vid_cap > - capture_init_format Maybe imx_media_mbus_fmt_to_pix_fmt should be moved into imx-media- capture.c be passed struct capture_priv to avoid duplicating the device type check? imx_media_capture_device_init() could gain a new parameter (or maybe replace legacy_api) to set the device type. regards Philipp
Hello, On Thu, 14 Oct 2021 13:26:26 +0200 Philipp Zabel <p.zabel@pengutronix.de> wrote: > Hi Dorota, > > On Wed, 2021-10-13 at 11:26 +0200, Dorota Czaplejewicz wrote: > > On Fri, 08 Oct 2021 14:19:41 +0200 Philipp Zabel <p.zabel@pengutronix.de> wrote: > [...] > > > I wonder: if you use 4-byte aligned width and odd height, does the CSI > > > write over the end of the buffer? > > > > I tested this case, and found a glitch which suggests the last 4 bytes are ignored: > > > > https://source.puri.sm/Librem5/linux-next/uploads/cfb59e3832431aaa3a69549455502568/image.png > > Thank you for testing, so it appears that at least without FBUF_STRIDE > the only requirement is that the whole image size must be a multiple of > 8 bytes. > > > That would be taken care of rounding up towards a number decided at runtime, like: > > > > divisor = 8 >> (mbus->height % 2); > > Which would then cause the CSI to write past the end of the buffer? > I'm not sure if you point out the mistake here (should be "4 <<"), or the fact that rounding is happening. If it's the latter, then it's of no concern: the values derived here are used to calculate buffer size. I'm submitting a new series where this is fixed. > I'd rather make sure that either the number of lines is even or the > width is a multiple of 8 bytes. > > > > > Signed-off-by: Dorota Czaplejewicz <dorota.czaplejewicz@puri.sm> > > > > --- > > > > > > > > Hello, > > > > > > > > my previous patch identified something that was not a problem, > > > > so I'm sending a different one. > > > > > > > > This has been tested on the Librem 5. > > > > > > > > Cheers, > > > > Dorota > > > > > > > > drivers/staging/media/imx/imx-media-utils.c | 4 ++-- > > > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > > > > > diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c > > > > index 5128915a5d6f..a303003929e3 100644 > > > > --- a/drivers/staging/media/imx/imx-media-utils.c > > > > +++ b/drivers/staging/media/imx/imx-media-utils.c > > > > @@ -545,13 +545,13 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix, > > > > } > > > > > > > > /* Round up width for minimum burst size */ > > > > - width = round_up(mbus->width, 8); > > > > + width = round_up(mbus->width, 4); > > > > > > > > /* Round up stride for IDMAC line start address alignment */ > > > > if (cc->planar) > > > > stride = round_up(width, 16); > > > > else > > > > - stride = round_up((width * cc->bpp) >> 3, 8); > > > > + stride = round_up((width * cc->bpp) >> 3, 4); > > > > > > Second, even if this works fine on the i.MX7/8M CSI, the alignment is > > > still required for the i.MX5/6 IPU, for which this code and the comments > > > were written. So we need a way to differentiate the two cases here. > > > > > > regards > > > Philipp > > > > How best to go about this? I can see in the file imx-media-capture.c > > that there the video device lives in struct capture_priv.vdev.vfd. > > Would that be the right place to query about the underlying hardware? > > > > Then the following functions would gain a new "small_divisor" parameter: > > - imx_media_mbus_fmt_to_pix_fmt (a GPL symbol) > > - imx_media_mbus_fmt_to_ipu_image (a GPL symbol) > > - __capture_try_fmt > > That sounds like it would work around the current code when it (at least > part of imx_media_mbus_fmt_to_pix_fmt()) should be split between i.MX5/6 > and i.MX7/8 implementations. For example rounding up the stride is not > useful on i.MX7/8, it just doesn't currently hurt because imx7-media-csi > is not using bytesperline to set up FBUF_STRIDE. And certainly the > comments don't apply. > > imx_media_mbus_fmt_to_ipu_image() is unused and should probably be > dropped, same as imx_media_ipu_image_to_mbus_fmt(). Done in next series. > > > Those would have to extract the device type from struct capture_priv: > > - __capture_legacy_try_fmt > > - capture_try_fmt_vid_cap > > - capture_s_fmt_vid_cap > > - capture_init_format > > Maybe imx_media_mbus_fmt_to_pix_fmt should be moved into imx-media- > capture.c be passed struct capture_priv to avoid duplicating the device > type check? > I opted not to, in favor of passing the actual device type. It comes out to the same thing, except a simple value is passed around instead of a device. > imx_media_capture_device_init() could gain a new parameter (or maybe > replace legacy_api) to set the device type. Thanks, this is what I was missing. Regards, Dorota > > regards > Philipp
On Sun, 2021-10-17 at 13:07 +0200, Dorota Czaplejewicz wrote: > Hello, > > On Thu, 14 Oct 2021 13:26:26 +0200 > Philipp Zabel <p.zabel@pengutronix.de> wrote: > > > Hi Dorota, > > > > On Wed, 2021-10-13 at 11:26 +0200, Dorota Czaplejewicz wrote: > > > On Fri, 08 Oct 2021 14:19:41 +0200 Philipp Zabel <p.zabel@pengutronix.de> wrote: > > [...] > > > > I wonder: if you use 4-byte aligned width and odd height, does the CSI > > > > write over the end of the buffer? > > > > > > I tested this case, and found a glitch which suggests the last 4 bytes are ignored: > > > > > > https://source.puri.sm/Librem5/linux-next/uploads/cfb59e3832431aaa3a69549455502568/image.png > > > > Thank you for testing, so it appears that at least without FBUF_STRIDE > > the only requirement is that the whole image size must be a multiple of > > 8 bytes. > > > > > That would be taken care of rounding up towards a number decided at runtime, like: > > > > > > divisor = 8 >> (mbus->height % 2); > > > > Which would then cause the CSI to write past the end of the buffer? > > > I'm not sure if you point out the mistake here (should be "4 <<"), or > the fact that rounding is happening. If it's the latter, then it's of > no concern: the values derived here are used to calculate buffer size. > > I'm submitting a new series where this is fixed. Thanks, I just didn't understand your intention. The name "divisor" threw me off, see the comment in the new series. regards Philipp
diff --git a/drivers/staging/media/imx/imx-media-utils.c b/drivers/staging/media/imx/imx-media-utils.c index 5128915a5d6f..a303003929e3 100644 --- a/drivers/staging/media/imx/imx-media-utils.c +++ b/drivers/staging/media/imx/imx-media-utils.c @@ -545,13 +545,13 @@ int imx_media_mbus_fmt_to_pix_fmt(struct v4l2_pix_format *pix, } /* Round up width for minimum burst size */ - width = round_up(mbus->width, 8); + width = round_up(mbus->width, 4); /* Round up stride for IDMAC line start address alignment */ if (cc->planar) stride = round_up(width, 16); else - stride = round_up((width * cc->bpp) >> 3, 8); + stride = round_up((width * cc->bpp) >> 3, 4); pix->width = width; pix->height = mbus->height;
Section 13.7.6.13 "CSI Image Parameter Register" of the i.MX 8M Quad Applications Processors Reference Manual states that the line size should be divisible by 8 bytes. However, the hardware also accepts sizes divisible by 4 bytes. This patch accepts line sizes divisible 4-bytes in non-planar mode. Signed-off-by: Dorota Czaplejewicz <dorota.czaplejewicz@puri.sm> --- Hello, my previous patch identified something that was not a problem, so I'm sending a different one. This has been tested on the Librem 5. Cheers, Dorota drivers/staging/media/imx/imx-media-utils.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)