media: v4l: vsp1: Fix offset calculation for plane cropping

Commit Message

Laurent Pinchart Feb. 28, 2022, noon UTC

From: Michael Rodin <[e-mail]>

The vertical subsampling factor is currently not considered in the
offset calculation for plane cropping done in rpf_configure_partition.
This causes a distortion (shift of the color plane) when formats with
the vsub factor larger than 1 are used (e.g. NV12, see
vsp1_video_formats in vsp1_pipe.c). This commit considers vsub factor
for all planes except plane 0 (luminance).

Fixes: e5ad37b64de9 ("[media] v4l: vsp1: Add cropping support")
Signed-off-by: Michael Rodin <[e-mail]>
Signed-off-by: LUU HOAI <[e-mail]>

Drop generalization of the offset calculation to reduce the binary size.

Signed-off-by: Laurent Pinchart <[e-mail]>
---
 drivers/media/platform/vsp1/vsp1_rpf.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Comments

Kieran Bingham Feb. 28, 2022, 12:35 p.m. UTC | #1

Quoting Laurent Pinchart (2022-02-28 12:00:58)
> From: Michael Rodin <[e-mail]>
> 
> The vertical subsampling factor is currently not considered in the
> offset calculation for plane cropping done in rpf_configure_partition.
> This causes a distortion (shift of the color plane) when formats with
> the vsub factor larger than 1 are used (e.g. NV12, see
> vsp1_video_formats in vsp1_pipe.c). This commit considers vsub factor
> for all planes except plane 0 (luminance).
> 
> Fixes: e5ad37b64de9 ("[media] v4l: vsp1: Add cropping support")
> Signed-off-by: Michael Rodin <[e-mail]>
> Signed-off-by: LUU HOAI <[e-mail]>
> 
> Drop generalization of the offset calculation to reduce the binary size.
> 
> Signed-off-by: Laurent Pinchart <[e-mail]>

Looks fine to me, and I now see that the extra tests identify the
failure experienced and resolve it so...

Reviewed-by: Kieran Bingham <[e-mail]>

> ---
>  drivers/media/platform/vsp1/vsp1_rpf.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
> index 85587c1b6a37..75083cb234fe 100644
> --- a/drivers/media/platform/vsp1/vsp1_rpf.c
> +++ b/drivers/media/platform/vsp1/vsp1_rpf.c
> @@ -291,11 +291,11 @@ static void rpf_configure_partition(struct vsp1_entity *entity,
>                      + crop.left * fmtinfo->bpp[0] / 8;
>  
>         if (format->num_planes > 1) {
> +               unsigned int bpl = format->plane_fmt[1].bytesperline;
>                 unsigned int offset;
>  
> -               offset = crop.top * format->plane_fmt[1].bytesperline
> -                      + crop.left / fmtinfo->hsub
> -                      * fmtinfo->bpp[1] / 8;
> +               offset = crop.top / fmtinfo->vsub * bpl
> +                      + crop.left / fmtinfo->hsub * fmtinfo->bpp[1] / 8;
>                 mem.addr[1] += offset;
>                 mem.addr[2] += offset;
>         }
> -- 
> Regards,
> 
> Laurent Pinchart
>

Michael Rodin March 2, 2022, 11 a.m. UTC | #2

Hi Laurent,

thank you for your work!

On Mon, Feb 28, 2022 at 02:00:58PM +0200, Laurent Pinchart wrote:
> From: Michael Rodin <[e-mail]>
> 
> The vertical subsampling factor is currently not considered in the
> offset calculation for plane cropping done in rpf_configure_partition.
> This causes a distortion (shift of the color plane) when formats with
> the vsub factor larger than 1 are used (e.g. NV12, see
> vsp1_video_formats in vsp1_pipe.c). This commit considers vsub factor
> for all planes except plane 0 (luminance).
> 
> Fixes: e5ad37b64de9 ("[media] v4l: vsp1: Add cropping support")
> Signed-off-by: Michael Rodin <[e-mail]>
> Signed-off-by: LUU HOAI <https://urldefense.proofpoint.com/v2/url?u=http-3A__hoai.luu.ub-40renesas.com&d=DwIDAQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=sWsgk3pKkv5GeIDM2RZlPY8TjNFU2D0oBeOj6QNBadE&m=nh16dwdZnPwCY5XIxQPE0ZD_5QL0r5Z--1UUoxkfyrU&s=OjqLr2oazZuor0qk9BfVD8w1mz9vKoHq6g0SNCFmFrY&e=>
> 
> Drop generalization of the offset calculation to reduce the binary size.

Dropping generalization which I have done in my initial patch [1] is ok as
long as this will not cause any troubles. I am not aware of any case where
bytesperline and bpp could be different between the chroma planes, so
probably it's fine.

> 
> Signed-off-by: Laurent Pinchart <[e-mail]>
> ---
>  drivers/media/platform/vsp1/vsp1_rpf.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
> index 85587c1b6a37..75083cb234fe 100644
> --- a/drivers/media/platform/vsp1/vsp1_rpf.c
> +++ b/drivers/media/platform/vsp1/vsp1_rpf.c
> @@ -291,11 +291,11 @@ static void rpf_configure_partition(struct vsp1_entity *entity,
>  		     + crop.left * fmtinfo->bpp[0] / 8;
>  
>  	if (format->num_planes > 1) {
> +		unsigned int bpl = format->plane_fmt[1].bytesperline;
>  		unsigned int offset;
>  
> -		offset = crop.top * format->plane_fmt[1].bytesperline
> -		       + crop.left / fmtinfo->hsub
> -		       * fmtinfo->bpp[1] / 8;
> +		offset = crop.top / fmtinfo->vsub * bpl
> +		       + crop.left / fmtinfo->hsub * fmtinfo->bpp[1] / 8;

Probably it makes sense to do the division after all multiplications are
done in order to avoid rounding errors? Consider the case when left = 3,
hsub = 2, bpp = 32. Then we would get for the second part of the offset:
  3 / 2 * 32 / 8 = 1 * 32 / 8 = 4
and if we do division as the last operation:
  (3 * 32) / (8 * 2) = 96 / 16 = 6
The first part of the offset can probably also cause the same rounding
issue.

>  		mem.addr[1] += offset;
>  		mem.addr[2] += offset;
>  	}
> -- 
> Regards,
> 
> Laurent Pinchart
> 

[1] https://lore.kernel.org/all/[e-mail]/T/

Laurent Pinchart April 14, 2022, 2:20 p.m. UTC | #3

Hi Michael,

Your previous mail slipped through the cracks, sorry about that.

On Wed, Mar 02, 2022 at 12:00:12PM +0100, Michael Rodin wrote:
> Hi Laurent,
> 
> thank you for your work!
> 
> On Mon, Feb 28, 2022 at 02:00:58PM +0200, Laurent Pinchart wrote:
> > From: Michael Rodin <[e-mail]>
> > 
> > The vertical subsampling factor is currently not considered in the
> > offset calculation for plane cropping done in rpf_configure_partition.
> > This causes a distortion (shift of the color plane) when formats with
> > the vsub factor larger than 1 are used (e.g. NV12, see
> > vsp1_video_formats in vsp1_pipe.c). This commit considers vsub factor
> > for all planes except plane 0 (luminance).
> > 
> > Fixes: e5ad37b64de9 ("[media] v4l: vsp1: Add cropping support")
> > Signed-off-by: Michael Rodin <[e-mail]>
> > Signed-off-by: LUU HOAI <[e-mail]>
> > 
> > Drop generalization of the offset calculation to reduce the binary size.
> 
> Dropping generalization which I have done in my initial patch [1] is ok as
> long as this will not cause any troubles. I am not aware of any case where
> bytesperline and bpp could be different between the chroma planes, so
> probably it's fine.
> 
> > Signed-off-by: Laurent Pinchart <[e-mail]>
> > ---
> >  drivers/media/platform/vsp1/vsp1_rpf.c | 6 +++---
> >  1 file changed, 3 insertions(+), 3 deletions(-)
> > 
> > diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
> > index 85587c1b6a37..75083cb234fe 100644
> > --- a/drivers/media/platform/vsp1/vsp1_rpf.c
> > +++ b/drivers/media/platform/vsp1/vsp1_rpf.c
> > @@ -291,11 +291,11 @@ static void rpf_configure_partition(struct vsp1_entity *entity,
> >  		     + crop.left * fmtinfo->bpp[0] / 8;
> >  
> >  	if (format->num_planes > 1) {
> > +		unsigned int bpl = format->plane_fmt[1].bytesperline;
> >  		unsigned int offset;
> >  
> > -		offset = crop.top * format->plane_fmt[1].bytesperline
> > -		       + crop.left / fmtinfo->hsub
> > -		       * fmtinfo->bpp[1] / 8;
> > +		offset = crop.top / fmtinfo->vsub * bpl
> > +		       + crop.left / fmtinfo->hsub * fmtinfo->bpp[1] / 8;
> 
> Probably it makes sense to do the division after all multiplications are
> done in order to avoid rounding errors? Consider the case when left = 3,
> hsub = 2, bpp = 32. Then we would get for the second part of the offset:
>   3 / 2 * 32 / 8 = 1 * 32 / 8 = 4
> and if we do division as the last operation:
>   (3 * 32) / (8 * 2) = 96 / 16 = 6

This was actually done on purpose :-) If the horizontal subsampling
factor is equal to 2, for instance for the NV12 chroma plane, the
horizontal offset must effectively be a multiple of 2. Otherwise you'll
swap the Cr and Cb components.

Taking your above example with a NV12 format (left=3, hsub=2, but
bpp=16), with the rounding in this patch,

	offset = crop.top / fmtinfo->vsub * bpl
	       + crop.left / fmtinfo->hsub * fmtinfo->bpp[1] / 8;
	       = [vertical offset]
	       + 3 / 2 * 16 / 8;
	       = [vertical offset]
	       + 2;

Byte: 0  1  2  3  4  5
      Cr Cb Cr Cb Cr Cb ...
            ^
            offset

With your rounding proposal,

	offset = crop.top / fmtinfo->vsub * bpl
	       + (crop.left * fmtinfo->bpp[1]) / (fmtinfo->hsub * 8);
	       = [vertical offset]
	       + (3 * 16) / (2 * 8);
	       = [vertical offset]
	       + 3;

Byte: 0  1  2  3  4  5
      Cr Cb Cr Cb Cr Cb ...
               ^
               offset

> The first part of the offset can probably also cause the same rounding
> issue.
> 
> >  		mem.addr[1] += offset;
> >  		mem.addr[2] += offset;
> >  	}
> > 
> 
> [1] https://lore.kernel.org/all/[e-mail]/T/

Michael Rodin April 19, 2022, 10:22 a.m. UTC | #4

Hi Laurent,

On Thu, Apr 14, 2022 at 05:20:14PM +0300, Laurent Pinchart wrote:
> Hi Michael,
> 
> Your previous mail slipped through the cracks, sorry about that.
> 
> On Wed, Mar 02, 2022 at 12:00:12PM +0100, Michael Rodin wrote:
> > Hi Laurent,
> > 
> > thank you for your work!
> > 
> > On Mon, Feb 28, 2022 at 02:00:58PM +0200, Laurent Pinchart wrote:
> > > From: Michael Rodin <[e-mail]>
> > > 
> > > The vertical subsampling factor is currently not considered in the
> > > offset calculation for plane cropping done in rpf_configure_partition.
> > > This causes a distortion (shift of the color plane) when formats with
> > > the vsub factor larger than 1 are used (e.g. NV12, see
> > > vsp1_video_formats in vsp1_pipe.c). This commit considers vsub factor
> > > for all planes except plane 0 (luminance).
> > > 
> > > Fixes: e5ad37b64de9 ("[media] v4l: vsp1: Add cropping support")
> > > Signed-off-by: Michael Rodin <[e-mail]>
> > > Signed-off-by: LUU HOAI <https://urldefense.proofpoint.com/v2/url?u=http-3A__hoai.luu.ub-40renesas.com&d=DwICaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=sWsgk3pKkv5GeIDM2RZlPY8TjNFU2D0oBeOj6QNBadE&m=XAmHpGpli5fGaRsYAxJsReuojH4FFIzGmp2Njkwt8ko&s=l9CsK_BwOB0w3jdi3p2OFRTiGdlxWl2EHtxac3eVSTU&e=>
> > > 
> > > Drop generalization of the offset calculation to reduce the binary size.
> > 
> > Dropping generalization which I have done in my initial patch [1] is ok as
> > long as this will not cause any troubles. I am not aware of any case where
> > bytesperline and bpp could be different between the chroma planes, so
> > probably it's fine.
> > 
> > > Signed-off-by: Laurent Pinchart <[e-mail]>
> > > ---
> > >  drivers/media/platform/vsp1/vsp1_rpf.c | 6 +++---
> > >  1 file changed, 3 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
> > > index 85587c1b6a37..75083cb234fe 100644
> > > --- a/drivers/media/platform/vsp1/vsp1_rpf.c
> > > +++ b/drivers/media/platform/vsp1/vsp1_rpf.c
> > > @@ -291,11 +291,11 @@ static void rpf_configure_partition(struct vsp1_entity *entity,
> > >  		     + crop.left * fmtinfo->bpp[0] / 8;
> > >  
> > >  	if (format->num_planes > 1) {
> > > +		unsigned int bpl = format->plane_fmt[1].bytesperline;
> > >  		unsigned int offset;
> > >  
> > > -		offset = crop.top * format->plane_fmt[1].bytesperline
> > > -		       + crop.left / fmtinfo->hsub
> > > -		       * fmtinfo->bpp[1] / 8;
> > > +		offset = crop.top / fmtinfo->vsub * bpl
> > > +		       + crop.left / fmtinfo->hsub * fmtinfo->bpp[1] / 8;
> > 
> > Probably it makes sense to do the division after all multiplications are
> > done in order to avoid rounding errors? Consider the case when left = 3,
> > hsub = 2, bpp = 32. Then we would get for the second part of the offset:
> >   3 / 2 * 32 / 8 = 1 * 32 / 8 = 4
> > and if we do division as the last operation:
> >   (3 * 32) / (8 * 2) = 96 / 16 = 6
> 
> This was actually done on purpose :-) If the horizontal subsampling
> factor is equal to 2, for instance for the NV12 chroma plane, the
> horizontal offset must effectively be a multiple of 2. Otherwise you'll
> swap the Cr and Cb components.
> 
> Taking your above example with a NV12 format (left=3, hsub=2, but
> bpp=16), with the rounding in this patch,
> 
> 	offset = crop.top / fmtinfo->vsub * bpl
> 	       + crop.left / fmtinfo->hsub * fmtinfo->bpp[1] / 8;
> 	       = [vertical offset]
> 	       + 3 / 2 * 16 / 8;
> 	       = [vertical offset]
> 	       + 2;
> 
> Byte: 0  1  2  3  4  5
>       Cr Cb Cr Cb Cr Cb ...
>             ^
>             offset
> 
> With your rounding proposal,
> 
> 	offset = crop.top / fmtinfo->vsub * bpl
> 	       + (crop.left * fmtinfo->bpp[1]) / (fmtinfo->hsub * 8);
> 	       = [vertical offset]
> 	       + (3 * 16) / (2 * 8);
> 	       = [vertical offset]
> 	       + 3;
> 
> Byte: 0  1  2  3  4  5
>       Cr Cb Cr Cb Cr Cb ...
>                ^
>                offset

Thank you very much for the clarification, I have missed this point!
Now the patch looks fine to me.

Reviewed-by: Michael Rodin <[e-mail]>

> > The first part of the offset can probably also cause the same rounding
> > issue.
> > 
> > >  		mem.addr[1] += offset;
> > >  		mem.addr[2] += offset;
> > >  	}
> > > 
> > 
> > [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_all_1637679566-2D76975-2D1-2Dgit-2Dsend-2Demail-2Dmrodin-40de.adit-2Djv.com_T_&d=DwICaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=sWsgk3pKkv5GeIDM2RZlPY8TjNFU2D0oBeOj6QNBadE&m=XAmHpGpli5fGaRsYAxJsReuojH4FFIzGmp2Njkwt8ko&s=zwktftJ_aVV0iA0D8dcfCy1_rRg5PSdi5OXfTZBs648&e=
> 
> -- 
> Regards,
> 
> Laurent Pinchart

Michael Rodin May 6, 2022, 9:58 a.m. UTC | #5

Hi Laurent,

do you see any open points for this patch? If not, do you know maybe when
it will be available on linux-media or maybe mainline branch?

Thank you!

On Tue, Apr 19, 2022 at 12:22:15PM +0200, Michael Rodin wrote:
> Hi Laurent,
> 
> On Thu, Apr 14, 2022 at 05:20:14PM +0300, Laurent Pinchart wrote:
> > Hi Michael,
> > 
> > Your previous mail slipped through the cracks, sorry about that.
> > 
> > On Wed, Mar 02, 2022 at 12:00:12PM +0100, Michael Rodin wrote:
> > > Hi Laurent,
> > > 
> > > thank you for your work!
> > > 
> > > On Mon, Feb 28, 2022 at 02:00:58PM +0200, Laurent Pinchart wrote:
> > > > From: Michael Rodin <[e-mail]>
> > > > 
> > > > The vertical subsampling factor is currently not considered in the
> > > > offset calculation for plane cropping done in rpf_configure_partition.
> > > > This causes a distortion (shift of the color plane) when formats with
> > > > the vsub factor larger than 1 are used (e.g. NV12, see
> > > > vsp1_video_formats in vsp1_pipe.c). This commit considers vsub factor
> > > > for all planes except plane 0 (luminance).
> > > > 
> > > > Fixes: e5ad37b64de9 ("[media] v4l: vsp1: Add cropping support")
> > > > Signed-off-by: Michael Rodin <[e-mail]>
> > > > Signed-off-by: LUU HOAI <https://urldefense.proofpoint.com/v2/url?u=http-3A__hoai.luu.ub-40renesas.com&d=DwICaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=sWsgk3pKkv5GeIDM2RZlPY8TjNFU2D0oBeOj6QNBadE&m=XAmHpGpli5fGaRsYAxJsReuojH4FFIzGmp2Njkwt8ko&s=l9CsK_BwOB0w3jdi3p2OFRTiGdlxWl2EHtxac3eVSTU&e=>
> > > > 
> > > > Drop generalization of the offset calculation to reduce the binary size.
> > > 
> > > Dropping generalization which I have done in my initial patch [1] is ok as
> > > long as this will not cause any troubles. I am not aware of any case where
> > > bytesperline and bpp could be different between the chroma planes, so
> > > probably it's fine.
> > > 
> > > > Signed-off-by: Laurent Pinchart <[e-mail]>
> > > > ---
> > > >  drivers/media/platform/vsp1/vsp1_rpf.c | 6 +++---
> > > >  1 file changed, 3 insertions(+), 3 deletions(-)
> > > > 
> > > > diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
> > > > index 85587c1b6a37..75083cb234fe 100644
> > > > --- a/drivers/media/platform/vsp1/vsp1_rpf.c
> > > > +++ b/drivers/media/platform/vsp1/vsp1_rpf.c
> > > > @@ -291,11 +291,11 @@ static void rpf_configure_partition(struct vsp1_entity *entity,
> > > >  		     + crop.left * fmtinfo->bpp[0] / 8;
> > > >  
> > > >  	if (format->num_planes > 1) {
> > > > +		unsigned int bpl = format->plane_fmt[1].bytesperline;
> > > >  		unsigned int offset;
> > > >  
> > > > -		offset = crop.top * format->plane_fmt[1].bytesperline
> > > > -		       + crop.left / fmtinfo->hsub
> > > > -		       * fmtinfo->bpp[1] / 8;
> > > > +		offset = crop.top / fmtinfo->vsub * bpl
> > > > +		       + crop.left / fmtinfo->hsub * fmtinfo->bpp[1] / 8;
> > > 
> > > Probably it makes sense to do the division after all multiplications are
> > > done in order to avoid rounding errors? Consider the case when left = 3,
> > > hsub = 2, bpp = 32. Then we would get for the second part of the offset:
> > >   3 / 2 * 32 / 8 = 1 * 32 / 8 = 4
> > > and if we do division as the last operation:
> > >   (3 * 32) / (8 * 2) = 96 / 16 = 6
> > 
> > This was actually done on purpose :-) If the horizontal subsampling
> > factor is equal to 2, for instance for the NV12 chroma plane, the
> > horizontal offset must effectively be a multiple of 2. Otherwise you'll
> > swap the Cr and Cb components.
> > 
> > Taking your above example with a NV12 format (left=3, hsub=2, but
> > bpp=16), with the rounding in this patch,
> > 
> > 	offset = crop.top / fmtinfo->vsub * bpl
> > 	       + crop.left / fmtinfo->hsub * fmtinfo->bpp[1] / 8;
> > 	       = [vertical offset]
> > 	       + 3 / 2 * 16 / 8;
> > 	       = [vertical offset]
> > 	       + 2;
> > 
> > Byte: 0  1  2  3  4  5
> >       Cr Cb Cr Cb Cr Cb ...
> >             ^
> >             offset
> > 
> > With your rounding proposal,
> > 
> > 	offset = crop.top / fmtinfo->vsub * bpl
> > 	       + (crop.left * fmtinfo->bpp[1]) / (fmtinfo->hsub * 8);
> > 	       = [vertical offset]
> > 	       + (3 * 16) / (2 * 8);
> > 	       = [vertical offset]
> > 	       + 3;
> > 
> > Byte: 0  1  2  3  4  5
> >       Cr Cb Cr Cb Cr Cb ...
> >                ^
> >                offset
> 
> Thank you very much for the clarification, I have missed this point!
> Now the patch looks fine to me.
> 
> Reviewed-by: Michael Rodin <[e-mail]>
> 
> > > The first part of the offset can probably also cause the same rounding
> > > issue.
> > > 
> > > >  		mem.addr[1] += offset;
> > > >  		mem.addr[2] += offset;
> > > >  	}
> > > > 
> > > 
> > > [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_all_1637679566-2D76975-2D1-2Dgit-2Dsend-2Demail-2Dmrodin-40de.adit-2Djv.com_T_&d=DwICaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=sWsgk3pKkv5GeIDM2RZlPY8TjNFU2D0oBeOj6QNBadE&m=XAmHpGpli5fGaRsYAxJsReuojH4FFIzGmp2Njkwt8ko&s=zwktftJ_aVV0iA0D8dcfCy1_rRg5PSdi5OXfTZBs648&e=
> > 
> > -- 
> > Regards,
> > 
> > Laurent Pinchart
> 
> -- 
> Best Regards,
> Michael

Laurent Pinchart May 7, 2022, 9:16 p.m. UTC | #6

Hi Michael,

On Fri, May 06, 2022 at 11:58:21AM +0200, Michael Rodin wrote:
> Hi Laurent,
> 
> do you see any open points for this patch? If not, do you know maybe when
> it will be available on linux-media or maybe mainline branch?

I've just sent a pull request that includes this fix, I hope it will
make it to v5.19.

> On Tue, Apr 19, 2022 at 12:22:15PM +0200, Michael Rodin wrote:
> > On Thu, Apr 14, 2022 at 05:20:14PM +0300, Laurent Pinchart wrote:
> > > Hi Michael,
> > > 
> > > Your previous mail slipped through the cracks, sorry about that.
> > > 
> > > On Wed, Mar 02, 2022 at 12:00:12PM +0100, Michael Rodin wrote:
> > > > Hi Laurent,
> > > > 
> > > > thank you for your work!
> > > > 
> > > > On Mon, Feb 28, 2022 at 02:00:58PM +0200, Laurent Pinchart wrote:
> > > > > From: Michael Rodin <[e-mail]>
> > > > > 
> > > > > The vertical subsampling factor is currently not considered in the
> > > > > offset calculation for plane cropping done in rpf_configure_partition.
> > > > > This causes a distortion (shift of the color plane) when formats with
> > > > > the vsub factor larger than 1 are used (e.g. NV12, see
> > > > > vsp1_video_formats in vsp1_pipe.c). This commit considers vsub factor
> > > > > for all planes except plane 0 (luminance).
> > > > > 
> > > > > Fixes: e5ad37b64de9 ("[media] v4l: vsp1: Add cropping support")
> > > > > Signed-off-by: Michael Rodin <[e-mail]>
> > > > > Signed-off-by: LUU HOAI <[e-mail]>
> > > > > 
> > > > > Drop generalization of the offset calculation to reduce the binary size.
> > > > 
> > > > Dropping generalization which I have done in my initial patch [1] is ok as
> > > > long as this will not cause any troubles. I am not aware of any case where
> > > > bytesperline and bpp could be different between the chroma planes, so
> > > > probably it's fine.
> > > > 
> > > > > Signed-off-by: Laurent Pinchart <[e-mail]>
> > > > > ---
> > > > >  drivers/media/platform/vsp1/vsp1_rpf.c | 6 +++---
> > > > >  1 file changed, 3 insertions(+), 3 deletions(-)
> > > > > 
> > > > > diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
> > > > > index 85587c1b6a37..75083cb234fe 100644
> > > > > --- a/drivers/media/platform/vsp1/vsp1_rpf.c
> > > > > +++ b/drivers/media/platform/vsp1/vsp1_rpf.c
> > > > > @@ -291,11 +291,11 @@ static void rpf_configure_partition(struct vsp1_entity *entity,
> > > > >  		     + crop.left * fmtinfo->bpp[0] / 8;
> > > > >  
> > > > >  	if (format->num_planes > 1) {
> > > > > +		unsigned int bpl = format->plane_fmt[1].bytesperline;
> > > > >  		unsigned int offset;
> > > > >  
> > > > > -		offset = crop.top * format->plane_fmt[1].bytesperline
> > > > > -		       + crop.left / fmtinfo->hsub
> > > > > -		       * fmtinfo->bpp[1] / 8;
> > > > > +		offset = crop.top / fmtinfo->vsub * bpl
> > > > > +		       + crop.left / fmtinfo->hsub * fmtinfo->bpp[1] / 8;
> > > > 
> > > > Probably it makes sense to do the division after all multiplications are
> > > > done in order to avoid rounding errors? Consider the case when left = 3,
> > > > hsub = 2, bpp = 32. Then we would get for the second part of the offset:
> > > >   3 / 2 * 32 / 8 = 1 * 32 / 8 = 4
> > > > and if we do division as the last operation:
> > > >   (3 * 32) / (8 * 2) = 96 / 16 = 6
> > > 
> > > This was actually done on purpose :-) If the horizontal subsampling
> > > factor is equal to 2, for instance for the NV12 chroma plane, the
> > > horizontal offset must effectively be a multiple of 2. Otherwise you'll
> > > swap the Cr and Cb components.
> > > 
> > > Taking your above example with a NV12 format (left=3, hsub=2, but
> > > bpp=16), with the rounding in this patch,
> > > 
> > > 	offset = crop.top / fmtinfo->vsub * bpl
> > > 	       + crop.left / fmtinfo->hsub * fmtinfo->bpp[1] / 8;
> > > 	       = [vertical offset]
> > > 	       + 3 / 2 * 16 / 8;
> > > 	       = [vertical offset]
> > > 	       + 2;
> > > 
> > > Byte: 0  1  2  3  4  5
> > >       Cr Cb Cr Cb Cr Cb ...
> > >             ^
> > >             offset
> > > 
> > > With your rounding proposal,
> > > 
> > > 	offset = crop.top / fmtinfo->vsub * bpl
> > > 	       + (crop.left * fmtinfo->bpp[1]) / (fmtinfo->hsub * 8);
> > > 	       = [vertical offset]
> > > 	       + (3 * 16) / (2 * 8);
> > > 	       = [vertical offset]
> > > 	       + 3;
> > > 
> > > Byte: 0  1  2  3  4  5
> > >       Cr Cb Cr Cb Cr Cb ...
> > >                ^
> > >                offset
> > 
> > Thank you very much for the clarification, I have missed this point!
> > Now the patch looks fine to me.
> > 
> > Reviewed-by: Michael Rodin <[e-mail]>
> > 
> > > > The first part of the offset can probably also cause the same rounding
> > > > issue.
> > > > 
> > > > >  		mem.addr[1] += offset;
> > > > >  		mem.addr[2] += offset;
> > > > >  	}
> > > > > 
> > > > 
> > > > [1] https://lore.kernel.org/all/1637679566-2D76975-1-git-send-email-mrodin-40de.adit-jv.com/T/

Patch

diff --git a/drivers/media/platform/vsp1/vsp1_rpf.c b/drivers/media/platform/vsp1/vsp1_rpf.c
index 85587c1b6a37..75083cb234fe 100644
--- a/drivers/media/platform/vsp1/vsp1_rpf.c
+++ b/drivers/media/platform/vsp1/vsp1_rpf.c
@@ -291,11 +291,11 @@  static void rpf_configure_partition(struct vsp1_entity *entity,
 		     + crop.left * fmtinfo->bpp[0] / 8;
 
 	if (format->num_planes > 1) {
+		unsigned int bpl = format->plane_fmt[1].bytesperline;
 		unsigned int offset;
 
-		offset = crop.top * format->plane_fmt[1].bytesperline
-		       + crop.left / fmtinfo->hsub
-		       * fmtinfo->bpp[1] / 8;
+		offset = crop.top / fmtinfo->vsub * bpl
+		       + crop.left / fmtinfo->hsub * fmtinfo->bpp[1] / 8;
 		mem.addr[1] += offset;
 		mem.addr[2] += offset;
 	}