diff mbox series

[07/16] media: ti-vpe: vpe: fix a v4l2-compliance failure causing a kernel panic

Message ID 20190927183650.31345-8-bparrot@ti.com (mailing list archive)
State New, archived
Headers show
Series media: vpe: maintenance | expand

Commit Message

Benoit Parrot Sept. 27, 2019, 6:36 p.m. UTC
v4l2-compliance fails with this message:

   warn: v4l2-test-formats.cpp(717): \
   	TRY_FMT cannot handle an invalid pixelformat.
   test VIDIOC_TRY_FMT: FAIL

This causes the following kernel panic:

Unable to handle kernel paging request at virtual address 56595561
pgd = ecd80e00
*pgd=00000000
Internal error: Oops: 205 [#1] PREEMPT SMP ARM
...
CPU: 0 PID: 930 Comm: v4l2-compliance Not tainted \
	4.14.62-01715-gc8cd67f49a19 #1
Hardware name: Generic DRA72X (Flattened Device Tree)
task: ece44d80 task.stack: ecc6e000
PC is at __vpe_try_fmt+0x18c/0x2a8 [ti_vpe]
LR is at 0x8

Because the driver fails to properly check the 'num_planes' values for
proper ranges it ends up accessing out of bound data causing the kernel
panic.

Since this driver only handle single or dual plane pixel format, make
sure the provided value does not exceed 2 planes.

Signed-off-by: Benoit Parrot <bparrot@ti.com>
Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ti.com>
---
 drivers/media/platform/ti-vpe/vpe.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Hans Verkuil Sept. 30, 2019, 8:35 a.m. UTC | #1
On 9/27/19 8:36 PM, Benoit Parrot wrote:
> v4l2-compliance fails with this message:
> 
>    warn: v4l2-test-formats.cpp(717): \
>    	TRY_FMT cannot handle an invalid pixelformat.
>    test VIDIOC_TRY_FMT: FAIL
> 
> This causes the following kernel panic:
> 
> Unable to handle kernel paging request at virtual address 56595561
> pgd = ecd80e00
> *pgd=00000000
> Internal error: Oops: 205 [#1] PREEMPT SMP ARM
> ...
> CPU: 0 PID: 930 Comm: v4l2-compliance Not tainted \
> 	4.14.62-01715-gc8cd67f49a19 #1
> Hardware name: Generic DRA72X (Flattened Device Tree)
> task: ece44d80 task.stack: ecc6e000
> PC is at __vpe_try_fmt+0x18c/0x2a8 [ti_vpe]
> LR is at 0x8
> 
> Because the driver fails to properly check the 'num_planes' values for
> proper ranges it ends up accessing out of bound data causing the kernel
> panic.
> 
> Since this driver only handle single or dual plane pixel format, make
> sure the provided value does not exceed 2 planes.
> 
> Signed-off-by: Benoit Parrot <bparrot@ti.com>
> Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ti.com>
> ---
>  drivers/media/platform/ti-vpe/vpe.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/media/platform/ti-vpe/vpe.c b/drivers/media/platform/ti-vpe/vpe.c
> index bbbf11174e16..1278d457f753 100644
> --- a/drivers/media/platform/ti-vpe/vpe.c
> +++ b/drivers/media/platform/ti-vpe/vpe.c
> @@ -1650,7 +1650,7 @@ static int __vpe_try_fmt(struct vpe_ctx *ctx, struct v4l2_format *f,
>  			      &pix->height, MIN_H, MAX_H, H_ALIGN,
>  			      S_ALIGN);
>  
> -	if (!pix->num_planes)
> +	if (!pix->num_planes || pix->num_planes > 2)
>  		pix->num_planes = fmt->coplanar ? 2 : 1;
>  	else if (pix->num_planes > 1 && !fmt->coplanar)
>  		pix->num_planes = 1;
> 

This looks weird.

Why not just unconditionally do:

	pix->num_planes = fmt->coplanar ? 2 : 1;

Regards,

	Hans
Benoit Parrot Sept. 30, 2019, 4:04 p.m. UTC | #2
Hans Verkuil <hverkuil@xs4all.nl> wrote on Mon [2019-Sep-30 10:35:05 +0200]:
> On 9/27/19 8:36 PM, Benoit Parrot wrote:
> > v4l2-compliance fails with this message:
> > 
> >    warn: v4l2-test-formats.cpp(717): \
> >    	TRY_FMT cannot handle an invalid pixelformat.
> >    test VIDIOC_TRY_FMT: FAIL
> > 
> > This causes the following kernel panic:
> > 
> > Unable to handle kernel paging request at virtual address 56595561
> > pgd = ecd80e00
> > *pgd=00000000
> > Internal error: Oops: 205 [#1] PREEMPT SMP ARM
> > ...
> > CPU: 0 PID: 930 Comm: v4l2-compliance Not tainted \
> > 	4.14.62-01715-gc8cd67f49a19 #1
> > Hardware name: Generic DRA72X (Flattened Device Tree)
> > task: ece44d80 task.stack: ecc6e000
> > PC is at __vpe_try_fmt+0x18c/0x2a8 [ti_vpe]
> > LR is at 0x8
> > 
> > Because the driver fails to properly check the 'num_planes' values for
> > proper ranges it ends up accessing out of bound data causing the kernel
> > panic.
> > 
> > Since this driver only handle single or dual plane pixel format, make
> > sure the provided value does not exceed 2 planes.
> > 
> > Signed-off-by: Benoit Parrot <bparrot@ti.com>
> > Reviewed-by: Tomi Valkeinen <tomi.valkeinen@ti.com>
> > ---
> >  drivers/media/platform/ti-vpe/vpe.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/drivers/media/platform/ti-vpe/vpe.c b/drivers/media/platform/ti-vpe/vpe.c
> > index bbbf11174e16..1278d457f753 100644
> > --- a/drivers/media/platform/ti-vpe/vpe.c
> > +++ b/drivers/media/platform/ti-vpe/vpe.c
> > @@ -1650,7 +1650,7 @@ static int __vpe_try_fmt(struct vpe_ctx *ctx, struct v4l2_format *f,
> >  			      &pix->height, MIN_H, MAX_H, H_ALIGN,
> >  			      S_ALIGN);
> >  
> > -	if (!pix->num_planes)
> > +	if (!pix->num_planes || pix->num_planes > 2)
> >  		pix->num_planes = fmt->coplanar ? 2 : 1;
> >  	else if (pix->num_planes > 1 && !fmt->coplanar)
> >  		pix->num_planes = 1;
> > 
> 
> This looks weird.
> 
> Why not just unconditionally do:
> 
> 	pix->num_planes = fmt->coplanar ? 2 : 1;

In order to not change the behavior, VPE would assume that NV12 format for
instance were always sent as 2 separate planes buffers. So for backward
compatibility this is order to handle both cases where NV12 could be
handled as both a single plane and a dual plane buffers based on what the
user space application intent on passing in/out.

Benoit

> 
> Regards,
> 
> 	Hans
diff mbox series

Patch

diff --git a/drivers/media/platform/ti-vpe/vpe.c b/drivers/media/platform/ti-vpe/vpe.c
index bbbf11174e16..1278d457f753 100644
--- a/drivers/media/platform/ti-vpe/vpe.c
+++ b/drivers/media/platform/ti-vpe/vpe.c
@@ -1650,7 +1650,7 @@  static int __vpe_try_fmt(struct vpe_ctx *ctx, struct v4l2_format *f,
 			      &pix->height, MIN_H, MAX_H, H_ALIGN,
 			      S_ALIGN);
 
-	if (!pix->num_planes)
+	if (!pix->num_planes || pix->num_planes > 2)
 		pix->num_planes = fmt->coplanar ? 2 : 1;
 	else if (pix->num_planes > 1 && !fmt->coplanar)
 		pix->num_planes = 1;