| Message ID | 20230126230219.3399415-1-m.grzeschik@pengutronix.de (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [v2] hantro: fix VIDIOC_ENUM_FRAMESIZES for non-coded formats without postproc | expand |
Le vendredi 27 janvier 2023 à 00:02 +0100, Michael Grzeschik a écrit : > The frmsize structure was left initialize to 0, as side effect, the > driver was reporting an invalid frmsize. > > Size: Stepwise 0x0 - 0x0 with step 0/0 > > The frmsize structure in hantro_fmt is only valid for bitstream formats. > So just adding frmsizes for non coded formats does not work to fix this. > > When codec_mode is HANTRO_MODE_NONE, then vidioc_enum_framesizes should > return with -ENOTTY. At least when hantro_needs_postproc returns false. > Which is the case for encoders. But with the latest postproc scaling > patch this is not the case anymore. This patch is fixing this back to > how it was but changing the EINVAL to ENOTTY since we touch this anyway. > > For now this is a good enough fix, but in the future enum_framesizes > needs to obtain different constraints in framesizes in combination of > coded to raw formats. > > Fixes: 79c987de8b35 ("media: hantro: Use post processor scaling capacities") > Signed-off-by: Michael Grzeschik <m.grzeschik@pengutronix.de> > Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> Acked-by: Nicolas Dufresne <nicolas.dufresne@collabora.com> > > --- > v1 -> v2: - changed EINVAL to ENOTTY as suggested by Nicolas > - updated the commit message to add more infos to the issue > > drivers/media/platform/verisilicon/hantro_v4l2.c | 7 +++++-- > 1 file changed, 5 insertions(+), 2 deletions(-) > > diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c > index 2c7a805289e7b8..30e650edaea8a3 100644 > --- a/drivers/media/platform/verisilicon/hantro_v4l2.c > +++ b/drivers/media/platform/verisilicon/hantro_v4l2.c > @@ -161,8 +161,11 @@ static int vidioc_enum_framesizes(struct file *file, void *priv, > } > > /* For non-coded formats check if postprocessing scaling is possible */ > - if (fmt->codec_mode == HANTRO_MODE_NONE && hantro_needs_postproc(ctx, fmt)) { > - return hanto_postproc_enum_framesizes(ctx, fsize); > + if (fmt->codec_mode == HANTRO_MODE_NONE) { > + if (hantro_needs_postproc(ctx, fmt)) > + return hanto_postproc_enum_framesizes(ctx, fsize); > + else > + return -ENOTTY; > } else if (fsize->index != 0) { > vpu_debug(0, "invalid frame size index (expected 0, got %d)\n", > fsize->index);
diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c index 2c7a805289e7b8..30e650edaea8a3 100644 --- a/drivers/media/platform/verisilicon/hantro_v4l2.c +++ b/drivers/media/platform/verisilicon/hantro_v4l2.c @@ -161,8 +161,11 @@ static int vidioc_enum_framesizes(struct file *file, void *priv, } /* For non-coded formats check if postprocessing scaling is possible */ - if (fmt->codec_mode == HANTRO_MODE_NONE && hantro_needs_postproc(ctx, fmt)) { - return hanto_postproc_enum_framesizes(ctx, fsize); + if (fmt->codec_mode == HANTRO_MODE_NONE) { + if (hantro_needs_postproc(ctx, fmt)) + return hanto_postproc_enum_framesizes(ctx, fsize); + else + return -ENOTTY; } else if (fsize->index != 0) { vpu_debug(0, "invalid frame size index (expected 0, got %d)\n", fsize->index);
The frmsize structure was left initialize to 0, as side effect, the driver was reporting an invalid frmsize. Size: Stepwise 0x0 - 0x0 with step 0/0 The frmsize structure in hantro_fmt is only valid for bitstream formats. So just adding frmsizes for non coded formats does not work to fix this. When codec_mode is HANTRO_MODE_NONE, then vidioc_enum_framesizes should return with -ENOTTY. At least when hantro_needs_postproc returns false. Which is the case for encoders. But with the latest postproc scaling patch this is not the case anymore. This patch is fixing this back to how it was but changing the EINVAL to ENOTTY since we touch this anyway. For now this is a good enough fix, but in the future enum_framesizes needs to obtain different constraints in framesizes in combination of coded to raw formats. Fixes: 79c987de8b35 ("media: hantro: Use post processor scaling capacities") Signed-off-by: Michael Grzeschik <m.grzeschik@pengutronix.de> Reviewed-by: Ezequiel Garcia <ezequiel@vanguardiasur.com.ar> --- v1 -> v2: - changed EINVAL to ENOTTY as suggested by Nicolas - updated the commit message to add more infos to the issue drivers/media/platform/verisilicon/hantro_v4l2.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)