From patchwork Mon Sep 10 15:29:54 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Philipp Zabel X-Patchwork-Id: 1432691 Return-Path: X-Original-To: patchwork-linux-media@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id 2AB25DFFCF for ; Mon, 10 Sep 2012 15:30:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757704Ab2IJPaX (ORCPT ); Mon, 10 Sep 2012 11:30:23 -0400 Received: from metis.ext.pengutronix.de ([92.198.50.35]:47403 "EHLO metis.ext.pengutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757528Ab2IJPaP (ORCPT ); Mon, 10 Sep 2012 11:30:15 -0400 Received: from dude.hi.pengutronix.de ([10.1.0.7] helo=dude.pengutronix.de) by metis.ext.pengutronix.de with esmtp (Exim 4.72) (envelope-from ) id 1TB5w9-0006ti-Fy; Mon, 10 Sep 2012 17:30:09 +0200 From: Philipp Zabel To: linux-media@vger.kernel.org Cc: Javier Martin , Mauro Carvalho Chehab , Richard Zhao , Laurent Pinchart , Sylwester Nawrocki , Kyungmin Park , Hans Verkuil , kernel@pengutronix.de, Philipp Zabel Subject: [PATCH v4 10/16] media: coda: fix sizeimage setting in try_fmt Date: Mon, 10 Sep 2012 17:29:54 +0200 Message-Id: <1347291000-340-11-git-send-email-p.zabel@pengutronix.de> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1347291000-340-1-git-send-email-p.zabel@pengutronix.de> References: <1347291000-340-1-git-send-email-p.zabel@pengutronix.de> X-SA-Exim-Connect-IP: 10.1.0.7 X-SA-Exim-Mail-From: p.zabel@pengutronix.de X-SA-Exim-Scanned: No (on metis.ext.pengutronix.de); SAEximRunCond expanded to false X-PTX-Original-Recipient: linux-media@vger.kernel.org Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org VIDIOC_TRY_FMT would incorrectly return bytesperline * height, instead of width * height * 3 / 2. Signed-off-by: Philipp Zabel Tested-by: Javier Martin fmt.pix.height, MIN_H, MAX_H, H_ALIGN, S_ALIGN); f->fmt.pix.bytesperline = round_up(f->fmt.pix.width, 2); - f->fmt.pix.sizeimage = f->fmt.pix.height * - f->fmt.pix.bytesperline; + f->fmt.pix.sizeimage = f->fmt.pix.width * + f->fmt.pix.height * 3 / 2; } else { /*encoded formats h.264/mpeg4 */ f->fmt.pix.bytesperline = 0; f->fmt.pix.sizeimage = CODA_MAX_FRAME_SIZE; @@ -492,11 +492,7 @@ static int vidioc_s_fmt(struct coda_ctx *ctx, struct v4l2_format *f) q_data->fmt = find_format(ctx->dev, f); q_data->width = f->fmt.pix.width; q_data->height = f->fmt.pix.height; - if (q_data->fmt->fourcc == V4L2_PIX_FMT_YUV420) { - q_data->sizeimage = q_data->width * q_data->height * 3 / 2; - } else { /* encoded format h.264/mpeg-4 */ - q_data->sizeimage = CODA_MAX_FRAME_SIZE; - } + q_data->sizeimage = f->fmt.pix.sizeimage; v4l2_dbg(1, coda_debug, &ctx->dev->v4l2_dev, "Setting format for type %d, wxh: %dx%d, fmt: %d\n",