From patchwork Wed Jun 17 12:54:31 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomi Valkeinen X-Patchwork-Id: 6625501 Return-Path: X-Original-To: patchwork-linux-fbdev@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 273A2C0433 for ; Wed, 17 Jun 2015 12:55:10 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 7EEE72085A for ; Wed, 17 Jun 2015 12:55:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 866A520858 for ; Wed, 17 Jun 2015 12:55:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755207AbbFQMzC (ORCPT ); Wed, 17 Jun 2015 08:55:02 -0400 Received: from comal.ext.ti.com ([198.47.26.152]:40740 "EHLO comal.ext.ti.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754426AbbFQMy7 (ORCPT ); Wed, 17 Jun 2015 08:54:59 -0400 Received: from dlelxv90.itg.ti.com ([172.17.2.17]) by comal.ext.ti.com (8.13.7/8.13.7) with ESMTP id t5HCsusq032195; Wed, 17 Jun 2015 07:54:56 -0500 Received: from DLEE71.ent.ti.com (dlee71.ent.ti.com [157.170.170.114]) by dlelxv90.itg.ti.com (8.14.3/8.13.8) with ESMTP id t5HCsuc2021751; Wed, 17 Jun 2015 07:54:56 -0500 Received: from dflp32.itg.ti.com (10.64.6.15) by DLEE71.ent.ti.com (157.170.170.114) with Microsoft SMTP Server id 14.3.224.2; Wed, 17 Jun 2015 07:54:56 -0500 Received: from deskari.ti.com (ileax41-snat.itg.ti.com [10.172.224.153]) by dflp32.itg.ti.com (8.14.3/8.13.8) with ESMTP id t5HCsi7Q002601; Wed, 17 Jun 2015 07:54:55 -0500 From: Tomi Valkeinen To: Laurent Pinchart , , CC: Tomi Valkeinen Subject: [PATCH 07/11] OMAPDSS: DISPC: check if scaling setup failed Date: Wed, 17 Jun 2015 15:54:31 +0300 Message-ID: <1434545675-477-8-git-send-email-tomi.valkeinen@ti.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1434545675-477-1-git-send-email-tomi.valkeinen@ti.com> References: <1434545675-477-1-git-send-email-tomi.valkeinen@ti.com> MIME-Version: 1.0 Sender: linux-fbdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fbdev@vger.kernel.org X-Spam-Status: No, score=-7.5 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The DISPC's scaling code seems to presume that decimation always succeeds, and so we always do find a suitable downscaling setup. However, this is not the case, and the algorithm can fail. When that happens, the code just proceeds with wrong results, causing issues later. Add the necessary checks to bail out if the scaling algorithm failed. Signed-off-by: Tomi Valkeinen --- drivers/video/fbdev/omap2/dss/dispc.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/drivers/video/fbdev/omap2/dss/dispc.c b/drivers/video/fbdev/omap2/dss/dispc.c index 2db1c986e989..0bdb587cb48c 100644 --- a/drivers/video/fbdev/omap2/dss/dispc.c +++ b/drivers/video/fbdev/omap2/dss/dispc.c @@ -2279,6 +2279,11 @@ static int dispc_ovl_calc_scaling_24xx(unsigned long pclk, unsigned long lclk, } } while (*decim_x <= *x_predecim && *decim_y <= *y_predecim && error); + if (error) { + DSSERR("failed to find scaling settings\n"); + return -EINVAL; + } + if (in_width > maxsinglelinewidth) { DSSERR("Cannot scale max input width exceeded"); return -EINVAL; @@ -2356,6 +2361,11 @@ again: } } while (*decim_x <= *x_predecim && *decim_y <= *y_predecim && error); + if (error) { + DSSERR("failed to find scaling settings\n"); + return -EINVAL; + } + if (check_horiz_timing_omap3(pclk, lclk, mgr_timings, pos_x, in_width, in_height, out_width, out_height, *five_taps)) { DSSERR("horizontal timing too tight\n");