From patchwork Wed Feb 7 00:04:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Gustavo A. R. Silva" X-Patchwork-Id: 10204703 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 09E29602D8 for ; Wed, 7 Feb 2018 07:42:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DDFC928417 for ; Wed, 7 Feb 2018 07:42:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CDE942858F; Wed, 7 Feb 2018 07:42:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 55E4728417 for ; Wed, 7 Feb 2018 07:42:30 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1A14F6E0EA; Wed, 7 Feb 2018 07:42:29 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org X-Greylist: delayed 1252 seconds by postgrey-1.36 at gabe; Wed, 07 Feb 2018 00:25:20 UTC Received: from gateway31.websitewelcome.com (gateway31.websitewelcome.com [192.185.143.5]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8B81689D39 for ; Wed, 7 Feb 2018 00:25:20 +0000 (UTC) Received: from cm15.websitewelcome.com (cm15.websitewelcome.com [100.42.49.9]) by gateway31.websitewelcome.com (Postfix) with ESMTP id 4CDFF85EED for ; Tue, 6 Feb 2018 18:04:26 -0600 (CST) Received: from gator4166.hostgator.com ([108.167.133.22]) by cmsmtp with SMTP id jDDueuTAOmzEzjDDuexCaq; Tue, 06 Feb 2018 18:04:26 -0600 Received: from [189.175.4.238] (port=39266 helo=embeddedgus) by gator4166.hostgator.com with esmtpa (Exim 4.89_1) (envelope-from ) id 1ejDDt-0032dU-PJ; Tue, 06 Feb 2018 18:04:25 -0600 Date: Tue, 6 Feb 2018 18:04:24 -0600 From: "Gustavo A. R. Silva" To: Bartlomiej Zolnierkiewicz Subject: [PATCH] video: fbdev: vermilion: use 64-bit arithmetic instead of 32-bit Message-ID: <20180207000424.GA32680@embeddedgus> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.24 (2015-08-30) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - gator4166.hostgator.com X-AntiAbuse: Original Domain - lists.freedesktop.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - embeddedor.com X-BWhitelist: no X-Source-IP: 189.175.4.238 X-Source-L: No X-Exim-ID: 1ejDDt-0032dU-PJ X-Source: X-Source-Args: X-Source-Dir: X-Source-Sender: (embeddedgus) [189.175.4.238]:39266 X-Source-Auth: gustavo@embeddedor.com X-Email-Count: 2 X-Source-Cap: Z3V6aWRpbmU7Z3V6aWRpbmU7Z2F0b3I0MTY2Lmhvc3RnYXRvci5jb20= X-Local-Domain: yes X-Mailman-Approved-At: Wed, 07 Feb 2018 07:42:28 +0000 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-fbdev@vger.kernel.org, "Gustavo A. R. Silva" , linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Cast _pitch_ to u64 in order to give the compiler complete information about the proper arithmetic to use. Notice that this variable is being used in a context that expects an expression of type u64 (64 bits, unsigned). The expression pitch * var->yres_virtual is currently being evaluated using 32-bit arithmetic and the result of the operation is being stored into variable mem, which is a variable of type u64. Based on that, chances are there is a potential integer overflow as a result of the operation. Addresses-Coverity-ID: 200655 ("Unintentional integer overflow") Signed-off-by: Gustavo A. R. Silva --- drivers/video/fbdev/vermilion/vermilion.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/video/fbdev/vermilion/vermilion.c b/drivers/video/fbdev/vermilion/vermilion.c index 6f8d444..5172fa5 100644 --- a/drivers/video/fbdev/vermilion/vermilion.c +++ b/drivers/video/fbdev/vermilion/vermilion.c @@ -651,7 +651,7 @@ static int vmlfb_check_var_locked(struct fb_var_screeninfo *var, } pitch = ALIGN((var->xres * var->bits_per_pixel) >> 3, 0x40); - mem = pitch * var->yres_virtual; + mem = (u64)pitch * var->yres_virtual; if (mem > vinfo->vram_contig_size) { return -ENOMEM; }