From patchwork Wed Oct 3 11:04:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eugeniy Paltsev X-Patchwork-Id: 10624579 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 33C5B112B for ; Wed, 3 Oct 2018 11:05:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1C2082879D for ; Wed, 3 Oct 2018 11:05:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 10280287A2; Wed, 3 Oct 2018 11:05:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 979972879D for ; Wed, 3 Oct 2018 11:05:10 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 334CF6E0AA; Wed, 3 Oct 2018 11:05:08 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from smtprelay.synopsys.com (smtprelay2.synopsys.com [198.182.60.111]) by gabe.freedesktop.org (Postfix) with ESMTPS id 107F06E0AA for ; Wed, 3 Oct 2018 11:05:06 +0000 (UTC) Received: from mailhost.synopsys.com (mailhost1.synopsys.com [10.12.238.239]) by smtprelay.synopsys.com (Postfix) with ESMTP id 2F97A10C03B3; Wed, 3 Oct 2018 04:05:05 -0700 (PDT) Received: from paltsev-e7480.internal.synopsys.com (paltsev-e7480.internal.synopsys.com [10.121.3.38]) by mailhost.synopsys.com (Postfix) with ESMTP id A031C53A4; Wed, 3 Oct 2018 04:05:00 -0700 (PDT) From: Eugeniy Paltsev To: dri-devel@lists.freedesktop.org Subject: [PATCH] drm: fb-helper: Validate requested pixel format against bpp Date: Wed, 3 Oct 2018 14:04:57 +0300 Message-Id: <20181003110457.9318-1-Eugeniy.Paltsev@synopsys.com> X-Mailer: git-send-email 2.14.4 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: David Airlie , Alexey Brodkin , linux-kernel@vger.kernel.org, stable@vger.kernel.org, Eugeniy Paltsev , linux-snps-arc@lists.infradead.org, Sean Paul MIME-Version: 1.0 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Validate requested pixel format against bits_per_pixel to reject invalid formats with subcomponents length sum is greater than requested bits_per_pixel. weston 5.0.0 with fbdev backend tries to set up an ARGB x8r8g8b8 pixel format without bits_per_pixel updating. So it can request x8r8g8b8 with 16 bpp which is obviously incorrect and should be rejected. Cc: stable@vger.kernel.org Signed-off-by: Eugeniy Paltsev --- drivers/gpu/drm/drm_fb_helper.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c index 16ec93b75dbf..4f39da07f053 100644 --- a/drivers/gpu/drm/drm_fb_helper.c +++ b/drivers/gpu/drm/drm_fb_helper.c @@ -1610,6 +1610,13 @@ int drm_fb_helper_check_var(struct fb_var_screeninfo *var, return -EINVAL; } + if ((var->green.length + var->blue.length + var->red.length + + var->transp.length) > var->bits_per_pixel) { + DRM_DEBUG("fb requested pixel format can't fit in %d bpp\n", + var->bits_per_pixel); + return -EINVAL; + } + switch (var->bits_per_pixel) { case 16: depth = (var->green.length == 6) ? 16 : 15;