From patchwork Tue Jul 3 19:14:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Thomas Hellstrom X-Patchwork-Id: 10505005 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6AF06601D3 for ; Tue, 3 Jul 2018 19:17:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 59D5528B5A for ; Tue, 3 Jul 2018 19:17:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4E34328BDD; Tue, 3 Jul 2018 19:17:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAD_ENC_HEADER,BAYES_00, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C7EE828B5A for ; Tue, 3 Jul 2018 19:17:06 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 9D59B6E8D3; Tue, 3 Jul 2018 19:16:38 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM05-BY2-obe.outbound.protection.outlook.com (mail-eopbgr710064.outbound.protection.outlook.com [40.107.71.64]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1D5FE6E8CB for ; Tue, 3 Jul 2018 19:16:37 +0000 (UTC) Received: from fedoratest.localdomain (155.4.205.56) by DM6PR05MB4587.namprd05.prod.outlook.com (2603:10b6:5:9f::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.930.11; Tue, 3 Jul 2018 19:16:34 +0000 From: Thomas Hellstrom To: dri-devel@lists.freedesktop.org Subject: [PATCH -next 07/15] drm/vmwgfx: Use modeset display memory validation for layout ioctl Date: Tue, 3 Jul 2018 21:14:52 +0200 Message-Id: <20180703191500.2374-8-thellstrom@vmware.com> X-Mailer: git-send-email 2.18.0.rc1 In-Reply-To: <20180703191500.2374-1-thellstrom@vmware.com> References: <20180703191500.2374-1-thellstrom@vmware.com> MIME-Version: 1.0 X-Originating-IP: [155.4.205.56] X-ClientProxiedBy: DM5PR21CA0042.namprd21.prod.outlook.com (2603:10b6:3:ed::28) To DM6PR05MB4587.namprd05.prod.outlook.com (2603:10b6:5:9f::24) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: baf7e166-322e-4953-a12e-08d5e1197e18 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989117)(5600053)(711020)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(2017052603328)(7153060)(7193020); SRVR:DM6PR05MB4587; X-Microsoft-Exchange-Diagnostics: 1; DM6PR05MB4587; 3:4p8FpOXoA9dqSCo9ngZWkzWBamLe4uifNis2cFZcBM9cwPF4ipRUgt21T3L6t69qWVLriNyT7i1Osa5JMSGg+GRzCl+rfUT6u9OwST6/+g7rDlcb/fqgVQRuzMwohnr/uCtm3zEW1JBv57e3RWhPEkZaah13jYr5jhGLNSRELFRpQ5W1wYpXPW/wg1YEdW85/gm3YteG/z4Flmq0KlXzjlV1xSn1oF1V8HGNEFjL79ipGr92k4MrjZ7/banXwr6c; 25:T1WSQ/EQOWqvjRXTrcc3UOQbRXb3oCq71BKnhupfVvgX7y1xz7ligESqllu0bq8RYKVLpVqo5Lg4vTAJ07wysnyjjnaSaq55C8PGFQ8wgc0vTnogyV9Y5mBFx20vBI2DOWjPkXV91pQi9NMJ0C+drIrK44+IxukhsICZQYN0NYvRq23QFm6koNaNicI31gN6QVcRlu3GWC2bjt4EjwYualMO5BEUCKeslQUQswB3fGbl+lf3MjSwcgCxSUPUlL22mlxvMWhbsPtXkOZKjBY8205Wo49ge0oonR/Q8rFGWRWEcJNeUTm5OFdFd1PPmx9Hwyu5/nLaX4QdL62cE0PlvA==; 31:uI3cDaS1nu07WwcrubGPI9TK0QAJJm3LdV1b7sOPg79MmXsYhm40kXA8gpm3RmhFo5f1dDmeKemMtPTxK6ZTEUkwuf0yEU+LxwIrn4demttye7TvLHhgnBN47xVO/nPJASPhvxF1bQDcWOMr0tpEQgHw1BJ0ziQM/mVqeJcZsdPHwsuuT5KgtW8AxLUkkYE/41K5x6eK5jgn7+YVoTVzXUii9RKQtKnF8p4DxmpA/wY= X-MS-TrafficTypeDiagnostic: DM6PR05MB4587: X-LD-Processed: b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0,ExtAddr X-Microsoft-Exchange-Diagnostics: 1; DM6PR05MB4587; 20:wR/2EQ+YkrBYony5LvJ/MSEQTyfnuIim1c+CNlvKrUl1m8GxqKjvc+ZCvoRvM/C5Tcuy1K8aS4jxwQ5EpbuKrfTGOEiKVhAKLFg4KIOXsiCp71ZAacExqre0xZ6GH0w1r0fnnhbhs+NzqVVnC+tR+6jWvSk2snsqKxtdbAqADscI2SppznL3E5M6ftl83ZdTDeI9V3nsgHYMH6vbqSCx6dhjHKicyoqqpTPXJ8Sna7J3rnHM2Ktmb5rkiglBzg98y5K9s1pYLte9tN2rmUyUW/68VKIgFnUQuohYHbud5zGhezYF+JgqR2vJcUl/yLHTWrySQXKb8hXSvHax0OoGpksUXpEmpPR/z+uGpbfqh0t5Lu3EIeg6OcvdtPRtaRDYacQ0Hs9sVwRSKlvjzDnY5HUODDq9KC655WFH4+l3I3dytOtzuOwGg7Ysn3lWyA7eit2RGPeIZrIy1sinSuR8ajIvkrDKApxbxwtEAj2dw3c3zugNBRuBLlYTlQO+2ivH; 4:XJS1saMw/nsQLGEKRcRfPL+3t6R5qD00Q6/1d0HExv9tPPg/QmgpHCZApjKDK2v7kVWKWkFvEKk6ww3Gxxz0UgJ7zt8HH7aVDZgt6I8+z7VB0lyFJm7M0fmEwVEV0UE6DkhGWP3bkjUhtOiCL1DBFG8ogHhcZDw/LZ4iiheR6ZwwExUZ9vaKcy9C7Yuh93W0+R1dsgoivYqUnXTBeJHUm5PxNecHPO6Va4a/4+h5TOlvMUewszqqJQGUz+nrp22C7xnkZBp6Y2vPcWggsz2j/6nRPz22wzcnp3rYl8GAz4QOsAItqaGgOxZRHBawG/HM X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(61668805478150); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(10201501046)(3231254)(944501410)(52105095)(93006095)(93001095)(149027)(150027)(6041310)(20161123558120)(20161123562045)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011)(7699016); SRVR:DM6PR05MB4587; BCL:0; PCL:0; RULEID:; SRVR:DM6PR05MB4587; X-Forefront-PRVS: 0722981D2A X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(136003)(366004)(39860400002)(396003)(376002)(346002)(199004)(189003)(52084003)(2361001)(53936002)(76176011)(68736007)(52116002)(51416003)(47776003)(26005)(2351001)(50226002)(16526019)(97736004)(86362001)(386003)(2906002)(6506007)(66066001)(186003)(316002)(16586007)(48376002)(446003)(6486002)(50466002)(54906003)(14444005)(11346002)(2616005)(1076002)(956004)(6116002)(6512007)(476003)(81166006)(8676002)(81156014)(36756003)(3846002)(5660300001)(486006)(107886003)(106356001)(305945005)(25786009)(7736002)(4326008)(478600001)(6916009)(105586002)(6666003)(8936002); DIR:OUT; SFP:1101; SCL:1; SRVR:DM6PR05MB4587; H:fedoratest.localdomain; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; Received-SPF: None (protection.outlook.com: vmware.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM6PR05MB4587; 23:B96+WAt2o8+zg6ZT8amCF/CMMEgR5F4KvzwQAxr2x?= =?us-ascii?Q?q1U4muffUs7BZ80upDupxrhTiZmxt4ZYuhwFFnRHzb3CYS8krsaCJOy12v2A?= =?us-ascii?Q?d1W7kw3TRIcdiWSrdKSHdu4VwKb/RQ68QoyVI7S2PWVBGaBifHcHpygpjZda?= =?us-ascii?Q?InTb6YOBIu3xFJ/2+avAQ1eHfUNqhLNGFJSC87PdXzM8xVAEIb5GY9PPEBSk?= =?us-ascii?Q?NQH5bE7xVAW93RzpVc5hZGkt31V65mMSbClc9AHBF7zUCZbfEcLrcqu9eeEL?= =?us-ascii?Q?5WzMsqoGtfaQxHSuGv/o0IiK/4kQdsVqNvISJDW95xZ9+WOSMrZh1QELZnx3?= =?us-ascii?Q?iCGIT89OelEid9BXN2sA3QaEi8p0Rvz8u5v0xlwTOLpySQoFvZHTVCIP5klt?= =?us-ascii?Q?8UjVS6UFL7aSUuB38eJsMswf/vLE5ef8g9algcGsBJokVGRVW62NxERZQxAk?= =?us-ascii?Q?NpCL+ENbofxxWXMYu4ZNoUvQIeQxhiC6/Bqoawea/02H02eeazOaG1DDJEGi?= =?us-ascii?Q?khYfDfC7umtiJ3SqqLJ5FcROxhiB5v7HxQfCID+o9RZYrBGf9xeL+8FdQlmN?= =?us-ascii?Q?VC1uoO9cQn2tKgFvxheMgKkn6JnVXLpCzZDcflMBfOX5YjKKS3vqjzv2eWBx?= =?us-ascii?Q?Gjmv7LcfgoLrDvp6rR7FuZgeyKbxPDttOEMhhSZ1xYSVg0a9holMtbbXp8f8?= =?us-ascii?Q?usk38cu3lO0z+r/O4KMWis3YIrNKbLwzcn9zXdFOrENi7uhduXYi8CGmKW9U?= =?us-ascii?Q?v+MpJkee56rZa0T6fm3iGbtEZoInaMtnJT8++8Kz3oM7IGYcRPCORKxof9E4?= =?us-ascii?Q?jBz+2V5NKcNHBikFDamdsCMyNu7aP/QVh0rYaTqsfRCCP2hvGJT0ypiD8KNR?= =?us-ascii?Q?3fTjT9u9kBOokRF9Q4YRrdy14+B/WEsk3QDC47psJuFfDlMnxJUZWrVcDNid?= =?us-ascii?Q?oCXB43NyyV07dgxBRLxAA79knIKnYVC4NhoLKECkVdfY2ZHp7jm8ocqwFojp?= =?us-ascii?Q?sfO+8TD95FHOZpm345dCWk9h7BdCqa+znsiYJzSl3Nqng00I/jgCNwe1aWzn?= =?us-ascii?Q?LHAtMqQMt4Umi3f1QgUINAhujMtfjsMIOlKfoSmibqHQcVlTlcBZNiQMQF9R?= =?us-ascii?Q?I3uoXnx78QmS3A6E6Or7wo6+36OgvSKg49ftslZNQOM2hhjkSvyGOfXtf/XH?= =?us-ascii?Q?LGKWRk1Txu1k0/xomaBQqVdm2vxy0PoJYyBnjp7ecGGCpA+fEw38TTH7PS0+?= =?us-ascii?Q?mSAwAymwLxZaE/rmXiS+xSMTCA4ECMGLt0vcdPxfFe3Yo/wqUvn7y0FAzeuW?= =?us-ascii?Q?qsSkq16IQKPFNJdPr4M7gU=3D?= X-Microsoft-Antispam-Message-Info: kaHV3v/TBcuT9hmhtwBtdvv7vDSe9LXEWQVhacj7VhVyfAOPZKE/mPu60rP4mH3y3dkDqfPt+jDblfX8LFIGF8U4Ptkg3OB1/IcCJwP/wfulqxEbWaHqeMUCqkwml1hwNzytqXSdbNAjonw61HWzihF7GRXn4uNHD4hzFStxb5Mxh5EPIEHUig/kSlSsPINoUiHGxoWkPRHcDkPhp74oeut8SaTdJyP1TatkSpg7j26ASGka49T4juzqZdSUb64SoqamZurOBapSSNNxsWXzj0S969dICPcODUE91R4pwtHDY56EL/U7LLnSZYDDrOHSwSd1oTu2QwYoIZZEEfaKoqL8oV/MV5mvQBQ3bDx3uYA= X-Microsoft-Exchange-Diagnostics: 1; DM6PR05MB4587; 6:8Bh30lDPv6HGeJJ8q4tmTXva7O53Z+VQ+FV9QUg+2ReFVkaip6ld2ZxjxCW+1fUg2ZVM9tm4y0JTMYbJpyYjIx+bvumrNig1GlUBK86/WYyYmdrPc2o677u89B7MQ3qmXIRafm99I8NbFup/gOGAVEmtWBbvlHiDpaDlt4EbKHn31Bn66C9f5GrP846yzKMCPvEB3fUDzOwltZ+hZsS7bvH1ZdBKO0lWGa4yZqmFxSX6rXEoaV9K/h7jA19iRbl7r8++SUGwpxIaPqVKxTwOe5/YVuHF3I8bi/fw4MnyxQWAZQh8KlIP/n88WPAmr3SwHzJniGjOuClTUQjOYgMSbQfMNn6cFg7Y94iEpYukqr3s2qx/j6Ta6xme1R8eLKeZYKU0HMvWqQtZCwy+t21O0D+Q7jJW9eQwfJfRYz1ahlhkQiRNwP91xOC0buJ+nB7z3gszAuP5HdWQPZIEcYGEgQ==; 5:1sgAoc8+599MRSVn5ujg3dieAXP4FBaDt5tQmom9e1xFjAL9bs4Y9kJXAfG7P6RX5aAY2I2vs5jLU/TuGkqptNgWQbQn39NYcCZHWt4gscUXxbZcK/+58DMpZiVF6VqGJrSOzKdVpmvAx6DAxPyz4wQvbb0feM/qVZEN1w8m/NU=; 24:DuE6ovede586ngArjGCq8nnUQaC4Aiux7gQ2+8i+Ny5AH5YSaYwdkBqTxAxxBjUuTiXRhMWgQOHWdzrQTFBidx/1rfKQ/t5G5qJQJUlwdH8= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DM6PR05MB4587; 7:C76DxtSPsYgPJAVpoXHR1t6jsvXnwZtbBXNQTH2fgEKV61T7HpLLT9hy5E21ezqWqG6DiQ7oZvWAxnXWL+1MfmaivDp1hIHhnx+NUv4UgV8B53kCfML4a69vaIwssnrj/c/h0Vv2CPuVfB3Z66PeVsoMHXdVp/c9wKcOkPD9yy/vbJ4NW3G7Pu7FEg9Gn+SiRXuBxHT9cx+7aKjzGq1BzRPta6FtAqNMpF6L29sxYrXpSofwDbYsDaCoGNJdwbq3; 20:NXFEPejHCNmEGIG1ulMO8uSVhcd/AhFfpvy/xlIUQxk9OBOJh6vaDpTDyX27lOoKbSig+ynbv0zsQFm0SfU2NUWaie+LMX4T85xBycGDs1kYMylGj6F95SuHxXU7z+qyG4vqgsWhd9XScvgVXOzGsga38hzHBGGz2/cOf7+447s= X-OriginatorOrg: vmware.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jul 2018 19:16:34.0392 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: baf7e166-322e-4953-a12e-08d5e1197e18 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b39138ca-3cee-4b4a-a4d6-cd83d9dd62f0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR05MB4587 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Hellstrom , Deepak Rawat , linux-graphics-maintainer@vmware.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: Deepak Rawat Call the same display memory validation function which is used by modeset_check. This ensure consistency that kernel change preferred mode/topology only if supported. Also change the internal function to use drm_rect instead of drm_vmw_rect. Signed-off-by: Deepak Rawat Reviewed-by: Thomas Hellstrom Signed-off-by: Thomas Hellstrom --- drivers/gpu/drm/vmwgfx/vmwgfx_kms.c | 120 ++++++++++++---------------- 1 file changed, 51 insertions(+), 69 deletions(-) diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c index a7c9a017fc3c..387bb39de839 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c @@ -1968,13 +1968,15 @@ void vmw_disable_vblank(struct drm_device *dev, unsigned int pipe) { } - -/* - * Small shared kms functions. +/** + * vmw_du_update_layout - Update the display unit with topology from resolution + * plugin and generate DRM uevent + * @dev_priv: device private + * @num_rects: number of drm_rect in rects + * @rects: toplogy to update */ - -static int vmw_du_update_layout(struct vmw_private *dev_priv, unsigned num, - struct drm_vmw_rect *rects) +static int vmw_du_update_layout(struct vmw_private *dev_priv, + unsigned int num_rects, struct drm_rect *rects) { struct drm_device *dev = dev_priv->dev; struct vmw_display_unit *du; @@ -1982,26 +1984,14 @@ static int vmw_du_update_layout(struct vmw_private *dev_priv, unsigned num, mutex_lock(&dev->mode_config.mutex); -#if 0 - { - unsigned int i; - - DRM_INFO("%s: new layout ", __func__); - for (i = 0; i < num; i++) - DRM_INFO("(%i, %i %ux%u) ", rects[i].x, rects[i].y, - rects[i].w, rects[i].h); - DRM_INFO("\n"); - } -#endif - list_for_each_entry(con, &dev->mode_config.connector_list, head) { du = vmw_connector_to_du(con); - if (num > du->unit) { - du->pref_width = rects[du->unit].w; - du->pref_height = rects[du->unit].h; + if (num_rects > du->unit) { + du->pref_width = drm_rect_width(&rects[du->unit]); + du->pref_height = drm_rect_height(&rects[du->unit]); du->pref_active = true; - du->gui_x = rects[du->unit].x; - du->gui_y = rects[du->unit].y; + du->gui_x = rects[du->unit].x1; + du->gui_y = rects[du->unit].y1; drm_object_property_set_value (&con->base, dev->mode_config.suggested_x_property, du->gui_x); @@ -2322,7 +2312,25 @@ vmw_du_connector_atomic_get_property(struct drm_connector *connector, return 0; } - +/** + * vmw_kms_update_layout_ioctl - Handler for DRM_VMW_UPDATE_LAYOUT ioctl + * @dev: drm device for the ioctl + * @data: data pointer for the ioctl + * @file_priv: drm file for the ioctl call + * + * Update preferred topology of display unit as per ioctl request. The topology + * is expressed as array of drm_vmw_rect. + * e.g. + * [0 0 640 480] [640 0 800 600] [0 480 640 480] + * + * NOTE: + * The x and y offset (upper left) in drm_vmw_rect cannot be less than 0. Beside + * device limit on topology, x + w and y + h (lower right) cannot be greater + * than INT_MAX. So topology beyond these limits will return with error. + * + * Returns: + * Zero on success, negative errno on failure. + */ int vmw_kms_update_layout_ioctl(struct drm_device *dev, void *data, struct drm_file *file_priv) { @@ -2331,15 +2339,12 @@ int vmw_kms_update_layout_ioctl(struct drm_device *dev, void *data, (struct drm_vmw_update_layout_arg *)data; void __user *user_rects; struct drm_vmw_rect *rects; + struct drm_rect *drm_rects; unsigned rects_size; - int ret; - int i; - u64 total_pixels = 0; - struct drm_mode_config *mode_config = &dev->mode_config; - struct drm_vmw_rect bounding_box = {0}; + int ret, i; if (!arg->num_outputs) { - struct drm_vmw_rect def_rect = {0, 0, 800, 600}; + struct drm_rect def_rect = {0, 0, 800, 600}; vmw_du_update_layout(dev_priv, 1, &def_rect); return 0; } @@ -2358,52 +2363,29 @@ int vmw_kms_update_layout_ioctl(struct drm_device *dev, void *data, goto out_free; } - for (i = 0; i < arg->num_outputs; ++i) { - if (rects[i].x < 0 || - rects[i].y < 0 || - rects[i].x + rects[i].w > mode_config->max_width || - rects[i].y + rects[i].h > mode_config->max_height) { - DRM_ERROR("Invalid GUI layout.\n"); - ret = -EINVAL; - goto out_free; - } - - /* - * bounding_box.w and bunding_box.h are used as - * lower-right coordinates - */ - if (rects[i].x + rects[i].w > bounding_box.w) - bounding_box.w = rects[i].x + rects[i].w; - - if (rects[i].y + rects[i].h > bounding_box.h) - bounding_box.h = rects[i].y + rects[i].h; + drm_rects = (struct drm_rect *)rects; - total_pixels += (u64) rects[i].w * (u64) rects[i].h; - } + for (i = 0; i < arg->num_outputs; i++) { + struct drm_vmw_rect curr_rect; - if (dev_priv->active_display_unit == vmw_du_screen_target) { - /* - * For Screen Targets, the limits for a toplogy are: - * 1. Bounding box (assuming 32bpp) must be < prim_bb_mem - * 2. Total pixels (assuming 32bpp) must be < prim_bb_mem - */ - u64 bb_mem = (u64) bounding_box.w * bounding_box.h * 4; - u64 pixel_mem = total_pixels * 4; - - if (bb_mem > dev_priv->prim_bb_mem) { - DRM_ERROR("Topology is beyond supported limits.\n"); - ret = -EINVAL; + /* Verify user-space for overflow as kernel use drm_rect */ + if ((rects[i].x + rects[i].w > INT_MAX) || + (rects[i].y + rects[i].h > INT_MAX)) { + ret = -ERANGE; goto out_free; } - if (pixel_mem > dev_priv->prim_bb_mem) { - DRM_ERROR("Combined output size too large\n"); - ret = -EINVAL; - goto out_free; - } + curr_rect = rects[i]; + drm_rects[i].x1 = curr_rect.x; + drm_rects[i].y1 = curr_rect.y; + drm_rects[i].x2 = curr_rect.x + curr_rect.w; + drm_rects[i].y2 = curr_rect.y + curr_rect.h; } - vmw_du_update_layout(dev_priv, arg->num_outputs, rects); + ret = vmw_kms_check_display_memory(dev, arg->num_outputs, drm_rects); + + if (ret == 0) + vmw_du_update_layout(dev_priv, arg->num_outputs, drm_rects); out_free: kfree(rects);