From patchwork Fri Aug 17 10:07:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Huang Rui X-Patchwork-Id: 10568571 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 983E214BD for ; Fri, 17 Aug 2018 10:08:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 857C92AFF3 for ; Fri, 17 Aug 2018 10:08:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 77F732B020; Fri, 17 Aug 2018 10:08:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 142142AFF3 for ; Fri, 17 Aug 2018 10:08:59 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0CE856E5FB; Fri, 17 Aug 2018 10:08:58 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0089.outbound.protection.outlook.com [104.47.38.89]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1A67B6E5FB; Fri, 17 Aug 2018 10:08:57 +0000 (UTC) Received: from MWHPR1201CA0006.namprd12.prod.outlook.com (2603:10b6:301:4a::16) by DM2PR12MB0249.namprd12.prod.outlook.com (2a01:111:e400:50d1::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1059.19; Fri, 17 Aug 2018 10:08:54 +0000 Received: from DM3NAM03FT021.eop-NAM03.prod.protection.outlook.com (2a01:111:f400:7e49::204) by MWHPR1201CA0006.outlook.office365.com (2603:10b6:301:4a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1059.20 via Frontend Transport; Fri, 17 Aug 2018 10:08:54 +0000 Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXCHOV01.amd.com (165.204.84.17) by DM3NAM03FT021.mail.protection.outlook.com (10.152.82.187) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.1059.14 via Frontend Transport; Fri, 17 Aug 2018 10:08:53 +0000 Received: from hr-intel.amd.com (10.34.1.3) by SATLEXCHOV01.amd.com (10.181.40.71) with Microsoft SMTP Server id 14.3.389.1; Fri, 17 Aug 2018 05:08:52 -0500 From: Huang Rui To: , Subject: [PATCH v4 3/5] drm/ttm: add bulk move function on LRU Date: Fri, 17 Aug 2018 18:07:59 +0800 Message-ID: <1534500481-15123-4-git-send-email-ray.huang@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1534500481-15123-1-git-send-email-ray.huang@amd.com> References: <1534500481-15123-1-git-send-email-ray.huang@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(136003)(396003)(39860400002)(346002)(376002)(2980300002)(428003)(189003)(199004)(450100002)(486006)(11346002)(2616005)(2906002)(476003)(126002)(446003)(336012)(426003)(53416004)(8936002)(4326008)(14444005)(47776003)(97736004)(106466001)(36756003)(5820100001)(105586002)(54906003)(110136005)(316002)(86362001)(104016004)(356003)(68736007)(72206003)(478600001)(81156014)(81166006)(8676002)(76176011)(2870700001)(23676004)(7696005)(77096007)(26005)(186003)(5660300001)(53936002)(50466002)(305945005)(50226002); DIR:OUT; SFP:1101; SCL:1; SRVR:DM2PR12MB0249; H:SATLEXCHOV01.amd.com; FPR:; SPF:None; LANG:en; PTR:InfoDomainNonexistent; MX:1; A:1; X-Microsoft-Exchange-Diagnostics: 1; DM3NAM03FT021; 1:k8y80Rqb3tHAewvpupiaT1ACzGhOUZjkdNYcxneB7FxgFrcfpWYA3DHMVQoHzgbaS+bVOnyZl1XbvyyTCbKFJ4hopdnPV2WuoMHspes+D5YxQkcKbuY83pfjCKHRoIDD X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 89c6ab80-2ff0-4159-4a1d-08d60429700a X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060); SRVR:DM2PR12MB0249; X-Microsoft-Exchange-Diagnostics: 1; DM2PR12MB0249; 3:XGP5o6exTqv0VGM9VKBtUxaHj2Ifrkrkl4Uv3gk6DKtF8d2uQeZKN3GB0GMzaJIJjeSTUrpvLm8G3ehcbuupza5ZWUl8Z2ig7qx+YVMbZT6WZbt3+KAbmT664yIMYAGFWv5DkXEsjIAXjItCiW2G870pfsvAaboIDkqS1KZY68Mqq0eQgodauBAuO8rsGo+Sg/LUrEitRkm48WGcbQifzJr3uUs/AH/Zlv08UYclr9928Jz3SBhpoRdSetQ8AYrzqKTpkSZhUQlmgZ1Luxhl7ykhXzaNnSikHstfzk8ihrEuxRZWZ/c3mzMB8Yf7DnM64ZoOzkjdhjGvFADu0gCo4CIhHWYxpL0JbyFUubkOXqI=; 25:2hmDLrUcTNB+agtSrUoFmZ+veUyKRunJ0npQ0c81uu4qlKsIOY1FrCvm0KxM0AO+2Yk1mzAHT4xawl4yNdPbw9/B5k+yHtRVMGucJg0wHy3VJrCYgaWOQujoTC2abzWZi6zQReNsIwJ6DdDkz1HZKMO/r4hx5+3l/TF91zIPlXtxqCnK2IB76iPpo+fSHGYBxyby4BXXUxJefLVmBCcs/bzqZbG1iyJnnw/XPNKluUJdsIb7YOC7TMX1BVu46YD/KqvL6QlvX9tccc6512XXrmlgqJ/gZ7hS1UgJ1J+ySHOqEeiyd0CM1qTunhTCmOsCyTO9ctSn+DGa8JRXgVGXPB9Ay+gEOyQ9dfvIyBJIXUM= X-MS-TrafficTypeDiagnostic: DM2PR12MB0249: X-Microsoft-Exchange-Diagnostics: 1; DM2PR12MB0249; 31:Tv+yPexN7PDzDT86/n9rMoiOuQcPo32NZFzaWow0btqbU0Cnq6YglM8GyxSpbje0mbGJ05dx0JeV2TuELZKNCZIKco00c+Kmi1crpHsKhl87QIMGyrlehEq2JLZfoVlF/Cwhxbq3oE4eVqjTa8D1k36lNPg2BajDh9fZ83d3Tw2JF8U+j6HZzdFtz8YHJyn6YE6PdX2QRpyRgTyyqzuXOYglZO88K+FVsAPli0XcPsk=; 20:mO+VsaMQ5EQ3DULeO0RHMJpaHAovxqdJggWuhWlkI5e9QKmEMtubTMvGtvFhe7Npr/HD52Bx5ggl3JAEyMcJmbhXKL+pxapLlXU09S39nrS4hNGKhyd16NzwkoQpvaB12zmKuWiJXyb1+0ARrhsGrzgEMBpUHQNdUo/0IXzO+G59Lw9Ej3l+zVgD7UNzkbS8c2hKw7lLiO1zxC/mhcNK8/nofCjsvgRvD2Ds+b5sr1WUvrXT+NpkF8IA0ie2otPJvF180ZkosoU71Y6qp8R/ZuIYAurkFu7uCDL3hhDI63RGuOff1if6RHbcvUoGpjhLXbtJVpAeYEbIBKc8RXyItV7S75yAVx/1pR1SAdhYOet/mTSvK1D+wtFjpCjtLJ448z8tkVk3y1bMQb6luSJ5ROHXnv2LDafxJ/vttsBOjYPb//i4HHL0sZ9aXNPdV3VP9/TrJkUK3XlQI9Un6RC6myWCMBcsEVmyHyhFZ+SHY9tDVLAWSyMJagY4+mdVUmag X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(93006095)(93003095)(3231311)(944501410)(52105095)(10201501046)(6055026)(149027)(150027)(6041310)(20161123560045)(20161123564045)(20161123562045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699016); SRVR:DM2PR12MB0249; BCL:0; PCL:0; RULEID:; SRVR:DM2PR12MB0249; X-Microsoft-Exchange-Diagnostics: 1; DM2PR12MB0249; 4:LEFZn2GlnT9boMeY+GFhDyeDt5anuDeUcSYHARYBShb9IVFvsY7YtI3hMBwABhIpj4w9iTdW+2mn06UDf6xK1nxYBecLjBZIkEmBeGmZl1tVzlVTj0p/xJcHlHORtFlDTo8roJAcG0yAHIFWIFGX4WEnXJXtE3+4wnuGrVHQLpEseg8jvA4Nt8uHFFLc2L+nRmP45/fpYuHxe8mi9YtA/hdAJTkV1+/84LW8VNJhM941xpqwEOYSAUcQhKRHTOfIF62xaEXYeB+FPQbSGjsDA/2kDZQcs+Csy56/p3cPnK2NLVZv3ADhnUgnpbgLB0iC X-Forefront-PRVS: 076777155F X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BDM2PR12MB0249=3B23=3ALr+VNG7?= =?utf-8?q?bGth/Kux+7pKBrbrs2umoEUDjDzPMAJP5vNz5lizNKdlrQlYiGOv/hWbgA2pJOV8x?= =?utf-8?q?wFlgLKLDJxZTWrFf96goR4hzVyiki80TH4OkQDfdS34+AGW3V2mTZwgAadr08lxRo?= =?utf-8?q?LbeVOIf5uvJm3a6Yrro77qhhWEOkw5XLh4DMRn+cBJtr6rIT7r9KsEwYkeeJRlHKD?= =?utf-8?q?nOY6T7Z3eSyYP/EoqCPcncQkJ54J+S5SVgXCauU5hjOQQoaoCR5J3jPMNDdgMLNNI?= =?utf-8?q?Hal31puopJhfrKZ47V4f3tTLDJj3FIKUWOvENxI7OglvVhGd45J94+MKoQiOFESBn?= =?utf-8?q?ESUKJm8kD4eTDfSJGW0IyVL9bL0kJddrRic3l5kjW8H2PRvrFRnbh6XElQhHx04uS?= =?utf-8?q?5W5NFB/Fj6RdbyIx0zOOClBYr2eap60yK/FBLM/UKTHDvDAfN79UKYs8qyT44V3Yy?= =?utf-8?q?jLvYIaDGepYf+f1HP0zf+O7dXtRB0jbbXuBN/w2JIoBR3V+ctGBkMo/3V7bxyS9b4?= =?utf-8?q?NiJrgbTYttcnuLbE0p3NpYJtEULDVo8WIvH6mVa+Wjm4NKVkKQrQy9D+HQjI7DjM2?= =?utf-8?q?6DzOxLCV3JXfTfxeUiM4lPfrLtT8HMh+qYiZDgiTwzToJ5Sz5kExRV88nMZjY056W?= =?utf-8?q?w3FLanqWiJV/xufaCTDKh3obprdrRfVuYHdJ+YWC0o749F8vYMNLdK4rBp4AQ7+NG?= =?utf-8?q?1b52m+A00ZJnIUoONxBdk2UFYZiyEYxXqmQUHYd31iAU+EoIV1E5ammyiBlPwJr6b?= =?utf-8?q?mtjRFveTHt6/1oSTVsoy68aawIp45+XvKCOzvY0c4rFxIWU+TMzPEJdCobGto1IQ/?= =?utf-8?q?ii6nG4kuB208mpZMM+eUk1u4cHOfn9Pae/ZsbJaHBiBb9QBGKTNtAbbmwKgy3SOmz?= =?utf-8?q?5sMRP96rNZrF0c7OuKtdoqYkUkkgDMWwCadCvjGgc9NA/EqDyCBvUq4AG9tP5sfSJ?= =?utf-8?q?3uVl08HRRo1xNdd+qTSYzJzmLMTCl3SPHinv1za5HaGIWGPRhafpgoR1nJxzRJVMv?= =?utf-8?q?a5YeMo9Evh3fkBJA6UNJ30Vl+ZO745DXa74shBi/T2CqLywjaYParTZdtVdOc03wA?= =?utf-8?q?1dE4ehE/nRXftJ1v0NCwxbXRAYuBUMD5OM3hwqQvk5sEcrYMCaWOTzbVAqORND7ss?= =?utf-8?q?e/qt7irxTf52A6XPCY=3D?= X-Microsoft-Antispam-Message-Info: AvAIFymgzTeO6BOMcTnGCGPv2Xm9R8FWPaF70n3PjcHMd1VCBBUDVxi6enJa9FhEt3orbWAMDa+VmBiOstUTCj8WximyKWv5HNjYO+UeDniV3kgUSYGjJxyleS2X8uYcPwRxXcWq0n9IFPcjLwLMQ6f2ALdcTij6xO6MTYTHPCrxvvvRc+lxH5Axoh0HzIHUwnK01qot7TjzEmR0S7NAAlUUQXum+Ot4GZgCkMkYhkeB5ArOlrrXOX1e0XdC7qQvWOnbURsQBcrK1JQhJIrC+pauJby4bTSKLrTUbSEjwHIo7HD3Lyl4auAINrOhZzRLlPs6QhQDOMmomX0R3wmti7Y6mWJSrXfxE/a1q+50Ph8= X-Microsoft-Exchange-Diagnostics: 1; DM2PR12MB0249; 6:31ABODFn14IgtbZ0g8Igi1NHoQU/4qysJ6CwmIqf8qPgPX9X3fHx3VM1eO6vHLOBEvoh+ux8vVYjBnFO+iKJisnHu66//LK2+p0r9ODrqqyV4v943xkTUguxBYH8Xv3/QkGHSdeY9/b2ZaJu9oytXEDm95a2Icprm0v2HIKVdhfHT5bUxE4pbm7hwd//aFiMaZuQhlywmGgoLg7ly3qAuLNAVd7+pbxr3V3P84592OZbsjNYQQ36+X9FjQ6S+X+0ycVaXbWk1BQv4FdlkMPSM/A9+/1Kaz+ZPaBS7FwfNMW4rHWJo8mlW5HvxdfbWWGTeZkxm1ho0Wzmv3uCvjc6TVY2qdvQ74U4dJGEyA78/MRyFUz035BrCt3gtytrAZzT16d4hvUqze1DIseWbKfrhcO2pnAp2x0sMjM0oLwJDQSiY3MdbZ4bKy+jNLKC16rtdn8/CV2wrTGbLt6bR5P+tw==; 5:pq+A59cyrqZRBlkKovvnpwecVvhw1simvl5d65jgdziHlm9vnW1VuC1ogPNLJkl6+J7mBRw84LYvsQsSHHaC761/I+NuqIl9s3Ep7aioVyVVcClf5ozaQluopjudJMaQMLbBV18/IympIcN+21O+HuzX2XJ0J5RmdQ/Dd2NdSWY=; 7:kG5Qaa4xy2LvxgFhIDA86L0ULUshTqWJ34ZTnQxh5vmCfHHAUM3rVBFyDTtGIPz0M7k1edEECRpLMsTMQttBuYBlwMP/FbCj6pyfFEQB3xZPuPmXaNAsJhvlIFJ3SvIHaGPaiwjiJ7zeRdgCOki2xww6AnaMoH2+uCzGJ4FVtEWYqIU+BIBqx5b8CxQrThyGP2RCSQq9LtWi3G7HsnOdVpnQhVN9OUSOgy8pDfuz71xNbG8nlW5w1h/LpfVlCocN SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DM2PR12MB0249; 20:VT4kN7yhbvh7ETmtpVHaBjlvvumGMdHe3M3Q1VKZNfrYJvyPJfnYVXpVGk6FoM3DMGRCrc9iua3Rn9AMrxRSHD59aMjWLaer78i8pGmgLLOgZkEki8LTyciPqRQ8WQ97X6DZvFXnjGocb87yLRew6Jch7IhYnjkUYdhWyzhYLOrsirXjeWMaZ/+JIT6Bb7hrwmtmXayYmjHysAhV8lqB5ZdV5aqlW23rH/uOjzk+keVYpzgEC0wKGTNK9CWrbBZ0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Aug 2018 10:08:53.7521 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 89c6ab80-2ff0-4159-4a1d-08d60429700a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXCHOV01.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM2PR12MB0249 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Huang Rui , =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP This function allow us to bulk move a group of BOs to the tail of their LRU. The positions of group of BOs are stored on the (first, last) bulk_move_pos structure. Signed-off-by: Christian König Signed-off-by: Huang Rui Tested-by: Mike Lothian Tested-by: Dieter Nützel Acked-by: Chunming Zhou Reviewed-by: Junwei Zhang --- drivers/gpu/drm/ttm/ttm_bo.c | 52 ++++++++++++++++++++++++++++++++++++++++++++ include/drm/ttm/ttm_bo_api.h | 10 +++++++++ 2 files changed, 62 insertions(+) diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index 7117b6b..39d9d55 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -247,6 +247,58 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, } EXPORT_SYMBOL(ttm_bo_move_to_lru_tail); +static void ttm_bo_bulk_move_helper(struct ttm_lru_bulk_move_pos *pos, + struct list_head *lru, bool is_swap) +{ + struct list_head entries, before; + struct list_head *list1, *list2; + + list1 = is_swap ? &pos->last->swap : &pos->last->lru; + list2 = is_swap ? pos->first->swap.prev : pos->first->lru.prev; + + list_cut_position(&entries, lru, list1); + list_cut_position(&before, &entries, list2); + list_splice(&before, lru); + list_splice_tail(&entries, lru); +} + +void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk) +{ + unsigned i; + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_mem_type_manager *man; + + if (!bulk->tt[i].first) + continue; + + man = &bulk->tt[i].first->bdev->man[TTM_PL_TT]; + ttm_bo_bulk_move_helper(&bulk->tt[i], &man->lru[i], false); + } + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_mem_type_manager *man; + + if (!bulk->vram[i].first) + continue; + + man = &bulk->vram[i].first->bdev->man[TTM_PL_VRAM]; + ttm_bo_bulk_move_helper(&bulk->vram[i], &man->lru[i], false); + } + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_lru_bulk_move_pos *pos = &bulk->swap[i]; + struct list_head *lru; + + if (!pos->first) + continue; + + lru = &pos->first->bdev->glob->swap_lru[i]; + ttm_bo_bulk_move_helper(&bulk->swap[i], lru, true); + } +} +EXPORT_SYMBOL(ttm_bo_bulk_move_lru_tail); + static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo, struct ttm_mem_reg *mem, bool evict, struct ttm_operation_ctx *ctx) diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index 0d4eb81..8c19470 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -417,6 +417,16 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, struct ttm_lru_bulk_move *bulk); /** + * ttm_bo_bulk_move_lru_tail + * + * @bulk: bulk move structure + * + * Bulk move BOs to the LRU tail, only valid to use when driver makes sure that + * BO order never changes. Should be called with ttm_bo_global::lru_lock held. + */ +void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk); + +/** * ttm_bo_lock_delayed_workqueue * * Prevent the delayed workqueue from running.