From patchwork Fri Aug 10 11:55:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Huang Rui X-Patchwork-Id: 10562647 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 49A111390 for ; Fri, 10 Aug 2018 11:57:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3BBA12B65B for ; Fri, 10 Aug 2018 11:57:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2D18E2B657; Fri, 10 Aug 2018 11:57:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B9D4E2B5BA for ; Fri, 10 Aug 2018 11:57:54 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A6A906E8BE; Fri, 10 Aug 2018 11:57:53 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on0079.outbound.protection.outlook.com [104.47.36.79]) by gabe.freedesktop.org (Postfix) with ESMTPS id 77E9B6E855; Fri, 10 Aug 2018 11:57:52 +0000 (UTC) Received: from MWHPR12CA0050.namprd12.prod.outlook.com (2603:10b6:300:103::12) by CY4PR1201MB0120.namprd12.prod.outlook.com (2603:10b6:910:1c::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1017.18; Fri, 10 Aug 2018 11:57:50 +0000 Received: from BY2NAM03FT003.eop-NAM03.prod.protection.outlook.com (2a01:111:f400:7e4a::203) by MWHPR12CA0050.outlook.office365.com (2603:10b6:300:103::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1038.22 via Frontend Transport; Fri, 10 Aug 2018 11:57:50 +0000 Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXCHOV01.amd.com (165.204.84.17) by BY2NAM03FT003.mail.protection.outlook.com (10.152.84.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.20.1038.3 via Frontend Transport; Fri, 10 Aug 2018 11:57:48 +0000 Received: from hr-intel.amd.com (10.34.1.3) by SATLEXCHOV01.amd.com (10.181.40.71) with Microsoft SMTP Server id 14.3.389.1; Fri, 10 Aug 2018 06:57:47 -0500 From: Huang Rui To: , Subject: [PATCH v2 3/5] drm/ttm: add bulk move function on LRU Date: Fri, 10 Aug 2018 19:55:57 +0800 Message-ID: <1533902159-14232-4-git-send-email-ray.huang@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1533902159-14232-1-git-send-email-ray.huang@amd.com> References: <1533902159-14232-1-git-send-email-ray.huang@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(39860400002)(396003)(346002)(136003)(376002)(2980300002)(428003)(189003)(199004)(450100002)(53936002)(14444005)(5660300001)(36756003)(4326008)(72206003)(2870700001)(86362001)(68736007)(356003)(478600001)(2906002)(81166006)(305945005)(81156014)(8676002)(53416004)(2616005)(486006)(476003)(126002)(50226002)(76176011)(446003)(105586002)(6666003)(11346002)(104016004)(426003)(23676004)(106466001)(7696005)(50466002)(54906003)(77096007)(336012)(110136005)(8936002)(186003)(316002)(97736004)(5820100001)(47776003)(26005); DIR:OUT; SFP:1101; SCL:1; SRVR:CY4PR1201MB0120; H:SATLEXCHOV01.amd.com; FPR:; SPF:None; LANG:en; PTR:InfoDomainNonexistent; MX:1; A:1; X-Microsoft-Exchange-Diagnostics: 1; BY2NAM03FT003; 1:fmaJ0RjFE0wB85OeLtVeQons3V8ztzcLKOJVmwcjZKr87nZ8omX0eSGlRUILGyuMs15xKbF6cBVbPOAwAs5rMj2Tw1BmDSV7pzLg6la7ojOEbi5jBhILej87SY6LQ/PF X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b005d80c-a228-4a39-c847-08d5feb87ea2 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989117)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060); SRVR:CY4PR1201MB0120; X-Microsoft-Exchange-Diagnostics: 1; CY4PR1201MB0120; 3:UvQOKpqRowzi0DbXahXCAnH2toCK5bgx+lrz+g4rGeug7zpIJiLRWh5/EdtM2fWE4Af57G8VcSDQ6n2/xccT3ZiGa29NHgPRtWJ2Yqnq2imhHJq4lxLu+As0gsBcin5McdNwp1WTKAhyZ+Hae2ecdgkGUFZzyhPKRs3Jhx+T6rjJjBHU3X3XZw7JB61TAyODbux40X1sKJS/baWR/voTztSpOy7/Q9MqMfOyIjaGKPxHoGMQF5VNVgWZgBr5/lr/8TGh7GqGjkSf0NfpnnfxxWuT1h803Nf6HwH/obwR7hcN3osMVFZvrqa6QyQ+p6ZETZEjqx8xwFB70/ziVryR8LOYs8tOcmHl24TUJG2Bc04=; 25:6+FTZ1ID5Y9xEGy8DJhdVPYjL6/+EzyPW2aj6H/y1BkxyMNzSXLZbk69Q914txvms2QnPFdb3oE6a18sJzC6x2tDOcSXLUvLwqQ+FthvGqw4TH+J+8ysqm0r7rL271PVVoFh4XDujBSTTepnXqCiRU035VjztSyq/2uoqi1BEe8NC9gbBldorWQNJUM1SUGKb0aRLpCBWqqTfEMLKbT5uZUsFuDPvmh/RoLyvLqd0TbEDAoyYfC83JECeV2ok6OMZCpERmJOxR8du6uQ4PHD/On+v14QvGW2FUYTdx+3WB7mInSdkfISMl429T9UHBI35Vw3zl9+5BRFLUk1PvzQWQ== X-MS-TrafficTypeDiagnostic: CY4PR1201MB0120: X-Microsoft-Exchange-Diagnostics: 1; CY4PR1201MB0120; 31:hjeVz0R4ynrJ1Qel1b3m8GsoWq53OofMD/VKfCiUn2C1jJBfat0IsRNwRL94FhqOagt5Ol2st1DF9PwsOuvm8WAGtBYP40PMMpnHG1Zd4ILk7YNhN327RWNFAJF7cOKoPOX04Wjr+15Yhsyj1rLS2iXcZ/14xXBqcE/2p7akOT4CFdYSXtotqmmlxOaANnq15V5Nq6iWICX3bS/0cYzC/tcybYtdC/V8NVLY8Xe1tgs=; 20:M04rd6Oe7RPSUZwKohwDkvxc3ZZukjiK5VfraAFUi9a936qYUSaiz2+RPQK/5TpyCSI0FGG9j1L+uW03vozBjw5bbStoZx3GRkobLHuR4tiOvEYh5dV0yU+emAs5zKmREdZKh3Wk/K7r7WcfrfXKasCPKWOonqcyXdeo2bAadovPp2hAZoMsd3kXN7UNastjbwux8Qjxg3ye5PNfOROJ4FKJvttIF+hfyN9Xn+azZDoul3VtlIhTV67BBSGth+Ckyn723NPqnrdoyN/n0mgPaEa6p3JxJC51+396luqeBrauLjXFbEW7XwAW+xoGftHpmzHL8jg8SgggqEX9q0yec1HmP5PgPFmpZItHNi8WgppqP8xJ8D7nTjeTfYV0PSm2w/BF+I/J3O/Eui1VFiVSS5FnZBsgSdEjm72/AiOKK3Vt29jp7fbuTIc9Xb5t3QTJCohhnOLBBMklAhkaewGKxKwiRCUAG1RLsZjYEGFMLcdcNUOGQ9Fv5oe2h0qv3VFO X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(93006095)(93003095)(3231311)(944501410)(52105095)(3002001)(10201501046)(6055026)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(20161123564045)(20161123558120)(6072148)(201708071742011)(7699016); SRVR:CY4PR1201MB0120; BCL:0; PCL:0; RULEID:; SRVR:CY4PR1201MB0120; X-Microsoft-Exchange-Diagnostics: 1; CY4PR1201MB0120; 4:EdQvdh6GE+MubXVzmoioz4zzyw23A2ZZelSlfjQN5CFEf9dBVm1/TNf5OzY8BUQYEl6ezxnqm8dgZqokauLKSIH+qNnM/rAJCJp/A5RLu30GWcQK8GzqbigWYnivRoMnrLu9HfAwhDcTcxFqKheXHkoQpHWpj4nKwzOX0ECGiRazU2SVi7QNZwTKwk7ihhJwFPxo6+je3Qs6wGbwEw9zVmkIgaVCM32aCwmOhiPmJMcoyrbemTquep2bF6omSNdbXFQw9NK2oWyVxLrqpML9lLu6MaFdGWbCShAXfdgb8uUZk3fwdyvSufGJjRnFl0fY X-Forefront-PRVS: 07607ED19A X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BCY4PR1201MB0120=3B23=3AAiPKT?= =?utf-8?q?y7zgm9JQk4mKEiidgHU5pKRWbvMQMJqa4iy2VN8x3MFd2IMriUjeWKmacZjBZRHcH?= =?utf-8?q?spDkC+vM0+sgGK5zbmScZ6vF2o/VWS5IGhY6y4t0jmupjn7yLfK20S7AhERg0jTCI?= =?utf-8?q?UNR9e8Ed1WQU99g+lxeVfZw4h8covW9YlvOISk8o81Hqo1UYFdzDGVkok0Tzu2BVI?= =?utf-8?q?L8LZ/LI2RlNtDbumYm0sXLz3IYD/jzmgE7iA5iLNpb/GRo9iX9dIWwSY7g0Sq1AyJ?= =?utf-8?q?dfN9VhNj4YHUMoRHLn/H3o3VeazEV5mJZ1FJJHq6MWqa3Uppe9r2gJLKFgRhgcKWz?= =?utf-8?q?Rvqo8cSzyqV/mfCgwQFTgtoTxD7+/Dzip7wC/nbCxaRks93mAB5gJZ4JbhnWx+R3N?= =?utf-8?q?+bIICdZyo1npo53KmSFXGq4IbSbbzwoHGBbu8TwKa56qJpqFRQaOuLJqBT9Tq4xuD?= =?utf-8?q?uEypHp47qq0JH089fqaSzZsL4oLyrMX7fC8+9cqfjvvN1aUhg0Jd4SVtxYgc2S8xS?= =?utf-8?q?rVmbZ4bE2YkOlAj3F8osYZfSwtAnAFb3IvkMXZwE1N4I9nHNqHxKf5gt936A30YF7?= =?utf-8?q?JGN8G0RbLdHceMeNgL4PTOlXH2oGiudLeZThdrtqKoSVl08rFgNnu9QFOpvWi6/uu?= =?utf-8?q?yHByH1e5ttG0uirDx53bUGfMmqKC7FqrPKet1M8kRhiSqqj1wgcI+DpmA8CVM3Dq4?= =?utf-8?q?cSEhytyoqQ3EtvOS8rnGNwICM17K3IEsbD5PwGB/asVCbg2CerY/STXzcc2FKyZP6?= =?utf-8?q?3Eqa/tM8DC0sQI3zuHsYSy3G6glq+6P0II3m6/MUStJkvH8b35lM7gPP6zNLkdpzz?= =?utf-8?q?taKWOrsLFq2esnArb7xMeD6l6gnqexZXQjbwPUn7n9oEwBrMoweWChSD6dWhTBRfs?= =?utf-8?q?2uiF+44BJICQPtdeLESIGDsmlumiIkWprxRMZfhunk5YCX2RqY6nmBXWRLo3PbVOJ?= =?utf-8?q?DviirO7/1a/aOwF1A61NdNxfwfqMQ46UIbZ0wXCU29Ohy3YGJRmvEIPGJ01VxhNp+?= =?utf-8?q?hvQFzZDaClJtTPI6J5SwbT7w5wT7O5Yzb6+zI4QrArzzxjFTQbE4Tx4na6YWjKjIJ?= =?utf-8?q?jG8VxVmaHahNuwOf1FxkaqsimjikuNBT4UX/S/E1DgaF4/i1GVw10Vbs3X6RGhyjm?= =?utf-8?q?0hU1KacbVWhGqvdQ6QCz8DCk46tfTLFItmuT3azIt?= X-Microsoft-Antispam-Message-Info: cL6cIFAuZqDBYOsi/d2Sj4F1pILHC3c95yd4kpZbTCjGE50Fz4xvUBeIJRp9ni2JoPKjdwfHdKsy4XrvqH5aY4GJF9ZngLRS53IhlB+AjeOGktYnHxfYB+pYy1qBSAZT2PvNBqXAdMVMdU1njXTcF06Q2SaCTB/ZzqgDDst3dJG9pa6wTeCqEEBSA0dWsjFaxOxvBfoLLnCzbL2fdL2MwEhx7H+EKXrHstNslDrBFgJTClUL5X6EuQmY/8vKe9mHtEoxc+e/xM7fqqvSiOUWZn+9vueDgQRqcXFWfWoCk7NBeAMd4PkXcuhuta0LCVWGtaoUspeCDmu8+xHK/0ErTojTmIQe2uWdlcEBe1OlgSw= X-Microsoft-Exchange-Diagnostics: 1; CY4PR1201MB0120; 6:VWAqWIFQrwRNp+TJZJTuUbHHjPMKUsCso3x2OVHE45QiTaax/hjaeUj+axyWNTd2Wfyz1F0Xb82raBxINLJ53Ckuq7LWYHSdq0t4F1Uvjay3XdtUxuUT8CfFsq5hs95K0S+3L6Hn0l65I7cpJ9LW5VBtPYoK6dC6lXlR9kBnXXo85dcLxonvHSKWTcFTKELTdYwnmnyZYfAQnhNxPECLbM3at9sEcJYu21HV1k22+n01VUkMhbCI1DsilzDaUYtXLyylad+vAUm26qZFPqFyrTtyAYjvupFWvPdyS9Nw8w6+A/FSBokTl2ktX+jTfvJEiTElUDuEV4zVahktw60vAC+5OJL2i33u2kV3w5HiqhH+qxDd9LhZPupqHfn73dqsu7af/mYa9KeGi+HH5HbvF8eW4350Hp89QtHBTGDz6cZCwHHnORvAdFXVZp6yjUVxQOQCkvL3tKjeVuVUh+mfGg==; 5:jiEUVkyZ5+ryt8/OsUFvvT7G9BIKAexvn9XmJKmZDrDP4Pg1rzg6+WEYl18mi3kz9Wtyp24HIEc6w/tTBrMOs3iwu55uyXIe4xj7YD1KZrqA20QBimcnLBvANfqBmXxlhHdkz69rFp4fU8UF9HUsXbssOwbuf8UHr818rF5AqcY=; 7:rkug3eDiK4tmo0Ve67Pw2eI3sX78H7hNzFg+gWcu5GwZVeTpsDTvymltxxEJx1xnT0IlYBx+gqufKayRWB0k5897F/K0XtGSkyxCNl5NivKMVfEpXjogfSqSc4HdTKc379rKdzrnE1NoiYOURMzV/8Y4a6+SFPBtEXesJIDPYaQeTUNXYy8h85Cv1tB0oM57WhszBvrsuRvDsXmRLqC+awh1qdOMyUs1U6RW0bZYzH7//Bc/UckrrurkK7YawviF SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY4PR1201MB0120; 20:ohYzHiFg4DLtij/+UpaZsYTQJTlxTR3yKdYhKQSUVXXgEY2KlZ5r3rLmI8ndxUeYA4icHBZkIAbE92k/jk0VVQNaf/ok1jCIMs0raYP3dU9zSAPaxeLaMk1C+CqkTJdSY8oKYwPIOZMhQyIgfJChJthUME6RU3i1oJSZ/xvF3FwpI3Qo4YseAuyOdGSCiNhy4hQAmKRE7AHtc+EJfYpISqSe5D39JQvk8kel3O8suFpiFQBjmmq61cu/mQnNcSs/ X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Aug 2018 11:57:48.7015 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b005d80c-a228-4a39-c847-08d5feb87ea2 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXCHOV01.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1201MB0120 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Huang Rui , =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP This function allow us to bulk move a group of BOs to the tail of their LRU. The positions of group of BOs are stored on the (first, last) bulk_move_pos structure. Signed-off-by: Christian König Signed-off-by: Huang Rui --- drivers/gpu/drm/ttm/ttm_bo.c | 52 ++++++++++++++++++++++++++++++++++++++++++++ include/drm/ttm/ttm_bo_api.h | 10 +++++++++ 2 files changed, 62 insertions(+) diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index 7117b6b..39d9d55 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -247,6 +247,58 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, } EXPORT_SYMBOL(ttm_bo_move_to_lru_tail); +static void ttm_bo_bulk_move_helper(struct ttm_lru_bulk_move_pos *pos, + struct list_head *lru, bool is_swap) +{ + struct list_head entries, before; + struct list_head *list1, *list2; + + list1 = is_swap ? &pos->last->swap : &pos->last->lru; + list2 = is_swap ? pos->first->swap.prev : pos->first->lru.prev; + + list_cut_position(&entries, lru, list1); + list_cut_position(&before, &entries, list2); + list_splice(&before, lru); + list_splice_tail(&entries, lru); +} + +void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk) +{ + unsigned i; + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_mem_type_manager *man; + + if (!bulk->tt[i].first) + continue; + + man = &bulk->tt[i].first->bdev->man[TTM_PL_TT]; + ttm_bo_bulk_move_helper(&bulk->tt[i], &man->lru[i], false); + } + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_mem_type_manager *man; + + if (!bulk->vram[i].first) + continue; + + man = &bulk->vram[i].first->bdev->man[TTM_PL_VRAM]; + ttm_bo_bulk_move_helper(&bulk->vram[i], &man->lru[i], false); + } + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_lru_bulk_move_pos *pos = &bulk->swap[i]; + struct list_head *lru; + + if (!pos->first) + continue; + + lru = &pos->first->bdev->glob->swap_lru[i]; + ttm_bo_bulk_move_helper(&bulk->swap[i], lru, true); + } +} +EXPORT_SYMBOL(ttm_bo_bulk_move_lru_tail); + static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo, struct ttm_mem_reg *mem, bool evict, struct ttm_operation_ctx *ctx) diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index 0d4eb81..8c19470 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -417,6 +417,16 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, struct ttm_lru_bulk_move *bulk); /** + * ttm_bo_bulk_move_lru_tail + * + * @bulk: bulk move structure + * + * Bulk move BOs to the LRU tail, only valid to use when driver makes sure that + * BO order never changes. Should be called with ttm_bo_global::lru_lock held. + */ +void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk); + +/** * ttm_bo_lock_delayed_workqueue * * Prevent the delayed workqueue from running.