From patchwork Wed Aug 22 07:52:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Huang Rui X-Patchwork-Id: 10572661 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 01836921 for ; Wed, 22 Aug 2018 07:54:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E7EED2AD5F for ; Wed, 22 Aug 2018 07:54:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DB8A12AE9B; Wed, 22 Aug 2018 07:54:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CEDB72AD5F for ; Wed, 22 Aug 2018 07:54:48 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C50A16E0F0; Wed, 22 Aug 2018 07:54:47 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0083.outbound.protection.outlook.com [104.47.33.83]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5471C6E0C9; Wed, 22 Aug 2018 07:54:46 +0000 (UTC) Received: from DM5PR12CA0002.namprd12.prod.outlook.com (2603:10b6:4:1::12) by BY2PR12MB0243.namprd12.prod.outlook.com (2a01:111:e400:585c::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1059.24; Wed, 22 Aug 2018 07:54:43 +0000 Received: from DM3NAM03FT020.eop-NAM03.prod.protection.outlook.com (2a01:111:f400:7e49::203) by DM5PR12CA0002.outlook.office365.com (2603:10b6:4:1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1080.13 via Frontend Transport; Wed, 22 Aug 2018 07:54:37 +0000 Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXCHOV01.amd.com (165.204.84.17) by DM3NAM03FT020.mail.protection.outlook.com (10.152.82.193) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.20.1080.9 via Frontend Transport; Wed, 22 Aug 2018 07:54:36 +0000 Received: from hr-intel.amd.com (10.34.1.3) by SATLEXCHOV01.amd.com (10.181.40.71) with Microsoft SMTP Server id 14.3.389.1; Wed, 22 Aug 2018 02:54:35 -0500 From: Huang Rui To: , Subject: [PATCH v5 3/5] drm/ttm: add bulk move function on LRU Date: Wed, 22 Aug 2018 15:52:53 +0800 Message-ID: <1534924375-5837-4-git-send-email-ray.huang@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1534924375-5837-1-git-send-email-ray.huang@amd.com> References: <1534924375-5837-1-git-send-email-ray.huang@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(39860400002)(396003)(346002)(136003)(376002)(2980300002)(428003)(189003)(199004)(81156014)(26005)(14444005)(2870700001)(97736004)(336012)(126002)(305945005)(105586002)(426003)(186003)(53416004)(110136005)(5660300001)(356003)(6666003)(2906002)(104016004)(53936002)(106466001)(81166006)(8676002)(486006)(77096007)(11346002)(2616005)(476003)(450100002)(446003)(4326008)(316002)(23676004)(7696005)(36756003)(76176011)(86362001)(72206003)(50466002)(50226002)(478600001)(47776003)(68736007)(8936002)(5820100001)(54906003); DIR:OUT; SFP:1101; SCL:1; SRVR:BY2PR12MB0243; H:SATLEXCHOV01.amd.com; FPR:; SPF:None; LANG:en; PTR:InfoDomainNonexistent; MX:1; A:1; X-Microsoft-Exchange-Diagnostics: 1; DM3NAM03FT020; 1:8LO29lVSY7sj3liUEQcpowotN6IiE8TvUzl3vOkU14VbvubiwW3oGLFFQmmi9Lc6ggxmKsmDaec5RBaLs1eqYRRjsi/6EPU2RvUjmqcPdsQkH4OK+aONUrmtcPBDI9LX X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7c235c04-d6a6-4c82-26a8-08d6080481ce X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060); SRVR:BY2PR12MB0243; X-Microsoft-Exchange-Diagnostics: 1; BY2PR12MB0243; 3:dNgM9D9sQ3rCBz5gIqrdWQr0DzngpvqVnA8vj4MlqZAuQO4nRWRN3bSmqaqgvEo5hnKS6pcUnoX7ZBrpALV/+l7kiQAmIxYyYGl6oIXcz440p9OL3T0h0mI5T2SWXsrl4yLGokUJCU+0Kl5fNX87ISIabB3QRLxSf+wAtaOsnIa6W44VmXf45UXrNVc95Frea6GmILjPgBe7EMhjhKQ1iNdhYDH+5P08Fh3Kyng9tdfOSK3xjShM03KTtBDTFXvQhkPFM3/JTDchFGCJxwHOkjvmqa8e8swD31eu4g7wjw4dC2T4NqYWI/TSFU04ad4dU8PgErAeRJAi/TnF8RuvILLnEBq873ApJCp/ePmy8Q0=; 25:DIcN1gd7qNtDney6UyV3ZQ6mRw91ZhrustasQKqA+3526fMHO2WkPYSUwTeEum6rE1wKw/ptKj/VwUcP4Fzg8orOZU9LuFAHYqDABY0+y5KrQl9HWNmHrLpYq0IlrOZLge5W5KRXby0N4zfTpxLBEubwrRUnOgpfe0+xVSG6XKi8FD+vyC07/eERWrCohkclM53xM/bfSDJ2ulAyH0OjIQNDQi32MVVFN7GEz631QFGGQAsR9ufWb2glKGT17ljQHMSP6ymQyssHvyImAihwWiMC4Qu+nIf9qVTjKSUIgFERiBYKia2T8jYm2Vjke7W/odXcChHVL1IoHkcbtikNMQ== X-MS-TrafficTypeDiagnostic: BY2PR12MB0243: X-Microsoft-Exchange-Diagnostics: 1; BY2PR12MB0243; 31:6djvJInHf+lFBOgZD4r0Gy0ZANDqifkN+vYnQwHMlY3K/MyDEpWzQ/RLg8kYgDoYJ6X1uvUtvfeJrpeI+H7mxinmVtSJP9br+aCyIYVxY8gOL4r3HSZD8KR95/m6QD71quhYZCmYiGO+rBZpjPunDTXUKtZR/By9AG1ul+yIxnauQSOOo99lN6oBe1PFNw5uhDtUvhiL4+Q9lVZwd7Mc2lKb700LuIG8fj5AP8GG3yo=; 20:LTqg9OxgbsUSSy2JVaN+TqKE6FMww42YtHrkfYaXeEsvlqls94voYek/Puwy1pk6yKN9kw6j0a8ihNAqsFVWTzB3muPZdLtj3nrH4xwAGf+BiBEpbMNoUYVmmj2kJSfOrC0qy3m/mqQlSa3nXLBhMxxcLxcu2RqtakJ9s7+bdfmHNtgNrfz7+nLXs2l7rzbqSG4d+HGupROA3HvMSx1/ZpithNUOtkslP9r9Nz49VpAfrMUKkGpd3FJ819Ffpj6TBZXtw15POlSlx003dFfcadtVr9+nIcueZ4Mp9IDIpUAgZFFJXQJ1Im0Vh8RyITB2I9f8MSFOSfQk+lmJg3C9i38ldTQ9Fr/rjTuiqliTEOB5+8l+E2sjSpJ+uulw4p9aaSBlz/VHHuyzcFMCj4K98AVNLQEhcg/3d/+DDTk6FPlVVmc5N+4ztFhxMn7ULxXrP99GTAD7LJCy/fpg1ZXsqjw5tz9qoi+0Gu5Ou/VRw2aqD9/08FZm2WaIXdoEzoeb X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(93006095)(93003095)(3231311)(944501410)(52105095)(3002001)(10201501046)(6055026)(149027)(150027)(6041310)(20161123560045)(20161123564045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(201708071742011)(7699016); SRVR:BY2PR12MB0243; BCL:0; PCL:0; RULEID:; SRVR:BY2PR12MB0243; X-Microsoft-Exchange-Diagnostics: 1; BY2PR12MB0243; 4:Bi7Sjwp3wB+4BJUsHlG5TvXIdsaPLEK+8phe/z++dInmvaJe/ipa5r8Tqi3eePeNYpumGC18JT9s964YkW0XqkE1cqN4WHkvPx1HeVfVmwFbMTn5GLk0HfPUFm3norljGqIfYpu+GEXZ7Xeu17d4N1rv/INlhEZ7sgFoklFTz7qtk4+4yAY24gAkCuWxJjmaMBKlbTJlfykIZTeTtbJnb/c1awamJEhuxsXXwMGXvibLxFPaP3QrIcwUPt20tUtlu6N4wLn/Zo0ecHhX1Pgq1uSg1HltUtHBEkofpxx0YVJoCfbtgWbkL7EF8ctNdKAy X-Forefront-PRVS: 0772E5DAD5 X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BBY2PR12MB0243=3B23=3ApF4MSS/?= =?utf-8?q?Ts7KjOeTfSsxElvlq6JPX3wuTRAKqmkFb8FVmBHtylfqiA9SlTR9PeNnYVzl79+jm?= =?utf-8?q?/knSA1FLmtlIk0YSjMuZoJhJJasWNmNuJuFQxwSRndNysfAiYzHKg0TltgBtNfbGD?= =?utf-8?q?o9NtmGNFQxTK6HHXDgg1ygx3HAgrgzkcx05SIU4MHRyC+3BY1lGK33Y7h9f0lzLVP?= =?utf-8?q?XVsmtyGRhtcACpBVIR9s2dU30qE4SlyHKHL/BO39+Rhzh6BHtwiqhfYveOM4o3BzX?= =?utf-8?q?OJLiMz9kblx8tCzNvhv835dOUOT7zEmyrgi5J8ol2zBY9BFjd1nybi2dTG/AYiwXU?= =?utf-8?q?4wrW0mnCW9tnDPy7+dtxCagmLlitIs+xZTIqFZKAeOlsmZ0fdhrwqI/z1uiaEd1GL?= =?utf-8?q?6yVclBT59AqCwy3IpjMhhhezmTwjdXzDhjw/Qv+ROZA4pvAf+0VxHIEVosTEyZr9o?= =?utf-8?q?6C2i2KQ9/vW/ga+EQQknu+vd4tAnfNgnQsRqJGZXQJj9GWdfywmKNJpkzGsRb8HX7?= =?utf-8?q?1sjoeOuTOgZ7YQxGvs+O26gzsPolbsCcMK5RKNTbaEg70/yQER7WZByy1JB2dW5yc?= =?utf-8?q?ogrvyO591a6VHqPZ7mZQrm60gPrNp9UmEAx0q920Nq7PldV1j6utRP2gkJ2I9KrC3?= =?utf-8?q?/0kXn/9jxCHQyy9b1EGbO0gPBpmajN5GtptWcaLa5RdacNGFltGHMDbDziIWYV1tP?= =?utf-8?q?ZzypxKQfSb1cBcrvEJd99jfttgoC7+1Fx5K2kj1OI8rgAl+ToxC8AkM2JKb7xUFgT?= =?utf-8?q?SvNH8Ywe6LxP8Ev77RyZHBhE5QieAadfs5YBc1oS4jmhuF5lK6TouTNylBL1W1Pwo?= =?utf-8?q?12itteAz846+F0KYl0/00FQntlnlgBSTgQq6kSBMW0n2NeNIYTGvfFA7OzQ/ZmZvk?= =?utf-8?q?A/lzqj4AgChzlo2vR5B5/UphR4ivgWsqD1MoWftl4CkDSmKBDDm12yKkaJxLDSn2g?= =?utf-8?q?N6/K9kQQJafc0omkkEyZVymtWkrXJxYuWMRyWo9Ebrl9e7ar8UfM3o50TZz0oSVs8?= =?utf-8?q?nPfCuzF08gISZO7lQP9UUZKg+g8UzOjQr4oynnzKuteO7ke4kRJz6YScKeZRqm5al?= =?utf-8?q?xFrDmYTmIUfuFIq2nn+dpiSd9hUE4QF7ihcfhXhH0H//NEf26Jyr1beQ/Yb47EM6/?= =?utf-8?q?M6KizSC06Vrz1orho79z4R4z4yrNuQUBIz2i8io?= X-Microsoft-Antispam-Message-Info: I/iciuAwDyY0puwTgvEYCuPT4vsXcYohBoOK1HoOb+N+0A6gpmssQBFJugf88S6KoDQoRAPhGRXqJQuSpKKhli1tL3Rdj3VRmatwo36zzWtxnV6sNpNMp3vgDwLgkGtkZcJWNYiK4SzPTP7deYLhdwl+EUr+4wJeIeYow5MKESFuxii4db6U/HhjtTMCOF5Go0AtPC2QSLVwzCF/A9eW5EndticPNZKsnlAaMBsVdYpY3Vyu+KAqHgD2UGX/ehDSfsyPm7m/S7RR5Vd5NEDeRyWqEBIqNhXIYxhgluouOrmGI5EvWcNSU8w+vINjIHom1mI/sVJtIWmrQR8UacYNEioB+HLy9ExzzygpMRa2gtg= X-Microsoft-Exchange-Diagnostics: 1; BY2PR12MB0243; 6:XGf4cjJ2hkbzcAALYaHG3awlYbh8Q66OVanfSAONkagk1JQ09Z5iJhENS0QudtFsnBuFot/3InIBbV/Tq2HGiS+58244FqLwCPQcseen9gcblX6XSkgl+sEmxpeXggW1wOSmjy9/H0BW6IYnbhahNA+E3eSdcvoREEK65HO0DfdvMAeVCk5gcnU5T77fEubl6AkFq+XuW7RZkGtkgDUJ7r4M9fOKkERPFMVcEhxNyJROXKKQwnYsO7oiIi4zaxy57Olz9PCutJ+O9Mmc6GE7Yne4qP+jS8y/rCrP0tsx0t994xCE1J9FHsUe8uOy4emCFdpl2pfh5bi23dxw/46s1liCTTW1XP/SbSx7xAs+QWGmZdBHYzavLbX1Zunn4K9RfkmcWkFKIiNfc/85OP1EW3x9s2gIqmmQBv/boZ8r5LEWy8Q2ujFIi1FM+/fMSPHR+iw6sd5xBriNjLcUsgfsUw==; 5:bbV63RpUJGsu8tWjh7mnvsjS3ZuXxNsFZnjBe5S+hmaeNvwwm4rYXiOGjR5TEgw9B0HD62RKiZLvzWk1yopKOnylgAW4chQZ3nITV8Jtm0dQZKfQcnmBDpBo5ESgThXLGBI11R+OdNitlFy5k3j5tuqyiTiLAVStGgcuYtnI/2Y=; 7:cfmVA82E+lrivm8kNI0FXCF51KoFwR2ISXTDYwVDiklf9aa1GbRquAqdl+vIwfKGzxOldtpU9PNkdIrKFFMVVF+Iv592jtUjJE8pKZWkhzvtAQFw5nJP68MfLl0235jg/QK+HgfpiyOG+nyWtQUqXuyEMiXVUKA4zISsyln+XS7b80m1J7RPCpOFpffHOvmFTb2CY1R0g0DjiA4V4lhrDf8Bu5tY0WSlHreiSWas1L8m3IhcF9or6r+VnA79NroQ SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BY2PR12MB0243; 20:+vI1ptmlhIEwcVLhqS/dYQOBQW7vF3JUzc+PhoRILfcvX41b/9w+Kd1IbBTMQzw95u2AXxbdd4sl8UyiwbYsIRav0HVhxF+3R7iacnrY2R0d9w4vi+qYzrzZkuSphpKX1zbCAyKTh126sKqNzL2kVtC5VZaiDuOfbZzlalkHaDPSg/LUIr7mtrPuyJGrbACaMiGzPGyEqbJyjYlE1Y2cMPL7JamvzrKconYL1rMZpJQOUNNPrwYXnk1uJlYT8+sP X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Aug 2018 07:54:36.8447 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7c235c04-d6a6-4c82-26a8-08d6080481ce X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXCHOV01.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR12MB0243 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Huang Rui , =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP This function allow us to bulk move a group of BOs to the tail of their LRU. The positions of group of BOs are stored on the (first, last) bulk_move_pos structure. Signed-off-by: Christian König Signed-off-by: Huang Rui Tested-by: Mike Lothian Tested-by: Dieter Nützel Acked-by: Chunming Zhou Reviewed-by: Junwei Zhang --- drivers/gpu/drm/ttm/ttm_bo.c | 52 ++++++++++++++++++++++++++++++++++++++++++++ include/drm/ttm/ttm_bo_api.h | 10 +++++++++ 2 files changed, 62 insertions(+) diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index 7117b6b..39d9d55 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -247,6 +247,58 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, } EXPORT_SYMBOL(ttm_bo_move_to_lru_tail); +static void ttm_bo_bulk_move_helper(struct ttm_lru_bulk_move_pos *pos, + struct list_head *lru, bool is_swap) +{ + struct list_head entries, before; + struct list_head *list1, *list2; + + list1 = is_swap ? &pos->last->swap : &pos->last->lru; + list2 = is_swap ? pos->first->swap.prev : pos->first->lru.prev; + + list_cut_position(&entries, lru, list1); + list_cut_position(&before, &entries, list2); + list_splice(&before, lru); + list_splice_tail(&entries, lru); +} + +void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk) +{ + unsigned i; + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_mem_type_manager *man; + + if (!bulk->tt[i].first) + continue; + + man = &bulk->tt[i].first->bdev->man[TTM_PL_TT]; + ttm_bo_bulk_move_helper(&bulk->tt[i], &man->lru[i], false); + } + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_mem_type_manager *man; + + if (!bulk->vram[i].first) + continue; + + man = &bulk->vram[i].first->bdev->man[TTM_PL_VRAM]; + ttm_bo_bulk_move_helper(&bulk->vram[i], &man->lru[i], false); + } + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_lru_bulk_move_pos *pos = &bulk->swap[i]; + struct list_head *lru; + + if (!pos->first) + continue; + + lru = &pos->first->bdev->glob->swap_lru[i]; + ttm_bo_bulk_move_helper(&bulk->swap[i], lru, true); + } +} +EXPORT_SYMBOL(ttm_bo_bulk_move_lru_tail); + static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo, struct ttm_mem_reg *mem, bool evict, struct ttm_operation_ctx *ctx) diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index 0d4eb81..8c19470 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -417,6 +417,16 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, struct ttm_lru_bulk_move *bulk); /** + * ttm_bo_bulk_move_lru_tail + * + * @bulk: bulk move structure + * + * Bulk move BOs to the LRU tail, only valid to use when driver makes sure that + * BO order never changes. Should be called with ttm_bo_global::lru_lock held. + */ +void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk); + +/** * ttm_bo_lock_delayed_workqueue * * Prevent the delayed workqueue from running.