From patchwork Wed Aug 8 09:59:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Huang Rui X-Patchwork-Id: 10559715 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7D14B13AC for ; Wed, 8 Aug 2018 10:00:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6B7432A95C for ; Wed, 8 Aug 2018 10:00:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5FDC32A9B7; Wed, 8 Aug 2018 10:00:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id DA8602AA1D for ; Wed, 8 Aug 2018 10:00:32 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BAEFE6E517; Wed, 8 Aug 2018 10:00:31 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0076.outbound.protection.outlook.com [104.47.42.76]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9EE7F6E517; Wed, 8 Aug 2018 10:00:30 +0000 (UTC) Received: from CY4PR1201CA0009.namprd12.prod.outlook.com (2603:10b6:910:16::19) by SN1PR12MB0255.namprd12.prod.outlook.com (2a01:111:e400:5143::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1017.15; Wed, 8 Aug 2018 10:00:28 +0000 Received: from CO1NAM03FT059.eop-NAM03.prod.protection.outlook.com (2a01:111:f400:7e48::206) by CY4PR1201CA0009.outlook.office365.com (2603:10b6:910:16::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1017.15 via Frontend Transport; Wed, 8 Aug 2018 10:00:28 +0000 Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXCHOV01.amd.com (165.204.84.17) by CO1NAM03FT059.mail.protection.outlook.com (10.152.81.192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.20.1038.3 via Frontend Transport; Wed, 8 Aug 2018 10:00:27 +0000 Received: from hr-intel.amd.com (10.34.1.3) by SATLEXCHOV01.amd.com (10.181.40.71) with Microsoft SMTP Server id 14.3.389.1; Wed, 8 Aug 2018 05:00:26 -0500 From: Huang Rui To: , Subject: [PATCH 3/5] drm/ttm: add bulk move function on LRU Date: Wed, 8 Aug 2018 17:59:35 +0800 Message-ID: <1533722377-27020-4-git-send-email-ray.huang@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1533722377-27020-1-git-send-email-ray.huang@amd.com> References: <1533722377-27020-1-git-send-email-ray.huang@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(396003)(346002)(376002)(39860400002)(136003)(2980300002)(428003)(189003)(199004)(476003)(97736004)(450100002)(126002)(2870700001)(6666003)(4326008)(336012)(47776003)(86362001)(53936002)(106466001)(486006)(5660300001)(53416004)(76176011)(77096007)(36756003)(54906003)(2616005)(23676004)(26005)(7696005)(110136005)(186003)(105586002)(50226002)(446003)(14444005)(8936002)(426003)(5820100001)(68736007)(72206003)(8676002)(81156014)(81166006)(305945005)(356003)(478600001)(2906002)(104016004)(316002)(11346002)(50466002); DIR:OUT; SFP:1101; SCL:1; SRVR:SN1PR12MB0255; H:SATLEXCHOV01.amd.com; FPR:; SPF:None; LANG:en; PTR:InfoDomainNonexistent; MX:1; A:1; X-Microsoft-Exchange-Diagnostics: 1; CO1NAM03FT059; 1:NMFBbAwUvzFcMc2dxM/aCwTsjYWI88MkaF2q/f3kEJMWTsigv4+ykuz3uIW56Y4vA6rKgkeaxPcivP3k7Qegb595n1zF7KXXErOGGDpvkbFvvPRht7IYE/4sQkFwUbaK X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 57840b57-2650-4f44-e06a-08d5fd15c4d0 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989117)(5600074)(711020)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(2017052603328)(7153060); SRVR:SN1PR12MB0255; X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0255; 3:fDYrOFFmr9z2yDmWmMuSkfYHCDr+Hx1nyHgB7iZA0hZZALKms51g3Dn3DlCUNu+L0BXOc4rwGFL0M8VstS0d4uuryz6D8KtG4kmc60+OMSE00HdmhF7uNPoW84ey0IDwi1O9USXKtjqCj+M3j9SM4i4Qm9ltUoftq9I9c5JfHf16siwx/sMHXdyVY9Jff7MgAYDmMgVdi/wbI3u4Ov5gsuoqq2x7I74/EkssmFYl58Gm/6GWbplN1k4vJM7aQRpXuVym92H/iLSPHvX87Hz9kmrsioVpFAiGyU8oWzWhqmePkbfb3Kzhbs/mvd/pZ0J2zEpaX1KCCi4y+5Q8qmvpVW87/1jqVf6K1+kv41dCe6E=; 25:ZTxuVog96VvanblE7xeJqpMd+jzzUyyPUUTtGQQSBCpLsdTTYUugIYR6vMTcsIxvQFKaOWgfSZAQ1Z8FcT7ediI2U2UFye6SgbN5xT1c4E9rilLgZ8lk6Cv7++HeDmSvCDSnaj5yZZge4PPm4FD8VAWpCK+weqGwMeBm32G9SYJpDTmGJm6q2HQvmqycb3RgCRWj8p8vNLEP6gKNGAHydXX/gVmLL4tLsPIggdmCjtn6BaVb3pYwBVFQLZoEQEOdBwMzZvl3BuAYohtpBGz9FRNCh0asDJkFmQmecz0NyVvaFokq1ZIg8ykNwJwD5VSIRNEk/8ZpaLSfdy3MtU5YnQ== X-MS-TrafficTypeDiagnostic: SN1PR12MB0255: X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0255; 31:G+v4/KpxOAf7Gm2Wyh8HM7GFWQDZ+3y18kNZAzqJLpmaUehc3WQH07sxxFjBo3Fa9tKIHn0DVEZj9C8FtbzLhac7JO9atCdzv/hnjCdh++uYSzK+DcF4mqA+NEngh3MW/CDdCZ99IjXu3icbpjCcFXGNpgUIUBEY0T6sKT7wZVghckDiZz6A0A0ut7ocqYZ2lAn/SSfWaRI+Rc21d+x1qyoNCcKcLrqasXL2pxncMdI=; 20:A1AbT2bB0WVO/cGIRV2fzSrLyOHVP2CpdTzLh17DtyvN3XdDnMz6tq393B0ZgwaDoVLLcZX8Shslyok/bbrnTh8Z/3nec0HyWY6T21BCZk6xdvoLd/hpuN7WRT8bROqcRvGBiJgeOXGCsvfK/BoONO2uaY6Ue7KpV+AMsDHRdnNrBMXFdMn0Thoapftk6kHyfSJSV17g85KYxzbmfSmz1jPouOrg4Kb1Ie62lLhQLXxVr70WMQcP6fZ9UySn7aEpk6aTobz17oqlVVoD3M7Kh00Z0lOZLQAISZ4GpUNfnAtPuf5IHhPu3QwA1gK7vtVl/wOoTyJbt0HQwaxK5ZBnQj0eZnIawTWV3neUcARbJYRqmwcW6GDjOC9aQqcEhb/ymHj6apY37ZlfaQfKGUgfKLyrhWbuEIeGOBsT8B3vcG8n9xY4M+e4Z5zYMuwU3LmYXPJEj225HX2aO/o3BlFweMaAHU4Jy81meCOgOE2q9dh7XqOAzmrYQL8xsbYpi2Be X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(3002001)(3231311)(944501410)(52105095)(93006095)(93003095)(6055026)(149027)(150027)(6041310)(20161123562045)(20161123564045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(6072148)(201708071742011)(7699016); SRVR:SN1PR12MB0255; BCL:0; PCL:0; RULEID:; SRVR:SN1PR12MB0255; X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0255; 4:pfJAFtx2ybCMNekJxPQXGgiFiUc7XiUCCDiaeuxkX4c3Ks9dpDBYMJmtaEAUTiE3Tuv9A8aT7k+feU0z6l33zvr2RpVh2X7YBn9f0cUDSOCyXmTMlMU8SrPh7CBgKWXkT9yb3y8P8C3npe1nEJH1H2rRW1eWBCDwU4MewSZQsW5LFZ8bPXZkqID3eKYxlk/cZHuWEe2Pa5beCdNQfmDD/ecoOG2NZPkqIL459JYJKn18/EbiBxrmvMe86Bfx692URPgxU/S3OgAwb/SdAkUCDhYqhS8Yo6THpLMX8zUJuwupP+eXFRzgREC3lJVcAIHp X-Forefront-PRVS: 07584EDBCD X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BSN1PR12MB0255=3B23=3AYD6Fqrk?= =?utf-8?q?BO4AAtM0T8nH1p2iNI3ldnalNXVNfmQq37ebcgRWKzlOt9ab3mKh0LMRl+ORyHjkv?= =?utf-8?q?ztrPGJdmmcN6NKpFijlffNJIq1cDZg6Z8P5rFergQY2MGJITL2xlN6po7mayJD1ka?= =?utf-8?q?zHGnjKBiuQU7sqVutX2tBnk0nDLpmIhewoh+krgOuzqnkEL7Jro9u3FttsGaVCvGP?= =?utf-8?q?JB3tY/6rhMJPBryGlpFpsnoooxoit2r4hZM9DHf+dzpyzDj76mlHeRsV7mir2KukB?= =?utf-8?q?1K/lXiz36HxcjdTTqO4ekuP8cw6lqlgB2ldb4BMrDHfzPHNW/BDBps6MxlAFu9Heg?= =?utf-8?q?x2nCInxmFAmyL1G6BEIOvsQO2KM8Mx8WJpgqDwQzpGIAmgg4K1a4QBcAgUqrsdBJg?= =?utf-8?q?dis3z2UOgFvlzVIbKVVVa5O2bmXWG2vI5+YByReGz4iPLTQw5wAG9wG3WYsI7+cNG?= =?utf-8?q?5yF1T/sdvL7gND0x9cL3CjIeKmVtVXPL4Sp3tN0K4IiPrFDSnnS7+tA224XKWjThe?= =?utf-8?q?EsSIgZJZFOUe+f7+KqJb/H5L85yegE1bqxyCEk7DMZHM2F47UVAtKoyXaNDqXvQtr?= =?utf-8?q?yZ+JcSFLjMa78XwCB/03Li3eUiHYckwuBGKK59UQd9OJFXf6VW9xH4krW9+/3Ju/q?= =?utf-8?q?NG3sUbiguEXfWcQZmGONlCvMSXGV7RSN22pzJmVoVzcfiNAkbZB/BGSdTqv3Z+8qC?= =?utf-8?q?Hmqw65E5vj+FgxHQAdQGN/0IAQzTpcTrx6NGQfnzgEPr5tRp9nilyYsuasbyrLJD4?= =?utf-8?q?185smbN3SZlv1O8w5FFPFBTxi8PzcKg3tq+Km4PoV2fFyCEsRcw60YXxWM6vD2QG7?= =?utf-8?q?Rt4L25+QmuZFfvPyncOKHwc6rq1PVH4W8peEymTZS9Grbo/uLROEC8L7FTrxpxwoP?= =?utf-8?q?epAUxzVDuAVgCLgVX03Wu2mLQZIbjExsjpWTZGV9/B9O0K6WeB6cLg0GQu+91L9Od?= =?utf-8?q?tiLVYE2cPETNnw/MCpflYIIv2mLjbBCnbi7CxPS8XP/dGq2C0l45CU4RXoltAx9mB?= =?utf-8?q?J6yL9JzWMzEd/7FYsMnikCdqsLS9C9vXrWDl0erPAqLQZq4fgC9oxEfEQz38Qs07J?= =?utf-8?q?J+E9fuThP0YsW1zfKko3mBU5eNY7/adTPxhSxYfPpkwB/YwXX0UkXEkUC270CD/09?= =?utf-8?q?v45v6YtOxLCbq/89VShrxnGvLGUO0vLLTXjuHB0?= X-Microsoft-Antispam-Message-Info: I9NecSsTDiQNTmBwhvxh9sEZkfKGEGO/0UCaWJkYwAmw2jcc3O7IzCUFj5W4N1itiyQskOdbthzrNgFNutcz7/JqpJOawNce9vhdirP5zM/sqvpZ3h/9SQAY0bo5n0tTTgbOEklxFh1/c7FcVNS0pTG90QFelGompgfK17U+zPBIR46D7/6O1QlEeif44BGfZDV2Iqr72T2E1Rjze+o90b0HBcXmKLOXCYifDoP+GU4foCEFBESAzRVn0Oy5TIqCCRO6fUi4GmzFwZV8qNsWaE17x5YSDfRmgmbelJpPr4i6MHufqTZwpFe5gtd6UqURfStZZrSu8Eqh5MgPtxgXHxnOx4uz/MyM65m55FvpdWU= X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0255; 6:f6x6cWO0vvTHzkDUsUNMwE8fc4xTD5444yJCkdKbbYA4QsxpgSWKpruqRuZDVTXdYplA/lj9Pk0Womgl5qnO/rBQgluO+jVofJu6nLfL6EyjOpY1QC1oMfpVoZ1rvrxhyHXN1cmS7uQcZ6qqqaVlbGKdFzM7uvURoshg0R6kwagkdaPm8ZQ2Kunx1YYAy8xWfYelQl0ydwGBHKzCZFG7I4LBrdqSfd59AfnSihcbmFo2uEPab+nK63mWjwYurYVRWShp8F66breKJGd+3leB1kNQ9qtqbSgZTi0yPr1qep8vb0VMyY1+ZkuVBZrngn3EtrOz6826btFRBpYeGWbiI+yUmEMMhCRo2zm9cjPsTu3jj7aP59/gcJ4XEv3Zo6BJKHrfLezi8qelabZWQaqWKotroId29m188FVfD/p1cPTKEHlS1eD0xLRXWSxrTQruVSTudT1DOgzCgwmdvN4eaA==; 5:okxeUr8YPESIFa9j92TsPHbkJB2zcnZm5a8QXKWjKeNHfUViHenJLnWAJXuFpHBYZw+/dxJqOj1gq/NsIoawiMdaU/upQrkTlR03i4NqUeWmjyA4lirqAX8TmyKzVdCSt5z0bWQeKzjFKUK6ZfYa9B3+RqG9GlE4+rG4kxcOB2E=; 7:xBl8KHJ272tZh7F8JvBv7F45AWh0DqNB9PEaA0ueFLc2TdtT1YUcA1fwkmadGS15mIJoUIzIeppAEwhMwwWmfzpcYKhktMSGR9IQNlm8ZYhD/PKptBWD3/M9ImqPQc6UPvVU7fBSFlWkYHJt4hVjmCL8mYuU6mQ8kPJxprNr0Bo+toHFiECXTsPyFk2qivszGFrNmk+na1EwYz3V7GFwWaosjcOB5eDNztE1D96Rs9XHuYbOX3t1aGqdoJ5q3usn SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; SN1PR12MB0255; 20:mVxlRd3YXGC7JzkAVwVXmT5y8OXbPL7y7QqLlhZTNH/wBYuKhL4ICuDR0U0vnm2QcKRoQzH6UE1ggIOREArrvNnR96Lb+NRDnyrP2uBazQCVGgYGoa+SsS3grfLpVHoT9vBQ+F6iO5xAyn8MS/Mlwq9m/lMQ/+f29wX/J/gG8g/U+9JGjX8YejZTjGWDEZ8/vyKXw4HvgeOichAjjnUVatRnRAndyd46QJwmolyhLHAl4a0/lpIhXvXQTe+wTdzi X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Aug 2018 10:00:27.8925 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 57840b57-2650-4f44-e06a-08d5fd15c4d0 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXCHOV01.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB0255 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Huang Rui , =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP This function allow us to bulk move a group of BOs to the tail of their LRU. The positions of group of BOs are stored on the (first, last) bulk_move_pos structure. Signed-off-by: Christian König Signed-off-by: Huang Rui --- drivers/gpu/drm/ttm/ttm_bo.c | 52 ++++++++++++++++++++++++++++++++++++++++++++ include/drm/ttm/ttm_bo_api.h | 10 +++++++++ 2 files changed, 62 insertions(+) diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index 7117b6b..39d9d55 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -247,6 +247,58 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, } EXPORT_SYMBOL(ttm_bo_move_to_lru_tail); +static void ttm_bo_bulk_move_helper(struct ttm_lru_bulk_move_pos *pos, + struct list_head *lru, bool is_swap) +{ + struct list_head entries, before; + struct list_head *list1, *list2; + + list1 = is_swap ? &pos->last->swap : &pos->last->lru; + list2 = is_swap ? pos->first->swap.prev : pos->first->lru.prev; + + list_cut_position(&entries, lru, list1); + list_cut_position(&before, &entries, list2); + list_splice(&before, lru); + list_splice_tail(&entries, lru); +} + +void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk) +{ + unsigned i; + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_mem_type_manager *man; + + if (!bulk->tt[i].first) + continue; + + man = &bulk->tt[i].first->bdev->man[TTM_PL_TT]; + ttm_bo_bulk_move_helper(&bulk->tt[i], &man->lru[i], false); + } + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_mem_type_manager *man; + + if (!bulk->vram[i].first) + continue; + + man = &bulk->vram[i].first->bdev->man[TTM_PL_VRAM]; + ttm_bo_bulk_move_helper(&bulk->vram[i], &man->lru[i], false); + } + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_lru_bulk_move_pos *pos = &bulk->swap[i]; + struct list_head *lru; + + if (!pos->first) + continue; + + lru = &pos->first->bdev->glob->swap_lru[i]; + ttm_bo_bulk_move_helper(&bulk->swap[i], lru, true); + } +} +EXPORT_SYMBOL(ttm_bo_bulk_move_lru_tail); + static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo, struct ttm_mem_reg *mem, bool evict, struct ttm_operation_ctx *ctx) diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index 0d4eb81..8c19470 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -417,6 +417,16 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, struct ttm_lru_bulk_move *bulk); /** + * ttm_bo_bulk_move_lru_tail + * + * @bulk: bulk move structure + * + * Bulk move BOs to the LRU tail, only valid to use when driver makes sure that + * BO order never changes. Should be called with ttm_bo_global::lru_lock held. + */ +void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk); + +/** * ttm_bo_lock_delayed_workqueue * * Prevent the delayed workqueue from running.