From patchwork Mon Aug 13 09:58:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Huang Rui X-Patchwork-Id: 10563969 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EF113139A for ; Mon, 13 Aug 2018 10:00:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DD6A228FD2 for ; Mon, 13 Aug 2018 10:00:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D091428EA0; Mon, 13 Aug 2018 10:00:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 6ACA428EA0 for ; Mon, 13 Aug 2018 10:00:00 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 775A66E09C; Mon, 13 Aug 2018 09:59:59 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM04-BN3-obe.outbound.protection.outlook.com (mail-eopbgr680062.outbound.protection.outlook.com [40.107.68.62]) by gabe.freedesktop.org (Postfix) with ESMTPS id 06E086E09C; Mon, 13 Aug 2018 09:59:58 +0000 (UTC) Received: from BN4PR12CA0008.namprd12.prod.outlook.com (2603:10b6:403:2::18) by BN6PR1201MB2497.namprd12.prod.outlook.com (2603:10b6:404:a7::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1038.23; Mon, 13 Aug 2018 09:59:56 +0000 Received: from CO1NAM03FT006.eop-NAM03.prod.protection.outlook.com (2a01:111:f400:7e48::208) by BN4PR12CA0008.outlook.office365.com (2603:10b6:403:2::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1038.19 via Frontend Transport; Mon, 13 Aug 2018 09:59:55 +0000 Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXCHOV02.amd.com (165.204.84.17) by CO1NAM03FT006.mail.protection.outlook.com (10.152.80.75) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.1059.14 via Frontend Transport; Mon, 13 Aug 2018 09:59:55 +0000 Received: from hr-intel.amd.com (10.34.1.3) by SATLEXCHOV02.amd.com (10.181.40.72) with Microsoft SMTP Server id 14.3.389.1; Mon, 13 Aug 2018 04:59:53 -0500 From: Huang Rui To: , Subject: [PATCH v3 3/5] drm/ttm: add bulk move function on LRU Date: Mon, 13 Aug 2018 17:58:49 +0800 Message-ID: <1534154331-11810-4-git-send-email-ray.huang@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1534154331-11810-1-git-send-email-ray.huang@amd.com> References: <1534154331-11810-1-git-send-email-ray.huang@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(346002)(136003)(396003)(376002)(39860400002)(2980300002)(428003)(199004)(189003)(478600001)(81166006)(8676002)(336012)(81156014)(7696005)(476003)(2616005)(5660300001)(23676004)(36756003)(126002)(105586002)(72206003)(11346002)(14444005)(446003)(86362001)(426003)(8936002)(47776003)(5820100001)(356003)(53416004)(450100002)(53936002)(316002)(54906003)(110136005)(104016004)(68736007)(6666003)(50466002)(26005)(77096007)(305945005)(486006)(76176011)(106466001)(97736004)(4326008)(2870700001)(50226002)(186003)(2906002); DIR:OUT; SFP:1101; SCL:1; SRVR:BN6PR1201MB2497; H:SATLEXCHOV02.amd.com; FPR:; SPF:None; LANG:en; PTR:InfoDomainNonexistent; A:1; MX:1; X-Microsoft-Exchange-Diagnostics: 1; CO1NAM03FT006; 1:Fld291riVt9bArd9M9ozHcNFK+VCnfev1yZU9LsH5u+Fj5xltU0uLGOiqxvfH2z9ssDjMdTqtKxgAGe6PxsMHq/i3qqg2x6OXZMWvxmsY33MqgViLCBKPIcY+8pY0/bh X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 34af3209-2d85-40f4-7f03-08d601038593 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989117)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060); SRVR:BN6PR1201MB2497; X-Microsoft-Exchange-Diagnostics: 1; BN6PR1201MB2497; 3:XQwms55p4kNdyKMtQ/mfGEX62CugXUjZe3lTkCJ1BflnAM4w03bGYsJu84ZpuXdB9uBobJsah5FfdCzMd7AEXqXUacNUjsDP1fLweKeOIsFIvoi2bMPCIR/DOVIdGzUjXcp/rqtxYCu/SeoKNOLzJWaCANaxVCcDdRliaB3S5tSsXSf2YFP6gqZjd+8rKEj9jMMP0oWJ+MbXQvc/3h3BUe1dNmW5TqKe/DANWjBXDCtAd8AJKs8Tf2YhnsbdrDPeCfv4WN3+gadOeKpYzKzLCzvIeVVXqYIH9vHC0L/VOtg7V0YmGMmhuqXsMv4OuPFt92WKoupaWIV+RF7k9JNLIXxRS+rO8LRfYS0wdkLNBMs=; 25:JfVU1gFyRvilJjmP/weaP1J9KFyfC6fKVg6Uv8Ux1q9VP+JdXLmidPN5mU4OYo7q3N/82KhB5NS1my4ESCbH3licGc9Ttf1vEgU7Y4uu1C40+5HaX52JbZ/ivaqhyq0Jzp81mEkEoB8yWZDZlhKUwHEJyLBZu/nvJi6k3WyBrjAm/OkScG3gewaX4s9jLCV4PQ7HK6U8G3UjfufFmGysCq4prKzyZ4zgrf/T9VvxhwJGcfMPYj7pLUXKf0Q+0OAx3LHKSOrSQQHk+HtKe/VL8VRGJWA6b4cJflVHU1zHiI591pysjfzLYBvWAzherzlraViUE/6R3W6nfDdpafoeCw== X-MS-TrafficTypeDiagnostic: BN6PR1201MB2497: X-Microsoft-Exchange-Diagnostics: 1; BN6PR1201MB2497; 31:rMUCubMOlWUYecU4HKzYL31aiD4oBHWUGw6GPYqK1gaz3oWLgzpFk1P4JIeQRVsu3FWUPKyOrib35Gn0Aulg7/3uzbXKUD2x8hvlj5HmoXPQl84QTyO8YB6rla3pJa3mrMa/TbPft3+n1sbSdkeDnl00WFwv2hmNIcFqSzXTXk3YTRM/Gy2/QhLU++WTnhR++gYqo73OzBFIFpq8xC9rLhs7PYCc2uyNge7+JQa6aIQ=; 20:LSbnAlobaQ4CrZmPtfsgOSZ1Kgd98Eugrz54dGp1nO7WgyLA8H++mgKRaP5//bXk9864Ot3bDip+j4S6gOBeA4gQG21mewdXjp87cy0rTvcnpIfssVZnz8OtRE+uZFkpp0unKna/2W+pwoAOC9lzbNX8hs2T3BMRlTULAuchqNZo0xnVJuQ2i49BOe/g8WQ7dzHCXcYPmWricY0zvkROG9VVa5lAQgSS85lYasBPlFsGCKN1UGYBVFuCz7vHe6cVP8tu0oB/EfhxKaGPa6iqvuN37cqOJ6ipMwPxNTOWonTtc2nAQdGAbm4C/052rfTi1CQco8rKaLu+Y142qaiVdwN8a5Y4whhdKHx7e8W5Al8rGBBe/YzlakD3mqGaIdxTtBm5aenzV/mWhnwwASTVo6I/peGhuO4U+mpnA10+F8inEF7Elb8FwCvvkitxwdaNx9Hyu1fMIpSmiGNF1D45cT4WffPtdKSBy7l9GY+UM+MWwqGdpPh1vVIwQm6WDxay X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(767451399110); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(93003095)(3231311)(944501410)(52105095)(10201501046)(3002001)(6055026)(149027)(150027)(6041310)(20161123560045)(20161123564045)(20161123562045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011)(7699016); SRVR:BN6PR1201MB2497; BCL:0; PCL:0; RULEID:; SRVR:BN6PR1201MB2497; X-Microsoft-Exchange-Diagnostics: 1; BN6PR1201MB2497; 4:JGGSo0GEPQU/9Slv1T1y3gN/cbRh8MeOzmz4wwWHIpJsj+FqxfGOc3sMeqlotzzTkT1omHp+q1Ti5I1M6SC4Wf43A7q38q3SzLvwTIiSbEoA6vsGe8Mefkv8NYVYMSPoPj25XCFiwMNgV1JdJMKr+H3u4YGQY/AMZDvW+0TEP21pdgpOYsVqO4cvMxa0T/Jll06RlVX1AVVgL/gTiBOUcs0/Ihk8zRXJLiFJmqEz9gSbZzeWFhIUPdSfuGgxDZX+O0bEZQ2H69BxzlQ3Mfh58H1Qg5Tfx8atopub3C4WHmb9XXa7jklkoqfMe6S3aSeI X-Forefront-PRVS: 07630F72AD X-Microsoft-Exchange-Diagnostics: =?utf-8?q?1=3BBN6PR1201MB2497=3B23=3AgrTv9?= =?utf-8?q?9QeObWtFy3O/VlA2OIOBMN839Mqtaso4zwv36yNod4JbLm5+vhQeUkxTXVGFmZZ+5?= =?utf-8?q?Ax+eu9jItbhTFw0yPqvvBVn54Qq091JJFZ76LBb0vlTiEPfRDTyna0WWv2gRHKRIw?= =?utf-8?q?/y04rccPVyc/5sbO1WwfCbpc8TCysTfluySlGd7NFDF/NWf/nqbMmHhqddmiexEyg?= =?utf-8?q?zqmkrFo+eIsFZAHWY9UbfR9tuFJivA8sxCNMfGRFWGxAi38xgNt0UbSlcALdD2gkL?= =?utf-8?q?QM7ze7O8wywlvzLzLaWJBhoV+mkZL9cYZX3I9lyOSSJJFCAHsN+RqDp3Ks371DjIm?= =?utf-8?q?BeIuNEnkHvwjjtWkSG/X8JMx/kYO5YFt34ekDXCBTuNQXcuoepMCMZpX0Y+zq47DF?= =?utf-8?q?qoV0Sy8LS+64sc6VioKinqxm4Fyhy4FqFM/yFiZB9xIK11BoZkQ3pObKWLy/YKU7f?= =?utf-8?q?gnUG+z7xSdjr5UbQjN5afsZT/ujoAdLJ1pJgGcHj4AakdaLbBL5m9qIay/qV+Ay4n?= =?utf-8?q?R8iRhq5446B3xIOD0geo2seMKTBl9oO3kAVCLAl1du6vzlx56X37LDB3CfFaC3vD0?= =?utf-8?q?2bUZII3aOABz430Tt3HYbOxSS7lCOOQmA5v8J2GVAoEBnNDSURPJw9gUV42JnSkk2?= =?utf-8?q?C7XAI6TrNbmBJH5Vih/OcgJnbnjnAAAg5QI7XM0uIr93CZsj3CEL81N9nHS4iZse4?= =?utf-8?q?Oz8I46cVFDLx8qIvuhvPhJPYXDZpwG2fOILEX1SgEd2dcqZUpQNtowJC2Nv0kKV9B?= =?utf-8?q?pRjEa07X8D80OCMrcgec72iG52Algnqrgn/10rZBedbzVrQvfu3v21foxV8jw0r59?= =?utf-8?q?Hgqo415gK89f3hpP1c0mxVRFRFBxBCV25OoaYpsX+t8+z5vCNrsOk1X43he8xB45O?= =?utf-8?q?UwQyRrrj8ZHO5Po1QzEQ/CN52Zohw0Q5pwBr5Fhqy3YzULOAXooeejLRTw4stJ3j3?= =?utf-8?q?07Nw3yJHqmhe5k3KMOOIvjzxyM8OW5fqFUw36FeITV+EjnoxtuCCC7dv8NcjZEzuQ?= =?utf-8?q?bZqLDOv3s+C4immtkwXhO+t4UESriIllYnu5DAF1tXtv3QSqDhpIGl9Y3egYFjhkn?= =?utf-8?q?jL3Fnd1bS2jq9iWDvsGXOiB5ELGeKT1fTDV8cHus+cjhNC1Qot42OnTuv5/k5GVWS?= =?utf-8?q?nCxk9rBFyfyaVDa6MjwhekiIy7u0Z5Aj9qbP+phRo?= X-Microsoft-Antispam-Message-Info: V3QX/nOWq7qZJs5pxA0xbz7e9yO1pxMSFRABtU2Lhur5EOyYkCU+BqkJuEUzivL/TRbI4DOQSNC1GNOOJ6XlXBhV11Yv2FLiD3tSnXJDFsF8UrxZdWt9lWAvOyZP9D6/Bdl4wnf/rSJMJcfjmZWykZgAJEcryoC/YMWRD8nGXViDFbALCElC8GyRujHb+Da0sTklCflXMo6yMok75XLNRqIRFJ1P8FO71MHvs6Vh2vt6xY8uFUqa6FnNgIO+fON91QGHQT1Z7oTPwkjT4mKNRFFWcGNsyo+4p0lZiw0kpKL2c9F1T9yCP+MxhvfGkyaYb+IlagjtgOagwu5oD8jl0StCNU+Hd2rJ/Zu/6HdaDQk= X-Microsoft-Exchange-Diagnostics: 1; BN6PR1201MB2497; 6:9R3Feiv1/ROdrKSYs/w1Gg4ta+W+3p9e8ZVM8B+DAd17eNOx6oQhI90Xvg7SPfLxm8N41zdoy+4jRQH9lp2GAknbbCgJRdzkLIodYzd8WcmkJkaVAyiEY0ey75hHlURR9Yutd1brBvg8FX+U2nb/uYxyAIfx9/GgxdpZgr33D2uDVEGmFHi+fEwwOJsTydf5J8fA5aOuS5luPmfLJ/tOwIQ8Q9YkL4sZDb2j0MWQ9tVRsJnvyjrSJK9TiWH6NWkxJ99uFNKmqoV07GCVmRQlUEb0AJfXa7RdmTPxkior1aH3TTrda94UuE9hjukTA2ThgmvqBlCZzBsBvQDg4Cod4rn95kOYxsKaK0gt2NY2HDcUl8PguyEYh6Q+eiyIihqdfjFJ1TQQgpwxJvRWfjoYcaE6ZVrCV+rgRrId4r7ritt8k6dFsoJfl6BgOqpWv6WY6nrfaAzy0lVDGeg18Uz4vg==; 5:sKn0rMMyRwlLpT2B540oIKgi2sIN31eywHuxu0ZmHKKojNiMJ7f3gU8ShZMKmAysvpheD9m4fqma83GIDNQu6J65a5TqVBn1PnVr4zXS+rRhYlIk3TXXOeMb9GTytUmd/1eOWL1TW1aSRhFziMQjxt0IddpDyX4kwV9Gzz/2xtQ=; 7:77mmk/Smz4k/LtLlJQgR4PiJpNraptRuLnW5FYlsgsTefIg2ykbpiLqVPw75XkIYjfrPXJlkNzyYfk8aPRVhMhQulu8NWhHGDs2B67wAfKbC25vZoo1jgnFB7ZFpiomf/L1C+D25KykCS3h+F5cOkKcA2td9yeE/AtwjVMuX75V5FQiWnJEos4qrc9/wQl1AZ5mSy2+vlFCq0eRK7l0bGEA5knvXkPHJVJFu+hNHgyMuVzWVNtmCgMPAyHfVDCnb SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BN6PR1201MB2497; 20:/40jt/D7KQzSGROJftRVO+GZa9Uy8LZilE61rnUP0xQdFK2z82AvBygwa0+GPCf73KiTm8irFZ2UztP1e5AdZA3A55uGL6c1fgMLew9+NuhASHHbQrqZRUy/nF7scrEmr425ca86hx+dAu+IcuSEAYuAF6akCjQjjTf19APxD+/vOIxCQmoPQb8r/FYdGl7/sKSUTzO1Jg8YgvU8IiJm5BwdZzMf1BlA5CLDWmhQy6I50qqtExnjIMpff6XyT2y8 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Aug 2018 09:59:55.4413 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 34af3209-2d85-40f4-7f03-08d601038593 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXCHOV02.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1201MB2497 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Huang Rui , =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP This function allow us to bulk move a group of BOs to the tail of their LRU. The positions of group of BOs are stored on the (first, last) bulk_move_pos structure. Signed-off-by: Christian König Signed-off-by: Huang Rui Tested-by: Mike Lothian --- drivers/gpu/drm/ttm/ttm_bo.c | 52 ++++++++++++++++++++++++++++++++++++++++++++ include/drm/ttm/ttm_bo_api.h | 10 +++++++++ 2 files changed, 62 insertions(+) diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index 7117b6b..39d9d55 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -247,6 +247,58 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, } EXPORT_SYMBOL(ttm_bo_move_to_lru_tail); +static void ttm_bo_bulk_move_helper(struct ttm_lru_bulk_move_pos *pos, + struct list_head *lru, bool is_swap) +{ + struct list_head entries, before; + struct list_head *list1, *list2; + + list1 = is_swap ? &pos->last->swap : &pos->last->lru; + list2 = is_swap ? pos->first->swap.prev : pos->first->lru.prev; + + list_cut_position(&entries, lru, list1); + list_cut_position(&before, &entries, list2); + list_splice(&before, lru); + list_splice_tail(&entries, lru); +} + +void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk) +{ + unsigned i; + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_mem_type_manager *man; + + if (!bulk->tt[i].first) + continue; + + man = &bulk->tt[i].first->bdev->man[TTM_PL_TT]; + ttm_bo_bulk_move_helper(&bulk->tt[i], &man->lru[i], false); + } + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_mem_type_manager *man; + + if (!bulk->vram[i].first) + continue; + + man = &bulk->vram[i].first->bdev->man[TTM_PL_VRAM]; + ttm_bo_bulk_move_helper(&bulk->vram[i], &man->lru[i], false); + } + + for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { + struct ttm_lru_bulk_move_pos *pos = &bulk->swap[i]; + struct list_head *lru; + + if (!pos->first) + continue; + + lru = &pos->first->bdev->glob->swap_lru[i]; + ttm_bo_bulk_move_helper(&bulk->swap[i], lru, true); + } +} +EXPORT_SYMBOL(ttm_bo_bulk_move_lru_tail); + static int ttm_bo_handle_move_mem(struct ttm_buffer_object *bo, struct ttm_mem_reg *mem, bool evict, struct ttm_operation_ctx *ctx) diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index 0d4eb81..8c19470 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -417,6 +417,16 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, struct ttm_lru_bulk_move *bulk); /** + * ttm_bo_bulk_move_lru_tail + * + * @bulk: bulk move structure + * + * Bulk move BOs to the LRU tail, only valid to use when driver makes sure that + * BO order never changes. Should be called with ttm_bo_global::lru_lock held. + */ +void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk); + +/** * ttm_bo_lock_delayed_workqueue * * Prevent the delayed workqueue from running.