From patchwork Wed Nov 8 06:58:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huan Yang X-Patchwork-Id: 13449622 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36185C4332F for ; Wed, 8 Nov 2023 06:59:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC1768D009D; Wed, 8 Nov 2023 01:59:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B69358D0058; Wed, 8 Nov 2023 01:59:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9BC1A8D009D; Wed, 8 Nov 2023 01:59:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 87B9A8D0058 for ; Wed, 8 Nov 2023 01:59:31 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4E2D1B5166 for ; Wed, 8 Nov 2023 06:59:31 +0000 (UTC) X-FDA: 81433886142.03.CFFB26A Received: from APC01-PSA-obe.outbound.protection.outlook.com (mail-psaapc01on2115.outbound.protection.outlook.com [40.107.255.115]) by imf29.hostedemail.com (Postfix) with ESMTP id 62C00120004 for ; Wed, 8 Nov 2023 06:59:28 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=vivo.com header.s=selector2 header.b=VpymtuJE; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=quarantine) header.from=vivo.com; spf=pass (imf29.hostedemail.com: domain of link@vivo.com designates 40.107.255.115 as permitted sender) smtp.mailfrom=link@vivo.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1699426768; a=rsa-sha256; cv=pass; b=JIGxEfHEiYDe0T51lFlOAdIqGW8cfJMlLaPpT/pktH6TPC3oGMOwsrjK0sQsPBbvJqq7ah F+iA9vRZRZ3f3V8UzLW95teZWiJFrQe8apRuQrUAKuO6wUW6iyx8jb44SM1otsdxezHufC HLXy61jaT+mCVOovU4QmF1hs61HX2FU= ARC-Authentication-Results: i=2; imf29.hostedemail.com; dkim=pass header.d=vivo.com header.s=selector2 header.b=VpymtuJE; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=quarantine) header.from=vivo.com; spf=pass (imf29.hostedemail.com: domain of link@vivo.com designates 40.107.255.115 as permitted sender) smtp.mailfrom=link@vivo.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699426768; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IUB6Be0jevwGt8NeqUa3bfJfaaD7+VKX6fpXnVPCeZM=; b=x62NI+/z9QO+i0e1RGo7CMfawdO06JTlev6sJXWrF8m7rYEeBdI/1jt3zCgQjBvD9GwdCw h6mPFyrN93NaflFFpRNEaqZrTd1E9mkidNhQRLTz/RkSVcPC4UwYmejI73S4j/mV1dUWdO k8ccCx098vRtDls+lzQ4dW0+hEnVYBw= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jj3EYZdlPGcZlM93m2vGlfXD2WRSoSpdKDpHd69Cq/1fVq6TzQC6lGKZDpW95LF1JMHjBA66f3LwANv1jaoCWUcB8EZadJ39sxY2SmcWkQIxy/AyRn0W26wQibhuJRQ1CCVBpo4Odpcp7B6Ae5G9koEgQsoCAFLm+W1DIHJMayvi7c9+rdlJphGmrdhow8qKBJhnDzDHwwgA59150Jiomb1RoB4CEEiiekW/KVskk2DFrtHWtYKivG2gDDf9+ztFXlyT9XSVoxJ17ldguL8HnehVMi1pIBj8JcehamVQV+c0gyG7SIWbEUCPlXQk75HbXUayBc1mC9YdVzB2/Lfiyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IUB6Be0jevwGt8NeqUa3bfJfaaD7+VKX6fpXnVPCeZM=; b=ZjbPsMDOwLgMI+m2iTFDfGoQBdd8hZ+tymVjIPkdGzYe9VQkqmh+UFhdRXHFk2/ZH0bMb4+Hckx+nUfdYv7zzQLN36R4KPFXizN/CgP3vVnIEsgthnRgxsWSSu/voTCliMJLlzXPLu361whrRxJd6joAmoEo2VeY76BTpM3/xxol6YgPy231hjVtQd0Uf3vB48k+v9NZMbSo4p2m3YKiKsFKY+GydArmG5f9AUGJKlacT6C9VCl5FRHYhVkHQjXUXAU1/kLoAtjA/r7reT50Kn4yL3B1V7B4ge5cywp1T8QoqXZnB4AcnByrNYYYlvfAMQoaWn5y0MiZZun4kgIUvg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IUB6Be0jevwGt8NeqUa3bfJfaaD7+VKX6fpXnVPCeZM=; b=VpymtuJEQ1SAbFMJnNq0jXihnjCNI2NXNoKUnxHf7gfsSfVQV5OVu2E8YXjQreSAW0n3Ws/Sx4v1whmsja2WqsCzj1zQ74xOfL/fGAAbCEWq2neTdw/LGsd2HQ3w5cG3bVwwka+ZHRGTXjBAAVU7RzId3NnU0mVpj8ACS3OeSaD2kMllJNL6zyivojo745Iec8X/CXjYF1iDEVH2XDGDK6rvIzl/OakhU0KZlRtNKPZp5sT55N4x0m/2ZK6UQhVf+9V30TfWsVq4ZmifZb2waJ4j2RKYIryoROWJkI7jV/CJggGXWwURScPgVcKlG5QEh6UT1bRIW8naN7Xi3I+jYQ== Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by TYZPR06MB3983.apcprd06.prod.outlook.com (2603:1096:400:2a::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.31; Wed, 8 Nov 2023 06:59:23 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::d754:7b3:dc4c:6b48]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::d754:7b3:dc4c:6b48%6]) with mapi id 15.20.6954.027; Wed, 8 Nov 2023 06:59:23 +0000 From: Huan Yang To: Tejun Heo , Zefan Li , Johannes Weiner , Jonathan Corbet , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , David Hildenbrand , Matthew Wilcox , Huang Ying , Yosry Ahmed , Kefeng Wang , Peter Xu , "Vishal Moola (Oracle)" , Liu Shixin , Yue Zhao , Hugh Dickins , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: opensource.kernel@vivo.com, Huan Yang Subject: [PATCH 2/4] mm: multi-gen LRU: MGLRU unbalance reclaim Date: Wed, 8 Nov 2023 14:58:13 +0800 Message-Id: <20231108065818.19932-3-link@vivo.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231108065818.19932-1-link@vivo.com> References: <20231108065818.19932-1-link@vivo.com> X-ClientProxiedBy: SG2PR04CA0187.apcprd04.prod.outlook.com (2603:1096:4:14::25) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|TYZPR06MB3983:EE_ X-MS-Office365-Filtering-Correlation-Id: f506a763-a0a4-427f-8a44-08dbe0283d19 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OIB58F67zvsEa5kDgR22kw9XiiibeYLh4UaPmd2yly1Pu41aePNxhJc1Xqv/wi+JO9bn2JEuSrFWM0xMePkjCYYvpNJMRd4oEkwuaIFXv55QFeaY6lNwgKsGc7n+b+db9O8J13dsJIJ7fgmYZUXG8Bop3cwVLQsgg7bWiITDC7FfswDSdN0IzBVjl3zgP8GJiMUU/CuvkuVboovoiNQ0DuYpNhC1RtfppAke2Wu/tmeg9NdC0WqX5riAubAM97g8rBpXlRx7maGIH+Qg5wVnOsmR5MimkBLXeAhdiXJEvdlyWVp0os1F67kQcdmBkn343gG3TLDZbarocgVmf/sm3/IXyVCdTWcBZxEjafQUdG+v0VEfkhphVawboRYEutVznFX7l609hhoYrP06AeAMI/4bNJ/Il8Uf99lDA7oiYFYoTRp/XOFLrg3hD+Ijz3cB4/+0ITwRcWRwWwNslNyoSa5e3smhUThIkYDzKZGjJ4q1jtFBON/BvzBS5zZoTuPlNXX77/+vyONfSlguhZkwQf/P3m5yplB9R04BKjS5/c2P3nE1rmk6qAkhSBTP1mbE8P37j8kIwgAdlpqGrpfa8N7jTJJS0zTxQfuTkjtzZhjXc8cvrWjHUNI0mqjjHVph8bKzgKbWmTYtobDgwimbAw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PUZPR06MB5676.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366004)(376002)(346002)(136003)(396003)(39850400004)(230922051799003)(64100799003)(186009)(1800799009)(451199024)(6486002)(6512007)(5660300002)(6506007)(478600001)(52116002)(6666004)(83380400001)(8936002)(8676002)(4326008)(7416002)(316002)(2906002)(26005)(110136005)(66556008)(66476007)(66946007)(86362001)(41300700001)(38100700002)(2616005)(107886003)(1076003)(38350700005)(921008)(36756003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 1Jz3+P9rWw2w0tLsgYmtAU7PuNVnP5eSZVzYTkeHX/SYdqBOMTqVQESCRn6Nq50XFlqKJa/JSAO+UMB1ycAJG0eGhWwixd2U0cc3O6iUQLZYL8GYxrE386fQgAKvbtwvlMndvDHYESAQd7TLSGT5enJTO6TmCKYG9ZtZnEmzIyC7LMNOagfOJMhwD4xD2LNjGTLkqGbCdZv6et3bv1WQTINZTgHN4097oatPzjt9KyDzQIr2H8/eMaJwi6RM+E6CiqWH1Csmdiw0I5YHTyNodsEl+rYspjcKxB55nIICTzSP6MPdgIB71n9d2DNl0nQK5NGJdVH+NlMErRIRGP4hpR61QM4lFw7xd5YTb2BSGuVEzTN9AZUbyheuRD/spvtduoNk3FZIbaH9jrCEenqKIUbRjsmtjyntcv+kN6bcYNa1veT4boi7bg0NzO4b3/Bz8doaBuEp1Bh5j4Sgodh4CfI7WBf1KzjYFs5qtCjqLC5eJOfmGWy92LQe4Djy44n9gE8qi8A0aJrtxCyg6uAiA+nd7ksEy7QOVBlOCzOKG9ptWPjLBup/DG0WBSFdgoc4499gd4ixuDrykfjXNwVB5XPGZfxwCGV0UleY8csewHYGkzv6e6u+OEN0EIC+ZENX9qhcZZhxBXBAobpDi45NGvHPko/FShdsH1Y4hOUWmzEx1g7b/k4Yy7buzn3CwjpIiSFLLLEmzs8/sBAwa9phU6NQb+XOkjznYLogS+gVz86c+RX/xc/QA+r60pCSDdet+YSyWfw4GjQ3ZxQ9cx8ZDZ2+FqeCErv+gnb3AsdkePQl5rsmlu+Ko8O211SVwNCmOzbQNWrsBA8DQqQiEp9GzJ6HVnLh/EObjWphoYEHiDvxHGkgIy9yckbJhCFosjuz9RAVp1PHAUWHuha7UaSwtawHdtAoX9mTkEsqNFX8FIik2/JdEgyIrKC2a0VSZtPWIoWn+SvoyO2W0d0NNFPHxuNzTICe7VFa6q82r2xjeP3eE0HLY2w1hvhyxqzYroI5L62u69rslam5lVBJCGvAwx1vi9ay8NIQ2YnMaZOgN5xeQ86tMbLgTVT7glGX5wTDDQQyZxXJ0ZewwB+0HbwIga0kJbhIveUM7uvXPhRM6+yifzDKlSe4mKmSTi1Gkd/D/PsK8fp/FubzU/MqVXBqrO/4z3MzK/7SSO3KqGxmPCT7diMQeVoihGPWrrwqGm8TFQdpAubODd++hWRQ4QqpjyqNIHMBQoFMBBiuyNzYCktdHXqnwLIV8xnrUoxOX8MEfHJ9HDeeP4iqjovj2Uo+56G7ny/qB5EZuIEQIy8XXcEy8vT88aazlAxLGMwujoJ8h16SztBnS7R5EoujYwmQv/ZyMoTMLW+OO0iIS3LyOyTp5HBJsoLJ10/jGXlJw3tROR+WXc3OvCuXK4ThlIBoh8MGl2aeHAiesxHk16mCtN8//A9lNJRsKrJjdwB2RHZasWXlHajWTwA4l3yDKDaa4X7utyitWtmw1OwB2446rO2P0YE2I9dVpU8wX9X3smSJXqBvsYWV8d3CvOmMYlUaTP2oik859qjOzGFRQIKlczxlZE6XQeVjhALMHW03iPJs X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: f506a763-a0a4-427f-8a44-08dbe0283d19 X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2023 06:59:23.3398 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Ug69hlALgR3GwUIA/+KIRoqdUDlRvTU77rJDAEmLouy0zHwxUB3+FWL8l6rYagluc3Lvmn8zvP0mvibh9eUCOQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYZPR06MB3983 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 62C00120004 X-Stat-Signature: tskzi5rx9xrnngkzou3w8muiut9gd5cb X-HE-Tag: 1699426768-452661 X-HE-Meta: U2FsdGVkX18zRkWLe1AF4SxA9QPcxgvQMRhVLt+xqjprBBROGAlbAiEtribLIYm4b7Qrd3rC5YhUnb2uZ/nWNMFj9B9Y05kNTGTNZ+6sYoIvpm2Q/uaqn/5W/KeAHhXf6JKuBDoXrFMdGIY3xbUcau9w2oX691N7wessbDYn8se/Ql1PA2Sj0geSf0VTY6iztD98hRG2W+z87ZsHS72mbYS3n4XokuWH8AJz9NYDv5ToTXNoIrqwSD391yMqNGe8GOtrgEAupjY9FZsfhwOCQ8ioSN0e3dNblcaT+2Io+5drapfDppQwyMMBI7ACFvMz4bX5w7LwqsBTSpr4ajr3F8+WV8qkSd9wCTVVp1IyDXVVq9Ncyw3yvMb/nYq8Lx4Us/KfT85T/PTGBjhECSZf7GC7ry0HZXQisrxsLkkil611iLdOr3p65/8vFo1jPfElSdbi+0Qj3KVoTO69NaG7zgOHpmM4NYy88oLnD48NQEdB03un5NOsu+gG+yHuj/b3xQgKQS0xoPAzOAXc78Nj0/stjLcUbFvYh8RDyQW34BYInFsbtZt6OjtLnOWfjx7miGvov0ItoVr1o5DeCK9uvyuRZShiWQ5fwLqGp0i7GbA4JB94HAFwprzXKqiy8f2boT92XYG2sYGrS4I7iJw48kyZ20EIdnP2XXqdFHdJoy4om62Kq2nLf/4etnpLL7lrlcXnw156TCPqZF/nYrCjVp5apioaT4vwgymyj3Zm8p3kpMP+c81jzhn8/rlzdJ7txe5G20amgGXIhJL1hSK28/jRjcUfiO8wymxjdh2S5Lw+cXI6mZCK8UB9HPmjnhqHjoE5vkByEH27aXAFQlw6QzJ8VRVq6NHtjYrECX7SCtz88yRjL+inUOtIgGCl2kY9PwzYbWxPqsqjUCjSBP9IjJb28lFgkCsDALSP1o7DsUujC6+5wE0QNoCz5waxkOSBbm9c+/aWq8+otI6qHns laff78aZ 9HaXIZlDXsrjdy1N4+mtaGfYl8zQUsQKLlFj9e5plUtWqwGYbI1HxgapRQeTl4tJm3jWtA212epAzNrnJJ0VtG6EMQkp9r4yeoOoPrZ/HAxGV4PeUFme4XWscLu11x1Lrlz52ZT1LkoNgIyuvcgFJCLMZCWyBleKZY58NMY8itsaQR0QU03srt8bTNB//pRgxy0QeAvBsL7WCS0dJdIYwquxm7odlQvRY6/QGooHx0IDi/6vsGJHrE2iBTml8AsR6vW5Q0PKNBfDuWBTnmzwO/xqIYIZo+pNobUy9Ddz1x8m3OCetJ1WZ/uVmPEYwYWBrtFpPoPuU++uqYYE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This patch apply unbalance reclaim into MGLRU, if unbalance reclaim, will not scan other type folios. Signed-off-by: Huan Yang --- mm/vmscan.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 6ed06e73143a..9243a1f0d606 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4481,6 +4481,7 @@ static int isolate_folios(struct lruvec *lruvec, struct scan_control *sc, int sw int type; int scanned; int tier = -1; + bool unbalance; DEFINE_MIN_SEQ(lruvec); /* @@ -4488,7 +4489,13 @@ static int isolate_folios(struct lruvec *lruvec, struct scan_control *sc, int sw * available from the same generation, interpret swappiness 1 as file * first and 200 as anon first. */ - if (!swappiness) + if (unlikely(unbalance_file_reclaim(sc, swappiness))) { + unbalance = true; + type = LRU_GEN_FILE; + } else if (unlikely(unbalance_anon_reclaim(sc, swappiness))) { + unbalance = true; + type = LRU_GEN_ANON; + } else if (!swappiness) type = LRU_GEN_FILE; else if (min_seq[LRU_GEN_ANON] < min_seq[LRU_GEN_FILE]) type = LRU_GEN_ANON; @@ -4504,7 +4511,7 @@ static int isolate_folios(struct lruvec *lruvec, struct scan_control *sc, int sw tier = get_tier_idx(lruvec, type); scanned = scan_folios(lruvec, sc, type, tier, list); - if (scanned) + if (scanned || unbalance) break; type = !type;