From patchwork Fri Oct 18 05:29:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jingxiang Zeng X-Patchwork-Id: 13841091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63896D3C549 for ; Fri, 18 Oct 2024 05:29:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DF01E6B0083; Fri, 18 Oct 2024 01:29:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DA1876B0085; Fri, 18 Oct 2024 01:29:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C67B96B0088; Fri, 18 Oct 2024 01:29:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A6E706B0083 for ; Fri, 18 Oct 2024 01:29:56 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 030708057F for ; Fri, 18 Oct 2024 05:29:45 +0000 (UTC) X-FDA: 82685595930.19.72B4BDE Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) by imf04.hostedemail.com (Postfix) with ESMTP id 7259640004 for ; Fri, 18 Oct 2024 05:29:40 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=dmjwvmUz; spf=pass (imf04.hostedemail.com: domain of jingxiangzeng.cas@gmail.com designates 209.85.216.48 as permitted sender) smtp.mailfrom=jingxiangzeng.cas@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729229248; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tMK51R3vi5oLmBALn45aEZ3BaYW1vEVFexGaGCJTMUY=; b=u2n41HM5xbu4roHAL3PBXpjEI8lO7k4Bgibyd77LOmcX8g3J37hU3i9YsGnEyocHa0TpBt vLh3zvS4JAoAomQngq8k6HqADBSAcA3AwgT1OMQPaHiP7zjae5mcF46/uncz/NU7vVSxEv 1peqbXpOZSj0xksYvkK4Oc46BWKabKk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729229248; a=rsa-sha256; cv=none; b=yVeBncCigqYCnY3uci6xd97hjeFX92j43m7VHKMIhubRUU4bGm4TmUJ7ee+lQw9m5E/cvL tg+dPWsHW0HBHDwbH3FAI6IZT5gH29MGRYfqGK2ZzxBATzIy06yFLnKF8Ka2dReJnJa9PI 4c60kq9gBatpZKEzj4OEuahhmFcMcTA= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=dmjwvmUz; spf=pass (imf04.hostedemail.com: domain of jingxiangzeng.cas@gmail.com designates 209.85.216.48 as permitted sender) smtp.mailfrom=jingxiangzeng.cas@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pj1-f48.google.com with SMTP id 98e67ed59e1d1-2e31af47681so1355297a91.2 for ; Thu, 17 Oct 2024 22:29:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729229393; x=1729834193; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tMK51R3vi5oLmBALn45aEZ3BaYW1vEVFexGaGCJTMUY=; b=dmjwvmUzs+olP/fKONqEA0/6lI590keSJejwznND0wnv9WEdTJ5Uv6tHC9tlXoBqnJ 5hO6osuTd3LBFx6DDAjSTLSZfLidrcQp1/XVktxWp9n1gXmErT7uCnCzxbJqZwxY59ao fiIdaRQ3GWkxgw1YbDBjcQSDAJ4SPZSC5Tcilrnab2HplpfbArCl/FOc7VZjBPtsGG16 dzNaEIuUInqXGP+WSvYaXfJ1Z+4y7JDKr006WC17zeGk94vKQ5e+IhN6HSvQH2SE441B IfWMLSEPCxHDn+bGnnIueCtXCeJmFA3NiJjKSmk8yXrwy0udvlALnFo+y4oRwnu+hUhN 1n1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729229393; x=1729834193; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tMK51R3vi5oLmBALn45aEZ3BaYW1vEVFexGaGCJTMUY=; b=ae7Xqn9cv3ngU5zyt92ROCDQbrG/hiPEF4tp4uLsOBIaslHzn2bcJKy2Bsannmx8Mc 6yHfzCvC8fgsrg0VL8Do3EiDTlGNDe5vsArWDsnFEhr/v0uLdpKNuclAagGXBoO/SttO VkkqrgOIBYi8XnvKlqm035C76TDdegpBP52uVaHIoWP9aRnJXYc+QVgGbcwt+QCLupjg 2GzG/qMvAJrZdNq+XfTkKSu4p+J6X2SWpqC/8OgDpUaHiViIKUUtpCC7nf7PtD3+/r1a 4k0UugOTsZruEa0zidTjJXKOHZ9gkVz4AIIfgHnLXlPHXEjutIEDQRWP/agt5PlcNbV3 /YkQ== X-Gm-Message-State: AOJu0YxyCtAET14xk8aXb6cB3wXBKb0xx0yakPXFlrzrGUQEoVBnq0kc LeR/CRxLD05QEMZW63mcNolZ6Q68OLLVOTXX9rNsa+yJrQdZmoAMh23gCKPM X-Google-Smtp-Source: AGHT+IHQ1FuN57UU+hgXhXlPcRmmLgC1mtvnpNrZqMy8gmnmiGtXNEswAg8iB0EXwi1QByLecQqkmw== X-Received: by 2002:a17:90a:8d14:b0:2e2:d859:1603 with SMTP id 98e67ed59e1d1-2e561900ccemr1428967a91.25.1729229392616; Thu, 17 Oct 2024 22:29:52 -0700 (PDT) Received: from localhost.localdomain ([14.116.239.33]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e55d7bf525sm944286a91.15.2024.10.17.22.29.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Oct 2024 22:29:52 -0700 (PDT) From: Jingxiang Zeng To: linux-mm@kvack.org Cc: akpm@linux-foundation.org, kasong@tencent.com, linuszeng@tencent.com, linux-kernel@vger.kernel.org, tjmercier@google.com, weixugc@google.com, yuzhao@google.com, chrisl@kernel.org Subject: [PATCH v5] mm/vmscan: wake up flushers conditionally to avoid cgroup OOM Date: Fri, 18 Oct 2024 13:29:42 +0800 Message-ID: <20241018052942.3810740-1-jingxiangzeng.cas@gmail.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7259640004 X-Stat-Signature: j7p9r9e4ecuxz3ie1e36toiynaajw3jp X-HE-Tag: 1729229380-610943 X-HE-Meta: U2FsdGVkX1+Fhgci0XbSjGQzhoUI1iJbRMGHnobrG+GjixlXdOzjKTyEf7ONcT18Q1qFaLOmMGuXypPtvZTsSPaUxS1hIolA4hZ/hY24u3QylwnPu7bf5yduIMncMmlk4GpWo52mPrGTmCpFKpg6NQ0RfZdwbdLBZnerxa4OoWhuhVOsevDYGO4hnmls73BIOB7bMWeQcfM+UxyMkb1R+Yi330igSB01MLab7abTAWbJ3I9ZuGGoAcXFb8pCeQvqC4L8PHrxmJfzuC3rlcF9pxyC8Qa1939BCE389Q5ArWmZLGpiZKnTpHm13AxZuqEo/OTCpVGNuQse+wNJ7cQh7K9t30rdqYRzIYksilVzm4PCXhjQZFZ4ENsEvSUwcRyDHRmH3hKZ43rAii3JH2xUYOS4rgqrVEG+cXinqUcBjirIqdF1ATq5ZXEJ9TEFmT+v6eesrVDvME00epyh3oGCsiEq/a3NVVjrdjIinK7nuFPeJ5EGksFtTKnxGhgcPQAN/ahDc6+1cljup4RPpKTjIwfoYfNguBy1kORsrwlJO54kOL2KtoCJray9N6GvDk3DnIBMfWptMLNnJ33zcI9quoBKV1YzLLAUCTdTFX6r9fCnfhxeXnie3Xc0F0OAjlCW+582OFA0/HuRce5XaSuQ3TW5+VbXMN8pXq6oSDHdZKLuWMHw0YhNl0oSYoB5ROIUNj1MNRtEqTbd1rI1EWPl1O8V32y/IK37fMi/lnyBKnx87Evg8459LqQv/lg90XYfz55jk0kRpknn82pLhD79Mw2c5AsHhZsgcvUvhNGrvkVBjTlHeQPacoVnNMHBfX42Y5yPLM5/nmWc1MC6xspKrOI6rB4kX5QhxQuNJY+DKL0ehPmeRei45woa916vbwlllwToqngkw1oT0FzJQ6FsjBOrPmSxO6AjUN7A9e9tp5/sJ2N0DNy0UUJmS875UvMARWB4Tng7AXXJkzqhNju cv1nOIeb JsQ+BYpL+G8Xg5Zt9cPU3NqlXGBmwFvngryc1v43M5reORlngGN6Uk+cjnysP1xJHOnsQhOrB8bG1tBdFKgmk7WKWCvHVR47cWJLXWAXywnJqFWKZUFtDD162UwTBzXq5aNf+AN2f1vT8Q9mmGAVvjS98R3agOdWHc/F8os/A6OwmJXOw+dHdAnXlR25PpStXVnYgAZFV+gqAgsgsAGmSm1dUVQFfpkpcoXh+2wrDV+X0odYRI7H0vr5H3Ocipd3wUVELCWbOcuZX3UZBMfsEfkV/NRdERjsc0y+X+FiCiWJ9mvMhmLEpRIgw1gB3SebqPty4ilJCEYNfXgIqn2jiqZmWWcGefa0ixdDsTHSeUGNWPBkXtQ3gbhBISN+UpOwRPt4odCAAE3+gPaOOp+TlGo6Ky5xWrfQuXHwukoS21t2A4GUkbqlMMv3Iteqy2YzarYfM X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Zeng Jingxiang Commit 14aa8b2d5c2e ("mm/mglru: don't sync disk for each aging cycle") removed the opportunity to wake up flushers during the MGLRU page reclamation process can lead to an increased likelihood of triggering OOM when encountering many dirty pages during reclamation on MGLRU. This leads to premature OOM if there are too many dirty pages in cgroup: Killed dd invoked oom-killer: gfp_mask=0x101cca(GFP_HIGHUSER_MOVABLE|__GFP_WRITE), order=0, oom_score_adj=0 Call Trace: dump_stack_lvl+0x5f/0x80 dump_stack+0x14/0x20 dump_header+0x46/0x1b0 oom_kill_process+0x104/0x220 out_of_memory+0x112/0x5a0 mem_cgroup_out_of_memory+0x13b/0x150 try_charge_memcg+0x44f/0x5c0 charge_memcg+0x34/0x50 __mem_cgroup_charge+0x31/0x90 filemap_add_folio+0x4b/0xf0 __filemap_get_folio+0x1a4/0x5b0 ? srso_return_thunk+0x5/0x5f ? __block_commit_write+0x82/0xb0 ext4_da_write_begin+0xe5/0x270 generic_perform_write+0x134/0x2b0 ext4_buffered_write_iter+0x57/0xd0 ext4_file_write_iter+0x76/0x7d0 ? selinux_file_permission+0x119/0x150 ? srso_return_thunk+0x5/0x5f ? srso_return_thunk+0x5/0x5f vfs_write+0x30c/0x440 ksys_write+0x65/0xe0 __x64_sys_write+0x1e/0x30 x64_sys_call+0x11c2/0x1d50 do_syscall_64+0x47/0x110 entry_SYSCALL_64_after_hwframe+0x76/0x7e memory: usage 308224kB, limit 308224kB, failcnt 2589 swap: usage 0kB, limit 9007199254740988kB, failcnt 0 ... file_dirty 303247360 file_writeback 0 ... oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=test, mems_allowed=0,oom_memcg=/test,task_memcg=/test,task=dd,pid=4404,uid=0 Memory cgroup out of memory: Killed process 4404 (dd) total-vm:10512kB, anon-rss:1152kB, file-rss:1824kB, shmem-rss:0kB, UID:0 pgtables:76kB oom_score_adj:0 The flusher wake up was removed to decrease SSD wearing, but if we are seeing all dirty folios at the tail of an LRU, not waking up the flusher could lead to thrashing easily. So wake it up when a mem cgroups is about to OOM due to dirty caches. I did run the build kernel test[1] on V5, with -j16 1G memcg on my local branch: Without the patch(10 times): user 1473.29 system 347.87 339.17 345.28 354.64 352.46 355.63 358.80 359.40 358.28 350.95 (avg 352.248) real 166.651 With the V5 patch(10 times): user 1470.7 system 339.13 350.58 350.07 355.58 348.96 344.83 351.78 336.39 350.45 343.31 (avg 347.108) real 165.821 Test results show that this patch has about 1% performance improvement, which should be caused by noise. --- Changes from v4: - Add the number of unqueued dirty pages in the shrink_folio_list function to sc->nr.unqueued_dirty. [Wei Xu, Jingxiang Zeng] - Reset sc->nr before calling lru_gen_shrink_node function. [Wei Xu, Jingxiang Zeng] - Modified the conditions for waking up the flusher thread to avoid interference from unevictable and anonymous pages. [Wei Xu, Jingxiang Zeng] - Link to v4: https://lore.kernel.org/all/20240929113050.76079-1-jingxiangzeng.cas@gmail.com/ Changes from v3: - Avoid taking lock and reduce overhead on folio isolation by checking the right flags and rework wake up condition, fixing the performance regression reported by Chris Li. [Chris Li, Kairui Song] - Move the wake up check to try_to_shrink_lruvec to cover kswapd case as well, and update comments. [Kairui Song] - Link to v3: https://lore.kernel.org/all/20240924121358.30685-1-jingxiangzeng.cas@gmail.com/ Changes from v2: - Acquire the lock before calling the folio_check_dirty_writeback function. [Wei Xu, Jingxiang Zeng] - Link to v2: https://lore.kernel.org/all/20240913084506.3606292-1-jingxiangzeng.cas@gmail.com/ Changes from v1: - Add code to count the number of unqueued_dirty in the sort_folio function. [Wei Xu, Jingxiang Zeng] - Link to v1: https://lore.kernel.org/all/20240829102543.189453-1-jingxiangzeng.cas@gmail.com/ --- Link: https://lore.kernel.org/all/CACePvbV4L-gRN9UKKuUnksfVJjOTq_5Sti2-e=pb_w51kucLKQ@mail.gmail.com/ [1] Fixes: 14aa8b2d5c2e ("mm/mglru: don't sync disk for each aging cycle") Signed-off-by: Zeng Jingxiang Signed-off-by: Kairui Song Cc: T.J. Mercier Cc: Wei Xu Cc: Yu Zhao --- mm/vmscan.c | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 2d0486189804..97e0af338ee0 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4292,6 +4292,7 @@ static bool sort_folio(struct lruvec *lruvec, struct folio *folio, struct scan_c int tier_idx) { bool success; + bool dirty, writeback; int gen = folio_lru_gen(folio); int type = folio_is_file_lru(folio); int zone = folio_zonenum(folio); @@ -4337,9 +4338,14 @@ static bool sort_folio(struct lruvec *lruvec, struct folio *folio, struct scan_c return true; } + dirty = folio_test_dirty(folio); + writeback = folio_test_writeback(folio); + if (type == LRU_GEN_FILE && dirty && !writeback) + sc->nr.unqueued_dirty += delta; + /* waiting for writeback */ - if (folio_test_locked(folio) || folio_test_writeback(folio) || - (type == LRU_GEN_FILE && folio_test_dirty(folio))) { + if (folio_test_locked(folio) || writeback || + (type == LRU_GEN_FILE && dirty)) { gen = folio_inc_gen(lruvec, folio, true); list_move(&folio->lru, &lrugen->folios[gen][type][zone]); return true; @@ -4455,7 +4461,8 @@ static int scan_folios(struct lruvec *lruvec, struct scan_control *sc, trace_mm_vmscan_lru_isolate(sc->reclaim_idx, sc->order, MAX_LRU_BATCH, scanned, skipped, isolated, type ? LRU_INACTIVE_FILE : LRU_INACTIVE_ANON); - + if (type == LRU_GEN_FILE) + sc->nr.taken += isolated; /* * There might not be eligible folios due to reclaim_idx. Check the * remaining to prevent livelock if it's not making progress. @@ -4589,6 +4596,7 @@ static int evict_folios(struct lruvec *lruvec, struct scan_control *sc, int swap return scanned; retry: reclaimed = shrink_folio_list(&list, pgdat, sc, &stat, false); + sc->nr.unqueued_dirty += stat.nr_unqueued_dirty; sc->nr_reclaimed += reclaimed; trace_mm_vmscan_lru_shrink_inactive(pgdat->node_id, scanned, reclaimed, &stat, sc->priority, @@ -4797,6 +4805,13 @@ static bool try_to_shrink_lruvec(struct lruvec *lruvec, struct scan_control *sc) cond_resched(); } + /* + * If too many file cache in the coldest generation can't be evicted + * due to being dirty, wake up the flusher. + */ + if (sc->nr.unqueued_dirty && !sc->nr.taken) + wakeup_flusher_threads(WB_REASON_VMSCAN); + /* whether this lruvec should be rotated */ return nr_to_scan < 0; } @@ -5942,6 +5957,7 @@ static void shrink_node(pg_data_t *pgdat, struct scan_control *sc) bool reclaimable = false; if (lru_gen_enabled() && root_reclaim(sc)) { + memset(&sc->nr, 0, sizeof(sc->nr)); lru_gen_shrink_node(pgdat, sc); return; }