From patchwork Fri Dec 8 06:14:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13484886 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A133EC4167B for ; Fri, 8 Dec 2023 06:14:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0CE456B0083; Fri, 8 Dec 2023 01:14:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 02F996B0085; Fri, 8 Dec 2023 01:14:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D9C066B0087; Fri, 8 Dec 2023 01:14:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BDE6C6B0083 for ; Fri, 8 Dec 2023 01:14:22 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 8C40D1C07F7 for ; Fri, 8 Dec 2023 06:14:22 +0000 (UTC) X-FDA: 81542636364.11.B605B1C Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf09.hostedemail.com (Postfix) with ESMTP id C83F014000A for ; Fri, 8 Dec 2023 06:14:20 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=d6lmE8ZI; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of 3PLRyZQYKCKEZVaIBPHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--yuzhao.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3PLRyZQYKCKEZVaIBPHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--yuzhao.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702016060; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XyO/ljh9FKCeCHVPdhFy9bPgiCVl0Vc+13vD9r1JAPY=; b=b5SEpHM+0gQ5EznBaWvzeuz2fD9DW/HUamL5CHZie/M0b6c5PrLMReaG8i+LToubhSdFut LO/XUZ3G/Mxi8iX66Qjo9gqhlMxsNRMgTiZxh/eTvH6xIqBeSTk/tRFsP4PaEDiydN4/Uw XMGmu8JEmZfSgRgkCyEvlUSznKaU4sg= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=d6lmE8ZI; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of 3PLRyZQYKCKEZVaIBPHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--yuzhao.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3PLRyZQYKCKEZVaIBPHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--yuzhao.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702016060; a=rsa-sha256; cv=none; b=KV01eid4BmKVLGQasP8CFd5yc7tLYsRbEtofoc1pcW1V9r8CZDlF2z0ov3BT4IY1jrY5Gn bQjCiUlqOsPS3acsRJ+0WhZluPM6I8CHd5cTJaEHvzY2CAfAHb38+/RUGtqwjAqgkv1RWW Rw1rHXlsULzGitv54fU8EeHhsLfcKZY= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-5d7e7e10231so11270367b3.1 for ; Thu, 07 Dec 2023 22:14:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1702016060; x=1702620860; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=XyO/ljh9FKCeCHVPdhFy9bPgiCVl0Vc+13vD9r1JAPY=; b=d6lmE8ZII3MDBDYQX265ZWmzjcDuTpDtCEN0OUbH1FHR3G/+D9JqWb4z+m1WT63ywA H7lkWdTTf6fH4VLVj5WfsOWnsCECqI+rSrU97baT6L6yXNEnDLhzWqselpZ5tPJDHiTd 4BSSA3BCFFH8sxHlLN1slK1tp3Bz/cZTFsMX6eRquWQwXY4jREcbLZ2RSVn0+kTefRhm MgJOwaOnPtVWom6BAmrmT2zfdxP+6vvGV6bd/nxzRRVEBM1MkIlGOxmb44l6VvBDM7Y1 feGTIhIybnj7EtydVt8zvaV1lG/RSWg4ecvZcpdwuAYhoC/UKxN0CNHuBRCVHLGh5S8S pZ7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702016060; x=1702620860; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=XyO/ljh9FKCeCHVPdhFy9bPgiCVl0Vc+13vD9r1JAPY=; b=XhwNwndhwmkjuT2XPPlz5DvSBQUwxPbky+9f/tLjS/gGj/UObBLSMK6aYUjNAfVUA/ 6Giom4Ts9Khse1ZWblD+h0uthbxnt0ae1nNBQ93U/v1AlhukyiIHX4SzAXzeX/ZDrzAS qH+R2YQOTjaLotNweJ/jlzsYbqhHPHytsFI3K0ZXVpw01cWl9j8mfm3BQaSSmvkScNDm 1NEQ8s5aJO4ShLkhS7qjVg/agCBWy+MOKGEzkLgg5k0WG2ITaUluxp66Z3HrmTzi3V9H Gv5v0UlLdsxu8QNQALnn+hUn5yP60zQBFMF3tsb2gjjW68eTMMSAYC2CLjT0sMekzKWP 13ZQ== X-Gm-Message-State: AOJu0YxVzpqhTjrmajb70vWic2mE2wNQpE06y914DtmIHSbeikWYcmxn 08aF6gWSHpPhQyc/ksrsTfQ4JLF82bA= X-Google-Smtp-Source: AGHT+IHgAD3BUwnh/ze1lDgA5VK1+NfzNOWSF6TnZDywN1IKmaY+ZlKGrPraa9vlIOKdRdO0XE83PMMSUHc= X-Received: from yuzhao2.bld.corp.google.com ([100.64.188.49]) (user=yuzhao job=sendgmr) by 2002:a05:690c:d05:b0:5d3:e8b8:e1fd with SMTP id cn5-20020a05690c0d0500b005d3e8b8e1fdmr7250ywb.3.1702016060001; Thu, 07 Dec 2023 22:14:20 -0800 (PST) Date: Thu, 7 Dec 2023 23:14:07 -0700 In-Reply-To: <20231208061407.2125867-1-yuzhao@google.com> Mime-Version: 1.0 References: <20231208061407.2125867-1-yuzhao@google.com> X-Mailer: git-send-email 2.43.0.472.g3155946c3a-goog Message-ID: <20231208061407.2125867-4-yuzhao@google.com> Subject: [PATCH mm-unstable v1 4/4] mm/mglru: reclaim offlined memcgs harder From: Yu Zhao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao , "T . J . Mercier" , stable@vger.kernel.org X-Rspamd-Queue-Id: C83F014000A X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: cri5mqztpndne8uneknb1ksw3mr8uczz X-HE-Tag: 1702016060-369306 X-HE-Meta: U2FsdGVkX1+jZonCBvqC8ScQ2mgAwUmB/YQsJLw55i1CLEvXiFcRDw6PJzdn+MvGycLqlci7kQYyp0dPVH/NA3WkuhHoBwi534sJ9y0rvFQLA/uVjDn5q25pYweOJVjW1UgTWLr2/HX/PxfkcPVjGxlRtFxGDJFdYjA48LeirmCZ8JHiDVZCp6XN/pXNxPAc99pbBKzRjpZWxluyunmy37GlRtjbREIvDlBrMKmCk8vWoAPZpyk3XViUQrlwZXZypF5UO2XVB0NfJzJX/NR5aNShBQR5Wwnkj1OFGkSQk0gL7F7hZRkqswcz5FEYtYMTKA/EhTgacWoS/3WbIwqC1SIkPA2w19IOcrdwafHeLaHZpbYZePFe5Sy5tLJ1EE9iUIwiCaGtO4AVcLb6KTNCPv/sy6CdO1H1m9BudPgFUAK9L51sJ6abV/6cl2QHmdqN94SipoTBCZEFK0/x3YRe+NydLhWZPsty5NpyN/oAhobu0+PKa5bqNyikPXNJoEh/EgeTFuZ9/o0IYvXbNnp14katkvZg0JYsaHkpftD4mDtoB1WezcfY02kWwFq37o6ZapqBQcsPt/fdwP7godz9/dmIwVo+ddE54lxrKRMj838vBXz2qJiM5OkaO7VkUY9oZxLUpE4urx91DrHgWhqSruXOAJSGRU8JB0GQF0igm5ymun65saBz3w1KvhWreQ7BeOey24EmUmXF4UYrGu6dejHEyqxiqTCWzgZxJQdQiHThiKFQ0dsnip1XTDTlczb4WVREIXKiK6SfiYd0fNRmPQJhVSi19U03b9V5Rr1tC9GNkuoFCdDZoevZbU4FxsjOy2ri/e2Um7u7LS2bMolLcUmPvVvQwVW8Nh9HGtsxRDu1HtQA/wyvfFNUF/7Qa6PyoXVBw3VGwxI8z4V4qO56E5WAV2AfzMPeKYiGglpWnsYSzXYcXl9kbn1gRAL1al9R5pfiS2ckQ6zLyEYYXAp tn+MP6dU MW7jOpXmBnhE7JSHIcvE0C2pJw3FDx5pUPpnjjWoUY90u34vF18GKzpDqTnLT7u5RjbrjbcAQXV2CxFj322K7IphdKj7ECyuaMvpQnvfthss1qmZnL/NV06Mpyzyv4JUuUORv2nE/Slpm1CpV2GEfJjdx8jpl3OnOvzgC/+JAU85BIim9wgCXPMd3ohD8Jt3LfDZTyecNAv0QOA+VVe4MfNB/JWR04jzTZ8ejg+UYaq3dz6WdpB+kx93FMml6DNtZUkamvwTlTCWyZkciVygzkgZQkkNQZCq0Sci+rpxraypag1lU6tXzMPa95DJfLzgFgdTtL0K4tFnZxwYPlgoFdkjW0aeH45bClOPcobliDSAAF63CJMZa5PceekYp7OLJ6l9SpB1p+YloM05oHMmCeLp7GgvwHz8ibuRJkHX1JxEid62Krdsq+Iz15sNwnBWa/OEB1IuSHMVbdWYUIFVBb2RxxQcWStCz/9KnBoAvzQ8I6slCGtqQyVKfZkdMYF4MjJeAwe2i5Y9ZLGYPKqZHJDiS5/yqBr+UwgDG/OT12Rah3CCVF25Fpb0lRnSMaVWBqbe1gVKbDMkkBUzm0yczy+Fdc+fkC1njuJg/aSNjWU5Lr6PNJb3iZFY5mteCCd5QJoTzZKGRvLTLbqQZe8PWVg3xzci3so41Wmg+HQcTTo1C7leNUUvm8ELeHw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.046434, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In the effort to reduce zombie memcgs [1], it was discovered that the memcg LRU doesn't apply enough pressure on offlined memcgs. Specifically, instead of rotating them to the tail of the current generation (MEMCG_LRU_TAIL) for a second attempt, it moves them to the next generation (MEMCG_LRU_YOUNG) after the first attempt. Not applying enough pressure on offlined memcgs can cause them to build up, and this can be particularly harmful to memory-constrained systems. On Pixel 8 Pro, launching apps for 50 cycles: Before After Change Zombie memcgs 45 35 -22% [1] https://lore.kernel.org/CABdmKX2M6koq4Q0Cmp_-=wbP0Qa190HdEGGaHfxNS05gAkUtPA@mail.gmail.com/ Fixes: e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists") Signed-off-by: Yu Zhao Reported-by: T.J. Mercier Tested-by: T.J. Mercier Cc: stable@vger.kernel.org --- include/linux/mmzone.h | 8 ++++---- mm/vmscan.c | 24 ++++++++++++++++-------- 2 files changed, 20 insertions(+), 12 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index e3093ef9530f..2efd3be484fd 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -524,10 +524,10 @@ void lru_gen_look_around(struct page_vma_mapped_walk *pvmw); * 1. Exceeding the soft limit, which triggers MEMCG_LRU_HEAD; * 2. The first attempt to reclaim a memcg below low, which triggers * MEMCG_LRU_TAIL; - * 3. The first attempt to reclaim a memcg below reclaimable size threshold, - * which triggers MEMCG_LRU_TAIL; - * 4. The second attempt to reclaim a memcg below reclaimable size threshold, - * which triggers MEMCG_LRU_YOUNG; + * 3. The first attempt to reclaim a memcg offlined or below reclaimable size + * threshold, which triggers MEMCG_LRU_TAIL; + * 4. The second attempt to reclaim a memcg offlined or below reclaimable size + * threshold, which triggers MEMCG_LRU_YOUNG; * 5. Attempting to reclaim a memcg below min, which triggers MEMCG_LRU_YOUNG; * 6. Finishing the aging on the eviction path, which triggers MEMCG_LRU_YOUNG; * 7. Offlining a memcg, which triggers MEMCG_LRU_OLD. diff --git a/mm/vmscan.c b/mm/vmscan.c index cac38e9cac86..dad4b80b04cd 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4626,7 +4626,12 @@ static bool should_run_aging(struct lruvec *lruvec, unsigned long max_seq, } /* try to scrape all its memory if this memcg was deleted */ - *nr_to_scan = mem_cgroup_online(memcg) ? (total >> sc->priority) : total; + if (!mem_cgroup_online(memcg)) { + *nr_to_scan = total; + return false; + } + + *nr_to_scan = total >> sc->priority; /* * The aging tries to be lazy to reduce the overhead, while the eviction @@ -4747,14 +4752,9 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc) bool success; unsigned long scanned = sc->nr_scanned; unsigned long reclaimed = sc->nr_reclaimed; - int seg = lru_gen_memcg_seg(lruvec); struct mem_cgroup *memcg = lruvec_memcg(lruvec); struct pglist_data *pgdat = lruvec_pgdat(lruvec); - /* see the comment on MEMCG_NR_GENS */ - if (!lruvec_is_sizable(lruvec, sc)) - return seg != MEMCG_LRU_TAIL ? MEMCG_LRU_TAIL : MEMCG_LRU_YOUNG; - mem_cgroup_calculate_protection(NULL, memcg); if (mem_cgroup_below_min(NULL, memcg)) @@ -4762,7 +4762,7 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc) if (mem_cgroup_below_low(NULL, memcg)) { /* see the comment on MEMCG_NR_GENS */ - if (seg != MEMCG_LRU_TAIL) + if (lru_gen_memcg_seg(lruvec) != MEMCG_LRU_TAIL) return MEMCG_LRU_TAIL; memcg_memory_event(memcg, MEMCG_LOW); @@ -4778,7 +4778,15 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc) flush_reclaim_state(sc); - return success ? MEMCG_LRU_YOUNG : 0; + if (success && mem_cgroup_online(memcg)) + return MEMCG_LRU_YOUNG; + + if (!success && lruvec_is_sizable(lruvec, sc)) + return 0; + + /* one retry if offlined or too small */ + return lru_gen_memcg_seg(lruvec) != MEMCG_LRU_TAIL ? + MEMCG_LRU_TAIL : MEMCG_LRU_YOUNG; } #ifdef CONFIG_MEMCG