From patchwork Mon Nov 27 19:36:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13470245 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B412DC07D5B for ; Mon, 27 Nov 2023 19:37:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E8DCA6B02B6; Mon, 27 Nov 2023 14:37:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D313C6B02BE; Mon, 27 Nov 2023 14:37:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9FC7B6B02B8; Mon, 27 Nov 2023 14:37:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 8967E6B02B6 for ; Mon, 27 Nov 2023 14:37:09 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 58986406E8 for ; Mon, 27 Nov 2023 19:37:09 +0000 (UTC) X-FDA: 81504742578.13.A91BACB Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf11.hostedemail.com (Postfix) with ESMTP id 7EFB44000B for ; Mon, 27 Nov 2023 19:37:07 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=mPNI9lsR; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701113827; a=rsa-sha256; cv=none; b=YrhdN8lYKaA9VadzOEBOOH5ef0pW1vmU+w+jC0vuDdBnhkM46jKkJ2C21fAAPS15Yz4w1i 0QERC7KaUQ3tmq2m70z+J8zBtfJ82fMhh0ci1r6yAGU7ynYUClopNW6bfiE3EXJ8cf6q/v DOePiYwqIoGs31Kwe/pRLraAGxaSvyA= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=mPNI9lsR; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.216.46 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701113827; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Yk3Ra7LxPSTcjlQasqmaE6aQE/pvNfDE/tLSZ7yZw8U=; b=MoFoM0j71Mu7pCQgTVpwARHCZ0hrtXqNIbG/PVBZ0dtV0SGnyU+otdbTxmNYfULaLUPA5+ 3xSeCVoJsEWjh3vqCLXvjuYM9dFd6cQylYiRR6m6K57gAPr2vyTQPA67Zu9sws4fa0wOOu mOKbnMdGNE63+Ki3ScT+bF6uvCE42NI= Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-2839c113cbcso3693850a91.3 for ; Mon, 27 Nov 2023 11:37:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701113826; x=1701718626; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Yk3Ra7LxPSTcjlQasqmaE6aQE/pvNfDE/tLSZ7yZw8U=; b=mPNI9lsRwjtFqCfUnrk+pW7RMev7kItGPEj0eh9ELLSzfQvAjXIW+jS5PjC5FARM66 eJycHqV5ey1ET2UK+a3oKsu4TJm3PzuXGWmVpylV/6WQasRqJtNlZy97fTGQvd5KQzC+ pQuLwmEN7HfZLraNDZW9ztOgQqQjwZOqpJMZ+h4H6ay/piPsRe85EmPpKpktTEJn9peE szphypO7vbtPlmJRTeufxyw9iyiuuE8qZeF13v6Hrn1hCl4Ik4KOO2lrkrBrHB5cOf7i 5c37SAWeejM1UcMgRycyao+e3PfdqZ4V7rdPc7i6JTkTC9Rea/qkn9AMXXW4+Z0pSI/Y yVXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701113826; x=1701718626; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Yk3Ra7LxPSTcjlQasqmaE6aQE/pvNfDE/tLSZ7yZw8U=; b=ouxUwJVPJp13bvFrHVD+5KpUmkZr/Hyy9ze8MsFW91Nd42VtrPgWXAf6H++Lp6xTph yDXgR6fE3Hpwf7RbxIrYEPAfVyIEOwe6B2ljM2GAmuxMgaNM6yjBTUTgFLXlDvTZblVu nPyV3bD1R710NKOXFHV7pYwRSoYONUK9UeEEpJs8koiQRRUWXSe0YQJxvEbzIcZrRqGg ufYJI68GbS+4Bv0XG7lQxV2COpQcDhoBDwf/0pXBd+wauBwP8jLOM8ea++8v/5gNYsQ3 Pg08biyuiaMzQFuYXMXNJQh02WG6ARtrcMFQa8leVLlN4elBz8l4Zh9565R0cQrxfGUk cN6g== X-Gm-Message-State: AOJu0YwUE/z9BZXnol2C9adOjeJ6GaaP7AeUFz3Q/HYbVMHiEi5k1K8P T90O2SOpdi7a6R2HqpniOz4= X-Google-Smtp-Source: AGHT+IESo6RUTEUw2quoScohVLKJzp+po5d5yitAfhgPXdhC6sxqbbGd5iWNr4rtvNf8otQ+V25FDw== X-Received: by 2002:a17:90b:3b92:b0:285:a72f:966f with SMTP id pc18-20020a17090b3b9200b00285a72f966fmr8454941pjb.46.1701113826185; Mon, 27 Nov 2023 11:37:06 -0800 (PST) Received: from localhost (fwdproxy-prn-026.fbsv.net. [2a03:2880:ff:1a::face:b00c]) by smtp.gmail.com with ESMTPSA id a21-20020a17090ad81500b002851e283c21sm7822831pjv.12.2023.11.27.11.37.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Nov 2023 11:37:05 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, cerasuolodomenico@gmail.com, yosryahmed@google.com, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, chrisl@kernel.org, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, shuah@kernel.org Subject: [PATCH v6 2/6] memcontrol: allows mem_cgroup_iter() to check for onlineness Date: Mon, 27 Nov 2023 11:36:59 -0800 Message-Id: <20231127193703.1980089-3-nphamcs@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231127193703.1980089-1-nphamcs@gmail.com> References: <20231127193703.1980089-1-nphamcs@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 7EFB44000B X-Stat-Signature: kgx6wqzct9ryoahqdzni7pfsfsqzdqqy X-HE-Tag: 1701113827-205626 X-HE-Meta: U2FsdGVkX1/BBcdSzokmxD2H/TefWLgdSe9ZFKHC4YEwDH7qRBEq23O+Ot0ZaNa5JoyV8+06UtHIaHqhP1drovjvQ7IHevs4NBrR6U45+TklJLFK9PPcwyhCmz4USCeRKAKi8zCm6RoIP7Fxks1a65SnMab9jbo6CXugowzyyEvPPqCmEvDX8D4vafMpZ6StcN+fG8j/mRBsfT3gfMUfqQs45yVV81E1oukANLAdyB4DY0VVo5eEGG+YyGrCCzmZroBvAztGm9g/QAnOW+tL8ksUQbsOCOL97C9IOW9AlsYFJ3fTykWw3o3ULsCI4F6uLCs+XeOKgNl7Jf+HWAT/Ibn9aHA7EIR5wGEEha0dRYBRk4AfqT9JXaXJDx104tZr6hk/LOV3ZXin6rIZ2gJCxEXmcM+KeTUB4l8bUusauqSNR9AKoGfh+1k4glvOoXPzrFoqm9zmiW8uk2JpWFjKP+1z4jde/ilQbQ4akqik9ttVnccRd854/IPFfT+NECdCwLCi4E0y5wwjpIBgXL3j4SpKWsRcSK12H+c6hS3G95OdZ7sBFC5GSfymWrnbPWKfm1SYoeK2t51EH9AtYbuXJSY5Y5D7MGwbkcSyzZIegF3kyo3BbRR13i1IepZgs3Qmis7nx+3dAeC82qSHbfaivp2pvciio28+nd60TUfhKuU8fYD/qO4kqJM2wX9HjrfIpq1pyTAxVAIIG9Y7N7RfQX6OQUocHQYgP3UWO15PwTSPZGFejVTM5bypEpONKoDrK+9ILpcbyerrkAYHmG0zkEmE+OUISYJ6lljFXjcwtvK7bPG8l3d9EVAV3PvzN3jVEpEr7/WGOncqG8AtNlZ6ShQOP3nfIRuhc+IwJ+Wg62P8hrpPRNwGPZXgLD5aO1jn6vxEtI0dzimDmDzgN+fFJtg6MbypypBpwVenJdnDSj7U9BgYhGg+TP65b5tBHnLpn+qpN+Dz75kqRpfFUhW 7+W/1gyb sMd2EC5O/pBUpRcMSxoR0TODmCh7NKFMCWHkwsmre8/BUE1stwBBWaCKg0ip1CmMAPTgDyqPVBnSLmCRKx/gOAoF4ZM+S116hY/0RYl2hMsB7ttXq2uuH187SJfIyCBFzVsM+Ks3NFl8FhH9ZhFwJkLS/PAq3EIRzcp0HEvUaf4GXpTJYPTl+GIu4BsLWAHJeceSY1BdYZzOLmlOavvnfUU9QDX8N1wE+WrhMFCO4JYcnQyjFFdzOT0+AepBglWk5eptFYpjts/EVMWpVgZX7ECnZtlMz1Lu0muZV7Gh9IGHVy0CvBZhas2yyYuQvuCbM3WzyosNXDjPugu5iX+dmRJSUb4pYGtTmIN+KiI+78Q9Bs3JGd3V9cOv6+4bW1apekreSh/9cfASVoeVWc1+wbYrTkSMRkTUyMH06gp46BlrbJEzBvv8G3nIrlBA1/NVIvjST X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The new zswap writeback scheme requires an online-only memcg hierarchy traversal. Add a new parameter to mem_cgroup_iter() to check for onlineness before returning. Signed-off-by: Nhat Pham --- include/linux/memcontrol.h | 4 ++-- mm/memcontrol.c | 17 ++++++++++------- mm/shrinker.c | 4 ++-- mm/vmscan.c | 26 +++++++++++++------------- 4 files changed, 27 insertions(+), 24 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 7bdcf3020d7a..86adce081a08 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -832,7 +832,7 @@ static inline void mem_cgroup_put(struct mem_cgroup *memcg) struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *, struct mem_cgroup *, - struct mem_cgroup_reclaim_cookie *); + struct mem_cgroup_reclaim_cookie *, bool online); void mem_cgroup_iter_break(struct mem_cgroup *, struct mem_cgroup *); void mem_cgroup_scan_tasks(struct mem_cgroup *memcg, int (*)(struct task_struct *, void *), void *arg); @@ -1381,7 +1381,7 @@ static inline struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio, static inline struct mem_cgroup * mem_cgroup_iter(struct mem_cgroup *root, struct mem_cgroup *prev, - struct mem_cgroup_reclaim_cookie *reclaim) + struct mem_cgroup_reclaim_cookie *reclaim, bool online) { return NULL; } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 564aa8f25b71..a1f051adaa15 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -221,14 +221,14 @@ enum res_type { * be used for reference counting. */ #define for_each_mem_cgroup_tree(iter, root) \ - for (iter = mem_cgroup_iter(root, NULL, NULL); \ + for (iter = mem_cgroup_iter(root, NULL, NULL, false); \ iter != NULL; \ - iter = mem_cgroup_iter(root, iter, NULL)) + iter = mem_cgroup_iter(root, iter, NULL, false)) #define for_each_mem_cgroup(iter) \ - for (iter = mem_cgroup_iter(NULL, NULL, NULL); \ + for (iter = mem_cgroup_iter(NULL, NULL, NULL, false); \ iter != NULL; \ - iter = mem_cgroup_iter(NULL, iter, NULL)) + iter = mem_cgroup_iter(NULL, iter, NULL, false)) static inline bool task_is_dying(void) { @@ -1115,6 +1115,7 @@ struct mem_cgroup *get_mem_cgroup_from_current(void) * @root: hierarchy root * @prev: previously returned memcg, NULL on first invocation * @reclaim: cookie for shared reclaim walks, NULL for full walks + * @online: skip offline memcgs * * Returns references to children of the hierarchy below @root, or * @root itself, or %NULL after a full round-trip. @@ -1129,7 +1130,8 @@ struct mem_cgroup *get_mem_cgroup_from_current(void) */ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, struct mem_cgroup *prev, - struct mem_cgroup_reclaim_cookie *reclaim) + struct mem_cgroup_reclaim_cookie *reclaim, + bool online) { struct mem_cgroup_reclaim_iter *iter; struct cgroup_subsys_state *css = NULL; @@ -1199,7 +1201,8 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, * is provided by the caller, so we know it's alive * and kicking, and don't take an extra reference. */ - if (css == &root->css || css_tryget(css)) { + if (css == &root->css || (!online && css_tryget(css)) || + css_tryget_online(css)) { memcg = mem_cgroup_from_css(css); break; } @@ -1812,7 +1815,7 @@ static int mem_cgroup_soft_reclaim(struct mem_cgroup *root_memcg, excess = soft_limit_excess(root_memcg); while (1) { - victim = mem_cgroup_iter(root_memcg, victim, &reclaim); + victim = mem_cgroup_iter(root_memcg, victim, &reclaim, false); if (!victim) { loop++; if (loop >= 2) { diff --git a/mm/shrinker.c b/mm/shrinker.c index dd91eab43ed3..54f5d3aa4f27 100644 --- a/mm/shrinker.c +++ b/mm/shrinker.c @@ -160,7 +160,7 @@ static int expand_shrinker_info(int new_id) new_size = shrinker_unit_size(new_nr_max); old_size = shrinker_unit_size(shrinker_nr_max); - memcg = mem_cgroup_iter(NULL, NULL, NULL); + memcg = mem_cgroup_iter(NULL, NULL, NULL, false); do { ret = expand_one_shrinker_info(memcg, new_size, old_size, new_nr_max); @@ -168,7 +168,7 @@ static int expand_shrinker_info(int new_id) mem_cgroup_iter_break(NULL, memcg); goto out; } - } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL)) != NULL); + } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL, false)) != NULL); out: if (!ret) shrinker_nr_max = new_nr_max; diff --git a/mm/vmscan.c b/mm/vmscan.c index d8c3338fee0f..9a65ee3a1bb7 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -397,10 +397,10 @@ static unsigned long drop_slab_node(int nid) unsigned long freed = 0; struct mem_cgroup *memcg = NULL; - memcg = mem_cgroup_iter(NULL, NULL, NULL); + memcg = mem_cgroup_iter(NULL, NULL, NULL, false); do { freed += shrink_slab(GFP_KERNEL, nid, memcg, 0); - } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL)) != NULL); + } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL, false)) != NULL); return freed; } @@ -3935,7 +3935,7 @@ static void lru_gen_age_node(struct pglist_data *pgdat, struct scan_control *sc) if (!min_ttl || sc->order || sc->priority == DEF_PRIORITY) return; - memcg = mem_cgroup_iter(NULL, NULL, NULL); + memcg = mem_cgroup_iter(NULL, NULL, NULL, false); do { struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); @@ -3945,7 +3945,7 @@ static void lru_gen_age_node(struct pglist_data *pgdat, struct scan_control *sc) } cond_resched(); - } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL))); + } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL, false))); /* * The main goal is to OOM kill if every generation from all memcgs is @@ -5037,7 +5037,7 @@ static void lru_gen_change_state(bool enabled) else static_branch_disable_cpuslocked(&lru_gen_caps[LRU_GEN_CORE]); - memcg = mem_cgroup_iter(NULL, NULL, NULL); + memcg = mem_cgroup_iter(NULL, NULL, NULL, false); do { int nid; @@ -5061,7 +5061,7 @@ static void lru_gen_change_state(bool enabled) } cond_resched(); - } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL))); + } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL, false))); unlock: mutex_unlock(&state_mutex); put_online_mems(); @@ -5164,7 +5164,7 @@ static void *lru_gen_seq_start(struct seq_file *m, loff_t *pos) if (!m->private) return ERR_PTR(-ENOMEM); - memcg = mem_cgroup_iter(NULL, NULL, NULL); + memcg = mem_cgroup_iter(NULL, NULL, NULL, false); do { int nid; @@ -5172,7 +5172,7 @@ static void *lru_gen_seq_start(struct seq_file *m, loff_t *pos) if (!nr_to_skip--) return get_lruvec(memcg, nid); } - } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL))); + } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL, false))); return NULL; } @@ -5195,7 +5195,7 @@ static void *lru_gen_seq_next(struct seq_file *m, void *v, loff_t *pos) nid = next_memory_node(nid); if (nid == MAX_NUMNODES) { - memcg = mem_cgroup_iter(NULL, memcg, NULL); + memcg = mem_cgroup_iter(NULL, memcg, NULL, false); if (!memcg) return NULL; @@ -5798,7 +5798,7 @@ static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc) struct mem_cgroup *target_memcg = sc->target_mem_cgroup; struct mem_cgroup *memcg; - memcg = mem_cgroup_iter(target_memcg, NULL, NULL); + memcg = mem_cgroup_iter(target_memcg, NULL, NULL, false); do { struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); unsigned long reclaimed; @@ -5855,7 +5855,7 @@ static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc) sc->nr_scanned - scanned, sc->nr_reclaimed - reclaimed); - } while ((memcg = mem_cgroup_iter(target_memcg, memcg, NULL))); + } while ((memcg = mem_cgroup_iter(target_memcg, memcg, NULL, false))); } static void shrink_node(pg_data_t *pgdat, struct scan_control *sc) @@ -6522,12 +6522,12 @@ static void kswapd_age_node(struct pglist_data *pgdat, struct scan_control *sc) if (!inactive_is_low(lruvec, LRU_INACTIVE_ANON)) return; - memcg = mem_cgroup_iter(NULL, NULL, NULL); + memcg = mem_cgroup_iter(NULL, NULL, NULL, false); do { lruvec = mem_cgroup_lruvec(memcg, pgdat); shrink_active_list(SWAP_CLUSTER_MAX, lruvec, sc, LRU_ACTIVE_ANON); - memcg = mem_cgroup_iter(NULL, memcg, NULL); + memcg = mem_cgroup_iter(NULL, memcg, NULL, false); } while (memcg); }