From patchwork Mon Mar 16 10:39:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 11440117 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EBD2192A for ; Mon, 16 Mar 2020 10:40:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AA6802071C for ; Mon, 16 Mar 2020 10:40:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DS6qyJAV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AA6802071C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E645E6B0005; Mon, 16 Mar 2020 06:40:31 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E3C916B0007; Mon, 16 Mar 2020 06:40:31 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2BA66B0008; Mon, 16 Mar 2020 06:40:31 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0098.hostedemail.com [216.40.44.98]) by kanga.kvack.org (Postfix) with ESMTP id BAA026B0005 for ; Mon, 16 Mar 2020 06:40:31 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 8E00F180AD81A for ; Mon, 16 Mar 2020 10:40:31 +0000 (UTC) X-FDA: 76600881462.13.ants30_21a7a9793a826 X-Spam-Summary: 2,0,0,c98844b5263147b1,d41d8cd98f00b204,laoar.shao@gmail.com,,RULES_HIT:2:41:69:355:379:541:800:960:973:988:989:1260:1345:1359:1437:1535:1605:1730:1747:1777:1792:2194:2198:2199:2200:2393:2553:2559:2562:2693:2897:2898:3138:3139:3140:3141:3142:3865:3867:3868:3870:3871:3872:3874:4049:4120:4250:4321:4605:5007:6261:6653:7514:9413:9592:10004:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12683:12895:13161:13172:13229:13255:14394:14687:21080:21444:21451:21627:21666:21966:21990:30054:30070:30090,0,RBL:209.85.215.196:@gmail.com:.lbl8.mailshell.net-66.100.201.100 62.50.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: ants30_21a7a9793a826 X-Filterd-Recvd-Size: 9147 Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Mon, 16 Mar 2020 10:40:31 +0000 (UTC) Received: by mail-pg1-f196.google.com with SMTP id t24so9519080pgj.7 for ; Mon, 16 Mar 2020 03:40:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=2GXt5l0BnU2RaVncy40HZUiqISCrmfLRyzDrfxLSYfo=; b=DS6qyJAVsjhyGFLKknVXpXrpm8SXdExRu5smq/INJvlVQNIksj/Z/DLTRGiDCGpf1g CJDalo4ezR0dkUepeQbBTC6a+nWdpToG3zBO+Cw4NpW38yZ7UNAAx/yOVopse2BmkcTR 0ag3iz6UBPaArUSqXLv6c4I50C65K1jEUDXZFBkjRiZMmaf6tHafTYptAhenll4rwRR+ vD86Fumx1e1+kMiqv0bUHeeS1hf8OgTtKSzwQ5VpkKvOmxsbQCzRV//YZObofz82x2k7 a1ItwC4dOxG4+53n6ZEmiv86CW2ZtPOE3lBWbIQ3iqNvXw+uz4yVnVVMLbPCz+OjLZrA WXJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=2GXt5l0BnU2RaVncy40HZUiqISCrmfLRyzDrfxLSYfo=; b=OCfH6KkNOyapYf0YO6aquYXOzz5Riy6XPN7JRItgZFlKbAA19VUYaWXzOo+WYG5MdU nK1ZultgEiAx90oI5tKY5lEMyvRKM4+xZcs/NLj6qHfY4pMlK+TRKDnDID7e0Q4EwSzz hzaHC71PeoLW28YAMZTDSvlV8imqtuuGASb17itLNFfzWzg/v84YVyv8VOZBKRhctJ/2 gn57OouuMIHV3VtYWlZKatUJwqZvXT0biLbLLEAF/bFGMeFS+9bJ8zfxRmjpM+MHVFIO TonaNxuh3OmmvdwSLVh758+xRRoe5uIhiHsWMSYwLspP9HOP2CSvUEWQSYwJWevKfScj jGvw== X-Gm-Message-State: ANhLgQ1g1IGgkLeMT2ETb1NWoeuVXNSizbOVkoFGbobp8UxIQKp8dmFp KhjEzZIWAtA5UKmbNLBEAu0= X-Google-Smtp-Source: ADFU+vsDFwodClKlUJFRep/E7/L5Oz3gg/x0Pio8rVsxenPZ+WUNteouZKf+PQdwdm5z0p4Mb76Hdg== X-Received: by 2002:a62:1dd3:: with SMTP id d202mr5005931pfd.47.1584355229901; Mon, 16 Mar 2020 03:40:29 -0700 (PDT) Received: from dev.localdomain ([203.100.54.194]) by smtp.gmail.com with ESMTPSA id h2sm19834276pjc.7.2020.03.16.03.40.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 16 Mar 2020 03:40:29 -0700 (PDT) From: Yafang Shao To: dchinner@redhat.com, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com, akpm@linux-foundation.org, viro@zeniv.linux.org.uk, willy@infradead.org Cc: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Yafang Shao Subject: [PATCH v6 1/3] mm, list_lru: make memcg visible to lru walker isolation function Date: Mon, 16 Mar 2020 06:39:56 -0400 Message-Id: <1584355198-10137-2-git-send-email-laoar.shao@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1584355198-10137-1-git-send-email-laoar.shao@gmail.com> References: <1584355198-10137-1-git-send-email-laoar.shao@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The lru walker isolation function may use this memcg to do something, e.g. the inode isolatation function will use the memcg to do inode protection in followup patch. So make memcg visible to the lru walker isolation function. Something should be emphasized in this patch is it replaces for_each_memcg_cache_index() with for_each_mem_cgroup() in list_lru_walk_node(). Because there's a gap between these two MACROs that for_each_mem_cgroup() depends on CONFIG_MEMCG while the other one depends on CONFIG_MEMCG_KMEM. But as list_lru_memcg_aware() returns false if CONFIG_MEMCG_KMEM is not configured, it is safe to this replacement. Another difference between for_each_memcg_cache_index() and for_each_mem_cgroup() is that for_each_memcg_cache_index() excludes the root_mem_cgroup because its kmemcg_id is -1, while for_each_mem_cgroup() includes the root_mem_cgroup. So we need to skip the root_mem_cgroup explicitly in the for loop. Cc: Matthew Wilcox Cc: Dave Chinner Signed-off-by: Yafang Shao --- include/linux/memcontrol.h | 18 ++++++++++++++++++ mm/list_lru.c | 47 +++++++++++++++++++++++++++------------------- mm/memcontrol.c | 15 --------------- 3 files changed, 46 insertions(+), 34 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index a7a0a1a5..39be2e4 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -445,6 +445,21 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *, int mem_cgroup_scan_tasks(struct mem_cgroup *, int (*)(struct task_struct *, void *), void *); +/* + * Iteration constructs for visiting all cgroups (under a tree). If + * loops are exited prematurely (break), mem_cgroup_iter_break() must + * be used for reference counting. + */ +#define for_each_mem_cgroup_tree(iter, root) \ + for (iter = mem_cgroup_iter(root, NULL, NULL); \ + iter != NULL; \ + iter = mem_cgroup_iter(root, iter, NULL)) + +#define for_each_mem_cgroup(iter) \ + for (iter = mem_cgroup_iter(NULL, NULL, NULL); \ + iter != NULL; \ + iter = mem_cgroup_iter(NULL, iter, NULL)) + static inline unsigned short mem_cgroup_id(struct mem_cgroup *memcg) { if (mem_cgroup_disabled()) @@ -945,6 +960,9 @@ static inline int mem_cgroup_scan_tasks(struct mem_cgroup *memcg, return 0; } +#define for_each_mem_cgroup(iter) \ + for (iter = NULL; iter; ) + static inline unsigned short mem_cgroup_id(struct mem_cgroup *memcg) { return 0; diff --git a/mm/list_lru.c b/mm/list_lru.c index 0f1f6b0..6daa8c6 100644 --- a/mm/list_lru.c +++ b/mm/list_lru.c @@ -207,11 +207,11 @@ unsigned long list_lru_count_node(struct list_lru *lru, int nid) EXPORT_SYMBOL_GPL(list_lru_count_node); static unsigned long -__list_lru_walk_one(struct list_lru_node *nlru, int memcg_idx, +__list_lru_walk_one(struct list_lru_node *nlru, struct mem_cgroup *memcg, list_lru_walk_cb isolate, void *cb_arg, unsigned long *nr_to_walk) { - + int memcg_idx = memcg_cache_id(memcg); struct list_lru_one *l; struct list_head *item, *n; unsigned long isolated = 0; @@ -273,7 +273,7 @@ unsigned long list_lru_count_node(struct list_lru *lru, int nid) unsigned long ret; spin_lock(&nlru->lock); - ret = __list_lru_walk_one(nlru, memcg_cache_id(memcg), isolate, cb_arg, + ret = __list_lru_walk_one(nlru, memcg, isolate, cb_arg, nr_to_walk); spin_unlock(&nlru->lock); return ret; @@ -289,7 +289,7 @@ unsigned long list_lru_count_node(struct list_lru *lru, int nid) unsigned long ret; spin_lock_irq(&nlru->lock); - ret = __list_lru_walk_one(nlru, memcg_cache_id(memcg), isolate, cb_arg, + ret = __list_lru_walk_one(nlru, memcg, isolate, cb_arg, nr_to_walk); spin_unlock_irq(&nlru->lock); return ret; @@ -299,25 +299,34 @@ unsigned long list_lru_walk_node(struct list_lru *lru, int nid, list_lru_walk_cb isolate, void *cb_arg, unsigned long *nr_to_walk) { - long isolated = 0; - int memcg_idx; + struct list_lru_node *nlru; + struct mem_cgroup *memcg; + long isolated; - isolated += list_lru_walk_one(lru, nid, NULL, isolate, cb_arg, - nr_to_walk); - if (*nr_to_walk > 0 && list_lru_memcg_aware(lru)) { - for_each_memcg_cache_index(memcg_idx) { - struct list_lru_node *nlru = &lru->node[nid]; + /* iterate the global lru first */ + isolated = list_lru_walk_one(lru, nid, NULL, isolate, cb_arg, + nr_to_walk); - spin_lock(&nlru->lock); - isolated += __list_lru_walk_one(nlru, memcg_idx, - isolate, cb_arg, - nr_to_walk); - spin_unlock(&nlru->lock); + if (!list_lru_memcg_aware(lru)) + goto out; - if (*nr_to_walk <= 0) - break; - } + nlru = &lru->node[nid]; + for_each_mem_cgroup(memcg) { + /* already scanned the root memcg above */ + if (mem_cgroup_is_root(memcg)) + continue; + + if (*nr_to_walk <= 0) + break; + + spin_lock(&nlru->lock); + isolated += __list_lru_walk_one(nlru, memcg, + isolate, cb_arg, + nr_to_walk); + spin_unlock(&nlru->lock); } + +out: return isolated; } EXPORT_SYMBOL_GPL(list_lru_walk_node); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 2058b8d..587c6b2 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -222,21 +222,6 @@ enum res_type { /* Used for OOM nofiier */ #define OOM_CONTROL (0) -/* - * Iteration constructs for visiting all cgroups (under a tree). If - * loops are exited prematurely (break), mem_cgroup_iter_break() must - * be used for reference counting. - */ -#define for_each_mem_cgroup_tree(iter, root) \ - for (iter = mem_cgroup_iter(root, NULL, NULL); \ - iter != NULL; \ - iter = mem_cgroup_iter(root, iter, NULL)) - -#define for_each_mem_cgroup(iter) \ - for (iter = mem_cgroup_iter(NULL, NULL, NULL); \ - iter != NULL; \ - iter = mem_cgroup_iter(NULL, iter, NULL)) - static inline bool should_force_charge(void) { return tsk_is_oom_victim(current) || fatal_signal_pending(current) ||