From patchwork Wed Jan 8 16:03:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 11323987 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 28186109A for ; Wed, 8 Jan 2020 16:04:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DB6652070E for ; Wed, 8 Jan 2020 16:04:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jHFsoUv+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DB6652070E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1B54A8E0007; Wed, 8 Jan 2020 11:04:19 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 164238E0001; Wed, 8 Jan 2020 11:04:19 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 004F58E0007; Wed, 8 Jan 2020 11:04:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0087.hostedemail.com [216.40.44.87]) by kanga.kvack.org (Postfix) with ESMTP id DF76E8E0001 for ; Wed, 8 Jan 2020 11:04:18 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 57388249D for ; Wed, 8 Jan 2020 16:04:18 +0000 (UTC) X-FDA: 76354938996.20.can15_429d4102a7515 X-Spam-Summary: 2,0,0,9cb1aae99c32953e,d41d8cd98f00b204,laoar.shao@gmail.com,:dchinner@redhat.com:hannes@cmpxchg.org:mhocko@kernel.org:vdavydov.dev@gmail.com:guro@fb.com:akpm@linux-foundation.org:viro@zeniv.linux.org.uk::linux-fsdevel@vger.kernel.org:laoar.shao@gmail.com,RULES_HIT:2:41:69:355:379:541:800:960:973:988:989:1260:1345:1359:1437:1535:1605:1730:1747:1777:1792:2194:2198:2199:2200:2393:2553:2559:2562:2693:2897:2898:3138:3139:3140:3141:3142:3865:3867:3868:3870:3871:3872:3874:4049:4120:4250:4321:4605:5007:6261:6653:7514:9413:9592:10004:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12683:12895:13161:13172:13229:13255:14394:14687:21080:21444:21451:21627:21666:21966:21990:30054:30070:30090,0,RBL:209.85.215.195:@gmail.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:38,LUA_SUMMARY:none X-HE-Tag: can15_429d4102a7515 X-Filterd-Recvd-Size: 9161 Received: from mail-pg1-f195.google.com (mail-pg1-f195.google.com [209.85.215.195]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Wed, 8 Jan 2020 16:04:17 +0000 (UTC) Received: by mail-pg1-f195.google.com with SMTP id l24so1807377pgk.2 for ; Wed, 08 Jan 2020 08:04:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=E+W6SlExR6PKwRnyZCifqOOllR7j0S8vS7QHNpyNirE=; b=jHFsoUv+fk7tRl48S/e6aBzrLKh2yNcQQgtLnKmG9XevPsfsPb48fW5yCEDc706k2y wKoqvBY1TG7vZDte52ZxqnML6yVbJmjFtwbWT8bGQmWzn2CpHkFhqywLIB8Gt1HTIFyL 6EPh72jl4eDFJUUl5mIewIJozdWcP/8AbjkJ0oGAEpUeVLXX71H15J0t0FkHIAy/tX/7 ziFFwGD0r6XzGpuduRS2SQ0fZYlSHQPaXmkZ9jebN6kBjO0ycc6dzYvzZbLYXNPCln4l LcZedYpV//pE9YMcfoqBnn2bISg1I8jr604D7b1GEsvT9FDtchax9jykRBaVPLBv6Gli a0IA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=E+W6SlExR6PKwRnyZCifqOOllR7j0S8vS7QHNpyNirE=; b=SKjeCvKiQyil189db1u2gXAhW7/dyVn8uUGE/5Vo33WfahWwvergHHUZxqJUndMkeu O1+tGd5hiNamItiXq15Nsi1Z/N6k7ULdJxsPeGe9LHOQiZdMrVVSs2fSjjwpGxUZs5dX f3YuPRDO/sa8Y+N+ouqBVN8IVBleRz/0T4/iBwbrnR2tFTat6b3ub114yUSJwEFyJUHz rlQaaFbBZXfKbQB8caGAKf8CurRnwcDOnB+3KG61edyeQWw5Ulii0tk8EjzqzGq17KPU NTLr+tQCtOo5SHVwhVlZlidZapbQV1P/iDlZTteL8Wb9+IEqRxkfTP3SWYMMJz5/Akkg 5pGA== X-Gm-Message-State: APjAAAWw1o2uF/+Lu4daS9nmJ0cSrhS/Aryi5f3aJRjS+e9dm/EsGXSQ HJ/QwdmV03qk6lqTJmv19w0= X-Google-Smtp-Source: APXvYqwCNKT0hQU2RWOH39HQnV30TOQjHn3K4hasN6PpsgAwO+KDKmGgw5JxPLmGIh7UdYu5S1epqQ== X-Received: by 2002:a63:5a64:: with SMTP id k36mr6064508pgm.323.1578499456356; Wed, 08 Jan 2020 08:04:16 -0800 (PST) Received: from dev.localdomain ([203.100.54.194]) by smtp.gmail.com with ESMTPSA id d22sm4079894pfo.187.2020.01.08.08.04.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Jan 2020 08:04:15 -0800 (PST) From: Yafang Shao To: dchinner@redhat.com, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com, akpm@linux-foundation.org, viro@zeniv.linux.org.uk Cc: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Yafang Shao Subject: [PATCH v3 1/3] mm, list_lru: make memcg visible to lru walker isolation function Date: Wed, 8 Jan 2020 11:03:55 -0500 Message-Id: <1578499437-1664-2-git-send-email-laoar.shao@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1578499437-1664-1-git-send-email-laoar.shao@gmail.com> References: <1578499437-1664-1-git-send-email-laoar.shao@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The lru walker isolation function may use this memcg to do something, e.g. the inode isolatation function will use the memcg to do inode protection in followup patch. So make memcg visible to the lru walker isolation function. Something should be emphasized in this patch is it replaces for_each_memcg_cache_index() with for_each_mem_cgroup() in list_lru_walk_node(). Because there's a gap between these two MACROs that for_each_mem_cgroup() depends on CONFIG_MEMCG while the other one depends on CONFIG_MEMCG_KMEM. But as list_lru_memcg_aware() returns false if CONFIG_MEMCG_KMEM is not configured, it is safe to this replacement. Another difference between for_each_memcg_cache_index() and for_each_mem_cgroup() is that for_each_memcg_cache_index() excludes the root_mem_cgroup because its kmemcg_id is -1, while for_each_mem_cgroup() includes the root_mem_cgroup. So we need to skip the root_mem_cgroup explicitly in the for loop. Cc: Dave Chinner Signed-off-by: Yafang Shao --- include/linux/memcontrol.h | 21 +++++++++++++++++++++ mm/list_lru.c | 47 +++++++++++++++++++++++++++------------------- mm/memcontrol.c | 15 --------------- 3 files changed, 49 insertions(+), 34 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index a7a0a1a5..a624c42 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -445,6 +445,21 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *, int mem_cgroup_scan_tasks(struct mem_cgroup *, int (*)(struct task_struct *, void *), void *); +/* + * Iteration constructs for visiting all cgroups (under a tree). If + * loops are exited prematurely (break), mem_cgroup_iter_break() must + * be used for reference counting. + */ +#define for_each_mem_cgroup_tree(iter, root) \ + for (iter = mem_cgroup_iter(root, NULL, NULL); \ + iter != NULL; \ + iter = mem_cgroup_iter(root, iter, NULL)) + +#define for_each_mem_cgroup(iter) \ + for (iter = mem_cgroup_iter(NULL, NULL, NULL); \ + iter != NULL; \ + iter = mem_cgroup_iter(NULL, iter, NULL)) + static inline unsigned short mem_cgroup_id(struct mem_cgroup *memcg) { if (mem_cgroup_disabled()) @@ -945,6 +960,12 @@ static inline int mem_cgroup_scan_tasks(struct mem_cgroup *memcg, return 0; } +#define for_each_mem_cgroup_tree(iter) \ + for (iter = NULL; iter; ) + +#define for_each_mem_cgroup(iter) \ + for (iter = NULL; iter; ) + static inline unsigned short mem_cgroup_id(struct mem_cgroup *memcg) { return 0; diff --git a/mm/list_lru.c b/mm/list_lru.c index 0f1f6b0..6daa8c6 100644 --- a/mm/list_lru.c +++ b/mm/list_lru.c @@ -207,11 +207,11 @@ unsigned long list_lru_count_node(struct list_lru *lru, int nid) EXPORT_SYMBOL_GPL(list_lru_count_node); static unsigned long -__list_lru_walk_one(struct list_lru_node *nlru, int memcg_idx, +__list_lru_walk_one(struct list_lru_node *nlru, struct mem_cgroup *memcg, list_lru_walk_cb isolate, void *cb_arg, unsigned long *nr_to_walk) { - + int memcg_idx = memcg_cache_id(memcg); struct list_lru_one *l; struct list_head *item, *n; unsigned long isolated = 0; @@ -273,7 +273,7 @@ unsigned long list_lru_count_node(struct list_lru *lru, int nid) unsigned long ret; spin_lock(&nlru->lock); - ret = __list_lru_walk_one(nlru, memcg_cache_id(memcg), isolate, cb_arg, + ret = __list_lru_walk_one(nlru, memcg, isolate, cb_arg, nr_to_walk); spin_unlock(&nlru->lock); return ret; @@ -289,7 +289,7 @@ unsigned long list_lru_count_node(struct list_lru *lru, int nid) unsigned long ret; spin_lock_irq(&nlru->lock); - ret = __list_lru_walk_one(nlru, memcg_cache_id(memcg), isolate, cb_arg, + ret = __list_lru_walk_one(nlru, memcg, isolate, cb_arg, nr_to_walk); spin_unlock_irq(&nlru->lock); return ret; @@ -299,25 +299,34 @@ unsigned long list_lru_walk_node(struct list_lru *lru, int nid, list_lru_walk_cb isolate, void *cb_arg, unsigned long *nr_to_walk) { - long isolated = 0; - int memcg_idx; + struct list_lru_node *nlru; + struct mem_cgroup *memcg; + long isolated; - isolated += list_lru_walk_one(lru, nid, NULL, isolate, cb_arg, - nr_to_walk); - if (*nr_to_walk > 0 && list_lru_memcg_aware(lru)) { - for_each_memcg_cache_index(memcg_idx) { - struct list_lru_node *nlru = &lru->node[nid]; + /* iterate the global lru first */ + isolated = list_lru_walk_one(lru, nid, NULL, isolate, cb_arg, + nr_to_walk); - spin_lock(&nlru->lock); - isolated += __list_lru_walk_one(nlru, memcg_idx, - isolate, cb_arg, - nr_to_walk); - spin_unlock(&nlru->lock); + if (!list_lru_memcg_aware(lru)) + goto out; - if (*nr_to_walk <= 0) - break; - } + nlru = &lru->node[nid]; + for_each_mem_cgroup(memcg) { + /* already scanned the root memcg above */ + if (mem_cgroup_is_root(memcg)) + continue; + + if (*nr_to_walk <= 0) + break; + + spin_lock(&nlru->lock); + isolated += __list_lru_walk_one(nlru, memcg, + isolate, cb_arg, + nr_to_walk); + spin_unlock(&nlru->lock); } + +out: return isolated; } EXPORT_SYMBOL_GPL(list_lru_walk_node); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 601405b..9bd4ea7 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -222,21 +222,6 @@ enum res_type { /* Used for OOM nofiier */ #define OOM_CONTROL (0) -/* - * Iteration constructs for visiting all cgroups (under a tree). If - * loops are exited prematurely (break), mem_cgroup_iter_break() must - * be used for reference counting. - */ -#define for_each_mem_cgroup_tree(iter, root) \ - for (iter = mem_cgroup_iter(root, NULL, NULL); \ - iter != NULL; \ - iter = mem_cgroup_iter(root, iter, NULL)) - -#define for_each_mem_cgroup(iter) \ - for (iter = mem_cgroup_iter(NULL, NULL, NULL); \ - iter != NULL; \ - iter = mem_cgroup_iter(NULL, iter, NULL)) - static inline bool should_force_charge(void) { return tsk_is_oom_victim(current) || fatal_signal_pending(current) || From patchwork Wed Jan 8 16:03:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 11323991 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BD675138C for ; Wed, 8 Jan 2020 16:04:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7C73A2070E for ; Wed, 8 Jan 2020 16:04:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="B5fOQPY0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7C73A2070E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 97D0C8E0009; Wed, 8 Jan 2020 11:04:24 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9077A8E0001; Wed, 8 Jan 2020 11:04:24 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A59B8E0009; Wed, 8 Jan 2020 11:04:24 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0195.hostedemail.com [216.40.44.195]) by kanga.kvack.org (Postfix) with ESMTP id 6562E8E0001 for ; Wed, 8 Jan 2020 11:04:24 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 12AB3180388DC for ; Wed, 8 Jan 2020 16:04:24 +0000 (UTC) X-FDA: 76354939248.04.time84_436d59b9ca417 X-Spam-Summary: 2,0,0,920fdfa6be744051,d41d8cd98f00b204,laoar.shao@gmail.com,:dchinner@redhat.com:hannes@cmpxchg.org:mhocko@kernel.org:vdavydov.dev@gmail.com:guro@fb.com:akpm@linux-foundation.org:viro@zeniv.linux.org.uk::linux-fsdevel@vger.kernel.org:laoar.shao@gmail.com,RULES_HIT:41:69:355:379:541:800:960:966:973:988:989:1260:1345:1359:1437:1535:1543:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2559:2562:2693:2731:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:4117:4321:4385:4605:5007:6261:6653:7514:7550:7875:7903:8660:9036:9413:10004:11026:11658:11914:12048:12294:12296:12297:12517:12519:12555:12895:12986:13148:13161:13229:13230:13255:14096:14181:14394:14687:14721:14877:21080:21433:21444:21451:21627:21666:21972:21990:30004:30034:30054,0,RBL:209.85.215.193:@gmail.com:.lbl8.mailshell.net-62.50.0.100 66.100.201.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMAR Y:none X-HE-Tag: time84_436d59b9ca417 X-Filterd-Recvd-Size: 6553 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Wed, 8 Jan 2020 16:04:23 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id k3so1801714pgc.3 for ; Wed, 08 Jan 2020 08:04:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=6tDRHfnOHhoFqOdXrEqpx02xS4p7Z8x/5vQQY0aLm8A=; b=B5fOQPY05C9Lq+n9bt52buT2A7zkUL4sgv4b5/aECGwJd39DWMxRsy2c8cF/pkfLyU WfKb7TzULr/fJztTjK8x2Ui/8G47hn0Y5MfDn10NN0zXi7fXrY921d+aW4z5EsMaQGHl 1nMAkJPUXIe0WM2pBF51WSkdzUJC2fR20TYqFFckLRPZURcOZOlQVOctax8yGfmmAETM ZwsFdFiQR+5CVXOF9LYIPsDlrakNHc6JzoFI2UexDMfzdNQwGjCKls1cqjYlaKO2dpNa JkFTLjRci3FahIpEeUq0kC6qrHFqK2UwB786bPemUQg35E8ECpLqlhqSxw52rdXrclg9 5bdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=6tDRHfnOHhoFqOdXrEqpx02xS4p7Z8x/5vQQY0aLm8A=; b=OV42OUChwcGjnsAmIyUQv40TOqWm8L9Y+WUNMEtbDvl/ct4/Wd0cW9gBdHucPAsjg8 enfmC/CYwzHDn3cbOnvJj00Zpnoneriq4Ga+T9pFWIEOOyysKBswhgbdhGzPWjVqI0Qo omjeLHstDGGpUlYB0DrWrMjOOKsOc33yAS6QwzAbFfhg4fzEeq+T1Wfps8YxiShc1r9a AS/apDs4/vCdOkhiStDh7OXAmUM5+Ryxm8HcGLM9rQzQqrJV0ZEV22FMdJx7/d3F4nRK cbD7DPO6hvqVlpCYIFGbSbkuDuXs7Flm/fTqNwF2L2FNzJS1Wih13QZrbRgwdl7EsFpN LoEw== X-Gm-Message-State: APjAAAUGJBUsbfoQ8IL9T/Dwh1UsIgSLyXqDm/na0EPUPHKqhFd6A311 lT8M3ob1c3nbTHHd0KB8w1U= X-Google-Smtp-Source: APXvYqwfRgPZUp3MgB5cdkyxCriBpjrgtBHzNG9GXRLTGn33q60rXLqX9nFm9V64JP+A+KjED3q8EQ== X-Received: by 2002:a62:ee11:: with SMTP id e17mr5905285pfi.48.1578499462405; Wed, 08 Jan 2020 08:04:22 -0800 (PST) Received: from dev.localdomain ([203.100.54.194]) by smtp.gmail.com with ESMTPSA id d22sm4079894pfo.187.2020.01.08.08.04.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Jan 2020 08:04:21 -0800 (PST) From: Yafang Shao To: dchinner@redhat.com, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com, akpm@linux-foundation.org, viro@zeniv.linux.org.uk Cc: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Yafang Shao Subject: [PATCH v3 2/3] mm, shrinker: make memcg low reclaim visible to lru walker isolation function Date: Wed, 8 Jan 2020 11:03:56 -0500 Message-Id: <1578499437-1664-3-git-send-email-laoar.shao@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1578499437-1664-1-git-send-email-laoar.shao@gmail.com> References: <1578499437-1664-1-git-send-email-laoar.shao@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A new member memcg_low_reclaim is introduced in shrink_control struct, which is derived from scan_control struct, in order to tell the shrinker whether the reclaim session is under memcg low reclaim or not. The followup patch will use this new member. Cc: Dave Chinner Signed-off-by: Yafang Shao --- include/linux/shrinker.h | 3 +++ mm/vmscan.c | 27 ++++++++++++++++----------- 2 files changed, 19 insertions(+), 11 deletions(-) diff --git a/include/linux/shrinker.h b/include/linux/shrinker.h index 0f80123..dc42ae5 100644 --- a/include/linux/shrinker.h +++ b/include/linux/shrinker.h @@ -31,6 +31,9 @@ struct shrink_control { /* current memcg being shrunk (for memcg aware shrinkers) */ struct mem_cgroup *memcg; + + /* derived from struct scan_control */ + bool memcg_low_reclaim; }; #define SHRINK_STOP (~0UL) diff --git a/mm/vmscan.c b/mm/vmscan.c index 5a6445e..c97d005 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -639,10 +639,9 @@ static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, /** * shrink_slab - shrink slab caches - * @gfp_mask: allocation context - * @nid: node whose slab caches to target * @memcg: memory cgroup whose slab caches to target - * @priority: the reclaim priority + * @sc: scan_control struct for this reclaim session + * @nid: node whose slab caches to target * * Call the shrink functions to age shrinkable caches. * @@ -652,15 +651,18 @@ static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, * @memcg specifies the memory cgroup to target. Unaware shrinkers * are called only if it is the root cgroup. * - * @priority is sc->priority, we take the number of objects and >> by priority - * in order to get the scan target. + * @sc is the scan_control struct, we take the number of objects + * and >> by sc->priority in order to get the scan target. * * Returns the number of reclaimed slab objects. */ -static unsigned long shrink_slab(gfp_t gfp_mask, int nid, - struct mem_cgroup *memcg, - int priority) +static unsigned long shrink_slab(struct mem_cgroup *memcg, + struct scan_control *sc, + int nid) { + bool memcg_low_reclaim = sc->memcg_low_reclaim; + gfp_t gfp_mask = sc->gfp_mask; + int priority = sc->priority; unsigned long ret, freed = 0; struct shrinker *shrinker; @@ -682,6 +684,7 @@ static unsigned long shrink_slab(gfp_t gfp_mask, int nid, .gfp_mask = gfp_mask, .nid = nid, .memcg = memcg, + .memcg_low_reclaim = memcg_low_reclaim, }; ret = do_shrink_slab(&sc, shrinker, priority); @@ -708,6 +711,9 @@ static unsigned long shrink_slab(gfp_t gfp_mask, int nid, void drop_slab_node(int nid) { unsigned long freed; + struct scan_control sc = { + .gfp_mask = GFP_KERNEL, + }; do { struct mem_cgroup *memcg = NULL; @@ -715,7 +721,7 @@ void drop_slab_node(int nid) freed = 0; memcg = mem_cgroup_iter(NULL, NULL, NULL); do { - freed += shrink_slab(GFP_KERNEL, nid, memcg, 0); + freed += shrink_slab(memcg, &sc, nid); } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL)) != NULL); } while (freed > 10); } @@ -2684,8 +2690,7 @@ static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc) shrink_lruvec(lruvec, sc); - shrink_slab(sc->gfp_mask, pgdat->node_id, memcg, - sc->priority); + shrink_slab(memcg, sc, pgdat->node_id); /* Record the group's reclaim efficiency */ vmpressure(sc->gfp_mask, memcg, false, From patchwork Wed Jan 8 16:03:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 11324007 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9A809109A for ; Wed, 8 Jan 2020 16:06:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5A5EF2073A for ; Wed, 8 Jan 2020 16:06:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UFK59NqJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5A5EF2073A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8B6AF8E0005; Wed, 8 Jan 2020 11:06:13 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 867DA8E0001; Wed, 8 Jan 2020 11:06:13 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77DB68E0005; Wed, 8 Jan 2020 11:06:13 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0144.hostedemail.com [216.40.44.144]) by kanga.kvack.org (Postfix) with ESMTP id 5F6D78E0001 for ; Wed, 8 Jan 2020 11:06:13 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 0C84B8249980 for ; Wed, 8 Jan 2020 16:06:13 +0000 (UTC) X-FDA: 76354943826.06.C8E11ED Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id 5ABBA1005C4AC for ; Wed, 8 Jan 2020 16:04:29 +0000 (UTC) X-Spam-Summary: 2,0,0,e91e5983059305ff,d41d8cd98f00b204,laoar.shao@gmail.com,:dchinner@redhat.com:hannes@cmpxchg.org:mhocko@kernel.org:vdavydov.dev@gmail.com:guro@fb.com:akpm@linux-foundation.org:viro@zeniv.linux.org.uk::linux-fsdevel@vger.kernel.org:laoar.shao@gmail.com,RULES_HIT:2:41:355:379:541:800:960:965:966:973:988:989:1260:1345:1359:1437:1535:1605:1606:1730:1747:1777:1792:1801:2196:2198:2199:2200:2393:2553:2559:2562:2693:2897:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4119:4250:4385:4390:4395:4470:4605:5007:6261:6653:7514:7903:9121:9413:10004:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12683:12895:13161:13229:14096:14394:14687:21080:21444:21450:21451:21627:21666:21796:21990:30036:30054:30090,0,RBL:209.85.215.194:@gmail.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:non e X-HE-Tag: fog48_44343ac839731 X-Filterd-Recvd-Size: 8907 Received: from mail-pg1-f194.google.com (mail-pg1-f194.google.com [209.85.215.194]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Wed, 8 Jan 2020 16:04:28 +0000 (UTC) Received: by mail-pg1-f194.google.com with SMTP id b137so1795019pga.6 for ; Wed, 08 Jan 2020 08:04:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Mf24uDxaxrXyVR4iCc3eAN/jtu0xKe1ZH34Ft0DWACg=; b=UFK59NqJPTz8BoeQ77BfQ+g5LONu/dr6l0d0g52YSUrbiG6xRjko26oVDtEVsHPsD0 sLsL5G/iP4iOGBCIeED9AwBiw9ntPiCBdbdcF9mDcRgt0W8fUJ+1MGHknNRkRcyGnGOa zQhZWUf4Wk7/i0RParuvUd+yMRMpHZniPBCS4iRGlfp0RACYKB3W+dJ3BR1ITnuZncL6 YE8WEeeW4P2NX933EJbqhtiqYtUCxXCP6d7ZUEiCDx5TAJcdZaHvlrfQ6/pbtfrPdNJ4 FVY0UhBCq0sf6biYqyWxXcbdST4mQyfOA7ssO8xVkdkPujOo8D5/SQEkPG2woll6nrJL YGhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Mf24uDxaxrXyVR4iCc3eAN/jtu0xKe1ZH34Ft0DWACg=; b=ErrRGodZcj2Et46RlHJDi/XK2SnIg2vPcbDyO3YNxpAmkYr6yHdOCCczjGbFupyw9U j2WzN8pW0whjn9uiWKDPdYfrOC3z9rc6sh3RoXnGtS3sQ9/nZkI47q38872zNbcgCK6Z 7rGPXSlSA5kRUMn+sieV6y14yX1G2lAfk3A3X4RXPYXyllMPfUf5lY+elPwPa01P+a9j vHbxys2PoTCneTu1INJhq/o+BpHVM/1HgYEfVZrwF/MeVx8raZkNunXXoHQ2CNssn/sZ mzify8hl2PjzAVSLlgPTIqWyKYnRk6Dy/g1FecFjvcq91tsSgcYGNVA81xh0rCajywBP 2YuA== X-Gm-Message-State: APjAAAXsYm2R8k7jWsgShNiD4uX8A/DI/I1MMZ82KI3vpmGPLkV2xFH+ suh32CAnIb1F10uXmHiiBvE= X-Google-Smtp-Source: APXvYqzpzaMo2o2yXGVPMqVWd7zOMF6/eFu/cnrHQ2glwGA4Vc7bHzW2fKgoLF43tcgijlEb5spNiA== X-Received: by 2002:a62:cdcb:: with SMTP id o194mr5659462pfg.117.1578499467505; Wed, 08 Jan 2020 08:04:27 -0800 (PST) Received: from dev.localdomain ([203.100.54.194]) by smtp.gmail.com with ESMTPSA id d22sm4079894pfo.187.2020.01.08.08.04.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 08 Jan 2020 08:04:26 -0800 (PST) From: Yafang Shao To: dchinner@redhat.com, hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, guro@fb.com, akpm@linux-foundation.org, viro@zeniv.linux.org.uk Cc: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, Yafang Shao Subject: [PATCH v3 3/3] memcg, inode: protect page cache from freeing inode Date: Wed, 8 Jan 2020 11:03:57 -0500 Message-Id: <1578499437-1664-4-git-send-email-laoar.shao@gmail.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1578499437-1664-1-git-send-email-laoar.shao@gmail.com> References: <1578499437-1664-1-git-send-email-laoar.shao@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On my server there're some running MEMCGs protected by memory.{min, low}, but I found the usage of these MEMCGs abruptly became very small, which were far less than the protect limit. It confused me and finally I found that was because of inode stealing. Once an inode is freed, all its belonging page caches will be dropped as well, no matter how may page caches it has. So if we intend to protect the page caches in a memcg, we must protect their host (the inode) first. Otherwise the memcg protection can be easily bypassed with freeing inode, especially if there're big files in this memcg. Supposes we have a memcg, and the stat of this memcg is, memory.current = 1024M memory.min = 512M And in this memcg there's a inode with 800M page caches. Once this memcg is scanned by kswapd or other regular reclaimers, kswapd <<<< It can be either of the regular reclaimers. shrink_node_memcgs switch (mem_cgroup_protected()) <<<< Not protected case MEMCG_PROT_NONE: <<<< Will scan this memcg beak; shrink_lruvec() <<<< Reclaim the page caches shrink_slab() <<<< It may free this inode and drop all its page caches(800M). So we must protect the inode first if we want to protect page caches. The inherent mismatch between memcg and inode is a trouble. One inode can be shared by different MEMCGs, but it is a very rare case. If an inode is shared, its belonging page caches may be charged to different MEMCGs. Currently there's no perfect solution to fix this kind of issue, but the inode majority-writer ownership switching can help it more or less. Cc: Dave Chinner Signed-off-by: Yafang Shao --- fs/inode.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 75 insertions(+), 3 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index 2b0f511..80dddbc 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -54,6 +54,12 @@ * inode_hash_lock */ +struct inode_isolate_control { + struct list_head *freeable; + struct mem_cgroup *memcg; /* derived from shrink_control */ + bool memcg_low_reclaim; /* derived from scan_control */ +}; + static unsigned int i_hash_mask __read_mostly; static unsigned int i_hash_shift __read_mostly; static struct hlist_head *inode_hashtable __read_mostly; @@ -713,6 +719,61 @@ int invalidate_inodes(struct super_block *sb, bool kill_dirty) return busy; } +#ifdef CONFIG_MEMCG_KMEM +/* + * Once an inode is freed, all its belonging page caches will be dropped as + * well, even if there're lots of page caches. So if we intend to protect + * page caches in a memcg, we must protect their host(the inode) first. + * Otherwise the memcg protection can be easily bypassed with freeing inode, + * especially if there're big files in this memcg. + * Note that it may happen that the page caches are already charged to the + * memcg, but the inode hasn't been added to this memcg yet. In this case, + * this inode is not protected. + * The inherent mismatch between memcg and inode is a trouble. One inode + * can be shared by different MEMCGs, but it is a very rare case. If + * an inode is shared, its belonging page caches may be charged to + * different MEMCGs. Currently there's no perfect solution to fix this + * kind of issue, but the inode majority-writer ownership switching can + * help it more or less. + */ +static bool memcg_can_reclaim_inode(struct inode *inode, + struct inode_isolate_control *iic) +{ + unsigned long cgroup_size; + unsigned long protection; + struct mem_cgroup *memcg; + bool reclaimable = true; + + if (!inode->i_data.nrpages) + goto out; + + /* Excludes freeing inode via drop_caches */ + if (!current->reclaim_state) + goto out; + + memcg = iic->memcg; + if (!memcg || memcg == root_mem_cgroup) + goto out; + + protection = mem_cgroup_protection(memcg, iic->memcg_low_reclaim); + if (!protection) + goto out; + + cgroup_size = mem_cgroup_size(memcg); + if (inode->i_data.nrpages + protection >= cgroup_size) + reclaimable = false; + +out: + return reclaimable; +} +#else /* CONFIG_MEMCG_KMEM */ +static bool memcg_can_reclaim_inode(struct inode *inode, + struct inode_isolate_control *iic) +{ + return true; +} +#endif /* CONFIG_MEMCG_KMEM */ + /* * Isolate the inode from the LRU in preparation for freeing it. * @@ -731,8 +792,9 @@ int invalidate_inodes(struct super_block *sb, bool kill_dirty) static enum lru_status inode_lru_isolate(struct list_head *item, struct list_lru_one *lru, spinlock_t *lru_lock, void *arg) { - struct list_head *freeable = arg; - struct inode *inode = container_of(item, struct inode, i_lru); + struct inode_isolate_control *iic = arg; + struct list_head *freeable = iic->freeable; + struct inode *inode = container_of(item, struct inode, i_lru); /* * we are inverting the lru lock/inode->i_lock here, so use a trylock. @@ -741,6 +803,11 @@ static enum lru_status inode_lru_isolate(struct list_head *item, if (!spin_trylock(&inode->i_lock)) return LRU_SKIP; + if (!memcg_can_reclaim_inode(inode, iic)) { + spin_unlock(&inode->i_lock); + return LRU_ROTATE; + } + /* * Referenced or dirty inodes are still in use. Give them another pass * through the LRU as we canot reclaim them now. @@ -798,9 +865,14 @@ long prune_icache_sb(struct super_block *sb, struct shrink_control *sc) { LIST_HEAD(freeable); long freed; + struct inode_isolate_control iic = { + .freeable = &freeable, + .memcg = sc->memcg, + .memcg_low_reclaim = sc->memcg_low_reclaim, + }; freed = list_lru_shrink_walk(&sb->s_inode_lru, sc, - inode_lru_isolate, &freeable); + inode_lru_isolate, &iic); dispose_list(&freeable); return freed; }