From patchwork Sat Apr 18 15:13:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 11496605 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C5FED6CA for ; Sat, 18 Apr 2020 15:13:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 85D0920776 for ; Sat, 18 Apr 2020 15:13:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="O5TCfJf7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 85D0920776 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AC8AE8E0006; Sat, 18 Apr 2020 11:13:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A793E8E0003; Sat, 18 Apr 2020 11:13:34 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 969018E0006; Sat, 18 Apr 2020 11:13:34 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0141.hostedemail.com [216.40.44.141]) by kanga.kvack.org (Postfix) with ESMTP id 80AF78E0003 for ; Sat, 18 Apr 2020 11:13:34 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 307B7824556B for ; Sat, 18 Apr 2020 15:13:34 +0000 (UTC) X-FDA: 76721319948.27.humor33_766f54cf2245e X-Spam-Summary: 2,0,0,53785edd0d57088d,d41d8cd98f00b204,laoar.shao@gmail.com,,RULES_HIT:2:41:69:355:379:541:800:960:966:973:988:989:1260:1345:1359:1437:1535:1605:1730:1747:1777:1792:2196:2198:2199:2200:2393:2559:2562:2731:3138:3139:3140:3141:3142:3865:3866:3867:3870:3871:3872:3874:4049:4120:4250:4321:4385:4605:5007:6119:6261:6653:7514:7903:9413:9592:11026:11232:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12683:12895:12986:13972:14096:14110:14394:14687:21080:21324:21444:21451:21611:21627:21666:21966:21990:30054:30070:30074,0,RBL:209.85.210.194:@gmail.com:.lbl8.mailshell.net-62.50.0.100 66.100.201.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:1:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: humor33_766f54cf2245e X-Filterd-Recvd-Size: 9018 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Sat, 18 Apr 2020 15:13:33 +0000 (UTC) Received: by mail-pf1-f194.google.com with SMTP id p25so2602771pfn.11 for ; Sat, 18 Apr 2020 08:13:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=fDbafwg9RkM9iVTg2J+7GHysyBGtm/+WKY/g4jULANI=; b=O5TCfJf7HOcoZ9jw6xSlAm0Meef+nuUzFltRn+U48inSG4LlR/xiQSXilsEUg2RHbZ 7ylxnbzC+TN/XSgY6z0sBU7MRgLB8Ad+/0a8fUfB6NSZTF/HZy0in3Wg3V3x1/l09xDd NMK4SCxCHYPedPyppHAF6U1KlJDFYXvMcoY9SxotvqOHJcEvQTSm0WhmRrTA1vvJu/A2 pacoq5OQkXvU4fNVoRwfW0PNDIYl/+hQxTORuktaa78TVk+Pkg+xY1jnskxGmkoBis7C tAtSpF8ZnRcjnjf2q9jbeGP83yl/lC+Xqfzc92+bdPNJ7NXVWlku/PcDnOT/HaFxtWUU gFbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=fDbafwg9RkM9iVTg2J+7GHysyBGtm/+WKY/g4jULANI=; b=eJQ+yjNVpNUAitDspewcCDMsoVpctAACWn2ZfNV6TuD2mi7E0neU2veUegL4fRNUvy IfjM2n15VbROaeJ/EC2l3jizQEWTHxdUsczDbFRyMy+20rK+aMCRrtSkj+ErFjhs8QiH Ae2MAJh/JKOAcX3oxuoCjt9cSx0eaMXzJxd+O30LLNY+d0kO9OS6BdNMvzqYdb9DS70w Nm5YDT8xRzgTdwv2Ew2UBUtVdZ42LrL8LLk43WD2I2eDXQOVku/OoKEndEmgU6F4dGXY F2tmLCiR+db0qKmpINL2d7VK7s6S6EzqYCGHJSlujTTYUHizZRd0D2c4ik1vuavNwWyP vEBg== X-Gm-Message-State: AGi0PuYpjQQ+tmPbTy3Iu0CCxFMcRYrVeu1Cs0WyymHkUM+IZeq83+yj 4jsxyj4f64sGjNmM7tfImbM= X-Google-Smtp-Source: APiQypKRc+pIJZrRFmbirBqrL21vS5pn2owvZ2MSdAUEugx9qaWRyVUyfh00il0BEBXCFlShTihnyQ== X-Received: by 2002:a63:5359:: with SMTP id t25mr7646092pgl.349.1587222812757; Sat, 18 Apr 2020 08:13:32 -0700 (PDT) Received: from localhost.localdomain ([203.100.54.194]) by smtp.gmail.com with ESMTPSA id j32sm9613028pgb.55.2020.04.18.08.13.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 18 Apr 2020 08:13:32 -0700 (PDT) From: Yafang Shao To: hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, Yafang Shao Subject: [PATCH 1/3] mm: change the return type of out_of_memory() Date: Sat, 18 Apr 2020 11:13:09 -0400 Message-Id: <20200418151311.7397-2-laoar.shao@gmail.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20200418151311.7397-1-laoar.shao@gmail.com> References: <20200418151311.7397-1-laoar.shao@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Change the return type of out_of_memory() from bool to enum oom_status for later use. The definition of enum oom_status is moved from mm/memcontrol.c to include/linux/oom.h. No functional change. Signed-off-by: Yafang Shao --- include/linux/memcontrol.h | 1 + include/linux/oom.h | 9 ++++++++- mm/memcontrol.c | 30 +++++++++++++----------------- mm/oom_kill.c | 14 +++++++------- mm/page_alloc.c | 3 ++- 5 files changed, 31 insertions(+), 26 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 1b4150ff64be..98bd8fb2f5c7 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -21,6 +21,7 @@ #include #include #include +#include struct mem_cgroup; struct page; diff --git a/include/linux/oom.h b/include/linux/oom.h index c696c265f019..3dca5ce189e6 100644 --- a/include/linux/oom.h +++ b/include/linux/oom.h @@ -22,6 +22,13 @@ enum oom_constraint { CONSTRAINT_MEMCG, }; +enum oom_status { + OOM_SUCCESS, + OOM_FAILED, + OOM_ASYNC, + OOM_SKIPPED +}; + /* * Details of the page allocation that triggered the oom killer that are used to * determine what should be killed. @@ -110,7 +117,7 @@ bool __oom_reap_task_mm(struct mm_struct *mm); extern unsigned long oom_badness(struct task_struct *p, unsigned long totalpages); -extern bool out_of_memory(struct oom_control *oc); +enum oom_status out_of_memory(struct oom_control *oc); extern void exit_oom_victim(void); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 5beea03dd58a..22418b55804f 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1570,8 +1570,9 @@ unsigned long mem_cgroup_size(struct mem_cgroup *memcg) return page_counter_read(&memcg->memory); } -static bool mem_cgroup_out_of_memory(struct mem_cgroup *memcg, gfp_t gfp_mask, - int order) +static enum oom_status mem_cgroup_out_of_memory(struct mem_cgroup *memcg, + gfp_t gfp_mask, + int order) { struct oom_control oc = { .zonelist = NULL, @@ -1580,16 +1581,20 @@ static bool mem_cgroup_out_of_memory(struct mem_cgroup *memcg, gfp_t gfp_mask, .gfp_mask = gfp_mask, .order = order, }; - bool ret; + enum oom_status ret; if (mutex_lock_killable(&oom_lock)) - return true; + return OOM_SUCCESS; /* * A few threads which were not waiting at mutex_lock_killable() can * fail to bail out. Therefore, check again after holding oom_lock. */ - ret = should_force_charge() || out_of_memory(&oc); + if (should_force_charge()) + return OOM_SUCCESS; + + ret = out_of_memory(&oc); mutex_unlock(&oom_lock); + return ret; } @@ -1767,13 +1772,6 @@ static void memcg_oom_recover(struct mem_cgroup *memcg) __wake_up(&memcg_oom_waitq, TASK_NORMAL, 0, memcg); } -enum oom_status { - OOM_SUCCESS, - OOM_FAILED, - OOM_ASYNC, - OOM_SKIPPED -}; - static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int order) { enum oom_status ret; @@ -1821,10 +1819,7 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int mem_cgroup_oom_notify(memcg); mem_cgroup_unmark_under_oom(memcg); - if (mem_cgroup_out_of_memory(memcg, mask, order)) - ret = OOM_SUCCESS; - else - ret = OOM_FAILED; + ret = mem_cgroup_out_of_memory(memcg, mask, order); if (locked) mem_cgroup_oom_unlock(memcg); @@ -6102,7 +6097,8 @@ static ssize_t memory_max_write(struct kernfs_open_file *of, } memcg_memory_event(memcg, MEMCG_OOM); - if (!mem_cgroup_out_of_memory(memcg, GFP_KERNEL, 0)) + if (mem_cgroup_out_of_memory(memcg, GFP_KERNEL, 0) != + OOM_SUCCESS) break; } diff --git a/mm/oom_kill.c b/mm/oom_kill.c index dfc357614e56..d5a941bea2d7 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -1042,18 +1042,18 @@ EXPORT_SYMBOL_GPL(unregister_oom_notifier); * OR try to be smart about which process to kill. Note that we * don't have to be perfect here, we just have to be good. */ -bool out_of_memory(struct oom_control *oc) +enum oom_status out_of_memory(struct oom_control *oc) { unsigned long freed = 0; if (oom_killer_disabled) - return false; + return OOM_FAILED; if (!is_memcg_oom(oc)) { blocking_notifier_call_chain(&oom_notify_list, 0, &freed); if (freed > 0) /* Got some memory back in the last second. */ - return true; + return OOM_SUCCESS; } /* @@ -1064,7 +1064,7 @@ bool out_of_memory(struct oom_control *oc) if (task_will_free_mem(current)) { mark_oom_victim(current); wake_oom_reaper(current); - return true; + return OOM_SUCCESS; } /* @@ -1075,7 +1075,7 @@ bool out_of_memory(struct oom_control *oc) * invoke the OOM killer even if it is a GFP_NOFS allocation. */ if (oc->gfp_mask && !(oc->gfp_mask & __GFP_FS) && !is_memcg_oom(oc)) - return true; + return OOM_SUCCESS; /* * Check if there were limitations on the allocation (only relevant for @@ -1093,7 +1093,7 @@ bool out_of_memory(struct oom_control *oc) get_task_struct(current); oc->chosen = current; oom_kill_process(oc, "Out of memory (oom_kill_allocating_task)"); - return true; + return OOM_SUCCESS; } select_bad_process(oc); @@ -1112,7 +1112,7 @@ bool out_of_memory(struct oom_control *oc) if (oc->chosen && oc->chosen != (void *)-1UL) oom_kill_process(oc, !is_memcg_oom(oc) ? "Out of memory" : "Memory cgroup out of memory"); - return !!oc->chosen; + return oc->chosen ? OOM_SUCCESS : OOM_FAILED; } /* diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 69827d4fa052..0926117eb921 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3917,7 +3917,8 @@ __alloc_pages_may_oom(gfp_t gfp_mask, unsigned int order, goto out; /* Exhausted what can be done so it's blame time */ - if (out_of_memory(&oc) || WARN_ON_ONCE(gfp_mask & __GFP_NOFAIL)) { + if (out_of_memory(&oc) == OOM_SUCCESS || + WARN_ON_ONCE(gfp_mask & __GFP_NOFAIL)) { *did_some_progress = 1; /* From patchwork Sat Apr 18 15:13:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 11496607 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E46816CA for ; Sat, 18 Apr 2020 15:13:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B19852076A for ; Sat, 18 Apr 2020 15:13:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="F19MROdL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B19852076A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E4C1F8E0007; Sat, 18 Apr 2020 11:13:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DFB9F8E0003; Sat, 18 Apr 2020 11:13:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D38908E0007; Sat, 18 Apr 2020 11:13:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0129.hostedemail.com [216.40.44.129]) by kanga.kvack.org (Postfix) with ESMTP id BDAAF8E0003 for ; Sat, 18 Apr 2020 11:13:39 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 69E34181AEF30 for ; Sat, 18 Apr 2020 15:13:39 +0000 (UTC) X-FDA: 76721320158.30.eggs86_7730b0a800341 X-Spam-Summary: 2,0,0,a9f4ab79c23ae4a2,d41d8cd98f00b204,laoar.shao@gmail.com,,RULES_HIT:41:69:355:379:541:800:960:973:988:989:1260:1345:1359:1437:1535:1542:1711:1730:1747:1777:1792:2198:2199:2393:2559:2562:2693:2731:3138:3139:3140:3141:3142:3353:3865:3867:3868:3870:3871:3872:3874:4321:4605:5007:6261:6653:7514:9413:9592:10004:11026:11658:11914:12043:12048:12114:12291:12296:12297:12438:12517:12519:12555:12895:12986:14096:14181:14394:14687:14721:21080:21324:21444:21451:21627:21666:21990:30054,0,RBL:209.85.210.194:@gmail.com:.lbl8.mailshell.net-62.50.0.100 66.100.201.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: eggs86_7730b0a800341 X-Filterd-Recvd-Size: 5255 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Sat, 18 Apr 2020 15:13:38 +0000 (UTC) Received: by mail-pf1-f194.google.com with SMTP id g30so2624937pfr.3 for ; Sat, 18 Apr 2020 08:13:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=wjmCBOEfzqW0cnWbfi1j4+3Gc0dBrwK/JfJg9kdoLoo=; b=F19MROdL9nDqzx8p/pTE5VVVG6XlqGG2qlaVqfWrEo0i/3Nu0OD8AHQZJcKTctnG4W EqT+bzJx5TS4vdJivpj14sCy8pBlfRGGxkM4dxHBg5jZf2kSpZZZEQZzcul5o0nRk+Vj BcTWqA//7EN/gj57BUW7GUO+skSgDkMLqpA3bDxXGtfSAS0t1DJ/WcsCdTahYH/AFTWO uOq0deuW3iIJzP29YkMCEkoPUYCO9K1hRxii5Z9sniEGMOO8ZfFDCEa5dsyYloNGzwFr NtU7WAvRI7Wrkip5vh+UbrVvFVrkWk9Y3TcRbT8g5Xg+X3kA7UF/CurPKZ6mIQNI3LGg IWSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=wjmCBOEfzqW0cnWbfi1j4+3Gc0dBrwK/JfJg9kdoLoo=; b=RF6zUEIdTjlmOi6mdxzphM4z8jqVyMRi1S6W52m6LGnsm11ik9EAiOLpT6Z4lBQjrs xryc6HxStGfQvMToIATZhzvYJ7Wrk5KQS/oV4Ve5WhPkWGCljR67jnbf7eKqnB2h+To9 u8VI4TEzGCAY27tFY0AW+FU1n65rU2W5F/yhpHGCcSoOKW9FUNk30oH/tSBmVUGXTFEL KI2A1AAz/NRndSi6Cpp28mg0xW03Tlwr4WeBMdMrKHoBec8NJDTwGMTVOn431GRWlbbb i2D/ElzHCoBwvKRM36nviqAVXWQ7NCK+RQQomhm7nvAOm8muDaEvzdmUYtRRGwxblX9q c8xQ== X-Gm-Message-State: AGi0PubtakQwDr582bF1HmcIg0tiGgpFnrY9MYel1nQ1LwU9Bi2tYWp/ qjPyHgmVyTJFSbWD8YWiivY= X-Google-Smtp-Source: APiQypJABOuzzbrqt3TJLfEQ3bEMTLs0C224RdypiUg6zJtjyMNNwOKGHIs3l1rCtH9ak2kXsSnFiQ== X-Received: by 2002:a65:498f:: with SMTP id r15mr8279109pgs.345.1587222817757; Sat, 18 Apr 2020 08:13:37 -0700 (PDT) Received: from localhost.localdomain ([203.100.54.194]) by smtp.gmail.com with ESMTPSA id j32sm9613028pgb.55.2020.04.18.08.13.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 18 Apr 2020 08:13:37 -0700 (PDT) From: Yafang Shao To: hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, Yafang Shao Subject: [PATCH 2/3] mm, memcg: introduce a new helper task_in_memcg_oom_set() Date: Sat, 18 Apr 2020 11:13:10 -0400 Message-Id: <20200418151311.7397-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20200418151311.7397-1-laoar.shao@gmail.com> References: <20200418151311.7397-1-laoar.shao@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A new helper task_in_memcg_oom_set() is introduced for later use. It will be used to bail out from the charge path if no victim can be found in memcg oom. Signed-off-by: Yafang Shao --- include/linux/memcontrol.h | 25 +++++++++++++++++++++++++ mm/memcontrol.c | 12 ++---------- 2 files changed, 27 insertions(+), 10 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 98bd8fb2f5c7..767bac135787 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -565,6 +565,23 @@ static inline bool task_in_memcg_oom(struct task_struct *p) return p->memcg_in_oom; } +/* Bail out from the charge path if no victim found in memcg oom */ +static inline enum oom_status task_in_memcg_oom_set(struct task_struct *p, + struct mem_cgroup *memcg, + gfp_t mask, + int order) +{ + if (!current->in_user_fault) + return OOM_SKIPPED; + + css_get(&memcg->css); + p->memcg_in_oom = memcg; + p->memcg_oom_gfp_mask = mask; + p->memcg_oom_order = order; + + return OOM_ASYNC; +} + bool mem_cgroup_oom_synchronize(bool wait); struct mem_cgroup *mem_cgroup_get_oom_group(struct task_struct *victim, struct mem_cgroup *oom_domain); @@ -1031,6 +1048,14 @@ static inline bool task_in_memcg_oom(struct task_struct *p) return false; } +static inline enum oom_status task_in_memcg_oom_set(struct task_struct *p, + struct mem_cgroup *memcg, + gfp_t mask, + int order) +{ + return OOM_SUCCESS; +} + static inline bool mem_cgroup_oom_synchronize(bool wait) { return false; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 22418b55804f..d6cb1b786045 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1800,16 +1800,8 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int * Please note that mem_cgroup_out_of_memory might fail to find a * victim and then we have to bail out from the charge path. */ - if (memcg->oom_kill_disable) { - if (!current->in_user_fault) - return OOM_SKIPPED; - css_get(&memcg->css); - current->memcg_in_oom = memcg; - current->memcg_oom_gfp_mask = mask; - current->memcg_oom_order = order; - - return OOM_ASYNC; - } + if (memcg->oom_kill_disable) + return task_in_memcg_oom_set(current, memcg, mask, order); mem_cgroup_mark_under_oom(memcg); From patchwork Sat Apr 18 15:13:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 11496609 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E9A981 for ; Sat, 18 Apr 2020 15:13:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D295B2076A for ; Sat, 18 Apr 2020 15:13:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Y32CSBdM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D295B2076A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E999E8E0008; Sat, 18 Apr 2020 11:13:43 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E4A288E0003; Sat, 18 Apr 2020 11:13:43 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D5F208E0008; Sat, 18 Apr 2020 11:13:43 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0243.hostedemail.com [216.40.44.243]) by kanga.kvack.org (Postfix) with ESMTP id BDD2C8E0003 for ; Sat, 18 Apr 2020 11:13:43 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 826BF63FA for ; Sat, 18 Apr 2020 15:13:43 +0000 (UTC) X-FDA: 76721320326.20.car83_77ca5596ebe07 X-Spam-Summary: 2,0,0,41253977819ddef8,d41d8cd98f00b204,laoar.shao@gmail.com,,RULES_HIT:1:2:41:355:379:541:800:960:966:973:988:989:1260:1345:1359:1437:1605:1730:1747:1777:1792:2196:2198:2199:2200:2393:2559:2562:2693:2731:2898:3138:3139:3140:3141:3142:3653:3865:3866:3867:3868:3870:3871:3872:3874:4050:4321:4385:4605:5007:6261:6653:7514:7903:8531:8555:9413:10004:11026:11233:11473:11658:11914:12043:12048:12049:12291:12296:12297:12438:12485:12517:12519:12555:12895:12986:13161:13229:14394:14664:14687:21080:21324:21444:21451:21627:21666:21972:21990:30054:30070,0,RBL:209.85.215.196:@gmail.com:.lbl8.mailshell.net-66.100.201.100 62.50.0.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: car83_77ca5596ebe07 X-Filterd-Recvd-Size: 10152 Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Sat, 18 Apr 2020 15:13:43 +0000 (UTC) Received: by mail-pg1-f196.google.com with SMTP id o15so2119268pgi.1 for ; Sat, 18 Apr 2020 08:13:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=rDs3m5c7QgR5dvRqd0B9A5lPWUbsdNS/xsLnI70SQss=; b=Y32CSBdMVLj5BVb72xkC3mQ2pMz5jPw1D+8bd/Pt5C6XhVIVKFPz02h+1Pv84tES+K WxOnKUxPslnAKuHqWu/0AetvzzZG+r8Hh9zsyNl8GOZd/4YNT1A08HMJKSYYqoSnGDn5 nymN0TsYz2fUTGNwjsh8GTxQqxN6WSN4+q0f4Gj3d1J/zPvmY4bgn2avHQwpSvtgYqbP ieaRfdrlfUvpqM88kWIwQPIloWr64lBX2nbSZEfJ7RPQPo1ZO6Th8HwW/zpvMMufFVZF a9ApWmLsuSQjDdU+JUBl5XaTkbZkcXIP/838fmLiyimw+APrrm1yNKbR7vUIYPnlZBLn I+cA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=rDs3m5c7QgR5dvRqd0B9A5lPWUbsdNS/xsLnI70SQss=; b=S6AIn6i9vGdlpKNo4yKX7hq7ft8ljq+zm2WFi+PVm4YK9WbetlCS7aKvenZFtfLzJc JiaEvc+jU+v9qQHrRM+YibCIPEdWpJ2R59hdc2K7neyV3AiCVqNPZYNJbwCII5EKMkuS WLZhkjsKxCOIqf5nYBtnBYc+NdpbFY74+4b0Np5Apk3ukPXQ7lyJn2k4vbk38xIaFQCj mXAnkK6BAf9JgnKey9Espi3Q6LoEQ7A1nclk5F6ZzXUAel6GmYtW7JRl6tr0JeMNf1KF CA5gTi9B5DtB+YdYJnjqlrODNv9iosgi6SwW7PwalmXCdrWj8tasRefrU6WeD8UTUbeL Bniw== X-Gm-Message-State: AGi0PuYNTaK9g9/ic/krZEW/ogZrNomW1e01jcovTdUd3AVOcjajrTkV Tof06oX0/CtbetLfiCeNZ5A= X-Google-Smtp-Source: APiQypIBndF88G89GiIx3faHP7ouPXOTAPd5mwmfOuwz9fUnPz140ZeUXGMUlq3WxEpnPBKd9Gnm/w== X-Received: by 2002:a62:62c3:: with SMTP id w186mr8264161pfb.238.1587222822222; Sat, 18 Apr 2020 08:13:42 -0700 (PDT) Received: from localhost.localdomain ([203.100.54.194]) by smtp.gmail.com with ESMTPSA id j32sm9613028pgb.55.2020.04.18.08.13.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 18 Apr 2020 08:13:41 -0700 (PDT) From: Yafang Shao To: hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, akpm@linux-foundation.org Cc: linux-mm@kvack.org, Yafang Shao Subject: [PATCH 3/3] memcg oom: bail out from the charge path if no victim found Date: Sat, 18 Apr 2020 11:13:11 -0400 Message-Id: <20200418151311.7397-4-laoar.shao@gmail.com> X-Mailer: git-send-email 2.18.1 In-Reply-To: <20200418151311.7397-1-laoar.shao@gmail.com> References: <20200418151311.7397-1-laoar.shao@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Without considering the manually triggered OOM, if no victim found in system OOM, the system will be deadlocked on memory, however if no victim found in memcg OOM, it can charge successfully and runs well. This behavior in memcg oom is not proper because that can prevent the memcg from being limited. Take an easy example. $ cd /sys/fs/cgroup/foo/ $ echo $$ > cgroup.procs $ echo 200M > memory.max $ cat memory.max 209715200 $ echo -1000 > /proc/$$/oom_score_adj Then, let's run a memhog task in memcg foo, which will allocate 1G memory and keeps running. $ /home/yafang/test/memhog & Then memory.current will be greater than memory.max. Run bellow command in another shell. $ cat /sys/fs/cgroup/foo/memory.current 1097228288 The tasks which have already allocated memory and won't allocate new memory still runs well. This behavior makes nonsense. This patch is to improve it. If no victim found in memcg oom, we should force the current task to wait until there's available pages. That is similar with the behavior in memcg1 when oom_kill_disable is set. In memcg2, the memcg oom can also be triggered manually - by reducing memory.max. We should distinguish this manually triggered memcg oom with other reguler memcg oom, so a magic key "oom_control.order == -2" is set in this situation. The tasks waiting in memcg oom will be waked up when we enlarge the memory.max. As these tasks is killable, we can also kill them directly. In memcg1, it cooperates well with memory.oom_control(oom_kill_disable). The tasks waiting in memcg1 oom can be waked up by enlarging memory.limit_in_bytes or disabling oom_kill_disable. As oom_kill_disable can be set manually, we should distinguish it with other reguler memcg oom as well. A member named memcg_oom_wait is introduced in struct task_struct to handle it. Signed-off-by: Yafang Shao --- include/linux/memcontrol.h | 8 ++++++-- include/linux/sched.h | 1 + mm/memcontrol.c | 25 +++++++++++++++++++++---- mm/oom_kill.c | 22 ++++++++++++++++++++++ 4 files changed, 50 insertions(+), 6 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 767bac135787..5df2c08e2720 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -569,7 +569,8 @@ static inline bool task_in_memcg_oom(struct task_struct *p) static inline enum oom_status task_in_memcg_oom_set(struct task_struct *p, struct mem_cgroup *memcg, gfp_t mask, - int order) + int order, + bool force) { if (!current->in_user_fault) return OOM_SKIPPED; @@ -578,6 +579,8 @@ static inline enum oom_status task_in_memcg_oom_set(struct task_struct *p, p->memcg_in_oom = memcg; p->memcg_oom_gfp_mask = mask; p->memcg_oom_order = order; + if (force) + p->memcg_oom_wait = true; return OOM_ASYNC; } @@ -1051,7 +1054,8 @@ static inline bool task_in_memcg_oom(struct task_struct *p) static inline enum oom_status task_in_memcg_oom_set(struct task_struct *p, struct mem_cgroup *memcg, gfp_t mask, - int order) + int order, + bool force) { return OOM_SUCCESS; } diff --git a/include/linux/sched.h b/include/linux/sched.h index 4418f5cb8324..cc1c7de7c248 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -774,6 +774,7 @@ struct task_struct { #endif #ifdef CONFIG_MEMCG unsigned in_user_fault:1; + unsigned memcg_oom_wait:1; #endif #ifdef CONFIG_COMPAT_BRK unsigned brk_randomized:1; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index d6cb1b786045..a637e13a4964 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1801,7 +1801,8 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int * victim and then we have to bail out from the charge path. */ if (memcg->oom_kill_disable) - return task_in_memcg_oom_set(current, memcg, mask, order); + return task_in_memcg_oom_set(current, memcg, mask, + order, false); mem_cgroup_mark_under_oom(memcg); @@ -1863,7 +1864,8 @@ bool mem_cgroup_oom_synchronize(bool handle) if (locked) mem_cgroup_oom_notify(memcg); - if (locked && !memcg->oom_kill_disable) { + if (locked && !memcg->oom_kill_disable && + !current->memcg_oom_wait) { mem_cgroup_unmark_under_oom(memcg); finish_wait(&memcg_oom_waitq, &owait.wait); mem_cgroup_out_of_memory(memcg, current->memcg_oom_gfp_mask, @@ -1883,7 +1885,10 @@ bool mem_cgroup_oom_synchronize(bool handle) */ memcg_oom_recover(memcg); } + cleanup: + if (current->memcg_oom_wait) + current->memcg_oom_wait = false; current->memcg_in_oom = NULL; css_put(&memcg->css); return true; @@ -6056,6 +6061,7 @@ static ssize_t memory_max_write(struct kernfs_open_file *of, struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); unsigned int nr_reclaims = MEM_CGROUP_RECLAIM_RETRIES; bool drained = false; + bool enlarge = false; unsigned long max; int err; @@ -6069,7 +6075,12 @@ static ssize_t memory_max_write(struct kernfs_open_file *of, for (;;) { unsigned long nr_pages = page_counter_read(&memcg->memory); - if (nr_pages <= max) + if (nr_pages < max) { + enlarge = true; + break; + } + + if (nr_pages == max) break; if (signal_pending(current)) @@ -6081,6 +6092,9 @@ static ssize_t memory_max_write(struct kernfs_open_file *of, continue; } + if (memcg->under_oom) + break; + if (nr_reclaims) { if (!try_to_free_mem_cgroup_pages(memcg, nr_pages - max, GFP_KERNEL, true)) @@ -6089,11 +6103,14 @@ static ssize_t memory_max_write(struct kernfs_open_file *of, } memcg_memory_event(memcg, MEMCG_OOM); - if (mem_cgroup_out_of_memory(memcg, GFP_KERNEL, 0) != + if (mem_cgroup_out_of_memory(memcg, GFP_KERNEL, -2) != OOM_SUCCESS) break; } + if (enlarge) + memcg_oom_recover(memcg); + memcg_wb_domain_size_changed(memcg); return nbytes; } diff --git a/mm/oom_kill.c b/mm/oom_kill.c index d5a941bea2d7..df564495c8b2 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -158,6 +158,15 @@ static inline bool is_sysrq_oom(struct oom_control *oc) return oc->order == -1; } +/* + * order == -2 means the oom kill is triggered by reducing memcg max, + * otherwise only for display puerposes. + */ +static bool is_max_write_oom(struct oom_control *oc) +{ + return oc->order == -2; +} + /* return true if the task is not adequate as candidate victim task. */ static bool oom_unkillable_task(struct task_struct *p) { @@ -1108,6 +1117,19 @@ enum oom_status out_of_memory(struct oom_control *oc) */ if (!is_sysrq_oom(oc) && !is_memcg_oom(oc)) panic("System is deadlocked on memory\n"); + + /* Bail out from the charge path if we can't find a victim. */ + if (is_memcg_oom(oc)) { + if (is_max_write_oom(oc)) + return OOM_SKIPPED; + + return task_in_memcg_oom_set(current, + oc->memcg, + oc->gfp_mask, + oc->order, + true); + } + } if (oc->chosen && oc->chosen != (void *)-1UL) oom_kill_process(oc, !is_memcg_oom(oc) ? "Out of memory" :