From patchwork Tue Oct 27 08:02:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11859627 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E018661C for ; Tue, 27 Oct 2020 08:03:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 651382224E for ; Tue, 27 Oct 2020 08:03:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=bytedance-com.20150623.gappssmtp.com header.i=@bytedance-com.20150623.gappssmtp.com header.b="J7QFPcYy" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 651382224E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 937F46B0062; Tue, 27 Oct 2020 04:03:23 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 90E016B006C; Tue, 27 Oct 2020 04:03:23 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 826466B006E; Tue, 27 Oct 2020 04:03:23 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0063.hostedemail.com [216.40.44.63]) by kanga.kvack.org (Postfix) with ESMTP id 50E056B0062 for ; Tue, 27 Oct 2020 04:03:23 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 02D318249980 for ; Tue, 27 Oct 2020 08:03:23 +0000 (UTC) X-FDA: 77416965486.29.cover01_5c16def2727a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id D105A18086CBE for ; Tue, 27 Oct 2020 08:03:22 +0000 (UTC) X-Spam-Summary: 1,0,0,89e4cc378fb4f12c,d41d8cd98f00b204,songmuchun@bytedance.com,,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:1801:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3866:4321:4605:5007:6261:6653:6737:8784:10004:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12895:13069:13311:13357:13894:14096:14181:14384:14394:14581:14721:21080:21444:21451:21627:21990:30054,0,RBL:209.85.216.65:@bytedance.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201;04y85rwstxb3we13ebhhjcyuyufcuopwbg5awkp1hsnmwf74asjbiswhfc84wmw.1uek4kqn63oyoznccppn57w1n7ppkqcx1rukw9k4xefwp1n5rgjab885x3ur6m6.6-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: cover01_5c16def2727a X-Filterd-Recvd-Size: 4505 Received: from mail-pj1-f65.google.com (mail-pj1-f65.google.com [209.85.216.65]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Tue, 27 Oct 2020 08:03:22 +0000 (UTC) Received: by mail-pj1-f65.google.com with SMTP id l2so343784pjt.5 for ; Tue, 27 Oct 2020 01:03:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pkKyxTwl8ekG841RoN58bB0vzsrEHZTiUbsxmBIxH0Y=; b=J7QFPcYyl+Rudib/U00XRzim7IuwfNQtPAZP0xb+Mv5iSY4+XHq5yo6c5JpkrTzH+x zMqLgUYoRMW6RLuijAzEUXPiwdUAv0OlP018X16jtfAE4iCBozAlIGusVJ379BvQF4ex euMiQdjzZbooFeBqwPBtKiAAMXvQMYa2laliQtiAMGa7S2aFAmb0VdDszsuq2FBeQIrn 6sGs6A7g90DTQsM0B1cOt3EBBB544p5q7HmT5QI3C7xvTdM987JLsiQjWQblfV4fCci2 1R7xoknLsUCfNO9cESLng0m+PikAlEnM8DcLgi/D28YoWPYN5FGqJexP7SZNeepLSpx9 bcRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pkKyxTwl8ekG841RoN58bB0vzsrEHZTiUbsxmBIxH0Y=; b=TYXsVOD1G5yW/CRdjPOha/s+JmN+T43RmowD1KGIFGXHHodx78Hjjfwd+ZZtqXqMCt oP7goadIi5Lioiv/HC1pzgjh5Q4VE5YYq+/w0z0MU4uEO9zRhUqV+SzAzhNaRkfsafQX 4jBntk2A0buDAtjkVvoRCFDR7D9Jls+I2xP0ZunyyN9lUe2xe7LlEeiGRITM97+66y3x KqeSpNnz7xgQz26qlqbFNHJVIvA5ccnvCPFAvfi9QaZpJYsbMiydRcdXtB4p+s4JAwkp csj5dPAPWlD4hc63GznsrWM5Uw0u2vVqnKDAjPryBqGQUAe7j6jHE9IpPFOSHHGHwjIH gf2A== X-Gm-Message-State: AOAM530sRpQj7keqUSrX5KlIE/6sjcy0qO6wRUwOdr5DR3eSy7+Lv31n GPu/p9ziqfZXYyDsZQgOMLOF5Q== X-Google-Smtp-Source: ABdhPJykmbE8wdjYC2X2wAsrBvfaEdwwrwqA0TK0rjcGxYjqEk01eqrw7luf4X70zO+qxkB7CaqFXg== X-Received: by 2002:a17:902:468:b029:d5:ad3c:cf52 with SMTP id 95-20020a1709020468b02900d5ad3ccf52mr1397871ple.7.1603785801285; Tue, 27 Oct 2020 01:03:21 -0700 (PDT) Received: from Smcdef-MBP.local.net ([103.136.220.89]) by smtp.gmail.com with ESMTPSA id p8sm1039580pgs.34.2020.10.27.01.03.14 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Oct 2020 01:03:20 -0700 (PDT) From: Muchun Song To: hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, akpm@linux-foundation.org, shakeelb@google.com, guro@fb.com, iamjoonsoo.kim@lge.com, laoar.shao@gmail.com, chris@chrisdown.name, christian.brauner@ubuntu.com, peterz@infradead.org, mingo@kernel.org, keescook@chromium.org, tglx@linutronix.de, esyr@redhat.com, surenb@google.com, areber@redhat.com, elver@google.com Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH 1/5] mm: memcg/slab: Fix return child memcg objcg for root memcg Date: Tue, 27 Oct 2020 16:02:52 +0800 Message-Id: <20201027080256.76497-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201027080256.76497-1-songmuchun@bytedance.com> References: <20201027080256.76497-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Consider the following memcg hierarchy. root / \ A B If we get the objcg of memcg A failed, the get_obj_cgroup_from_current can return the wrong objcg for the root memcg. Fixes: bf4f059954dc ("mm: memcg/slab: obj_cgroup API") Signed-off-by: Muchun Song --- mm/memcontrol.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 1337775b04f3..fcbd79c5023e 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2945,7 +2945,7 @@ struct mem_cgroup *mem_cgroup_from_obj(void *p) __always_inline struct obj_cgroup *get_obj_cgroup_from_current(void) { - struct obj_cgroup *objcg = NULL; + struct obj_cgroup *objcg; struct mem_cgroup *memcg; if (memcg_kmem_bypass()) @@ -2964,6 +2964,9 @@ __always_inline struct obj_cgroup *get_obj_cgroup_from_current(void) } rcu_read_unlock(); + if (memcg == root_mem_cgroup) + return NULL; + return objcg; } From patchwork Tue Oct 27 08:02:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11859629 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5C69C61C for ; Tue, 27 Oct 2020 08:03:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 05FAD207BB for ; Tue, 27 Oct 2020 08:03:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=bytedance-com.20150623.gappssmtp.com header.i=@bytedance-com.20150623.gappssmtp.com header.b="0mmq9zyn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 05FAD207BB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1498E6B006C; Tue, 27 Oct 2020 04:03:33 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1203A6B006E; Tue, 27 Oct 2020 04:03:33 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 037496B0070; Tue, 27 Oct 2020 04:03:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0031.hostedemail.com [216.40.44.31]) by kanga.kvack.org (Postfix) with ESMTP id CC0EC6B006C for ; Tue, 27 Oct 2020 04:03:32 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 71EFC180AD806 for ; Tue, 27 Oct 2020 08:03:32 +0000 (UTC) X-FDA: 77416965864.09.voice61_1405e802727a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id 59A66180AD802 for ; Tue, 27 Oct 2020 08:03:32 +0000 (UTC) X-Spam-Summary: 1,0,0,e0c406d29acba01c,d41d8cd98f00b204,songmuchun@bytedance.com,,RULES_HIT:41:355:379:541:800:960:965:966:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1540:1711:1730:1747:1777:1792:2196:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3352:3865:3866:3867:3872:3874:4385:4390:4395:5007:6261:6653:6737:8603:9036:10004:11026:11658:11914:12048:12296:12297:12517:12519:12555:12679:12895:13069:13161:13229:13311:13357:13894:14096:14181:14384:14394:14721:21080:21444:21451:21627:30054:30070,0,RBL:209.85.215.196:@bytedance.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201;04yrpns3ysz6k4nkpdj57ug6bzf7fopxjrhttu45t6os1jqrbhqcs93gybyresq.xiw6hqi9b3yq749o9pa6osb84twsdyx57duikzeqrasxmm66gwyptex8k5gqemp.o-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:34,LUA_SUMMARY:none X-HE-Tag: voice61_1405e802727a X-Filterd-Recvd-Size: 4291 Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by imf12.hostedemail.com (Postfix) with ESMTP for ; Tue, 27 Oct 2020 08:03:31 +0000 (UTC) Received: by mail-pg1-f196.google.com with SMTP id o7so321762pgv.6 for ; Tue, 27 Oct 2020 01:03:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YWIAYpPs9+svXY8PEtGZ5WZ0CNEgBLTQowd7Fmk6Ks0=; b=0mmq9zynkfizT4Ql1IxpOGnGDMmXfmaRO/jTjmX5y5xzgLSWldW1D0OmZmiaqfd3Jj uTnV/2ojetEMYoqEy6o95NADBO7FFfWXq+J8lRdeuLuCL+w6GFggpi1WoXcq7Ai0Ouln kMd+2CMAl09mrojtd/edMDGXtdwUoq2EPfkdkJl8IJ42pR927Gy1+02gSJnULXNN7/AR Tin3noEy1hSSySrqkHSsnkchL+m1SAd/NAdjV9jF1hVqVfziOhHfoalcTD5MZA+F0CcN Wsvz53aafL+3Nhw88pYQat+6bd/CjXTZhks3P7F+G145xKZfz8Ji/eNFy8hAV5ap0QgC lJjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YWIAYpPs9+svXY8PEtGZ5WZ0CNEgBLTQowd7Fmk6Ks0=; b=HWQKLqgc2r7Dgq9C4xgz3HpmKkfrcRO44FHMRJo1/F8wkjnOIng2FRATPM23KALty/ UCfLzJn/4BF+8EkMvwGBdDUPeRkAr8Qeb5XhRnSQ5Pjyer3/RhKRcG3zSmRjmALZP91e rmPKOIFzX+EXfXZdAyxxKkfL/UFxEbOj2akUjfdwE+sEQI3pEs7oxkaVg7+taAKIWP3x Zi6nP1+PMPEljSeBaRMQqlHYbpNFhsRzZwtGyERsm5epQ8zfekj0AsDoWWM0qkMNVdRg JGSCALho98cESac8RKKBnQbBbzMwJIxIgzqfa87bC1x38106mKHv+LFhzN/gk45eS+wM BQfQ== X-Gm-Message-State: AOAM533SSipr1EepnyPgF3I7AyAPozcOv7H3D1DlmOF/4bMPwZNsaITp /xZEFQSXKWjZTb1994Ha0eRnAw== X-Google-Smtp-Source: ABdhPJz1hfXJG1lmVYIhQsXjHZGILbCTSJSIyuvmPZNDlDiLd4f3AkFLLlZK7QOTjOFm2stL4K/P9g== X-Received: by 2002:a63:5421:: with SMTP id i33mr938048pgb.316.1603785810689; Tue, 27 Oct 2020 01:03:30 -0700 (PDT) Received: from Smcdef-MBP.local.net ([103.136.220.89]) by smtp.gmail.com with ESMTPSA id p8sm1039580pgs.34.2020.10.27.01.03.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Oct 2020 01:03:30 -0700 (PDT) From: Muchun Song To: hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, akpm@linux-foundation.org, shakeelb@google.com, guro@fb.com, iamjoonsoo.kim@lge.com, laoar.shao@gmail.com, chris@chrisdown.name, christian.brauner@ubuntu.com, peterz@infradead.org, mingo@kernel.org, keescook@chromium.org, tglx@linutronix.de, esyr@redhat.com, surenb@google.com, areber@redhat.com, elver@google.com Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH 2/5] mm: memcg/slab: Fix use after free in obj_cgroup_charge Date: Tue, 27 Oct 2020 16:02:53 +0800 Message-Id: <20201027080256.76497-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201027080256.76497-1-songmuchun@bytedance.com> References: <20201027080256.76497-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The rcu_read_lock/unlock only can guarantee that the memcg will not be freed, but it cannot guarantee the success of css_get to memcg. This can be happened when we reparent the memcg. So using css_tryget instead of css_get. Fixes: bf4f059954dc ("mm: memcg/slab: obj_cgroup API") Signed-off-by: Muchun Song Acked-by: Roman Gushchin --- mm/memcontrol.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index fcbd79c5023e..c0c27bee23ad 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3223,8 +3223,10 @@ int obj_cgroup_charge(struct obj_cgroup *objcg, gfp_t gfp, size_t size) * independently later. */ rcu_read_lock(); +retry: memcg = obj_cgroup_memcg(objcg); - css_get(&memcg->css); + if (!css_tryget(&memcg->css)) + goto retry; rcu_read_unlock(); nr_pages = size >> PAGE_SHIFT; From patchwork Tue Oct 27 08:02:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11859631 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A93136A2 for ; Tue, 27 Oct 2020 08:03:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 61EA621D24 for ; Tue, 27 Oct 2020 08:03:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=bytedance-com.20150623.gappssmtp.com header.i=@bytedance-com.20150623.gappssmtp.com header.b="ldzMtw/9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 61EA621D24 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 714AD6B006E; Tue, 27 Oct 2020 04:03:40 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6EBAF6B0070; Tue, 27 Oct 2020 04:03:40 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6018C6B0071; Tue, 27 Oct 2020 04:03:40 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0187.hostedemail.com [216.40.44.187]) by kanga.kvack.org (Postfix) with ESMTP id 333AF6B006E for ; Tue, 27 Oct 2020 04:03:40 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id BBBE21EE6 for ; Tue, 27 Oct 2020 08:03:39 +0000 (UTC) X-FDA: 77416966158.17.price00_480c5b72727a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id 9BC7B180D0185 for ; Tue, 27 Oct 2020 08:03:39 +0000 (UTC) X-Spam-Summary: 1,0,0,ed0dabbbba906ec2,d41d8cd98f00b204,songmuchun@bytedance.com,,RULES_HIT:2:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1605:1606:1730:1747:1777:1792:2393:2559:2562:2890:3138:3139:3140:3141:3142:3865:3866:3867:3871:3874:4042:4119:4321:4605:5007:6261:6653:6737:10004:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12895:12986:13161:13229:13894:14096:14394:14877:21080:21094:21323:21444:21450:21451:21627:21740:21966:21990:30001:30029:30054,0,RBL:209.85.214.196:@bytedance.com:.lbl8.mailshell.net-66.100.201.201 62.2.0.100;04y8r5hs86918dfkd1kq9mnmp3qznocys16a8fje36m6wx7ujtsmeynpdweydhq.9bzdmusznq3muhtgku6kyw3i4mfou3myhqh3gug1qwozpz9y383c6fk4ppkdh57.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: price00_480c5b72727a X-Filterd-Recvd-Size: 8781 Received: from mail-pl1-f196.google.com (mail-pl1-f196.google.com [209.85.214.196]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Tue, 27 Oct 2020 08:03:38 +0000 (UTC) Received: by mail-pl1-f196.google.com with SMTP id b19so353680pld.0 for ; Tue, 27 Oct 2020 01:03:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CcP1qCO+NxV4ZfTtTYq+9NLHwEqCtxnTam0h3E0eXoY=; b=ldzMtw/9aoBNu1NB1gYXBiySt/fmyfIdYqovv3lB+Khh/+hI0IBXFQUEpvIN9McZNz z1OfclFxVWOlxOv6eyKxiYNP84Z9/k53WzIkHOOme5HItxqZrw3poX87ZcKT8BZmcXny 0zD6i8b9iYMQ31vz6dwG6zVXvOPpaH8S9UENtwmf/O2j+K+FpzKEk9npn8NYZratmrwO LJzA3fHzBoQMJTjxFr23ghu7QjOY9FLRtF2VjKfAREtrrWeNputyCME8katlLf0A0pL4 EXQ9aU6tuyu1SqHU0tYDKNI8b2paDbWhksM/SCbkTCuz2LAGj+Hq3jiafOpvIgCGHbMi QhvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CcP1qCO+NxV4ZfTtTYq+9NLHwEqCtxnTam0h3E0eXoY=; b=e/K0vDiHY1mF/Q1GYLGz3etay8EyE3peIL1AXGtyTW8OTutuEG6H65cWoYumotfwsc 6QZmbmM5Rp7P6fXNWMFA0ZIDRqcaD6oVvbRuWIxw6lBA2MkdQexXpNkXQy+7jXj0xxlD 3oFAyD67J/kg77i9ejkmPSoC4f2aK7OUoLGoiKSlMMhxCRXLreuQIShL+lVeTKLwHXfm wROLvSwyJIW3kZ9xwS0+KcgZKop8FKNs54PvUtSvQmvHonMOnBvVfLS/bqLZC+YbjoVk uR60B4wyY8n9bnBTl+kR2Acw+QXsJHJOHaidg/85wonHwdPpCaJp3ak1Glf9SCDuOW9u K4nQ== X-Gm-Message-State: AOAM530d3ztKBWfIjDzU2fRjCnGNCRCqog8gj4Y8FY88MHfZ8Drq4OQa IaoSAq8oBirpEnzoyycvOBDVnA== X-Google-Smtp-Source: ABdhPJwcAbffc6htWbV7MxS/83vNzm6AujW6V0vE+GjnoU5Zb2sPH+c74AGETf8TVykW/jNgYO3VLw== X-Received: by 2002:a17:90a:a008:: with SMTP id q8mr914987pjp.211.1603785818068; Tue, 27 Oct 2020 01:03:38 -0700 (PDT) Received: from Smcdef-MBP.local.net ([103.136.220.89]) by smtp.gmail.com with ESMTPSA id p8sm1039580pgs.34.2020.10.27.01.03.31 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Oct 2020 01:03:37 -0700 (PDT) From: Muchun Song To: hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, akpm@linux-foundation.org, shakeelb@google.com, guro@fb.com, iamjoonsoo.kim@lge.com, laoar.shao@gmail.com, chris@chrisdown.name, christian.brauner@ubuntu.com, peterz@infradead.org, mingo@kernel.org, keescook@chromium.org, tglx@linutronix.de, esyr@redhat.com, surenb@google.com, areber@redhat.com, elver@google.com Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH 3/5] mm: memcg/slab: Rename *_lruvec_slab_state to *_lruvec_kmem_state Date: Tue, 27 Oct 2020 16:02:54 +0800 Message-Id: <20201027080256.76497-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201027080256.76497-1-songmuchun@bytedance.com> References: <20201027080256.76497-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The *_lruvec_slab_state is also suitable for pages allocated from buddy, not just for the slab objects. But the function name seems to tell us that only slab object is applicable. So we can rename the keyword of slab to kmem. Signed-off-by: Muchun Song Acked-by: Roman Gushchin --- include/linux/memcontrol.h | 18 +++++++++--------- kernel/fork.c | 2 +- mm/memcontrol.c | 3 ++- mm/workingset.c | 8 ++++---- 4 files changed, 16 insertions(+), 15 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index d7e339bf72dc..95807bf6be64 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -793,15 +793,15 @@ void __mod_memcg_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx, int val); void __mod_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx, int val); -void __mod_lruvec_slab_state(void *p, enum node_stat_item idx, int val); +void __mod_lruvec_kmem_state(void *p, enum node_stat_item idx, int val); -static inline void mod_lruvec_slab_state(void *p, enum node_stat_item idx, +static inline void mod_lruvec_kmem_state(void *p, enum node_stat_item idx, int val) { unsigned long flags; local_irq_save(flags); - __mod_lruvec_slab_state(p, idx, val); + __mod_lruvec_kmem_state(p, idx, val); local_irq_restore(flags); } @@ -1227,7 +1227,7 @@ static inline void mod_lruvec_page_state(struct page *page, mod_node_page_state(page_pgdat(page), idx, val); } -static inline void __mod_lruvec_slab_state(void *p, enum node_stat_item idx, +static inline void __mod_lruvec_kmem_state(void *p, enum node_stat_item idx, int val) { struct page *page = virt_to_head_page(p); @@ -1235,7 +1235,7 @@ static inline void __mod_lruvec_slab_state(void *p, enum node_stat_item idx, __mod_node_page_state(page_pgdat(page), idx, val); } -static inline void mod_lruvec_slab_state(void *p, enum node_stat_item idx, +static inline void mod_lruvec_kmem_state(void *p, enum node_stat_item idx, int val) { struct page *page = virt_to_head_page(p); @@ -1330,14 +1330,14 @@ static inline void __dec_lruvec_page_state(struct page *page, __mod_lruvec_page_state(page, idx, -1); } -static inline void __inc_lruvec_slab_state(void *p, enum node_stat_item idx) +static inline void __inc_lruvec_kmem_state(void *p, enum node_stat_item idx) { - __mod_lruvec_slab_state(p, idx, 1); + __mod_lruvec_kmem_state(p, idx, 1); } -static inline void __dec_lruvec_slab_state(void *p, enum node_stat_item idx) +static inline void __dec_lruvec_kmem_state(void *p, enum node_stat_item idx) { - __mod_lruvec_slab_state(p, idx, -1); + __mod_lruvec_kmem_state(p, idx, -1); } /* idx can be of type enum memcg_stat_item or node_stat_item */ diff --git a/kernel/fork.c b/kernel/fork.c index 4b328aecabb2..4fb0bbc3b041 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -384,7 +384,7 @@ static void account_kernel_stack(struct task_struct *tsk, int account) mod_lruvec_page_state(vm->pages[0], NR_KERNEL_STACK_KB, account * (THREAD_SIZE / 1024)); else - mod_lruvec_slab_state(stack, NR_KERNEL_STACK_KB, + mod_lruvec_kmem_state(stack, NR_KERNEL_STACK_KB, account * (THREAD_SIZE / 1024)); } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c0c27bee23ad..22b4fb941b54 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -866,7 +866,7 @@ void __mod_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx, __mod_memcg_lruvec_state(lruvec, idx, val); } -void __mod_lruvec_slab_state(void *p, enum node_stat_item idx, int val) +void __mod_lruvec_kmem_state(void *p, enum node_stat_item idx, int val) { pg_data_t *pgdat = page_pgdat(virt_to_page(p)); struct mem_cgroup *memcg; @@ -2920,6 +2920,7 @@ struct mem_cgroup *mem_cgroup_from_obj(void *p) if (mem_cgroup_disabled()) return NULL; + VM_BUG_ON(!virt_addr_valid(p)); page = virt_to_head_page(p); /* diff --git a/mm/workingset.c b/mm/workingset.c index 50d53f3699e4..2c707c92dd89 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -445,12 +445,12 @@ void workingset_update_node(struct xa_node *node) if (node->count && node->count == node->nr_values) { if (list_empty(&node->private_list)) { list_lru_add(&shadow_nodes, &node->private_list); - __inc_lruvec_slab_state(node, WORKINGSET_NODES); + __inc_lruvec_kmem_state(node, WORKINGSET_NODES); } } else { if (!list_empty(&node->private_list)) { list_lru_del(&shadow_nodes, &node->private_list); - __dec_lruvec_slab_state(node, WORKINGSET_NODES); + __dec_lruvec_kmem_state(node, WORKINGSET_NODES); } } } @@ -541,7 +541,7 @@ static enum lru_status shadow_lru_isolate(struct list_head *item, } list_lru_isolate(lru, item); - __dec_lruvec_slab_state(node, WORKINGSET_NODES); + __dec_lruvec_kmem_state(node, WORKINGSET_NODES); spin_unlock(lru_lock); @@ -564,7 +564,7 @@ static enum lru_status shadow_lru_isolate(struct list_head *item, * shadow entries we were tracking ... */ xas_store(&xas, NULL); - __inc_lruvec_slab_state(node, WORKINGSET_NODERECLAIM); + __inc_lruvec_kmem_state(node, WORKINGSET_NODERECLAIM); out_invalid: xa_unlock_irq(&mapping->i_pages); From patchwork Tue Oct 27 08:02:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11859635 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4EE7B6A2 for ; Tue, 27 Oct 2020 08:03:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DADA021D24 for ; Tue, 27 Oct 2020 08:03:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=bytedance-com.20150623.gappssmtp.com header.i=@bytedance-com.20150623.gappssmtp.com header.b="PJog1NMt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DADA021D24 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0F9C16B0070; Tue, 27 Oct 2020 04:03:49 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0D0266B0071; Tue, 27 Oct 2020 04:03:49 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F03C16B0072; Tue, 27 Oct 2020 04:03:48 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0023.hostedemail.com [216.40.44.23]) by kanga.kvack.org (Postfix) with ESMTP id C18C36B0070 for ; Tue, 27 Oct 2020 04:03:48 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 604FB8249980 for ; Tue, 27 Oct 2020 08:03:48 +0000 (UTC) X-FDA: 77416966536.20.joke13_3b17e452727a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id 39EC2180C07A3 for ; Tue, 27 Oct 2020 08:03:48 +0000 (UTC) X-Spam-Summary: 1,0,0,60d471ac2f5c77ba,d41d8cd98f00b204,songmuchun@bytedance.com,,RULES_HIT:41:355:379:541:800:960:966:973:988:989:1260:1311:1314:1345:1359:1437:1515:1534:1541:1711:1730:1747:1777:1792:2196:2199:2393:2559:2562:2895:3138:3139:3140:3141:3142:3352:3865:3866:3867:3871:3872:4321:4385:5007:6261:6653:6737:8603:10004:11026:11473:11658:11914:12048:12296:12297:12438:12517:12519:12555:12679:12895:12986:13069:13161:13229:13311:13357:13894:14096:14181:14384:14394:14721:21080:21444:21451:21627:21740:30054:30070,0,RBL:209.85.214.194:@bytedance.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201;04ygkoqq73zbb11z7e3xqoeirueimocdr1qf13tsaqupjf99h7rh85433diacgg.8t118q1qw3ktrnruzywh1puc6tjsuzhwzkttb5fh5tsmtwdibz9c1m7fpcgka54.r-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: joke13_3b17e452727a X-Filterd-Recvd-Size: 4703 Received: from mail-pl1-f194.google.com (mail-pl1-f194.google.com [209.85.214.194]) by imf16.hostedemail.com (Postfix) with ESMTP for ; Tue, 27 Oct 2020 08:03:47 +0000 (UTC) Received: by mail-pl1-f194.google.com with SMTP id x10so327218plm.13 for ; Tue, 27 Oct 2020 01:03:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LN3286O1QAMAK+WN2hg/G+SRa6WlZWbQzoibyYb4wWo=; b=PJog1NMte5d9AcJQ2N25yRGouMltF5NBzhqBTv5dYQmj59PEpXsswhsrU0LQ+VWVfn aLdhc9z+7+C3pAXFkSsi/4lSgs4MYKwF7uVxVl+NsaEQOQzS04XB6RLzkTkl9RkPB0Oq SkIb23JF1hsg16qTRBKdvWjTQabtx+3Q9R5e4k+VNRf43+TfAiWbkEICxD3t3+7NB7p7 L8yLqDncE2eSZPNsDjZSHXJ2Wv2O3UCu1Gk29TEbivdr3CCZBt8Zstueo74xQ/0viaBY EDV9Pv6BY3iPIabSdepfkGEm9H2JQlMqBjsX0yyAsdIyNWdN466cnPESkPb7kZHS7hKf LJSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LN3286O1QAMAK+WN2hg/G+SRa6WlZWbQzoibyYb4wWo=; b=HGUc7xKyIk4RSz/3ZlT709YE+jgGzBxRVssVXiS0Xu4a2ArEF7kff/VRwLFkX1J8s9 9mFdLwxSY2f1BmM8orgKApo++f1KrSF1zXe2Fx7GhTIlVWukcktgF/ZsiyuufA0CdfGZ wiPWB6c+rUCJ4KljkBqMVQ1RHRDoVRax1JbEP1ks0oMGva17T//zRy34JrElW7FpZfDC 52zO5VxSCMX399CRPSkPYQ7bcJ5KwbXGB0F9SKq9SGP8zJG7PKqzxQ2v2mJMaHHDUuw4 4BsxEr7luMlGbZXX1QdASsuLFrPUsMyLyk4snAaWkDTfC/xNDKKQKeYC+VOGw8zyxn8C uyLQ== X-Gm-Message-State: AOAM530HyQmP63qbN8eY+lcUZeiUDooTZ78h87L6x5/uMAM2UzAEdBgk BK94voXZwfbxS5sEKlROuJU24w== X-Google-Smtp-Source: ABdhPJy+4tj9XYz/K9k/YpdyvUdWeb1R34jmZQ+GTNvy94neoHvWyQPy95s/Z5NTlj+jeS4gGAA20g== X-Received: by 2002:a17:90a:9a1:: with SMTP id 30mr971273pjo.85.1603785826642; Tue, 27 Oct 2020 01:03:46 -0700 (PDT) Received: from Smcdef-MBP.local.net ([103.136.220.89]) by smtp.gmail.com with ESMTPSA id p8sm1039580pgs.34.2020.10.27.01.03.38 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Oct 2020 01:03:46 -0700 (PDT) From: Muchun Song To: hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, akpm@linux-foundation.org, shakeelb@google.com, guro@fb.com, iamjoonsoo.kim@lge.com, laoar.shao@gmail.com, chris@chrisdown.name, christian.brauner@ubuntu.com, peterz@infradead.org, mingo@kernel.org, keescook@chromium.org, tglx@linutronix.de, esyr@redhat.com, surenb@google.com, areber@redhat.com, elver@google.com Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH 4/5] mm: memcg/slab: Fix root memcg vmstats Date: Tue, 27 Oct 2020 16:02:55 +0800 Message-Id: <20201027080256.76497-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201027080256.76497-1-songmuchun@bytedance.com> References: <20201027080256.76497-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If we reparent the slab objects to the root memcg, when we free the slab object, we need to update the per-memcg vmstats to keep it correct for the root memcg. Now this at least affects the vmstat of NR_KERNEL_STACK_KB for !CONFIG_VMAP_STACK when the thread stack size is smaller than the PAGE_SIZE. Fixes: ec9f02384f60 ("mm: workingset: fix vmstat counters for shadow nodes") Signed-off-by: Muchun Song Acked-by: Roman Gushchin --- mm/memcontrol.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 22b4fb941b54..70345b15b150 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -875,8 +875,13 @@ void __mod_lruvec_kmem_state(void *p, enum node_stat_item idx, int val) rcu_read_lock(); memcg = mem_cgroup_from_obj(p); - /* Untracked pages have no memcg, no lruvec. Update only the node */ - if (!memcg || memcg == root_mem_cgroup) { + /* + * Untracked pages have no memcg, no lruvec. Update only the + * node. If we reparent the slab objects to the root memcg, + * when we free the slab object, we need to update the per-memcg + * vmstats to keep it correct for the root memcg. + */ + if (!memcg) { __mod_node_page_state(pgdat, idx, val); } else { lruvec = mem_cgroup_lruvec(memcg, pgdat); From patchwork Tue Oct 27 08:02:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 11859637 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 352356A2 for ; Tue, 27 Oct 2020 08:03:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D6F81207BB for ; Tue, 27 Oct 2020 08:03:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=bytedance-com.20150623.gappssmtp.com header.i=@bytedance-com.20150623.gappssmtp.com header.b="P+Nz2tG9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D6F81207BB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0AC336B0071; Tue, 27 Oct 2020 04:03:57 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 05B4C6B0072; Tue, 27 Oct 2020 04:03:57 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E8C0A6B0073; Tue, 27 Oct 2020 04:03:56 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0161.hostedemail.com [216.40.44.161]) by kanga.kvack.org (Postfix) with ESMTP id BE8586B0071 for ; Tue, 27 Oct 2020 04:03:56 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 59BBE1EE6 for ; Tue, 27 Oct 2020 08:03:56 +0000 (UTC) X-FDA: 77416966872.18.neck64_0c030f02727a Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin18.hostedemail.com (Postfix) with ESMTP id 4A98E100ED0FC for ; Tue, 27 Oct 2020 08:03:56 +0000 (UTC) X-Spam-Summary: 1,0,0,2fa91a600cd4714e,d41d8cd98f00b204,songmuchun@bytedance.com,,RULES_HIT:41:69:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1437:1515:1535:1544:1605:1711:1730:1747:1777:1792:2194:2199:2393:2559:2562:2890:2901:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:4042:4118:4250:4321:4605:5007:6261:6653:6737:7875:9010:9592:10004:11026:11658:11914:12043:12048:12114:12291:12296:12297:12438:12517:12519:12555:12683:12895:13894:14096:14110:14181:14394:14721:21080:21444:21450:21451:21627:21966:21990:30001:30012:30054,0,RBL:209.85.215.193:@bytedance.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201;04y8skxze44tibewxjfxpsjf5aiyjycmoeafw17uhd6q5wmu95hskmsnyt7x6wo.h8swa5urbf9twb84kpgbogbyyqbec51yxqhhemms6jw67afjqyr3uyf1urfzf1o.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: neck64_0c030f02727a X-Filterd-Recvd-Size: 7666 Received: from mail-pg1-f193.google.com (mail-pg1-f193.google.com [209.85.215.193]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Tue, 27 Oct 2020 08:03:55 +0000 (UTC) Received: by mail-pg1-f193.google.com with SMTP id g12so317224pgm.8 for ; Tue, 27 Oct 2020 01:03:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uiglRpBNWkrLFEUzSubzu7HQqvI3OKCdlqP2Lx2y8is=; b=P+Nz2tG9WB/qhj+FCO+gcFTyNF1I6MsxBKRw9kmsx8p6kTqOEUDww4WezfUMzA7ids ac0Zw4GET/vXhxCf8QG68dkLY5mwIlJRUt9Gm+jLihmcX0NoOz+Cp5twgR8DGsYW67qd PZLKcV4nEUGWtiN0RpyT1yxo4Vpm4Vkn4poFL4T2cKC8G6zrJxeA05Boleywdh/UF3ZC mRUnhoac0MFVouztQxbXWFTjsi/iFZUkv7BWXoxoXhK2tMu9Ey2RFTz7zACV5EhVfAwX vnFuZ5csx1UlaEVc6ELswrsJ5ZO+KZAEF/R9wChhpmuL/z63EgKtfFu5WfRU6MxRFo7g u0PA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uiglRpBNWkrLFEUzSubzu7HQqvI3OKCdlqP2Lx2y8is=; b=lwbj9xGc5JMJVHyoYx3VZQaogFTrMgX9jYEh3jjGBx511LddAVxONsiILN6NjVNZib QOj6Wb2+Geum3w6k/M2xYkzB0Odgib1ZObPwokC+15XxRfJmXK2U0g37AzfaJJNJ4ATb 2wAqpD69MjYOlK88duIeT7OHfcjC91M4ZdsLTJlAx/6zVFG3IDEsEs7ao9vrR7uRC5tO VrjHwU2JaI11xcU/sW5ZQBwbgrf8G9L9ls0s5AkF3UNZ30Ia7pCI6hOfNlqlUBaEQUQW 0AjWsb43Uy6on0Pvxm3hMe0n5YtDYwYjz36s7RWsAvGuE3iB2kReUcNY2qiP6jF6szga +6Mw== X-Gm-Message-State: AOAM532uXtVhnDIcA3fE8OV9liZc65Ucxp+GVtfssvLRtQ1j2+sLViTT JJ7xEs7W1OwddW0jD3tz9/HB7Q== X-Google-Smtp-Source: ABdhPJxqNnFSCS9aSZpbv6+4QVNMOZB8yeIGk564y0BvOrg9NvzTp9pcjm2oymtvYwf9KBriDCrUgQ== X-Received: by 2002:aa7:91c8:0:b029:155:c7c1:3fae with SMTP id z8-20020aa791c80000b0290155c7c13faemr1060014pfa.74.1603785834731; Tue, 27 Oct 2020 01:03:54 -0700 (PDT) Received: from Smcdef-MBP.local.net ([103.136.220.89]) by smtp.gmail.com with ESMTPSA id p8sm1039580pgs.34.2020.10.27.01.03.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 27 Oct 2020 01:03:54 -0700 (PDT) From: Muchun Song To: hannes@cmpxchg.org, mhocko@kernel.org, vdavydov.dev@gmail.com, akpm@linux-foundation.org, shakeelb@google.com, guro@fb.com, iamjoonsoo.kim@lge.com, laoar.shao@gmail.com, chris@chrisdown.name, christian.brauner@ubuntu.com, peterz@infradead.org, mingo@kernel.org, keescook@chromium.org, tglx@linutronix.de, esyr@redhat.com, surenb@google.com, areber@redhat.com, elver@google.com Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Muchun Song Subject: [PATCH 5/5] mm: memcontrol: Simplify the mem_cgroup_page_lruvec Date: Tue, 27 Oct 2020 16:02:56 +0800 Message-Id: <20201027080256.76497-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20201027080256.76497-1-songmuchun@bytedance.com> References: <20201027080256.76497-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We can reuse the code of mem_cgroup_lruvec() to simplify the code of the mem_cgroup_page_lruvec(). Signed-off-by: Muchun Song --- include/linux/memcontrol.h | 44 +++++++++++++++++++++++++++----------- mm/memcontrol.c | 40 ---------------------------------- 2 files changed, 32 insertions(+), 52 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 95807bf6be64..5e8480e54cd8 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -451,16 +451,9 @@ mem_cgroup_nodeinfo(struct mem_cgroup *memcg, int nid) return memcg->nodeinfo[nid]; } -/** - * mem_cgroup_lruvec - get the lru list vector for a memcg & node - * @memcg: memcg of the wanted lruvec - * - * Returns the lru list vector holding pages for a given @memcg & - * @node combination. This can be the node lruvec, if the memory - * controller is disabled. - */ -static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg, - struct pglist_data *pgdat) +static inline struct lruvec *mem_cgroup_node_lruvec(struct mem_cgroup *memcg, + struct pglist_data *pgdat, + int nid) { struct mem_cgroup_per_node *mz; struct lruvec *lruvec; @@ -473,7 +466,7 @@ static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg, if (!memcg) memcg = root_mem_cgroup; - mz = mem_cgroup_nodeinfo(memcg, pgdat->node_id); + mz = mem_cgroup_nodeinfo(memcg, nid); lruvec = &mz->lruvec; out: /* @@ -486,7 +479,34 @@ static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg, return lruvec; } -struct lruvec *mem_cgroup_page_lruvec(struct page *, struct pglist_data *); +/** + * mem_cgroup_lruvec - get the lru list vector for a memcg & node + * @memcg: memcg of the wanted lruvec + * + * Returns the lru list vector holding pages for a given @memcg & + * @node combination. This can be the node lruvec, if the memory + * controller is disabled. + */ +static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg, + struct pglist_data *pgdat) +{ + return mem_cgroup_node_lruvec(memcg, pgdat, pgdat->node_id); +} + +/** + * mem_cgroup_page_lruvec - return lruvec for isolating/putting an LRU page + * @page: the page + * @pgdat: pgdat of the page + * + * This function relies on page->mem_cgroup being stable - see the + * access rules in commit_charge(). + */ +static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page, + struct pglist_data *pgdat) +{ + return mem_cgroup_node_lruvec(page->mem_cgroup, pgdat, + page_to_nid(page)); +} struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 70345b15b150..7097f3fc4dee 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1332,46 +1332,6 @@ int mem_cgroup_scan_tasks(struct mem_cgroup *memcg, return ret; } -/** - * mem_cgroup_page_lruvec - return lruvec for isolating/putting an LRU page - * @page: the page - * @pgdat: pgdat of the page - * - * This function relies on page->mem_cgroup being stable - see the - * access rules in commit_charge(). - */ -struct lruvec *mem_cgroup_page_lruvec(struct page *page, struct pglist_data *pgdat) -{ - struct mem_cgroup_per_node *mz; - struct mem_cgroup *memcg; - struct lruvec *lruvec; - - if (mem_cgroup_disabled()) { - lruvec = &pgdat->__lruvec; - goto out; - } - - memcg = page->mem_cgroup; - /* - * Swapcache readahead pages are added to the LRU - and - * possibly migrated - before they are charged. - */ - if (!memcg) - memcg = root_mem_cgroup; - - mz = mem_cgroup_page_nodeinfo(memcg, page); - lruvec = &mz->lruvec; -out: - /* - * Since a node can be onlined after the mem_cgroup was created, - * we have to be prepared to initialize lruvec->zone here; - * and if offlined then reonlined, we need to reinitialize it. - */ - if (unlikely(lruvec->pgdat != pgdat)) - lruvec->pgdat = pgdat; - return lruvec; -} - /** * mem_cgroup_update_lru_size - account for adding or removing an lru page * @lruvec: mem_cgroup per zone lru vector