From patchwork Wed May 1 17:26:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13651039 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB528C25B4F for ; Wed, 1 May 2024 17:26:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2C50A6B008C; Wed, 1 May 2024 13:26:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 22DD16B0095; Wed, 1 May 2024 13:26:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED5946B0092; Wed, 1 May 2024 13:26:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id C52B96B008A for ; Wed, 1 May 2024 13:26:33 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 41D6814096C for ; Wed, 1 May 2024 17:26:33 +0000 (UTC) X-FDA: 82070506266.16.AD4B988 Received: from out-186.mta0.migadu.com (out-186.mta0.migadu.com [91.218.175.186]) by imf03.hostedemail.com (Postfix) with ESMTP id D051920049 for ; Wed, 1 May 2024 17:26:29 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=tIfgRTnl; spf=pass (imf03.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.186 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714584390; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=d5S9dkOJqha+rpI1CHtwyB9wbo4t9Wc2EBy5O17ehws=; b=l08xoFlAQzy6LqQkIhsh+5OoZYBreNrw7DPZ/vDN8xT04yvhfALdipe2om8NIwXepup5r8 GJfuSvYlCx/YIPJtRFF/xqad0ZJrVs+yevpoYp7pzUc2a5z/mjlB1IEuroAp/29AJDvPAI tLjbRrtXqqoZLm4+JDmuX1tz97y9wWM= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=tIfgRTnl; spf=pass (imf03.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.186 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714584390; a=rsa-sha256; cv=none; b=JHdvtzTUTLt8ov1Hd55YpZ0Efyy///DMyaEoMIla7SMmPxowaKbyGqAe4gPZBGX+9d4n4A wAvo998YAOunNX9TLVKl9XzL+ZG4fi8pZCudH5HEwSCu3uQfS6lz/KvZyJDJBpVm/9Z8S1 OTp0DEUiVCDYyf9/CGUIgvXO4ef14Fw= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714584387; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d5S9dkOJqha+rpI1CHtwyB9wbo4t9Wc2EBy5O17ehws=; b=tIfgRTnl/mhCgo0S97RFAXjLL/WVu5BE3Xyfq+9Mm73A1FvvUmiRo1lxo7b4Xy3VsjEWSv x7EoZ6cGnTqWTDf9G9b6HpQZYMpFZEEwHjoreUm7T9cMzcP9nw8sl2uOSArKJ1uG3TCahI /Jtd6f/SoU72qn5FtRtSozJCOvm6c/w= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 1/8] memcg: reduce memory size of mem_cgroup_events_index Date: Wed, 1 May 2024 10:26:10 -0700 Message-ID: <20240501172617.678560-2-shakeel.butt@linux.dev> In-Reply-To: <20240501172617.678560-1-shakeel.butt@linux.dev> References: <20240501172617.678560-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Stat-Signature: n5yd68eh6cy8h5asry8xw9zbdwzsxgn4 X-Rspam-User: X-Rspamd-Queue-Id: D051920049 X-Rspamd-Server: rspam05 X-HE-Tag: 1714584389-13334 X-HE-Meta: U2FsdGVkX19UgfjsdgBd6FCoxjKyA9c9NH7bwuTivu6oInlrRjlx2Rg+qX1T6DwyHMHeyruISZriQYlHes8YcHtf0e1DiuSCmSFTJX8leSweQO9YnowvZoYgz3qFaszlsLnlXQ4g05G7r61L+e2F13jWShOo+GXFPlRRmuDLp+4Olkf0KtxTEIe1g7vEhcXCObr7+LzP0UJ8cjpTALryVOry/L3N+zV94KHbNRfTrBOG6ioHA7C7fskAJm9Egf6SpQkphJzc7sNIu4zQdr47El5cwB8MW5EEBBpUaBrLgr6mL/+eVcxHY0fbcYDmQXKNxZOHpVf/52eH20WufZWtmaYpqfokvYSNAVa5s7Kk/AC8LVub3wIvS4Jano1qjQz2vnrFemAdZ4nUGmrdAkKgimGA6gnffBiPf2zl8KSLgfAcsjL5rOGCPzVlbDWezAyk3Tu/49pDCnOsYumou/O4SAPW7njHhAKVaoYGUPTSzj9pzLIEOBxpjbH7UxWefzvCTWlUhFwnKbGyh5ZP1gj872IoOXWhHHO6VlNanK0SrUawk3aHRT1VnSvrdy9aq/+WnQj/TWX45p6TaGMQ6w73BpVnE5fhVgExXWZ55cwz9xO1R5DS+zmWvO8NsOqQnd6TOyvLFakmM3fSB62UkNqIRWVYrOArIayGnK0SzZrcS6wGp8XKpv6cxjrJ9SG6/Xd1RaTolWcuqv1+2l4c6DsZE6KHxpvqwTvagACsEbTISyxv0otVQlONH3r8ifBAljraeLETchKJMtrg+YQ6nuI3ICcov+HiLQe4KfMnXvmQkLF+oiailj5On0TAa2UMHBd2x/tssDqwH83v1xA5RJufwpmTdnaEjpTwaTNYhp7joBoXF/BNfSQp2lTLDRkGmEaAFdzig1nR+YiVX/Vkah7gTGxeESN+c7Jg3vTYilxEMGbS+OKnXm0llx5LBnDx2G7aAAos/yQEzUnwnsLEHHw pAqi6+1s YGlrK7EkVtMnDVzsklC1R7q3GsRKMTNo9OSxbMuTR2d42y3oHutR6eSNm0KSfudPKaQf1d940Uuof6d905vM9l7CHfm7uS68f0Q/8vUqx0IGsbAQTkn2zj9NI1cfbWqwcZ3rl2+E6cPU/WrnXNbvX0XiFWA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: mem_cgroup_events_index is a translation table to get the right index of the memcg relevant entry for the general vm_event_item. At the moment, it is defined as integer array. However on a typical system the max entry of vm_event_item (NR_VM_EVENT_ITEMS) is 113, so we don't need to use int as storage type of the array. For now just use int8_t as type and add a BUILD_BUG_ON(). Another benefit of this change is that the translation table fits in 2 cachelines while previously it would require 8 cachelines (assuming 64 bytes cacheline). Signed-off-by: Shakeel Butt Reviewed-by: Roman Gushchin Reviewed-by: Yosry Ahmed Reviewed-by: T.J. Mercier --- Changes since v3: - N/A mm/memcontrol.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 602ad5faad4d..c146187cda9c 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -607,11 +607,13 @@ static const unsigned int memcg_vm_event_stat[] = { }; #define NR_MEMCG_EVENTS ARRAY_SIZE(memcg_vm_event_stat) -static int mem_cgroup_events_index[NR_VM_EVENT_ITEMS] __read_mostly; +static int8_t mem_cgroup_events_index[NR_VM_EVENT_ITEMS] __read_mostly; static void init_memcg_events(void) { - int i; + int8_t i; + + BUILD_BUG_ON(NR_VM_EVENT_ITEMS >= S8_MAX); for (i = 0; i < NR_MEMCG_EVENTS; ++i) mem_cgroup_events_index[memcg_vm_event_stat[i]] = i + 1; From patchwork Wed May 1 17:26:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13651040 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF2B8C4345F for ; Wed, 1 May 2024 17:26:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C18736B0092; Wed, 1 May 2024 13:26:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B7A586B0093; Wed, 1 May 2024 13:26:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F4EB6B0095; Wed, 1 May 2024 13:26:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7C2A36B0092 for ; Wed, 1 May 2024 13:26:35 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1DDB940BAD for ; Wed, 1 May 2024 17:26:35 +0000 (UTC) X-FDA: 82070506350.04.A5A0D11 Received: from out-181.mta0.migadu.com (out-181.mta0.migadu.com [91.218.175.181]) by imf08.hostedemail.com (Postfix) with ESMTP id 96FB5160005 for ; Wed, 1 May 2024 17:26:33 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=oO8ERjmx; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf08.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.181 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714584393; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fBg/xQS/MIq7cEhcQ96zq9LHH7YtwZWaDugtQaMbEjA=; b=Ao+9Nc0Kl09FeL1oOVnrUIRCiUtt7cPI+9samMNvwzhbAtxcygPhkFXDvKTtk5s08Vm1ki D18JUlnQKqwWF7LJjK9F8JwU1ZzKMHnJOcI4Sdmh1PcUU08dhg82EVZGd/D1uS1ICyeiH3 TK5sNhNIIOxuuqV2rrAEHrkrg41Fspw= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=oO8ERjmx; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf08.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.181 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714584393; a=rsa-sha256; cv=none; b=uQy6Ldxo84/VUqij6o20+x5M+q10CWkEsZ9/3xFo3cK/emV1Ipke2449u2DsvgCBabIduh t7EmwXMvInKQ5tgsoWbDIeDS72DED9UsAwlHMSfapM45dIhuW3l26TphPbNDjwvqNP10lp N92iCEklX7EJ0nOCuwIQt/eK0UvFQig= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714584392; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fBg/xQS/MIq7cEhcQ96zq9LHH7YtwZWaDugtQaMbEjA=; b=oO8ERjmxEmOnQNxm4RLisy/xS7iu1hwJijzbGr5XjDAhgynlzk14K/ZUCNh7l6xdxVVh6i VmXZzO2NisPVHuroxULEK/AH/2QxNLCkvwgMItOxL8K00gWQGYiU1iY2ubooM4yA3Vat5A gsvk25slnPnudsqUwOVnqs+0FRq83OQ= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 2/8] memcg: dynamically allocate lruvec_stats Date: Wed, 1 May 2024 10:26:11 -0700 Message-ID: <20240501172617.678560-3-shakeel.butt@linux.dev> In-Reply-To: <20240501172617.678560-1-shakeel.butt@linux.dev> References: <20240501172617.678560-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 96FB5160005 X-Stat-Signature: 9spbsjo6zofxgc8mn8hxmuu6uc9so319 X-Rspam-User: X-HE-Tag: 1714584393-941011 X-HE-Meta: U2FsdGVkX1/nyEtqra0dX3f1balpKqUDss+irDhmZFjDhXGsXmfu+7MGO1LLK3szOUmG/wvSKfSExGogk8nT6491uq/XroZ8k1bxIZIR2K2e9UY60BExRxMMPSNjoq2mxSWRVoh+BVI30uzcF79pYjKhM8GOMB821a5jiHIm508Iv0/3BGu5eAVSZw0hgKyQyK5qciN1qYQKcQn1vHladYxWRP/WA0C+5H+Kyv2mMiyAXDpVW+OxaHDjvitQhQLQen34qjkTgep7ub3CBrmg+ury4ppq5Uucf+Qe1DJJSNKITTaWnoaz2zmxdpl6tOXyBcoUH4giZy7utlpOhZ0H4vt59LYiEcpC6KakiZAZUSxeGPgrdvG5DLkAhuQwr6Jl3Hw1b8M7Wc0gFramREKa8hiQi5SnGFzSvovDnNguqH1QlWCP4yKmEzjxlCYVNXeljT7a6ugtT0jkGLePwAYsKEEVcjCtRQEZeBFo2IQPVBEuzDRbGJKozbr7ro5AGUTZFrTO1O+pHprR/ebvb7mruquNEcGQ5cksaSQLfdOCD4c2ovZ8oz0Qsnw/WiHoaiTepCyhf9SiaXhk4aEybr3YRJpEO7G7vzSveJ4gnsY++ddtISGoYkC4PPhzvsBW1ijBr9vQiePOLGg8vY/SWMhRoMrba5oJ+pYwtXeYZY7IwdHhb5J26RbYGPXcA2cCSIN2YQ/3WTVDKw4P17zyEVZtEWbo9ppBCvfDGKsAaE1nq7b7PtbjPmt6kxpx4+sxOqgF/qtmCHi4cMfieRtGrFqWu2S6sO/YU9mUKxGrPfqDXKk6VdsfxI4M3HEaOTImdnabRuy1tOQudxhX9G5oXFLoUcltpYNnXkMXCYEeLCFjMHO32GuvChATWoY/7Rjoi3c0AwlcaSDk7WM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To decouple the dependency of lruvec_stats on NR_VM_NODE_STAT_ITEMS, we need to dynamically allocate lruvec_stats in the mem_cgroup_per_node structure. Also move the definition of lruvec_stats_percpu and lruvec_stats and related functions to the memcontrol.c to facilitate later patches. No functional changes in the patch. Signed-off-by: Shakeel Butt Reviewed-by: Yosry Ahmed Reviewed-by: T.J. Mercier Reviewed-by: Roman Gushchin --- Changes since v3: - N/A include/linux/memcontrol.h | 62 +++------------------------ mm/memcontrol.c | 87 ++++++++++++++++++++++++++++++++------ 2 files changed, 81 insertions(+), 68 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 9aba0d0462ca..ab8a6e884375 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -83,6 +83,8 @@ enum mem_cgroup_events_target { struct memcg_vmstats_percpu; struct memcg_vmstats; +struct lruvec_stats_percpu; +struct lruvec_stats; struct mem_cgroup_reclaim_iter { struct mem_cgroup *position; @@ -90,25 +92,6 @@ struct mem_cgroup_reclaim_iter { unsigned int generation; }; -struct lruvec_stats_percpu { - /* Local (CPU and cgroup) state */ - long state[NR_VM_NODE_STAT_ITEMS]; - - /* Delta calculation for lockless upward propagation */ - long state_prev[NR_VM_NODE_STAT_ITEMS]; -}; - -struct lruvec_stats { - /* Aggregated (CPU and subtree) state */ - long state[NR_VM_NODE_STAT_ITEMS]; - - /* Non-hierarchical (CPU aggregated) state */ - long state_local[NR_VM_NODE_STAT_ITEMS]; - - /* Pending child counts during tree propagation */ - long state_pending[NR_VM_NODE_STAT_ITEMS]; -}; - /* * per-node information in memory controller. */ @@ -116,7 +99,7 @@ struct mem_cgroup_per_node { struct lruvec lruvec; struct lruvec_stats_percpu __percpu *lruvec_stats_percpu; - struct lruvec_stats lruvec_stats; + struct lruvec_stats *lruvec_stats; unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; @@ -1037,42 +1020,9 @@ static inline void mod_memcg_page_state(struct page *page, } unsigned long memcg_page_state(struct mem_cgroup *memcg, int idx); - -static inline unsigned long lruvec_page_state(struct lruvec *lruvec, - enum node_stat_item idx) -{ - struct mem_cgroup_per_node *pn; - long x; - - if (mem_cgroup_disabled()) - return node_page_state(lruvec_pgdat(lruvec), idx); - - pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x = READ_ONCE(pn->lruvec_stats.state[idx]); -#ifdef CONFIG_SMP - if (x < 0) - x = 0; -#endif - return x; -} - -static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec, - enum node_stat_item idx) -{ - struct mem_cgroup_per_node *pn; - long x = 0; - - if (mem_cgroup_disabled()) - return node_page_state(lruvec_pgdat(lruvec), idx); - - pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x = READ_ONCE(pn->lruvec_stats.state_local[idx]); -#ifdef CONFIG_SMP - if (x < 0) - x = 0; -#endif - return x; -} +unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx); +unsigned long lruvec_page_state_local(struct lruvec *lruvec, + enum node_stat_item idx); void mem_cgroup_flush_stats(struct mem_cgroup *memcg); void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c146187cda9c..7126459ec56a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -576,6 +576,60 @@ mem_cgroup_largest_soft_limit_node(struct mem_cgroup_tree_per_node *mctz) return mz; } +struct lruvec_stats_percpu { + /* Local (CPU and cgroup) state */ + long state[NR_VM_NODE_STAT_ITEMS]; + + /* Delta calculation for lockless upward propagation */ + long state_prev[NR_VM_NODE_STAT_ITEMS]; +}; + +struct lruvec_stats { + /* Aggregated (CPU and subtree) state */ + long state[NR_VM_NODE_STAT_ITEMS]; + + /* Non-hierarchical (CPU aggregated) state */ + long state_local[NR_VM_NODE_STAT_ITEMS]; + + /* Pending child counts during tree propagation */ + long state_pending[NR_VM_NODE_STAT_ITEMS]; +}; + +unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx) +{ + struct mem_cgroup_per_node *pn; + long x; + + if (mem_cgroup_disabled()) + return node_page_state(lruvec_pgdat(lruvec), idx); + + pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); + x = READ_ONCE(pn->lruvec_stats->state[idx]); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; +} + +unsigned long lruvec_page_state_local(struct lruvec *lruvec, + enum node_stat_item idx) +{ + struct mem_cgroup_per_node *pn; + long x = 0; + + if (mem_cgroup_disabled()) + return node_page_state(lruvec_pgdat(lruvec), idx); + + pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); + x = READ_ONCE(pn->lruvec_stats->state_local[idx]); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; +} + /* Subset of vm_event_item to report for memcg event stats */ static const unsigned int memcg_vm_event_stat[] = { PGPGIN, @@ -5491,18 +5545,25 @@ static int alloc_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) if (!pn) return 1; + pn->lruvec_stats = kzalloc_node(sizeof(struct lruvec_stats), GFP_KERNEL, + node); + if (!pn->lruvec_stats) + goto fail; + pn->lruvec_stats_percpu = alloc_percpu_gfp(struct lruvec_stats_percpu, GFP_KERNEL_ACCOUNT); - if (!pn->lruvec_stats_percpu) { - kfree(pn); - return 1; - } + if (!pn->lruvec_stats_percpu) + goto fail; lruvec_init(&pn->lruvec); pn->memcg = memcg; memcg->nodeinfo[node] = pn; return 0; +fail: + kfree(pn->lruvec_stats); + kfree(pn); + return 1; } static void free_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) @@ -5513,6 +5574,7 @@ static void free_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) return; free_percpu(pn->lruvec_stats_percpu); + kfree(pn->lruvec_stats); kfree(pn); } @@ -5865,18 +5927,19 @@ static void mem_cgroup_css_rstat_flush(struct cgroup_subsys_state *css, int cpu) for_each_node_state(nid, N_MEMORY) { struct mem_cgroup_per_node *pn = memcg->nodeinfo[nid]; - struct mem_cgroup_per_node *ppn = NULL; + struct lruvec_stats *lstats = pn->lruvec_stats; + struct lruvec_stats *plstats = NULL; struct lruvec_stats_percpu *lstatc; if (parent) - ppn = parent->nodeinfo[nid]; + plstats = parent->nodeinfo[nid]->lruvec_stats; lstatc = per_cpu_ptr(pn->lruvec_stats_percpu, cpu); for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { - delta = pn->lruvec_stats.state_pending[i]; + delta = lstats->state_pending[i]; if (delta) - pn->lruvec_stats.state_pending[i] = 0; + lstats->state_pending[i] = 0; delta_cpu = 0; v = READ_ONCE(lstatc->state[i]); @@ -5887,12 +5950,12 @@ static void mem_cgroup_css_rstat_flush(struct cgroup_subsys_state *css, int cpu) } if (delta_cpu) - pn->lruvec_stats.state_local[i] += delta_cpu; + lstats->state_local[i] += delta_cpu; if (delta) { - pn->lruvec_stats.state[i] += delta; - if (ppn) - ppn->lruvec_stats.state_pending[i] += delta; + lstats->state[i] += delta; + if (plstats) + plstats->state_pending[i] += delta; } } } From patchwork Wed May 1 17:26:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13651041 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E545C4345F for ; Wed, 1 May 2024 17:26:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A435A6B0098; Wed, 1 May 2024 13:26:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9F7096B0099; Wed, 1 May 2024 13:26:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 894C96B009A; Wed, 1 May 2024 13:26:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6761F6B0098 for ; Wed, 1 May 2024 13:26:42 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 16F7D1405F0 for ; Wed, 1 May 2024 17:26:42 +0000 (UTC) X-FDA: 82070506644.20.239BA23 Received: from out-178.mta1.migadu.com (out-178.mta1.migadu.com [95.215.58.178]) by imf10.hostedemail.com (Postfix) with ESMTP id 03252C0016 for ; Wed, 1 May 2024 17:26:38 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=If080VrU; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf10.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.178 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714584399; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JuviNSLTVsDm2/Um5Hfjfi5K4HkGkzN+hyrTetNNaTI=; b=Ox8oUUlNFgR8EXrp03txx2tG2pRpo8JcEueCwkcpmJFZTox+AndAiiKnaIA9O5NcHjWa+n 4skfQmP8JBj7DR7fLl/yskj1XWtOtjYyHZGg589Q3l9j8oYHeAiZl/90frR6m+oDq9mwpX 3jN+GGFWjyAGKb99D59W53+MBaJ1ANk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714584399; a=rsa-sha256; cv=none; b=sTMpyl0WoVo4DaGBdcXYZXR8I/lbpxexJBdNQYQayBWlbC/c2wQooo2rIN8ahEE/bD9AOD uuyKDWDWCICOrTHLVOKWPXtSy8fvTK0+ckSj6p371HC2OQjOZBawm5xPcpqquy635ErpCF Td175RFHVb1IoWdpgoxAuvJ3pzCRBII= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=If080VrU; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf10.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.178 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714584397; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JuviNSLTVsDm2/Um5Hfjfi5K4HkGkzN+hyrTetNNaTI=; b=If080VrUZeaO5mlZ3lObU0HLh9SFBUBimkTYvPuIkRA/tpnOeALwyjhGWuu/NaO77tLhLx mmezzwbTym0qQ927euj7Kk9h1CWx4JQ0J96wFEZT3cAJNQPrqkIkEpHJ7L8GlkhL/RHPiy KbmHTH6I7hXFRF+V0wq38poukU67Sr0= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 3/8] mm: memcg: account memory used for memcg vmstats and lruvec stats Date: Wed, 1 May 2024 10:26:12 -0700 Message-ID: <20240501172617.678560-4-shakeel.butt@linux.dev> In-Reply-To: <20240501172617.678560-1-shakeel.butt@linux.dev> References: <20240501172617.678560-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 03252C0016 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: 3k45dsfk5n6rshb3x8peyx5pyryzuyxi X-HE-Tag: 1714584398-972462 X-HE-Meta: U2FsdGVkX1/xwhsneVz0gqskQmmqGGtm7gdI+jCN03A9XA0k4UULFlv47HscpF4kjICGAWz2+T5dQzNaSwSl+Z51iNH6lYIrAjs/v7Mbm3JRzVqBFfaQJef7wBLnDiUb8qNe8yjbuYZ59ZzBUpLQdkdU9us9DHWd2ChfYURQyn/xVLKuyHgY6102ORfL+I6AdjzxkuuqZWmeP9g1xr0ic3KWVeJTAt5SZ4dV71bgSMrhYi2Wcll2NaCUcXK95DYfj2EfV+mY87FVuHhbkuh32XPiTFAs1bgNF82wVDSS18GHqdvPMOvr5W1f+Me3wreVZzkUPIH090SZMb0ICPSASFK5auMwZPddkeak2aBtlurJfSCucSXY7DvrUH7RUZU1W15vPaENgjaXfhpa+OujLARcHud41AnzxzgXLmWj5bkHJdmZEgPovMxHXdd5ftXIdcz1yjeEeZAMOsFLHWIZ8XS5bcVJlGTi6yTbZSYkMDnP27Nl2rURtsjps/4yshbThwXAWGFK8pEMkaTZtRop5Je5KAJekGeQPHlLFe7LfDm+c+aOsGSs6JMkJuiZp84wfGqbEYLUVe5oIlDjtustj6V83K3wCcdOkHyBb5UnuTiU4UpKDif4zyH75sq2ffvpmCvZiszHsOGo1FgmnB2aiSeUHyKPG/cXY5GGH0TznyTGFAHR9nnZElV1m/OEoE4LoJjmOJpzejhqOgj/k+xABUeH+gxcogT47hj4sS+SVFehemnF4/10LKg57/d/sQjo1WupK/I7nrHxOSdbserPxniI7LgwydO/Hogv8ZKpG5bqAiofa2vsIZ/czqyLMDR/Uh5qE5aHLxhx8Wuz9o9k2A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Roman Gushchin The percpu memory used by memcg's memory statistics is already accounted. For consistency, let's enable accounting for vmstats and lruvec stats as well. Signed-off-by: Roman Gushchin Signed-off-by: Shakeel Butt Reviewed-by: Yosry Ahmed Reviewed-by: T.J. Mercier --- Changes since v3: - N/A mm/memcontrol.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7126459ec56a..434cff91b65e 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5545,8 +5545,8 @@ static int alloc_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) if (!pn) return 1; - pn->lruvec_stats = kzalloc_node(sizeof(struct lruvec_stats), GFP_KERNEL, - node); + pn->lruvec_stats = kzalloc_node(sizeof(struct lruvec_stats), + GFP_KERNEL_ACCOUNT, node); if (!pn->lruvec_stats) goto fail; @@ -5617,7 +5617,8 @@ static struct mem_cgroup *mem_cgroup_alloc(struct mem_cgroup *parent) goto fail; } - memcg->vmstats = kzalloc(sizeof(struct memcg_vmstats), GFP_KERNEL); + memcg->vmstats = kzalloc(sizeof(struct memcg_vmstats), + GFP_KERNEL_ACCOUNT); if (!memcg->vmstats) goto fail; From patchwork Wed May 1 17:26:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13651042 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D17BC25B4F for ; Wed, 1 May 2024 17:26:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7FCE66B0099; Wed, 1 May 2024 13:26:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7935C6B009A; Wed, 1 May 2024 13:26:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6249C6B009B; Wed, 1 May 2024 13:26:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 41D676B0099 for ; Wed, 1 May 2024 13:26:43 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 08876A0997 for ; Wed, 1 May 2024 17:26:43 +0000 (UTC) X-FDA: 82070506686.17.6719AE7 Received: from out-182.mta1.migadu.com (out-182.mta1.migadu.com [95.215.58.182]) by imf01.hostedemail.com (Postfix) with ESMTP id 4865940022 for ; Wed, 1 May 2024 17:26:41 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=Rx80bQZR; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf01.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.182 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714584401; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=782J2O+/QbH+Qs8ly7QzrLsj4aQaOzuT54G3wfieUlU=; b=pD99rQ3kxLB/SN4fSZ1WkRvs3W9mPIMMeln8SspgYh5bx7xsPaySeRN3PwJ8XQEH9Aw0/9 boyIvNHZYFdhMwoNwnIEqUKim0S0riSf5EiiYHEAD6LjVbOYkLji0YtaXtquvdbUYb/wlD yIVZxgJfsxFXJ3l3quf0p2cT37lmX7k= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=Rx80bQZR; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf01.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.182 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714584401; a=rsa-sha256; cv=none; b=HcEAIISmvRKU8gp1pMZ2cgw5jNYyCtRJz/BZvW9QfT/uDQi6zpnMwYjgi0MzbhZzk8WVGG XZ9EwRMAu7T8BGT3hL26Gx9lV47bnDmWxs8tsm8hXyMf0XB88w7XVVnseuEwVrP+yLqGbd liZlO36jegzb6sLeDZOWYc4a6+r0NMc= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714584399; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=782J2O+/QbH+Qs8ly7QzrLsj4aQaOzuT54G3wfieUlU=; b=Rx80bQZRjPoWmePKWXel0j8/uGt+bp6kwdcwb2X/rVhyNqx0jO2DaXhk8/6dqhqoUXjS1s jfLdgauMcx1tZ+EbNyBv5iMVTW+lYWkKOIURBeLAetgm8Brah2VYMLle6RwU1+VmXFaVSw ns70BirvQu3LMJsoq+bcE6kRjIZIpPU= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 4/8] memcg: reduce memory for the lruvec and memcg stats Date: Wed, 1 May 2024 10:26:13 -0700 Message-ID: <20240501172617.678560-5-shakeel.butt@linux.dev> In-Reply-To: <20240501172617.678560-1-shakeel.butt@linux.dev> References: <20240501172617.678560-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Stat-Signature: 7au1ed6kogbf37zd3gb7cxh6tmb61ztb X-Rspamd-Queue-Id: 4865940022 X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1714584401-931587 X-HE-Meta: U2FsdGVkX18GVd8yTPH2a5OfetLQsq1nTOQ4ze0rBPme6ChjZUwMX+WchK3d2B0nQN26sJIKOtjE9mDexrStx8iXppMbeR4hKvhHqMKTSD8E10rOHUEWQUpKcJbpMM7qBePl7TEyV2FgYFRrbheakq09lQIbgFDjAYKFNtSIWtyanVs8A9iaqR0DcskX/PeMxghQy2/hz3eKluLU7VLb0HlmEN0RzQG7ZXZwk8qyHrfb99lCTJ3jBnqIZLlAIy/BsGF34rydBp7nzpTMTNYf6UW1FDPVpv3qgbaq8QehjlAS/SGUQiJsjagBmT0l1iKC55OIMcwR9xRfPUf15LUOZtWdzeMk6FNLUX10qKXLZowXUoSDsr/by+yc0FkqwiwLI48NrJhFu+Yx0Q8+dwqud0sWVtLfqDaM9S+q5krdpfN89nXJlpt6+/lQ6wJhhWBcscFOprzA+x9Hbl5npJ3+e9IlEtGdOhI0PlkwUnr0kZTtvLXWeoSE48nFzTGj1pSYZYZVZNirP+8Du6nDmakIcrdGbvrswXAj5aNoQAojB7ObqAhvS0lDW+qnDJJzK8OM53rrwhVnRLTs0uLQORRlSUEDcX0VaaCEJkiRrqXykDjXdC1J3+QWOz470xxPeMBjLYoxuCSATXx6YpYehSi4HZLvgzGAfGDk2hVPdsZ3zhLoHiwKoriHZQ2vpkGwFb0bmeoc2NfevwlvcaNkBuWnDjj+J+nOeD4sItn3gKULK24guAsyxYVpL19ziSWyEIb+vBQUPxeTlnZG8w4I5TyrK8VMUNqjJofrBMvpyYkSf93mvjmnxeo7QoYvKnzZ9BXIS9J0Qxp3rYxiKVxC0R9+ne/7hPJPDGzLYVtPIyBVksP2SAURfet0XJYbW7NrZR34C/ZvqCwQ2eOkMN5MdCXCd9lFYv2LhTiuswGjpnY6F4oXvGnCrZ42p4TSKYKmmB/Hrr6TjkhNthzuhlSjwrx 8Lflkx5A u/kisNhFpAz0+A8uL60oHb3eGOfWEoFR3k2T3cQZrqGbxTVuf1GjRYsh76Pjp5mKkGXhmhylT4XwgtEA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: At the moment, the amount of memory allocated for stats related structs in the mem_cgroup corresponds to the size of enum node_stat_item. However not all fields in enum node_stat_item have corresponding memcg stats. So, let's use indirection mechanism similar to the one used for memcg vmstats management. For a given x86_64 config, the size of stats with and without patch is: structs size in bytes w/o with struct lruvec_stats 1128 648 struct lruvec_stats_percpu 752 432 struct memcg_vmstats 1832 1352 struct memcg_vmstats_percpu 1280 960 The memory savings is further compounded by the fact that these structs are allocated for each cpu and for each node. To be precise, for each memcg the memory saved would be: Memory saved = ((21 * 3 * NR_NODES) + (21 * 2 * NR_NODES * NR_CPUS) + (21 * 3) + (21 * 2 * NR_CPUS)) * sizeof(long) Where 21 is the number of fields eliminated. Signed-off-by: Shakeel Butt Reviewed-by: Roman Gushchin --- Changes since v3: - Use S8_MAX instead of 127 (Yosry) - Change the ordering of if conditions (Yosry) - Changed the stat array size macro to MEMCG_VMSTAT_SIZE (Johannes) mm/memcontrol.c | 134 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 114 insertions(+), 20 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 434cff91b65e..f3b6be5a0605 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -576,35 +576,106 @@ mem_cgroup_largest_soft_limit_node(struct mem_cgroup_tree_per_node *mctz) return mz; } +/* Subset of node_stat_item for memcg stats */ +static const unsigned int memcg_node_stat_items[] = { + NR_INACTIVE_ANON, + NR_ACTIVE_ANON, + NR_INACTIVE_FILE, + NR_ACTIVE_FILE, + NR_UNEVICTABLE, + NR_SLAB_RECLAIMABLE_B, + NR_SLAB_UNRECLAIMABLE_B, + WORKINGSET_REFAULT_ANON, + WORKINGSET_REFAULT_FILE, + WORKINGSET_ACTIVATE_ANON, + WORKINGSET_ACTIVATE_FILE, + WORKINGSET_RESTORE_ANON, + WORKINGSET_RESTORE_FILE, + WORKINGSET_NODERECLAIM, + NR_ANON_MAPPED, + NR_FILE_MAPPED, + NR_FILE_PAGES, + NR_FILE_DIRTY, + NR_WRITEBACK, + NR_SHMEM, + NR_SHMEM_THPS, + NR_FILE_THPS, + NR_ANON_THPS, + NR_KERNEL_STACK_KB, + NR_PAGETABLE, + NR_SECONDARY_PAGETABLE, +#ifdef CONFIG_SWAP + NR_SWAPCACHE, +#endif +}; + +static const unsigned int memcg_stat_items[] = { + MEMCG_SWAP, + MEMCG_SOCK, + MEMCG_PERCPU_B, + MEMCG_VMALLOC, + MEMCG_KMEM, + MEMCG_ZSWAP_B, + MEMCG_ZSWAPPED, +}; + +#define NR_MEMCG_NODE_STAT_ITEMS ARRAY_SIZE(memcg_node_stat_items) +#define MEMCG_VMSTAT_SIZE (NR_MEMCG_NODE_STAT_ITEMS + \ + ARRAY_SIZE(memcg_stat_items)) +static int8_t mem_cgroup_stats_index[MEMCG_NR_STAT] __read_mostly; + +static void init_memcg_stats(void) +{ + int8_t i, j = 0; + + BUILD_BUG_ON(MEMCG_NR_STAT >= S8_MAX); + + for (i = 0; i < NR_MEMCG_NODE_STAT_ITEMS; ++i) + mem_cgroup_stats_index[memcg_node_stat_items[i]] = ++j; + + for (i = 0; i < ARRAY_SIZE(memcg_stat_items); ++i) + mem_cgroup_stats_index[memcg_stat_items[i]] = ++j; +} + +static inline int memcg_stats_index(int idx) +{ + return mem_cgroup_stats_index[idx] - 1; +} + struct lruvec_stats_percpu { /* Local (CPU and cgroup) state */ - long state[NR_VM_NODE_STAT_ITEMS]; + long state[NR_MEMCG_NODE_STAT_ITEMS]; /* Delta calculation for lockless upward propagation */ - long state_prev[NR_VM_NODE_STAT_ITEMS]; + long state_prev[NR_MEMCG_NODE_STAT_ITEMS]; }; struct lruvec_stats { /* Aggregated (CPU and subtree) state */ - long state[NR_VM_NODE_STAT_ITEMS]; + long state[NR_MEMCG_NODE_STAT_ITEMS]; /* Non-hierarchical (CPU aggregated) state */ - long state_local[NR_VM_NODE_STAT_ITEMS]; + long state_local[NR_MEMCG_NODE_STAT_ITEMS]; /* Pending child counts during tree propagation */ - long state_pending[NR_VM_NODE_STAT_ITEMS]; + long state_pending[NR_MEMCG_NODE_STAT_ITEMS]; }; unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx) { struct mem_cgroup_per_node *pn; long x; + int i; if (mem_cgroup_disabled()) return node_page_state(lruvec_pgdat(lruvec), idx); + i = memcg_stats_index(idx); + if (i < 0) + return 0; + pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x = READ_ONCE(pn->lruvec_stats->state[idx]); + x = READ_ONCE(pn->lruvec_stats->state[i]); #ifdef CONFIG_SMP if (x < 0) x = 0; @@ -617,12 +688,17 @@ unsigned long lruvec_page_state_local(struct lruvec *lruvec, { struct mem_cgroup_per_node *pn; long x = 0; + int i; if (mem_cgroup_disabled()) return node_page_state(lruvec_pgdat(lruvec), idx); + i = memcg_stats_index(idx); + if (i < 0) + return 0; + pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x = READ_ONCE(pn->lruvec_stats->state_local[idx]); + x = READ_ONCE(pn->lruvec_stats->state_local[i]); #ifdef CONFIG_SMP if (x < 0) x = 0; @@ -689,11 +765,11 @@ struct memcg_vmstats_percpu { /* The above should fit a single cacheline for memcg_rstat_updated() */ /* Local (CPU and cgroup) page state & events */ - long state[MEMCG_NR_STAT]; + long state[MEMCG_VMSTAT_SIZE]; unsigned long events[NR_MEMCG_EVENTS]; /* Delta calculation for lockless upward propagation */ - long state_prev[MEMCG_NR_STAT]; + long state_prev[MEMCG_VMSTAT_SIZE]; unsigned long events_prev[NR_MEMCG_EVENTS]; /* Cgroup1: threshold notifications & softlimit tree updates */ @@ -703,15 +779,15 @@ struct memcg_vmstats_percpu { struct memcg_vmstats { /* Aggregated (CPU and subtree) page state & events */ - long state[MEMCG_NR_STAT]; + long state[MEMCG_VMSTAT_SIZE]; unsigned long events[NR_MEMCG_EVENTS]; /* Non-hierarchical (CPU aggregated) page state & events */ - long state_local[MEMCG_NR_STAT]; + long state_local[MEMCG_VMSTAT_SIZE]; unsigned long events_local[NR_MEMCG_EVENTS]; /* Pending child counts during tree propagation */ - long state_pending[MEMCG_NR_STAT]; + long state_pending[MEMCG_VMSTAT_SIZE]; unsigned long events_pending[NR_MEMCG_EVENTS]; /* Stats updates since the last flush */ @@ -844,7 +920,13 @@ static void flush_memcg_stats_dwork(struct work_struct *w) unsigned long memcg_page_state(struct mem_cgroup *memcg, int idx) { - long x = READ_ONCE(memcg->vmstats->state[idx]); + long x; + int i = memcg_stats_index(idx); + + if (i < 0) + return 0; + + x = READ_ONCE(memcg->vmstats->state[i]); #ifdef CONFIG_SMP if (x < 0) x = 0; @@ -876,18 +958,25 @@ static int memcg_state_val_in_pages(int idx, int val) */ void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val) { - if (mem_cgroup_disabled()) + int i = memcg_stats_index(idx); + + if (mem_cgroup_disabled() || i < 0) return; - __this_cpu_add(memcg->vmstats_percpu->state[idx], val); + __this_cpu_add(memcg->vmstats_percpu->state[i], val); memcg_rstat_updated(memcg, memcg_state_val_in_pages(idx, val)); } /* idx can be of type enum memcg_stat_item or node_stat_item. */ static unsigned long memcg_page_state_local(struct mem_cgroup *memcg, int idx) { - long x = READ_ONCE(memcg->vmstats->state_local[idx]); + long x; + int i = memcg_stats_index(idx); + + if (i < 0) + return 0; + x = READ_ONCE(memcg->vmstats->state_local[i]); #ifdef CONFIG_SMP if (x < 0) x = 0; @@ -901,6 +990,10 @@ static void __mod_memcg_lruvec_state(struct lruvec *lruvec, { struct mem_cgroup_per_node *pn; struct mem_cgroup *memcg; + int i = memcg_stats_index(idx); + + if (i < 0) + return; pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); memcg = pn->memcg; @@ -930,10 +1023,10 @@ static void __mod_memcg_lruvec_state(struct lruvec *lruvec, } /* Update memcg */ - __this_cpu_add(memcg->vmstats_percpu->state[idx], val); + __this_cpu_add(memcg->vmstats_percpu->state[i], val); /* Update lruvec */ - __this_cpu_add(pn->lruvec_stats_percpu->state[idx], val); + __this_cpu_add(pn->lruvec_stats_percpu->state[i], val); memcg_rstat_updated(memcg, memcg_state_val_in_pages(idx, val)); memcg_stats_unlock(); @@ -5702,6 +5795,7 @@ mem_cgroup_css_alloc(struct cgroup_subsys_state *parent_css) page_counter_init(&memcg->kmem, &parent->kmem); page_counter_init(&memcg->tcpmem, &parent->tcpmem); } else { + init_memcg_stats(); init_memcg_events(); page_counter_init(&memcg->memory, NULL); page_counter_init(&memcg->swap, NULL); @@ -5873,7 +5967,7 @@ static void mem_cgroup_css_rstat_flush(struct cgroup_subsys_state *css, int cpu) statc = per_cpu_ptr(memcg->vmstats_percpu, cpu); - for (i = 0; i < MEMCG_NR_STAT; i++) { + for (i = 0; i < MEMCG_VMSTAT_SIZE; i++) { /* * Collect the aggregated propagation counts of groups * below us. We're in a per-cpu loop here and this is @@ -5937,7 +6031,7 @@ static void mem_cgroup_css_rstat_flush(struct cgroup_subsys_state *css, int cpu) lstatc = per_cpu_ptr(pn->lruvec_stats_percpu, cpu); - for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { + for (i = 0; i < NR_MEMCG_NODE_STAT_ITEMS; i++) { delta = lstats->state_pending[i]; if (delta) lstats->state_pending[i] = 0; From patchwork Wed May 1 17:26:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13651043 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32C69C4345F for ; Wed, 1 May 2024 17:26:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F00156B009B; Wed, 1 May 2024 13:26:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E88C56B009D; Wed, 1 May 2024 13:26:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D0DD96B009B; Wed, 1 May 2024 13:26:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id B05D16B009B for ; Wed, 1 May 2024 13:26:46 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 640D040945 for ; Wed, 1 May 2024 17:26:46 +0000 (UTC) X-FDA: 82070506812.11.A38BA52 Received: from out-173.mta0.migadu.com (out-173.mta0.migadu.com [91.218.175.173]) by imf29.hostedemail.com (Postfix) with ESMTP id D409612001B for ; Wed, 1 May 2024 17:26:43 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=RYJMprjH; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf29.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.173 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714584404; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ememgP1JocqN9S+WLAK1U+7C9ihTSU1uk5FHGEEDYr4=; b=Oam2OLKKx07xdCpPH9bhe9zPSvgnYEj+boy7wlqMPWAq8qXW5fsp/2Xq7kaMk7JBrUbsxL PXYu/k0EEwA8uFLZ9BO5hxLa00/NJnshGcXl4PdgGaGX/z5efCqbpmMjhXo4kKee6jFrK1 Qu2ENeatQaUhb65KTqUgV0vTCD/Q7mY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714584404; a=rsa-sha256; cv=none; b=XI7iSINXexwdlvESQJlDOgEgYVtZqAt2t/ktd/xydcz/O4xkwfG8C3Sf1RiAjuZI3K7EHi YzMk8G/52ZSPFUhBS4JEmrnjLqwCbw3xamtZGo9uF2XKV/WfWWfcjVgt2Z9yfe3PrAzUdg OUnGxZzAx8uP0UbQwPcRQjMO6BY1/Zc= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=RYJMprjH; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf29.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.173 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714584402; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ememgP1JocqN9S+WLAK1U+7C9ihTSU1uk5FHGEEDYr4=; b=RYJMprjHMDS8e79Oq3QGMPyZEM9cde+P1t4xIvdp2lQdMigU28nY2+mJ+fRYkJXbRFOr7Z qX8L+scG0lHGdqMKsjfVCgN7bbJJe3LmLnlAZ+XUaTSy9gzBjchqIpCV85Os45LMjdQMMP ARQLfXyvt3IftYfCXIXe7crcs2JLri4= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 5/8] memcg: cleanup __mod_memcg_lruvec_state Date: Wed, 1 May 2024 10:26:14 -0700 Message-ID: <20240501172617.678560-6-shakeel.butt@linux.dev> In-Reply-To: <20240501172617.678560-1-shakeel.butt@linux.dev> References: <20240501172617.678560-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: D409612001B X-Stat-Signature: mhjaqmczexhku44ket6s6z9k8benb9a9 X-Rspam-User: X-HE-Tag: 1714584403-127564 X-HE-Meta: U2FsdGVkX18m68keBu9Hh/LEX8bN3/uxlRk8+OGd5H9WfNkIDkNKV1WHZ8sK69BCfv2Ar0BvqPWti3xOi4R6T99inbgdKHIH1W/bGDAuNc+Sd1bdmbdXqk+RIBFAYXfNVWWUHUcc7HUBASL57VT6sIZw3azd5QUxlsgMiZ/nr/xlMBHV5vydfN42zhrhmUM9JHurDduu6BTMJa5ud7LzFBWpVlbc6Fedm5smKWS1PsAGJ/FOhWjrJ/FMKmWvoeroq52FIiZIGG+rIK24UXUz//JNxrELQ3PInuXeZ6/iEAv3sTymlV0zbRxvSKmzYpuOjus8/llxdmlN+Iui7h7FYYRdeoqEgjZ4a9FGrUE1zJoCVqpkziSUe/q5iDp86BX3XWjuyRoC//4g0MCv3UjfVT04suCM3JZAmHmMU2bf/ir+8ZdotnuKY1IJz41zvHlbWe6kP7ZW9T1V9mVSgDbsDIUBqEsT7UlxIRNcimdm2KEIfd8W80wJY2MQZmpRELLAtANGOGerRlEv7aFMnxFB9WrZ2qnM8Dg19j33a5GgyZVDKUY+4Ka7Hk2MuuJx4uS+tLH6tyTqRjugSDnxWkA3SkYAvPVI5rnrI098aFchitbIk4ahjKv/pqj7C91UzVSJABzypMUhf5rUl1SoJ3x0wVQO3iwpk6LshIZUA72fIhH51pkNqRzgqil4XLogY5yxHLQVVOQpxwtbNUfJHLu0CBrraFq3tp0AB/kcBM7TvGHidVs3DZlX1zQriQnPhoNng/nz1SlArlfiKctYmhQaMfNxdYKVKl2PJC8hUbPz5OR2/kvxA86PvjMlSAhuw5J5B24fXxnH5FnzgSMxbL8y4Ed2nD/7sY1q86HFeDPh1yF68jrH9vzZPLc2QFP8LGzM1ncOQ/DrWn4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There are no memcg specific stats for NR_SHMEM_PMDMAPPED and NR_FILE_PMDMAPPED. Let's remove them. Signed-off-by: Shakeel Butt Reviewed-by: Yosry Ahmed Reviewed-by: Roman Gushchin Reviewed-by: T.J. Mercier --- Changes since v3: - N/A mm/memcontrol.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f3b6be5a0605..a78cf00dd537 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1010,8 +1010,6 @@ static void __mod_memcg_lruvec_state(struct lruvec *lruvec, case NR_ANON_MAPPED: case NR_FILE_MAPPED: case NR_ANON_THPS: - case NR_SHMEM_PMDMAPPED: - case NR_FILE_PMDMAPPED: if (WARN_ON_ONCE(!in_task())) pr_warn("stat item index: %d\n", idx); break; From patchwork Wed May 1 17:26:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13651044 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB182C4345F for ; Wed, 1 May 2024 17:26:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 983BE6B009C; Wed, 1 May 2024 13:26:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 92F926B009D; Wed, 1 May 2024 13:26:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D1EE6B009E; Wed, 1 May 2024 13:26:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 59E2A6B009C for ; Wed, 1 May 2024 13:26:48 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 229B2140CAB for ; Wed, 1 May 2024 17:26:48 +0000 (UTC) X-FDA: 82070506896.24.6CCDBF9 Received: from out-189.mta0.migadu.com (out-189.mta0.migadu.com [91.218.175.189]) by imf26.hostedemail.com (Postfix) with ESMTP id 6227D140007 for ; Wed, 1 May 2024 17:26:46 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=YllzTqli; spf=pass (imf26.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.189 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714584406; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=y9l4h9rie3ivYcnawsodk45GJOrF/TCVwGQRSWBbU5Y=; b=pLlEf0lJ04h2J2d5qXqSvNPgdneyR1Y/9iTEsr6Hb75kSjWIe8MwCB/2iM0k950ShiZpo5 TuzoUh0toEzRc6M8yuF4tpQMFqnPIemRm8wWucPppQP/muwNv7JajQEpLxRzJIp7/YbWge uR/5F9q/DxtaX9VD0ZUZyftoH/kJ4Z8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714584406; a=rsa-sha256; cv=none; b=a0g5UsJi1FS4Nn2D9w3GH16YVUAr24crAkSgqFrapvBAlFW7ZyNXj03WY2mviMRnBe+ZFZ Ao54b7PEXtnXoFImZbPXOHfGAZxI/xZgWhuNjqU96P4gdtcpCM1HlpGtmBAmGTHUevSPpD G3Z+pXv8GyDzV95qvNIeCdkP0j0m5Hk= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=YllzTqli; spf=pass (imf26.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.189 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714584405; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y9l4h9rie3ivYcnawsodk45GJOrF/TCVwGQRSWBbU5Y=; b=YllzTqliiBcI+Aeoe8N7nlIwm57TMV4GBbDUW7Gs09N6g0ncTSgttX6g5AuD2XQq1L1kKx wwfv4dy2Hbq8pAeX2MHK9w6Z6P0ndH05+Ncdg1vb5QCnTHF9BevviekazDKX8/0cJw5QhC b+SnksRmtYa+1xk4xF+paE+q/1cuoI8= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 6/8] mm: cleanup WORKINGSET_NODES in workingset Date: Wed, 1 May 2024 10:26:15 -0700 Message-ID: <20240501172617.678560-7-shakeel.butt@linux.dev> In-Reply-To: <20240501172617.678560-1-shakeel.butt@linux.dev> References: <20240501172617.678560-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Stat-Signature: bzrgqzcxiznn4ds4e418xa61rghfryf1 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 6227D140007 X-HE-Tag: 1714584406-32268 X-HE-Meta: U2FsdGVkX1+eJ+I5lb9RSymsMtZSLOdjm0NWrFVUtw5v1yuv0kf2rlEIEONQmwCHL295DbxK/YzkgFbncCE2cQ0Q3wUX+JLB+ZjiR69dmBvE/pC1DKu7j6TQfLFzvenBH83eePOPJX0NNp5IBf4dJlSFXmoX6x7dVh6iXl/qK5XZRbOnPsvruggnjnLF1KCEJGxhytWWxsoWf+Oyqw0PrSNJpZKo9q0dD/4rzMW9bqvLt2k8hcB+CiK+xaORBMoRx7O8Q3cxYm854k4l3TPmdoSVFEOnI/M4C+vUyJ0aRJ0SIkktwXsoTuVNJ+OtcnuSAsUW3n7cL0EClBe6AqN/AeaV7g87MW31EWTAG1HYLEcuPT0KEWovd54RtylTLg99MJlr+Gr5GPXmopx8/TTCVJlZD4udunIhZdkF6QYawl+rTefiv9/gDGBPktzgvIFYet32Qa5Dq2VsEKCT1cOKOaOGC4KFEIyKgXsvz4cHvhIkv99iJtu6Qh7nekY/rrt4ZDdBhB9A+Mg1s128EDaEhAW019ecWikU9NPo7FGscvA55PYxnzNDtUP6AEZwkgsWqFhAwZ6rkG0VPtMM1eFo1wNUZttsr6WDGiWu5QxrsSgsanCQi0QJr/cfak4OMDk2B4VgVMUlbWj5kztc2hi9324eUKlNUWLUvhgBUbOmkQl+PBZnLR9IHJ4/5C3JF7TxdF99FjF0JQFgU7oBronGJAZIImG3Jh3wckulO5q8HH5OPKaox+0KgLIs+rYuaR9qNpnGK7MbubeZQHUc+RPhQcv5Gn7EAhOkBhnoF/yL/XGSLQGhoaKlQPZ8iY2WnecTNOS6SJ9bpp/BaJoQSllPMdbORiZAk3DalxgVFVdWokyHTGfpS/4oTUp+jR2aNaca7+tlgvLBkDhQNYhs8KiOOngtqwl5CE+buPc1qqMKg30lJ25TWTAO4PqlGZoXvG0jsTQWi/+iFKakPbWGUIX 8EGIj/jH YX+Tu3GCWZxD/xMinJmbNFQPx/2cAcINodH6B+6YcYYN5SMkJnpz7DXeX308r/xfmwOjJsTDhWaFCPoBzszP6r/F3CKYG77fTnCI19Kamsb5e5FQaPq3orvVtpBaXXaad6ZsoN4thWgpsixluFfJrq3mPWw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: WORKINGSET_NODES is not exposed in the memcg stats and thus there is no need to use the memcg specific stat update functions for it. In future if we decide to expose WORKINGSET_NODES in the memcg stats, we can revert this patch. Signed-off-by: Shakeel Butt Reviewed-by: Roman Gushchin Reviewed-by: T.J. Mercier --- Changes since v3: - N/A mm/workingset.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mm/workingset.c b/mm/workingset.c index f2a0ecaf708d..c22adb93622a 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -618,6 +618,7 @@ struct list_lru shadow_nodes; void workingset_update_node(struct xa_node *node) { struct address_space *mapping; + struct page *page = virt_to_page(node); /* * Track non-empty nodes that contain only shadow entries; @@ -633,12 +634,12 @@ void workingset_update_node(struct xa_node *node) if (node->count && node->count == node->nr_values) { if (list_empty(&node->private_list)) { list_lru_add_obj(&shadow_nodes, &node->private_list); - __inc_lruvec_kmem_state(node, WORKINGSET_NODES); + __inc_node_page_state(page, WORKINGSET_NODES); } } else { if (!list_empty(&node->private_list)) { list_lru_del_obj(&shadow_nodes, &node->private_list); - __dec_lruvec_kmem_state(node, WORKINGSET_NODES); + __dec_node_page_state(page, WORKINGSET_NODES); } } } @@ -742,7 +743,7 @@ static enum lru_status shadow_lru_isolate(struct list_head *item, } list_lru_isolate(lru, item); - __dec_lruvec_kmem_state(node, WORKINGSET_NODES); + __dec_node_page_state(virt_to_page(node), WORKINGSET_NODES); spin_unlock(lru_lock); From patchwork Wed May 1 17:26:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13651045 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53644C4345F for ; Wed, 1 May 2024 17:26:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A6C386B009E; Wed, 1 May 2024 13:26:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9CB976B009F; Wed, 1 May 2024 13:26:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81FBC6B00A0; Wed, 1 May 2024 13:26:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 623996B009E for ; Wed, 1 May 2024 13:26:51 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2CDCAC0B21 for ; Wed, 1 May 2024 17:26:51 +0000 (UTC) X-FDA: 82070507022.14.CD902CE Received: from out-172.mta0.migadu.com (out-172.mta0.migadu.com [91.218.175.172]) by imf23.hostedemail.com (Postfix) with ESMTP id 96959140016 for ; Wed, 1 May 2024 17:26:49 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=trfIiJqg; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf23.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.172 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714584409; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GSfZvMT0meM6+Tg2n6OjVJNbS0vHa+pAGl0pke03oZo=; b=qp3JqKPXI8zxFKpEFD5He11k61G6iSSWtfKwP3AnoeGYclYiHzqZpRkwJ/2xNG17ev8mbB BBDx2fLq6fVMVpKYkrcfapIDsaVXWriwSMiC9yMEqxYEY1u3S3JBJujqQ4jdnVNy1ni33z J33MFfHGMOmPiYGvZtHai2Y5zAHfPgE= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=trfIiJqg; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf23.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.172 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714584409; a=rsa-sha256; cv=none; b=Epns8QYxl2FjWLQmvicx1Mr27QDNF7JRlc2OmK/aniCWlznhWcaxVcbFPg5WZPZAdoCnTT qDU0lBOOCs+Qvnp3V4xnmZ+56FBhN2X/rBnjzYfk8gK0HXLmhjnW4Y/8n1w78W5sOgLkWu lQlh0yMl/JUmC0WzRkNtn3pfLXXz9Gg= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714584408; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GSfZvMT0meM6+Tg2n6OjVJNbS0vHa+pAGl0pke03oZo=; b=trfIiJqgV4cZZHqhDCNO3VkVT22d/YhoirgzSeIxHk0Kyp4BRRYVdbzVdcRQUlf99KJVIB +sqEruaqw97BM6RkWEWKQYlkT3DaUVJvFUGmcuMT4MflewgGqt6Zg8xgqKU5yk0B6aoJF4 tPo0pBwMJJ1keoewuqZXlJvbt60TGN0= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 7/8] memcg: warn for unexpected events and stats Date: Wed, 1 May 2024 10:26:16 -0700 Message-ID: <20240501172617.678560-8-shakeel.butt@linux.dev> In-Reply-To: <20240501172617.678560-1-shakeel.butt@linux.dev> References: <20240501172617.678560-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Stat-Signature: 5urbcdydf8cxwckbxfk8wckcaxsu3pbd X-Rspamd-Queue-Id: 96959140016 X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1714584409-90533 X-HE-Meta: U2FsdGVkX1+uT8pbw2K5C5pXSH+DVJpmlCwVhgNjAKlRv5sgMo3Iay6EtfCjLLLdNmxjqrqk4Uzz2GzkoybiCSJEoG2NpmCqt1zv2Vp7seBX//+Ufcf6MqBEG1xbAG+LyF4LhoYDDZIHz6Q7wLaDYxgIFIaIe3LcAEI3T9PgZWykMfowLn1y4UNSjcYAAxZ1owTOo2sLyV9awKEvjztiopYznktYjW6V/pZ+UtPVTQU9w1aDvszMRAg2N7vM5Yf3tHkt1fOt1OocsBfeLW3ZeINaibqYhl94mKGngY5jO1zF1vd4wAWfLuTJpsqLtvQvNakNPrdcAdDfezYeVmszx61abjqxM50wN6wuWQKNQA57baLmx7VxhCeGT6Tk1lPZn9E0nsmsSkUCiAv2XiAO82Z+VELkKyQUz7MUy9LJda9tPUYdQKUPp+tJ7mYOSTZlPp0M6sNRA4Nf0BzOzHN/PAK2F3hVfNskTRqpqrTpWOOV78Mrb+8qJuJHCx8CrD/655G/TXXuu3PDoe7rPfD7BX9cdUiBx86e+B2e/QWXxhu+SP6wCjgMjnUezzOiGk2AsuoMyp894VNrvGN2Y0GCUiMCG4+N0CTgWxh8q3kE5cT8jwgEu8QD+tAEGq0ZNdCMQOlkchaK28GxsVMJTgMyra59gBeMLZXI4ucKTD5ZJI7vsEAHzWa2s9y4KCmeszLxrfhZMSV45Gax0or5OpCX/TQCzWQC54UlxyJ2EDbcmewMaUuyTe3UtGHLGfmMyw+li5oMEQOqkZB/9hIt7m/xJjVF6BFWEU2u2AfYCcUgsqg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To reduce memory usage by the memcg events and stats, the kernel uses indirection table and only allocate stats and events which are being used by the memcg code. To make this more robust, let's add warnings where unexpected stats and events indexes are used. Signed-off-by: Shakeel Butt Reviewed-by: Roman Gushchin --- Changes since v3: - N/A mm/memcontrol.c | 39 +++++++++++++++++++++++---------------- 1 file changed, 23 insertions(+), 16 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index a78cf00dd537..b4a1b4bb599d 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -671,7 +671,7 @@ unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx) return node_page_state(lruvec_pgdat(lruvec), idx); i = memcg_stats_index(idx); - if (i < 0) + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, idx)) return 0; pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); @@ -687,14 +687,14 @@ unsigned long lruvec_page_state_local(struct lruvec *lruvec, enum node_stat_item idx) { struct mem_cgroup_per_node *pn; - long x = 0; + long x; int i; if (mem_cgroup_disabled()) return node_page_state(lruvec_pgdat(lruvec), idx); i = memcg_stats_index(idx); - if (i < 0) + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, idx)) return 0; pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); @@ -923,7 +923,7 @@ unsigned long memcg_page_state(struct mem_cgroup *memcg, int idx) long x; int i = memcg_stats_index(idx); - if (i < 0) + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, idx)) return 0; x = READ_ONCE(memcg->vmstats->state[i]); @@ -960,7 +960,10 @@ void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val) { int i = memcg_stats_index(idx); - if (mem_cgroup_disabled() || i < 0) + if (mem_cgroup_disabled()) + return; + + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, idx)) return; __this_cpu_add(memcg->vmstats_percpu->state[i], val); @@ -973,7 +976,7 @@ static unsigned long memcg_page_state_local(struct mem_cgroup *memcg, int idx) long x; int i = memcg_stats_index(idx); - if (i < 0) + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, idx)) return 0; x = READ_ONCE(memcg->vmstats->state_local[i]); @@ -992,7 +995,7 @@ static void __mod_memcg_lruvec_state(struct lruvec *lruvec, struct mem_cgroup *memcg; int i = memcg_stats_index(idx); - if (i < 0) + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, idx)) return; pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); @@ -1106,34 +1109,38 @@ void __mod_lruvec_kmem_state(void *p, enum node_stat_item idx, int val) void __count_memcg_events(struct mem_cgroup *memcg, enum vm_event_item idx, unsigned long count) { - int index = memcg_events_index(idx); + int i = memcg_events_index(idx); - if (mem_cgroup_disabled() || index < 0) + if (mem_cgroup_disabled()) + return; + + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, idx)) return; memcg_stats_lock(); - __this_cpu_add(memcg->vmstats_percpu->events[index], count); + __this_cpu_add(memcg->vmstats_percpu->events[i], count); memcg_rstat_updated(memcg, count); memcg_stats_unlock(); } static unsigned long memcg_events(struct mem_cgroup *memcg, int event) { - int index = memcg_events_index(event); + int i = memcg_events_index(event); - if (index < 0) + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, event)) return 0; - return READ_ONCE(memcg->vmstats->events[index]); + + return READ_ONCE(memcg->vmstats->events[i]); } static unsigned long memcg_events_local(struct mem_cgroup *memcg, int event) { - int index = memcg_events_index(event); + int i = memcg_events_index(event); - if (index < 0) + if (WARN_ONCE(i < 0, "%s: missing stat item %d\n", __func__, event)) return 0; - return READ_ONCE(memcg->vmstats->events_local[index]); + return READ_ONCE(memcg->vmstats->events_local[i]); } static void mem_cgroup_charge_statistics(struct mem_cgroup *memcg, From patchwork Wed May 1 17:26:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13651046 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77976C4345F for ; Wed, 1 May 2024 17:26:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7FB636B00A1; Wed, 1 May 2024 13:26:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7AA9A6B00A2; Wed, 1 May 2024 13:26:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 539846B00A3; Wed, 1 May 2024 13:26:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3142B6B00A1 for ; Wed, 1 May 2024 13:26:55 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id D67A0A16BF for ; Wed, 1 May 2024 17:26:54 +0000 (UTC) X-FDA: 82070507148.15.CE4A45C Received: from out-189.mta1.migadu.com (out-189.mta1.migadu.com [95.215.58.189]) by imf25.hostedemail.com (Postfix) with ESMTP id 31DEDA0018 for ; Wed, 1 May 2024 17:26:52 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=YISvIA9+; spf=pass (imf25.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.189 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714584413; a=rsa-sha256; cv=none; b=BUeUp2of3IbnXud6YUkU1lH/Q3dSLlfqjaoJLPAJ+CrFfdqKoho77qysTBsVR44ladBQu/ wXf19tOe8Q9gfcdIQmSO0Teh1BpWPphXPKka5y+6umuTkWG5VpCS89GUjfiCOAIuHaakx+ arLUYqjPusKw4y5uEmxHkxWmYepg2AA= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=YISvIA9+; spf=pass (imf25.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.189 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714584413; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tCoAbtrV5bh5IqBSP9ezmAWDh3uY8sBNqfpz/AA9bcI=; b=sXjLQSUo8LMswyUzAV2baAIHAXHca6d4a2OOF032YpXWfUHlhSxaI1QGvKaTmf1HDtLC5f 9alQyw7KsO7ot1j7nEVcOUOjHdhA8YDlJAiFUCxgFNR4JbK4OpDg9WxYjvXL99AfTZe0+C flsJHJBAjbJK0MApZiZFmkxeab62g8U= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714584411; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tCoAbtrV5bh5IqBSP9ezmAWDh3uY8sBNqfpz/AA9bcI=; b=YISvIA9+kLtsFEh8MdixEMDSsd8PoXaMrMDODnf1eEjM6JbuLCUks/ImDQ8J8NJ2jIUepy HiLDsiFIiMZF5j0TtXoAglBooeB6fZvdFyN7ggcqruY6WnAoev3cXxwgDUVNyvSM4NV8On wbeqCV6XdRAOfOrUrhbc9rSs7VW6dGg= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Yosry Ahmed , "T . J . Mercier" Cc: kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 8/8] memcg: use proper type for mod_memcg_state Date: Wed, 1 May 2024 10:26:17 -0700 Message-ID: <20240501172617.678560-9-shakeel.butt@linux.dev> In-Reply-To: <20240501172617.678560-1-shakeel.butt@linux.dev> References: <20240501172617.678560-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Stat-Signature: tdkuweenaoontfk6dsumudksya7d3z3m X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 31DEDA0018 X-HE-Tag: 1714584412-821849 X-HE-Meta: U2FsdGVkX19qzMy78n/sLlA4CcusGtMH2zwG+4DeQFFOV1+A+C4HwTxuvMJFwYSWUNYw39Ew5MnChvsbbog0AaQNBDKEVWxXalYSeAvzhAUVamJrrf28KlEcWM/drLQV2qED8V/QvoRYvRdEm8q2Pfjz36g1/a6btQn2exjZuCqqBqfnBba9Fv9ZM7HVAQkfNgBnxirUOO4/FFB5xfSQDie/od3KSXqhJzMQmJa408X1ehWcKRK6N6/TU9r8rQmVVyUWwbkksooCQXbm1XNmWIzbPDDPoPIsNLO1Ftq3QRn9nVoie0cfAtwM8XkNjfS+A6P93OLJb3EtLDhpQeaEqKvNs1h5l8mcH+9ue8cjK6Nm8bvOkXvZRtPh39gEvfOmW7mYAIlrAQ1KvTgLBSOB2WnxgcjFNP9BxMkhpf5P3TPNlkeSJBbZVTbOAskP2L1zUaDuKqxL8h4QtpkYi8F/uZGDfN63WwgRnFNyHDvWWGXaCtRMgS+nO2h5Zp2TJBpb///66rJLzwoa0PttDfk1vkbYjTV9Bo4+to/gepxIouurszvE8LUZemAzVAtQUyqC4LQHpQ4qlxTv6Fa6Q5u0ED+DCDv4xAcDu9RXekXklad9ERxEUq1LPxgMqbWtsFzSZKetnNzJiWoMWBPwOLQQNZ/7wePI82WvylWi+vCNqJJowG3Np6ouZH0mUD2KZbsockeCDUN392BT8ciE5WEmAAhBedTpjQUaC+KhXCHSXqmhxcvjC+j63glPmJZqIeSJKxz/lWUI4NQccntSMzD9fKEo+xqqnFSObpr++tpLFaoR/2iaR6BPaI976pUsiSKKV0XB7lmlekUiDLq4D+SSvRgtrK5zqxhOvuC7u5KwvJHOFI8PqhkrPSyy6DlGykSWoHnEGmiuXTaRjg0Xhad+3AI1OriiDiNocGrv+AfiMVIHrnB3v8ZmNhCvWVF+OncN2qeEn2bTmIwPCeqRfvP IW2C0uCU JfQzEBeLhX+qIJyeKxKt8Ob50/C7dAyQx/v7hRoyOiXxY30ofYhCukjfT8Rf8dyrvF7xYDMlVNM8yEQbQocxYfH+RnEi3V9cEo5ORup1wPLLLhqhy4jtt2U8RKwN4yq8SWKHxpzT7rAuj8BlNfxOwwByu4g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The memcg stats update functions can take arbitrary integer but the only input which make sense is enum memcg_stat_item and we don't want these functions to be called with arbitrary integer, so replace the parameter type with enum memcg_stat_item and compiler will be able to warn if memcg stat update functions are called with incorrect index value. Signed-off-by: Shakeel Butt Reviewed-by: T.J. Mercier Reviewed-by: Roman Gushchin --- Changes since v3: - N/A include/linux/memcontrol.h | 13 +++++++------ mm/memcontrol.c | 3 ++- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index ab8a6e884375..030d34e9d117 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -974,7 +974,8 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg); void folio_memcg_lock(struct folio *folio); void folio_memcg_unlock(struct folio *folio); -void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val); +void __mod_memcg_state(struct mem_cgroup *memcg, enum memcg_stat_item idx, + int val); /* try to stablize folio_memcg() for all the pages in a memcg */ static inline bool mem_cgroup_trylock_pages(struct mem_cgroup *memcg) @@ -995,7 +996,7 @@ static inline void mem_cgroup_unlock_pages(void) /* idx can be of type enum memcg_stat_item or node_stat_item */ static inline void mod_memcg_state(struct mem_cgroup *memcg, - int idx, int val) + enum memcg_stat_item idx, int val) { unsigned long flags; @@ -1005,7 +1006,7 @@ static inline void mod_memcg_state(struct mem_cgroup *memcg, } static inline void mod_memcg_page_state(struct page *page, - int idx, int val) + enum memcg_stat_item idx, int val) { struct mem_cgroup *memcg; @@ -1491,19 +1492,19 @@ static inline void mem_cgroup_print_oom_group(struct mem_cgroup *memcg) } static inline void __mod_memcg_state(struct mem_cgroup *memcg, - int idx, + enum memcg_stat_item idx, int nr) { } static inline void mod_memcg_state(struct mem_cgroup *memcg, - int idx, + enum memcg_stat_item idx, int nr) { } static inline void mod_memcg_page_state(struct page *page, - int idx, int val) + enum memcg_stat_item idx, int val) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b4a1b4bb599d..39f8b0df46f6 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -956,7 +956,8 @@ static int memcg_state_val_in_pages(int idx, int val) * @idx: the stat item - can be enum memcg_stat_item or enum node_stat_item * @val: delta to add to the counter, can be negative */ -void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val) +void __mod_memcg_state(struct mem_cgroup *memcg, enum memcg_stat_item idx, + int val) { int i = memcg_stats_index(idx);