From patchwork Sat Apr 27 00:37:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13645512 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09B03C4345F for ; Sat, 27 Apr 2024 00:37:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 75ABD6B008A; Fri, 26 Apr 2024 20:37:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 70A9A6B008C; Fri, 26 Apr 2024 20:37:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F8F56B0092; Fri, 26 Apr 2024 20:37:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 4181C6B008A for ; Fri, 26 Apr 2024 20:37:47 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E96AA12142B for ; Sat, 27 Apr 2024 00:37:46 +0000 (UTC) X-FDA: 82053448932.23.FEB0349 Received: from out-187.mta1.migadu.com (out-187.mta1.migadu.com [95.215.58.187]) by imf12.hostedemail.com (Postfix) with ESMTP id 3C62440019 for ; Sat, 27 Apr 2024 00:37:45 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=Nm5yyRv3; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf12.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.187 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714178265; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Gfv3iMSIWQC4QbQ/6GXaLS8Cjd7JxtChDyD5rQgDX9Q=; b=8G70YkHb4xnktpG+TAUfIsPjUjDzsK1beDKpgUU46CpCVrCvw7Y2z+5kusceADYDNhJSj2 wQQZ46nsPap5Oa3NwXIuDQtA3gfDTOzHm+3wB3xsCAaOxuEG10/wk62Q9vIbgY2ysP7x7Y p+ZiqpF5Q2pVNwq5xB3VBPAvoWT5Izg= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=Nm5yyRv3; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf12.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.187 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714178265; a=rsa-sha256; cv=none; b=Yep+GErOPdzU2jJxa/PYXHrkj310KGPyEuYNBqG7IUFdtkSeSFye0E0PWH28E7burLmaB2 3ok4LHWj8vXQ2U9Z3NSPYzx/wCQpGVcIas/jtx1DpAYhoAlF+xslnGswY3soLKLK8rNKNs SX3lcA1BDhR3kAFWUKps7BjCKR6nkVs= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714178263; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gfv3iMSIWQC4QbQ/6GXaLS8Cjd7JxtChDyD5rQgDX9Q=; b=Nm5yyRv30txwaGJ50+RpR3dkkQIdU/cAkah0rz5E9NaWS6JEnqIlf8pCbX0XdNDBFnxc6N Hbz4uF6xZb+psVAA6G6MVW0R9CvKnIaxzNY6sPTfTQEuMd3bY+WiOBBBHx+n4a1jgGVc7m SpP+ZPeUku+xQYxxAw2TOWgKmw/JcxQ= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/7] memcg: reduce memory size of mem_cgroup_events_index Date: Fri, 26 Apr 2024 17:37:27 -0700 Message-ID: <20240427003733.3898961-2-shakeel.butt@linux.dev> In-Reply-To: <20240427003733.3898961-1-shakeel.butt@linux.dev> References: <20240427003733.3898961-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 3C62440019 X-Stat-Signature: nb3dbthjtu76yaabpryxpxk8igfbf3r8 X-HE-Tag: 1714178265-144274 X-HE-Meta: U2FsdGVkX19y+cJB67p126E5ReOWKBKmTRpPREKVlT61q7yfPhNJpQ7F7O+MhYZ6Qc6swIuvs0OeiY6aybhnPlfqjfeigl2X5V1tJxjSV4NFzrYqUPI3o25KCQBVOcjsCcCxSFQvJnbADODDkX/ah+Regz/ZSFaYmV5QmcDBEE1yjLkt+noW0x03lZ9jGdj8SXz9MH/63PFkyRaNvCWV4evMaTWQQ+idjvPNitzVsehzNwR/BDK+zGHm03Rmmb0H8nL78ekUZ+INSyc8CINsvBAoWFxUi29uULvLmMS91+lSIAfrcZ6nd8a+8Uv9wQIn754uH1U3TFtqYStrWX+DBuIGma/FaQ1NLES1qlSMjubXZ7h1NExbKRadsyVzkQNMQ+L0LsmeQfGfyVcCbC+xGcuhNy4ayMyIRm6xgrj3rI/a2LWfPV0kDPwPKvMqbmsyEAQJMgOt9DBp9X6S8v+FDtJLSHJwxlbz+unVSutI4EuVUfZaCmZrXYqMhByAFo/f+uDEfjGTzuHKukPgQPbSttBsovMAnSxwXGvN4xZ54ij8x2ByCA2cIvW28NkkL07zUNQAiTOPFmLmHMx/ePqrMRB2XW71K7Tn4kus3l8l9DtgOWisXZ6I1obeq7r6VJ32NJiyf6oEnSSnLFJD3bSOLue1YRn/+MrRNIH3onIxjJddIj4EIpwNdspb7bsBgHxSv7WUvU+VZPotP/BiVMlGanYxNgFcYsF7XFicNa+OgrpMW22IO4nPxpa0aNEgJwxlB7MKkxITNLxUoBt58HA9mERtN7PXQTer2BDKvshnMaCJLA/m0hbO5xGkgayuLuWrf2BFRUBegLgpBuych4gdv1XmMAzNFHToRQxWDtuuisRSV/GAKwwZK6uNsujBai2IFhIlTfEOqKY59NcsxL9uL1Vf88fUHXNBh01e2oSXTzdYLxZSI36Eu5FqiZwXZ5QYD9/zruMRJlQVYB3vLyf F4Yh52bW FALujQj2tv+BGx31NNhdvPKYEX3WGm0/VnDF+0o6/PKhFC3JQXSRZqx3sX7XRHbQyr2uoNt+ZOh4Lr1XF1xb32NASw0JLMJ/lHoUiZwtnXgD1KlWahoyFoDZROA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: mem_cgroup_events_index is a translation table to get the right index of the memcg relevant entry for the general vm_event_item. At the moment, it is defined as integer array. However on a typical system the max entry of vm_event_item (NR_VM_EVENT_ITEMS) is 113, so we don't need to use int as storage type of the array. For now just use int8_t as type and add a BUILD_BUG_ON() and will switch to short once NR_VM_EVENT_ITEMS touches 127. Signed-off-by: Shakeel Butt Reviewed-by: Roman Gushchin --- mm/memcontrol.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 602ad5faad4d..53769d06053f 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -607,11 +607,14 @@ static const unsigned int memcg_vm_event_stat[] = { }; #define NR_MEMCG_EVENTS ARRAY_SIZE(memcg_vm_event_stat) -static int mem_cgroup_events_index[NR_VM_EVENT_ITEMS] __read_mostly; +static int8_t mem_cgroup_events_index[NR_VM_EVENT_ITEMS] __read_mostly; static void init_memcg_events(void) { - int i; + int8_t i; + + /* Switch to short once this failure occurs. */ + BUILD_BUG_ON(NR_VM_EVENT_ITEMS >= 127 /* INT8_MAX */); for (i = 0; i < NR_MEMCG_EVENTS; ++i) mem_cgroup_events_index[memcg_vm_event_stat[i]] = i + 1; From patchwork Sat Apr 27 00:37:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13645513 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CAE2C4345F for ; Sat, 27 Apr 2024 00:37:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DEB436B0092; Fri, 26 Apr 2024 20:37:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D9A8A6B0093; Fri, 26 Apr 2024 20:37:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3B3F6B0095; Fri, 26 Apr 2024 20:37:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A38E36B0092 for ; Fri, 26 Apr 2024 20:37:49 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5F7C7160836 for ; Sat, 27 Apr 2024 00:37:49 +0000 (UTC) X-FDA: 82053449058.18.A0AFD13 Received: from out-188.mta1.migadu.com (out-188.mta1.migadu.com [95.215.58.188]) by imf20.hostedemail.com (Postfix) with ESMTP id CDF0C1C0010 for ; Sat, 27 Apr 2024 00:37:47 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b="TTzxVCz/"; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf20.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.188 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714178268; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5HIEk0SwiGrn4sd8u1onunMHRY8Lr8R1eMIi9eNXAwg=; b=ar69QgRHDlQ54j1J6tLUwp6RytbKVu1eGON9nHZs47qyMaybqyrX7+TyYzqlpal9BPUl19 HswwJFPBrIqIC+trFaifg4SkuDnqDHoRY5RephGOCjfh50IdWYTmPayUr7wlWMeoA4XXeX GI/OM4e3JYLBvJWBssGaiZek2+CA27Y= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714178268; a=rsa-sha256; cv=none; b=7wvn2TAt8y+va3uNtK2eHmpjR0Rq8xcYiXMw3r6z/hmwm/85jQmrYUIVtZHQkdL7SDOKWS 4wre57IjdJMPvwaZy1oSY2Bc6WmhWzKMp7pS6FOGzoVqUMQIZoE3sb7la1ZR87HZ9WgAlt IWJPQKC2wDw4M9U+19/nm2g7M1cvKSM= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b="TTzxVCz/"; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf20.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.188 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714178266; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5HIEk0SwiGrn4sd8u1onunMHRY8Lr8R1eMIi9eNXAwg=; b=TTzxVCz/SkpxFmZQf92Tga9m8XT6tVu0QWSUn9K3qKo0E1mv96BJzNsXIz76H/EcjW0Psm aOZme5snQD4/yIdO02Ba+xutkIyUD+b/G5a8DIZz/9L/SuinYiiA5H2V5R0NuZ+SUXmxdq n/1LeeGxekFuaPZVKjK+TbCPzXMu6FA= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/7] memcg: dynamically allocate lruvec_stats Date: Fri, 26 Apr 2024 17:37:28 -0700 Message-ID: <20240427003733.3898961-3-shakeel.butt@linux.dev> In-Reply-To: <20240427003733.3898961-1-shakeel.butt@linux.dev> References: <20240427003733.3898961-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: CDF0C1C0010 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: f38x6cj6yk5dkcecks9xfsoxcg6knatp X-HE-Tag: 1714178267-235413 X-HE-Meta: U2FsdGVkX1+UXPv1K820PDcoyh63N1N3nAFk/BB1jODG51yxxuMNIr/HuTDpD9XpIAHv+TmFIAo6ENrtaFoIgO7TpRJXES1Kbmf0PvciNJfyOqlL2xTQHKDx2MFw/+X+FqKtyFVGskZlXrbTuKgVm5C6baRkqAg+qa1CNm06AKvuHjKQ3C+o42Sawvj1fQeaEmZy6GhnOe9odNvrckbm7mvXpkyq5fMLQugyJz5ImCVUwPz/oVhVYdz8D0dCrJYTbLdGZu4j4ecOiewRzHZc5L5zw8oL5ZeFMCME41y1tcr2mh+oifXI907CqwYr7fUrobud7xpVyfRou7buVqZ7UQ9yRHp+neXyWWCRVQn5bqzew2xN3U7ZTnrNRKcTXHfu2XiEuniumEDNvwkBuYIbibbrEtdWDTTK8FYlBa3vMv9Z/aGcM9P/b/GpH0YGhxl5srt2JS9+rvDaU7QvztFD9kaftFcKMzViI8zKgQFkotXwKck+tKEBfRWedqgZTfiwU95n91tFd6CP9aZqiha74T5lo+y24LdphPaH8ECyAscdLiU7fAt7nPF66oB7lNCgdVB9O71g7JZYNwAU0IxL8uLauUS+vXVV3J5+5LkVl0tBGvkrcrEYzmIi+HwVCCam3tF/nieVLffqo1tSPKvYYWpzkdqX+ITh9zgCZhcT286wR2RuTghDZdThQJLwRC5qsO2Tky/0rWKQWJzWXZwLL0adPo29DubkRVQTA64qKx/8C6Bv7/V7Ku9+LkymlgbGEUR8T0sZFuPWe4FwfilRC6OrgeMV5LzrDAMAvJlBg688CxbaaBANQZujvwtphasMXWQ3bYtqAqerHAsECRYzMXSxidbS1tfrNNNylEOyoA0M5KBuXjCV6D3QmAZTUCmK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To decouple the dependency of lruvec_stats on NR_VM_NODE_STAT_ITEMS, we need to dynamically allocate lruvec_stats in the mem_cgroup_per_node structure. Also move the definition of lruvec_stats_percpu and lruvec_stats and related functions to the memcontrol.c to facilitate later patches. No functional changes in the patch. Signed-off-by: Shakeel Butt Signed-off-by: Roman Gushchin Acked-by: Shakeel Butt --- include/linux/memcontrol.h | 62 +++------------------------ mm/memcontrol.c | 87 ++++++++++++++++++++++++++++++++------ 2 files changed, 81 insertions(+), 68 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 9aba0d0462ca..ab8a6e884375 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -83,6 +83,8 @@ enum mem_cgroup_events_target { struct memcg_vmstats_percpu; struct memcg_vmstats; +struct lruvec_stats_percpu; +struct lruvec_stats; struct mem_cgroup_reclaim_iter { struct mem_cgroup *position; @@ -90,25 +92,6 @@ struct mem_cgroup_reclaim_iter { unsigned int generation; }; -struct lruvec_stats_percpu { - /* Local (CPU and cgroup) state */ - long state[NR_VM_NODE_STAT_ITEMS]; - - /* Delta calculation for lockless upward propagation */ - long state_prev[NR_VM_NODE_STAT_ITEMS]; -}; - -struct lruvec_stats { - /* Aggregated (CPU and subtree) state */ - long state[NR_VM_NODE_STAT_ITEMS]; - - /* Non-hierarchical (CPU aggregated) state */ - long state_local[NR_VM_NODE_STAT_ITEMS]; - - /* Pending child counts during tree propagation */ - long state_pending[NR_VM_NODE_STAT_ITEMS]; -}; - /* * per-node information in memory controller. */ @@ -116,7 +99,7 @@ struct mem_cgroup_per_node { struct lruvec lruvec; struct lruvec_stats_percpu __percpu *lruvec_stats_percpu; - struct lruvec_stats lruvec_stats; + struct lruvec_stats *lruvec_stats; unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; @@ -1037,42 +1020,9 @@ static inline void mod_memcg_page_state(struct page *page, } unsigned long memcg_page_state(struct mem_cgroup *memcg, int idx); - -static inline unsigned long lruvec_page_state(struct lruvec *lruvec, - enum node_stat_item idx) -{ - struct mem_cgroup_per_node *pn; - long x; - - if (mem_cgroup_disabled()) - return node_page_state(lruvec_pgdat(lruvec), idx); - - pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x = READ_ONCE(pn->lruvec_stats.state[idx]); -#ifdef CONFIG_SMP - if (x < 0) - x = 0; -#endif - return x; -} - -static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec, - enum node_stat_item idx) -{ - struct mem_cgroup_per_node *pn; - long x = 0; - - if (mem_cgroup_disabled()) - return node_page_state(lruvec_pgdat(lruvec), idx); - - pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x = READ_ONCE(pn->lruvec_stats.state_local[idx]); -#ifdef CONFIG_SMP - if (x < 0) - x = 0; -#endif - return x; -} +unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx); +unsigned long lruvec_page_state_local(struct lruvec *lruvec, + enum node_stat_item idx); void mem_cgroup_flush_stats(struct mem_cgroup *memcg); void mem_cgroup_flush_stats_ratelimited(struct mem_cgroup *memcg); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 53769d06053f..5e337ed6c6bf 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -576,6 +576,60 @@ mem_cgroup_largest_soft_limit_node(struct mem_cgroup_tree_per_node *mctz) return mz; } +struct lruvec_stats_percpu { + /* Local (CPU and cgroup) state */ + long state[NR_VM_NODE_STAT_ITEMS]; + + /* Delta calculation for lockless upward propagation */ + long state_prev[NR_VM_NODE_STAT_ITEMS]; +}; + +struct lruvec_stats { + /* Aggregated (CPU and subtree) state */ + long state[NR_VM_NODE_STAT_ITEMS]; + + /* Non-hierarchical (CPU aggregated) state */ + long state_local[NR_VM_NODE_STAT_ITEMS]; + + /* Pending child counts during tree propagation */ + long state_pending[NR_VM_NODE_STAT_ITEMS]; +}; + +unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx) +{ + struct mem_cgroup_per_node *pn; + long x; + + if (mem_cgroup_disabled()) + return node_page_state(lruvec_pgdat(lruvec), idx); + + pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); + x = READ_ONCE(pn->lruvec_stats->state[idx]); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; +} + +unsigned long lruvec_page_state_local(struct lruvec *lruvec, + enum node_stat_item idx) +{ + struct mem_cgroup_per_node *pn; + long x = 0; + + if (mem_cgroup_disabled()) + return node_page_state(lruvec_pgdat(lruvec), idx); + + pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); + x = READ_ONCE(pn->lruvec_stats->state_local[idx]); +#ifdef CONFIG_SMP + if (x < 0) + x = 0; +#endif + return x; +} + /* Subset of vm_event_item to report for memcg event stats */ static const unsigned int memcg_vm_event_stat[] = { PGPGIN, @@ -5492,18 +5546,25 @@ static int alloc_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) if (!pn) return 1; + pn->lruvec_stats = kzalloc_node(sizeof(struct lruvec_stats), GFP_KERNEL, + node); + if (!pn->lruvec_stats) + goto fail; + pn->lruvec_stats_percpu = alloc_percpu_gfp(struct lruvec_stats_percpu, GFP_KERNEL_ACCOUNT); - if (!pn->lruvec_stats_percpu) { - kfree(pn); - return 1; - } + if (!pn->lruvec_stats_percpu) + goto fail; lruvec_init(&pn->lruvec); pn->memcg = memcg; memcg->nodeinfo[node] = pn; return 0; +fail: + kfree(pn->lruvec_stats); + kfree(pn); + return 1; } static void free_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) @@ -5514,6 +5575,7 @@ static void free_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) return; free_percpu(pn->lruvec_stats_percpu); + kfree(pn->lruvec_stats); kfree(pn); } @@ -5866,18 +5928,19 @@ static void mem_cgroup_css_rstat_flush(struct cgroup_subsys_state *css, int cpu) for_each_node_state(nid, N_MEMORY) { struct mem_cgroup_per_node *pn = memcg->nodeinfo[nid]; - struct mem_cgroup_per_node *ppn = NULL; + struct lruvec_stats *lstats = pn->lruvec_stats; + struct lruvec_stats *plstats = NULL; struct lruvec_stats_percpu *lstatc; if (parent) - ppn = parent->nodeinfo[nid]; + plstats = parent->nodeinfo[nid]->lruvec_stats; lstatc = per_cpu_ptr(pn->lruvec_stats_percpu, cpu); for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { - delta = pn->lruvec_stats.state_pending[i]; + delta = lstats->state_pending[i]; if (delta) - pn->lruvec_stats.state_pending[i] = 0; + lstats->state_pending[i] = 0; delta_cpu = 0; v = READ_ONCE(lstatc->state[i]); @@ -5888,12 +5951,12 @@ static void mem_cgroup_css_rstat_flush(struct cgroup_subsys_state *css, int cpu) } if (delta_cpu) - pn->lruvec_stats.state_local[i] += delta_cpu; + lstats->state_local[i] += delta_cpu; if (delta) { - pn->lruvec_stats.state[i] += delta; - if (ppn) - ppn->lruvec_stats.state_pending[i] += delta; + lstats->state[i] += delta; + if (plstats) + plstats->state_pending[i] += delta; } } } From patchwork Sat Apr 27 00:37:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13645515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B485C4345F for ; Sat, 27 Apr 2024 00:37:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8960F6B0096; Fri, 26 Apr 2024 20:37:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 843876B0098; Fri, 26 Apr 2024 20:37:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6E7566B0099; Fri, 26 Apr 2024 20:37:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 4C12C6B0096 for ; Fri, 26 Apr 2024 20:37:53 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 00834C1441 for ; Sat, 27 Apr 2024 00:37:52 +0000 (UTC) X-FDA: 82053449184.16.21F790B Received: from out-174.mta1.migadu.com (out-174.mta1.migadu.com [95.215.58.174]) by imf17.hostedemail.com (Postfix) with ESMTP id 4142140008 for ; Sat, 27 Apr 2024 00:37:50 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=q3eCDp0J; spf=pass (imf17.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.174 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714178271; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CiP6jJ3z/Zb+7HCzkE9EgNFU2GbKmyiZu4OerwR5i68=; b=mDJiG68CvO1FKOs33ieHr/cJ1sVls/WYWjg6EHLFUxqw+r4ZyT9AGPXBlf6obYNSSLjrgy l8GWINbXiqhhKT/XcYGbvNDCdBolHFFhOhYl8erfKz6MQxSjkQb0BCanN4bt7iM/Nktamt NZqJufH6IDUlGfce5q3jABnvpezCv1E= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=q3eCDp0J; spf=pass (imf17.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.174 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714178271; a=rsa-sha256; cv=none; b=8oFF9ZiLubTb5b1nGfdzf9/vingO0oFOZqGP9K7uj57IvYLxTvactGK7lO6qhi6Mk4PCJG C0c8J8KvoWU7onsw9zyIlNEQh+ITNJcHzAg5bt/a2q8ZZUPQVdrVqUwjyiZ/t4ukdlIHY3 7W5gKN2aNAHIZsxf/2c65WPnYnBKe0w= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714178269; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CiP6jJ3z/Zb+7HCzkE9EgNFU2GbKmyiZu4OerwR5i68=; b=q3eCDp0J9gOqeEqaX8tgBo8ZzkToNZNxyobMgMVMJQ9HF/mBNKsNmj1YUeUiJcf9PW3Sre yx9grJUbPPPWOTn07Rw0XLZzPbR+tJ83YdB8f1HVsij6xYHqrTTowobd5UzmTg2kJ+OKbX cChmVNaWnbQODbqk5iXRD5Oq8ndtlew= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 3/7] memcg: reduce memory for the lruvec and memcg stats Date: Fri, 26 Apr 2024 17:37:29 -0700 Message-ID: <20240427003733.3898961-4-shakeel.butt@linux.dev> In-Reply-To: <20240427003733.3898961-1-shakeel.butt@linux.dev> References: <20240427003733.3898961-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Queue-Id: 4142140008 X-Rspamd-Server: rspam06 X-Stat-Signature: 9j488om87d8aq5yoone7rzm3zznr8ie7 X-HE-Tag: 1714178270-116948 X-HE-Meta: U2FsdGVkX18i6Wa5z8BnPHNnFKkxXFnT2AmW5rs/r7V2UFFrlSY8KV43DvIdEIZwVa9plHZmv1wtbJ0d+YGlbRDjxiq4OwxBX8P//WrmpcIOUMMRhfUSeYdJ5PRKzPOt3M0e+yTAzeJrqs4XlKWywN4+mqwt0kE4Z8xB0Y9fwLcL1rg0Op4A02wtphVPqNFxeXFpE8b3F6pwK0q4Dt3SfQHu/K2Bbzy/fElkrYEeLJaXEn4hNgNr1bOSQUHWJ1cweayzibbEtk+ycnisw78CNZw2oFmsn18Z1lCg4vQDM8GB1Qbx2hk/O1V2TzJQt0Uvacb5NOw/jutTw6Pq+PyrCdVDoHR4Ot7h9fc/Yq+09gF4CilOXe1csrpWx7e3aUxmgeCNqVidHfcvv0j0/r3eYzUarRcU8SsMJXQdDrBVDt008/fz0A0iWIx0zR9UwZv44aU1gsZzcG/S4F8IUaqbCgVVb72B5h5SiRXszK3Il7xDt1W4kxnWZiXSyNTc3AViWGan+Gmx5TmWF63olHg90RIIxSGxaaW79sJ6g7u4Ap/Xfhm6JnVMjsDjzB+PK5J5XwxBeUPKxIEkTRin936exeCakgcYuTfQ/GkJmHbnixgTe2iNjWokNfOEe8AJUIogrYJ6/FJ5qFQHK2ZTiv8JANBQ76QENlSkp8CjHfn1bZRugyA2x/SZu10PC4Y/ET5TKANJK9W9tMHlUb1lmA3CnhpzfKjF+2xUqULA48YjCBhcnhS0X8LrnMJaGhBMX8tNENU0Xh4f4OhTPM/XbEcTW3WOknrEP/ioDUGy4n/wq+sbFNGcaNK2YERkeVXsHddazkmkfC8ex/af9EqypfvLl7A2SnSxI56cztuQYOQyWN1/k0V62ZdzF3tNIeG/uLIyCdyCUgZZbXQrpsDd3L266MUFAZVNU5uUxLmZ7OxaSuPrP7vNvotkfdPQtoVrEpeClUfZCCxk4rNyt2+qOn0 GLgPk8/X LM7quqrz/PE16QknjNhrit7MeEKMJd27MuMaVhYnAxRF9japaMbrYuggkMUTAFtxHPL+mao/9lLCq8js= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: At the moment, the amount of memory allocated for stats related structs in the mem_cgroup corresponds to the size of enum node_stat_item. However not all fields in enum node_stat_item has corresponding memcg stats. So, let's use indirection mechanism similar to the one used for memcg vmstats management. For a given x86_64 config, the size of stats with and without patch is: structs size in bytes w/o with struct lruvec_stats 1128 648 struct lruvec_stats_percpu 752 432 struct memcg_vmstats 1832 1352 struct memcg_vmstats_percpu 1280 960 The memory savings is further compounded by the fact that these structs are allocated for each cpu and for each node. To be precise, for each memcg the memory saved would be: Memory saved = ((21 * 3 * NR_NODES) + (21 * 2 * NR_NODS * NR_CPUS) + (21 * 3) + (21 * 2 * NR_CPUS)) * sizeof(long) Where 21 is the number of fields eliminated. Signed-off-by: Shakeel Butt --- mm/memcontrol.c | 138 ++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 115 insertions(+), 23 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 5e337ed6c6bf..c164bc9b8ed6 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -576,35 +576,105 @@ mem_cgroup_largest_soft_limit_node(struct mem_cgroup_tree_per_node *mctz) return mz; } +/* Subset of node_stat_item for memcg stats */ +static const unsigned int memcg_node_stat_items[] = { + NR_INACTIVE_ANON, + NR_ACTIVE_ANON, + NR_INACTIVE_FILE, + NR_ACTIVE_FILE, + NR_UNEVICTABLE, + NR_SLAB_RECLAIMABLE_B, + NR_SLAB_UNRECLAIMABLE_B, + WORKINGSET_REFAULT_ANON, + WORKINGSET_REFAULT_FILE, + WORKINGSET_ACTIVATE_ANON, + WORKINGSET_ACTIVATE_FILE, + WORKINGSET_RESTORE_ANON, + WORKINGSET_RESTORE_FILE, + WORKINGSET_NODERECLAIM, + NR_ANON_MAPPED, + NR_FILE_MAPPED, + NR_FILE_PAGES, + NR_FILE_DIRTY, + NR_WRITEBACK, + NR_SHMEM, + NR_SHMEM_THPS, + NR_FILE_THPS, + NR_ANON_THPS, + NR_KERNEL_STACK_KB, + NR_PAGETABLE, + NR_SECONDARY_PAGETABLE, +#ifdef CONFIG_SWAP + NR_SWAPCACHE, +#endif +}; + +static const unsigned int memcg_stat_items[] = { + MEMCG_SWAP, + MEMCG_SOCK, + MEMCG_PERCPU_B, + MEMCG_VMALLOC, + MEMCG_KMEM, + MEMCG_ZSWAP_B, + MEMCG_ZSWAPPED, +}; + +#define NR_MEMCG_NODE_STAT_ITEMS ARRAY_SIZE(memcg_node_stat_items) +#define NR_MEMCG_STATS (NR_MEMCG_NODE_STAT_ITEMS + ARRAY_SIZE(memcg_stat_items)) +static int8_t mem_cgroup_stats_index[MEMCG_NR_STAT] __read_mostly; + +static void init_memcg_stats(void) +{ + int8_t i, j = 0; + + /* Switch to short once this failure occurs. */ + BUILD_BUG_ON(NR_MEMCG_STATS >= 127 /* INT8_MAX */); + + for (i = 0; i < NR_MEMCG_NODE_STAT_ITEMS; ++i) + mem_cgroup_stats_index[memcg_node_stat_items[i]] = ++j; + + for (i = 0; i < ARRAY_SIZE(memcg_stat_items); ++i) + mem_cgroup_stats_index[memcg_stat_items[i]] = ++j; +} + +static inline int memcg_stats_index(int idx) +{ + return mem_cgroup_stats_index[idx] - 1; +} + struct lruvec_stats_percpu { /* Local (CPU and cgroup) state */ - long state[NR_VM_NODE_STAT_ITEMS]; + long state[NR_MEMCG_NODE_STAT_ITEMS]; /* Delta calculation for lockless upward propagation */ - long state_prev[NR_VM_NODE_STAT_ITEMS]; + long state_prev[NR_MEMCG_NODE_STAT_ITEMS]; }; struct lruvec_stats { /* Aggregated (CPU and subtree) state */ - long state[NR_VM_NODE_STAT_ITEMS]; + long state[NR_MEMCG_NODE_STAT_ITEMS]; /* Non-hierarchical (CPU aggregated) state */ - long state_local[NR_VM_NODE_STAT_ITEMS]; + long state_local[NR_MEMCG_NODE_STAT_ITEMS]; /* Pending child counts during tree propagation */ - long state_pending[NR_VM_NODE_STAT_ITEMS]; + long state_pending[NR_MEMCG_NODE_STAT_ITEMS]; }; unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx) { struct mem_cgroup_per_node *pn; - long x; + long x = 0; + int i; if (mem_cgroup_disabled()) return node_page_state(lruvec_pgdat(lruvec), idx); - pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x = READ_ONCE(pn->lruvec_stats->state[idx]); + i = memcg_stats_index(idx); + if (i >= 0) { + pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); + x = READ_ONCE(pn->lruvec_stats->state[i]); + } #ifdef CONFIG_SMP if (x < 0) x = 0; @@ -617,12 +687,16 @@ unsigned long lruvec_page_state_local(struct lruvec *lruvec, { struct mem_cgroup_per_node *pn; long x = 0; + int i; if (mem_cgroup_disabled()) return node_page_state(lruvec_pgdat(lruvec), idx); - pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - x = READ_ONCE(pn->lruvec_stats->state_local[idx]); + i = memcg_stats_index(idx); + if (i >= 0) { + pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); + x = READ_ONCE(pn->lruvec_stats->state_local[i]); + } #ifdef CONFIG_SMP if (x < 0) x = 0; @@ -690,11 +764,11 @@ struct memcg_vmstats_percpu { /* The above should fit a single cacheline for memcg_rstat_updated() */ /* Local (CPU and cgroup) page state & events */ - long state[MEMCG_NR_STAT]; + long state[NR_MEMCG_STATS]; unsigned long events[NR_MEMCG_EVENTS]; /* Delta calculation for lockless upward propagation */ - long state_prev[MEMCG_NR_STAT]; + long state_prev[NR_MEMCG_STATS]; unsigned long events_prev[NR_MEMCG_EVENTS]; /* Cgroup1: threshold notifications & softlimit tree updates */ @@ -704,15 +778,15 @@ struct memcg_vmstats_percpu { struct memcg_vmstats { /* Aggregated (CPU and subtree) page state & events */ - long state[MEMCG_NR_STAT]; + long state[NR_MEMCG_STATS]; unsigned long events[NR_MEMCG_EVENTS]; /* Non-hierarchical (CPU aggregated) page state & events */ - long state_local[MEMCG_NR_STAT]; + long state_local[NR_MEMCG_STATS]; unsigned long events_local[NR_MEMCG_EVENTS]; /* Pending child counts during tree propagation */ - long state_pending[MEMCG_NR_STAT]; + long state_pending[NR_MEMCG_STATS]; unsigned long events_pending[NR_MEMCG_EVENTS]; /* Stats updates since the last flush */ @@ -845,7 +919,13 @@ static void flush_memcg_stats_dwork(struct work_struct *w) unsigned long memcg_page_state(struct mem_cgroup *memcg, int idx) { - long x = READ_ONCE(memcg->vmstats->state[idx]); + long x; + int i = memcg_stats_index(idx); + + if (i < 0) + return 0; + + x = READ_ONCE(memcg->vmstats->state[i]); #ifdef CONFIG_SMP if (x < 0) x = 0; @@ -877,18 +957,25 @@ static int memcg_state_val_in_pages(int idx, int val) */ void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val) { - if (mem_cgroup_disabled()) + int i = memcg_stats_index(idx); + + if (mem_cgroup_disabled() || i < 0) return; - __this_cpu_add(memcg->vmstats_percpu->state[idx], val); + __this_cpu_add(memcg->vmstats_percpu->state[i], val); memcg_rstat_updated(memcg, memcg_state_val_in_pages(idx, val)); } /* idx can be of type enum memcg_stat_item or node_stat_item. */ static unsigned long memcg_page_state_local(struct mem_cgroup *memcg, int idx) { - long x = READ_ONCE(memcg->vmstats->state_local[idx]); + long x; + int i = memcg_stats_index(idx); + + if (i < 0) + return 0; + x = READ_ONCE(memcg->vmstats->state_local[i]); #ifdef CONFIG_SMP if (x < 0) x = 0; @@ -902,6 +989,10 @@ static void __mod_memcg_lruvec_state(struct lruvec *lruvec, { struct mem_cgroup_per_node *pn; struct mem_cgroup *memcg; + int i = memcg_stats_index(idx); + + if (i < 0) + return; pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); memcg = pn->memcg; @@ -931,10 +1022,10 @@ static void __mod_memcg_lruvec_state(struct lruvec *lruvec, } /* Update memcg */ - __this_cpu_add(memcg->vmstats_percpu->state[idx], val); + __this_cpu_add(memcg->vmstats_percpu->state[i], val); /* Update lruvec */ - __this_cpu_add(pn->lruvec_stats_percpu->state[idx], val); + __this_cpu_add(pn->lruvec_stats_percpu->state[i], val); memcg_rstat_updated(memcg, memcg_state_val_in_pages(idx, val)); memcg_stats_unlock(); @@ -5702,6 +5793,7 @@ mem_cgroup_css_alloc(struct cgroup_subsys_state *parent_css) page_counter_init(&memcg->kmem, &parent->kmem); page_counter_init(&memcg->tcpmem, &parent->tcpmem); } else { + init_memcg_stats(); init_memcg_events(); page_counter_init(&memcg->memory, NULL); page_counter_init(&memcg->swap, NULL); @@ -5873,7 +5965,7 @@ static void mem_cgroup_css_rstat_flush(struct cgroup_subsys_state *css, int cpu) statc = per_cpu_ptr(memcg->vmstats_percpu, cpu); - for (i = 0; i < MEMCG_NR_STAT; i++) { + for (i = 0; i < NR_MEMCG_STATS; i++) { /* * Collect the aggregated propagation counts of groups * below us. We're in a per-cpu loop here and this is @@ -5937,7 +6029,7 @@ static void mem_cgroup_css_rstat_flush(struct cgroup_subsys_state *css, int cpu) lstatc = per_cpu_ptr(pn->lruvec_stats_percpu, cpu); - for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { + for (i = 0; i < NR_MEMCG_NODE_STAT_ITEMS; i++) { delta = lstats->state_pending[i]; if (delta) lstats->state_pending[i] = 0; From patchwork Sat Apr 27 00:37:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13645516 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5874C4345F for ; Sat, 27 Apr 2024 00:37:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A0D316B0098; Fri, 26 Apr 2024 20:37:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9A4F46B0099; Fri, 26 Apr 2024 20:37:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 86DA36B009A; Fri, 26 Apr 2024 20:37:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 652036B0098 for ; Fri, 26 Apr 2024 20:37:56 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2734D806DB for ; Sat, 27 Apr 2024 00:37:56 +0000 (UTC) X-FDA: 82053449352.20.3ABEEF8 Received: from out-180.mta1.migadu.com (out-180.mta1.migadu.com [95.215.58.180]) by imf20.hostedemail.com (Postfix) with ESMTP id A35251C0011 for ; Sat, 27 Apr 2024 00:37:54 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=b6gjrMhu; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf20.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.180 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714178274; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=StJiZmI3BplMAQI82xrhJ2SVU48fOC0iWz8HtDgUnJo=; b=WYO9hL49MV8vAjl3+jC+06d1EAy5EEPTAG+ojWGaLYD5Vl5r8zLDO/lPljPbdIr2VwK99y mI+mpYH2g5JQmL1+CQK1MsgRieOOwDe75/UwXDosPMK3pSbaclEy/Eph/Rwji0/Lyt9OFX WDTsE91puAbB36dEzRcnEoo9sgVOxv0= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=b6gjrMhu; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf20.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.180 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714178274; a=rsa-sha256; cv=none; b=Z7RElAGr9cD+Q/c01rpL60qj2Si+evas71Zv4CwWouLNLiZhjw021Ja8qvvpEY3jpXfEmB dyqwo0S9WXXV4Af04K12hMabZrRbyOKAsP/rxjekPYXxsZjt/h/XyAUAC9s4u3CQ/jl6DQ hd5eI53jTXi0Uu1AbOZ0aBNfBGyjPGo= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714178273; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=StJiZmI3BplMAQI82xrhJ2SVU48fOC0iWz8HtDgUnJo=; b=b6gjrMhuur3+QzQZZhAjDMsktwDbCY5i5Y/S3EnI6cT4PrqrIopqb7fjdGAwV1+f85fJkj oXiUuqCSxo3ONQADEVCNbYBnGKPDW1uurVXpqggrM7xrGlwYB8WMKdS6/0/lRWwBF5Hmo5 wt+amLltg08cTAW21mxIRKMXJMRUjFc= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 4/7] memcg: cleanup __mod_memcg_lruvec_state Date: Fri, 26 Apr 2024 17:37:30 -0700 Message-ID: <20240427003733.3898961-5-shakeel.butt@linux.dev> In-Reply-To: <20240427003733.3898961-1-shakeel.butt@linux.dev> References: <20240427003733.3898961-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Stat-Signature: r8uaiehu14dkdtwrwbpqm76zoybi5y5u X-Rspamd-Queue-Id: A35251C0011 X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1714178274-758128 X-HE-Meta: U2FsdGVkX1+0Dk5Nue56rznXvDcQ7Sy61MkevuWzXKKFBlS+QF9pj1EV9Gw2DByXF9nHDWm7lmbtZ6rbQ0ZhXuCKpUjqwqOlgE63vH/GBkyXruaKDlmlAPjibcBb3DXLNMHgEtFDij7lCrElhvo8sbSnAsQ6h+4nWihf/2elQYXnVuUhMHZnPNqt7Od92bZfJQyviWeQL3UeSJnkwigxHeynFmOVkVVzmIC1Wf6FagxzjpIGJJsM+bSMvxzmAjiKG4VjHPTr6KlZNY1EwPnt9LcfT6B86AO4Uj+zLgC066zBYknMOEP9gCDNbhs2ZjY+pAIeQdzsqPgG75Dz6iOD8ptswu/kvgrKvS5FVru5doejIN4D4TmD+N8iZPiyWHMVEWKpEsePCUMy6pc1P0r4i5kyR8PE4sDDyObOkrNSYbYvcpBSU9TxB0XEljLgaLHYHEGsEJqLCMhb3e0cqzvpiw39HoPc1KA9MPdC+SvcojqYXaSfL7zl3PVyJj5oKXyuN6JoW8Zfu6tUOZDT0jnJHRcZ/U9cpt88kSP8WgUId6vEsLtFH8tssmG4pgKC+m/jPR/vfopisgJX2N4HQxzC8NuY7GkkkSQgSr1ZNUAdOiNx0qQJoa8fr4g7yjIOUQzSlYnZoAtiIRo58GESgTADgksqD6MQmhKo9ighx3V9gNzxw5U6s3BdD4+602oMIqp7rm6PJKEpVIrtKF08KZN74IdFIqCLgUCQniv42ppSPGTYH+WXtQO813SSiCtw8j6qNfx4NF65ITdrmKGr5R7Ua7r3FfNfxE4JVOjyaZ6X5w7vCpn13xHdxgXGzOqLW38MgJBXq950zZSwgOzUl4U4ZtfYsHV7p6HiythGirCoTGopxMxK6EoFwlvUvheT55yk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There are no memcg specific stats for NR_SHMEM_PMDMAPPED and NR_FILE_PMDMAPPED. Let's remove them. Signed-off-by: Shakeel Butt Reviewed-by: Yosry Ahmed Reviewed-by: Roman Gushchin --- mm/memcontrol.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c164bc9b8ed6..103e0e53e20a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1009,8 +1009,6 @@ static void __mod_memcg_lruvec_state(struct lruvec *lruvec, case NR_ANON_MAPPED: case NR_FILE_MAPPED: case NR_ANON_THPS: - case NR_SHMEM_PMDMAPPED: - case NR_FILE_PMDMAPPED: if (WARN_ON_ONCE(!in_task())) pr_warn("stat item index: %d\n", idx); break; From patchwork Sat Apr 27 00:37:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13645517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D615C4345F for ; Sat, 27 Apr 2024 00:38:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9CF0E6B0099; Fri, 26 Apr 2024 20:37:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 908C26B009B; Fri, 26 Apr 2024 20:37:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 75BE86B009C; Fri, 26 Apr 2024 20:37:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 4EEC96B0099 for ; Fri, 26 Apr 2024 20:37:59 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id CA90C4015B for ; Sat, 27 Apr 2024 00:37:58 +0000 (UTC) X-FDA: 82053449436.23.FDBB1A3 Received: from out-176.mta1.migadu.com (out-176.mta1.migadu.com [95.215.58.176]) by imf18.hostedemail.com (Postfix) with ESMTP id 1CAD91C0002 for ; Sat, 27 Apr 2024 00:37:56 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=houO09vD; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf18.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.176 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714178277; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zLHD4gb8t6zDRcF8qzenGgWmsGCHlQSRICk2vCIqKfk=; b=OAfeVf4mkUrics7k88HviOdGVMdk2hgpH0wUVk7p4oWJMD2Kvu2C8xQYY7eDCI36YUbH1E DMI+k1HpCPL565JvX1wdzZoMk82UvlfZUU3PBXlooBTcGoyod3MK6SX+NbiptKYWHQnL/n RLmW7d2zIw4KmRFzqEhOyT83PR4whB8= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=houO09vD; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf18.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.176 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714178277; a=rsa-sha256; cv=none; b=fGXyTZBEzah2cSN97B6lASpFM6iIHbXhGijsPVVsNLmjIsuCo+ZJadYUa0Nalz+NUqWuZ3 uLXiWxjlazZznVA6kPs2LihBMCA80JI57AvledDsLAJCt7jP1KH0TkhvlRL7j0BYy6qpLf 69hsEs54M2dsNr3A9sqTGD+D2sEKQKU= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714178275; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zLHD4gb8t6zDRcF8qzenGgWmsGCHlQSRICk2vCIqKfk=; b=houO09vDhohFagXTbOfhLSJqhpc84FITSFNLRlY6ylNtuLjFRkyPyDJyqpgi6EbQHK4FZn RhE2P7AnzkgfYA/sfqlnm1T5oG8du97LuVbDILIJXqYXe1+56MCMYjkGUrV8Oga2UzpQz5 EeBGScH0Y4HpVoFLboUXkjNMFabeuPk= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 5/7] memcg: pr_warn_once for unexpected events and stats Date: Fri, 26 Apr 2024 17:37:31 -0700 Message-ID: <20240427003733.3898961-6-shakeel.butt@linux.dev> In-Reply-To: <20240427003733.3898961-1-shakeel.butt@linux.dev> References: <20240427003733.3898961-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 1CAD91C0002 X-Stat-Signature: 3omnu3eoikq6iosz6ib6ixw8tkpqjtp7 X-HE-Tag: 1714178276-136177 X-HE-Meta: U2FsdGVkX1+uyHXxVIYQ+xqVgUxM4Jfi/5BYDY8LIO7TAxxoQrK4TOCPEb3LCcz8Nkd1WT7eTWfZJ4AzKhN8nSg8E6J1HvBsCDxUcqZlIc6gj+WQs+cOfanaMTUe/K9eFlie3NRAn/WVhvqcuul+89jWB3o/Z1IBNUXjOmAORjCVzqaQg//R94BxgD+11HlSV3rcp6xJRLFnSSENYhYDMbgZE4HGKpdhljVg2BkJV/gcUj7abVd6uWNa8UVgsJNzcv0VFMdxbkySKRolkjlPAu09Q/FeJZZFuW4RksSncIrr1nt2+tibZdOggtlR5ZID+FOWCt2xjm7lrxtWGuD9xGALmZIEc++ekvKKLlaEf7u9HEzS5nra8aYgxixK0YYYQ4S94vVgm9JqARQhW2BVp50w7LFjumSVXWyQmYqiqJJWv1it2IEfIuDBOJHXU7ZP/n8bkrLtLao8dGbfhFaLiq+fVWhiD9O4hzM/+5+DLUsCTdF8MuPg0EnTrJdciEUgeLh+Gf7Hq0FgDU5Z0kkLCgW5tliA6zT29H/VzTbBdDrNTsY5Bi19iX42l4H/61hUasJK3HtMh/qSNATWafDUqz2NDTJBi3TCoxjr6oXjxpyiobDQ3tvriO0N2GCQF2HraKLgXSCMN0otprzhBBBnD1mmiBmMvP/5jaTyRewxTq/XvrbcPxBoBncF/D4+ExThkj6iFTv8L2Dvrg4qy1A/eCktdjnQnXApxL/oAESAZZ41Prsq3z0ZQ0NxR3zVuMPKeItIsRf74HNbdAjlkoV+/u6LJD4cAaiNtEpsIpALLpezzuwM78/iRCj2OP+q93SIB8XtXvQIHwSMebhh/gIebtal+0t2wIpmZAXqiyN3rHgOsP2fEci8pOMxCsjEEXPi93ptWAD/4kXoXEDUdJ+xJAdHIRR24+LreyMB845L2jtJMow8KonfCE+WHT3JT0I8yT1usG2HTBrwrdMhevF KynBpIUp pSZkkcBb+pPcKEHEwzReJrFW5OMvD1t+Mjec9J9Flo6wSg0RfQSPnOBvTaAlK6TLLuPIWxgXyGjhNbmr9Cqd+ZOfhr5MwM6a2RqQxqri7UT7D0bW9AAedFvJnMg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To reduce memory usage by the memcg events and stats, the kernel uses indirection table and only allocate stats and events which are being used by the memcg code. To make this more robust, let's add warnings where unexpected stats and events indexes are used. Signed-off-by: Shakeel Butt --- mm/memcontrol.c | 43 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 34 insertions(+), 9 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 103e0e53e20a..36145089dcf5 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -671,9 +671,11 @@ unsigned long lruvec_page_state(struct lruvec *lruvec, enum node_stat_item idx) return node_page_state(lruvec_pgdat(lruvec), idx); i = memcg_stats_index(idx); - if (i >= 0) { + if (likely(i >= 0)) { pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); x = READ_ONCE(pn->lruvec_stats->state[i]); + } else { + pr_warn_once("%s: stat item index: %d\n", __func__, idx); } #ifdef CONFIG_SMP if (x < 0) @@ -693,9 +695,11 @@ unsigned long lruvec_page_state_local(struct lruvec *lruvec, return node_page_state(lruvec_pgdat(lruvec), idx); i = memcg_stats_index(idx); - if (i >= 0) { + if (likely(i >= 0)) { pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); x = READ_ONCE(pn->lruvec_stats->state_local[i]); + } else { + pr_warn_once("%s: stat item index: %d\n", __func__, idx); } #ifdef CONFIG_SMP if (x < 0) @@ -922,8 +926,10 @@ unsigned long memcg_page_state(struct mem_cgroup *memcg, int idx) long x; int i = memcg_stats_index(idx); - if (i < 0) + if (unlikely(i < 0)) { + pr_warn_once("%s: stat item index: %d\n", __func__, idx); return 0; + } x = READ_ONCE(memcg->vmstats->state[i]); #ifdef CONFIG_SMP @@ -959,8 +965,13 @@ void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val) { int i = memcg_stats_index(idx); - if (mem_cgroup_disabled() || i < 0) + if (mem_cgroup_disabled()) + return; + + if (unlikely(i < 0)) { + pr_warn_once("%s: stat item index: %d\n", __func__, idx); return; + } __this_cpu_add(memcg->vmstats_percpu->state[i], val); memcg_rstat_updated(memcg, memcg_state_val_in_pages(idx, val)); @@ -972,8 +983,10 @@ static unsigned long memcg_page_state_local(struct mem_cgroup *memcg, int idx) long x; int i = memcg_stats_index(idx); - if (i < 0) + if (unlikely(i < 0)) { + pr_warn_once("%s: stat item index: %d\n", __func__, idx); return 0; + } x = READ_ONCE(memcg->vmstats->state_local[i]); #ifdef CONFIG_SMP @@ -991,8 +1004,10 @@ static void __mod_memcg_lruvec_state(struct lruvec *lruvec, struct mem_cgroup *memcg; int i = memcg_stats_index(idx); - if (i < 0) + if (unlikely(i < 0)) { + pr_warn_once("%s: stat item index: %d\n", __func__, idx); return; + } pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); memcg = pn->memcg; @@ -1107,8 +1122,13 @@ void __count_memcg_events(struct mem_cgroup *memcg, enum vm_event_item idx, { int index = memcg_events_index(idx); - if (mem_cgroup_disabled() || index < 0) + if (mem_cgroup_disabled()) + return; + + if (unlikely(index < 0)) { + pr_warn_once("%s: event item index: %d\n", __func__, idx); return; + } memcg_stats_lock(); __this_cpu_add(memcg->vmstats_percpu->events[index], count); @@ -1120,8 +1140,11 @@ static unsigned long memcg_events(struct mem_cgroup *memcg, int event) { int index = memcg_events_index(event); - if (index < 0) + if (unlikely(index < 0)) { + pr_warn_once("%s: event item index: %d\n", __func__, event); return 0; + } + return READ_ONCE(memcg->vmstats->events[index]); } @@ -1129,8 +1152,10 @@ static unsigned long memcg_events_local(struct mem_cgroup *memcg, int event) { int index = memcg_events_index(event); - if (index < 0) + if (unlikely(index < 0)) { + pr_warn_once("%s: event item index: %d\n", __func__, event); return 0; + } return READ_ONCE(memcg->vmstats->events_local[index]); } From patchwork Sat Apr 27 00:37:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13645518 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B05A8C4345F for ; Sat, 27 Apr 2024 00:38:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3DAB86B009D; Fri, 26 Apr 2024 20:38:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 388766B009E; Fri, 26 Apr 2024 20:38:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2034A6B009F; Fri, 26 Apr 2024 20:38:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 02BB16B009D for ; Fri, 26 Apr 2024 20:38:04 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7653A406C4 for ; Sat, 27 Apr 2024 00:38:04 +0000 (UTC) X-FDA: 82053449688.16.9A51479 Received: from out-177.mta0.migadu.com (out-177.mta0.migadu.com [91.218.175.177]) by imf18.hostedemail.com (Postfix) with ESMTP id BC1B31C0006 for ; Sat, 27 Apr 2024 00:38:02 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=sivGlrSp; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf18.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.177 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714178282; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Gx3PKav0CetNdFI0jFUffF6lUGwk4q4qKzC3INQ2E8A=; b=LNZPVg0gmZ2w/l6I3Cuq9bsGksDONVbQIJ75Ph07itG1cCgpUsyR9squtAty6JYVhEdvsw KAq2wS1e9dMHp6JQgDAtBFZzkbaaJxqjSrCicD50qo7HbUMOoWnHJaBbV5MAWlkwRAjDxZ Dyu7a5kpjfKsOuj//RHeNkKT4rHy5OA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714178282; a=rsa-sha256; cv=none; b=awXmOo8lzvdCLsn7g9ptHZRBHkWjHUtYfTCUYj4zeVKR2tAYT2MyOg6ElRI7sS9XD+k7l/ pVAIGV1L0U3j7Z8cSpRl2GkjquC4wX5WSmoCcTDvKPBEYa8VbyF4THecOe3WhnulyQpIWw flDWc9GAMC6VKxb9cfuNIeaRqNc7axg= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=sivGlrSp; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf18.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.177 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714178280; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gx3PKav0CetNdFI0jFUffF6lUGwk4q4qKzC3INQ2E8A=; b=sivGlrSpl81qjvi6K7a/ZlE+lX7x9zLqRfOxT3jTy3TpHNeCbRa/CRH5yGHNGDKvqIijXu BGuhy0LIWV81fc/WqvGITWNss9qyDo50w1qvtd+jaICAMwbaUac4gGWLUWsi6XL7uBHCmV pOaOEV7xvE4/E41E676iu4kaYa9KQZU= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 6/7] memcg: use proper type for mod_memcg_state Date: Fri, 26 Apr 2024 17:37:32 -0700 Message-ID: <20240427003733.3898961-7-shakeel.butt@linux.dev> In-Reply-To: <20240427003733.3898961-1-shakeel.butt@linux.dev> References: <20240427003733.3898961-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: BC1B31C0006 X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: 681szb4xskrj9itiw1gbwreikhf3ahr9 X-HE-Tag: 1714178282-689502 X-HE-Meta: U2FsdGVkX1/Lj7Q2InDKv9G30ZyTmt7MRaScs7tckDyqV3ttDrUy43zEeq5GCv/ihoiUgJC4XKmHoEPWdsIvgyBc3Hc4HZyjn1pkOcDgWU+crtCeE8ZkBbxBsMxn4uVn6gfIX1bZ2MWRcfEC5m3lXOva9PweMIOJzm9xFqsoVkh8KeWX4TmLxxmMIejYbt1PapIlNUFWb9NlIVrEezx4LASJnUE0PvAJXLZtK3FmmjsFZVcvY8uKBJNnHpetbbKqWa1B74B9751vn4eDOZEV0kTqbWvpKVo6UTXVpmuqQbO3dRIUcjVRSCGuSPvc+vkcUHfVzvrg04C8p3fWOcveoOTeWSMIyIEzVVH0FD/kMw7tIlJx+P4iYKIK5d4ckJ7BhOrEtRn9OgnWZ5Lcyr1DLQb9zYDF/TBCFb7xkOXnQg8zZoaNplW7ynRss+urOIWAO+mVOpKJg/Iw2oO+EbGLGFOT/fdPMZhjkfqmxE7v0YMOtKo6j1N0hsDrede4e5WEVeRC5Bzp2QUbcdcXtdp3lIPI54+F6EUO5VIKg4c+q/oPBAf1FyPjYJp1E92ZPboD/rqodNFtDgw4Mlu0zi4xX/xc9U5aF2g+ZpqOFn5/0bBa11Pjr+2+YZrkhN9aAsA6UjxeUNKh420hlkBVws2AdQp1KenF9EJaFxFm8DK2rVRg+q06Mdw355q/LjUCP10kuvCK5yEN16Tw+5Pgczn3mr/Q4IqF1JSIBVb8nxnSvfExFDmPBvWFVcBbiz5r69ggz4kh3yxbPm4Y4WQ8Cf+mD5dmACkfn/a0H5rUw+q2HD9iZkAcHTLrsuG7zL2sVLZbaD5t+Lt+fGktQogia5J65SNHcdklE8Te9LKQ+NGtP9JrHvclGVVLimhX2ch6iCgKd4u+g55yT0dwGJYw4wUixE1QPqJvxPTi8GHvGYLtzOzZzrTm5RxiOTBpOZD8cbFMWajEaZPsn2OFpY4kGuH Wk7r8avo tH0OAbu7C9/9RL1NFb8z3toIZxjNRH/7bThwXa/Qej2Ib4/FsTs+6L7VOTVvZrWtcDXGe4zXqgrU1AHuYVY01FKrPkFJCM/DJkGqPTea2CQ9ts4tnogOJKDLInaglD8NcgQhu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The memcg stats update functions can take arbitrary integer but the only input which make sense is enum memcg_stat_item and we don't want these functions to be called with arbitrary integer, so replace the parameter type with enum memcg_stat_item and compiler will be able to warn if memcg stat update functions are called with incorrect index value. Signed-off-by: Shakeel Butt --- include/linux/memcontrol.h | 13 +++++++------ mm/memcontrol.c | 3 ++- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index ab8a6e884375..73cad69dfb5a 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -974,7 +974,8 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg); void folio_memcg_lock(struct folio *folio); void folio_memcg_unlock(struct folio *folio); -void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val); +void __mod_memcg_state(struct mem_cgroup *memcg, enum memcg_stat_item idx, + int val); /* try to stablize folio_memcg() for all the pages in a memcg */ static inline bool mem_cgroup_trylock_pages(struct mem_cgroup *memcg) @@ -995,7 +996,7 @@ static inline void mem_cgroup_unlock_pages(void) /* idx can be of type enum memcg_stat_item or node_stat_item */ static inline void mod_memcg_state(struct mem_cgroup *memcg, - int idx, int val) + enum memcg_stat_item idx, int val) { unsigned long flags; @@ -1005,7 +1006,7 @@ static inline void mod_memcg_state(struct mem_cgroup *memcg, } static inline void mod_memcg_page_state(struct page *page, - int idx, int val) + enum memcg_stat_item idx, int val) { struct mem_cgroup *memcg; @@ -1491,19 +1492,19 @@ static inline void mem_cgroup_print_oom_group(struct mem_cgroup *memcg) } static inline void __mod_memcg_state(struct mem_cgroup *memcg, - int idx, + enum memcg_stat_item idx, int nr) { } static inline void mod_memcg_state(struct mem_cgroup *memcg, - int idx, + enum memcg_stat_item idx, int nr) { } static inline void mod_memcg_page_state(struct page *page, - int idx, int val) + enum memcg_stat_item idx, int val) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 36145089dcf5..d11536ef59ef 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -961,7 +961,8 @@ static int memcg_state_val_in_pages(int idx, int val) * @idx: the stat item - can be enum memcg_stat_item or enum node_stat_item * @val: delta to add to the counter, can be negative */ -void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val) +void __mod_memcg_state(struct mem_cgroup *memcg, enum memcg_stat_item idx, + int val) { int i = memcg_stats_index(idx); From patchwork Sat Apr 27 00:37:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13645519 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5633C4345F for ; Sat, 27 Apr 2024 00:38:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2EF996B009F; Fri, 26 Apr 2024 20:38:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 29F6F6B00A0; Fri, 26 Apr 2024 20:38:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0F37F6B00A1; Fri, 26 Apr 2024 20:38:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id E35F96B009F for ; Fri, 26 Apr 2024 20:38:07 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 695F412021D for ; Sat, 27 Apr 2024 00:38:07 +0000 (UTC) X-FDA: 82053449814.24.7DFD8E9 Received: from out-182.mta1.migadu.com (out-182.mta1.migadu.com [95.215.58.182]) by imf27.hostedemail.com (Postfix) with ESMTP id C592440005 for ; Sat, 27 Apr 2024 00:38:05 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=SncPF2vo; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf27.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.182 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714178285; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XaVmG+nL1Z/KgIhQvtyGGfErTWQGfN9RM5fOWCMfM60=; b=iLjax5SLqb48HnKHmpv4/7aMRUk2vriVeo/ZqQM7V/bmgLnyVoJL8PFj+9oDJHhF6yzOh5 fooiTjbPQ4mtPNlpoIuv22CBftslGzkWJpHDl1KTQL1RrSGb6jcvv6HjxqnZj4/IU0ohJB mRPtQSPWktKwwkI64SQ3IfsstVLTu3w= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=SncPF2vo; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf27.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.182 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714178286; a=rsa-sha256; cv=none; b=Kun+ejlleshwLmubyCV5lL92MzEgCcm8oTOMRT3E0rG1/Hi6lgsU3agzXTRPMawUMU9t+9 kiDUI59MdfQndQbrzhWooBqXnZafTp4PT1nywKJNfhOehZQRpWA4gXVXuix6LoikqoTEJP LC06Zy3VvWx/nYQtgppP4l4vW78fqX4= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714178284; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XaVmG+nL1Z/KgIhQvtyGGfErTWQGfN9RM5fOWCMfM60=; b=SncPF2vo7Gj5pGR2HtW7z18CEr5iYUMsR9J3GEV4lMaLOx6+PYwok7wMxTGAqdmd/XJczt pZgDrZ1qNeEJIRnvjIs1Aeyf4VLQRcEnJYOnsAE/Dy3rzsexN2xS0tWE7DQUd+u6a/yuZU qWm6PPDolOHny72+IBIloElPU+qJuFI= From: Shakeel Butt To: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 7/7] mm: cleanup WORKINGSET_NODES in workingset Date: Fri, 26 Apr 2024 17:37:33 -0700 Message-ID: <20240427003733.3898961-8-shakeel.butt@linux.dev> In-Reply-To: <20240427003733.3898961-1-shakeel.butt@linux.dev> References: <20240427003733.3898961-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam01 X-Stat-Signature: s5hzsgoykws6ehniip5n8uhydt6j1czq X-Rspam-User: X-Rspamd-Queue-Id: C592440005 X-HE-Tag: 1714178285-515199 X-HE-Meta: U2FsdGVkX1+ZdOgDjJcVN+uIURLxvHimQsujzSjyRInsU9niZ/ApVVhoiiPoIMZ8HzKPl+xZ8Iz0Eo7QZUbhiwO0w/HmHHAjx4qQthKPYwq5r81osvYujnxL5fqMzqtS1I4YCt8eiYR6uJwkxtek0+EjXhsY54nS+kTqccSTXPTR3nEACI5bFZ20JDqsujwxCjw35/hoI0ZyWjRzc3rACRwUV+4wG/ZTsAxZfk10BaAZpcBkgefe6fQlfoNPTq16cUtfBuB6pNJsh6A0xF/4GNyv12754vU9hlD+yGKXhPCfU5wj3LLlRrxe96W8EfMDOLy7ZePCCdShxvNf+yu9lvvrBUkiVhx6CNTZyUCxgmUt4ak8bKonps0dauraB6sSZb+yNWuoo+x90fGnWf7RdowGGLoRz1XT+F7QbcC3NbC/Qzy2l5/R/dgenKthcFE14kiI23IeE9f9SLPh065f2fb/AAOo83AWeuJWYmBBQgq+xmfb7O+zp2N7krRiKx0uBc5ig9f+mYgQ37m48ksW7jCMLu5sggQlqnCjgP9IZXKg+HIwj+QypbiyWAnGoFfXn0vEP4PF5B4x4Vmz9Wp17ML3tK3LXYYDTl+ABdyr60Nc2YNjC7m4BIWWQPPBuV29csXxBmkPH2qImYvbI2sMrd2xAt27SbKITwcAU97//0JytjR8f1M+wwp33b+o8lmN9pCYIlBy5DtC5Y0mgcfHNHsswewDizNvYVreAR6DQDzWh6gQiiMQa3gyHFJTv2bUgjtSX5tv3Un3bZiSAe1TBSi/BvpJ9RW7Bp2uU6TKE4fMfOOkCC7nyjEk56ApwS6gEz4oTQOr6cNXM5kSqz5/jytmA4JDiIEZOaPvTrpCm4OosMBPjcQuUghnpCDhb25mKrevW+P0qKpZ7pDRNQvXk7BpF+aYrRmGn10HgZguGdNCNeSg84RAtY/1TQSMTZL3WsmRM7csVLpGT7mJONX y30/oqRq IlagIwnaKQ6ZprEryi8p4h5GCi/QgMGaqlcxmGSVwP0mSTkjK6VVsIWws3hkPg3ydgsk9FT2f0MPhJaexBxIapASCu3ZgJg9/3YLkK5hLf8kAmmlEB77bk3LZDDMov4zkte7j X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: WORKINGSET_NODES is not exposed in the memcg stats and thus there is no need to use the memcg specific stat update functions for it. In future if we decide to expose WORKINGSET_NODES in the memcg stats, we can revert this patch. Signed-off-by: Shakeel Butt Reviewed-by: Roman Gushchin --- mm/workingset.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mm/workingset.c b/mm/workingset.c index f2a0ecaf708d..c22adb93622a 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -618,6 +618,7 @@ struct list_lru shadow_nodes; void workingset_update_node(struct xa_node *node) { struct address_space *mapping; + struct page *page = virt_to_page(node); /* * Track non-empty nodes that contain only shadow entries; @@ -633,12 +634,12 @@ void workingset_update_node(struct xa_node *node) if (node->count && node->count == node->nr_values) { if (list_empty(&node->private_list)) { list_lru_add_obj(&shadow_nodes, &node->private_list); - __inc_lruvec_kmem_state(node, WORKINGSET_NODES); + __inc_node_page_state(page, WORKINGSET_NODES); } } else { if (!list_empty(&node->private_list)) { list_lru_del_obj(&shadow_nodes, &node->private_list); - __dec_lruvec_kmem_state(node, WORKINGSET_NODES); + __dec_node_page_state(page, WORKINGSET_NODES); } } } @@ -742,7 +743,7 @@ static enum lru_status shadow_lru_isolate(struct list_head *item, } list_lru_isolate(lru, item); - __dec_lruvec_kmem_state(node, WORKINGSET_NODES); + __dec_node_page_state(virt_to_page(node), WORKINGSET_NODES); spin_unlock(lru_lock);