From patchwork Tue Jun 30 00:44:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 11632947 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D08856C1 for ; Tue, 30 Jun 2020 00:44:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8F5ED2078D for ; Tue, 30 Jun 2020 00:44:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="GCzQ3CZw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8F5ED2078D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8A1588D0022; Mon, 29 Jun 2020 20:44:25 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 851868D001D; Mon, 29 Jun 2020 20:44:25 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 740BD8D0022; Mon, 29 Jun 2020 20:44:25 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0187.hostedemail.com [216.40.44.187]) by kanga.kvack.org (Postfix) with ESMTP id 5E5068D001D for ; Mon, 29 Jun 2020 20:44:25 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id DAF6A181AC9C6 for ; Tue, 30 Jun 2020 00:44:24 +0000 (UTC) X-FDA: 76984032048.20.hand90_290ce8e26e73 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id B3664180C0609 for ; Tue, 30 Jun 2020 00:44:24 +0000 (UTC) X-Spam-Summary: 1,0,0,c924c8eef3923ddc,d41d8cd98f00b204,354r6xggkcoqyngqkkrhmuumrk.iusrotad-ssqbgiq.uxm@flex--shakeelb.bounces.google.com,,RULES_HIT:1:2:41:69:152:355:379:541:800:960:966:973:988:989:1260:1277:1313:1314:1345:1437:1516:1518:1593:1594:1605:1730:1747:1777:1792:2196:2199:2393:2559:2562:2890:3138:3139:3140:3141:3142:3152:3308:3865:3866:3867:3868:3870:3871:3874:4042:4052:4321:4385:4605:5007:6119:6261:6653:7903:8957:9121:9592:9969:10004:11026:11232:11233:11473:11658:11914:12043:12296:12297:12438:12555:12895:12986:14394:14659:14877:21063:21080:21433:21444:21450:21451:21611:21627:21660:21987:21990:30001:30054,0,RBL:209.85.216.74:@flex--shakeelb.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100;04yfxxetn8wys9drrfmyjk1cjdi4eocx435p15hxob7nsywpt5y47tmrdzmmym5.w399hwmumyir1ca4ahxgjxuxwbj3b7saokppdupek7yfbfpb1ybm7qhsbzo3nat.4-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neut ral,Cust X-HE-Tag: hand90_290ce8e26e73 X-Filterd-Recvd-Size: 11640 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Tue, 30 Jun 2020 00:44:24 +0000 (UTC) Received: by mail-pj1-f74.google.com with SMTP id a6so4080400pjd.4 for ; Mon, 29 Jun 2020 17:44:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to:cc; bh=0aKSg4enppw3omuJsgtHx/x0BaQCXGbr10nLU30c2nM=; b=GCzQ3CZw8ycKukijr4zhfY+6CQEHGJLBhAHyCWFBPXLE9bnms1tCMPLeqaw6PrLXlE +0TJ215AiVdAQwZ8BFlJpnc8kXzC1Yey7XRZ2hjiOFBC6KizBQl7sMINwLzACCzpVKej /5tRbO19HGO7M+G1mY+QI70BKIkuiGJ0aQHONRR1eTiP/+99BvuwsDa2D8UC81ZiK2i2 zcnWWknB09h6r1T9rXlU+TpAsP0lDbTvipQ7d7xucyPFF3dLXxvIubeE4BG4PlCfg/dz be4Wh2RYXz12rdxW0oXmjxU4SKEKUlOl/uxz3STIkfiLPX2fu7+Akk07zQhJq4YeDqKF MsGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=0aKSg4enppw3omuJsgtHx/x0BaQCXGbr10nLU30c2nM=; b=MdJSn0NoTzssTi+6QmkO7KOsMX+Kc8bgDw8RdTsM4aW+NcTHvE0tnsaDbA+eCGtbAO mZyJX+gaMzvuk32C/OEYsggGUoYJCvq+qXaV9iZn7L6kXRdRbwRERCF71yDjclm+O2xB /NrHKgFU8E3ms0aqx6/NaD21nu7Wi/4+IsGWYlWbLzgdpchN+DvjckQqHe12wTlXwtIO DBMQ6WY9wZM/fB91RMV4JgOUBoQBol+ua48Izh1JnDOrirlaMiPiPGOq0FZUIZpyZoLC AbOjjRxwnCtYxPzwO/FF4rnySsG55hxvehd/cBSlFaxvLIxZSHa3ew08STDEVf80+FFb KC9A== X-Gm-Message-State: AOAM5333cvtk1NMchAW58ah7oFrZ1PCSU7CDDAlMDQqTAEvLrL5AmM4Q SZ3iGxzYmH9r9UeEgP1Mb+qz0JUVmjdWsw== X-Google-Smtp-Source: ABdhPJwR392dp05vt9q5riFi+yk62Bo2Xp3HnK2dHLZXb07dH5n/F2zNwT4qqjPPABu82UpmZQURnOyy2BEcTQ== X-Received: by 2002:a17:90a:a115:: with SMTP id s21mr14264564pjp.232.1593477863146; Mon, 29 Jun 2020 17:44:23 -0700 (PDT) Date: Mon, 29 Jun 2020 17:44:13 -0700 Message-Id: <20200630004413.1423733-1-shakeelb@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.27.0.212.ge8ba1cc988-goog Subject: [PATCH] mm: memcontrol: account kernel stack per node From: Shakeel Butt To: Johannes Weiner , Roman Gushchin , Michal Hocko Cc: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Shakeel Butt X-Rspamd-Queue-Id: B3664180C0609 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently the kernel stack is being accounted per-zone. There is no need to do that. In addition due to being per-zone, memcg has to keep a separate MEMCG_KERNEL_STACK_KB. Make the stat per-node and deprecate MEMCG_KERNEL_STACK_KB as memcg_stat_item is an extension of node_stat_item. Signed-off-by: Shakeel Butt --- drivers/base/node.c | 4 ++-- fs/proc/meminfo.c | 4 ++-- include/linux/memcontrol.h | 2 -- include/linux/mmzone.h | 8 ++++---- kernel/fork.c | 29 ++++++++++------------------- kernel/scs.c | 2 +- mm/memcontrol.c | 2 +- mm/page_alloc.c | 16 ++++++++-------- mm/vmstat.c | 8 ++++---- 9 files changed, 32 insertions(+), 43 deletions(-) diff --git a/drivers/base/node.c b/drivers/base/node.c index 0cf13e31603c..508b80f6329b 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -440,9 +440,9 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_FILE_MAPPED)), nid, K(node_page_state(pgdat, NR_ANON_MAPPED)), nid, K(i.sharedram), - nid, sum_zone_node_page_state(nid, NR_KERNEL_STACK_KB), + nid, node_page_state(pgdat, NR_KERNEL_STACK_KB), #ifdef CONFIG_SHADOW_CALL_STACK - nid, sum_zone_node_page_state(nid, NR_KERNEL_SCS_KB), + nid, node_page_state(pgdat, NR_KERNEL_SCS_KB), #endif nid, K(sum_zone_node_page_state(nid, NR_PAGETABLE)), nid, 0UL, diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index f262bff3ca31..887a5532e449 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -101,10 +101,10 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "SReclaimable: ", sreclaimable); show_val_kb(m, "SUnreclaim: ", sunreclaim); seq_printf(m, "KernelStack: %8lu kB\n", - global_zone_page_state(NR_KERNEL_STACK_KB)); + global_node_page_state(NR_KERNEL_STACK_KB)); #ifdef CONFIG_SHADOW_CALL_STACK seq_printf(m, "ShadowCallStack:%8lu kB\n", - global_zone_page_state(NR_KERNEL_SCS_KB)); + global_node_page_state(NR_KERNEL_SCS_KB)); #endif show_val_kb(m, "PageTables: ", global_zone_page_state(NR_PAGETABLE)); diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index ba1e42715ecf..a3ddb236898e 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -33,8 +33,6 @@ enum memcg_stat_item { MEMCG_SWAP = NR_VM_NODE_STAT_ITEMS, MEMCG_SOCK, MEMCG_PERCPU_B, - /* XXX: why are these zone and not node counters? */ - MEMCG_KERNEL_STACK_KB, MEMCG_NR_STAT, }; diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 8e859444927a..b79f73ce8b57 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -153,10 +153,6 @@ enum zone_stat_item { NR_ZONE_WRITE_PENDING, /* Count of dirty, writeback and unstable pages */ NR_MLOCK, /* mlock()ed pages found and moved off LRU */ NR_PAGETABLE, /* used for pagetables */ - NR_KERNEL_STACK_KB, /* measured in KiB */ -#if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) - NR_KERNEL_SCS_KB, /* measured in KiB */ -#endif /* Second 128 byte cacheline */ NR_BOUNCE, #if IS_ENABLED(CONFIG_ZSMALLOC) @@ -201,6 +197,10 @@ enum node_stat_item { NR_KERNEL_MISC_RECLAIMABLE, /* reclaimable non-slab kernel pages */ NR_FOLL_PIN_ACQUIRED, /* via: pin_user_page(), gup flag: FOLL_PIN */ NR_FOLL_PIN_RELEASED, /* pages returned via unpin_user_page() */ + NR_KERNEL_STACK_KB, /* measured in KiB */ +#if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) + NR_KERNEL_SCS_KB, /* measured in KiB */ +#endif NR_VM_NODE_STAT_ITEMS }; diff --git a/kernel/fork.c b/kernel/fork.c index 73fdfa9674b5..ee5393350ef7 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -278,7 +278,7 @@ static inline void free_thread_stack(struct task_struct *tsk) for (i = 0; i < THREAD_SIZE / PAGE_SIZE; i++) { mod_memcg_page_state(vm->pages[i], - MEMCG_KERNEL_STACK_KB, + NR_KERNEL_STACK_KB, -(int)(PAGE_SIZE / 1024)); memcg_kmem_uncharge_page(vm->pages[i], 0); @@ -381,32 +381,23 @@ static void account_kernel_stack(struct task_struct *tsk, int account) { void *stack = task_stack_page(tsk); struct vm_struct *vm = task_stack_vm_area(tsk); + struct page *page; BUILD_BUG_ON(IS_ENABLED(CONFIG_VMAP_STACK) && PAGE_SIZE % 1024 != 0); if (vm) { - int i; - BUG_ON(vm->nr_pages != THREAD_SIZE / PAGE_SIZE); + page = vm->pages[0]; - for (i = 0; i < THREAD_SIZE / PAGE_SIZE; i++) { - mod_zone_page_state(page_zone(vm->pages[i]), - NR_KERNEL_STACK_KB, - PAGE_SIZE / 1024 * account); - } } else { - /* - * All stack pages are in the same zone and belong to the - * same memcg. - */ - struct page *first_page = virt_to_page(stack); - - mod_zone_page_state(page_zone(first_page), NR_KERNEL_STACK_KB, - THREAD_SIZE / 1024 * account); - - mod_memcg_obj_state(stack, MEMCG_KERNEL_STACK_KB, + page = virt_to_page(stack); + mod_memcg_obj_state(stack, NR_KERNEL_STACK_KB, account * (THREAD_SIZE / 1024)); } + + /* All stack pages are in the same node. */ + mod_node_page_state(page_pgdat(page), NR_KERNEL_STACK_KB, + THREAD_SIZE / 1024 * account); } static int memcg_charge_kernel_stack(struct task_struct *tsk) @@ -431,7 +422,7 @@ static int memcg_charge_kernel_stack(struct task_struct *tsk) return ret; mod_memcg_page_state(vm->pages[i], - MEMCG_KERNEL_STACK_KB, + NR_KERNEL_STACK_KB, PAGE_SIZE / 1024); } } diff --git a/kernel/scs.c b/kernel/scs.c index 5d4d9bbdec36..4ff4a7ba0094 100644 --- a/kernel/scs.c +++ b/kernel/scs.c @@ -17,7 +17,7 @@ static void __scs_account(void *s, int account) { struct page *scs_page = virt_to_page(s); - mod_zone_page_state(page_zone(scs_page), NR_KERNEL_SCS_KB, + mod_node_page_state(page_pgdat(scs_page), NR_KERNEL_SCS_KB, account * (SCS_SIZE / SZ_1K)); } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b1a644224383..06de63901f81 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1485,7 +1485,7 @@ static char *memory_stat_format(struct mem_cgroup *memcg) (u64)memcg_page_state(memcg, NR_FILE_PAGES) * PAGE_SIZE); seq_buf_printf(&s, "kernel_stack %llu\n", - (u64)memcg_page_state(memcg, MEMCG_KERNEL_STACK_KB) * + (u64)memcg_page_state(memcg, NR_KERNEL_STACK_KB) * 1024); seq_buf_printf(&s, "slab %llu\n", (u64)(memcg_page_state(memcg, NR_SLAB_RECLAIMABLE_B) + diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 174c849ba9f2..0568b126f719 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5402,6 +5402,10 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) " anon_thp: %lukB" #endif " writeback_tmp:%lukB" + " kernel_stack:%lukB" +#ifdef CONFIG_SHADOW_CALL_STACK + " shadow_call_stack:%lukB" +#endif " all_unreclaimable? %s" "\n", pgdat->node_id, @@ -5423,6 +5427,10 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(node_page_state(pgdat, NR_ANON_THPS) * HPAGE_PMD_NR), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), + node_page_state(pgdat, NR_KERNEL_STACK_KB), +#ifdef CONFIG_SHADOW_CALL_STACK + node_page_state(pgdat, NR_KERNEL_SCS_KB), +#endif pgdat->kswapd_failures >= MAX_RECLAIM_RETRIES ? "yes" : "no"); } @@ -5454,10 +5462,6 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) " present:%lukB" " managed:%lukB" " mlocked:%lukB" - " kernel_stack:%lukB" -#ifdef CONFIG_SHADOW_CALL_STACK - " shadow_call_stack:%lukB" -#endif " pagetables:%lukB" " bounce:%lukB" " free_pcp:%lukB" @@ -5479,10 +5483,6 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(zone->present_pages), K(zone_managed_pages(zone)), K(zone_page_state(zone, NR_MLOCK)), - zone_page_state(zone, NR_KERNEL_STACK_KB), -#ifdef CONFIG_SHADOW_CALL_STACK - zone_page_state(zone, NR_KERNEL_SCS_KB), -#endif K(zone_page_state(zone, NR_PAGETABLE)), K(zone_page_state(zone, NR_BOUNCE)), K(free_pcp), diff --git a/mm/vmstat.c b/mm/vmstat.c index 2c5a96694490..96bf8bfffd1d 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1158,10 +1158,6 @@ const char * const vmstat_text[] = { "nr_zone_write_pending", "nr_mlock", "nr_page_table_pages", - "nr_kernel_stack", -#if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) - "nr_shadow_call_stack", -#endif "nr_bounce", #if IS_ENABLED(CONFIG_ZSMALLOC) "nr_zspages", @@ -1212,6 +1208,10 @@ const char * const vmstat_text[] = { "nr_kernel_misc_reclaimable", "nr_foll_pin_acquired", "nr_foll_pin_released", + "nr_kernel_stack", +#if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) + "nr_shadow_call_stack", +#endif /* enum writeback_stat_item counters */ "nr_dirty_threshold",