From patchwork Mon Apr 15 08:12:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13629609 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32F04C00A94 for ; Mon, 15 Apr 2024 08:12:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EFB5A6B0092; Mon, 15 Apr 2024 04:12:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E5C326B0093; Mon, 15 Apr 2024 04:12:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D4C616B0095; Mon, 15 Apr 2024 04:12:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id AF76C6B0092 for ; Mon, 15 Apr 2024 04:12:50 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 7806C1A04C0 for ; Mon, 15 Apr 2024 08:12:50 +0000 (UTC) X-FDA: 82011050100.14.7A99749 Received: from szxga05-in.huawei.com (szxga05-in.huawei.com [45.249.212.191]) by imf30.hostedemail.com (Postfix) with ESMTP id A274880012 for ; Mon, 15 Apr 2024 08:12:47 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf30.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713168768; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pb0vQZr6BCZfEtJKRV34ebWNy0uWYuBoFM9f7l27QP8=; b=wTfzGdFu22ikRxHL6BWQC1QZv4kI8KCKj2qJs8vLaus1cGv+abBAHKXlKy+/C2dKjFbiBB +2hiMLEUmc0bLrcrmzPsR/qEvz3F+MOM4ptxQ1AzmBTz+3ZAKAqE/D9R0jqASIkU06AdvU khzEh5b5FF7tIlQsOCKVnwVC8qucT4E= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf30.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713168768; a=rsa-sha256; cv=none; b=ae7+gSU9R1ECtDxdCatf+yUcykYQto+EyrC6+I4T8wvxOqikdqvmZMpmp09IrnPeZHxveL wvAE1e9b+rs0wCkyGygUrb17ib5KagnnbAU2MtKIvDEl/MF4RWyZwWGsZN7ZfiwOrh8wep pvRA6uz8o9xWQw30g1lrfNjD6ALhXD4= Received: from mail.maildlp.com (unknown [172.19.163.44]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4VJ0Fm0c6sz1hwST; Mon, 15 Apr 2024 16:09:48 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id D1FF414011F; Mon, 15 Apr 2024 16:12:43 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Mon, 15 Apr 2024 16:12:43 +0800 From: Kefeng Wang To: Andrew Morton CC: Huang Ying , Mel Gorman , Ryan Roberts , David Hildenbrand , Barry Song , Vlastimil Babka , Zi Yan , "Matthew Wilcox (Oracle)" , Jonathan Corbet , Yang Shi , Yu Zhao , , Kefeng Wang Subject: [PATCH rfc 3/3] mm: pcp: show per-order pages count Date: Mon, 15 Apr 2024 16:12:20 +0800 Message-ID: <20240415081220.3246839-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240415081220.3246839-1-wangkefeng.wang@huawei.com> References: <20240415081220.3246839-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: A274880012 X-Stat-Signature: bttunhxpnpxcgmj1e1qj3zzsgr5ynriy X-HE-Tag: 1713168767-527320 X-HE-Meta: U2FsdGVkX1/ywEI2i3Cn6wG0AfBaIADMJkBmw9PL8u5JygiPqxW+YoLNBiU+Fymwt/f11QFCDNmK9Rl9axogeeOEQAwyMnDL7wArHJjpnivx4byvopr63kEXANC7mG1Mv5s1aW9hPR+pC/j/wpk+LBPCYNQ1CihNheNlYsFXnj3WBH3ThfgFV9tXP2sPoi9Xv5hux9H9W8d3UinqmOVDOcioD6QWVi1ahuN38xysBjbtkMeKf8WSqaIc8cdSScYmTk4lESRwX5qbhL7IUHM0B1ziL5dg6sPXkuNSdJDkCrXoX4/SuuAo11tlx+9oi/eFhtOR01EHryfLQdZc/Di3AOm7oc/7aPmLvf1DKt/R4Yoz9QLqKJpdgd+8UPyC85u+lOxPt1/QZ2YjskHa1JF9f8qoOP0yO22BGGlM5GDluD/kLBgo8d2wCVbPPDYJymN5dwA5T112Rr7m9K5EzF86Dfo9vpOE4zALykBIITVBjbVtLlJJ3xwAaV2RpF8A+7d1JcFyjSs1hQQ0wL0EdjtC2o2ujHelBSopCR8BP1DeYOS45m8ezvLFA3FgWdspxJW/SKGy+AAgyaSeqIAXjKnaKJNZOdJR7w1089QHSYFnxX+GxSmcE/V1Saej7oOF43Nzd5JIUm842qey6Y4nZUM8sfvIOTRj+vSwBSJwef8Jentu7n9b9TTzvtiNkqOzhIBSVqfkV8WR1jSXx393A6mtAw1HQBOfk2NRfE0IVw0XpTZGGAW1AujDwuqALCSfnRsG2Rdw8Vn5B5NjN4l6+iCWyR09vnPQAib7WExB9zMoQueUtrzSXyf+ti9+LNJadLHZ5lreUo+/vOGfGf9Ryy8PTkCcHja5tCQbV2RytAo7oq5QYtTy++znKbf8nBd2JoS1ugmTzadgbYh2+O54z37xy6cSBnnSyFdSZ9Cao3HueuxW/64BfRmFfkWoSHwICaWjT94g2tXQKvJ45YAm/Ox WiH4BwI5 i2UyGNTzZDL3wvqn+oaxI0M40pQ24myhJZ9PtQ8qyi2zsAeFj74odNEL1egLL4cCdYst2TNmJEc4qXOSlwZApdrbp4XDmJQlyd5fbVQOd8vQhXex0FJCgptEeN+kMyLsn5IGO/JJWDNRvUeEeFgkiSb3VcQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: THIS IS ONLY FOR DEBUG. Show more detail about per-order page count on each cpu in zoneinfo, and a new pcp_order_stat shows the total counts of each hugepage size in sysfs. #cat /proc/zoneinfo .... cpu: 15 count: 275 high: 529 batch: 63 order0: 59 order1: 28 order2: 28 order3: 6 order4: 0 order5: 0 order6: 0 order7: 0 order8: 0 order9: 0 #cat /sys/kernel/mm/transparent_hugepage/hugepages-64kB/stats/pcp_order_stat 10 Signed-off-by: Kefeng Wang --- include/linux/mmzone.h | 6 ++++++ include/linux/vmstat.h | 19 +++++++++++++++++++ mm/Kconfig.debug | 8 ++++++++ mm/huge_memory.c | 27 +++++++++++++++++++++++++++ mm/page_alloc.c | 4 ++++ mm/vmstat.c | 16 ++++++++++++++++ 6 files changed, 80 insertions(+) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index c745e2f1a0f2..c32c01468a77 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -665,6 +665,9 @@ enum zone_watermarks { #define NR_LOWORDER_PCP_LISTS (MIGRATE_PCPTYPES * (PAGE_ALLOC_COSTLY_ORDER + 1)) #define HIGHORDER_PCP_LIST_INDEX (NR_LOWORDER_PCP_LISTS - (PAGE_ALLOC_COSTLY_ORDER + 1)) #define NR_PCP_LISTS (NR_LOWORDER_PCP_LISTS + NR_PCP_THP) +#ifdef CONFIG_PCP_ORDER_STATS +#define NR_PCP_ORDER (PAGE_ALLOC_COSTLY_ORDER + NR_PCP_THP + 1) +#endif #define min_wmark_pages(z) (z->_watermark[WMARK_MIN] + z->watermark_boost) #define low_wmark_pages(z) (z->_watermark[WMARK_LOW] + z->watermark_boost) @@ -702,6 +705,9 @@ struct per_cpu_pages { /* Lists of pages, one per migrate type stored on the pcp-lists */ struct list_head lists[NR_PCP_LISTS]; +#ifdef CONFIG_PCP_ORDER_STATS + int per_order_count[NR_PCP_ORDER]; /* per-order page counts */ +#endif } ____cacheline_aligned_in_smp; struct per_cpu_zonestat { diff --git a/include/linux/vmstat.h b/include/linux/vmstat.h index 735eae6e272c..91843f2d327f 100644 --- a/include/linux/vmstat.h +++ b/include/linux/vmstat.h @@ -624,4 +624,23 @@ static inline void lruvec_stat_sub_folio(struct folio *folio, { lruvec_stat_mod_folio(folio, idx, -folio_nr_pages(folio)); } + +static inline void pcp_order_stat_mod(struct per_cpu_pages *pcp, int order, + int val) +{ +#ifdef CONFIG_PCP_ORDER_STATS + pcp->per_order_count[order] += val; +#endif +} + +static inline void pcp_order_stat_inc(struct per_cpu_pages *pcp, int order) +{ + pcp_order_stat_mod(pcp, order, 1); +} + +static inline void pcp_order_stat_dec(struct per_cpu_pages *pcp, int order) +{ + pcp_order_stat_mod(pcp, order, -1); +} + #endif /* _LINUX_VMSTAT_H */ diff --git a/mm/Kconfig.debug b/mm/Kconfig.debug index afc72fde0f03..57eef0ce809b 100644 --- a/mm/Kconfig.debug +++ b/mm/Kconfig.debug @@ -276,3 +276,11 @@ config PER_VMA_LOCK_STATS overhead in the page fault path. If in doubt, say N. + +config PCP_ORDER_STATS + bool "Statistics for per-order of PCP (Per-CPU pageset)" + help + Say Y to show per-order statistics of Per-CPU pageset from zoneinfo + and pcp_order_stat in sysfs. + + If in doubt, say N. diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 9b8a8aa36526..0c6262bb8fe4 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -599,12 +599,39 @@ DEFINE_MTHP_STAT_ATTR(anon_swpout, MTHP_STAT_ANON_SWPOUT); DEFINE_MTHP_STAT_ATTR(anon_swpout_fallback, MTHP_STAT_ANON_SWPOUT_FALLBACK); DEFINE_MTHP_STAT_ATTR(anon_swpin_refault, MTHP_STAT_ANON_SWPIN_REFAULT); +#ifdef CONFIG_PCP_ORDER_STATS +static ssize_t pcp_order_stat_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + int order = to_thpsize(kobj)->order; + unsigned int counts = 0; + struct zone *zone; + + for_each_populated_zone(zone) { + struct per_cpu_pages *pcp; + int i; + + for_each_online_cpu(i) { + pcp = per_cpu_ptr(zone->per_cpu_pageset, i); + counts += pcp->per_order_count[order]; + } + } + + return sysfs_emit(buf, "%u\n", counts); +} + +static struct kobj_attribute pcp_order_stat_attr = __ATTR_RO(pcp_order_stat); +#endif + static struct attribute *stats_attrs[] = { &anon_alloc_attr.attr, &anon_alloc_fallback_attr.attr, &anon_swpout_attr.attr, &anon_swpout_fallback_attr.attr, &anon_swpin_refault_attr.attr, +#ifdef CONFIG_PCP_ORDER_STATS + &pcp_order_stat_attr.attr, +#endif NULL, }; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 25fd3fe30cb0..f44cdf8dec50 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1185,6 +1185,7 @@ static void free_pcppages_bulk(struct zone *zone, int count, list_del(&page->pcp_list); count -= nr_pages; pcp->count -= nr_pages; + pcp_order_stat_dec(pcp, order); __free_one_page(page, pfn, zone, order, mt, FPI_NONE); trace_mm_page_pcpu_drain(page, order, mt); @@ -2560,6 +2561,7 @@ static void free_unref_page_commit(struct zone *zone, struct per_cpu_pages *pcp, pindex = order_to_pindex(migratetype, order); list_add(&page->pcp_list, &pcp->lists[pindex]); pcp->count += 1 << order; + pcp_order_stat_inc(pcp, order); batch = READ_ONCE(pcp->batch); /* @@ -2957,6 +2959,7 @@ struct page *__rmqueue_pcplist(struct zone *zone, unsigned int order, migratetype, alloc_flags); pcp->count += alloced << order; + pcp_order_stat_mod(pcp, order, alloced); if (unlikely(list_empty(list))) return NULL; } @@ -2964,6 +2967,7 @@ struct page *__rmqueue_pcplist(struct zone *zone, unsigned int order, page = list_first_entry(list, struct page, pcp_list); list_del(&page->pcp_list); pcp->count -= 1 << order; + pcp_order_stat_dec(pcp, order); } while (check_new_pages(page, order)); return page; diff --git a/mm/vmstat.c b/mm/vmstat.c index db79935e4a54..632bb1ed6a53 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1674,6 +1674,19 @@ static bool is_zone_first_populated(pg_data_t *pgdat, struct zone *zone) return false; } +static void zoneinfo_show_pcp_order_stat(struct seq_file *m, + struct per_cpu_pages *pcp) +{ +#ifdef CONFIG_PCP_ORDER_STATS + int j; + + for (j = 0; j < NR_PCP_ORDER; j++) + seq_printf(m, + "\n order%d: %i", + j, pcp->per_order_count[j]); +#endif +} + static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, struct zone *zone) { @@ -1748,6 +1761,9 @@ static void zoneinfo_show_print(struct seq_file *m, pg_data_t *pgdat, pcp->count, pcp->high, pcp->batch); + + zoneinfo_show_pcp_order_stat(m, pcp); + #ifdef CONFIG_SMP pzstats = per_cpu_ptr(zone->per_cpu_zonestats, i); seq_printf(m, "\n vm stats threshold: %d",