From patchwork Fri Apr 12 07:37:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13627053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4110FC4345F for ; Fri, 12 Apr 2024 07:38:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B2BB76B0088; Fri, 12 Apr 2024 03:38:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AB44F6B0089; Fri, 12 Apr 2024 03:38:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9073E6B008A; Fri, 12 Apr 2024 03:38:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 6C89D6B0088 for ; Fri, 12 Apr 2024 03:38:07 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 059A31C12D2 for ; Fri, 12 Apr 2024 07:38:07 +0000 (UTC) X-FDA: 82000076214.10.62564EC Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf01.hostedemail.com (Postfix) with ESMTP id 515C84000F for ; Fri, 12 Apr 2024 07:38:05 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=IxzzxZiJ; spf=pass (imf01.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712907485; a=rsa-sha256; cv=none; b=UDqFScucNl057oxa50vHRRq9orQpxPlvXLZhUTZSlHAAjXYnmDhMEzJ1p3qUm+xMR+y3q9 CKBkflu94CbXgxWhmgdrt6E6E0Mey022TzpMJJ3euvsp64iYinwBQaXsgW7byCWC8gJYwl dAK2FBFNrI7O35OQZahGbnC+BptCrTk= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=IxzzxZiJ; spf=pass (imf01.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712907485; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=grQ4HG8GdSNxCcCAmJJR/tpqgvcFuaNbvEKv4Is26aY=; b=Ssa7q5fJ3YF6AE65TwmPjDtAK78OgVae9EQsnxlS3agy+h2dD5doKFs161QAq6c1KW+0DJ KKchwxyS7fXmV8IeEk9FY9+wbpX8V3/QZSY+IrgC234QJtttM/Ek7zVSFLy9h2cAB98ysg FauWD3qnaWK2bbRhSTugXZ6ebikqCnM= Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-1e504f58230so5668285ad.2 for ; Fri, 12 Apr 2024 00:38:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712907484; x=1713512284; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=grQ4HG8GdSNxCcCAmJJR/tpqgvcFuaNbvEKv4Is26aY=; b=IxzzxZiJVCjuE/eZ05nw41mrSCfZTMXYXexPQh3U2GM2Z1xfr9v9T3xTL595TzIvwl eXyHpe1Us8GFcYq7fjosWesUp2hajLYfqpfhR7PIX+W/e2fcPcF5hA9GMfjVhq4ZPh8D a2jNlOemVAxU9kDkbXegq81RuBI5ZGCInbZ2QxmP5FMYAJeRDCuZS2dNTxgMib8cXYl/ oa1fAQg28gPpFic06An1BeJVCC9UqCJnhRrDe/Hqv4WsMw44G3nIuxTdfdm/o9MYl6g7 fEps2rWYHhlvsZtoZ0f7HLgDDTVsrqGnkCYJHUdrLrsQVdR1AOoxf1ykncKcHVgUXkMf Xxng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712907484; x=1713512284; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=grQ4HG8GdSNxCcCAmJJR/tpqgvcFuaNbvEKv4Is26aY=; b=j5JEiVw0Mkg5uHjWeOJGWDZ+Qr2raxYOF6G9G+KtcgNPQD3E5ZYNNolI+Zl8G+MGqu hmsRNDwVjJV0Wettq4lChEPo4ieIl2pxVJ/5KEz5MpF/mIvHPr+Eac8vhwW3iDUotfaN 9dH9h1sBLlmXVQLYkTQUTnGHUBm9EioZuhq5iY/IbYgo8ap7KSYC77yNdESR0oYb1PIY XerZ/aJThNEh1PjTHS0eSxma28Ygj35oRCGnsZArZqRAc7ZlUIrkFROrA5rluNPmdMrR SLeL5+xycLE1jvGiBL+WZOR9Migie6ZKqUTZcXqRGvQC53SZ2hN7ovPcD3DWBTRXXG4S gzdA== X-Forwarded-Encrypted: i=1; AJvYcCW6l5LltJmgz9kaWAyRCy+9vJeIZ/5yEjXtSe2kE/Swjm2DgVuUIkon10P/EZLKgU2enQqLsY37LIswDOfg+Rpsi60= X-Gm-Message-State: AOJu0YytQRuM/dbMOVzlsd+sn1TuVmSm9rDZ6Dqho4sWhVN3I4xxCHPf okrMhB+kowEVoa6f9M9QTjHkVDRUpNIbuRj8L0AtiUfTsykrZ/Fj X-Google-Smtp-Source: AGHT+IE+Kawy3AjgOA5jbKXP+/9bdEvPRSrd/EvA5EuSv+JyLrfRn6UvgmK3uhhBkgHSzBuIuxpkcA== X-Received: by 2002:a17:902:ea10:b0:1dd:135e:2604 with SMTP id s16-20020a170902ea1000b001dd135e2604mr1985869plg.40.1712907484158; Fri, 12 Apr 2024 00:38:04 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id n5-20020a170903110500b001e478f359bdsm2344011plh.127.2024.04.12.00.37.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Apr 2024 00:38:03 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: cerasuolodomenico@gmail.com, chrisl@kernel.org, david@redhat.com, kasong@tencent.com, linux-kernel@vger.kernel.org, peterx@redhat.com, ryan.roberts@arm.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, yosryahmed@google.com, yuzhao@google.com, corbet@lwn.net Subject: [PATCH v5 1/4] mm: add per-order mTHP anon_fault_alloc and anon_fault_fallback counters Date: Fri, 12 Apr 2024 19:37:37 +1200 Message-Id: <20240412073740.294272-2-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412073740.294272-1-21cnbao@gmail.com> References: <20240412073740.294272-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 515C84000F X-Stat-Signature: jkgyukerq8d6n5wq77cmp1yxfjt47xdh X-Rspam-User: X-HE-Tag: 1712907485-236222 X-HE-Meta: U2FsdGVkX189Kf6wwmUpHPNcjRzKCyALnWTtBKikZZv9Lgxt4NriGefnV0eIWqZtXHn62JmlNkKHoSF8MYUTZjDaNviaOZOlRI9+32j7VPlaV+VGvv7g7UuCvAXYRaHj17DR1ImRQZR+bzPnVrUnBXxsvg+fV4epBiCJJabdyVDsJxAv3pl8rPKlARYUP8OWzq+T9sCBUkFofmULF8m6HVr552mDRcAKalCMXmr1rKomxNYymgLPDTqdz8DJtnVx5DuB1wG1RYpbNI/H9HOP4/C0km2K8jSZsXnaILB6o5jrIcvjHn1usQs+JeWzHtXeQlvMccMYZCv6pgRdp6ytULFg2zfdPOGC3LcNUHaETYAlAkYmoOkuNJ/KfO5YGfyd8ffBruPUnp/biLRoEdn5ttqEDo84zJF5oO/fbz933wkGpf9k1+QA8Wi5xiPRwcWJQy0M2WjrHmvqJicG1SABakbMylBQ+No0m5Vz7ywa6BvArnWH8+Oc3RZAW6EjGrwSMDLP7i35CX3YF8VedCnKfj2wagcm4HOQsT9k2A0Vb3cSiSsfrNevQJk5DlKL4O6o55TKtoYDmM9C3Tr0Zhj7rIzRUV+aMyKLhKvqNNxwQHxH1bWHhpqZBNdUzluslXzCXd3wELcz1xkF4ByAwjRENnDH5anJQarLZkRK2E8F8+dsN0/eUe7LEI4PR5oq5tM8tHG2mayg9J6YHZ1lQlp7Prq3Pr9MbpmuXfNMSqhtZbQTVZygwLim2bfs8ms+hKK9RUSg8uVNP3LaX2GrcJA3JiiyAhjLJFYH5KOavfvMFUVuNwM2OQEnuJNSdVK0fnET3NuZ3chqDOPQ6ikFZkjKRqf5VQZuDD3Z645p8eSQtEuQoJoZV+khsRYCpWca+1JRq5ZaG6H43+IdGBOYQoYENTlcFWuiYgO9kycoMUvsLNQ2aztKUzatcnNHWe8EZIHHu2Qn16UzdA+GsL5z6lo kW+pgIiG HFlou9Y0uGXNM3VMH/H8Kq7cRfSaULLDDnLu6QPbp6RtoRZQpo3EbuG/g1fYLwntHswoeoRHaCsVvDLfTLsjYgI4KNhfC/jVx4Tptwjtt8a6JQRsjB0v864YrmCxa0y6qNt39HTc8xhqvFBMZkXV3d2cItZWjrxUYZjiy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song Profiling a system blindly with mTHP has become challenging due to the lack of visibility into its operations. Presenting the success rate of mTHP allocations appears to be pressing need. Recently, I've been experiencing significant difficulty debugging performance improvements and regressions without these figures. It's crucial for us to understand the true effectiveness of mTHP in real-world scenarios, especially in systems with fragmented memory. This patch establishes the framework for per-order mTHP counters. It begins by introducing the anon_fault_alloc and anon_fault_fallback counters. Additionally, to maintain consistency with thp_fault_fallback_charge in /proc/vmstat, this patch also tracks anon_fault_fallback_charge when mem_cgroup_charge fails for mTHP. Incorporating additional counters should now be straightforward as well. Signed-off-by: Barry Song Cc: Chris Li Cc: David Hildenbrand Cc: Domenico Cerasuolo Cc: Kairui Song Cc: Matthew Wilcox (Oracle) Cc: Peter Xu Cc: Ryan Roberts Cc: Suren Baghdasaryan Cc: Yosry Ahmed Cc: Yu Zhao --- include/linux/huge_mm.h | 51 ++++++++++++++++++++++++++++++++++ mm/huge_memory.c | 61 +++++++++++++++++++++++++++++++++++++++++ mm/memory.c | 3 ++ mm/page_alloc.c | 4 +++ 4 files changed, 119 insertions(+) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index e896ca4760f6..c5beb54b97cb 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -264,6 +264,57 @@ unsigned long thp_vma_allowable_orders(struct vm_area_struct *vma, enforce_sysfs, orders); } +enum mthp_stat_item { + MTHP_STAT_ANON_FAULT_ALLOC, + MTHP_STAT_ANON_FAULT_FALLBACK, + MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE, + __MTHP_STAT_COUNT +}; + +struct mthp_stat { + unsigned long stats[0][__MTHP_STAT_COUNT]; +}; + +extern struct mthp_stat __percpu *mthp_stats; + +static inline void count_mthp_stat(int order, enum mthp_stat_item item) +{ + if (order <= 0 || order > PMD_ORDER || !mthp_stats) + return; + + this_cpu_inc(mthp_stats->stats[order][item]); +} + +static inline void count_mthp_stats(int order, enum mthp_stat_item item, long delta) +{ + if (order <= 0 || order > PMD_ORDER || !mthp_stats) + return; + + this_cpu_add(mthp_stats->stats[order][item], delta); +} + +/* + * Fold the foreign cpu mthp stats into our own. + * + * This is adding to the stats on one processor + * but keeps the global counts constant. + */ +static inline void mthp_stats_fold_cpu(int cpu) +{ + struct mthp_stat *fold_stat; + int i, j; + + if (!mthp_stats) + return; + fold_stat = per_cpu_ptr(mthp_stats, cpu); + for (i = 1; i <= PMD_ORDER; i++) { + for (j = 0; j < __MTHP_STAT_COUNT; j++) { + count_mthp_stats(i, j, fold_stat->stats[i][j]); + fold_stat->stats[i][j] = 0; + } + } +} + #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ (1<stats[order][item]; + } + cpus_read_unlock(); + + return sum; +} + +#define DEFINE_MTHP_STAT_ATTR(_name, _index) \ +static ssize_t _name##_show(struct kobject *kobj, \ + struct kobj_attribute *attr, char *buf) \ +{ \ + int order = to_thpsize(kobj)->order; \ + \ + return sysfs_emit(buf, "%lu\n", sum_mthp_stat(order, _index)); \ +} \ +static struct kobj_attribute _name##_attr = __ATTR_RO(_name) + +DEFINE_MTHP_STAT_ATTR(anon_fault_alloc, MTHP_STAT_ANON_FAULT_ALLOC); +DEFINE_MTHP_STAT_ATTR(anon_fault_fallback, MTHP_STAT_ANON_FAULT_FALLBACK); +DEFINE_MTHP_STAT_ATTR(anon_fault_fallback_charge, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); + +static struct attribute *stats_attrs[] = { + &anon_fault_alloc_attr.attr, + &anon_fault_fallback_attr.attr, + &anon_fault_fallback_charge_attr.attr, + NULL, +}; + +static struct attribute_group stats_attr_group = { + .name = "stats", + .attrs = stats_attrs, +}; + static struct thpsize *thpsize_create(int order, struct kobject *parent) { unsigned long size = (PAGE_SIZE << order) / SZ_1K; @@ -549,6 +593,12 @@ static struct thpsize *thpsize_create(int order, struct kobject *parent) return ERR_PTR(ret); } + ret = sysfs_create_group(&thpsize->kobj, &stats_attr_group); + if (ret) { + kobject_put(&thpsize->kobj); + return ERR_PTR(ret); + } + thpsize->order = order; return thpsize; } @@ -691,6 +741,11 @@ static int __init hugepage_init(void) */ MAYBE_BUILD_BUG_ON(HPAGE_PMD_ORDER < 2); + mthp_stats = __alloc_percpu((PMD_ORDER + 1) * sizeof(mthp_stats->stats[0]), + sizeof(unsigned long)); + if (!mthp_stats) + return -ENOMEM; + err = hugepage_init_sysfs(&hugepage_kobj); if (err) goto err_sysfs; @@ -725,6 +780,8 @@ static int __init hugepage_init(void) err_slab: hugepage_exit_sysfs(hugepage_kobj); err_sysfs: + free_percpu(mthp_stats); + mthp_stats = NULL; return err; } subsys_initcall(hugepage_init); @@ -880,6 +937,8 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, folio_put(folio); count_vm_event(THP_FAULT_FALLBACK); count_vm_event(THP_FAULT_FALLBACK_CHARGE); + count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_FALLBACK); + count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); return VM_FAULT_FALLBACK; } folio_throttle_swaprate(folio, gfp); @@ -929,6 +988,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, mm_inc_nr_ptes(vma->vm_mm); spin_unlock(vmf->ptl); count_vm_event(THP_FAULT_ALLOC); + count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_ALLOC); count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC); } @@ -1050,6 +1110,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) folio = vma_alloc_folio(gfp, HPAGE_PMD_ORDER, vma, haddr, true); if (unlikely(!folio)) { count_vm_event(THP_FAULT_FALLBACK); + count_mthp_stat(HPAGE_PMD_ORDER, MTHP_STAT_ANON_FAULT_FALLBACK); return VM_FAULT_FALLBACK; } return __do_huge_pmd_anonymous_page(vmf, &folio->page, gfp); diff --git a/mm/memory.c b/mm/memory.c index 649a547fe8e3..06048af7cf9a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4368,6 +4368,7 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf) folio = vma_alloc_folio(gfp, order, vma, addr, true); if (folio) { if (mem_cgroup_charge(folio, vma->vm_mm, gfp)) { + count_mthp_stat(order, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); folio_put(folio); goto next; } @@ -4376,6 +4377,7 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf) return folio; } next: + count_mthp_stat(order, MTHP_STAT_ANON_FAULT_FALLBACK); order = next_order(&orders, order); } @@ -4485,6 +4487,7 @@ static vm_fault_t do_anonymous_page(struct vm_fault *vmf) folio_ref_add(folio, nr_pages - 1); add_mm_counter(vma->vm_mm, MM_ANONPAGES, nr_pages); + count_mthp_stat(folio_order(folio), MTHP_STAT_ANON_FAULT_ALLOC); folio_add_new_anon_rmap(folio, vma, addr); folio_add_lru_vma(folio, vma); setpte: diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b51becf03d1e..3135b5ca2457 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5840,6 +5840,10 @@ static int page_alloc_cpu_dead(unsigned int cpu) */ vm_events_fold_cpu(cpu); +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + mthp_stats_fold_cpu(cpu); +#endif + /* * Zero the differential counters of the dead processor * so that the vm statistics are consistent. From patchwork Fri Apr 12 07:37:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13627054 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D93A7C4345F for ; Fri, 12 Apr 2024 07:38:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E9B06B008A; Fri, 12 Apr 2024 03:38:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4734E6B008C; Fri, 12 Apr 2024 03:38:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 313FC6B0092; Fri, 12 Apr 2024 03:38:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 0F7456B008A for ; Fri, 12 Apr 2024 03:38:13 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id B5BAD40DA9 for ; Fri, 12 Apr 2024 07:38:12 +0000 (UTC) X-FDA: 82000076424.16.0F5D372 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by imf26.hostedemail.com (Postfix) with ESMTP id E5DD414000F for ; Fri, 12 Apr 2024 07:38:10 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BOEpGMX0; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712907491; a=rsa-sha256; cv=none; b=fp9oyMc0kuD9/h/kqCJkjGVlpskNeMs1aE6cmzbZ1to5Qw/K/mVuyU5qrbUQlUO5H/S+Su vtwhI7kZpHdg7KP8ZbORvEe134TY0VhbnE7q96p/axYgHXidynwO7blwulxCAiaHiX9sfD ymx/558KnEZJbx8nV/hz55cQQoo5tlI= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BOEpGMX0; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf26.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712907491; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hsOHXEK8Nh5pewumsjjrvdM3HKJLRrjO6yqH5K5tp/s=; b=BjwKwjEES3tPMzyG1L2/JK3KA9iAUNshavXWwhpqVIR1DzsmB1hiWwmqRF3Q4kP9Put6PQ oXfNtcWEGvX7lKCDfymChXjS4/b4znmG6oKI7hGohICrfWCYfrB55HN92IzjCDIedGH7XE ypliF9sgPFDRNr/PUYLa/+AaKe1CZCg= Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-1e40042c13eso4839885ad.2 for ; Fri, 12 Apr 2024 00:38:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712907490; x=1713512290; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hsOHXEK8Nh5pewumsjjrvdM3HKJLRrjO6yqH5K5tp/s=; b=BOEpGMX0TlJhnbYI2dOKSSrYRKqsLhMFjcgH+HpLDDk7uQPBm3VDPHyM/4Wd05t5fZ S5OVAtTcM3Qj53VfpPz8JTr0hkonMCotreIGzpJ8gQiU49y/CJoH17dtRbcqD6cSSZsP cu6Aga96yfwS78lu6zIJ9df9v1n+ljYE3fmtn3lTZ0HfkzGCBfeNjcCXw7C3+T+2MqrM 8HQR4OYBMV/+HL1eOljogXpx0sh9HtUHRhcKd3sGJPxtVHaK5r6Y1oiqKWnbVPA8efUO bSNUcQ0fDNvbTAxkGAYrj2YjA8I6CHiIkhWgKgMyNdhGaeWkY6xMhtKHvvBCMVD9+xG7 yF8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712907490; x=1713512290; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hsOHXEK8Nh5pewumsjjrvdM3HKJLRrjO6yqH5K5tp/s=; b=jaWO18BUm8BRUsPqSlfomDwWB5dguk3dJnex5ChphyM1Aus/z7ONqjCkYGJkkDVaVe tTje0dAppBAtJPMiEwKAo/vViabr84sz+SnRWbeftCX31YXMQ82VIOm5c3qHpmGT5gud 71Jfu7uZr1y8hF08BpLNWpEiFxS1iHqdnPAc4LA9hDjPxkkpUDpKJsejak+Ne7HcLDSu eIEQFwmGB0xKXQ5FyfJZ/0ljofeI9ojiQ5XZIgURg5iqcOvWRp1jSL+/GIPbTIcbDGpQ EFbrdhrp/d9wmPRF/esTkE8iPl+CUpPac/HCIFRl4C/P7BHfj3N4IA9ThIiWmNrEJK5D PPpg== X-Forwarded-Encrypted: i=1; AJvYcCVgbrKz8V6zvrrwzgNbT47Hvu6LGhG04rXM4lBRXoZoFUES4FTSMamvf6IMmIb9J+8R9WA96P9PmQNYZR4v8PAZ0tM= X-Gm-Message-State: AOJu0YxiSCCSade4bXY2iMmRe88mcFuivjdgqvGytd2Tp1TjMenG5lXD bk20WBGtUcMJUOxP3t1nkTT66q36jSyMK5NEGW2d1379QFQdS4gcr599o0su X-Google-Smtp-Source: AGHT+IHWZYew+Q9NtnWF+a6SE88ptdUjsAwDnwkkWmi3sA6yX2D3E8zqruIKdheWkebuC2R11yEKEg== X-Received: by 2002:a17:902:9341:b0:1e2:952b:81fc with SMTP id g1-20020a170902934100b001e2952b81fcmr1436795plp.29.1712907489843; Fri, 12 Apr 2024 00:38:09 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id n5-20020a170903110500b001e478f359bdsm2344011plh.127.2024.04.12.00.38.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Apr 2024 00:38:09 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: cerasuolodomenico@gmail.com, chrisl@kernel.org, david@redhat.com, kasong@tencent.com, linux-kernel@vger.kernel.org, peterx@redhat.com, ryan.roberts@arm.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, yosryahmed@google.com, yuzhao@google.com, corbet@lwn.net Subject: [PATCH v5 2/4] mm: add per-order mTHP anon_swpout and anon_swpout_fallback counters Date: Fri, 12 Apr 2024 19:37:38 +1200 Message-Id: <20240412073740.294272-3-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412073740.294272-1-21cnbao@gmail.com> References: <20240412073740.294272-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: E5DD414000F X-Stat-Signature: 4a153mdd8qwqcbfkubt5qnamg14rbycg X-HE-Tag: 1712907490-841216 X-HE-Meta: U2FsdGVkX1/RNIOuoooyDh0SlmMfoOCVbkLqhyqRKtST38WGKaHFX3Aq/lZArhe/ywMk46OAxUNiflenkX+U5vY0AUnkt2RwhXHRDV/wnMhrtA/zpqRtt4nAvw/eQIK7U4UHJThtyli2p7Bai+nThadnQsNhuezOge0EBHLIGAkG8YXaXw6pP2VoejJpHUEFOneosjFxjS/ySK400AjhT6Jjx1BrkqpcOoCf1PaQn+lIHazPx1UTEYIwaF2PO7QEQVWpDHno0JoV7mbClkAEIqvuF+AuyoOvqGx4xfNCWkFoUzDv33btJch0PyfSGZy5xFgbvKINrnHXUiqJhva4Sifvnbn8uptgftaRfx7rODLwaxJsrG12HUeZBnU45IWl+XJY6fM3xOS9+uk3iY95g2HMWqsQTFHfkDv7big3R1iLA781WM8hMgtGYlNaVwZmRpVQlaBiNI4n1zoMpfGCDmKguL/IaGiXBbJqgp9JEatMPDbXUfPvcsgxpMzHTvRneYr2PduyjN59QVnQJcJNwR9N46oRv6UVMdRRvv9dCQfOlNUg2ixW4lGcx+A+tF8asK9YlsLiHoS+3Hq1qKHOhpQ0kjOYE8FL0kwkYr+KLUf3leKhBiv6M3aaVY9iFmHMBlPvFuT1rG+mrRaxWGSLNHmyBNpcTWRjNdaYZQlZ+OvMcR+iSEqK6gJ1XM4i9cudug6qJSDnFJM/TIQAb9iT7d1O5Vd4u1asfu0VfsQFKFixs9femNQFAbgbjtJY04yOVEt1M75fVFQDa+QHmhF/lQNMKAjyvPXthLSTqbL31HE7qsFSejLZzJp+Vi4SchMTu0cAN48l+w9DDQNsD0J+JRNEsjTzRwPf7G7CcOfu0R+JUeazLUSaSPCaTuDlsIaV+VJnUFOSKgEnAL+NlHP/4xASdeGgfHg3tC1J2U3l+t1S44oDbvRxYRTpv9s02XfLpuBZeSJ3N19TaPlaYAr 72G141go XV5yOXQ3D/08NiPY3v60Ll9PTKOgXlvh1lVLPLeo+6g2B222c0RoK4nQtFJ2xNlWSZ5lLA+WgIEeZEbyTNn3SmnCUkLesL77UYOfDkLVTHQ0XU+b0DMDFftwDQJ28JJF2zIbizZCKAlyN2iszAJ1znqp5NYhFP7y20PoZZo9hP/rVv5rMnyeyNVWkTJZbSzUP/pa1EsVwhcTYL+MFQdMvQHTIVWyTQTbbBhcS+LifqShSNuUYtsLWFTzbHNq0SS4O9OGZHBOoWhJ0uKXd85Hdnd/oKqiKuPGzm8Mh/nicboVvM7ifFRfwL++JSiqsu3/7adnYrhCR5I2qF2OJj62QgD7H3C1FiA2CtjVf+5X602Rp5mKSNM6SOoMoXMxziPumn4CwAUNoyYcX4et+KGVhlpY6Azzxaiz43afIjuHCuE8ShX0vzmnEyJ/5ZYjsb65wjCqd/jw07sk5aX60u7hZmaIiWWK5c3MUuNwpciasi8miH1t2/5ETWs5KwgCNLXPvk31JufJ0iYJznGkMzWztNbyaa4t0/KgDkhqcc5tsGuEN6112bBUL8MT512ZBUOWXMERY52E9DqtZpcaJjuMqwIKNzrwYbH/0ux/AC1ZV/a/4ML34aCvjaAiFiCOXK/xNyrZ0hBSE/A01W/PnFwah8wS1PaA+O+o4/3tBgztbcUxE6LiZqiMh8/KEF8dcqbfMdmsykj7Ze/cvrlA8adf45ydVU+hkLLm+kK2uB8GGSN52iQLmNsl5HaomqIptYRLOpkmIDHFkpi2Qsd1PhnczgG5nSzpcpeg+nNbPS4t8+RUXYAZBFEcwPv95vMNuPgpLvMmGfHcHa82qftqOj6uOT5G/2A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song This helps to display the fragmentation situation of the swapfile, knowing the proportion of how much we haven't split large folios. So far, we only support non-split swapout for anon memory, with the possibility of expanding to shmem in the future. So, we add the "anon" prefix to the counter names. Signed-off-by: Barry Song Cc: Chris Li Cc: David Hildenbrand Cc: Domenico Cerasuolo Cc: Kairui Song Cc: Matthew Wilcox (Oracle) Cc: Peter Xu Cc: Ryan Roberts Cc: Suren Baghdasaryan Cc: Yosry Ahmed Cc: Yu Zhao Reviewed-by: Ryan Roberts --- include/linux/huge_mm.h | 2 ++ mm/huge_memory.c | 4 ++++ mm/page_io.c | 1 + mm/vmscan.c | 3 +++ 4 files changed, 10 insertions(+) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index c5beb54b97cb..b69c3b3e1436 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -268,6 +268,8 @@ enum mthp_stat_item { MTHP_STAT_ANON_FAULT_ALLOC, MTHP_STAT_ANON_FAULT_FALLBACK, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE, + MTHP_STAT_ANON_SWPOUT, + MTHP_STAT_ANON_SWPOUT_FALLBACK, __MTHP_STAT_COUNT }; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 21c4ac74b484..13e74724d0c3 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -557,11 +557,15 @@ static struct kobj_attribute _name##_attr = __ATTR_RO(_name) DEFINE_MTHP_STAT_ATTR(anon_fault_alloc, MTHP_STAT_ANON_FAULT_ALLOC); DEFINE_MTHP_STAT_ATTR(anon_fault_fallback, MTHP_STAT_ANON_FAULT_FALLBACK); DEFINE_MTHP_STAT_ATTR(anon_fault_fallback_charge, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); +DEFINE_MTHP_STAT_ATTR(anon_swpout, MTHP_STAT_ANON_SWPOUT); +DEFINE_MTHP_STAT_ATTR(anon_swpout_fallback, MTHP_STAT_ANON_SWPOUT_FALLBACK); static struct attribute *stats_attrs[] = { &anon_fault_alloc_attr.attr, &anon_fault_fallback_attr.attr, &anon_fault_fallback_charge_attr.attr, + &anon_swpout_attr.attr, + &anon_swpout_fallback_attr.attr, NULL, }; diff --git a/mm/page_io.c b/mm/page_io.c index a9a7c236aecc..46c603dddf04 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -217,6 +217,7 @@ static inline void count_swpout_vm_event(struct folio *folio) count_memcg_folio_events(folio, THP_SWPOUT, 1); count_vm_event(THP_SWPOUT); } + count_mthp_stat(folio_order(folio), MTHP_STAT_ANON_SWPOUT); #endif count_vm_events(PSWPOUT, folio_nr_pages(folio)); } diff --git a/mm/vmscan.c b/mm/vmscan.c index bca2d9981c95..49bd94423961 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1231,6 +1231,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, goto activate_locked; } if (!add_to_swap(folio)) { + int __maybe_unused order = folio_order(folio); + if (!folio_test_large(folio)) goto activate_locked_split; /* Fallback to swap normal pages */ @@ -1242,6 +1244,7 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, THP_SWPOUT_FALLBACK, 1); count_vm_event(THP_SWPOUT_FALLBACK); } + count_mthp_stat(order, MTHP_STAT_ANON_SWPOUT_FALLBACK); #endif if (!add_to_swap(folio)) goto activate_locked_split; From patchwork Fri Apr 12 07:37:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13627055 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A661CC4345F for ; Fri, 12 Apr 2024 07:38:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 33C9E6B0092; Fri, 12 Apr 2024 03:38:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C59A6B0093; Fri, 12 Apr 2024 03:38:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0F2296B0095; Fri, 12 Apr 2024 03:38:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DF4F06B0092 for ; Fri, 12 Apr 2024 03:38:18 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7291880DEE for ; Fri, 12 Apr 2024 07:38:18 +0000 (UTC) X-FDA: 82000076676.02.F1BEB31 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf18.hostedemail.com (Postfix) with ESMTP id A07751C000A for ; Fri, 12 Apr 2024 07:38:16 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=mtHZlrQA; spf=pass (imf18.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712907496; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HKv+dOKRCJhG1tenSkeU7/8yjqgzALDAmJSETtZIvnY=; b=XBvTFuZ1dzql/jS03lzpRUxQ7W2utrcQ8CMN6QVPBvxoRGGlVdfsco9yTbLfNwue9nnuaS N+c6tnlaEN/dsW4dlIe14lQY0GwBeMhUVa35xTxnlqvOcn5MKsn/WOEbkhdv7py90/3a1/ wvmGPuQIapNP6BrZ1Anni0IPLJ/z3Qs= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=mtHZlrQA; spf=pass (imf18.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712907496; a=rsa-sha256; cv=none; b=o2X02UDXaX2Lz4BGDCLMh1wCPasid3KO6BGFqevPVCEefM3yIldeVkj0pWGJV5vvRa3TEC HRLda6ssrPIAbpplT7RSyN0UPeWqvdLRl9GkOqJYja4ldDZz93knM9SyEB8dMPmDyCtcQQ ESgYM8XCCh8qmIt4VE0ZsEw1gPIPaQk= Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1e4266673bbso5830625ad.2 for ; Fri, 12 Apr 2024 00:38:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712907495; x=1713512295; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HKv+dOKRCJhG1tenSkeU7/8yjqgzALDAmJSETtZIvnY=; b=mtHZlrQA/AARalhIGpi81MvEZ4Nzrqj4He5OXUTo7Kg8ujtkPnq3BlwbfZRdrRN8ns wVScXXZd7z8UNSdvdfEFCOf+emZxwSkskMCE9Ojm1dgqlaY0EEAs+7FDnwASyCNZs8HG 9PdUxir2tHruxJgGWgYkzTVWlFsZJ/jZC0W7Ck06H2OE1XdOEdon2ngAWfgPzbKOcisb YHRTpSqupVic4rw4ZGYnL0xF8nq8poPHg1aITdiiytZqMuXqyTep60pOzYbfWbO6nw31 eE4mWrXDWW3ezioGDEnndNnvWiQCnm1x1QBVA5l4n6EXGEh7Tv+P0a/I5vxynD2FlFNE YkuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712907495; x=1713512295; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HKv+dOKRCJhG1tenSkeU7/8yjqgzALDAmJSETtZIvnY=; b=TOkkre0LVctMlRhIvcf/cD1gF/ERGfVX1emXYW76Opf8TGJu1eqxIY6Y5Y8nede6m5 eAQVFzVb5vP0RRuiQSxMR3iBjC0yZ80lxM8uASSJQ9iE11OVdpDaxDY+hmfsOBIVnt0M LNlSJ8QkoqyGsshd+YluFS9lU4m3C+IeuRsikzU9Vz9tRhSBzrn26L5s9IgZ8O/aqr6l iCRF+p8vbb7giHE9+cfQMDAoEMmJ8Lmuzef0zBE0t85suKjIIQhZwZV/WpYcupkqchpn edykpYPHd8ifNuYsH/1Kb05AcXZAZFNJt2dBV8MF3Br1boSygwVf8tnQEPT7oskpYbWr K6fw== X-Forwarded-Encrypted: i=1; AJvYcCVUUwQF8aLrJecQapb4ZIph1c+2nqNxoRs5pKiat2i7iZnagfaso5vpXJd75y4d7NS9f2Wmc1OV8EIBhG6MmAyKhE0= X-Gm-Message-State: AOJu0YxLDWT1YrRRHMYlPwot08f2dxEuWFa4rBBFwCTX+ymNwgPI7i9I i/aarlNN0HbsH8VjcGyQDY2v/gFMLmYI1DfuEwBQ0bbajvEtsAPe X-Google-Smtp-Source: AGHT+IEs8mtcjto8rOXz5YMEcwZFJOMI1fiWwDFq30NXF3pGJzIMZvBmCkp2JLPcrzhMVIiefWw4TQ== X-Received: by 2002:a17:902:e950:b0:1e3:e0a7:3c9 with SMTP id b16-20020a170902e95000b001e3e0a703c9mr2171282pll.14.1712907495496; Fri, 12 Apr 2024 00:38:15 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id n5-20020a170903110500b001e478f359bdsm2344011plh.127.2024.04.12.00.38.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Apr 2024 00:38:15 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: cerasuolodomenico@gmail.com, chrisl@kernel.org, david@redhat.com, kasong@tencent.com, linux-kernel@vger.kernel.org, peterx@redhat.com, ryan.roberts@arm.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, yosryahmed@google.com, yuzhao@google.com, corbet@lwn.net Subject: [PATCH v5 3/4] mm: add docs for per-order mTHP counters and transhuge_page ABI Date: Fri, 12 Apr 2024 19:37:39 +1200 Message-Id: <20240412073740.294272-4-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412073740.294272-1-21cnbao@gmail.com> References: <20240412073740.294272-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: A07751C000A X-Rspam-User: X-Stat-Signature: ttkpt3tgwq7bt19ajn1ymh57uezy9zej X-Rspamd-Server: rspam01 X-HE-Tag: 1712907496-775743 X-HE-Meta: U2FsdGVkX1/dpEBgZGlr/kIU+XbejiHy0d9Uv/3SBi+2g11MstRv/1RKqQFOtA5+XgPVRnNYWieYAysvcgSOQaNZYWjIsl9KYhCLKvFFpnTs2Qss7hoXoxx5aAItfB2vzGBfBHLe+pXiNXVfd8F091b75/Rw8dNNaBfongL27J//QS84tiQxMb7zA5kiQOaYUjqWvEx4XcorfSPSKRpRL0LG88UPx3YepSOcjhPDbetrSb9sPOOuogfWsd3mv1U+1M+GTUki96bj1e/2cSqzM3saADcgDS9BYMgepru3yRLrTsMsWEfpaDhZhwQfe+AxeeE0T5hBKv9sVyc0hKmdx65q42R5GrQqzLkvGKRk3HRNmyregVvhPJn/WT0aIZ1WtTQh0FdzwuJryu0Wjv4o7Csw2f1dPU/aZBTQwo9gESAU2E6aqA/UvldDx84imsGbF1DiARxJGb7jgytxWoEc4ifiL/3wU/izmcvaoDdqRb0QhvSvLb/vT+hZI8221Q6sUvoBiRR25++topsj/FgzeslS4QFvu6UoO82YwiUlPg+k5VLdMmKy6e2y2jLmxb/U8yPSlAMBxNRmiub9n5hd7fcG3DyXBELjnpRc5FH2P866jFJxcG5043Z6yXJ+L6YSh0wSUPQpDI8mx4AYn8euLbLFBrZRsg8J/t7WVcfnwSEElGfBWpo/iStpCCbfs52FeF2sOJE0ubLBZXmqlUfeRwY1+49ZZjB+MgBjThzw8jxI3Fx0h2exEJy7QPx59sst46Wr1OCH7TGeLDcmXdv4KjfyTpUl5Sbypc1Hvy6H3ifpLvzmXMvALWPyqmp6IJL0HZMqtN/yGzy8tbilfA8UrMfwTtwCQUJbgXLveVMLWderXup+w7kSDmjAHcu7eVwLPlE8GRCZ/984IKUG8n1wtGJf5mJW5XRZlvu/H27ZD8SlBBh0ZK7mYnp3nY8PndjZ8tJKepvG5R7k/55BiAm dNre0wxW hmj4hfBINi+kxQSbbXixO8AxKSZtWmrREma4mMFWsTf6qgriBF6c7iXsUlWrElFXQu9z6afiNfz0G+PLGUteo8pGKhgfOVFIk06Dq5C3xxAZBC/zQwiHo+hJMJ07PEl57J9uLyCYgefUYS0rSyGN577DfM4sYBaZLRtJynfTM9pwPDZzTYFRapI3fFQBG/pH54sUPlsQOjMGbDC8/WXwpQL6TG0FGXgM2zxH8qnfjtNPYhH/EXFRNfyUk2/E+JysdybJ/ezYVvM1VF+m4lcwBQXj+TC3Qvi3Pbq4KREKGKvBPwpPpVZThDKa2TbPm4SyWR1OYCAUgEuQt5qkTrvxBpn3sOrTl8ajkvBO5U0Hvu2UGTcoTKCuVLJ7mSfcCSrvwQGuI8xxVKc9bjIN+bqKQsexMVMSi5UxJUB5iRhoiynK/mzwMxRJQcrq2SLfYaADI7z7T6SCex7BZEhiQ9K+zWCvuXVBql+FUMxw10yzDe9mN0VgWbuZNGtceu1Q7cpjlykJ48hQlcRlDlN6C1YG5prfc69q/SnHkC+ATm3WkQQWNP+RxzVRdcGoApySfA5mYk9ATAHXwmVuWLdJstE864xf59Tit9RUvdQS3Tf/00TlZ56iq4tAbO56g3K5x/sPNkhJY18BlFSgtpvYK1DpTQFnywfUDb46IBIHO4JzmNFRAPH+LToN9woCfQTf2+vvhExTf+cD7Sg8WYdoAMZpfH/WF+Ynk4hZubabZ1BLNzPx1tuvt5AWoERy2Y4nzImfi1IMdDbKYjyjXtJ3+JONu6lqUJN8Qzbh06NUMzYnQYhldhS11Xxh1hkCLWwTlCTb0tvBkXcY/oBapW5swCSUaSsZAPjh6E5fNz1HEgAzIEz0lbS4zzHPBw6UM+7tFU4zODAMLm4NeAyv8OcB1QzjeFkjkDLDz8GarKAcQkHB1UeVwwV4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song This patch includes documentation for mTHP counters and an ABI file for sys-kernel-mm-transparent-hugepage, which appears to have been missing for some time. Signed-off-by: Barry Song Cc: Chris Li Cc: David Hildenbrand Cc: Domenico Cerasuolo Cc: Kairui Song Cc: Matthew Wilcox (Oracle) Cc: Peter Xu Cc: Ryan Roberts Cc: Suren Baghdasaryan Cc: Yosry Ahmed Cc: Yu Zhao Cc: Jonathan Corbet Reviewed-by: Ryan Roberts --- .../sys-kernel-mm-transparent-hugepage | 17 +++++++++++ Documentation/admin-guide/mm/transhuge.rst | 28 +++++++++++++++++++ 2 files changed, 45 insertions(+) create mode 100644 Documentation/ABI/testing/sys-kernel-mm-transparent-hugepage diff --git a/Documentation/ABI/testing/sys-kernel-mm-transparent-hugepage b/Documentation/ABI/testing/sys-kernel-mm-transparent-hugepage new file mode 100644 index 000000000000..80dde0fd576c --- /dev/null +++ b/Documentation/ABI/testing/sys-kernel-mm-transparent-hugepage @@ -0,0 +1,17 @@ +What: /sys/kernel/mm/hugepages/ +Date: April 2024 +Contact: Barry Song +Description: + /sys/kernel/mm/transparent_hugepage/ contains a number of files and + subdirectories, + - defrag + - enabled + - hpage_pmd_size + - khugepaged + - shmem_enabled + - use_zero_page + - subdirectories of the form hugepages-kB, where + is the page size of the hugepages supported by the kernel/CPU + combination. + + See Documentation/admin-guide/mm/transhuge.rst for details. diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index 04eb45a2f940..f436ff982f22 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -447,6 +447,34 @@ thp_swpout_fallback Usually because failed to allocate some continuous swap space for the huge page. +In /sys/kernel/mm/transparent_hugepage/hugepages-kB/stats, There are +also individual counters for each huge page size, which can be utilized to +monitor the system's effectiveness in providing huge pages for usage. Each +counter has its own corresponding file. + +anon_fault_alloc + is incremented every time a huge page is successfully + allocated and charged to handle a page fault. + +anon_fault_fallback + is incremented if a page fault fails to allocate or charge + a huge page and instead falls back to using huge pages with + lower orders or small pages. + +anon_fault_fallback_charge + is incremented if a page fault fails to charge a huge page and + instead falls back to using huge pages with lower orders or + small pages even though the allocation was successful. + +anon_swpout + is incremented every time a huge page is swapout in one + piece without splitting. + +anon_swpout_fallback + is incremented if a huge page has to be split before swapout. + Usually because failed to allocate some continuous swap space + for the huge page. + As the system ages, allocating huge pages may be expensive as the system uses memory compaction to copy data around memory to free a huge page for use. There are some counters in ``/proc/vmstat`` to help From patchwork Fri Apr 12 07:37:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13627056 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C413C4345F for ; Fri, 12 Apr 2024 07:38:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8E2556B0095; Fri, 12 Apr 2024 03:38:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 86BD46B0096; Fri, 12 Apr 2024 03:38:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6E53F6B0098; Fri, 12 Apr 2024 03:38:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4C0E86B0095 for ; Fri, 12 Apr 2024 03:38:24 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 1A620160DB4 for ; Fri, 12 Apr 2024 07:38:24 +0000 (UTC) X-FDA: 82000076928.08.C1C5B28 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf11.hostedemail.com (Postfix) with ESMTP id 3B7FA40004 for ; Fri, 12 Apr 2024 07:38:22 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="bYfWyum/"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712907502; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vLivAlSFWVO5qmFvXnJVwwxKOFwucVM2YU4CcxrDCQY=; b=rTIylY0R6e90Tgg3lDtfQEFHSZAeAwNkrD/taWHS5D07H5DzShr+GiJ+zwXUgWiGOOIP+J W8dcf1ca+2qqmNt6kfzf7LmtAxLnAWjB31leSP8jRzVKo5pzkHQsxRblts+v99LqUaRGpb 5k0R8PpJ79bWObQ9M487fpROgg6fChk= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="bYfWyum/"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712907502; a=rsa-sha256; cv=none; b=xwB5xn9SJnXx5vW8h3g77Nnc3LhVuJ92FpKjc8HPZpkdovk0vdpYLuzqINZ6SlcH4gS2Yi AfQ2eeEC4UeDxH40tJI2HQMnJmxf5afWS9jhCTQg8YDqD+rRCbBg/SrrqCE+17FFyRvctN 9CHSheO1y0f1ZM+vph2sc94ISOACXg0= Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1e3f17c64daso5151195ad.3 for ; Fri, 12 Apr 2024 00:38:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712907501; x=1713512301; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vLivAlSFWVO5qmFvXnJVwwxKOFwucVM2YU4CcxrDCQY=; b=bYfWyum/VLCMIqrmuZ17Wg8NBQM3KabiZSNfFonqE5m80HQBfqkcIOq0DcBLtbe1bs I11ibJ0SH+dC40DLmRK97pbTqbmAVz5HusMh+Udl0NpmJS0eC8F96fTgK1Il107i9/IM XqC4Bv+dOTeHL0/hWhnVpJhiiiv11JoGFKsMYaG5k89TzI5anipjRSychFtqrdDR+Hig vlWrjctcfKEmRGtpDJjiq6fTIsf+XYp3csG6TVd7JNiRRpF1j5PNXp7L2HHB0jCSueVP gw0o5GMoUG/Sn8NdOrfsQw6wh6BzMFfNQuHtjxDMK8qjUaR4hCR9wK1mCXtLrdQ7HulB MksA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712907501; x=1713512301; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vLivAlSFWVO5qmFvXnJVwwxKOFwucVM2YU4CcxrDCQY=; b=UbdpBp0RWYceSrUZch1w2Bk6VYLdIzUoTFwpB4iC7jPjeUhIAx7GLw1A8Kz+OqSZD9 zM8x069KBIl3s3o/YfVeAL+jbNtuX4z/fzRlE9uJf+7wMjIQ/YQxeIufpQH6Jb9knxkf VCV0vnnQV+Si0lacIvsmaXP1WCrXgFkVhc+YnQpa8HjYfaWddt921w58oNBbXHM2qb0U RSinEhoGa0r1nFkEe6kiJXw8yuenTa24UujW/z6rQ6n8O/yj7NKdu8OUy1BFexWXp2Sp xk6jIjv/HLEG5LQD9vxI7i454nk0zlDiFL49pWODS4tDS0rkDXxpxL5eoCBDn5VVDO07 ljDw== X-Forwarded-Encrypted: i=1; AJvYcCVmB6IMHbBkVtm3/GNWp1smPaiwgUNxKPHO411UdFu0YVYZEc5UG+CjaXsK50RDbTSJY3QhAJfpwXWnEWWCEIPJa8w= X-Gm-Message-State: AOJu0YwhIbBE5DDVz3mBIegyTxpYEGwJhEbSjNPAy6vdagfP7xZhznyN +4vaRAUbrcJ0m7HM3Db43/TkXW5XFqCiUcc0ln7fwUGm2qrvOvuA X-Google-Smtp-Source: AGHT+IHgBpc7UAQvJAcCBWbdfiIookBgasRuDbzAV/jyU0fOVDPDt6/r0/rQy9bL8YhPtuipQpw50g== X-Received: by 2002:a17:902:ec8e:b0:1e2:9ac2:4a03 with SMTP id x14-20020a170902ec8e00b001e29ac24a03mr1874586plg.28.1712907501110; Fri, 12 Apr 2024 00:38:21 -0700 (PDT) Received: from localhost.localdomain ([2407:7000:8942:5500:aaa1:59ff:fe57:eb97]) by smtp.gmail.com with ESMTPSA id n5-20020a170903110500b001e478f359bdsm2344011plh.127.2024.04.12.00.38.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Apr 2024 00:38:20 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: cerasuolodomenico@gmail.com, chrisl@kernel.org, david@redhat.com, kasong@tencent.com, linux-kernel@vger.kernel.org, peterx@redhat.com, ryan.roberts@arm.com, surenb@google.com, v-songbaohua@oppo.com, willy@infradead.org, yosryahmed@google.com, yuzhao@google.com, corbet@lwn.net Subject: [PATCH v5 4/4] mm: correct the docs for thp_fault_alloc and thp_fault_fallback Date: Fri, 12 Apr 2024 19:37:40 +1200 Message-Id: <20240412073740.294272-5-21cnbao@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240412073740.294272-1-21cnbao@gmail.com> References: <20240412073740.294272-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 3B7FA40004 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 7z4p6a7fp6c8pgf1y9pcg748uj8wjgp6 X-HE-Tag: 1712907502-767892 X-HE-Meta: U2FsdGVkX1/JCU9vpjbNE7f9Ci2a+BWL1iR/MJ1lGIuAu2/T1U6vxSi8VHcy3lDSjY5sB9TtkGTNWTd+9h2N4ifQgy3o6rPjlqhdxqmMbZiUdgzWo85UHlOQKn4AbBVbxexGeAsAUwx6SIbcywYSGj4LdOLwz30g4rF/MXbMq5lMvlWguFbYjP5ROCCvUYRjED2Xvcoz7s4/toUfXAeWhAkvtumUsunDMoNnStFfn+bKzfG/pGuYtvkijY5eZTZlq6FHOlX5Ilk4h1wYp7qUtsJeVe8OwJ3VcALQG1R0SEzkWF2tscsiLL4Jl/ql79lQ3VVPyvZTAktQpsq6INTfUl2u+9lW+pNlX0BUltUR1+2Tgr4XxKMIE8d1Yc9v2KEnrvWaeEjD1MJ6BSvJ8gCFUO3wyx9e8JyU5JWGdLHNfoVPRrSLLebWqRys04b01TQMH7dzAiWf/V6MDMaivcd2nz6FqU1pwEEaJcTpYJUDm6dh7O6hvXZNnTcGCB4buKD3frzuYB/OMu7c2Ts+bTE27KjbzTaGhUVdlVSvD32uGzANm1W4L30WlJj6prh5MImyEl/ZIBmaQTcHy8ykce28KgrAF8kQFopzTs2RKwKDP6Mc0khuVdQkE0eFjK2MrsZ3E2KJ1Tde4/zbmIBNtc95EYVVEM1ILDHDmpnUMGGi7hOBfmV0c2KnMzXBgRkkn8V29RYEUIqlTpvlGiiVzVwJOVDLx6fYKHcIs0/2rNNySTASLjjjvaQtjY30p5aUCxuM9Elw2uzT2Ou/419x1MdLybh28b0tSZ0Qrqq+Emr0W/u5xrVZl8X/4faB7S1AqmvgEdDUw8bAiBlfS37k2XOZiITCUfrQzOtBsX/Tt5NjLm1VbJPbhGtUSoDTUci9+dIjYzu4gwsyHH3p1z6/HTuIh2KMuuqQkXFoTd1Jr5FX9dOEsPvv/ZIPpPFMUl1mPpJRuGWgQNrtSrjRmlMnPam b/6H8oBP hKtVrUpGVh6o5S4TjbBFln3T6BnQ+UjHT3HjyYCyRW0xKKkbyNKie5pkGX/w6V1OEYUil55nlg/T2fgDtbqIMu9ED30eA/tDUj8aCz/txpH07tmPTO0mvFbtFApRoavcZ1tlUcoZnC3QRGtDarbWc8LVrb6ZEbrMUR5cm/RB+7GxYFk2VX3CeEiGNzkNSELXTPts3YGtVbo1ACt820N1Zdywha7yBZIDJV/YFS+NVss8z422hkgGYJ5Mh/H8QC0IbPVze4ItapiCXjSxhuNsc2QDz6u6SENbXnEOBduBKuV3el0XrnvZgxMUI4WZcLLp/Al2DmFxjUlCstlaQcoqX00VMAakQEYuJaBUTGPy7s4SP7DLeYzWAtEsNyGs5UwKcks/VdXUvZPj5v4pIDtRjAt6pKQHoE8uLWyJgnFISQDobr0vkhxYBZRDXTpGErsR9XQSsAgHBr3lcWyBWBzYxQJcsGzznMp02C1VbiTGLvhg1e+c9XOSTkyX0vBBYqwUs9WkOFD2MLRSbZVK8xhi/OHObC4lY+ffVv2vrE2QSTh5KrXmjbqRKo1YEDQuOodFPAfXQzTr40BceijhVMKM5cL693MuPq3MGhH00sBPjXNTsDv7zOuuR6Ifgv1jFBX2J99znLV2p/su1UlSjwtzUICuBaj+FpmPNs4u4AnzE5alobhpguzs9qCxvzc066gBKWnadN3iOQ0yY3sL70QnOoGRh+tDvWyoMGF0CG9hDoFii90dc6yqqDEfE+tKkHFMZ6n7fyZLVpiUmvXh4XC0vftuXQbdxwAwemLeGX/qNkaWLDcYCe/tloj1XQg4aCSLB0sQJjvQT/LqgIfPORwMXsiAfGRFbpzcXA8JffOb3Q24n9d8QDl0vb+9D2Xle+ENuNDJBjShcpmorR5NBiEFyO5w8hg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song The documentation does not align with the code. In __do_huge_pmd_anonymous_page(), THP_FAULT_FALLBACK is incremented when mem_cgroup_charge() fails, despite the allocation succeeding, whereas THP_FAULT_ALLOC is only incremented after a successful charge. Signed-off-by: Barry Song Cc: Chris Li Cc: David Hildenbrand Cc: Domenico Cerasuolo Cc: Kairui Song Cc: Matthew Wilcox (Oracle) Cc: Peter Xu Cc: Ryan Roberts Cc: Suren Baghdasaryan Cc: Yosry Ahmed Cc: Yu Zhao Cc: Jonathan Corbet Reviewed-by: Ryan Roberts --- Documentation/admin-guide/mm/transhuge.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index f436ff982f22..98e3a99ea780 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -369,7 +369,7 @@ monitor how successfully the system is providing huge pages for use. thp_fault_alloc is incremented every time a huge page is successfully - allocated to handle a page fault. + allocated and charged to handle a page fault. thp_collapse_alloc is incremented by khugepaged when it has found @@ -377,7 +377,7 @@ thp_collapse_alloc successfully allocated a new huge page to store the data. thp_fault_fallback - is incremented if a page fault fails to allocate + is incremented if a page fault fails to allocate or charge a huge page and instead falls back to using small pages. thp_fault_fallback_charge