From patchwork Sat Aug 24 01:04:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13776156 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73820C5321E for ; Sat, 24 Aug 2024 01:05:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 09855800DA; Fri, 23 Aug 2024 21:05:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 04881800D4; Fri, 23 Aug 2024 21:05:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DB672800DA; Fri, 23 Aug 2024 21:05:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id BBF67800D4 for ; Fri, 23 Aug 2024 21:05:12 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 3782AA02F2 for ; Sat, 24 Aug 2024 01:05:12 +0000 (UTC) X-FDA: 82485345264.27.DFEED7B Received: from mail-ot1-f44.google.com (mail-ot1-f44.google.com [209.85.210.44]) by imf25.hostedemail.com (Postfix) with ESMTP id 461C5A001D for ; Sat, 24 Aug 2024 01:05:10 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=eLZpFOda; spf=pass (imf25.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.44 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724461445; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=L/o1HIzXNMob30fXF5zmCRt2n3hoqmCYtF+NPffM/eQ=; b=i+L60TsGwhSaW3wg1IIlUXneVebcU2SVtOzuBqnJ6oZDlLZ1HsSMKDzuVUjbGZJYPvjxV/ +bYdJwRJ+epe+n7DCt0GWsOt7UJ1E/IhfrA/TIxpRC2ZDqcb3RJKRoHczurWxDsJBHHIKp 2DBMSpKumz0Ni2jofBghY21OqG9+5cA= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=eLZpFOda; spf=pass (imf25.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.44 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724461445; a=rsa-sha256; cv=none; b=2RPCm4e7n4V9268OTGQwrfQlLNkavSIDigAp/RGIbSpkajPAL+C0381obdfWISIeKK3mbn AxLaq+cJkpueag2Hf6/c3FIcAUQwZECq72ezYoGNu22dt8bxvxMzzNPHe3bZdfxsY4O38B +lQAOj5LRRRZpvq+LS5AMt+LWyuql2o= Received: by mail-ot1-f44.google.com with SMTP id 46e09a7af769-70945a007f0so2107695a34.2 for ; Fri, 23 Aug 2024 18:05:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724461509; x=1725066309; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=L/o1HIzXNMob30fXF5zmCRt2n3hoqmCYtF+NPffM/eQ=; b=eLZpFOda6ihPWXVNRJ4KRQYXYvAqOJBBVNn2ZK0Q/8RxXAemYd3uTlA+FZFdKzO5bg 6XE5BszlnA+09bkkciX5MbMuJMdwco+ip9r2qzcIj6EHRtxR50aHuLX8wMwfLRaGZWA0 eoYGpzP8zkgsS2p32zlqgr8e71Zh4u3/ztk3ARfbbcza8CiJn22T2caEkDMGkSGmgPv+ XogEDV7SbA4WTK105LCDg/YHqoPE9r7b/oqeXl3ESNQBsBPmuC0zUG/LGeVEYdWhYRcz ozR+ZK4vYxcZIYchCkKL/0as2CJHf+T+p4CdUfxt7DwMTCShVGafB1vibCqYbx4rUSvf fr9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724461509; x=1725066309; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=L/o1HIzXNMob30fXF5zmCRt2n3hoqmCYtF+NPffM/eQ=; b=AIBEBdLKSk5BLUL5PVljC0NoWKWPlMQSeUmqUKeQ/m9CxKLMiV1Vd/Cg2YEVvUqjnq 2caGFKT4NNoUNWo5kC0Z76afgfqnHV7lY684Owsn4P6Q0ARapuvHieVNL1n58jD3OvPZ T+kVTZv4qNI1msSyWxckXRyGa7DXWROHAvRhmczoW4ndFd9Z8WYeJ/kPjLCa2qp2lqQh f0AXrTIIX6BR3Zc2qCssNiyCnWh/Izl+rv4fSe7WMNp2Mka5p9C4SafNlxb2NsB2rOiQ zNSseauQIsQSA8Nf64Yyl2zvyeYgId2m6STjtCFshaLwXlYFm5m2Sp94mVz4FxficdVP eL/Q== X-Forwarded-Encrypted: i=1; AJvYcCUtQJJ8h8DiXoXiCsr/tN5XzH1IvXNq5Tz1H9QYDxZgXgsVfOu+wBA3EEwk3y8OVdDs74bZRIfmDA==@kvack.org X-Gm-Message-State: AOJu0YxwGrdqi4tlOzvX99nXcI8hs0KVP4DfxPTgEjJSBEWutCcdX9Sq ZL2lfgLRCySvq5SIoMjl+BLxl2nUJ/c7P6WZD1mHxZAs5MhWtkMh X-Google-Smtp-Source: AGHT+IHSjqtgJC33yaq8apeRF8rlt8GA3MppGd4QfCOxrURvtmHnsAtQ/+hq9+EVB/CoETVjurGJLg== X-Received: by 2002:a05:6808:18a2:b0:3d9:303a:fc6d with SMTP id 5614622812f47-3de2a8d41a7mr5142441b6e.41.1724461509124; Fri, 23 Aug 2024 18:05:09 -0700 (PDT) Received: from Barrys-MBP.hub ([2407:7000:8942:5500:8d8:dd4b:c921:b282]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ad56c9fsm3274064a12.64.2024.08.23.18.05.03 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 23 Aug 2024 18:05:08 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hanchuanhua@oppo.com, ioworker0@gmail.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, ryan.roberts@arm.com, usamaarif642@gmail.com, v-songbaohua@oppo.com, yuanshuai@oppo.com, ziy@nvidia.com Subject: [PATCH v4 1/2] mm: count the number of anonymous THPs per size Date: Sat, 24 Aug 2024 13:04:40 +1200 Message-Id: <20240824010441.21308-2-21cnbao@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240824010441.21308-1-21cnbao@gmail.com> References: <20240824010441.21308-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Stat-Signature: wdruhz6rq7xg69os3wbzqf5dhfyhqxbq X-Rspam-User: X-Rspamd-Queue-Id: 461C5A001D X-Rspamd-Server: rspam02 X-HE-Tag: 1724461510-720022 X-HE-Meta: U2FsdGVkX1+sSssLcRC3hlXtKvVwIDCZ+y9xdgTqpG6IHqXo+LA/Py4o5xHZqthTH8/Azmv9IBe6HehZQavS6HDZnr4VgUm9Jpmtx6vQfrD9Zr9DXowM9SRgHt8jWDeNhOBW9AfZqvD7VJfBmNMw2GSFh6+1PdHU837UZX+nJm2N+mJQgBy5/qzlheSkdeG4+lOT/CxphcilxWj2PtDfT4AoaTQWDTqUyE/50fU7LW9cl8zuHT5IRvIo81HNHCBnY/1wpoZaLRJvtZwquP31itbT2WZAB8GCVwUCdCkE+HJlY2IrZ3S8epM37FhRfBFWGzE2W53PyXoUGX+obHLL0gZwEblfeqcs6dEW+87Rb3iQiQjCfCiQn7sQLnARF4y8Hc0imiaXRF4dpXahTKlCK4GJn6vdDqc5gsWv8mv0pRXISb2dQajfUL+Z+41sqiuJZwkQq7TCTyTT/GTCnA2cLfr5iIo6V54KBIAp/JNdTCawMDPR8ww8s+mulFVyGQgCvDMCEeeAHVSQCJ+HPLR5O7SXJrqkP0QGmtEtHEUcjeyDvPTm4youyaJ1JiGygkqH9yBhhdcN2bQmSS/M3yNlY6ooo76LPBKGyNNbN6CEQrNYPMXXSKxwFMoPsn0paVZHrw9bFEiXyCozM1d9m0ZvGZiceP+Uix6HXb1lPdlGLDHNtGkldGoNCyjadW7OlFVm8jvrkdy46nbJ/XoREsD9VimOP5FOwzyICDbLWHUyJsKGAfh613TG1Sl06Azk7ihNXrggwKMEcZk2IDorYBsPI2rzlh76hLhPvqutJsBA2nFhsos7EdmQZVsFX67UI0KULIANfULC8wWJw/X52f9GZ5bFrHrnZiw9WGwtFuCX2ihq/JoNZestNg0o+VDbS40f6rHyzGc0FgymdQvUTHyYeQMXlccFHW4/DySBnYghE+MsD+UPFUZEc6JvQu83aSE9JjHUaLbeevsVEbAkVX5 ViYD/yo1 /87V0EEFDvbL4s0MKbmthu/mmE6GfoJVGpRkPkywUS1nEpsU5mTUjfwAv32KzR/ZBjCn2zC1zkcBHXsnjYPF2nnb1Ad58poLl1ZFPsfS2kcCQM8EOwKGg0ejM895Eb06S/JhLbwbjQADtdayoMEHsGd5/E636mE6fE56KEKMCWF/MXfffvBiAHOnB8iEio96MWCGDK1IDpF6ajOY1ev8PqI1plfDpIzXHV422U8akM/rLji4unIlXi0pyPqAt/2me32v2xGt4RtEfBVtXgpLDI0KpAOobtVvigHDrF/K4yQSqpP1QN/ZobBkAiUnAt0uNY/3bnjCiK29esALPOnasUHlpYzjn2XkZqi79y9JJJFHznFIANegJFSFsUf4ILb3UDq0T9DcyVNVFcvTH5F1qJ8VTYNIFZ4U8bX9EP4cs8kCH1oe5H2R9skmqazvHj8kNP09k/3VHJpPMDeM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song Let's track for each anonymous THP size, how many of them are currently allocated. We'll track the complete lifespan of an anon THP, starting when it becomes an anon THP ("large anon folio") (->mapping gets set), until it gets freed (->mapping gets cleared). Introduce a new "nr_anon" counter per THP size and adjust the corresponding counter in the following cases: * We allocate a new THP and call folio_add_new_anon_rmap() to map it the first time and turn it into an anon THP. * We split an anon THP into multiple smaller ones. * We migrate an anon THP, when we prepare the destination. * We free an anon THP back to the buddy. Note that AnonPages in /proc/meminfo currently tracks the total number of *mapped* anonymous *pages*, and therefore has slightly different semantics. In the future, we might also want to track "nr_anon_mapped" for each THP size, which might be helpful when comparing it to the number of allocated anon THPs (long-term pinning, stuck in swapcache, memory leaks, ...). Further note that for now, we only track anon THPs after they got their ->mapping set, for example via folio_add_new_anon_rmap(). If we would allocate some in the swapcache, they will only show up in the statistics for now after they have been mapped to user space the first time, where we call folio_add_new_anon_rmap(). Signed-off-by: Barry Song Acked-by: David Hildenbrand --- Documentation/admin-guide/mm/transhuge.rst | 5 +++++ include/linux/huge_mm.h | 15 +++++++++++++-- mm/huge_memory.c | 13 ++++++++++--- mm/migrate.c | 4 ++++ mm/page_alloc.c | 5 ++++- mm/rmap.c | 1 + 6 files changed, 37 insertions(+), 6 deletions(-) diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index 79435c537e21..b78f2148b242 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -551,6 +551,11 @@ split_deferred it would free up some memory. Pages on split queue are going to be split under memory pressure, if splitting is possible. +nr_anon + the number of transparent anon huge pages we have in the whole system. + These huge pages could be entirely mapped or have partially + unmapped/unused subpages. + As the system ages, allocating huge pages may be expensive as the system uses memory compaction to copy data around memory to free a huge page for use. There are some counters in ``/proc/vmstat`` to help diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 4c32058cacfe..2ee2971e4e10 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -126,6 +126,7 @@ enum mthp_stat_item { MTHP_STAT_SPLIT, MTHP_STAT_SPLIT_FAILED, MTHP_STAT_SPLIT_DEFERRED, + MTHP_STAT_NR_ANON, __MTHP_STAT_COUNT }; @@ -136,14 +137,24 @@ struct mthp_stat { DECLARE_PER_CPU(struct mthp_stat, mthp_stats); -static inline void count_mthp_stat(int order, enum mthp_stat_item item) +static inline void mod_mthp_stat(int order, enum mthp_stat_item item, int delta) { if (order <= 0 || order > PMD_ORDER) return; - this_cpu_inc(mthp_stats.stats[order][item]); + this_cpu_add(mthp_stats.stats[order][item], delta); +} + +static inline void count_mthp_stat(int order, enum mthp_stat_item item) +{ + mod_mthp_stat(order, item, 1); } + #else +static inline void mod_mthp_stat(int order, enum mthp_stat_item item, int delta) +{ +} + static inline void count_mthp_stat(int order, enum mthp_stat_item item) { } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 513e7c87efee..26ad75fcda62 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -597,6 +597,7 @@ DEFINE_MTHP_STAT_ATTR(shmem_fallback_charge, MTHP_STAT_SHMEM_FALLBACK_CHARGE); DEFINE_MTHP_STAT_ATTR(split, MTHP_STAT_SPLIT); DEFINE_MTHP_STAT_ATTR(split_failed, MTHP_STAT_SPLIT_FAILED); DEFINE_MTHP_STAT_ATTR(split_deferred, MTHP_STAT_SPLIT_DEFERRED); +DEFINE_MTHP_STAT_ATTR(nr_anon, MTHP_STAT_NR_ANON); static struct attribute *anon_stats_attrs[] = { &anon_fault_alloc_attr.attr, @@ -609,6 +610,7 @@ static struct attribute *anon_stats_attrs[] = { &split_attr.attr, &split_failed_attr.attr, &split_deferred_attr.attr, + &nr_anon_attr.attr, NULL, }; @@ -3314,8 +3316,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, struct deferred_split *ds_queue = get_deferred_split_queue(folio); /* reset xarray order to new order after split */ XA_STATE_ORDER(xas, &folio->mapping->i_pages, folio->index, new_order); - struct anon_vma *anon_vma = NULL; + bool is_anon = folio_test_anon(folio); struct address_space *mapping = NULL; + struct anon_vma *anon_vma = NULL; int order = folio_order(folio); int extra_pins, ret; pgoff_t end; @@ -3327,7 +3330,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, if (new_order >= folio_order(folio)) return -EINVAL; - if (folio_test_anon(folio)) { + if (is_anon) { /* order-1 is not supported for anonymous THP. */ if (new_order == 1) { VM_WARN_ONCE(1, "Cannot split to order-1 folio"); @@ -3367,7 +3370,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, if (folio_test_writeback(folio)) return -EBUSY; - if (folio_test_anon(folio)) { + if (is_anon) { /* * The caller does not necessarily hold an mmap_lock that would * prevent the anon_vma disappearing so we first we take a @@ -3480,6 +3483,10 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, } } + if (is_anon) { + mod_mthp_stat(order, MTHP_STAT_NR_ANON, -1); + mod_mthp_stat(new_order, MTHP_STAT_NR_ANON, 1 << (order - new_order)); + } __split_huge_page(page, list, end, new_order); ret = 0; } else { diff --git a/mm/migrate.c b/mm/migrate.c index 4f55f4930fe8..3cc8555de6d6 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -450,6 +450,8 @@ static int __folio_migrate_mapping(struct address_space *mapping, /* No turning back from here */ newfolio->index = folio->index; newfolio->mapping = folio->mapping; + if (folio_test_anon(folio) && folio_test_large(folio)) + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON, 1); if (folio_test_swapbacked(folio)) __folio_set_swapbacked(newfolio); @@ -474,6 +476,8 @@ static int __folio_migrate_mapping(struct address_space *mapping, */ newfolio->index = folio->index; newfolio->mapping = folio->mapping; + if (folio_test_anon(folio) && folio_test_large(folio)) + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON, 1); folio_ref_add(newfolio, nr); /* add cache reference */ if (folio_test_swapbacked(folio)) { __folio_set_swapbacked(newfolio); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 8a67d760b71a..7dcb0713eb57 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1084,8 +1084,11 @@ __always_inline bool free_pages_prepare(struct page *page, (page + i)->flags &= ~PAGE_FLAGS_CHECK_AT_PREP; } } - if (PageMappingFlags(page)) + if (PageMappingFlags(page)) { + if (PageAnon(page)) + mod_mthp_stat(order, MTHP_STAT_NR_ANON, -1); page->mapping = NULL; + } if (is_check_pages_enabled()) { if (free_page_is_bad(page)) bad++; diff --git a/mm/rmap.c b/mm/rmap.c index 1103a536e474..78529cf0fd66 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1467,6 +1467,7 @@ void folio_add_new_anon_rmap(struct folio *folio, struct vm_area_struct *vma, } __folio_mod_stat(folio, nr, nr_pmdmapped); + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON, 1); } static __always_inline void __folio_add_file_rmap(struct folio *folio, From patchwork Sat Aug 24 01:04:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13776157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF1B9C5321E for ; Sat, 24 Aug 2024 01:05:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3F541800DB; Fri, 23 Aug 2024 21:05:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A4E6800D4; Fri, 23 Aug 2024 21:05:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2450D800DB; Fri, 23 Aug 2024 21:05:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 01E13800D4 for ; Fri, 23 Aug 2024 21:05:19 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id ADE38A02F4 for ; Sat, 24 Aug 2024 01:05:19 +0000 (UTC) X-FDA: 82485345558.21.69A8E65 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf22.hostedemail.com (Postfix) with ESMTP id CDDFDC0013 for ; Sat, 24 Aug 2024 01:05:17 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=j88GMPPq; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724461500; a=rsa-sha256; cv=none; b=nr45ZVh4W+sBSBifg0CXVitcJLXuyZNk4XaxJfVYS9aqpetzg1qyGU98EQ1cLgWz2Gbxit C3/Unep9iEmSQtwx7Aigb844XsyrI9ov6xbSB5AuoFGyFuaQTdyQYSa8gpd4FtqTykkoNb 0SaKqUm3qRIiQxesSx58BL8h73veMJY= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=j88GMPPq; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724461500; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/fvnizBuO11jjlZtTiefVW+ElYgnwDD8L4QEiBXi3dY=; b=xDTknHcyJcwwbHht5baYSykE7YfOpTLQtE5eUdczPW0+NA5CsxqKDgCyqowSCM56gTasu9 tShbqAR6oLr1ghVv2YFqQwIy7QPvsu/bLxYVijsaPaapzGnAn5UQOtMCkjbP9+cEb0D1rM 7te7NA6Nii5NXypo6HyqQ/xE0Ay+JNo= Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2021c03c13aso19957205ad.1 for ; Fri, 23 Aug 2024 18:05:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724461516; x=1725066316; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/fvnizBuO11jjlZtTiefVW+ElYgnwDD8L4QEiBXi3dY=; b=j88GMPPq6AI8I47oE4FVzoRDRR5dDuWCfIhnf1CD80kqkhSN1j7uTWs3U50Xx8K7sO TaHkxtOrKifyRth9ybyv9G8rjYuXIWLpqVM5YRPfrrKOGIm7UCLQcNufL3uAN2oYTBbr T2Cg79HRbTvc9MoIuSGKQ1oaSrTh6Jo6n+zc0kPSRi5PFt3dQcM63CsReVwxtr8KnIwK xGOUOA23WRUuwKLu7JfRag22zC0jICzIRH4ySk0tPIul9MLtTKmjv+jB1WPKXMSIZCxt 7uyTlY0z4h21KkoJPWR8nxk9a2T8NueABjj+/b3WdaCmY4N3LHFplGE6jCQ2JkUPPa0J /zcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724461516; x=1725066316; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/fvnizBuO11jjlZtTiefVW+ElYgnwDD8L4QEiBXi3dY=; b=hhWejyXwvZ8q8VGQURvZ5qzNdpwpOLEV0/TjN8kYN2c07EtB23J32l/tDcYXBsOMNE jNsSHobZ4+lKrJn9YlW2wr9YXpmus0AXsUDMo3WhPJOq0fBNLvVLFKwDPAtCh/DRR5Fw UuMZkEZ1wRyBdz1CmdBFdp+1d5T+F15xgQOhSEfWlIgwPXx84+LElktozIu8SbS0DVg0 6VbtviUhag4/U1M7YaSDEcc1X/VnE+XdIy4yozhObFlS31MMXXrZanyxWNCafcUk6spY TTJWK4iDvmlKW5AExtzSGI/ynSDII+6MptQIsTPmMG7+agm9ePmeYLXg/CbRhHM1LDf1 y97g== X-Forwarded-Encrypted: i=1; AJvYcCWiuqi5Um9XZ3NW8cz4VhUCYxcJ4P6cgS4RfSYNuFruPZYJu8Fql0g4LfMIzXWmG8o4o5q56eNEVw==@kvack.org X-Gm-Message-State: AOJu0Yy06XVa1yMuuF3jR+ze3q/7f44OimZ347yDHZWV8fAN/qdPIDiJ kuA/kqYOo4H+Cm1LNYkuBCVJIu6cWXhHBofUEhMIt/zyBFRdrH1d X-Google-Smtp-Source: AGHT+IFUsnJOcIozfJLBYBiMbcDGe/SYAgg1aJHSClzsIiFxd+TuEBD6xCjLHozi/xa8m8owGr2qkQ== X-Received: by 2002:a17:902:dad1:b0:201:f2a4:cf74 with SMTP id d9443c01a7336-2037fe1ac56mr132147095ad.22.1724461516426; Fri, 23 Aug 2024 18:05:16 -0700 (PDT) Received: from Barrys-MBP.hub ([2407:7000:8942:5500:8d8:dd4b:c921:b282]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7cd9ad56c9fsm3274064a12.64.2024.08.23.18.05.10 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Fri, 23 Aug 2024 18:05:16 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: baolin.wang@linux.alibaba.com, chrisl@kernel.org, david@redhat.com, hanchuanhua@oppo.com, ioworker0@gmail.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, ryan.roberts@arm.com, usamaarif642@gmail.com, v-songbaohua@oppo.com, yuanshuai@oppo.com, ziy@nvidia.com Subject: [PATCH v4 2/2] mm: count the number of partially mapped anonymous THPs per size Date: Sat, 24 Aug 2024 13:04:41 +1200 Message-Id: <20240824010441.21308-3-21cnbao@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20240824010441.21308-1-21cnbao@gmail.com> References: <20240824010441.21308-1-21cnbao@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: CDDFDC0013 X-Rspamd-Server: rspam01 X-Stat-Signature: qeuy9wg855sxcrpcx34hpbj5mbpj8pb4 X-HE-Tag: 1724461517-718502 X-HE-Meta: U2FsdGVkX1/McRcjBaFY76KJglKiBL4W2CiEUQR1iXXPHhSGTXpkntO+YopuZwSShUk+Ln+2E3bJrYgdayCKnGEqbnpFqEId96ym4vo63pp77Od+wveqE7QV6hbVxrLJ/Z5atSWoy9TE+dOYw2NX8oBkh9rBBwbCEtCJiwc7cLf4/yjxarZJtod9U1W6S/aAVrxPvfe/ObM0YPYDbUMd0hED+B4QNMtr3B4uKMpni0p4nRDy4vyAv1ULdKLhH/m0kmE02lBI1izohBAVTKdbnJggGPxaAmBS/SAOM9LF/Niu43yKjoYR0iFgPqiFdCJxJf2g7Sa00aer7FN8fcxo4vyAUTe9fm97te71do4m3vyE5pjs/YUAzEp1CkGzqjLZgulmOik1suo82fNyiz5AC+1Ht37iVq/EVtp9wmYsYHGQuMBJVxPrfu4DYHit508neQiihFTDoqRvUa3IoW8101WaCr9+q9BDG02O1dzsI0FFOUlx14k4PFBDNS7H+1Ws5lOGDQBuR4qiQdSoasmjWHtrbxR/VysDCtAeFx1UuYaLdoleJKeMbZyovFZDSLo7/n0feJ39oS191qNQdVuZScPIGDaQ5prT1vLz1izhFzXx0fwdR2L5MPgqmJMvCKnSLrJMrt0eqEeaGXSsujrCjW14gLMxo1SRmaFV/TnQXtWm8JKVwydc0wC3m2JoHvK3SGrYyW7XyzPozg4BNaVEexV/5MFO2M/6bmOSOQzngUsYCmyo28v87tk8rDIW3ACHj5bvV3sYriNm4TCrKR6Ptk26Bip5FUiNKUhl7vTiE4+TDZCZTiG8ORLVSv1R0es47tFGccboN3nI/V7N/mqFZCsM2OMtTO/8VunCg5m8PzI6aT8HjVU+/m8VorjMGpR/DVYO5K+4w54sFXPEbWj7Ehc1EaDjbXDq9m1sZHg/f+WWrRNasIAqGxRb/75pVppoOpJiuNYYuVfZ4vO1wH7 ObC748ET SaMN18n8CpZ8UTqMHSvXnSRy07X0LScENBC62o+Uu2PcnP0xTOVOeTT/eKiZ4jEMG/5Tf+cXc4EhrPx8PrGtMnv/f2zqsn0M3U5g59szBoZdlEndEowDhuoCbdojovkETJ8vWXraug6zCjLDxu5f8Fp6rNPbhgS1T/mLpPRrMwcc8eHVBH/ZrCYDJFmcY6yru6F2UZ2M+mZ2Yj8MxUXuw/kWqTA7w2uhH/rCgS5e5rB0cZ6P+wRFnUgHOcf8saTZJdn4fTbNu6j+dOM0kZ7NtnLCQitDIqmE09BLYpMh4byYZiUFwD2nF/uKg7yLUcVDaMb13LWEBbiw1xErAcHk73QN5S4Jh73o0QQ4mML5y55q9488sTojE+3K+L7ELhbkzci+Eu7GCY7RO0J84ghEQYS2d+wUcp5aky92XbcyEBn++arlQaL6o6PH2FDr++ZfuKU/fIlC32/9nn2k= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song When a THP is added to the deferred_list due to partially mapped, its partial pages are unused, leading to wasted memory and potentially increasing memory reclamation pressure. Detailing the specifics of how unmapping occurs is quite difficult and not that useful, so we adopt a simple approach: each time a THP enters the deferred_list, we increment the count by 1; whenever it leaves for any reason, we decrement the count by 1. Signed-off-by: Barry Song Acked-by: David Hildenbrand --- Documentation/admin-guide/mm/transhuge.rst | 7 +++++++ include/linux/huge_mm.h | 1 + mm/huge_memory.c | 6 ++++++ 3 files changed, 14 insertions(+) diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index b78f2148b242..6630f2ed14ee 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -556,6 +556,13 @@ nr_anon These huge pages could be entirely mapped or have partially unmapped/unused subpages. +nr_anon_partially_mapped + the number of anonymous THP which are likely partially mapped, possibly + wasting memory, and have been queued for deferred memory reclamation. + Note that in corner some cases (e.g., failed migration), we might detect + an anonymous THP as "partially mapped" and count it here, even though it + is not actually partially mapped anymore. + As the system ages, allocating huge pages may be expensive as the system uses memory compaction to copy data around memory to free a huge page for use. There are some counters in ``/proc/vmstat`` to help diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 2ee2971e4e10..4902e2f7e896 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -127,6 +127,7 @@ enum mthp_stat_item { MTHP_STAT_SPLIT_FAILED, MTHP_STAT_SPLIT_DEFERRED, MTHP_STAT_NR_ANON, + MTHP_STAT_NR_ANON_PARTIALLY_MAPPED, __MTHP_STAT_COUNT }; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 26ad75fcda62..a81eab98d6b8 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -598,6 +598,7 @@ DEFINE_MTHP_STAT_ATTR(split, MTHP_STAT_SPLIT); DEFINE_MTHP_STAT_ATTR(split_failed, MTHP_STAT_SPLIT_FAILED); DEFINE_MTHP_STAT_ATTR(split_deferred, MTHP_STAT_SPLIT_DEFERRED); DEFINE_MTHP_STAT_ATTR(nr_anon, MTHP_STAT_NR_ANON); +DEFINE_MTHP_STAT_ATTR(nr_anon_partially_mapped, MTHP_STAT_NR_ANON_PARTIALLY_MAPPED); static struct attribute *anon_stats_attrs[] = { &anon_fault_alloc_attr.attr, @@ -611,6 +612,7 @@ static struct attribute *anon_stats_attrs[] = { &split_failed_attr.attr, &split_deferred_attr.attr, &nr_anon_attr.attr, + &nr_anon_partially_mapped_attr.attr, NULL, }; @@ -3457,6 +3459,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, if (folio_order(folio) > 1 && !list_empty(&folio->_deferred_list)) { ds_queue->split_queue_len--; + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON_PARTIALLY_MAPPED, -1); /* * Reinitialize page_deferred_list after removing the * page from the split_queue, otherwise a subsequent @@ -3523,6 +3526,7 @@ void __folio_undo_large_rmappable(struct folio *folio) spin_lock_irqsave(&ds_queue->split_queue_lock, flags); if (!list_empty(&folio->_deferred_list)) { ds_queue->split_queue_len--; + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON_PARTIALLY_MAPPED, -1); list_del_init(&folio->_deferred_list); } spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags); @@ -3564,6 +3568,7 @@ void deferred_split_folio(struct folio *folio) if (folio_test_pmd_mappable(folio)) count_vm_event(THP_DEFERRED_SPLIT_PAGE); count_mthp_stat(folio_order(folio), MTHP_STAT_SPLIT_DEFERRED); + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON_PARTIALLY_MAPPED, 1); list_add_tail(&folio->_deferred_list, &ds_queue->split_queue); ds_queue->split_queue_len++; #ifdef CONFIG_MEMCG @@ -3611,6 +3616,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, list_move(&folio->_deferred_list, &list); } else { /* We lost race with folio_put() */ + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON_PARTIALLY_MAPPED, -1); list_del_init(&folio->_deferred_list); ds_queue->split_queue_len--; }