From patchwork Mon Dec 16 16:50:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dev Jain X-Patchwork-Id: 13910076 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54BA1E7717F for ; Mon, 16 Dec 2024 16:52:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AD9FF6B00BE; Mon, 16 Dec 2024 11:51:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A648D6B00BF; Mon, 16 Dec 2024 11:51:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DDB36B00C0; Mon, 16 Dec 2024 11:51:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6A1A76B00BE for ; Mon, 16 Dec 2024 11:51:59 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 25BBD418AC for ; Mon, 16 Dec 2024 16:51:59 +0000 (UTC) X-FDA: 82901412972.08.F27DD16 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf09.hostedemail.com (Postfix) with ESMTP id 59468140011 for ; Mon, 16 Dec 2024 16:51:37 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of dev.jain@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=dev.jain@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734367903; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QkzU1v8iL0Z+ISbOeDDWbdecNqDcUREegtxV4C6vrZE=; b=bHVy6m6XwWgqsFBA4iy9aeAxWTJ1ySRYcxdGW4jI/vnjGu1iwY9Mh+px7ghP5LW0oTJV/U HIzAV/2E4oXziLLdaEluyJATgAT8ET/wA/VO4S3ACSWF/PI/bsNYqTVqSeZjeTpgSVvdtw JpW5wVGdgTfcHuqkkOfJYsH6mUR0aTA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734367903; a=rsa-sha256; cv=none; b=WiORuTvmldoS4z/j0nagKsrRPaIR+vqKRP2pTUDKlIVma3qzuNPqbNuv4tDAJ8XDthQNXn 0QS8gO4MfeKtxs6WqUU5qF21apnvi4Boo1PwsbRtXWAggTCS8pca+o4YxR8H+AbIIJ+W8t /7oby9XISnU1WqGHjj80w0qHSYvd9jM= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of dev.jain@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=dev.jain@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C211B1692; Mon, 16 Dec 2024 08:52:24 -0800 (PST) Received: from K4MQJ0H1H2.arm.com (unknown [10.163.78.212]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0E60D3F528; Mon, 16 Dec 2024 08:51:45 -0800 (PST) From: Dev Jain To: akpm@linux-foundation.org, david@redhat.com, willy@infradead.org, kirill.shutemov@linux.intel.com Cc: ryan.roberts@arm.com, anshuman.khandual@arm.com, catalin.marinas@arm.com, cl@gentwo.org, vbabka@suse.cz, mhocko@suse.com, apopple@nvidia.com, dave.hansen@linux.intel.com, will@kernel.org, baohua@kernel.org, jack@suse.cz, srivatsa@csail.mit.edu, haowenchao22@gmail.com, hughd@google.com, aneesh.kumar@kernel.org, yang@os.amperecomputing.com, peterx@redhat.com, ioworker0@gmail.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, jglisse@google.com, surenb@google.com, vishal.moola@gmail.com, zokeefe@google.com, zhengqi.arch@bytedance.com, jhubbard@nvidia.com, 21cnbao@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Dev Jain Subject: [RFC PATCH 02/12] khugepaged: Generalize alloc_charge_folio() Date: Mon, 16 Dec 2024 22:20:55 +0530 Message-Id: <20241216165105.56185-3-dev.jain@arm.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: <20241216165105.56185-1-dev.jain@arm.com> References: <20241216165105.56185-1-dev.jain@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 59468140011 X-Stat-Signature: mwt8antawtwae5bw9eqb5gpctcuq81mt X-Rspam-User: X-HE-Tag: 1734367897-708722 X-HE-Meta: U2FsdGVkX18IgtJBe/bMUK9jLSji0zGZMIslIvi4349ZkpJy8q4fcwF8i3NKK7mB9BF0eXCOIO5ptTkfCZbUBqZ80hL/LLvMKtga9zyo4iYXeL6hjlwz8kzSCbgdtf7ABFeyt+kJOph0+NH41nw1/UqlXnEzUgs8jww8/TVmHRruxveEDOpDkJpmYaxBmuohcivD128sIelzIhsc9uXn2uPQi1/JWBDjvU4EZXuvnNyioQWEIP1fLJP/XGZFXj+TbZhbGNTRebRuTx0dQ2HjpMih1KsgcHLVQemdBeLZX0Tro+i0fhbAm8w0isnzFNCPFAvbtNjeX3Mq6pdCTV7ZE6NSdba3q3J5B117kllDdIGpYIIQ+OdtZ9RBtRt31ER2OquROCMmtACMvl9KiRvlOoj/s+SjzarFCd3r20lEG3Dbj6Gvnu3MFA95+6/PMblRcobxfxHUMSSmBDo5Xhczit0oWJIKQbg6jncLajTMKx9RdldUlQ+X+BkJhkwW58LwhjwIHAlcDLkd4UHX4C0QPudDItg2O1KAe7gciKwNTEKV9FWD3XThHsR95PY45TlDM3DoaNwbXVkKeUIld2rZVuC5PLeQTSbLom6JR/uJ/wQoFPMB8XZVYHXQA7sKWACPqD1DPw4kPOqh1Ok40679HwIiab4KRLCdcy12FHwyAsJ3rVDjjGt74cA9s/0JRlBvY+BjvN4ZbYWYcqJAnCeBYVZdlNhXcSe97xignleqajRpv0TqXZ49OGUxC7H7pVGR6j/t48yVy29GUgBxIh6o57TuhpTlUQPkLRGY76Gag2r0tpgSyrM3NAd03lII9D7BF57FERcTweYHv9XgmVAkxDVACmzRRiyrnfja07hd7yfXmYQALjIq0tg9IQaCmimhuWFw/6xwTgPAFp3RWK2Pybi4zyzuLX8oZxvfnC+PGIR4lvdNknTekOQJ+B69Rx3A6ODn1qzxZKggt54mVpI 1QIcRpNY zv5eK9Rg9i2vXRtXZZMpFMk45ZljxwTuC5VI4RT2vJACAI5zyfsWPbaPSFk6rFlcuXQL0MhrJEMCFMJJntzh2wuc8aiETV/fdyRT9ZvUpeMP66iekOE7c+Ape1YXxmFrIXPeB9dGAqkgF56Af/6Q4lCjjddR/147Wy/ptDniaDxr7IH/IkQ/bgg05gB8ybJIX0rZwzpY2VPUrXKcxvJ3QGdYY13Mb2wsHRfdsBgvNG0RbFcmPYqogOYCOpvcp44HO/DyVmvZk+IfM84lwaLZKJXVMmRZnYrBikU9SFpK4Dgzg7J5IvNqeaydTvlfXC6jKP+4bTgVe2arWAx+YmMNmZsyAd7Ad8I8QvnFotkI6FYStdCVTt+s8FddLQHWEhGLZAUgie1cnT4DrNsE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Pass order to alloc_charge_folio() and update mTHP statistics. Signed-off-by: Dev Jain --- include/linux/huge_mm.h | 2 ++ mm/huge_memory.c | 4 ++++ mm/khugepaged.c | 13 +++++++++---- 3 files changed, 15 insertions(+), 4 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 93e509b6c00e..8b6d0fed99b3 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -119,6 +119,8 @@ enum mthp_stat_item { MTHP_STAT_ANON_FAULT_ALLOC, MTHP_STAT_ANON_FAULT_FALLBACK, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE, + MTHP_STAT_ANON_COLLAPSE_ALLOC, + MTHP_STAT_ANON_COLLAPSE_ALLOC_FAILED, MTHP_STAT_ZSWPOUT, MTHP_STAT_SWPIN, MTHP_STAT_SWPIN_FALLBACK, diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 2da5520bfe24..2e582fad4c77 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -615,6 +615,8 @@ static struct kobj_attribute _name##_attr = __ATTR_RO(_name) DEFINE_MTHP_STAT_ATTR(anon_fault_alloc, MTHP_STAT_ANON_FAULT_ALLOC); DEFINE_MTHP_STAT_ATTR(anon_fault_fallback, MTHP_STAT_ANON_FAULT_FALLBACK); DEFINE_MTHP_STAT_ATTR(anon_fault_fallback_charge, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); +DEFINE_MTHP_STAT_ATTR(anon_collapse_alloc, MTHP_STAT_ANON_COLLAPSE_ALLOC); +DEFINE_MTHP_STAT_ATTR(anon_collapse_alloc_failed, MTHP_STAT_ANON_COLLAPSE_ALLOC_FAILED); DEFINE_MTHP_STAT_ATTR(zswpout, MTHP_STAT_ZSWPOUT); DEFINE_MTHP_STAT_ATTR(swpin, MTHP_STAT_SWPIN); DEFINE_MTHP_STAT_ATTR(swpin_fallback, MTHP_STAT_SWPIN_FALLBACK); @@ -636,6 +638,8 @@ static struct attribute *anon_stats_attrs[] = { &anon_fault_alloc_attr.attr, &anon_fault_fallback_attr.attr, &anon_fault_fallback_charge_attr.attr, + &anon_collapse_alloc_attr.attr, + &anon_collapse_alloc_failed_attr.attr, #ifndef CONFIG_SHMEM &zswpout_attr.attr, &swpin_attr.attr, diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 95643e6e5f31..02cd424b8e48 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1073,21 +1073,26 @@ static int __collapse_huge_page_swapin(struct mm_struct *mm, } static int alloc_charge_folio(struct folio **foliop, struct mm_struct *mm, - struct collapse_control *cc) + int order, struct collapse_control *cc) { gfp_t gfp = (cc->is_khugepaged ? alloc_hugepage_khugepaged_gfpmask() : GFP_TRANSHUGE); int node = hpage_collapse_find_target_node(cc); struct folio *folio; - folio = __folio_alloc(gfp, HPAGE_PMD_ORDER, node, &cc->alloc_nmask); + folio = __folio_alloc(gfp, order, node, &cc->alloc_nmask); if (!folio) { *foliop = NULL; count_vm_event(THP_COLLAPSE_ALLOC_FAILED); + if (order != HPAGE_PMD_ORDER) + count_mthp_stat(order, MTHP_STAT_ANON_COLLAPSE_ALLOC_FAILED); return SCAN_ALLOC_HUGE_PAGE_FAIL; } count_vm_event(THP_COLLAPSE_ALLOC); + if (order != HPAGE_PMD_ORDER) + count_mthp_stat(order, MTHP_STAT_ANON_COLLAPSE_ALLOC); + if (unlikely(mem_cgroup_charge(folio, mm, gfp))) { folio_put(folio); *foliop = NULL; @@ -1124,7 +1129,7 @@ static int collapse_huge_page(struct mm_struct *mm, unsigned long address, */ mmap_read_unlock(mm); - result = alloc_charge_folio(&folio, mm, cc); + result = alloc_charge_folio(&folio, mm, order, cc); if (result != SCAN_SUCCEED) goto out_nolock; @@ -1850,7 +1855,7 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, VM_BUG_ON(!IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && !is_shmem); VM_BUG_ON(start & (HPAGE_PMD_NR - 1)); - result = alloc_charge_folio(&new_folio, mm, cc); + result = alloc_charge_folio(&new_folio, mm, HPAGE_PMD_ORDER, cc); if (result != SCAN_SUCCEED) goto out;