From patchwork Thu Jul 11 07:29:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13730096 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81BFCC3271E for ; Thu, 11 Jul 2024 07:29:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 154796B00A0; Thu, 11 Jul 2024 03:29:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0DA806B00A1; Thu, 11 Jul 2024 03:29:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E94CA6B00A2; Thu, 11 Jul 2024 03:29:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C94FA6B00A0 for ; Thu, 11 Jul 2024 03:29:43 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 73660A0452 for ; Thu, 11 Jul 2024 07:29:43 +0000 (UTC) X-FDA: 82326647046.12.6DD4310 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf24.hostedemail.com (Postfix) with ESMTP id C8064180004 for ; Thu, 11 Jul 2024 07:29:41 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf24.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720682957; a=rsa-sha256; cv=none; b=a9zQ7f/k1KGj5c2XqUTwe1pUggHScUF/Ex2NWOFpMzXv3ufo+60t+1v6EHpnGNGJN1/WfW B3f3RSj7j8UykSVW5/ym5CiM0VuUhCeP0t49GRhdgF3eT4l5UMkML5cQ1+3fseJOzZqKaC OhmtYZ5I8fAkhEXNwbMe5pnupboUhlE= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf24.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720682957; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6CvvoxGBKwXTXUMYHZFV2RIww44uZBcSMHZeXFKKQos=; b=PJxKh2cMuvIl8c7WJe85KcjcfI+1R4B6jxhnXgTg4Wxyn6pzmKhz9JU/RbK4PyRvKCEBNk QE9uIiq8+xD6r4xkTQKmbDG4silz5/GLghPh3KUgPkqfuSekiFZoMj0x0pPzMcog+fh28I cjYptOZEdcqzCTz6kXDU/NE2f9cYg3M= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2804E1042; Thu, 11 Jul 2024 00:30:06 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.27]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3F6DB3F762; Thu, 11 Jul 2024 00:29:39 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Hugh Dickins , Jonathan Corbet , "Matthew Wilcox (Oracle)" , David Hildenbrand , Barry Song , Lance Yang , Baolin Wang Cc: Ryan Roberts , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v1 1/2] mm: Cleanup count_mthp_stat() definition Date: Thu, 11 Jul 2024 08:29:24 +0100 Message-ID: <20240711072929.3590000-2-ryan.roberts@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240711072929.3590000-1-ryan.roberts@arm.com> References: <20240711072929.3590000-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: C8064180004 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: iitebr9dtdnrx8ocgzu76jgazmkqc8nz X-HE-Tag: 1720682981-312108 X-HE-Meta: U2FsdGVkX1+/bXorZDXMj/7Rf45jD+Zp7fwKK/gtoauuTiaJI8hbvOiadxYy7/Q7cBSxwzPR+k7Sm5i4bmjEpzTtyLZtkBOqfuG2qAeiMFN358PKnajTgVzSv9kYrFFItcDDguEwN30MHKTYOtdzDSQiN33mTQwjPKDe694FQjX/F4vkxoJfvtt4XRPFcLPh6hujwioS2XsLRQgHa+MNKq0EdN4BtYcYgjo1cy1d/08prYou7foo8K9RTYXFmGZhDahzq8VH7KRKU8WsM2uUIgv/XGFH+QGkZ6RR8/PRMHv9QdXFaI7FqHUkr3U7nl7/5urwRkUTpYBZuk51Fqe94NkRG0u4+LqFaznE8UY8s9M3EOQuXftKzP71k8v7FLVifkz6gYhym9tl4ylYRmtPp5hmEK8oVpu/nXFsl0FAzsBcN6eFs0i3tKICIRuHzdI+MFB9aaqm/zx2Ol2OSpnr0Xn5JISIaYnq8MFHhN15q+hIFvlsD8wd/nTMA16A4WaDcibrV9E0NJTuRoNm7CwJQUuA8+Jcc8/aMAwCCHLFaM4m7o/IdV0v2cHTW8hmL4eVKy6CPsvKA+kscPcI5Zjoz8IegcA67zQwNscNjmUWeq+0wisvyydLlGKORCCF3hi/zBIaAmpoPEiMHHA/7KboytjDrPXH5hfaqgUjcm2O3pHrRvx1O2jXDcYDz+cdIdK8j0CDZ4Pyn36GwDSe5tdOGQldvYk9FhEfIksh4XJgvz/SWWySnR3AlbCCTsxVgnL/ud8YQlJq9Yj7ngUQad3mxUbgsSPoU7IxnYj4ozz9OcqleFa4ujmzP7xYRapqsZbP3nQpyYVffWA4XnyjR422asnmFM5H7WfdMhcHKBM7MGNaWdnKg7BygMakrPDjvXDyyAdBNScD6ZR2FG0jXNpJDv/M8DTsTlry1kcP6PM39m/IfbjWVSFWdcwY0TodEPAglEXzqmrJjG2gdw2Dt9d oSHAIMAL 7KMMiYHNcjOzh2ywS31Ju+G7AdOUx0j9MVQyvEiM/WeDRaxW8GcWqNKdkpxGQbdicLUhS1ZYBkYucq/eNXO1f+64UQu6qwUrLBdKQuNJm4IgxEILZF18gr8wjN0DalKshzmlsDkVUelYUn5lMrOKhtj2+ispnV8TH4wnu43s0oD8FQqNVlqE67+E6AgTxr48dACVdz3Mm7u6/GkI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Let's move count_mthp_stat() so that it's always defined, even when THP is disabled. Previously uses of the function in files such as shmem.c, which are compiled even when THP is disabled, required ugly THP ifdeferry. With this cleanup, we can remove those ifdefs and the function resolves to a nop when THP is disabled. I shortly plan to call count_mthp_stat() from more THP-invariant source files. Signed-off-by: Ryan Roberts Acked-by: Barry Song Reviewed-by: Baolin Wang Reviewed-by: Lance Yang --- include/linux/huge_mm.h | 70 ++++++++++++++++++++--------------------- mm/memory.c | 2 -- mm/shmem.c | 6 ---- 3 files changed, 35 insertions(+), 43 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index cff002be83eb..cb93b9009ce4 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -108,6 +108,41 @@ extern struct kobj_attribute thpsize_shmem_enabled_attr; #define HPAGE_PUD_MASK (~(HPAGE_PUD_SIZE - 1)) #define HPAGE_PUD_SIZE ((1UL) << HPAGE_PUD_SHIFT) +enum mthp_stat_item { + MTHP_STAT_ANON_FAULT_ALLOC, + MTHP_STAT_ANON_FAULT_FALLBACK, + MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE, + MTHP_STAT_SWPOUT, + MTHP_STAT_SWPOUT_FALLBACK, + MTHP_STAT_SHMEM_ALLOC, + MTHP_STAT_SHMEM_FALLBACK, + MTHP_STAT_SHMEM_FALLBACK_CHARGE, + MTHP_STAT_SPLIT, + MTHP_STAT_SPLIT_FAILED, + MTHP_STAT_SPLIT_DEFERRED, + __MTHP_STAT_COUNT +}; + +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && defined(CONFIG_SYSFS) +struct mthp_stat { + unsigned long stats[ilog2(MAX_PTRS_PER_PTE) + 1][__MTHP_STAT_COUNT]; +}; + +DECLARE_PER_CPU(struct mthp_stat, mthp_stats); + +static inline void count_mthp_stat(int order, enum mthp_stat_item item) +{ + if (order <= 0 || order > PMD_ORDER) + return; + + this_cpu_inc(mthp_stats.stats[order][item]); +} +#else +static inline void count_mthp_stat(int order, enum mthp_stat_item item) +{ +} +#endif + #ifdef CONFIG_TRANSPARENT_HUGEPAGE extern unsigned long transparent_hugepage_flags; @@ -263,41 +298,6 @@ struct thpsize { #define to_thpsize(kobj) container_of(kobj, struct thpsize, kobj) -enum mthp_stat_item { - MTHP_STAT_ANON_FAULT_ALLOC, - MTHP_STAT_ANON_FAULT_FALLBACK, - MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE, - MTHP_STAT_SWPOUT, - MTHP_STAT_SWPOUT_FALLBACK, - MTHP_STAT_SHMEM_ALLOC, - MTHP_STAT_SHMEM_FALLBACK, - MTHP_STAT_SHMEM_FALLBACK_CHARGE, - MTHP_STAT_SPLIT, - MTHP_STAT_SPLIT_FAILED, - MTHP_STAT_SPLIT_DEFERRED, - __MTHP_STAT_COUNT -}; - -struct mthp_stat { - unsigned long stats[ilog2(MAX_PTRS_PER_PTE) + 1][__MTHP_STAT_COUNT]; -}; - -#ifdef CONFIG_SYSFS -DECLARE_PER_CPU(struct mthp_stat, mthp_stats); - -static inline void count_mthp_stat(int order, enum mthp_stat_item item) -{ - if (order <= 0 || order > PMD_ORDER) - return; - - this_cpu_inc(mthp_stats.stats[order][item]); -} -#else -static inline void count_mthp_stat(int order, enum mthp_stat_item item) -{ -} -#endif - #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ (1<vm_mm, MM_ANONPAGES, nr_pages); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE count_mthp_stat(folio_order(folio), MTHP_STAT_ANON_FAULT_ALLOC); -#endif folio_add_new_anon_rmap(folio, vma, addr, RMAP_EXCLUSIVE); folio_add_lru_vma(folio, vma); setpte: diff --git a/mm/shmem.c b/mm/shmem.c index f24dfbd387ba..fce1343f44e6 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1776,9 +1776,7 @@ static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf, if (pages == HPAGE_PMD_NR) count_vm_event(THP_FILE_FALLBACK); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE count_mthp_stat(order, MTHP_STAT_SHMEM_FALLBACK); -#endif order = next_order(&suitable_orders, order); } } else { @@ -1803,10 +1801,8 @@ static struct folio *shmem_alloc_and_add_folio(struct vm_fault *vmf, count_vm_event(THP_FILE_FALLBACK); count_vm_event(THP_FILE_FALLBACK_CHARGE); } -#ifdef CONFIG_TRANSPARENT_HUGEPAGE count_mthp_stat(folio_order(folio), MTHP_STAT_SHMEM_FALLBACK); count_mthp_stat(folio_order(folio), MTHP_STAT_SHMEM_FALLBACK_CHARGE); -#endif } goto unlock; } @@ -2180,9 +2176,7 @@ static int shmem_get_folio_gfp(struct inode *inode, pgoff_t index, if (!IS_ERR(folio)) { if (folio_test_pmd_mappable(folio)) count_vm_event(THP_FILE_ALLOC); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE count_mthp_stat(folio_order(folio), MTHP_STAT_SHMEM_ALLOC); -#endif goto alloced; } if (PTR_ERR(folio) == -EEXIST)